I’ve written already about the type of proposal that is likely to be accepted to linux.conf.au, this is a discussion of how the process worked.
Our process aims to find a good set of talks. Past conferences have asked for written papers too, but we do not believe they are widely read and some authors have simply not sent them in, which is possibly unfair to people who believed the given requirements and wrote their paper. This year we didn’t ask. By not asking for papers, conferences like linux.conf.au are missing one opportunity to actually check that our speakers have had more than a paragraph worth of thoughts concerning their talk. Hence the emphasis on known good technical quality and known speaking ability in the criteria.
I’d like to make a quick comparison here with academic computing conferences. Firstly to clear up a common misconception about academic conferences: people don’t just read their papers out loud; or at least not in computing. I’m told they do in philosophy. It’s meant to be an engaging narrative about a problem and its solution, much like a technical conference talk. (Both types of conferences have speakers that fail at this.) The selection is very different though: for an academic conference you submit an abstract or a full academic paper, usually in the 8–15 pages range, and selection is usually based entirely on the quality of the research as demonstrated in the paper, rather than on your history as an engaging or hugely popular public speaker. And the papers are actually important, in computing they will contain (or ideally contain) enough details to allow people to replicate the research (in traditional experimental science, that stuff goes in journals, in computing journals tend to contain only very serious and really stellar work). People wanting to do serious critiques of the work or to extend it will refer extensively to the paper; the paper matters in the way that code does in Free Software. Reviewers will study the paper in detail: ten conference papers would be a very very high reviewing load for a single conference.
This year all program committee members were asked to review all proposals. We voted on them, literally, on a scale of 1–5, which I personally interpreted as please no
through to I will die if we reject this
, although other reviewers may have calibrated differently. We did not provide feedback that was intended for the authors. We did not, therefore, do what would be called peer review
, which is about extensive constructive criticism of the work suggesting ways to improve it, even if it is being rejected. That’s expensive for reviewers and would require drawing reviewers from a broader range of backgrounds: the kind of expertise required to say this talk is not terribly exciting
is not the same as the expertise required to write a letter to the author suggesting technical improvements to their work. I called the linux.conf.au process Am I hack or not?
initially, although since our acceptance rate is about 25% this turned out to be unfair to people who were rejected. Many were actually hack.
That acceptance rate does have certain effects when it comes to our criteria. We are not able to take many chances on people without a track record. We do not have the reviewing manpower to make any useful suggestions to people about their work or their talk proposal, although this would be possible with some other processes we could have used. The abstracts length for this conference makes proper peer review impossible (we could offer suggestions about making a better abstract, but not about doing better work as such even if we had the manpower). We can aim to possibly only select good or excellent talks.
I’ll be interested to compare the PyCon process, particularly since they’re pattern nuts and have found a series of patterns around which you can organise your committee meetings. I have to say an occupation hazard of doing these things is that you really want to go to the conference afterwards. I’d kill to go to PyCon now, if it wasn’t that that wouldn’t help me get a ticket to Texas one bit.
In other news, the linux.conf.au programme is available. Here’s talks I’m particularly looking forward to:
- The Kernel Report (Jonathan Corbet)
- Fixing suspend for fun and profit (Matthew Garrett)
- Digital Preservation – The National Archives of Australia, Open Standards and Open Source (Michael Carden) [although unfortunately this is up against Val Henson, who I’d also like to see]
- The OzDMCA: what it means for FOSS (Kimberlee Weatherall)
- Tutorial:GIMP Uncovered: Understanding Images and Image Editing (Akkana Peck) [I’ll have to catch either Kimberlee Weatherall or Akkana Peck on video though, another clash]
- Starting an Open Source business (Paul Fenwick)
- How to Herd Cats and Influence People (Jono Bacon)
- Concurrency and Erlang (André Pang)
- Making Sausage: How the OLPC Machine Was Designed (Jim Gettys)
Andrew has already put his hand up for the cricket match and he doesn’t even have permission to take the leave yet.
The linux.conf.au review process by Mary Gardiner is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Comments
From my US vantage point, “big of a deal” is very common.
And I too have those moments of looking at a word for a while and it not looking like a word anymore. I know what you mean.
Posted by katie on February 25, 2004 01:17 PM
I just came across “How large of a litter does a rat have at one time?” in the TREC QA 2003 questions too.
I can’t do that with “of”.
Posted by Mary on February 25, 2004 03:51 PM
From a Norwegian: is this supposed to be the same as “it’s not a big deal”, or is THAT just some Norwenglish and gramatically incorrect phrase?
Posted by Magni on February 25, 2004 09:06 PM
Magni:
“It’s not a big deal” is perfectly fine, and you wouldn’t blink hearing it in Australia. The “that” simply lets you add emphasis, it’s always said “it’s not /that/ big a deal”. Having it there at all is a little colloquial.
Posted by Mary on February 26, 2004 02:24 PM
I hate the word “of” in a lot of contexts that seem to be loved by Americans.
“Get off of my lawn”.
It just grates me, I know not why.
In fact, I hate most Americanisms: “one hundred two dollars”, “write me”, “most all”… Hmmm, there’s one more, but I can’t remember it for now.
Posted by TimC on March 4, 2004 10:03 PM
Katie: In the photo of tornado kitty, simon and you napping, is Simon sucking his leg and sewing? Because that’s exactly the thing that one of our ginger kitties, imaginitively called “Ginger” did when happy. Always sucked his left rear leg. It would be really cool if there was another identical ginger kitty (they look exactly the same) was just as stupid as our one
Posted by TimC on March 4, 2004 10:11 PM
One I love though, is “Way!” in response to “No way!”
In fact, I tend to like most back-formations.
Posted by Mary on March 10, 2004 11:33 AM