"Just leave if you don't like it"

A note on the arguments following Mark Pesce’s keynote. There’s one in particular that bugs me: “just leave if you don’t like it.”

The thing is, it isn’t normal at linux.conf.au (unlike at a Bar Camp) to just exit a talk from, say, the front section in the middle of a row. Unless you are at the very edge of the room, it’s considered rude to just leave, to the point where some speakers or session chairs might actually yell at you. (I had university lecturers do that.) And I suspect LCA, for organisational reasons as well as for speaker comfort, would rather not encourage an atmosphere of people just traipsing in and out of talks through the centre of rooms. So… the environment is (somewhat) coercive: if you don’t like the talk, you have to be actively rude to the speaker and the rest of the audience in protecting yourself from the talk.

If an environment could be created where someone could leave a talk from any place in the audience with a minimum of fuss and without risk of social retribution, and if people really did do so for all kinds of reasons, and thus an exit during Pesce’s talk would not have been immediately visible to everyone as “I have a strangulation phobia, if you would like to bother me in future, please mime strangling me”[1], I’d at least take this argument seriously. But in the LCA context it currently equates to: “don’t like the talk? embarrass yourself and be rude to the speaker!”

(Note to LCA people: I have a comment policy, and if your comment annoys me I won’t publish it.)

[1] I do have a strong reaction to strangulation, although probably not technically phobic, and if anyone uses this information to harass me even as a joke[2], they will not be my friend thereafter.

[2] People who have physical triggers, like having sharp objects pointed at their eyes, or disliking their neck being touched without warning, and who admit them, do suddenly find that half their acquaintance immediately does that to find out what happens. Consider yourself warned about what will happen.

Ethics of Free Software community research

Most of this entry is exactly a year old today and it’s just sat around in draft form all that time. Since I posted something similar on Geek Feminism about research into women in tech and similar topics, I thought I’d get it out there.

In January 2009 a researcher named Anne Chin of Monash University Law emailed the chat list for the linux.conf.au 2009 conference asking for research subjects to be interviewed about licencing and Open Source software. There were several responses criticising her use of HTML email and Microsoft Word attachments. I’ll leave the specifics of this alone except that people should be (and probably are) aware that this is almost always an unknowing violation of community norms.

I did, though, think about making some notes on research ethics and Free Software research. A bit about my background: I am not a specialist in ethics. I’m somewhat familiar with ethics applications to work with human subjects, but not from the perspective of evaluating them. I’ve made them, and I’ve been a subject in a study that had made them.

For people who haven’t seen this process, the ethical questions arising from using human subjects in your research in general covers the question of whether the good likely to arise from the outcomes of the study outweighs the harm done to the subjects, together with issues of consent to that harm. (There are many philosophical assumptions underlying this ethical framework, I don’t intend to treat them here.) Researchers in universities, hospitals, schools and research institutes usually have to present their experimental designs to an ethics committee who will determine this question for them and approve their experiment. Researchers who work across several of these (eg, a PhD student who wants to interview schoolchildren) will need to do several ethics applications, a notable chore when the forms and guidelines aren’t standardised and occasionally directly conflict. Researchers working for private commercial entities may or may not have a similar requirement. Researchers who use animals also have to have ethical reviews, these are done by animal ethics committees, which are usually separate.

At my university, essentially any part of your research that involves measuring or recording another person’s response to a research question and using it to help answer that question needs a human ethics application.

The good/harm balance may include very serious dilemmas: is there a health risk to subjects? how will the researcher manage the conflict between maintaining subject confidentiality and research integrity and the good of her subjects or the requirements of the law if she uncovers, say, episodes of abuse or violence? But it also involves less immediately obvious and serious ethical questions. Is this study a giant waste of subjects’ time? is considered a question of ethics by ethics committees, and is in fact the most serious problem for linguistics research, since there’s very seldom an outcome of particular interest to the subjects themselves.

The study in which I took part a few years back was towards the serious end actually: it was a study into the psychological profiles of people who have an immediate family member who had cancer as a child and involved both questionnaires and a phone interview with a psychologist. Both because the study explored memories of the illness and because the profiling included evaluating depressive episodes, suicidal ideation and so on, it came with a detailed consent form and with information about a counselling service that had been informed of the study and was prepared to work with its subjects.

In the case of the Free Software community the ethical questions are often more towards the waste of time? end of the spectrum than the more immediately serious end. It’s important to understand that this isn’t necessarily the case though. Here are some more cutting ethical problems:

  • getting findings that expose your subjects and/or their employers to intellectual property claims; or
  • revealing that your subjects are breaching employment contracts in some way (generally also related to IP) and thus exposing them to job loss and possible civil action.

Getting ethics approval to carry out workplace studies can be fairly hard precisely due to problems like these. But in the rest of this post I will treat the waste of time problem.

Firstly the basics: are your subjects going to be identifiable in your final reports or to the general public? If not, who will know who they are? Can a subject opt to have their responses removed from the study? When and how? All this should be explained at the start. (Usually if an ethics committee has been involved, there’s a consent form.) If doing a survey look into survey design, in order to construct non-leading questions and such.

Now, for specifics. Most of them arise from this principle: there are a lot of researchers working, in various ways, on the Free Software community, possibly making it a slightly over-studied group if anything. This places the onus on the individual researcher to demonstrate to the community that their project is worthwhile and that they’re going to do what they say. Thus:

  1. demonstrate some familiarity with the background. Depending on your research level this could mean anything from demonstrating a knowledge of existing anthropological work on Free Software (say, if the research project is for your anthropology PhD) down to at least understanding the essential concepts and core history (say, a project at high school level). This can be demonstrated by research design, eg asking sensible well-informed questions, but actually mostly requires a bigger time investment: making appearances in the community, either virtually or physically, ideally for a little time before asking the community to help you get your PhD/A-grade/pass.
  2. don’t get the community to design your experiment for you. Have a specific goal, more specific than get people to write me lengthy essays about Free Software, and get ideas from that and write about them. In the general case, the ask people incredibly vague stuff and hope they say something interesting technique fails the waste-of-time test.
  3. give your results back to the community. The most common problem with the various surveys, interviews and questionnaires sent to the Free Software community is that responding to them is like shouting into a black hole. It is not unheard of, of course, to see the thesis or essay or roundup that comes out of these, but it is unusual, relative to the number of requests. Most of the time the researcher promptly disappears. Researchers should come to the Free Software community with an explanation of when and where they will make the results of the study available. They should explain the aims in advance unless this would compromise the results. (On that note: Anne Chin is giving a linux.conf.au talk this year.)

Creative Commons License
Ethics of Free Software community research by Mary Gardiner is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

Possible structures for a technical talk

Edited version of a private email I sent about linux.conf.au 2010 preparation, but not conference-specific. The email discussion led to me enumerating the kind of talks that I think you can give about technical subjects at a linux-conf.au-style (non-academic developer-oriented) conference.

Recitation of facts; or the architecture diagram talk

In this kind of talk, the speaker walks the audience through, say, the architecture of a project. The talk contents are dictated by the architecture. Five minutes on the input module, five minutes on preprocessing, five minutes on transforming into a new set of matrix bases, five minutes on postprocessing, five minutes on sanity checking, five minutes on rendering.

This is the most obvious structure for a technical talk. It’s also the driest structure. It is useful mostly for audience members who want to work on or with the project code, which is for most talks a small fraction of the audience. It best suits very major projects with years of cruft in their architecture and a wide potential userbase.

Insane hack adventure; or “hacking the Tivo”, Andrew Tridgell, linux.conf.au 2001

In this kind of talk, the speaker usually talks about a project he or she personally undertook and structures it as a battle against the odds. So of course Y is the most obvious solution, although not trivial. So I tried Y for a couple of months. And then it turns out that the X protocol absolutely makes Y impossible for these reasons [dramatic pause] and at this point any normal human being would have given up, but I retired to a monastery for six months and returned refreshed and a lotus blossom I saw on my return inspired me to try Q…

This is probably the most effective structure for a technical talk and it’s surprisingly widely applicable (you can frame a lot of things in terms of here’s the current problem, here’s what you think the obvious solution is, here’s what happened when I tried it). The biggest limitation is that it’s very hard to do it about work you didn’t do yourself so it’s not available to all speakers. And you can misjudge it: so of course I tried Y says the speaker uh, Y is obviously dumb, why didn’t you go straight to Q? asks the audience (at some conferences, they might well ask it out loud, repeatedly). It doesn’t work for speakers who don’t have expertise (at least narrowly on this one topic) over that the audience has.

Demonstration of surprising ease

In this talk, the speaker proposes a task which sounds too formidable to complete in the talk slot. (Pretty much any task, for some talk slots.) They then proceed to show how it can be done despite the odds using their tool. Edward Hervey edited a video PiTiVi at linux.conf.au 2008. At OSDC 2008 Thomas Lee added a ‘unless’ keyword to the Python language.

This structure can be fun but it has a lot of pitfalls. Edward was continually battling to type while dealing with a handheld microphone. Speakers should recruit an assistant for these talks. The Python keyword talk was kind of fun, but it was also hard to follow: and now I am going over to this totally other tree and adding several lines to this other file at line 1099 and 1346 and is obviously required: he did not succeed in making me think I could easily edit the Python language. In all cases, it needs a decent amount of rehearsal: even more so than other talks.

Dropping of wisdoms

These talks are generally (at linux.conf.au anyway) talks about the social/community/artful aspects of coding. For example, Jonathan Corbet’s talk about how kernel hacking actually works in terms of the trees and maintainers etc at 2009. (One of Linus Torvalds’s bigger 2009 contributions was being in the audience for that talk and adding occasional commentary on his various prejudices.) Andrew Bennetts has had some good luck with a series of talks that began at OSDC 2008, in which he essentially lists every tool he can think of to help you make your Python programs faster. I didn’t see it but I would guess Paul Fenwick’s talk about awesome things you’ve never heard of in Perl was the linux.conf.au equivalent in 2009.

These are useful talks, the main pitfall is that they require considerable expertise over the bulk of the audience. Otherwise someone is just listing a bunch of stuff you do every day back to you. They also need to be about tasks or tools that a lot of people actually use or want to use. No one wants the hidden tricks to using libnousersexceptme.

Creative Commons License
Possible structures for a technical talk by Mary Gardiner is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

On girl stuff

In both of my recent talks involving women and Free Software the audience has latched onto something I didn’t expect. At OSDC it was the GNOME finding that they only got women applying for their summer of code projects once they created special ones for women. I think I expected people to have heard about that already, but they hadn’t. (GNOME had zero applications from women for Google Summer of Code, and some hundreds for the Women’s Summer Outreach variant.) There were probably a couple of things going on there aside from women responding to a specific invitation — in particular, computer science academics at some universities getting excited about being able to give their women students a specific invitation — but clearly invitations are part of what’s going on.

There is a karmic debt to do some work already incurred by giving these talks, but since the work I do isn’t Free Software and wouldn’t be generally useful if I released it as such (I know a lot of people say this about their work, but I try and predict word usage based on the opinion of the document, this really is quite niche software) and I had a reasonable idea for a variant on this kind of talk, I gave a second one anyway, at the LinuxChix miniconf. It was titled ‘Starting Your Free Software Adventure’ and happened to use women as examples. The idea was to show people what the first steps look like. I conducted (extremely short) interviews of several women involved in Free Software or Culture or their communities, including Kristen Carlson Accardi, Brenda Wallace and Terri Oda among others. (I intend to make the slides available, but since I quoted the subjects extensively and directly, it will require gathering permission and then a bit of work editing them.)

As I noted previously this talk was a failure all up, because the wrong audience turned up for it. But one thing stood out and kept coming up all week: Terri mentioning that she had resisted at times working on things perceived as ‘girl stuff’. In Free Software this includes but is not limited to documentation, usability research, community management and (somewhat unusually) sometimes management in general. The audience immediately hit on it, and it swirled around me all week.

This is a perennial problem for professional women: software development is by no means unique in having developed a hierarchy that goes from high status roles disproportionately occupied by and associated with men to somewhat lower status roles disproportionately occupied by and associated with women. (In the case of software, disproportionately occupied by women still means male dominated, by the way, at least in the English-speaking world.) It’s difficult to disentangle the extent to which women and/or their mentors and teachers self-select for the lower status roles (and I would hardly argue that the self-selection occurs in a vacuum either) versus the extent to which they are more or less barred from high status roles versus the extent to which the association is actually flipped and professions and jobs within them have become low status because women started doing them. Other well-known examples, are, for example, the concentration of women in biological sciences as opposed to, say, physics, the different specialisation choices of male and female medical doctors and surgeons, and so on. Sometimes, as in the war between sciences, the status of a field is somewhere between a joke and real, to the extent that those can be differentiated, but often it isn’t: there’s a correlation between the male to female ratio of a medical specialty and its pay.

In all of these cases, a woman who is conscious of this problem tends to face a choice. Do the ‘girl stuff’, or not? (Of course, ideally one rejects the dichotomy, but no individual woman is responsible for constructing it, and if you are sincerely of the belief that one is not programmed to a frightening and unavoidable extent by one’s social context we’re working from very different premises and don’t have a lot to say to each other.) And some, although I don’t know what proportion, of women feel guilty about their choice, especially if they do choose to do girl stuff. Just go ahead and imagine your own scare quotes from now on, by the way.

It also gets messy in various other ways. There’s the extent to which a woman who doesn’t do girl stuff is invested in maintaining the status she has chosen and also the aforementioned loop where if women are doing something, it will come to be seen as not particularly hard or noteworthy.

Most concretely, I usually see this tension bubble away underneath outreach programmes promoting computing careers (you know what, I have my own status issues and I still resist calling it IT) to women. There’s the people who want to go for yeah we all know coding is populated by weirdos, and male weirdos at that, luckily you don’t have to be a geek and you don’t have to code, phew! I tend to hear about that one only once my outreach friends have gotten involved and staged a coup, admittedly. There’s the there’s so many opportunities in computing, and yes, coding is one of them and its fulfilling and it’s something you can do, but dammit, coders get all the cred and attention and dammit can we talk about something else? Women who admin/write/test/manage rock! And there’s you know, women coders don’t exactly rule the world yet, and furthermore isn’t all this oh-yes-you-could-code-I-guess-and-that’s-a-fine-thing but look! something for folks with people skills! talk basically a soft version of ew coding that’s for boys, also, last I checked, math is hard?

I observe again that there’s no right answer here in the real world right now. Women doing girl stuff have good reasons to feel dissatisfied that their hard-won skills are underpaid and under-respected, women doing boy stuff (scare quotes! please insert!) want other women to know that there’s fun to be had over here, thank you.

One crucial point in my thoughts about this I stumbled on only after the conversation Brianna Laugher recounts, over Indian on the Friday night (the location of all major conference breakthroughs worldwide). She said — paraphrased — that she didn’t feel that she should have a problem or be criticised for doing what she is good at, or what’s so desperately needed in her communities, and have to be just another coder in order to be fully respected. And I said that while this was certainly true, women also need to have the opportunity, to give themselves the opportunity, to be selfish: if they want to code, or do something else they are currently either bad at or not notably good at, or for that matter which they are good at but in which they’d have competitors, they should consider doing that, rather than automatically looking for and filling the space that is most obviously empty.

I had a brief, but related conversation with Jeff Waugh at the Professional Delegates Networking Session — an attempt to formally recreate the Indian diner breakthrough environment —  at which he commented that he continued to find the invitation culture (the same one I discussed in my OSDC talk) of women in Free Software mystifying and frustrating. (Not his exact words, if you have better adjectives Jeff let me know.) I took that one somewhere else: specifically to invitation cultures outside Anglo culture and then to honorific use in the Korean language, but when considering the question of women I think this is intertwined with the be selfish thing: women are reluctant to enter places where they aren’t obviously welcomed, and what better way to be welcomed than to do work that needs doing and not become just another person doing the coding free-for-all and delaying external validation for potentially quite a long time?

I have no answers. Just the perennial question of distinguishing what other people want, what other people claim they want, the genuine satisfaction of being of service to someone, and the genuine satisfaction of knowing you’ve done a good job of something hard. Take a look at where you’re standing on that one occasionally.

The linux.conf.au review process

I’ve written already about the type of proposal that is likely to be accepted to linux.conf.au, this is a discussion of how the process worked.

Our process aims to find a good set of talks. Past conferences have asked for written papers too, but we do not believe they are widely read and some authors have simply not sent them in, which is possibly unfair to people who believed the given requirements and wrote their paper. This year we didn’t ask. By not asking for papers, conferences like linux.conf.au are missing one opportunity to actually check that our speakers have had more than a paragraph worth of thoughts concerning their talk. Hence the emphasis on known good technical quality and known speaking ability in the criteria.

I’d like to make a quick comparison here with academic computing conferences. Firstly to clear up a common misconception about academic conferences: people don’t just read their papers out loud; or at least not in computing. I’m told they do in philosophy. It’s meant to be an engaging narrative about a problem and its solution, much like a technical conference talk. (Both types of conferences have speakers that fail at this.) The selection is very different though: for an academic conference you submit an abstract or a full academic paper, usually in the 8–15 pages range, and selection is usually based entirely on the quality of the research as demonstrated in the paper, rather than on your history as an engaging or hugely popular public speaker. And the papers are actually important, in computing they will contain (or ideally contain) enough details to allow people to replicate the research (in traditional experimental science, that stuff goes in journals, in computing journals tend to contain only very serious and really stellar work). People wanting to do serious critiques of the work or to extend it will refer extensively to the paper; the paper matters in the way that code does in Free Software. Reviewers will study the paper in detail: ten conference papers would be a very very high reviewing load for a single conference.

This year all program committee members were asked to review all proposals. We voted on them, literally, on a scale of 1–5, which I personally interpreted as please no through to I will die if we reject this, although other reviewers may have calibrated differently. We did not provide feedback that was intended for the authors. We did not, therefore, do what would be called peer review, which is about extensive constructive criticism of the work suggesting ways to improve it, even if it is being rejected. That’s expensive for reviewers and would require drawing reviewers from a broader range of backgrounds: the kind of expertise required to say this talk is not terribly exciting is not the same as the expertise required to write a letter to the author suggesting technical improvements to their work. I called the linux.conf.au process Am I hack or not? initially, although since our acceptance rate is about 25% this turned out to be unfair to people who were rejected. Many were actually hack.

That acceptance rate does have certain effects when it comes to our criteria. We are not able to take many chances on people without a track record. We do not have the reviewing manpower to make any useful suggestions to people about their work or their talk proposal, although this would be possible with some other processes we could have used. The abstracts length for this conference makes proper peer review impossible (we could offer suggestions about making a better abstract, but not about doing better work as such even if we had the manpower). We can aim to possibly only select good or excellent talks.

I’ll be interested to compare the PyCon process, particularly since they’re pattern nuts and have found a series of patterns around which you can organise your committee meetings. I have to say an occupation hazard of doing these things is that you really want to go to the conference afterwards. I’d kill to go to PyCon now, if it wasn’t that that wouldn’t help me get a ticket to Texas one bit.

In other news, the linux.conf.au programme is available. Here’s talks I’m particularly looking forward to:

  • The Kernel Report (Jonathan Corbet)
  • Fixing suspend for fun and profit (Matthew Garrett)
  • Digital Preservation – The National Archives of Australia, Open Standards and Open Source (Michael Carden) [although unfortunately this is up against Val Henson, who I’d also like to see]
  • The OzDMCA: what it means for FOSS (Kimberlee Weatherall)
  • Tutorial:GIMP Uncovered: Understanding Images and Image Editing (Akkana Peck) [I’ll have to catch either Kimberlee Weatherall or Akkana Peck on video though, another clash]
  • Starting an Open Source business (Paul Fenwick)
  • How to Herd Cats and Influence People (Jono Bacon)
  • Concurrency and Erlang (André Pang)
  • Making Sausage: How the OLPC Machine Was Designed (Jim Gettys)

Andrew has already put his hand up for the cricket match and he doesn’t even have permission to take the leave yet.

Creative Commons License
The linux.conf.au review process by Mary Gardiner is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

Getting a talk into linux.conf.au

We had a programming committee meeting for linux.conf.au 2007 on Saturday. Decisions were made. They may be revised based on budget. But the general consensus was that it’s the papers that linux.conf.au rejects that makes linux.conf.au the best. And here’s the more cuddly than Rusty guide to being among the best.

First a note. We had in the order of 250 proposals for 60 talk slots. (The ratio is a bit better for tutorials, about 2 proposals for every slot available.) We reject most of what we get, and we reject a fair number of things we suspect or know would be perfectly fine talks. It’s a competitive conference.

  1. Software talked about or that is core to your talk must be available under an Open Source licence. This is not negotiable, with a tiny bit of wiggle room for people who are waiting for their employer to sign off on an Open Source release. Only a little wiggle room, mind you.
  2. It is getting towards being a requirement that you are a core member of a project, or of the part of it you’re talking about. You need to have written a fair chunk of the code, initiated the documentation project, done the benchmarks, whatever. Sweated the sweat. Tutorials are a little different: for a tutorial, evidence of ability to convey enough knowledge well is generally important, and depending on your intended audience might trump not being a major developer of the tool in question.
  3. Project maturity is not essential, but is desirable. If it hasn’t been merged yet, or you are the only user, it will have to be great to be accepted.
  4. Enormous maturity can be a disadvantage, or at least it is if it leads to the the style of proposal that goes here’s the update on my LCA 2005 talk about [some project]. It’s easier to get accepted if you submit a talk focusing on a particular new feature or development.
  5. Being known as a good enough speaker is a big advantage. Standards here are high, but I feel not crazy. You can be accepted without being an amazing speaker. It is, however, essential to convince the review committee somehow that you have had and can convey 45 minutes worth of thoughts about your subject and that people will want to hear it. Being known as a good speaker from other conferences or events is excellent, and a high quality abstract can be convincing in some cases too.
  6. Insane coolness is another huge advantage. In particular, people who’ve built things they can hold in their hands, put their arms around or have a sword fight with, tend to get their papers accepted. Most proposals do not fall into this category, those that do have a high acceptance rate.
  7. Not submitting a kernel talk helps your chances of acceptance. This one is interesting. The problem is that we get a huge number of very good kernel proposals. linux.conf.au accepts a fair number of kernel talks, but is not a kernel conference and doesn’t intend to become one. So to get a proposal accepted into this stream, you must not only be good, but be very very good.
  8. Not submitting a general commentary on your experiences in the Open Source world also helps your chances of acceptance. Again, we accept some of these, but almost everyone has opinions on how to run an Open Source project, and they submit a variety of them. We need some special reason to believe you have something to say that the audience can’t easily think up for themselves or read about.
  9. Having some relevance to a primarily Australian audience is useful. This is really only meaningful for the above mentioned commentaries, for things like kernels it doesn’t matter, and if it’s hella cool, it also doesn’t matter.

For comprehensive information about submission statistics and a list of all the program committee’s blog entries, see John Ferlito’s entry.

Creative Commons License
Getting a talk into linux.conf.au by Mary Gardiner is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.