puzzling.org

I tried to find useful third party guidelines for filing bugs against documentation, and wasn’t encouraged to see something I wrote on the second page of Google’s results for “filing documentation bugs”. So I wrote something better.

I’ve been notionally working on Twisted’s documentation for about six months and I’ve been disappointed to find it’s more something to be avoided rather than something to avoid with. It is at least true that I’ve found bug reports useful as a way in: “need to fix bug” is a much more useful starting point for me than “need to improve documentation.” I’m not absolutely sure about this, but I seem to be one of the only Twisted developers (hah! I develop not!) who files bugs against themself. It’s my only consistent adoption of todo lists in my entire life to date.

However, I’ve written an entire CMS and now a series of articles (which I’ve decided are so detailed that noone who’s read them would ever think they were remotely competent enough to buy and use a domain name) rather than write or edit Twisted docs. This is very disappointing, but I’m not contemplating giving up at the moment, despite the occasional nagging fear that my presence as docs editor is holding back someone vastly more committed waiting in the wings. (I suspect there is no such person, but if there is, do get in touch.)

It’s been interesting when I do work on docs though to discover exactly what parts of it I like doing: it wasn’t what I suspected when I began. Writing text for them is very hard. My output is about a paragraph an hour except in rare cases where I understand the code sufficiently well that I don’t have to read or write 100 lines of code to convince myself I understand what’s going on. I’m ambivalent about editing other people’s text for clarity or style — in many ways I love this, but it feels too easy to slip into the “make it as if I wrote it” rather than “improve it” trap. I quite enjoy producing example code, possibly more than I enjoy producing real code. This is probably the right way around — unless I’m a very atypical documentation user, example code is worth far more than text except in the case of design discussions.

I think the main problem I really have is that I’m terrible at ongoing tasks. My coding is the same: I like to have something working at the end of every coding “run” (which for me is about three hours of solid concentration, of which I can currently do two in a day if I have a good day). I like to have a passable document, or at least section, at the end of every docs run (same length as a code run, three hours must be my natural concentration cycle). I’m discouraged from starting anything that can’t be broken up into three hour chunks. In the case of documentation, in which it’s very hard to tell when I’ve finished, this is occasionally paralysing.

Now that I’m hosting this in the same place as my main log, it wants to become my main log’s brother. It wants me to tell it all about whatever spiritual hiccups have smeared my professional programming today. It wants to be my tech confessional. But I’m refraining. Perhaps it might get lucky and be used as a non programming tech log one day.

Currently, I’m struggling with Backwards design issues, specifically storage issues. This stuff, while not quite as dull as rule based information extraction or user input validation, is still pretty dull, and every time I think about it I think about downloading Zope 3 and using that and then I come to my senses and realise that I’d have to rewrite the whole thing.

It’s a backend problem. puzzling.org has always been pretty much a simple tree in structure so the filesystem makes sense as a storage mechanism. Except when it doesn’t, in precisely those cases when puzzling.org stops looking like a tree. For example, consider the logs. their tree structure is: root, year, month, day. However, I want the leaves of the tree (the entries) to be a doubly linked list, ie to have previous and next links.

I can search the tree in this case (if I’m looking or the entry before the 1st of January, I crawl up to the root and down into the previous year to find the 31st of December), but if I decide not to, I need to calculate some data and store it somewhere. Where? Well, I only have made 566 diary entries over the past three years, I could just about store the list in memory. But if I don’t, I need to figure out where to put it.

The case for a links blog (which doesn’t exist yet) is harder. If it is to look like my del.icio.us page, each url needs to be associated with a title, a description and a list of categories. But the sane web tree configuration is root, category, url, (as opposed to root, url, category) which means being able to make the "what urls are in this category?" query easily. When urls are in multiple categories, how do you represent that in the filesystem with a < O(n) complexity query time (n being the total number of urls)? Symlinks?

Well, it’s a trick question as far as I can see. The filesystem’s fairly strict tree structure and limited query mechanisms mean that it isn’t a good backend for this. Which means databases of course. Which means researching databases and choosing between them and learning to use my choice and dependencies (because Nevow isn’t a major dependency, no!) and ew. Hmph. I like the filesystem.

So Dave Winer has pulled all (most?) of the free weblogs.com content. Authors can, at some point, take advantage of a one time offer to get a copy of their content. Nice Dave, good boy. In the mean time, criticism is supposedly muted because people’s content is being ‘held hostage’ for at least the rest of the month. (I don’t actually know about that, all I’ve read is the criticism. I haven’t bothered with the nicey-nice stuff.) [Update in the interests of completeness: there is now a transition plan to 90-day free hosting on buzzword.com. Content has been restored.]

A couple of things concern me about this. One is this persistent notion that’s probably been around since the beginning of time and will probably be around until the end that having the right to do something is a justification for doing that thing. (Hint: "but I’m allowed to do that" is a non-defence against criticisms of your failures of courtesy, generosity or general personability. The whole point of that stuff is that it requires you to do more for others than the bare minimum that you’re compelled to do.)

The other is this notion that if you’re getting something for free, you deserve what you get when it all turns sour. As others have noticed, this is the same stuff that was levelled at people who were shocked about Movable Type’s new licencing schemes. Mark Pilgrim re-wrote that debate in his terms in his Freedom 0 essay. What the people who wanted something for free did wrong wasn’t trying to get something for free ("something free", if you don’t like people playing fast-and-loose with the multiple senses of "free"), it was not getting a guarantee of that freedom. Hopefully Shelley Powers can do something similar with her thoughts on The Value of Free:

There’s nothing wrong with not doing the free thing. However, there’s also nothing wrong with the people who accepted the free thing, freely given… Each person who accepted these free things also gave something back in return: whether it was bodies when webloggers were few, or grateful acknowledgement when webloggers were many. Though those who have benefited from these free services in the past should be grateful, they don’t deserve to be called "cheap" or cut loose without warning. Free does not equate to no value.

Shelley Powers

The point of money is to abstract over some notion of value in a way that allows values to be compared. It’s efficient to be able to compare price tags. But the consistent confusion of money and the value it represents in some cases is concerning for all kinds of reasons. Limiting concern to Free Software alone, it would mean that there is no quality without money; that there are no ethical obligations without money; and that nothing of value is exchanged without money.

My personal instincts about this favour social changes that move from a rights based discussion ("I’m allowed to do this, I’m not compelled to do that") to a courtesy and generosity based discussion. What were the nice things Dave Winer could have done if he couldn’t provide free hosting anymore? What’s an ethical way to write software? What’s an appropriately thankful way to use it? I know, oh, I know that people have been talking in these terms for thousands of years too. I still wish they’d do it more often.

This thing has been around for a day and already people have asked me about comments. I don’t get asked about comments for the other log very much. I turned them on on Livejournal (this is cross-posted) with some trepidation and it’s worked out surprisingly well.

There are two main reasons I’ve avoided comments or web editing in Backwards. The first is that I’m not a big user of web editing: I’ve had too many crashes that cost me work, accidental cuts I can’t undelete, and old cached copies of the page in the form (thank you Zope 2, or was that Squid?). Plus it’s always someone else’s UI and they’ve always set something to be too small. The other reason is that input validation is currently competing with rule based information extraction (just don’t even ask) for my "least exciting programming chore" award. The beauty of writing the entire site myself is that I don’t have to check for malicious mark up, logins, cookies and other horrible things. I have all the power, no one else has any. Easiest authentication problem ever.

But it all comes down to the fact that I instinctively dislike the idea of comments on puzzling.org because it’s all mine, precious. Maybe I’ve spent too much time in the wrong comments threads, but I just don’t see the appeal of spending however many millions of hours I’ve spent this year in order to give people a forum to attack me and a guaranteed audience for their troll-fest. I want to put a click between me and my critics. Given the Livejournal experiment, this is a bit silly: people use my comments to say things like "let’s go crazy Spanish style" rather than "I will eat your young, ignorant evil-doer." Even so. Precious. One day someone’s spam robot would leave a comment and I’d feel personally violated.

There’s a pot and kettle problem though, because I prefer it when other people leave comments on (or in the case of my fellows who write their own CMS, write a comments system and then leave it on). There’s a certain social niche comments fill. Writing an entry to say "happy birthday" or "wasn’t it a nice day?" in response to other people’s entries is a noise problem more than a social activity. Sending an email works a bit better, but people are protective of their inboxes. Plus you miss out on interaction between commenters.

Wow, it really is possible to talk yourself into things isn’t it? Good thing I didn’t try and balance out the pros and cons of writing input validation, or I’d be spending today adding a comment facility to this thing. As it is I need to add some features for Andrew so that I can acquire my first user.

The extended absence of advogato.org has finally goaded me into doing what I’ve considered doing for ages on and off: moving my tech log to a server I control. I’m sure I’m far from alone too, especially since advogato.org posters showed up frequently on the Planets. I may work out some way of cross-posting, but I’m not sure anyone read the advogato version of this. It appears on various aggregators, hopefully they will all point at the new version soon.

Who would have thought that puzzling.org would be at all close to advogato.org in reliability?

Moving my tech log, or at least the bits of it that I had archived on my desktop, helped me iron out a bunch of kinks in Backwards too. It’s coming along nicely and maybe I’ll even do a tarball release and suggest it to the unwary on #twisted.web in a few months. Still, I had to change a large amount of code just to get it to let me use two logs with one install, so perhaps it isn’t quite that sound yet.

Incidentally, a side-effect of this is that entries I make on eyes will no longer appear on LinuxChix Live. You can subscribe to it directly if you’re interested though.

Heads up: A call for Python related papers for the first Australian OSDC (Open Source Developers’ Conference) went to the python-au list this afternoon.

The OSDC is between the 1st and 3rd December 2004 in Melbourne. It sounds like there will be a whole 12 hours between that and ALTA‘s summer school and workshop… in Sydney! Pfft, there’s just about time to drive between them with that kind of timing.

I’m tempted to work up a paper for OSDC, because I sure won’t have one for ALTW. It’s a pity my Python expertise is a proper subset of spiv‘s. And I’ve been doing web development again anyway, and it lacks awesomeness. Perhaps I need to develop newer and cooler Python expertise in a hurry.

Assuming that you have good reasons for keeping an online diary, there are a few things you can do to improve your chances of making your diary readable. I’ll begin by stating the general principles, and then by reviewing a few breakable rules of thumb that, in my experience, are good indicators of an interesting diary.

The general principle of good writing is to determine your audience, and write for them. An online diarist will normally encounter some tension here — the diarists are often writing partly for themselves or their future selves, and the desire to record events that were important to them may conflict with the desire to record events in an interesting way. You will need to decide to what extent you are intending to resolve this tension in the audience’s favour.

It is the case, if I am part of your audience, that your choice of material is generally meaningless to me, and the use to which you put your material is everything, which is why most of these tips tend towards the stylistic.

Tell a story

Of the beginning, middle and end structure, online diarists struggle most with the ending, often because they don’t know it yet. The most successful stories are often trivial anecdotes. However, there may be an ongoing story that you don’t want to record only in hindsight. In this case, you will want to return to it periodically.

It is very very hard to make a story out of emotions you are still experiencing, unless you’re a brutally honest and particularly insightful person, so if you want to write a powerful emotional entry, you may be better off writing an entry that looks back a year or more.

Write long entries

A long diary entry gives you the chance to tell a story, rather than writing an instant message to your readership, and most good online diaries contain at least the odd long entry scattered in their archives.

Very few online diarists seem to be poets, and so generally very few short entries will not become the highlights of your diary.

Drama is the biggest online diary cliche

If your entry is an allusion to misery that only your three best friends in the world can comprehend, your entry will be boring. The high points of an online diary are very seldom the most dramatic entries, save in the case of diaries that resemble an emotional car crash. For the rest, you will need to hone your ability to make the prosaic interesting, because it is actually much easier to do that than to make secretive drama interesting.

Make your entries complete within themselves

Again, if your entry is full of allusions to events you cannot describe in full, and people you cannot say anything about, and feelings that you are unwilling to share, your entry will be boring. If you need to censor something that is crucial to understanding a story, you may as well censor the entire story. In other cases, tell the story in such a way that it is a complete anecdote, even if it is not totally uncensored. If your reader can tell that you’ve left part of the story out, your entry is not as good as it could be.

A subtle style will serve you well

A diary with a unique voice is often an interesting read. One of the easiest ways to achieve this is to let your spoken style influence your written style. It should be relatively sparing, but a touch of spoken mannerisms in a diary makes it more readable.

I think there are several bad reasons to keep an online diary, including using it as a poor substitute for a paper diary, using it to experiment with hyperlinking writing, or using it as a forum for your opinions. Each of these needs is better served by alternative forms. On the other hand, online diaries are maligned as being necessarily uninteresting due to their trivial nature. Trivial and uninteresting do not always go hand in hand, as diarists and letter writers have appreciated for hundreds of years.

The online diary is a format held in peculiar contempt, for several reasons. Most of those reasons are due to the usual meaning of ‘diary’ — that is, a more-or-less secret record of one’s life, written, presumably, for your satisfaction alone, and deriving much of its power from the fact that it has no readers, freeing the author both from the stylistic constraints of writing for an audience, and from the judgements of that audience.

The online diary format naturally loses much of that power. The disadvantages of the online diary format compared to the paper diary format include less honesty (or less sweeping honesty anyway), and much less privacy. It also leaves the author wide open to charges of narcissism, since they are writing about themself for an audience of other people.

So, let’s free the online diary from those constraints. You do not keep an online diary for the same reasons you keep a paper diary. The disadvantages include a lack of complete honesty and privacy. If you want to write with complete honesty and privacy you should keep a paper diary or correspond in private with trusted friends who will destroy your missives rather than hand them to anyone else.

I also suggest that you do not keep an online diary in order to experiment with stylised writing, because you’re likely to attract the wrong audience. Audiences seeking experimental writing styles don’t expect to find it in online diaries, and audiences reading online diaries don’t expect highly stylised writing, or content that deviates radically from the normally online diary content (that is, a person’s record of their life).

Most of the good stylised writing I’ve seen on the Web has been noticably free from the constraints of chronology. Online diaries are tied to a date based format, and people who are interested in telling stories or linking ideas together would be better off with a more integrated site, all of which is an ongoing work. I consider gruntle, raze, and the Jargon File to be excellent examples of the power that loosely organised, heavily hyperlinked sites offer to writers interested in experimenting with style and content that doesn’t fit in a chronological format. If you want to tell stories, I highly recommend this form over the online diary format.

If you’re interested in writing opinion pieces, rather than snippets of your daily life, I suggest you consider blogging, rather than keeping an online diary. Blogging and online diaries are both presently primarily chronological formats, and there is a gray area between them, since people use the same tools for both. The primary distinction between the stereotypical blog and the stereotypical online diary is the amount of linking in the former. Blogs link to websites, link to each other, comment on each other, discuss each other, discuss links, and discuss ideas. If you’re interested in taking part in intellectual crossfire, the blogging tools and communities will be much more satisfying than the online diary format.

Where experimental sites link internally, and blogs link externally, online diaries are largely hyperlink-free. The form requires authors to relate chosen aspects of their life on a loosely chronological basis. They attract readers who like to follow simple story lines, who like to feel involved in the lives of others. As often as not, the readership is made up of people who know the author and people who would like to.

So, what are good reasons to keep an online diary? If you need pre-digital examples of online diary-like writing, consider letter writing one hundred years or more ago. Letters of this time were often gossipy, personal, entertaining, bitchy and informative. In retrospect, some of the writing in informal letters is not only historically interesting, but very very good. So, if you think that one hundred years ago you would have liked to sit in your drawing room and write to your sister in the next town about your housekeeping, giving interest to the mundanities of your life, then online diary writing is probably a format you would enjoy.