Working in natural language processing is so disappointing sometimes — probably a lot of AI work is similar. You imagine somehow that all these clever clogs are verbing nouns and nouning verbs and otherwise initiating computers into the mysteries of human language, but really, you spend an awful lot of time writing regular expressions and using other tools at a similarly shallow level.
Case in point: gazetteers. A very common approach to the problem of locating place names or person names in a piece of text is to to have an enormous list of place names and person names and simply see if your candidate is in the list (yeah, you can make it more intelligent by checking for common surnames or the like too). It’s kind of understandable when you try and articulate rules for telling person names from place names (c’mon, can’t you imagine a person named “Ayers Rock” and some kind of landmark named “Apple Martin”?) but it still feels … un-fun.
I don’t work with them much, but I can see why the field is so fascinated with statistical approaches at the moment. Generalising over a set of data is so exciting. Stephen gave a very interesting talk on Monday about what Regina Barzilay and Mirella Lapata are doing in sentence ordering, and it didn’t seem to involve them having to sit down and manually write out any rules at all about how sentences are ordered. Very refreshing. Rigorous logical approaches have the same kind of appeal, but they’re less commonly used.
A lot of the time I don’t think I’m made to be an experimenter. But then I remember that I don’t think I’m made to be a programmer either. What I’m made to do is sit on a bean bag and have lackeys bring me interesting things to read. I can’t see that working as a job description.