Tech interviews, too much homework, and the motherhood question

There’s a fascinating discussion around technical interviews recently; would both candidate experience and hiring signal be improved by revising the current round of (basically Google-inspired) non-runnable algorithm-centric coding examples completed under time pressure?

I’ve been following Thomas Ptacek’s tweets about it for a few months, for example:

“We could have you write and test code like a normal person, but instead we interview you based on something utterly unlike coding”.

— Thomas H. Ptacek (@tqbf) March 28, 2016

@gayle @edropple Yes, I DEFINITELY think no interview better than algorithm interview.

— Thomas H. Ptacek (@tqbf) March 9, 2015

Then last week Julia Grace wrote A Walkthrough Guide to Finding an Engineering Job at Slack:

We’ve put a great deal of effort into designing our interview process so that it is comprehensive and consistent, and are working hard to remove as many points of bias as possible. To date we’ve found it successfully identifies people who will succeed here — those with a high degree of technical competence who also embody Slack’s values: empathy, courtesy, craftsmanship, solidarity, playfulness, and thriving[…]

We’d like to get an idea of how you write code in the real world, since we feel this is the best indicator of how you’d write code day to day here at Slack. Granted, the Slack codebase is larger and more complicated than any technical exercise, but we have found the technical exercise to be a good indicator of future performance on the job. There are great engineers at big name companies and at small ones, so this gives everyone a chance to shine independent of where they are now.

This varies by position, but generally you’ll have a week to complete a technical exercise and submit the code and working solution back to us.

Uncritical praise of take home exams started to ring alarm bells for me. I recall take home exams from university; one of my majors was philosophy, which tended to assign a long essay (eg, 4000 words) to be completed over 6 weeks or so, and a take-home exam (eg, 2 essays of 1000–2000 words) to be completed with a four day deadline. I moved out of a rural town to go to university and lived on my own. From age 19, I also financially supported myself. I loathed take home exams, because I was competing with people who would get the exam, go home, and work on it all week in the house they lived in with their parents. No job. No housework. (Admittedly, no self-imposed decision to take 125% of a normal course load every year for four years of university either, that one was on me.)

And that was before I had two children. I’m not at all excited about tech interviews moving to a model where I’m doing a huge amount of work in my own time, because I do not have a huge amount of free time. Anecdotally, I have already heard of people spending in excess of 20 hours on Slack’s coding exercise. Freeing 20 hours in a week is a non-starter for me, especially if I’m not a clear finalist for the job. Slack is administering these take home assignments prior to on-site interviews, and is a very sought after workplace; it’s quite possible their process will be widely copied and people will regularly be doing a couple of days of coding before in-person interviews, for many many jobs.

To be fair, I have also read through Steve Yegge’s Get that job at Google and estimate that, at my current levels of free time, it would probably take me a couple of years to complete the preparation he recommends. (I have an undergraduate degree in computer science and mathematics — the philosophy major was a separate degree — and a PhD in computing, but at this distance I am far from passing an exam in discrete maths.) But I also wouldn’t be required to submit work samples proving I’d spent that time.

I am also aware that other positions require extensive preparatory work for job interviews for senior candidates, such as preparing sample budgets or strategy presentations or similar, but it’s at least more common only to give such large amounts of work to later-stage candidates for the position.

Let’s not get uncritically excited about adding (yet another!) screen for “isn’t a mother of young children”. I am thrilled that Camille Fournier has made several similar points in Thoughts on Take Home Interviews (also available on her blog):

On twitter, a discussion ensued about whether asking people to spend time at home doing exercises didn’t itself cause bias, against those who did not have a lot of spare time to be doing take-home exercises. Julia mentioned that they expect it to take 2–4 hours, but admitted that some people got really into the project and spent far longer than that[…]

The creative take-home also seems likely to select for those with free time, because if it is really an exercise that some people want to overdo, they will overdo it and you will have a hard time not rewarding that enthusiasm (why shouldn’t you!). And while it’s ok to ask for a few hours, building something that rewards those who can spend far longer is likely to bias against those who have, say, kids to take care of after work and on weekends, or other activities that limit their free time.

Gaëtan Voyer-Perraul also notes in a reply:

If this thing catches on, then it’s going to become a gating mechanism for every developer job in existence. New grads will be faced with hundreds of hours of “take-home” work that goes into the same black hole as their resumés.

Also worth a read: Rod Begbie gives a postmortem of a take-home interview question he used to administer.

I’m excited about revising the technical interviewing process, which will require both experimentation and evidence. While experimenting, and as the tech industry actively seeks candidates from under-represented backgrounds, the ability of candidates to interview with your organisation without tens of hours of free time for take-homes in addition to time for on-site interviews should be a core design principle for your interview process.