Update
I consider this horribly out of date, I'm only leaving it here for posterity, and much of what I used to do has been dropped. I might right and update post one day, but my new process is more exploratory and less adversarial.
The Setup
This is an overview of my interviewing processes, along with some background, that I've formulated over the years. The post itself was inspired in part by a conversation and after reading an incredulous look at claims of job candidate failure rates through hacker news.
Different Strokes for Different Folks
Primarily there are two formats that I've run, one of which is a bog standard interview (60 mins), the other is much more involved, spanning multiple interviews/sessions. The former was simply me being asked to assess technical skills based on a resume, while the latter was a process designed with a dev team a couple years back.
The standard interview, at least the time box aspect, was ostensibly foisted upon me. My employer at the time walked up to me, handed me a resume, and told that I'd be interviewing someone the day after. All I had to go on was their resume, and could only spare 30 minutes of prep time, and so the resume was my focus. It was only in the latter part of the first interview, where I realised that I had been grilling the candidate rather hard, and wasn't feeling entirely convinced they were going to cut the mustard. At which point I felt sorry for them, and tried to find out if they might have other redeeming qualities. That's when I asked a few prying questions and stumbled upon three that would be the core of this interview format going forward.
Hold the Cream and Sugar
The standard interview has a fairly simple premise; it's based on assuming the worst, that the person I'm interviewing is a flunky and a liar. The prep is simple, grab a pen and markup their resume with incredulous comments about their accomplishments and a quick checklist of the major items you care about that are relevant to the position (TDD, CI, DDD, SOLID, Design Patterns, etc...). These act as a guide as I probe into the person's past, and ensure that they're on the up and up. This deep dive into their professional background will end up taking the majority of the allotted time. If things are going well, I break into the actual interview questions, which, outside of the veracity of their resume, is what I've been after all along. The list is short, but given the time frame it makes sense, but here they are along with my expectations and insights:
1) What are you excited about in regards to software engineering/computing?
Here I'd like the candidate to talk about software, technology, architecture, industry trend, process, etc... anything to show that they're engaged, that they're passionate about computing, and are so now. This can also be an opportunity to get insight into how current they are, mind you this is not going to be a definitive answer in that regard, so it entirely depends on why they're excited about it.
2) Do you have a reading list, informal or otherwise, slashdot, blogs, mailing lists, books, documentation, papers?
Push come to shove, anyone remotely worthwhile reads, and reads a lot, even if they don't think they do. It's a matter of life and death for one's career. I tend to afford a lot of leniency when getting answers for this question, and pry and dig deeper, asking them if they: read any blogs regularly; have an aggregator, such as Google Reader; frequent sites like Hacker News or SlashDot; have read any academic or white papers; subscribe to mailing lists such as a news group for an OSS project. Quite literally anything will do, but nothing is a non-negotiable game over, in my mind; though, I'm not convinced that's better or worse than, "If I get stuck, I Google around and find the answer". I believe what bothers me most about the latter is that the only time one seeks new knowledge is when they're observably stuck. It puts into question how proactive they are; moreover, how capable they are in identifying that they're in trouble in the first place. In my view, as a developer, our situation is as follows: we read code, we read emails, we read commit logs, we read documentation, we read all the time. We have to love reading; performing it as a mere job function doesn't suffice. Not only that, the inability, or lack of motivation, to stomach technical material is not an option. Lastly, and most importantly, reading is the gateway to learning, and a reading habit is a strong indication of the desire to educate oneself.
3) What would you like to tell me about yourself that I haven't asked you about, something that I should really know?
Here the candidate could talk about something that wasn't on the resume, or didn't come up, or revisit a prior thread in the interview. It's open mic, they shouldn't let it go to waste. This is merely a last chance for them to make their 'pitch'.
As for the "plain Jane interview", that's it. Those three questions are really what I'm after. If the resume has made it this far, and their story checks out, they're probably plenty qualified, at a technical level.
How it happened
The lengthy and involved interview process was only run a few times, and needs to be iterated upon further; moreover, the initial runs were not carried out entirely to my liking. It would be a disservice to simply render that here as the final product; moreover seeing the evolution of is key in highlighting the impetus of its design. So in order to ensure that it can be taken and carried forward successfully, I'm providing a summary of its history.
The first step is filtering the resumes. Simply look for the major no-nos: typos, grammatical errors, etc... The only additional criteria was having a Linux background so we didn't have to teach them the basics -- that would have been torture -- and an experience quality and quantity commensurate to the role. I could go into a rant about people lacking a Unix background, I won't, but I will leave you with this, of the people who come from one and jump from the other, I find one of those groups have little issue gaining competency in their new environment while the other are a liability.
The first interview was a short, 10 minute, chat to make sure they're not crazy or smelly, and then they were handed the test. The test was as follows: the first page, with which I disagree strongly but the rest of the team wanted it, was 8 or 10 ridiculous esoteric PHP questions about obscure semantics of rarely used PHP functions. The next page was the famous, or infamous as the case may be, fizzbuzz, answerable in any language with which they were comfortable. I somewhat agree with the author that some might take fizzbuzz to be insulting. Perhaps we should have put up a warning, to the effect of, "please don't be insulted, this is a simple litmus test", and another, "please walk through your code, there are some very subtle but easy mistakes made if you rush". The last page was three questions: 1) name two variables in the my.cnf, which ensured that they had at one point configured MySQL, 2) design a normalized table structure for about 10 or 12 items of a person's bio, name, address1 and address2, phone1, phone2, etc... 3) and a simple select statement. Afterwards the team reviews the test outcome; primarily focusing on FizzBuzz, and we were very forgiving when marking it, making sure the logic is sound, while syntax errors are perfectly fine. As for normalisation we expected three tables, though we're willing to take less normalized designs so long as there was a blurb showing that they recognize what they did. If we're happy with the performance on to the next step.
The candidate would be given a simple take home, they are provided a sandbox, and asked to read the data off the file system, and output a web page with some formatting. Just some simple code, we wanted to see approach, I think the test was too simple and it didn't allow anyone to flex any real OAD skills without it being an embarrassingly obese solution. It's best done procedurally, and therefore largely a failure when attempting to get any design sense.
Onto the real technical interview, here I'm going to switch tenses, more to the point, I'm going to describe the intension. The problem is simple, give the candidate a whiteboard, the floor, and in front of at least some or all of the team they'll be working with. Then ask them to design netflix -- having them dive in head first, it'll give you an idea of the things they care about and confident with, see what sort of questions they have, act more like the business owner at this point. Now, structure it, ask them some questions about the areas they haven't covered, expose the things they didn't think about or were trying to skirt. We had a list of things that we were hoping for, something about the web presentation layer, something about the domain, something about code organisation, something about server architecture, and something about database design. These are simple expectations that the team can come up with according to the position, spanning the major concerns of the business currently -- if you haven't guessed it, you should be wearing the fellow programmer hat at this stage. In the previous two stages it's important to help coach the candidate through any 'dead air', make sure they feel they're not being grilled and help keep them moving. Otherwise, you're wasting everyone's time. Each of the stages should last approximately 20 minutes. Then let them ask the team questions, talk about fit, talk about culture, talk about inappropriate humour at the office, the team needs to open up and let the candidate know exactly what they're getting into, this is going to be a new family after all.
It's My Party
Currently, this would be my game plan, at least for a product oriented shop, service programming is another matter entirely.
Have a member of the team do an initial smell test, and the business should evaluate them for their soft-skills, end off by giving them a programming test. The test should consist of some sort of fizzbuzz, and whatever else is relevant to the industry.
If things are still green, have them do a take home test, for which I would give them a large problem and ask them to get as far as they can -- and idea lifted from a colleague who did this with me and others he's interviewed. If time permits, you could do even better by having them come in and pair for and hour or two with a senior team member on a problem.
Satisfied with their performance on the take home, ask them to come in. Review the test with them, and then do a shortened version of an incredulous look at their resume. In particular, I would incorporate the insights gained from reviewing their test. Finally ask them the three questions to find out if they're going to do more than contribute at the level they're currently at, or are they going to grow.
Then setup the team interview, "The Netflix Design Session", or frankly any other large system they're familiar with or would be a good way stress their design skills.
If they're good to go, then let the business work out the hiring details.
In the case where you have to turn away the candidate, don't waste their time, and don't help perpetuate the issue. Give them a 15 to 30 minute call and coach them, tell them their sore spots out right and hopefully they'll attack them. You could even give them a timeline to shoot for to complete those objectives and ask them to reapply when they feel they've had a chance to address those issues.
Perspective
All that said and done, you can be a vicious interviewer and cut apart everyone who comes before you, and I could have done that too. But I try not to, everyone has a path that they follow and whether you like it or not, everyone is a work in progress, and you're going to be a part of finishing it up. Decide on a reasonable baseline, which is determined by a lot of things, the work, it's quality and demands; the location; compensation; benefits; and the learning. I know many people want the best, but fall short on two key points, first and foremost, chances are they themselves are not the best and so cream of the crop will not bother with them, and secondly, they can't afford it. Push come to shove, Google pays six figures, the stock is ridiculous, signing bonuses, etc. For the Vancouverites, I imagine California is a pretty awesome place to live, so the entire location thing isn't as big a deal. At the end of the day, you can be very stringent, only take the awesome candidate (read: no one, for the aforementioned reasons), or you could be realistic, and realise that there is a middle ground, it's a minimum level of knowledge, and the character required to do the job well.
