Scenario
Interviewer: tell me Mr. Williams how would you, while on a Solaris 6 machine, view network traffic to determine what was the cause of your network connections to host xyz failiing? Is there any tool that you would use to see what might be happening on the wire?
Candidate: Well sure, I might start by first determining whether I'm getting a timeout or a connection refused, then I'd take a look at the network traffic with tcpdump if I had root privileges on the machine.
Interviewer: Is that the only tool you'd use to look at the traffic?
Candidate: From the wire; yes. I might save it to a file and process the information with other tools but tcpdump will grab everything that I need so form that point its just a matter of extracting and interpreting it.
Interviewer: I'm sorry but that's not right
Candidate: What's not right?
Interviewer: tcpdump is not the right answer. I was looking for snoop. You're obviously not a good fit.
evaluation
In this situation Candidate gave a perfectly valid answer and one that was applicable to multiple systems. Interviewer had only worked in an environment with Solaris and assumed that anyone who's worked with Solaris would use snoop (which is specific to Solaris). He had never before heard of tcpdump and didn't know that it serves the same function as snoop. As a result he didn't recognize the validity of answer that was more right than the one he was expecting and dismissed a candidate with greater experience and merit than his own.
This sort of thing happens all the time. I've both witnessed and been subject to it repeatedly. Whether it's tcpdump v. snoop, od v. hexdump, ed v. ex it's all frustratingly common and really should be stopped.
The Problem
Even when the people asking the questions are themselves technical, and in some cases especially so, there is a tendency, in many cases, to expect a specific (set of) answer(s) as opposed to evaluating what's said by the candidate to determine whether it might actually be right. In the Unix/Linux world this problem is compounded by the fact that there are so many ways to get things done in general and each type of system might have several idiosyncratic ways to achieve something in a way that's specific to that particular family of systems and no others.
What it is that they're after
Some interviewers just want to feel superior. They get their sense of importance from being in what they perceive as a position of power. That is an individual issue. The organization is usually looking for someone who can perform well in the position for which they are looking to hire. This is where the whole issue of litmus test questions starts to break down most often.
It's only natural to presume that if you have an open position, one way to make sure that you hire a qualified candidate is to find someone who's doing or had done exactly the same thing. It's rare but it does happen. More often than not it's the case that you have to interview candidates who display the skills required to perform well in a given role and make a determination based on their experience in several areas as opposed to one given thing. This requires much more insight into their background than can be had from just going down a predetermined list of queries.
Don't get it twisted
I'm not saying that no one should use a list of questions when they go into an interview. Sometimes people purposely ask vague and open questions to see how a person thinks or to find out what assumptions the candidate will make. This is a perfectly useful technique when appropriately qualified. Unfortunately too many people feel empowered by their interview questions and don't do enough to make sure that they are used effectively. Having a good set of key points to be covered is a great way to make sure that you're comfortable with the decision you make about a candidate. The point here is that interviews (particularly those for technical positions) are not conducted like hollywood read-throughs. They are not pre-scripted.
Tips for successful evaluation of technical candidates
- Be specific - If you're asking technical questions you should provide some context. Let the candidate know what your assumptions are about the environment.
- Listen - If you ask a question learn how to listen to and evaluate the answer
- Let them have rope - Giving someone enough rope to hang themselves might also mean they have enough to swing to safety. Ask open but pointed questions that allow a candidate to expound.
- Do your research - If you're looking for a specific response to a question you should make sure that you know what all of the appropriate responses are. Saying "well I've been doing this for 12 years" does not count as research.
- Get ready to be wrong - Letting a candidate prove that their solution is valid is sometimes the only way to know that they know what they are talking about. While not alway practical it's valuable at times.
Happy head hunting
