Panel interviews don’t work

There’s a Right Way to conduct job interviews: one-on-one, with a single interviewer per interview session. If you need multiple interviewers (you probably do), schedule multiple sessions, each one-on-one.

The alternate approach, panel interviews – having multiple interviewers in a session at once – is almost always a bad practice. It increases stress on the candidate, risks measuring the wrong things, and doesn’t lead to better results. Avoid panel interviews: they don’t work. If you need convincing, read on.

Stress lowers the predictive power of interviews

Let’s remind ourselves of what we’re trying to do with an interview question. We’ve got this role we’re trying to fill, and there’s some skill that’s required to be successful in that role. So we’ll ask some questions about using that skill and attempt to use their response to predict their eventual job performance. The key word here is “predict”: the basic premise of interview practice is that there’s a correlation between performance on an interview question and eventual job performance in that area. So, our goal as interviewers is to maximize the predictive power of the interview.

That’s a tall order, of course, because interviews are pretty different from doing the job! It’ll always be an imperfect correlation, so we have to do everything we can to minimize the gap.

And what’s something that makes interviewing the most unlike a real job? Stress.

Interviewing is stressful. This affects some more than others, but nearly everyone experiences some anxiety while interviewing. Candidates are being put on the spot in a way that very rarely happens anywhere else. And they’re being judged in a way that’s super-uncomfortable, and also very unlike almost any other situation. This stress gets in the way of their ability to answer interview questions well. I’ve seen candidates have trouble remembering things, fumble for words, crack jokes or say things out of stress they wouldn’t normally say, etc. This puts interviewers in a tricky spot: if a candidate gives a bad answer does that mean they lack the skill? Or are they just stressed out?

This leads to the risk that we’ll end up selecting candidates not based on their ability to do the job, but on their ability to handle a stressful interview. That’s a huge failure: I want to hire someone good at writing software; I don’t care if they’re good at interviewing!

So, good interviewers do everything they can to lower the stress level on candidates. For example, I try to ask questions in a friendly and casual way, to try to make the interview feel a bit more like a conversation. I also give candidates lots of detail about what the interview will cover ahead of time – often sharing the actual questions themselves – so they can prepare and be ready. I structure the interview to have easy questions first to try to help candidates relax. And so on.

Panel interviews do the opposite: they ratchet up stress. Multiple interviewers can lead to a feeling that the candidate is being ganged up on. With a single interviewer, sometimes you can sell the fiction that it’s “just a conversation”; the candidate can somewhat forget they’re being judged. With multiple interviewers that’s nearly impossible: candidates feel like they’re on trial and this is the jury. Because, in a sense, it is! Panel interviews also add group dynamics on top of everything else that’s stressful about an interview. This further increases stress, particularly for more introverted candidates.

You should avoid panel interviews because they ratchet up stress. This leads to candidates having worse performance because they’re stressed, which means you end up measuring “ability to handle panel interviews”, rather than the underlying job skills you care about.

Panel interviews lead to groupthink, not multiple perspectives

The argument in favor of panel interviews usually goes something like this: “if we have multiple people observing a candidate’s interview, we’ll get multiple perspectives on their answer, and therefore panel interviews collect more data than a single interview.” Sometimes, this is linked to preventing bias: “multiple interviewers mean there’s less of a chance of a single interviewer’s unconscious bias coloring results.”

These are reasonable assumptions; they make a sort of intuitive sense. In practice, though, I’ve never seen it turn out this way. Instead, every panel interview I’ve seen results in groupthink: I’ve never seen an argument after a panel interview. The panelists don’t have differing opinions; they all more or less agree on what they observed.

I think what’s going on is that you have multiple people all watching the same performance, so everyone on the panel is going to observe the same behavior. Panel interviews don’t collect more data; they collect the same data with multiple sensors. So the only differences that could even arise would be matters of interpretation. And then group dynamics will mean that the panelists will tend to want to agree with each other. (Or maybe the power dynamic is such that panelists don’t feel comfortable expressing disagreement?) Thus, we don’t end up with a range of opinions on a candidate; we end up with the same opinion repeated multiple times.

There is a way to collect multiple perspectives and to help prevent bias: multiple interviews. Instead of having three interviewers conduct a panel interview, have each interviewer meet with the candidate separately. If you’re particularly interested in some area or skill, have each interviewer ask about that same area. They can even ask the same question (though, if you’re going to do that make sure you let the candidate know that this will happen).

I’ve seen this produce far better results than panel interviews. When I run interview rounds with multiple overlapping interviews, I do see discussion and disagreement among interviewers. I think this is because interviewers aren’t seeing the same behavior: they’re each seeing a different response to the same or a similar question. Multiple interviews do collect more data because they’re observing more behavior.

Multiple interviews also give you a chance to course-correct or follow up. If one interviewer hears something concerning or weird, they can tell the next person to ask about that area again or in more depth. If something important isn’t covered, they can make sure later interviewers get to it. None of that is possible with panel interviews; it’s nearly impossible to make these kinds of adjustments in real-time.

So, although it can seem like panel interviews allow for better data collection, that’s not actually what happens. They merely gather the same data multiple times. If you want more data, or to better control for bias, run multiple one-on-one interviews.

Exceptions

There aren’t a lot of exceptions: nearly all the time, interviews should be one-on-one. But there are a couple:

1. Roles where performing in a group-interview-like scenario is part of the job.

   This is rare, but there are roles where you do get interviewed by a group of somewhat-judgemental people. Media relations/public relations is one: the job can sometimes literally involve being interviewed, so it’s reasonable to want to select based on someone’s skill in an interview.

   The other example is executive roles, particularly C-level roles (CEO, CTO, CISO, etc.). Talking to a board can be a lot like a panel interview; I once briefed a board on a security incident and the questions were far more probing and relentless than any interview I’ve ever had. Not hostile, exactly, but not comfortable or easy.

   Of course, what’s happening here is that you’re turning the interview into a work sample test. I’d still prefer an explicitly-designed work sample test to an accidental one as part of an interview, but if you must combine in this way, go for it.

2. A second person attends as an observer, for the purpose of training.

   As I wrote about in my piece on training interviewers, inviting new interviewers to observe real interviews is a great way to train them. But as that article covers, there’s a way to do this to avoid turning it into a panel interview:

   I conduct interviews with observers in a very specific way. At the beginning of the interview, I introduce the observer, explain that they’re there for training purposes, and won’t be talking or be involved in the decision-making at all. After that, I have the observer turn off their audio and video (or, for in-person interviews, move out of the sight-line of the candidate). This helps us both pretend the observer’s not there.

   This strikes a reasonable balance between letting you use the interview to help level up new interviewers while minimizing any additional stress.

Avoid panel interviews

Thanks for reading. I hope I’ve convinced you not to use panel interviews – or if you are, to stop. If you’re not convinced, email me and let me know what it’d take to change your mind: jacob @ this domain.