Jacob Kaplan-Moss

Tag: interviewing

My interview kickoff script, annotated November 29th, 2018

When I interview, I say nearly the same thing at the beginning of the interview. It’s a script I’ve practiced and honed over the years . It’s only eleven sentences, but each has a specific purposes. I’ve iterated on this for years, and it’s pretty tightly honed at this point. I published this script in the guide to interviewing I wrote at 18F last year, but never got a chance to break down where it comes from…

IQ isn't enough to get hired April 3rd, 2019

I’ve interviewed hundreds of people for technical roles, and a pattern has emerged. In general, we reject many more candidates for social skills than for technical competence. In fact, most technical interview funnels are arranged so that technical screens are earlier. This means that if you reach an interview, you’ve probably (mostly) passed the technical part, and are now being judged mostly on EQ - emotional intelligence. If you get rejected at this point, it’s probably because of social skills.…

My questions for prospective employers (Director/VP roles) April 23rd, 2019

Last time I was looking for a job, I wrote up a list of questions I wanted to ask prospective employees. I just ran across the list again, and figured I’d share. I was looking for a senior management role (Director/VP-level) in Engineering or Security, so the questions are sloped in that direction. Also note that I was in a fairly strong position; I didn’t need the a job immediately. So, I was able to ask fairly direct, challenging questions.…

Training Interviewers September 8th, 2020

What’s the best way to train folks to conduct job interviews? I have a process I’ve used for about five years that seems to work well. It’s loosely based on the “see one, do one, teach one” methodology used by many medical schools.

Unpacking Interview Questions: “Explain a Topic At Multiple Levels…” February 8th, 2021

Part 1 of my Unpacking Interview Questions series, where I share one of the questions I use when I interview for technical roles. Today: asking candidates to explain a topic at multiple levels. This is one of my favorite questions to ask for engineering roles; strong performance on this question correlates very highly with high job performance on my teams.

Unpacking Interview Questions: “Tell Me About a Project You Led…” February 9th, 2021

Part 2 of my Unpacking Interview Questions series, where I share one of the questions I use when I interview for technical roles. Today: measuring a manager’s ability to lead projects and manage them effectively.

Unpacking Interview Questions: Diversity, Equity, and Inclusion February 10th, 2021

Part 3 of my Unpacking Interview Questions series, where I share one of the questions I use when I interview for technical roles. Today: making sure candidates align with organizational values of diversity, equity, and inclusion.

Unpacking Interview Questions: “Tell Me About a Disagreement…” February 11th, 2021

Part 4 of my Unpacking Interview Questions series, where I share one of the questions I use when I interview for technical roles. Today, an oldie-but-goodie: looking into a candidate’s ability to disagree and resolve conflict professionally.

Unpacking Interview Questions: The Weakness Question February 12th, 2021

The fifth and final part of my Unpacking Interview Questions series, where I share one of the questions I use when I interview for technical roles. Today’s question is the most difficult-to-ask of the series, but also one of the most valuable: asking a candidate to discuss one of their weaknesses.

Unpacking Interview Questions: Interview Question Series Wrap Up February 15th, 2021

A summary and wrap-up of my Unpacking Interview Questions series, covering why I wrote this series in the first place, some advice on developing your own questions, and answers to a few questions.

Unpacking Interview Questions: Types of Interview Questions March 1st, 2021

There are three types of interview questions: behavioral, hypothetical, and trivia. Behavioral questions are the gold standard; they’re the most effective at predicting job performance. Hypothetical questions can be useful in certain circumstances, if used correctly. Avoid trivia.

“Why Did You Leave Your Last Job?” March 8th, 2021

Asking candidates why they left their last job is common… but should it be? I’m not sure. What do you think?

Mailbag: Adapting Interview Questions for Junior Candidates March 29th, 2021

Sébastien asks: “Would you say that your questions are equally applicable to junior and senior positions?”

Work Sample Tests: Introduction to Work Sample Tests November 9th, 2021

Earlier this year, I wrote a series on interview questions. Good interview questions are one key to hiring well, but they’re not the only key. Today, I’m starting a new series on another critical factor in effective hiring: using work sample tests, aka practical exercises. This is part 1: what are work sample tests, and why do we need them?

Work Sample Tests: The tradeoff between inclusivity and predictive value November 10th, 2021

Good hiring processes try to maximize inclusivity and predictive value, but unfortunately, work sample tests bring these goals into conflict. There’s always a tradeoff between predictive value and inclusivity. The guiding principle of work sample tests is: construct a test that balances predictive value and inclusivity. Fair work sample tests will be predictive enough to give you a high degree of confidence that you’re making a good hire, while also being designed to be as accessible to as many candidates as possible.

Work Sample Tests: A Framework for Good Work Sample Tests: Eight Rules for Fair Tests November 17th, 2021

What makes a work sample test “good” – fair, inclusive, and with high predictive value? Here’s my framework: eight principles that, if followed, give you a great shot at constructing a good work sample test.

Work Sample Tests: Coding “Homework” November 23rd, 2021

Coding homework is my default work sample test: I use it for all engineering roles unless it’s obvious that another kind of exercise is better. There are good reasons to make homework-style work sample tests the default: they’re relatively easy to construct, they scale reasonably well to large hiring rounds, they’re accurate simulations of real work, and easier than most other kinds of tests to construct in a way that maximizes inclusivity. Here’s how to conduct a coding homework work sample test.

Work Sample Tests: Pair Programming November 30th, 2021

I tend to prefer asynchronous work sample tests. The flexible scheduling of asynchronous exercises (i.e. “work on this whenever you like”) works better for the majority of candidates. But for some candidates, and some roles, synchronous exercises work better. By “synchronous” I mean: work sample tests that are explicitly scheduled, and that has both the interviewer and the candidate working directly together at the same time. In these cases, I often turn to pair programming.

Work Sample Tests: Bring Your Own Code December 7th, 2021

If you’re hiring engineers, some candidates will already have code they can share: side projects, open source, and so on. It’s silly to ask those candidates to write new code just for your interview if they already have code they can share. So, if you’re asking candidates to code as a work sample test, you should also offer to let candidates submit something they’ve previously written. Here’s how.

Work Sample Tests: ‘Reverse’ Code Review December 15th, 2021

For most software engineering roles, the best work sample test will be some combination of the exercises I covered earlier in this series. But not every role; there are some circumstances where other types of tests fit better or are better at revealing some critical piece of information relevant to hiring. This post covers one of them: a “reverse” code review, where instead of you reviewing the candidate’s code, you have them review yours.

Work Sample Tests: Labs & Simulation Environments December 24th, 2021

The work sample tests I’ve covered in this series so far all involve software development. But what about roles that don’t involve day-to-day coding: roles like security analysis, penetration testing, technical support, bug bounty triage, project or program management, systems administration, technical operations, and so on? For those roles, I turn to simulated, “lab”-style environments. Here are some examples of that kind of test.

Work Sample Tests: What doesn't work (and why) December 30th, 2021

I’ve written about a bunch of effective work sample tests and the “rules of the road” that make them effective. One thing I haven’t talked about is counter-examples: types of work sample tests that don’t work. I tend not to do this sort of thing: I find it’s usually more useful to talk about what does work than to pick apart what doesn’t. But here, I think it’s illustrative: looking at why certain kinds of work sample tests fail can help illustrate the principles of effective tests. Let’s look at a few kinds of work sample tests that (usually) fail, and why.

Work Sample Tests: Wrap Up and Q&A January 6th, 2022

This is the final post in my series on work sample tests. It’s a wrap-up post: I’ll address a few random points I couldn’t quite fit in elsewhere, and answer some questions from readers.

Developing a Values Interview Question January 31st, 2022

How do you develop an interview question that measures a core value?

Panel interviews don't work July 8th, 2022

There’s a Right Way to conduct job interviews: one-on-one, with a single interviewer per interview session. If you need multiple interviewers (you probably do), schedule multiple sessions, each one-on-one. The alternate approach, panel interviews – having multiple interviewers in a session at once – is almost always a bad practice. It increases stress on the candidate, risks measuring the wrong things, and doesn’t lead to better results. Avoid panel interviews: they don’t work.

Taking notes in interviews August 12th, 2022

Techniques for effective note-taking during interviews.

Post-interview recommendations: a case against 'maybe' September 23rd, 2022

If you’re ever an interviewer on a role I’m hiring for, there’s this one thing I’m going to ask you to do that might feel weird. After you conduct that interview, I’m going to ask you to send me a recommendation, and I’m going to insist that the recommendation begins with a very clear “hire” or “no hire”. I won’t accept any form of “maybe”.

When Is Short Tenure a Red Flag? October 14th, 2022

A pattern of short tenure – multiple short jobs over a small period of time – can be a red flag, but necessarily. I’ll dig into short tenure in this article. What’s “short”? What’s a “pattern”? When you’re thinking of switching jobs, when and how should your tenure factor into the decision? For hiring managers, when is it reasonable to be concerned about job tenure, and what should we do when we become concerned?

The Intersection of Tenure and Seniority October 20th, 2022

Patterns of short tenure are normal at the beginning of a career, but are more of a red flag in more senior roles. Here’s why.