What's the most dreaded interview in the standard software engineering hiring funnel?  No no, not as an applicant...but an interviewer!  Is it the technical?  Coming up with questions that are neither brain teasers nor walks in the park?  The team-fit?  What do you even ask to get the right signals without injecting bias?

None of the above!  I think it's the phone screen.  In a phone screen you have limited time to make a judgment about a candidate...someone you have just met and know practically nothing about.  Worse, for many of us, phone screens are our first foray into interviewing.  Junior and mid-level engineers are given phone screens as a practice ground before being allowed into on-site technicals.  This folly aside, the phone screen touches every candidate in your pipeline and thus determines who even gets a chance to show what they're capable of in the later rounds.  Getting the phone screen wrong can have dire consequences for your hiring pipeline.

Is this candidate worth your time?

Each stage of an hiring funnel is designed to gather a specific type of data about the candidate, and that guides how those conversations are structured.  Unlike later stages, like the technical interview or the team-fit interview, the phone screen does not have an obvious main topic.  The consequence is that many interviewers have a natural inclination to treat the phone interview as a little bit of everything,  capturing a very incomplete picture of the candidate.  Without enough information to make a firm decision, our natural curiosity takes over and the candidate is let through to the next stage.

Such false positives can be a tremendous risk:

1. It wastes the time/resources of both parties, as the next stage is usually a technical.  For the candidate this requires preparation, taking time-off, and usually some travel.  For the company, this can be 8+ hours of lost productivity building and executing the interview.
2. The candidate is one step closer to a job offer, but you haven't actually made any progress confirming they are the right person for the job!  The cost of a bad hire is astronomical, so making sure each stage of the hiring funnel is actually contributing to making better decisions is more than just sound economics.

Every stage of the hiring funnel has a different goal that can be used to frame the conversation.  The technical and team-fit conversations are about getting to "yes."  You should leave such interviews convinced the candidate is the one for the job, that you would unequivocally answer "yes" when asked "should we hire this person?"  If you leave those meetings a "maybe," "I don't know," or "more information required," most companies will ere on the side of caution and pass on the candidate.

The phone screen is different.  The conclusion of a phone screen is not "yes" or "no"... it is "no" or "not no."

Bad phone screens happen when interviewers ask questions that are about getting to "yes," rather than getting to "not no."  There isn't enough time to build an adequate picture of the candidate, and the medium is poorly suited for the nuanced conversations that build up the confidence required to say "yes."  That means the data you extract from the phone screen is unlikely to be useful in making the final hiring decision as later interviews will do a much better job of revealing the pertinent facts about a person.

Therefore the phone screen serves to gate access to the more intensive technical and team-fit interviews, that's it.  It answers the question, "Is this candidate worth our time?"

A more altruistic way to frame this is, "Would this candidate have a positive experience with our interview process?"  If someone is just going to get steamrolled by the technical prompts in the on-site, or feel utterly out of their depth in the behavioral questions from the hiring manager, then save them some humiliation and pass now.

Getting to "Not No"

I won't suggest a generic template for all phone screens, the most effective interview loops are designed and consider not only the requisite job skills for the position, but also company values and culture.  What I outline here are the behaviors you can use as an interviewer to execute the design in a way that ensures the phone screen pulls its weight.

Keep it short and sweet

Phone screens should take 30-45 minutes at most, and feature a single interviewer—no panels.  By virtue of coming early in the hiring funnel, you will be doing a lot of them.  This also applies to the candidates who will be having similar calls with many other companies simultaneously.  Long, panel-based phone screens run counter to their proposed value of cheaply disqualifying candidates that would struggle in later stages of the interview.

Phone screens can be kept on time by limiting them to a single purpose: assessing the candidate.  They are not about explaining the role, bootstrapping the relationship, or preparing the candidate for the interview process...that should happen before the phone screen.

Therefore you should always precede phone screens with a "warm up" call.  The warm up call is your first high-fidelity moment of contact with the candidate—giving you an opportunity to start selling the position, answer questions about the company, and explain the interview procedure.  This post isn't about hosting warm-up calls or techniques for closing however: try this or this or this if you're interested.

Once you have 30-45 minutes of time dedicated to assessing the candidate, the next step is being realistic with what you can accomplish in that time window.  In my experience you can fit 4-5 simple technical questions (e.g. "Explain the difference between a process and a thread."), OR 1-2 behavioral questions (e.g. "Tell me about a time you ____" and follow-ups).

Without role-specific guidance, my general approach is to blend the two as follows:

• Open with a behavioral question based on something on their resume.  People like talking about themselves, it will make them comfortable and it shows you're interested.  As a secondary benefit, you get a quick read on how much of their resume is bluster and how much is legit.
• Move on with 3-4 technical questions.  Never make them up on the spot, and ideally practice them on a peer before trying them "in production."
• If there is any time left over, ask if they have any questions.

The hardest part about this format is it will feel awkward.  Transitioning cleanly and conversationally between different prompts is hard, and the result is it will feel a bit like recitation.  If the warm-up call has set the context correctly, then the candidate will come in expecting this and no harm, no foul.  Keep practicing, as masters of the craft will be able to keep the entire thing conversational, and thereby directly contribute towards closing the candidate.

Remove yourself from the conversation

The phone screen is supposed to be about them, not you.  It's very easy to slip into a call where the real thing being tested is how well someone can hold an engaging conversation.  Signs you may be taking over the conversation include:

• Faux active listening: you repeat back what you heard the candidate say, but you "clean it up" by applying your own knowledge to remove rough edges.
• Long speeches: you find yourself giving long explanations, usually in response to an implied question, as a way of sharing your ideology towards removing some of the mystery about yourself and the company.

The result of either is the same: it pulls the center of attention away from the person being interviewed, and onto you.  It's an unconscious assertion of dominance, and the candidates that pick up on this shift and play along with the appropriate rejoinders and conversational norms are likely to receive high marks from you in interview feedback.  You probably won't even realize the real person you're evaluating in the feedback is yourself.  This is a recipe for unconscious bias.

The root of the problem is that the power-dynamic in any interview strongly favors the interviewer.  Humans are biologically wired to key off power signals, even adopting similar manners of speech.  Beyond that, it's just sound strategy to figure out what an interviewer likes/wants and to provide that in the conversation.  By over-participating, you effectively give the candidate the answers by showing them something to imitate that they know you'll respond positively towards.

The solution?  Shut up.  Stop talking and listen.  Get the urge to correct them or "subtly" nudge them towards a better answer?  Don't.  Let them show you what they know and drive the conversation.  If you're actually interested in these candidates as people and not just resources, you'll give them the courtesy of being themselves.  Then, all you need to do is determine if their unique skills and personality will be a boon to the team.

Be selective

50% of all candidates should fail the phone screen.

If that number sounds high, consider that even at Verve—a tiny, no-name industrial software company where I led engineering—about 50 people made it through to the phone screen per open requisition.  With a 50% pass rate, that was still 25 people x 8 hours x 3 interviewers = 600 hours of interviewing in the subsequent stages.

More than that, if >50% of people are passing it's probably because you aren't collecting the right information about the candidate.  You probably tried to figure out if they are a fit for the job, rather than if they would thrive in your interview process.  There's just no way to do that in a single sitting over the phone, and your natural curiosity likely leads you to "giving them the benefit of the doubt" and forwarding them through to the next round.  You don't want to miss out on a good candidate right!?

Instrument your hiring funnel, and keep an eye on the number of candidates that are passing.  When that number crests above 50%, it's time to conduct an investigation and figure out what's going on in your interviews.  Sometimes the cause will be benign—an uptick in recruiter-sourced candidates is likely to correspond to an uptick in phone screen success—other times it's more serious.  I call process health indicators like this "trip wires," and they strike a balance between negligence and hyper-vigilance.

Don't talk about experience

At first glance, it's not obvious what sorts of questions work well over the phone.  Technical questions are limited by the lack of access to a whiteboard, IDE, or other tools.  If you take the advice of "removing yourself from the conversation," than you may be inclined to ask about the candidate's prior work.  That should reveal a lot about them right?

Maybe, but it is more likely to lead to the same outcome: a feeling of uncertainty and an inclination to give them the benefit of the doubt.

The problem with asking people about their experience is that they almost certainly know more about it than you.  Unless they have worked in an identical area, you don't have enough expertise to assess the caliber of their answers.  There can be no "wrong" answers when talking about experience, so you're liable to apply your own personal judgments to their background and whether or not it is relevant to the position at hand.  Instead, you need to let the candidate show you that their experience has prepared them for the role.  This allows you to open the door to people from non-traditional and under-represented backgrounds that might be at a disadvantage under experienced-based regimes.

The sorts of questions that allow a candidate's to apply their experiences rather than recite them, while upholding an objective standard, are also the same sorts of question that make it easy for you to shut up and listen rather than dominate the conversation.

Good technical questions for a phone screen...

• ...permit a range of responses that will reflect a candidate's knowledge.  Think like a grading rubric from primary school: needs improvement, developing, proficient, accomplished, and distinguished.  You-know-it-or-you-don't questions like, "What is the signature of the copy-constructor in C++?" are a bad choice because the range of responses is binary.
• ...are specific and do not allow a candidate to hand-wave an answer without revealing something about their knowledge level.  "Describe how decorators are defined and used in Python," is better than, "Tell me about your level of proficiency with Python?"
• ...tie back to a specific job requirement.  The phone screen shouldn't feel like a CS quiz.  Not going to have this person write sorting algorithms once they get hired?  Then you probably shouldn't ask about quicksort.
• ...challenge the candidate and solicit a complex response.  Questions that can receive 1-sentence answers, or seem excessively trivial, will make the candidate feel like they are completing a verbal version of fizz buzz.  I have actually watched someone ask a candidate, "What is an IP address?" only 30 seconds after they got done answering a 20-minute long design question.

A question I frequently use, despite it's surface appearance as CS trivia, is "Explain to me how a hashtable works in as much detail as possible."  The number of possible answers and levels of detail is endless, it's just about the only data structure 100% of developers use no matter the job, and I can sit back and let the candidate drive instead of having to "pull" them through a multi-part scenario-driven question.

Technical questions aren't the only safe haven for avoiding experiential questions. Start by picking a functional skill that is important to the job.   For example building consensus is critical for senior-level developers that are expected to be change agents.  Then probe the candidate to explain how that functional skill is done e.g. "What does it mean to be good at building consensus?" and "What do you do when you are trying to build consensus?"  Candidates will have a tendency to talk in abstract, so push them to talk about what they would do (or have done) specifically.

Candidates who can do the job will be able to answer any of the above questions without making appeals to their experience to make their points.  It shows they can apply analytical reasoning to determine the best course of action, and would be able to share that reasoning to build support and cohesion once they are in the role.

Prioritizing the Litmus Test

The software engineering interview is an evolving art, and as screening procedures have been found to be ineffective or prejudice, we have discarded them.  It's been a long time since I've seen companies use brain teasers, and whiteboard-centric algorithms tests seem like they might be next on the chopping block.

Phone screens, however, are unlikely to leave us anytime soon because many of the interview processes that are emerging to replace teasers, whiteboards, and algorithms are even more time-intensive.  That means we need increasingly effective early-stage screening to make such time investments tenable for both the organization and the candidate.

If this is your first foray into interviewing, don't assume it's just a training ground before you move on to the "big leagues" of on-site technicals.  The phone screen is an equally critical node in the hiring funnel; you do yourself, your employer, and your candidates a service by taking the time to master the art.

I have started referring to these types of phone screens as litmus tests (a metaphor for a practice by the same name in politics).  It's a nifty short-hand to combine the mindsets of "getting to not no," "are they worth my time?" and "will they have a positive experience?"  For those that are mathematically inclined, you may prefer the expression "necessary but not sufficient."  With these first principles, you will be able to go out in the world and host effective phone screens—phone screens that are short, emphasize the candidate's skills over their background/conversation abilities, and selective.