How to interview software engineers remotely

My experience hiring coders during the pandemic

Photo by Headway on Unsplash

I am a software engineer and the team lead of a development group. Over recent months, my team has had to fill several positions, and we needed to update our hiring process in light of this pandemic. Our normal interview process brought people in for an interview, let the group meet them, and did some whiteboard coding. Admittedly, this was a very outdated process and, after February, none of that was an option any longer.

This gave us a great opportunity to modernize, so I worked with management and other lead engineers to come up with a better solution.

Some of the very big tech companies have interview processes that can last many days, but no one is going to put up with that for smaller companies. We needed a process that fit a candidate's schedule.

We settled on using a web platform called Coderbyte to conduct a focused skills test. We then used Microsoft Teams to give “in person” interviews. While using this method, I interviewed several candidates and eventually hired two of them.

Here's how we conducted successful interviews:

Coderbyte

Coderbyte has a developer side that offers interview prep and an employer side that offers interviewer capabilities (note: these are not affiliate links, I get no kickbacks from Coderbyte). We used the employer side to create a short skills test to verify candidates basic knowledge and communication skills. Our goal was to design a test that would take less than an hour to complete, but would give us insights into the candidate and weed out those who clearly didn’t fit.

Setting up a skills test in Coderbyte isn't difficult. We chose a format using 4 to 8 multiple choice questions, 3 short answer questions, and 3 "challenge" questions. Coderbyte lets you select from existing questions or write your own. With the exception of the challenge questions, we chose to write our own based on interview questions we have used in the past.

Below I will explain the methodology and not the questions themselves, both because Coderbyte offers plenty of free questions to use, and because "what questions should I ask in an interview" is a topic that's been covered to death. This article is about why the format worked and what to focus on in each category.

Photo by Ferenc Almasi on Unsplash

The short answer questions were tailored to the position, programming language, and position skill level and were meant to ensure the candidate understood aspects of the language that would be necessary for their job. Particularly, we might ask one or two conceptual questions about the frameworks or architectures we use, and a few "what will this code output" questions that show basic knowledge of the language. If a candidate couldn't answer these questions correctly (or mostly correctly), we would know right away that they would be a poor fit for the position.

For the short answer questions, the question itself didn't matter much. The main goal of these questions was to see how well the candidate communicated in writing. In the current world of remote working, being a strong communicator is a key skill.

Finally, Coderbyte offers "challenge" questions, which are short coding problems. Here we used built in questions from Coderbyte's library. Coderbyte records the candidate's work, does some rudimentary plagiarism detection, and then tests the answer against multiple inputs to generate a score. We were looking for both programming style and correctness in this section.

We calibrated these tests against our own engineers in similar positions. When our engineers scored higher than 70% for a test at their level and finished in about 45 minutes, we considered the test to be ready.

Following a successful skills test, the candidate would get a call from HR to determine possible start dates, share benefits packages, and to let them ask a few basic questions. The manager in charge of the team followed up with a phone call or MS Teams call to introduce themselves and decide if the candidate logistically could start working with us (salary range, relocation expenses, working hours, etc.). If those two quick phone calls both went well, it was on to the interview by the team.

Microsoft Teams Interview

Photo by Austin Distel on Unsplash

For the final “team” interview, we would start with a fifteen minute virtual meeting that included myself, one or two other senior engineers, and a junior engineer. We began by reviewing the candidate's Coderbyte test, taking note of wrong answers to the multiple choice section and issues or style questions in the challenge section. We also went over the candidate's resume.

Next, we created a sample of about fifty lines of code that was complete enough to compile and run, but had some bugs in it. We'd use this for an on the fly debugging session. Our code was pulled directly from our developed products and simplified into isolated components. We could also use this to describe our shop's code style.

We were then ready for our video interview. Video is important as it helps you all connect and gives the candidate a much better feel for the group.

The interview format itself was as follows:

• Less than five minutes: Manager introduction of the team and the candidate. Manager leaves the interview after this introduction.
• Five minutes: Team lead introduces themself and the project / company, including a quick introduction to processes the team follows (agile, monorepo, etc) and the format of this interview. This lets the candidate know what's coming and where they have time for questions.
• Five minutes: Quick introduction of the engineers on the interview, including their perspective of the project, manager, process, and company.
• Ten minutes: Let the candidate introduce themselves. Ask follow up soft questions: Why do you want to work here, what sounds good, what sounds difficult, what is interesting about this project, etc.
• Ten minutes: Go over the Coderbyte questions. I mostly liked going over the challenge questions (screen share) to let the candidate explain their thinking. If any challenge questions were incorrect, it was an opportunity to talk about edge cases or improvement.
• Fifteen minutes: Next, it was time for the live debugging session. We'd share the text via Teams' file sharing feature (warning: we often had to rename it to .txt as Teams can block several file formats), ask the candidate to share their screen, and start with a the issue a customer may have discovered, e.g.: "When I press this button, nothing happens." We'd let the candidate talk us through debugging from there. The first bug would be fairly straightforward. There'd be a second bug in the code that we'd introduce which would be much harder to find (hopefully). This was to see how the candidate asked for help when we introduced the second problem. Asking for help is a great sign of a quality candidate even though it's frustrating to do.
• Remaining time, usually ten minutes: Finally, it was the candidate's turn for Q&A. Hopefully we had built up some camaraderie so the candidate felt open in asking those "tough" questions they might not want to ask of management or company representatives. Questions about our direct managers and about the "real" company culture were common and encouraged.

After the interview, our internal team would schedule a final fifteen minutes to discuss the candidate. Usually this was very straightforward. We asked ourselves if the candidate was a good fit for the team and what level we thought they would fall into (junior dev, dev, senior dev, etc). Then we'd make that recommendation to management and it was up to management and the candidate to find a deal that worked for them.

Image by mohamed Hassan from Pixabay

And there you have it. Not too onerous for the candidate (a little more than two hours of their time), not too much time taken for the larger engineering team, and a pretty good way to judge an incoming candidate while weeding out people who likely shouldn't be interviewed. The first time or two creating these tests were slow for me, but I quickly was able to build tests once I had gone through the process a time or two.

Our company was successful in finding and hiring several quality developers using this process. I hope that even once we're able to return to whiteboard in-person interviews that we choose not to and instead keep a process like this one.

If you found this helpful, see our other articles on interviews and interview prep:

This Simple Interview Question Is Also The Hardest To Answer

This Is The Most Overlooked Step In Interview Prep

Citizen Upgrade is a community of experts covering technology, society, and personal development. Visit us at our website, on Facebook, or on Twitter.