How to Use Coding Challenges with Your Interview Process

Entire libraries could be filled with tips and tricks about how to ace a coding challenge. This step of the interview process is so feared that few developers think about the other side of the coin: what goes into creating an effective coding challenge? How do you design a problem that will select candidates that your company wants to hire?

What a coding challenge should accomplish

The goal of a coding challenge, like any other step of an interview, is to determine what kind of employee the candidate would be. A good challenge should reveal a coder’s hard technical skills, their approach to problem solving, and their ability to think clearly under the pressure of an approaching deadline. One common mistake many CTOs make is using a challenge with a widely-known solution, or creating a challenge that does not accurately reflect a professional coding environment. These mistakes do no favors to you or to the candidates you are vetting. Here are some steps to take that will make your challenge more than a formality.

Use your resources wisely

You don’t necessarily need to reinvent the wheel. Sometimes holding your own coding challenge is impractical. If you have a large number of candidates to get through, giving each one a coding challenge and then analyzing each one can take too long, particularly for smaller start-ups. If this is the case for you, there are a number of third-party companies that challenge prospective candidates for you: a choice that reduces some of the costs of coding challenges. The most popular of these are TopCoder, Coderbyte, and HackerRank.

Ensuring your challenge effectively screens candidates

Choose the right topic

The goal of a coding challenge should be to help you determine how well the candidate would fit into your team. In order to make a judgment about that, your coding challenge should be representative of the kind of work that they would be doing if you hire them. Classic coding challenges like FizzBuzz or designing a simple REST interface won’t help you here, because these standardized challenges won’t help you distinguish candidates: any programmer worth interviewing can handle these. This means that there is no one best coding challenge; it has to be customized so that it can simulate the working conditions and needs of your company.

In-person or take-home?

Another serious consideration is how to administer the coding challenge; should they take it home or do it in person? Neither option is better than the other, and many companies do both. What type of challenge you use depends on what kind of role you want to fill, and where your priorities are.

The primary advantage of an in-person coding challenge is that it gives you the chance to see how a candidate approaches a problem in real time. It is the best way to determine how well they respond to a deadline in a professional environment. If your team pair programs frequently, consider having paired programming be a part of this challenge. When attempting to find somebody who will be a good cultural fit into the company, an in-person challenge is perfect, because if your experience with them during the interview doesn’t go well, they’re not likely to impress you any further if you hire them.

Giving candidates a take-home challenge gives you insight into other strengths and weakness. Since take-home challenges tend to be longer, you can examine their approach to larger projects. More importantly, you’ll see how they code when a prospective employer isn’t looking over their shoulder. Do they still adhere to best practices when they aren’t being watched?

Whether you administer an in-person challenge, a take-home challenge, or both, consider how long it should take to finish. An overly long challenge will not give you any more insight than a shorter one; all you’ll do is drive away candidates who might not be able to afford to take a lot of time off of work. A general rule of thumb is that an in-person challenge shouldn’t last longer than three hours, and a take-home challenge shouldn’t take more than eight.

Evaluating a candidate’s performance

Once the coding challenge is done, how should it be judged? The most important thing to keep in mind is not to fixate on whether or not they got the “correct” answer. Look at the candidate’s process, and think about what your team values most. If test driven development is a priority, look at their unit test.

Another thing to assess is if they understood the requirements of the challenge. Any developer needs to show an ability to grasp the requirements of a given project, and a willingness to ask for clarification if they are confused. When you look at the mistakes they made, look to see if they were due to a misunderstanding or to lack of attention to detail.

Finally, it’s worth taking the time to talk through the challenge with the candidate afterwards. This will help you understand their thought process, and most importantly, it will reveal how they respond to criticism. If you rely only on comparing the candidate’s solution to the one you had in mind, you can overlook a strong coder’s bad qualities, or a weaker coder’s good qualities. Skills can be developed further, but if the person responds poorly to criticism or is difficult to have a conversation with, they may be more of a detriment than a help to your team.

It’s not just about the code

A good coding challenge can reveal a candidate’s strengths and weaknesses from both a technical and interpersonal view. The ROI on a well-executed and reviewed coding challenge is high, since it’s the closest thing you’ll get to seeing how well a person would work in your team’s environment. Coding challenges are  too valuable a tool for a CTO to phone in. If you don’t have the time or inclination to create a unique challenge, at least find a way to put some kind of twist on one of the classics. Remember, your coding challenge should be challenging and realistic, and you should set clear expectations. Giving candidates a “gotcha” challenge will not help you determine their value, and will hurt your company’s reputation. After all, an interview is also a chance for prospective employees to understand your values and philosophy, and word of unreasonable interviews tends to spread. If you take the time and effort to craft a strong coding challenge, you will find yourself hiring stronger coders, and better employees in general.

Summary of strategies

Your time is valuable, so here are the things to keep in mind when considering how to approach coding challenges:

1. Don’t forget that you can outsource coding challenges if it would be more cost-effective
2. Either create a code challenge to reflect the needs of your company or adapt an existing challenge to reflect what you need the developer to be able to do on day one of the job
3. Don’t make the coding challenge too long or too cute
4. Determine from the outset how you will evaluate a candidate’s performance on the coding challenge
5. If possible, talk through the coding challenge with prospective hires to see how they respond to feedback and criticism