With the tests in the pairing exercise, do you expect a senior to solve four of them and a senior+ all five of them? Or do you pick one or two based on level and just do those?
Hi Dan! My general rule of thumb when designing the method was to start a senior candidate at Level 3 (mid-level) and let them go from there. With only 40 minutes and also being realistic about skill level, asking a senior candidate to change a button colour is a waste of time.
Have you tried asking an AI to look for potential issues in the code — thinking about that one hour upfront period they get — and what happens if you do?
I haven't but it's a good idea! I'll try it :) That said, it doesn't matter because the tasks are usually about implementing something specific, rather than just looking for problems in the existing code. Though that could potentially kick in at the senior+ level.
Thanks for those insights. AI cheating is real, and I’ve seen two extreme schools of thought when it comes to handling it. The first go back to face to face interviews (where it’s still possible), and the second adopt a more liberal approach similar to the one you’ve share. I love how your approach handles the “shame” factor in using AI tools. What’s been your experience with the quality of engineers you’ve hired with this process compared to other approaches? Also do you have any data on what other tech companies are doing these days (esp hi tech)?
Hi Gaurav, I don't have much data on other companies. My goal here was to focus on what we were doing and to improve on that, since for example FAANG's focus on leetcode comes from a different goal than ours.
This is also a relatively new process so I don't have enough data to share on the quality of the engineers yet - though I will say that with this new approach we switched from it being a 'boolean interview' (yes/no) to it being a contributing interview. In other words, we're using this test to level candidates and to get an idea of their strengths/weaknesses, rather than to exclude them from the interview process. As the hiring manager, I combine this information together with the rest of what we learn about them, and try to go with the best fit in terms of adding something new to our teams, bringing the appropriate skills, having the right motivation/drive, but also if we can offer them what they're looking for in their career development. So of course I think I make good hires (I'm a little biased!) but even if I do, I don't think that's entirely down to the technical interview.
It's great to see more thought that includes a better experience for the candidates and making better decisions for the company. How many technical interviews are there, and how many employees are in each round with the candidate? Do you have a single 90-minute technical session or multiple?
This is very good
With the tests in the pairing exercise, do you expect a senior to solve four of them and a senior+ all five of them? Or do you pick one or two based on level and just do those?
Hi Dan! My general rule of thumb when designing the method was to start a senior candidate at Level 3 (mid-level) and let them go from there. With only 40 minutes and also being realistic about skill level, asking a senior candidate to change a button colour is a waste of time.
Makes sense, thank you!
Have you tried asking an AI to look for potential issues in the code — thinking about that one hour upfront period they get — and what happens if you do?
I haven't but it's a good idea! I'll try it :) That said, it doesn't matter because the tasks are usually about implementing something specific, rather than just looking for problems in the existing code. Though that could potentially kick in at the senior+ level.
Enjoyed reading the article, guys, Gregor and Anna! 🙌
Thank you!
Thanks for those insights. AI cheating is real, and I’ve seen two extreme schools of thought when it comes to handling it. The first go back to face to face interviews (where it’s still possible), and the second adopt a more liberal approach similar to the one you’ve share. I love how your approach handles the “shame” factor in using AI tools. What’s been your experience with the quality of engineers you’ve hired with this process compared to other approaches? Also do you have any data on what other tech companies are doing these days (esp hi tech)?
Hi Gaurav, I don't have much data on other companies. My goal here was to focus on what we were doing and to improve on that, since for example FAANG's focus on leetcode comes from a different goal than ours.
This is also a relatively new process so I don't have enough data to share on the quality of the engineers yet - though I will say that with this new approach we switched from it being a 'boolean interview' (yes/no) to it being a contributing interview. In other words, we're using this test to level candidates and to get an idea of their strengths/weaknesses, rather than to exclude them from the interview process. As the hiring manager, I combine this information together with the rest of what we learn about them, and try to go with the best fit in terms of adding something new to our teams, bringing the appropriate skills, having the right motivation/drive, but also if we can offer them what they're looking for in their career development. So of course I think I make good hires (I'm a little biased!) but even if I do, I don't think that's entirely down to the technical interview.
It's great to see more thought that includes a better experience for the candidates and making better decisions for the company. How many technical interviews are there, and how many employees are in each round with the candidate? Do you have a single 90-minute technical session or multiple?