I like this take-home approach. Usually, I've been against that kind of outside work but in rare cases (i.e. leadership positions) have assigned take-home case study style presentation work to gauge communication skills. For engineers, though, I've taken a different approach.
If they've passed the basic screening with the TA folks and I've validated they didn't just regurgitate buzzwords to get in the door, I'll bring them in for an on-site (or do a virtual onsite). This consists of culture fit interviews with their direct team as well as with key stakeholders from elsewhere in the org. And a coding test. Directly with me.
But the coding test isn't LeetCode. It's an intentionally incomplete and ambiguous task related to the work they'll be doing in the job gauged less at evaluating their technical skills (though that is in fact a part) and more validating whether or not they ask questions and can engage with stakeholders.
For example - "Here's a dump of time-series data from one of our APIs. The key consumers of this data don't understand JSON feeds so they need you to build a visualization atop the data."
That's it. That's the task. Anyone who dives in and just starts hacking away is probably not a fit (they might be coachable ... but that's something we feel out through this and later followup conversations). Someone who stops and asks "ok, what does this data actually represent" or "who are the key consumers and what will they need to accomplish with this data" _before_ they dive into the code are the folks I want on the team.
In other words - am I looking at a great communicator with excellent collaboration skills, or is this someone who just wants to be told what to do and will take no ownership over the product direction? The most successful teams I've built bias heavily towards the former.
If they've passed the basic screening with the TA folks and I've validated they didn't just regurgitate buzzwords to get in the door, I'll bring them in for an on-site (or do a virtual onsite). This consists of culture fit interviews with their direct team as well as with key stakeholders from elsewhere in the org. And a coding test. Directly with me.
But the coding test isn't LeetCode. It's an intentionally incomplete and ambiguous task related to the work they'll be doing in the job gauged less at evaluating their technical skills (though that is in fact a part) and more validating whether or not they ask questions and can engage with stakeholders.
For example - "Here's a dump of time-series data from one of our APIs. The key consumers of this data don't understand JSON feeds so they need you to build a visualization atop the data."
That's it. That's the task. Anyone who dives in and just starts hacking away is probably not a fit (they might be coachable ... but that's something we feel out through this and later followup conversations). Someone who stops and asks "ok, what does this data actually represent" or "who are the key consumers and what will they need to accomplish with this data" _before_ they dive into the code are the folks I want on the team.
In other words - am I looking at a great communicator with excellent collaboration skills, or is this someone who just wants to be told what to do and will take no ownership over the product direction? The most successful teams I've built bias heavily towards the former.