Thinking out loud, here’s one idea for an LLM-assisted interview:
- Spin up a Digital Ocean droplet
- Add the candidate’s SSH key
- Have them implement a basic API. It must be publicly accessible.
- Connect the API to a database. Add more features.
- Set up a basic deployment pipeline. Could be as simple as script that copies the code from your local machine to the server.
Anything would be fair game. The goal would be to see how the candidate converses with the LLM, how they handle unexpected changes, and how they make decisions.
For the time being, I’ve banned LLMs in my interviews.
I want to see how the candidate reasons about code. So I try to ask practical questions and treat them like pairing sessions.
- Given a broke piece of code, can you find the bug and get it working?
- Implement a basic password generator, similar to 1Password (with optional characters and symbols)
If you can reason about code without an LLM, then you’ll do even better with an LLM. At least, that’s my theory.
I never ask trick questions. I never pull from Leetcode. I hardly care about time complexity. Just show me you can reason about code. And if you make some mistakes, I won’t judge you.
I’m trying to be as fair as possible.
I do understand that LLMs are part of our lives now. So I’m trying to explore ways to integrate them into the interview. But I need more time to ponder.
His book, Rebuilding Rails, had a huge impact on me, especially early in my career (when I started learning Rails). Such a great teacher. He encouraged me to be curious, dig deeply, and to understand my tools.
Such a badass developer. And such a kind and genuine person.
- Spin up a Digital Ocean droplet
- Add the candidate’s SSH key
- Have them implement a basic API. It must be publicly accessible.
- Connect the API to a database. Add more features.
- Set up a basic deployment pipeline. Could be as simple as script that copies the code from your local machine to the server.
Anything would be fair game. The goal would be to see how the candidate converses with the LLM, how they handle unexpected changes, and how they make decisions.
Just a thought.