Linux sysadmin interview cheating countermeasures(threads.com)
threads.com
Linux sysadmin interview cheating countermeasures
https://www.threads.com/@nixcraft/post/DR6kxiYjNt9
4 comments
Yes it's come to this for us, too. We had 3/3 interviews this week clearly and mechanically reading definitions at us instead of discussing tradeoffs. We're a full remote company but we're considering onsite interviews now.
Consider pair work (a generalization of pair programming):
Spin up a VM for you and the candidate to SSH into. Start a shared screen/tmux session. Work on some sysadmin task(s) together. Switch between driving/typing and navigating/directing at regular intervals (15 minutes is typical, though a 30 minute interview would require a <10 minute interval).
You'll have to come up with a task (or set of tasks) that deals with the skills you need. The task(s) would ideally be very difficult for an LLM or search engine to provide a complete answer for; but they shouldn't be gotchas, since that might be false negatives.
Test your task(s) with other people in your company first, so you'll be comfortable when doing it with a candidate.
I can't say that this will 100% prevent cheating. But, you will surely learn a lot about the candidate's habits, in terms of reasoning and communication. If the candidate pastes commands, then that's a possible indication of LLM-usage; if the candidate doesn't use tab-completion, then they might be typing text in from somewhere else (e.g. an LLM, a guide).
I enjoy using the command-line, so I suspect I would enjoy this sort of interview.
At any rate, I hope you find a good candidate.
Spin up a VM for you and the candidate to SSH into. Start a shared screen/tmux session. Work on some sysadmin task(s) together. Switch between driving/typing and navigating/directing at regular intervals (15 minutes is typical, though a 30 minute interview would require a <10 minute interval).
You'll have to come up with a task (or set of tasks) that deals with the skills you need. The task(s) would ideally be very difficult for an LLM or search engine to provide a complete answer for; but they shouldn't be gotchas, since that might be false negatives.
Test your task(s) with other people in your company first, so you'll be comfortable when doing it with a candidate.
I can't say that this will 100% prevent cheating. But, you will surely learn a lot about the candidate's habits, in terms of reasoning and communication. If the candidate pastes commands, then that's a possible indication of LLM-usage; if the candidate doesn't use tab-completion, then they might be typing text in from somewhere else (e.g. an LLM, a guide).
I enjoy using the command-line, so I suspect I would enjoy this sort of interview.
At any rate, I hope you find a good candidate.
never ever waste interview time asking candidates questions that can be looked up in a book.
seriously, interview 101, your poor interview skills are the problem if you are worried about ‘cheating’.
seriously, interview 101, your poor interview skills are the problem if you are worried about ‘cheating’.
Absolutely. We give a system design problem in a blank shared document. as a conversation starter.
Even open questions like "how did you select that technology component over this other one, and what are the tradeoffs?" We're getting long-winded, overly formal, mechanical definitions instead of an honest conversation.
Even open questions like "how did you select that technology component over this other one, and what are the tradeoffs?" We're getting long-winded, overly formal, mechanical definitions instead of an honest conversation.