CloudFlare “Interview Questions”(blog.cloudflare.com)
blog.cloudflare.com
CloudFlare “Interview Questions”
https://blog.cloudflare.com/cloudflare-interview-questions/
75 comments
My first reaction to the "fastest traceroute" question would be, is this a trick question? This is because many routers rate-limit the number of ICMP packets they forward. Causing a lot of ICMP replies could easily exceed this limit and result in missing hops. This requires being deliberately slow. A lot of the early Internet measurement studies had to deal with these problems, though ideally a single traceroute is less likely to trigger this behavior.
However, your hint indicates that there's something other than inducing a number of TTL exceeded packets. Can't verify as I'm on mobile, but are you thinking of the TCP Record Route option?
However, your hint indicates that there's something other than inducing a number of TTL exceeded packets. Can't verify as I'm on mobile, but are you thinking of the TCP Record Route option?
I'd think sending multiple packets (parallelism) with varying TTLs (1,2,3..n) without waiting for responses from each hops to increment the TTL, would probably give us faster traceroute. n being the number of hops you "expect" the destination subnet to be.
TTL's given by the Fibonacci sequence?
Not really. You'd want every routers in the path to reply to see all the hops in between. This would be unreliable though, since the ICMP TTL exceeded message might come in out-of-order to the sender, who is sending these probes(with varying TTLs) simultaneously. But assuming, you could somehow figure out the hop-order, this IMO is faster than current traceroute implementation.
Let's collaboratively collect the answers to these (trivia!) questions: http://collabedit.com/rsjy8
(Saving current progress in case someone abuses the page again: http://pastie.org/pastes/10182802/text [16:08 Monday, May 11, 2015 UTC].)
And they did!
Thanks, this is fun!
Hey look, a dickbutt!
Everybody is taking these questions too seriously. These are not interview questions, and these are quite fun actually - I know the answer to a few of them (not because I know RFCs by heart but because I had to deal with them in the past), and the others are just intriguing enough that I may actually go look for answers.
I actually just did interviews with Cloudflare and found it to be quite compreehensive, from the tech and personal stand point. People seems to know that they are doing and they all seem very chill and nice. Probably one of the best overall interviews I've done to date. The only problem was the delay between the interviews, they are huge! All in all, seems like a really nice company to work at.
> These are not interview questions
These should become interview questions when somebody declares himself/herself an expert in TCP/IP.
These should become interview questions when somebody declares himself/herself an expert in TCP/IP.
If the purpose of an interview is to deflate claims people make about their expertise, sure. That's not a productive goal for an interview, though.
Is it not? Hypothetically, is it not worth knowing that someone has exaggerated or lied about their skills in something? That would seem to be useful information in an interview; i.e. can someone say "I don't know" or will they make up something instead.
It can be, but I think the idea that it should be is probably the biggest misconception in hiring, and the biggest impediment most startups have to effectively hiring engineers.
Consider: there's a difference, perhaps subtle, between "deflating claimed expertise" and "assessing ability to perform a job".
Consider: there's a difference, perhaps subtle, between "deflating claimed expertise" and "assessing ability to perform a job".
> "deflating claimed expertise" and "assessing ability to perform a job".
This is a very important point, and also why I never claim to know anything I can't prove (speaking as a Linux Admin, not even developer). Everything in my resume points back to the github so whoever is interviewing can know exactly how far I've studied or had experience with a specific subject, and acts as a reference point for the job of course. Also lets me know which companies to avoid if they just care about random trivia questions rather than how much demonstrable value you can offer them.
This is a very important point, and also why I never claim to know anything I can't prove (speaking as a Linux Admin, not even developer). Everything in my resume points back to the github so whoever is interviewing can know exactly how far I've studied or had experience with a specific subject, and acts as a reference point for the job of course. Also lets me know which companies to avoid if they just care about random trivia questions rather than how much demonstrable value you can offer them.
Oh, I agree. It should not be the only thing you're looking for. I have, however, gained very valuable information from figuring out that someone likes to aggrandize their own capabilities in a particular topic and/or is unable to say "I don't know." We specifically look for people that are able to say that, since honesty and not being afraid to learn (usually the result of saying "I don't know") is important to us.
But it is /definitely/ not the only thing we look for.
But it is /definitely/ not the only thing we look for.
My experience is that once I figured out how to reliably screen for ability to perform the job I was hiring for, I stopped caring about psychological tea-leaves like these almost entirely.
Being a raving asshole could get you evicted from the Matasano hiring pipeline, but otherwise, if you could knock out the challenges we gave candidates, you had a mortal lock on our attention.
I was responsible for candidates from first contact to offer letter, and in the last year and a half I ran the process, I looked at a total of zero resumes. Many, many hires. Guess how many didn't work out.
So I guess my subtext is, if you have the mental cycles to reason from signals like "do I disagree with this candidate about what the word 'expertise' implies", you probably don't have the job aptitude prediction stuff nailed yet.
Being a raving asshole could get you evicted from the Matasano hiring pipeline, but otherwise, if you could knock out the challenges we gave candidates, you had a mortal lock on our attention.
I was responsible for candidates from first contact to offer letter, and in the last year and a half I ran the process, I looked at a total of zero resumes. Many, many hires. Guess how many didn't work out.
So I guess my subtext is, if you have the mental cycles to reason from signals like "do I disagree with this candidate about what the word 'expertise' implies", you probably don't have the job aptitude prediction stuff nailed yet.
I think that's going a bit far. I think we're saying the same thing but using different signals; the ability to say "I don't know" is actually an important gauge of aptitude in my opinion, because anyone that is unable to say it will possibly have too much hubris to take the time to learn something they don't know. It is not the only signal I rely on, and if they truly know the topic, there are other signals I rely on either way. But it is /a/ signal which I don't believe is invalid. I don't actually care if someone agrees with my definition of expertise, I just care about their ability to learn, which starts with admitting they don't know something.
Challenges are another great way to do that, and don't necessarily need to be time-limited; MicroCorruption is a great example. We will occasionally do similar "take home" style challenges. The only time we'll time limit them is specifically when we're looking for how /quickly/ someone can get up to speed on a topic they're unfamiliar with, which is also occasionally a valid question; i.e. do they need to become an expert to build something in it or can they get dangerous enough quickly. Not everyone can. In those cases, we provide plenty of resources, make ourselves fully available, and make the time limits a matter of days, not hours.
Edit: to clarify, it is far more important to me that someone have the ability to learn new things quickly than already contain an existing bit of knowledge in their head.
Challenges are another great way to do that, and don't necessarily need to be time-limited; MicroCorruption is a great example. We will occasionally do similar "take home" style challenges. The only time we'll time limit them is specifically when we're looking for how /quickly/ someone can get up to speed on a topic they're unfamiliar with, which is also occasionally a valid question; i.e. do they need to become an expert to build something in it or can they get dangerous enough quickly. Not everyone can. In those cases, we provide plenty of resources, make ourselves fully available, and make the time limits a matter of days, not hours.
Edit: to clarify, it is far more important to me that someone have the ability to learn new things quickly than already contain an existing bit of knowledge in their head.
I think people think I'm talking about "how well people can learn new things" when I talk about this. That's potentially a part of it, but it's not what I interviewed for. I interviewed for, literally, simply, "ability to do the job we had Matasano consultants perform".
Microcorruption and the Crypto Challenges were outreach for Matasano, but they were not part of the hiring process. Our hiring challenges were slightly more boring, and explicitly tuned to generate the exact signal we wanted.
I don't know whether there's any valid signal to recover from candidate psychology. I don't have to reach that argument, because in my universe, there's a giant, very accurate signal available that makes looking at other signals a poor use of my time. The fact that the big signal also keeps me from using dubious signals and making bad decisions is just a knock-on benefit. :)
Microcorruption and the Crypto Challenges were outreach for Matasano, but they were not part of the hiring process. Our hiring challenges were slightly more boring, and explicitly tuned to generate the exact signal we wanted.
I don't know whether there's any valid signal to recover from candidate psychology. I don't have to reach that argument, because in my universe, there's a giant, very accurate signal available that makes looking at other signals a poor use of my time. The fact that the big signal also keeps me from using dubious signals and making bad decisions is just a knock-on benefit. :)
That's great that you had a very specific job req and set of things you were looking for. Not all of us are that lucky ;)
Startups sometimes look for generalists (i.e. people who can learn new things) specifically because nobody knows what the stack will look like, how the product will change, etc. a year from now.
Startups sometimes look for generalists (i.e. people who can learn new things) specifically because nobody knows what the stack will look like, how the product will change, etc. a year from now.
I completely disagree. I find these types of questions incredibly useful at measuring depth of knowledge and exposure.
Of course I don't expect anyone to know the answers to all of them, but, if you've been working in Java for a decade and can't answer any nuanced questions about garbage collection in the JVM, then I start doubting either your experience or curiosity.
Of course I don't expect anyone to know the answers to all of them, but, if you've been working in Java for a decade and can't answer any nuanced questions about garbage collection in the JVM, then I start doubting either your experience or curiosity.
I was actually asked some of these for a position in a SRE team. So these are interview questions you might encounter.
This is almost unrelated to the post, but the question on the MD5 checksum was the inspiration. It seems like we could have a DoS mitigation strategy with a protocol extension that required the packets to be checksummed with a key stretching algorithm (similar to bcrypt). The worse the DoS got, the higher the number of iterations that could be required. If routers at the network edge could verify the checksums somehow then the traffic would be dropped at distributed nodes rather than at a central node. Ideally you'd want something easy to verify but hard to calculate for the client. Of course, I know nothing about DoS attacks, so this is probably trivially wrong in some way.
Here is how I did a TCP simultaneous open over the loopback interface. http://stackoverflow.com/a/2263853/78720
Wow, this is awesome. So it works!
So, for those interested in learning could we possibly have a quick short answer for each one of them?
K
K
This is what I was looking forward to but never saw a list of answers.
What specific questions are you finding hard to look up? (Some of them are easy to find, some aren't.)
Since this is essentially a list of trivia / interesting questions I was hoping for a list of answers just as easy to consume. I was able to look-up most of the ones I cheery-picked.
cheery-picked
That made me smile. Mental image: cherry-picking after a few pints.
That made me smile. Mental image: cherry-picking after a few pints.
>13) Can a SYN packet have a payload? (hint: new RFC proposals)
Hint is ok but RFC 1379 and RFC 1644 predate your hint. I have personally seen this happen as I implemented it :-)
Hint is ok but RFC 1379 and RFC 1644 predate your hint. I have personally seen this happen as I implemented it :-)
I've always thought a job at CloudFlare would be awesome, so much so that I took their list of requirements for a SRE role and set them as goals to work towards. After reading this list of questions, however, maybe I should lower my expectations, I could barely answer 25%.
I start at CloudFlare in a week. I was fortunate in that I came via a recommendation.
Still, for whatever it's worth, I was genuinely impressed with everyone I spoke with throughout the chain.
Still, for whatever it's worth, I was genuinely impressed with everyone I spoke with throughout the chain.
I love these questions. I wouldn't expect any interviewee to know all of them. But I'd expect them to be able to talk intelligently about some of them.
For a software candidate a simple question like "tell me what you know about TCP/IP" is a great start to just probe what they know about networking.
For a software candidate a simple question like "tell me what you know about TCP/IP" is a great start to just probe what they know about networking.
Ah... this bring back memories! When I was doing dev ops I had to know a lot more about the nitty gritty details of networking. Still know a few of these, but most have been forgotten.
I do CS (software mainly) graduating next year And I only know a couple of these .. Should I be worried ?
No. Even if you applied to a new job that related to all this it would generally be understood that as a new grad you're not going to know this stuff, and either A: the job would never have been posted in such a way as to lead you to apply or B: they understand there's going to be on-the-job training.
That said, if you're specifically applying to CloudFlare, you just got handed a great study sheet to differentiate yourself very handily from your peers. You'd think that "everyone" looks for this sort of thing, but given the ongoing streams of reports of "people who fail FizzBuzz", no, seriously, few people look at this sort of thing before interviews!
That said, if you're specifically applying to CloudFlare, you just got handed a great study sheet to differentiate yourself very handily from your peers. You'd think that "everyone" looks for this sort of thing, but given the ongoing streams of reports of "people who fail FizzBuzz", no, seriously, few people look at this sort of thing before interviews!
Only if you plan on becoming a low-level TCP/IP implementation engineer or some other such niche field in the very immediate future.
The vast majority of working programmers can be extremely effective without knowing much more than basic socket usage and perhaps the basics of how NAT setups can work against you if you're doing something other than just pure web-browser networking.
Having said that, knowing how in-the-wild network protocols work in general is one of those things (like knowing how compilers work) that can broaden your horizons and are probably worth learning for that reason, just don't worry too much about retaining minutia about them that you can easily look up should the need arise.
The vast majority of working programmers can be extremely effective without knowing much more than basic socket usage and perhaps the basics of how NAT setups can work against you if you're doing something other than just pure web-browser networking.
Having said that, knowing how in-the-wild network protocols work in general is one of those things (like knowing how compilers work) that can broaden your horizons and are probably worth learning for that reason, just don't worry too much about retaining minutia about them that you can easily look up should the need arise.
No. And if you run into a company where they do ask these questions (and others like them) and take them seriously, then try really hard not to work there because it will probably suck.
I think it's just confirmation that we can't know everything and there is always room to learn more.
Why should you? These are tricky domain specific questions that a network engineer with intimate knowledge of TCP/IP could answer some of. I certainly wouldn't expect a fresh CS graduate to know these things unless you did a lot of work in that area for some reason.
No. If you can't figure them out from reading the relevant RFCs, then maybe (reading specs is a vital skill), but this is absolutely not stuff that's worth memorizing (at least for most CS jobs).
Do you have any projects outside of school? or work experience? if answer is no then yes, you should be worried.
Can't someone learn these in a couple of months.
Why does a candidate have to know all these already?
Why does a candidate have to know all these already?
For quite some time we've been grilling our candidates
about dirty corners of TCP/IP stack.
[...]
I'm joking of course, [...]
The article was joking about the interview questions. But let's say they weren't. I'd answer your question thusly: Interest. If someone interviewed and had an answer to most of these questions, it would at least show that they had an interest in the material and weren't just looking for a job.
Days. The article pretty quickly diverts away from the subject of interview questions.
"The goal is to encourage our readers to review the dusty RFCs, get interested in the inner workings of the network stack and generally spread the knowledge about the protocols we rely on so much."
"The goal is to encourage our readers to review the dusty RFCs, get interested in the inner workings of the network stack and generally spread the knowledge about the protocols we rely on so much."
The subject was to mock interview questions in interviews, as depicted by the use of quotes around "interview questions". The article then goes on to peak the interest of the reader about TCP/IP quirks that cloudflare has faced over the past couple of months; i.e. real work challenges that potential employees should be able to research and answer.
"For quite some time we've been grilling our candidates about dirty corners of TCP/IP stack. Every engineer here must prove his/her comprehensive understanding of the full network stack. For example: what are the differences in checksumming algorithms between IPv4 and IPv6 stacks?"
I read that and was blown away, uttering "Bloody hell." I would have never known that from the top of my head. Good joke, well played, sir.
I read that and was blown away, uttering "Bloody hell." I would have never known that from the top of my head. Good joke, well played, sir.
What you really need to know the dirty little secrets of the various implementations of the stacks and the short cuts (or blatant ignoring of standards).
My Boss at BT could do this from a x.400 packet trace and point to the offending dword and say that is XXX's POS stack.
There is also the story of a guy (at BT Labs) who took the CCIE and only scored 98% he then wrote a personal letter to John Chambers explaining why he was right and Cisco was not.
...
Turns out he was one of the 3 inventors of Ethernet
My Boss at BT could do this from a x.400 packet trace and point to the offending dword and say that is XXX's POS stack.
There is also the story of a guy (at BT Labs) who took the CCIE and only scored 98% he then wrote a personal letter to John Chambers explaining why he was right and Cisco was not.
...
Turns out he was one of the 3 inventors of Ethernet
Do you have some link where I can read more about it? I googled but nothing came up. The part about the CCIE.
2/3 top-level comments are confusion re. the title. Can we have a different title for HN in this case?
EDIT: Title as I write this: CloudFlare “Interview Questions”
EDIT: Title as I write this: CloudFlare “Interview Questions”
It's etiquette to use the original title that you're linking to, unless it's a bit older (then you can include a year), or if the original title is too ambiguous. I think it's reasonable as it is right now.
https://news.ycombinator.com/newsguidelines.html
The guidelines allow for changing a misleading title, even if it was unintentionally misleading. From the comments here, clearly many were misled.
The guidelines allow for changing a misleading title, even if it was unintentionally misleading. From the comments here, clearly many were misled.
The quotes are there for a reason.
I understood the quote marks myself, but when you look at many of the comments here, it's obvious many didn't. So the reason failed.
I've mentioned it on the blogpost but: Do you have some metrics about the success of your candidates replying to those questions?
These are not actually interview questions.
Well, I know that there are "interview questions", but they are trying to show us the kind of knowledge that is needed to join a Company as Cloudfare so I am pretty sure that some of those are going to be questions in some point of their interview process.
I just wanted to know that in case that the questions are formulated that way, what's the ratio of people that can successfully answer those.
I just wanted to know that in case that the questions are formulated that way, what's the ratio of people that can successfully answer those.
The article says:
"For quite some time we've been grilling our candidates about dirty corners of TCP/IP stack. Every engineer here must prove his/her comprehensive understanding of the full network stack. For example: what are the differences in checksumming algorithms between IPv4 and IPv6 stacks? I'm joking of course, but in the spirit of the old TCP/IP pub game I want to share some of the amusing TCP/IP quirks I've bumped into over the last few months while working on CloudFlare's automatic attack mitigation systems."
That is, these are not real interview questions. They are trivia questions.
I doubt most engineers even at Cloudflare can answer these without research.
"For quite some time we've been grilling our candidates about dirty corners of TCP/IP stack. Every engineer here must prove his/her comprehensive understanding of the full network stack. For example: what are the differences in checksumming algorithms between IPv4 and IPv6 stacks? I'm joking of course, but in the spirit of the old TCP/IP pub game I want to share some of the amusing TCP/IP quirks I've bumped into over the last few months while working on CloudFlare's automatic attack mitigation systems."
That is, these are not real interview questions. They are trivia questions.
I doubt most engineers even at Cloudflare can answer these without research.
mu out of zero people asked successfully answered.
Cloudflare needs people who understand this, since they're a man-in-the-middle security hole/backdoor. Note that they're not asking questions about security. This shows their priorities.
Oh, please.
Cloudflare reserves the right to add adware and spyware to your pages as they pass through Cloudflare's MITM center. ("CloudFlare may: ... Add script to your pages to, for example, add services, Apps, or perform additional performance tracking."[1])
[1] https://www.cloudflare.com/terms
[1] https://www.cloudflare.com/terms
You should have continued:
CloudFlare will make it clear whenever a feature will modify your content and, whenever possible, provide you a mechanism to allow you to disable the feature.
CloudFlare will make it clear whenever a feature will modify your content and, whenever possible, provide you a mechanism to allow you to disable the feature.
When privacy and security require an opt-out, you can be sure the vendor plans to violate them. Their "free HTTPS everywhere" plan has to be monetized somehow.
A fun set of trivia questions involves inconsistent retransmission: in out-of-order delivery, with some data buffered --- IP fragments or TCP segments --- what happens when the packet that allows reassembly to proceed (a) overlaps data already sent and (b) that overlapping data isn't the same as what was already buffered? (When we wrote the IDS paper, Vern Paxson told us he'd been observing this happening in the wild, which blows my mind).
Another fun set of interview questions pertains to how you'd design the fastest possible "traceroute" program. You can do better than parallelizing/pipelining. :)
I used to like asking people what the utility of discard, chargen, and echo were.