Crypto 101 – Introductory course on cryptography(crypto101.io)
crypto101.io
Crypto 101 – Introductory course on cryptography
https://www.crypto101.io/
147 comments
> I'm streets ahead of the average Joe
Has anyone seen "streets ahead" being used by anyone else besides Pierce Hawtorn and the OP? I mean, is it widely used now? (I'm not native speaker, obviously)
Has anyone seen "streets ahead" being used by anyone else besides Pierce Hawtorn and the OP? I mean, is it widely used now? (I'm not native speaker, obviously)
I'm using it as much as possible just to push it into language.
I read it as a Pierce Hawthorneism.
It's been common parlance in English for centuries.
Same here, however I think it is better for us to talk about what we do know more, so that it increases the discussion and thus awareness and knowledge sharing (as well as getting corrected when wrong).
So we went out on a limb and made these 1 minute layman video explainers on cryptography: http://gun.js.org/explainers/data/security.html , I hope they are useful. Anything incorrect?
So we went out on a limb and made these 1 minute layman video explainers on cryptography: http://gun.js.org/explainers/data/security.html , I hope they are useful. Anything incorrect?
Logged in to post pretty much exactly this.
Since the author is reading this: thanks. Read the first few lines and am already hooked. I have been looking for a long time, but most texts on crypto are either way below my level of understanding or miles above. You're a godsend!
Since the author is reading this: thanks. Read the first few lines and am already hooked. I have been looking for a long time, but most texts on crypto are either way below my level of understanding or miles above. You're a godsend!
I'm not sure if it still applies, but there is a saying:
"There are maybe 5 people in the world that may call themselves cryptographers"
"There are maybe 5 people in the world that may call themselves cryptographers"
And they're called Alice, Bob, Chris, Dave and Eve.
Carol is the preferred C party :P
The old Cryptopals challenges (http://cryptopals.com/) seem to cover the same material in a pedagogically very different way - they don't feed you the information as this book does, but give you a practical task which can be easily done with e.g. reading the specification of an algorithm from wikipedia, but figuring out the implementation of the attack yourself gives a much better understanding than simply reading about it.
Although this book claims a "Learn by doing" approach, I didn't find any specific assignments or data samples to facilitate that.
Although this book claims a "Learn by doing" approach, I didn't find any specific assignments or data samples to facilitate that.
Hi! I'm the author. Also a big fan of Cryptopals; I was one of the reviewers for Set 8, and working with some of the authors. It's interesting to hear that point of view; because I thought C101 took the same approach as Cryptopals in the sense that it focused on teaching crypto by breaking it. When I say "learn by doing", I'm referring to stuff like walking through a bit-flipping attack just like cryptopals does. I recommend (and have organized/will be organizing) study groups where we work through cryptopals using Crypto 101; and I think that's a great idea. I was originally going to add exercises for C101, but honestly, Cryptopals was already that, and there wasn't much point in reinventing the wheel. I guess I should probably link to them?
I think it would be good to link them, I started reading, got to page 30 and thought it would be nice to try the challenge. This reminds me when I thought I could handle myself in Linux, but when I played OverTheWire wargames, I still learned a ton of things that you only learn by doing.
I had a similar POV as well -- I went into reading this book thinking there would be exercises and projects to reinforce the ideas. In particular, the following sentence (and remaining paragraph) in Chapter 1 set me up for that:
> The exercises accompanying this book focus on teaching cryptography by breaking inferior systems.There seems to be a lot of comments asking about the quality of this piece. I read through this the last time it was posted to HN, and I just have to say that this is the perfect balance of having enough detail to understand how things work, but not so much that it's overwhelming. That's a really difficult balance when it comes to crypto, so major props to the author. Fantastic work.
Whenever I have taken the small amount of cryptography knowledge I already have and tried to use it in a project, I've often been shutdown with "the system already does that" when it doesn't, or "this will be too complicated for the user, instead lets just roll our own [ad-hoc cryptography method]".
For those reading:
How do you convince people that it's worth using best practices?
Is there a good heuristic to measure the value of something, when deciding how much time and money to spend on securing it?
What are good library/SaaS solutions to help build secure applications with less chance of shooting yourself in the foot, better UX and lower cost? (Keybase, etc.)
For those reading:
How do you convince people that it's worth using best practices?
Is there a good heuristic to measure the value of something, when deciding how much time and money to spend on securing it?
What are good library/SaaS solutions to help build secure applications with less chance of shooting yourself in the foot, better UX and lower cost? (Keybase, etc.)
The video claims that the Python standard library doesn't check certificates by default. In fact, it has done for at least a couple of years ([0] quotes the documentation as saying that it changed two years ago - in 2.7.9 and 3.4.3).
Although the video is marked 2015, the overlay at the start shows it's from PyCon 2013.
[0]: http://stackoverflow.com/a/28325763/2492
Although the video is marked 2015, the overlay at the start shows it's from PyCon 2013.
[0]: http://stackoverflow.com/a/28325763/2492
For idiots like myself, I found this video, Public key cryptography - Diffie-Hellman Key Exchange (full version), to be completely enlightening using mixed colors to explain the most basic features of a cryptography algorithm.[0]
[0] https://www.youtube.com/watch?v=YEBfamv-_do
[0] https://www.youtube.com/watch?v=YEBfamv-_do
For anyone interested I found this to be a good book on working through some cryto implementations in Go:
https://leanpub.com/gocrypto
Its free to read online but its also very reasonably priced. Its written by an engineer over at Cloudflare.
https://leanpub.com/gocrypto
Its free to read online but its also very reasonably priced. Its written by an engineer over at Cloudflare.
This is great! Kudos to the author and thanks Rackspace for sponsoring this as well.
It's really encouraging to see this increased democratization of crypto not necessarily in the engineering of it per se but rather the awareness and understanding of it.
It's really encouraging to see this increased democratization of crypto not necessarily in the engineering of it per se but rather the awareness and understanding of it.
cryptography textbooks get very difficult. I get lost in a sea of hashes and the prime number theorem
I downloaded this about a year ago and loved it. Incredibly informative. Is this an updated version?
Maybe I am being too harsh, but it is clear the author does not have a formal education in the subject [0] nor any track in breaking non-toy crypto implementations [1]. This alone makes me a bit wary of any recommendation one may read in the material.
There seems to be more attention to listing all the beasts in the cryptographic zoo than to the few fundamental tools required to really understand the mechanics (e.g. birthday paradox, PRFs, some prime number theory).
Sure, I can't spot anything fundamentally wrong and it all reads pretty smoothly, but calling this a "course" is highly misleading. If the intention is to guide people in selecting good crypto primitives, then maybe "guide" is a more honest word?
For those interested, I would strongly recommend to bite the bullet and dedicate time to Boneh's course on Coursera.
[0] I don't have any either [1] Ditto
There seems to be more attention to listing all the beasts in the cryptographic zoo than to the few fundamental tools required to really understand the mechanics (e.g. birthday paradox, PRFs, some prime number theory).
Sure, I can't spot anything fundamentally wrong and it all reads pretty smoothly, but calling this a "course" is highly misleading. If the intention is to guide people in selecting good crypto primitives, then maybe "guide" is a more honest word?
For those interested, I would strongly recommend to bite the bullet and dedicate time to Boneh's course on Coursera.
[0] I don't have any either [1] Ditto
Hi! I'm the author. You're mistaken about [0] and [1]. I'd like to address your specific point about being a guide rather than a course. I agree that a crypto zoo would be a guide and not a course; although I disagree that either the book or the talk are a zoo. It's possible that that hasn't come through effectively enough. I could understand why someone might think that from looking at the table of contents for example; it's certainly a lot clearer in the talk. The approach is instead to walk someone through the kinds of primitives that exist, but more importantly, why they exist. In that context, when I say "primitive", I mean "block cipher" or "MAC"; not something like "AES". That includes incidentally talking about PRFs, although I dance around that term and use terminology they're going to find in their crypto libraries. Similarly, there's an appendix on modular arithmetic; but I try not to get too lost in proofs about group theory.
I could say "IND-CCA2" with a formal description, but in my experience that makes people's eyes glaze over. It's a lot easier to show them a bunch of reasonably-looking-yet-fatally-broken unauthenticated encryption to drive the point home. My target audience is curious programmers, not new academic cryptographers. If that's your criticism, that's absolutely valid. If you want to be the person to design SHA-4; this book isn't for you.
It's true that I take a different approach than Dan Boneh does. That's not a criticism: Boneh's course is great, it's just different. I think showing people how to break stuff is a useful educational tool. If your threat model is random web apps that have the letters "AES" or "MD5" in their code, I think it's a more effective one than a rigorous mathematical approach that will quickly dissuade the curious programmer.
I could say "IND-CCA2" with a formal description, but in my experience that makes people's eyes glaze over. It's a lot easier to show them a bunch of reasonably-looking-yet-fatally-broken unauthenticated encryption to drive the point home. My target audience is curious programmers, not new academic cryptographers. If that's your criticism, that's absolutely valid. If you want to be the person to design SHA-4; this book isn't for you.
It's true that I take a different approach than Dan Boneh does. That's not a criticism: Boneh's course is great, it's just different. I think showing people how to break stuff is a useful educational tool. If your threat model is random web apps that have the letters "AES" or "MD5" in their code, I think it's a more effective one than a rigorous mathematical approach that will quickly dissuade the curious programmer.
I am curious about what leads you to the conclusion but it is clear the author does not have a formal education in the subject?
lvh (https://lvh.io), care to weigh in?
I was under the impression he was formally educated in cryptography.
This ebook has been cited by many computer security university courses.
I was under the impression he was formally educated in cryptography.
This ebook has been cited by many computer security university courses.
What "prime number theory" would you be referring to?
I checked the PDF and this looks very interesting and comprehensive, any change you could give an eta for the final release and more specific the epub release?
Thanks!
Thanks!
Hi! I'm the author. I should remove that "pre-release" notice. Sure; there's plenty more that I could talk about, but there's also plenty of stuff there that you can use right now (and most of the stuff I wanted to talk about). Suggestions come in faster than I have time to implement them, so... Maybe this will never be done, and maybe that doesn't stop it from being a useful resource :-) I hope you enjoy it!
This is currently on edx. Its more advanced that the courses mentioned here. I do not know what edx will do after the course ends, but if you want it you can get it while it ss still available.
https://www.edx.org/course/quantum-cryptography-caltechx-del...
Quantum Cryptography by Thomas Vidick (Caltech) and Stephanie Wehner (Delft University)
https://www.edx.org/course/quantum-cryptography-caltechx-del...
Quantum Cryptography by Thomas Vidick (Caltech) and Stephanie Wehner (Delft University)
So-called Quantum Cryptography is largely snake oil. The most important things you need to know about it is that:
* In order to do QC you need an authenticated channel first. QC proponents hardly mention that or try to obscure it, but it basically means you can't have QC unless you already have some other secure cryptography.
* QC has severe practical limits. It needs a point to point connection capable of sending physical particles. That means: No Wifi, no mobile Internet and no connections over large distances. Given that these people recently started talking about a "Quantum Internet" makes this simply ridiculous.
* In order to do QC you need an authenticated channel first. QC proponents hardly mention that or try to obscure it, but it basically means you can't have QC unless you already have some other secure cryptography.
* QC has severe practical limits. It needs a point to point connection capable of sending physical particles. That means: No Wifi, no mobile Internet and no connections over large distances. Given that these people recently started talking about a "Quantum Internet" makes this simply ridiculous.
Of the first point I shall plead Wittgenstein and not speak, but as regards the second: In what way are photons, the medium of wireless telecommunication, not physical particles? In what way are, indirect connections not decomposable into multiple direct connections (broadcast vs. point-to-point is a more robust difference, I agree)?
For QC to work, the receiver must receive the exact same photons the transmitter created. Those photons can not be tampered with, what means they can not be relayed, or reflected by most surfaces.
Is there a working and practical implementation somewhere? I was under the impression it wasn't practical yet.
You can buy it:
http://www.idquantique.com/
It's probably insecure (these things get broken regularly), not practical for any real use case and total obvious nonsense to buy it, but I think there are some banks who thought they have some money leftover that they'd sink in such devices.
It's probably insecure (these things get broken regularly), not practical for any real use case and total obvious nonsense to buy it, but I think there are some banks who thought they have some money leftover that they'd sink in such devices.
This is certainly not the first thing you want to learn in cryptography, for many reasons.
Suggestion: Add some notification medium for when the book is ready.
Agreed. The one time I actually want to sign up to a mailing list, and there's no way of doing it.
Hi! I'm the author.
I should really remove that pre-release stuff. There's no useful point for "done", and it already includes most of the stuff I wanted to talk about.
I should really remove that pre-release stuff. There's no useful point for "done", and it already includes most of the stuff I wanted to talk about.
Oh, well in that case I'll get reading! Thanks so much :)
Thank you!
Also here is is a Dan Boneh cryptography playlist https://www.youtube.com/playlist?list=PL9oqNDMzcMClAPkwrn5dm...
And as context, Professor Dan Boneh teaches the main introductory crypto class at Stanford (CS255) - it's a great intro for someone with some good CS fundamentals. The course syllabus is here:
http://crypto.stanford.edu/~dabo/cs255/syllabus.html
(Disclosure, I took the class a few years ago)
http://crypto.stanford.edu/~dabo/cs255/syllabus.html
(Disclosure, I took the class a few years ago)
This is a great course. I also took it a few years ago and have been waiting for his Crypto 2 course since then. It seems like it's always "Coming Soon Fall 201x" though.
With everything in Crypto I have to wonder: Is the information correct? I really have no way of verifying if I'm learning the correct DHE, and I know that it's easy to get wrong. Perhaps I can do some testing in code, but I may test it incorrectly too, and those small errors can be exploited.
Hi! I'm the author of Crypto 101.
Firstly, I'm a real, honest-to-God cryptographer. I don't know if there are any particular people you had in mind whose recommendations you'd like to see, but there are a few HN bigwigs who'd probably be willing to generally endorse it :-) Also, it's been posted on HN a few times before, so it's had some scrutiny. That doesn't mean I don't make mistakes, but generally speaking, an active reader should be OK.
The other thing is in the way the book is structured. I teach you to break crypto; so when I say something is broken, I prove it by showing you how to break it.
Finally, the goal of this book is absolutely not to help you implement DHE. In an ideal world, the primitives we offer people are hard to misuse. Crypto 101 then only exists to satisfy programmer curiosity. It is not a replacement for a traditional academic education that will help you design new primitives; it also doesn't show you how to write secure implementations. However, Crypto 101 is still useful beyond merely satisfying curiosity now, because most cryptographic libraries _do not_ provide that easy-to-use API. Using regular hashes for password storage, various forms of broken AES-CBC (unauthenticated, key=IV, static IV...), et cetera are very real problems for real code, and Crypto 101 teaches you how to avoid that minefield.
I'm also working on the "better, more accessible" crypto part, but I only have so much free time :)
Firstly, I'm a real, honest-to-God cryptographer. I don't know if there are any particular people you had in mind whose recommendations you'd like to see, but there are a few HN bigwigs who'd probably be willing to generally endorse it :-) Also, it's been posted on HN a few times before, so it's had some scrutiny. That doesn't mean I don't make mistakes, but generally speaking, an active reader should be OK.
The other thing is in the way the book is structured. I teach you to break crypto; so when I say something is broken, I prove it by showing you how to break it.
Finally, the goal of this book is absolutely not to help you implement DHE. In an ideal world, the primitives we offer people are hard to misuse. Crypto 101 then only exists to satisfy programmer curiosity. It is not a replacement for a traditional academic education that will help you design new primitives; it also doesn't show you how to write secure implementations. However, Crypto 101 is still useful beyond merely satisfying curiosity now, because most cryptographic libraries _do not_ provide that easy-to-use API. Using regular hashes for password storage, various forms of broken AES-CBC (unauthenticated, key=IV, static IV...), et cetera are very real problems for real code, and Crypto 101 teaches you how to avoid that minefield.
I'm also working on the "better, more accessible" crypto part, but I only have so much free time :)
My trust chain: I don't know crypto and didn't know who you were (so I might have asked the same question) but a few days ago I found out you are involved with tptacek with latacora.com so that was all I needed to know and trust what you said.
I Ctrl-Fd "tptacek" as soon as I opened the page
That's a little scary but I appreciate the compliment.
It's sort of the HN version of 'yeah, he's a goodfella'.
Separately with latacora I suggest that you wrap in some ongoing residual that carries on far past the 'eventually you staff your own security team' phase (as added insurance for 'leadership').
Separately with latacora I suggest that you wrap in some ongoing residual that carries on far past the 'eventually you staff your own security team' phase (as added insurance for 'leadership').
Thanks! We're not looking to make money from clients in perpetuity. One of the things we like about the model is that we plan our own obsolescence and then get out of the way. It lines up our incentives.
Thanks for putting this together!
My g/f is getting interested in this sort of thing and she'll find this thrilling.
My g/f is getting interested in this sort of thing and she'll find this thrilling.
> I really have no way of verifying if I'm learning the correct DHE, and I know that it's easy to get wrong. Perhaps I can do some testing in code, but I may test it incorrectly too, and those small errors can be exploited.
Well this is step 0 in cryptography engineering: You don't implement primitives. Use reviewed components providing primitives and implement your protocol on top of that. Step -1 is to not design your protocol, but use a reviewed protocol and implement that. Finally, step -2 is to not implement a protocol, but rather use a reviewed implementation of a fitting protocol.
This isn't sarcasm and I don't mean to attack you.
Well this is step 0 in cryptography engineering: You don't implement primitives. Use reviewed components providing primitives and implement your protocol on top of that. Step -1 is to not design your protocol, but use a reviewed protocol and implement that. Finally, step -2 is to not implement a protocol, but rather use a reviewed implementation of a fitting protocol.
This isn't sarcasm and I don't mean to attack you.
There's a difficult rabbithole here - eventually you have to draw a line and decide to trust someone. Otherwise it's just layer after layer of paranoia until you implode in a puff of Philip Dick :)
Of course; my point is (and I assume that's also the point of other people who assert similar stances on the topic) that the average programmer or software engineer should leave these things to recognized cryptographers. This also highlights a peculiarity with cryptography, in that it is very hard to understand for laymen (assuming laymen are ppl with some CS or math education), so most arguments in discussions of laymen will be appeals to authority (eg. djb said ..., Schneier wrote ..., Rogaway wrote ...), and that very subtle mistakes can have very large consequences (on the other hand SE is full of these things; your entire software can be great and bug-free, yet a single off-by-one could get it 0wned anyway).
> Of course; my point is (and I assume that's also the point of other people who assert similar stances on the topic) that the average programmer or software engineer should leave these things to recognized cryptographers.
If "crypto is for cryptographers" is your stance I think you're doing yourself a disservice. Understanding crypto, as in learning the foundations of it, is important for everyone. Having a good grasp of cryptography helps with making decisions in your code base w.r.t to how you use cryptography primitives and tends to help you in identifying places where it's gone wrong.
I agree that it's easy to get these things wrong but I don't think that's a very good reason not to try to write or read up on the subject. And yes, it's possible that what you get taught might have holes too. But it's easier to fix these holes, for example by having discussions around the topic with others and validating what you're being taught, than acquiring the knowledge to begin with.
Now the author in question (lvh) is also known for their contributions to caesium (Clojure implementation of libsodium/NaCl) and Python Cryptographic Authority's cryptography library. They've got some solid chops in the area but make of it what you will.
That said, rolling your own crypto should be left to cryptographers with extensive peer reviewing and so on.
If "crypto is for cryptographers" is your stance I think you're doing yourself a disservice. Understanding crypto, as in learning the foundations of it, is important for everyone. Having a good grasp of cryptography helps with making decisions in your code base w.r.t to how you use cryptography primitives and tends to help you in identifying places where it's gone wrong.
I agree that it's easy to get these things wrong but I don't think that's a very good reason not to try to write or read up on the subject. And yes, it's possible that what you get taught might have holes too. But it's easier to fix these holes, for example by having discussions around the topic with others and validating what you're being taught, than acquiring the knowledge to begin with.
Now the author in question (lvh) is also known for their contributions to caesium (Clojure implementation of libsodium/NaCl) and Python Cryptographic Authority's cryptography library. They've got some solid chops in the area but make of it what you will.
That said, rolling your own crypto should be left to cryptographers with extensive peer reviewing and so on.
I agree that everyone should learn more about crypto. It helps people understand what is actually going on, and what they can count on and not count on.
Engineers working around crypto that will be responsible for anything important, even doing seemingly simple things like feeding the entry point a password, need to either have their code audited by someone with experience here, or be a person who could audit it. This is because there are so very many ways that seemingly inconsequential things (function timing, hardware behavior in the face of voltage manipulation, nonzero-ed memory in deallocated buffers... and these are the more obvious things) that can lead to a break.
And that's really only a starting point... this stuff is hard. I won't try to write it other than for my own amusement.
Engineers working around crypto that will be responsible for anything important, even doing seemingly simple things like feeding the entry point a password, need to either have their code audited by someone with experience here, or be a person who could audit it. This is because there are so very many ways that seemingly inconsequential things (function timing, hardware behavior in the face of voltage manipulation, nonzero-ed memory in deallocated buffers... and these are the more obvious things) that can lead to a break.
And that's really only a starting point... this stuff is hard. I won't try to write it other than for my own amusement.
I think it's important to have a decent understanding of types of crypto (including salt/hashing) as well as where to apply what. IMHO it gets harder when it comes down to protocols used for communications, sync/async, etc.
Also, if you're ever interested in something.. find a protocol in a language, and try to refactor it, but first create a bunch of test cases, and make SURE to include unicode characters, etc in any strings you're testing, high byte characters are often missed as a use case, and it gets weird.
Getting multiple systems in multiple languages talking can sometimes be frustrating to say the least.
Also, if you're ever interested in something.. find a protocol in a language, and try to refactor it, but first create a bunch of test cases, and make SURE to include unicode characters, etc in any strings you're testing, high byte characters are often missed as a use case, and it gets weird.
Getting multiple systems in multiple languages talking can sometimes be frustrating to say the least.
Look at DES or MD5. Were they not designed by cryptographers?
Or how about the whole Dual_EC_RBG backdoor fiasco? Yes, there was criticism right from the start, but it was adopted and used widely as far as I know. Heck, Dual_EC_RBG was only ousted in 2013:
"The ANSI X9F1 Tool Standards and Guidelines Group which discussed the backdoor also included three employees from the prominent security company RSA Security.[6] In 2004, RSA Security made an implementation of Dual_EC_DRBG which contained the NSA backdoor the default CSPRNG in their RSA BSAFE as a result of a secret $10 million deal with NSA. In 2013, after the New York Times reported that Dual_EC_DRBG contained a backdoor by the NSA, RSA Security said they had not been aware of any backdoor when they made the deal with NSA, and told their customers to switch CSPRNG."
Besides, I think it's a good learning exercise to try to implement cryptographic primitives based on existing implementations. My opinion is that such language would dissuade people from even trying to design one for fun.
Disclaimer: do not design your own crypto primitive for production use.
Or how about the whole Dual_EC_RBG backdoor fiasco? Yes, there was criticism right from the start, but it was adopted and used widely as far as I know. Heck, Dual_EC_RBG was only ousted in 2013:
"The ANSI X9F1 Tool Standards and Guidelines Group which discussed the backdoor also included three employees from the prominent security company RSA Security.[6] In 2004, RSA Security made an implementation of Dual_EC_DRBG which contained the NSA backdoor the default CSPRNG in their RSA BSAFE as a result of a secret $10 million deal with NSA. In 2013, after the New York Times reported that Dual_EC_DRBG contained a backdoor by the NSA, RSA Security said they had not been aware of any backdoor when they made the deal with NSA, and told their customers to switch CSPRNG."
Besides, I think it's a good learning exercise to try to implement cryptographic primitives based on existing implementations. My opinion is that such language would dissuade people from even trying to design one for fun.
Disclaimer: do not design your own crypto primitive for production use.
Cryptographers were not fond of DUAL_EC. They also recommended against DES and MD5 long before attacks really appeared.
Nitpick: DUAL_EC was adopted despite cryptographer's complaints; I don't think there any cryptographers vociferously supporting it.
I do agree that learning crypto is a good thing, because it gives you an idea of what your primitives should do, and when it is appropriate to use them. People should always refrain from using their own home-grown crypto, however.
I do agree that learning crypto is a good thing, because it gives you an idea of what your primitives should do, and when it is appropriate to use them. People should always refrain from using their own home-grown crypto, however.
> that the average programmer or software engineer should leave these things to recognized cryptographers
Study and experimentation are how you end up with recognized cryptographers.
This very sensible attitude of "Use the recommended library with recommended settings" seems to get pushed to the point where it becomes "Don't bother learning". I don't think it's intentional of course.
Study and experimentation are how you end up with recognized cryptographers.
This very sensible attitude of "Use the recommended library with recommended settings" seems to get pushed to the point where it becomes "Don't bother learning". I don't think it's intentional of course.
To clarify: I didn't mean that people shouldn't learn or read about the topic, or try designing their own block or stream ciphers, hashes, KDFs and of course protocols (which are great fun!), not at all. Both my original comment and this one are directed at production use of cryptography.
Hi! I'm the author of Crypto 101.
I couldn't agree more. I think the fact that we have so much crypto foot guns available is a UX issue that we should address. Ideally, Crypto 101 would only be to satisfy one's own curiosity; not mandatory reading for anyone who wants to do something cryptographic. Alas, that's also not the world we live in right now. Realistically, plenty of crypto libraries will offer up e.g. unauthenticated AES-CBC. Then you probably want to know about some of the attacks on that, why you want a MAC, and where that MAC should live; if only to convince your coworkers that the concerns you have are real.
I couldn't agree more. I think the fact that we have so much crypto foot guns available is a UX issue that we should address. Ideally, Crypto 101 would only be to satisfy one's own curiosity; not mandatory reading for anyone who wants to do something cryptographic. Alas, that's also not the world we live in right now. Realistically, plenty of crypto libraries will offer up e.g. unauthenticated AES-CBC. Then you probably want to know about some of the attacks on that, why you want a MAC, and where that MAC should live; if only to convince your coworkers that the concerns you have are real.
Isn't this the point of [Twitter]NaCl (Bernstein et al)? To provide a library which, when used correctly, makes it harder to make the obvious mistakes?
I think it's great, but it doesn't solve the protocol issue.
I think it's great, but it doesn't solve the protocol issue.
Sure! But folks don't always use NaCl; they can have all sorts of stuff currently in play.
It's not really a rabbithole in most cases - unless you have exceptional expertise (and you'd know), you should trust someone else right away, not eventually, because you should trust yourself even less.
Even if you have good reason to believe that all available third party systems are vulnerable to one or another intelligence agency, using that would still be more secure than doing everything yourself, which you can't do properly (noone can do that without extensive review from a large community, so you definitely can't as well), and thus making a system that's vulnerable to everyone.
Even if you have good reason to believe that all available third party systems are vulnerable to one or another intelligence agency, using that would still be more secure than doing everything yourself, which you can't do properly (noone can do that without extensive review from a large community, so you definitely can't as well), and thus making a system that's vulnerable to everyone.
The only problem with that is that at some point you get into outright (justified or not) paranoia. Is your computer the victim of a supply chain attack? Did someone intercept your switch/router in delivery?
Well, one benefit of wide exposure (like here on HN) is that if it was wrong, somebody would be bound to come along and point it out.
FWIW, I consider myself pretty well-versed on cryptography and although I haven't gone over this book with a fine-toothed comb (yet!), it looks very, very accurate to me.
FWIW, I consider myself pretty well-versed on cryptography and although I haven't gone over this book with a fine-toothed comb (yet!), it looks very, very accurate to me.
> Well, one benefit of wide exposure (like here on HN) is that if it was wrong, somebody would be bound to come along and point it out.
I wouldn't expect that subtle implementation errors or even protocol flaws would get noticed on any news site, or even by contributors / co-developers or whatever.
I wouldn't expect that subtle implementation errors or even protocol flaws would get noticed on any news site, or even by contributors / co-developers or whatever.
There is some level of trust when you read books like this (or really any technical book); for that reason I'd probably pass on it for something by an authority in the field (or their recommendations).
You learn about DHE reading this. You implement DHE following specs and standards.
When I was taking Aikido, there was a day when the sensei was going through all of our techniques and showed how the uke (initiator of the attack, receiver of the technique) could turn things around on the tori. (receiver of the attack, initiator of the technique) It seemed like there were a half dozen ways each that a technique could go seriously wrong, and that many of them didn't require much skill, only determination and the opportunity provided by a mistake. That day made me question the validity of the entire notion of self defense.
I wonder if there shouldn't be a software engineering class where people try to set up a secure web app, with their own homegrown algorithms and protocols, which is then attacked by a tiger team which includes a conspirator on the inside? Perhaps there are such classes now.
I wonder if there shouldn't be a software engineering class where people try to set up a secure web app, with their own homegrown algorithms and protocols, which is then attacked by a tiger team which includes a conspirator on the inside? Perhaps there are such classes now.
I've always thought it would be neat to have something like the Netflix "Chaos Monkey", but rather than sitting around on your machines taking them down at random, it sits on your network (or as part of the offering of the PaaS you've deployed to) trying to break into your stack using Metasploit et al, and then will immediately shut down/isolate every piece of software it manages to gain access to.
If you deployed vulnerable code, you'd see your stack fall over ~10mins later, along with an accompanying notice from the Penetrator Monkey that you've got work to do.
If you deployed vulnerable code, you'd see your stack fall over ~10mins later, along with an accompanying notice from the Penetrator Monkey that you've got work to do.
I worry that this would give a false sense of security. Definitely it's better to have something that will highlight problems than to have the problems go unnoticed, but there's no way that something like a Penetrator Monkey could be comprehensive, and less experienced people might fall into the trap of believing that the monkey is as competent as an actual attacker.
Something similar happens with code coverage. It's pretty easy to write tests that provide 100% code coverage yet still miss basic bugs. 100% code coverage gives you little more guarantee than "this code doesn't crash for naive inputs" unless the tests providing the coverage were well written - code coverage gives you no introspection to how good the tests might actually be.
In short, I agree that this would be a cool + useful tool. But it should come with a disclaimer that it's not a substitute for more thorough testing + verification.
Something similar happens with code coverage. It's pretty easy to write tests that provide 100% code coverage yet still miss basic bugs. 100% code coverage gives you little more guarantee than "this code doesn't crash for naive inputs" unless the tests providing the coverage were well written - code coverage gives you no introspection to how good the tests might actually be.
In short, I agree that this would be a cool + useful tool. But it should come with a disclaimer that it's not a substitute for more thorough testing + verification.
There should be. In industry, we call those things CTFs, and there are both attacker and defender CTFs. My experience has been that they are extremely effective learning tools.
extremely effective learning tools
As far as they go, maybe; but I worry that people don't appreciate the limitations. Making sure that someone can't code up an exploit against your system in half an hour is absolutely essential, but it's not the only thing that matters -- a serious cryptographer is also going to be concerned about "two years of coding and ten million dollars of computing power" attacks, which can be safely disregarded in CTFs.
As far as they go, maybe; but I worry that people don't appreciate the limitations. Making sure that someone can't code up an exploit against your system in half an hour is absolutely essential, but it's not the only thing that matters -- a serious cryptographer is also going to be concerned about "two years of coding and ten million dollars of computing power" attacks, which can be safely disregarded in CTFs.
Not exactly what you describe but related: people build intentionally-vulnerable applications for the sake of teaching security by example.
In the sub-domain of web security, django.nV is such a "purposefully vulnerable Django application" that comes as companion to a websec tutorial: https://github.com/nVisium/django.nV
No disclaimer, I have no affiliation with the project, I just enjoy their work.
In the sub-domain of web security, django.nV is such a "purposefully vulnerable Django application" that comes as companion to a websec tutorial: https://github.com/nVisium/django.nV
No disclaimer, I have no affiliation with the project, I just enjoy their work.
This is slightly tangential since you specified a conspirator on the inside, but how easy is it to break a homegrown encryption algorithm if you don't have the source code? I assume there are tools (what are they?) that will break a simple caesar cipher if you have more than a sentence or so of plain text to work with. But if you strung together 2-3 broken algorithms and your attacker doesn't know which ones, is it still trivial to decrypt?
People who can break it won't spend the time breaking your homegrown crypto, so you won't get proof it's broken. But it's still broken. If lots of money or lives of political dissidents are at stake, it will be broken.
To have really capable people work on breaking your crypto for free, you have to be an insider. You become an insider by breaking other people's crypto. You can publish a break in an insider's crypto even if you are unknown. After you publish a few such papers, you become an insider and can publish your own crypto other people will spend their time trying to break.
People can learn the state of the art and develop an alternative to the common (NIST) choices which are no worse, but also no better. Some of those are blessed as "national pride ciphers" (GOST, Camellia, SEED, etc.).
To have really capable people work on breaking your crypto for free, you have to be an insider. You become an insider by breaking other people's crypto. You can publish a break in an insider's crypto even if you are unknown. After you publish a few such papers, you become an insider and can publish your own crypto other people will spend their time trying to break.
People can learn the state of the art and develop an alternative to the common (NIST) choices which are no worse, but also no better. Some of those are blessed as "national pride ciphers" (GOST, Camellia, SEED, etc.).
Ciphers aren't the place where security most often fails. The failures have to do with implementation. More commonly, they have to do with implementation of protocols and systems using the protocols.
I agree, but I replied to someone who talked about ciphers.
Of course. I was also (mainly) posting for the benefit of 3rd parties.
We're specifically talking about the scenario where you have a "tiger team" (strcredzero's phrasing) trying to break it. I interpreted GP as asking just how hard a time the tiger team would have if they don't have source code.
I've seen a few fun articles about people breaking home grown encryption. The question is mostly about how motivated someone might be to find a problem.
Making a secure encryption algorithm requires a lot of presence of mind, and a lot of industry knowledge. If your threat model is incomplete, you lose. If you forgot one tiny thing at one tiny point in the algorithm, you lose.
If you don't have people checking your work, how do you know? If someone is determined to break your encryption, they are capable of spending a lot more time analyzing it than you spent building it. And they only need to find one mistake.
It's definitely better to use the tools that experts have spent lots of time, lots of breadth of knowledge, and lots of depth of knowledge inspecting.
Making a secure encryption algorithm requires a lot of presence of mind, and a lot of industry knowledge. If your threat model is incomplete, you lose. If you forgot one tiny thing at one tiny point in the algorithm, you lose.
If you don't have people checking your work, how do you know? If someone is determined to break your encryption, they are capable of spending a lot more time analyzing it than you spent building it. And they only need to find one mistake.
It's definitely better to use the tools that experts have spent lots of time, lots of breadth of knowledge, and lots of depth of knowledge inspecting.
Those articles (and their sources) sound interesting. Think you can share any of them?
how easy is it to break a homegrown encryption algorithm if you don't have the source code?
A better question to ask is, how easy is it to break the protocols and the software using the protocols? We have secure ciphers. Those aren't where the problems in computer and network security lie.
A better question to ask is, how easy is it to break the protocols and the software using the protocols? We have secure ciphers. Those aren't where the problems in computer and network security lie.
Also interested.
Is there a general way to attack naive ciphers?
I fall of at even a proper Vigenère cipher, although I guess I could hack together a terribly inefficient python script to test all options and then print them line by line and start visually scanning for patterns :-/
Is there a general way to attack naive ciphers?
I fall of at even a proper Vigenère cipher, although I guess I could hack together a terribly inefficient python script to test all options and then print them line by line and start visually scanning for patterns :-/
Vigenère ciphers based on xor: xor the ciphertext with itself at varying offsets. Stop when you detect likely plaintext in the result.
There are penetration testing classes, but doubt that they are bundled together with classical CS, thought never know what some uni or courses may add.
And there are project like those for practicing, few pointers:
- https://sourceforge.net/projects/metasploitable/
- https://github.com/ethicalhack3r/DVWA
- https://www.vulnhub.com/
If you go this direction you will find much more applications prepared for Capture The Flag competitions. Some people are even posting walk-throughs how to hack and fix them. It is very interesting & hard.
- https://sourceforge.net/projects/metasploitable/
- https://github.com/ethicalhack3r/DVWA
- https://www.vulnhub.com/
If you go this direction you will find much more applications prepared for Capture The Flag competitions. Some people are even posting walk-throughs how to hack and fix them. It is very interesting & hard.
Aikido in particular seems to be very subtle, both in my experience and the opinion of an aikido black belt I've asked about it. If you restrict yourself to aikido techniques and your attacker has no such reservations, you have to be extremely good. I think it would be a mistake to give up on self-defense in general because of that. In any case, you have to go into it knowing it's not a guarantee, it just increases your odds.
Yeah, I studied Aikido briefly in college and found it quite powerful when a petite 65yr old Japanese lady could inflict the most excruciating drop-to-your-knees pain I've ever felt with barely a touch. But it takes a decade or more of diligent practice to achieve that kind of skill, and I found Judo, BJJ, and Krav Maga quicker to become effective with.
Freestyle Olympic wrestling also has many useful techniques for winning a real street fight, and as an activity it builds the type of strength that is very effective in a fight. Worth taking a look at wrestler and BJJ practitioner Daniel Cormier's recent win of the UFC light heavyweight championship over Anthony Johnson, an extremely talented striker, to see why this is the case. Johnson essentially had a small window of time to knock Cormier out, and he nearly did, but once Cormier manage to get a hold of him the fight was brutal after that.
Strength training and general athleticism alone too will often allow you to win over someone who practices a martial art that is purely focused on striking, especially one of the more ornamental ones. Hard to win against someone that can just pick you up and slam you on the concrete ala Quinton Jackson.
Strength training and general athleticism alone too will often allow you to win over someone who practices a martial art that is purely focused on striking, especially one of the more ornamental ones. Hard to win against someone that can just pick you up and slam you on the concrete ala Quinton Jackson.
Aikido in particular seems to be very subtle
Which aids in the analogy.
If you restrict yourself to aikido techniques and your attacker has no such reservations, you have to be extremely good.
Good for the analogy.
I think it would be a mistake to give up on self-defense in general because of that. In any case, you have to go into it knowing it's not a guarantee, it just increases your odds.
Good points for the analogy.
Which aids in the analogy.
If you restrict yourself to aikido techniques and your attacker has no such reservations, you have to be extremely good.
Good for the analogy.
I think it would be a mistake to give up on self-defense in general because of that. In any case, you have to go into it knowing it's not a guarantee, it just increases your odds.
Good points for the analogy.
The act of reversing a technique is called "kaeshiwaza". Kaeshiwaza is possible only if there there is an opening (weakness) in the waza.
That is all there is to it.
Some examples here https://www.youtube.com/watch?v=C_SB0TqvUb8
That is all there is to it.
Some examples here https://www.youtube.com/watch?v=C_SB0TqvUb8
What would be the purpose of such a class? If the point is to show that rolling your own security is bad, it seems like having an inside man would just help leave people convinced that they could have succeeded if only it weren't for that other guy.
Something tells me a conspirator wouldn't be required anyways. Your security is either sound or broken, i.e. probably broken unless you use proven implementations of proven cryptosystems with proven configurations.
E.g. even if RSA is (currenly) safe, as well as its primitives and common configuration... rolling your own implementation will probably be subject to timing attacks.
E.g. even if RSA is (currenly) safe, as well as its primitives and common configuration... rolling your own implementation will probably be subject to timing attacks.
since im not in the market for downvoting. ill just say.
eh.
RSA is a simple formula. takes a bit of fiddling and skill to get BigIntegers working. but anyone who passed highschool algebra could roll their own. in fact many high schools even set it as homework.
timing attacks you say. erm, not remotely connected.
eh.
RSA is a simple formula. takes a bit of fiddling and skill to get BigIntegers working. but anyone who passed highschool algebra could roll their own. in fact many high schools even set it as homework.
timing attacks you say. erm, not remotely connected.
If you implement RSA from the textbook, as a simple formula, you'll leave yourself vulnerable to padding based attacks that can recover the private key. It turns out that though the formulas are straightforward and the math can be explained after a decent undergraduate education, the implementation is still very subtle.
padding attacks are only really related to encrypting data not sharing the product of two very large primes. You clearly havent got the faintest idea what you are talking about.
Rather than putting the rest of us at risk by putting zero value on the security of anything you make and guessing someone [a spook] will do it for you, why dont you recommend everyone takes some time out to at least learn the basics.
Yes, there are several "mines" you need to be wary of.
But they all pale into insignificance compared to linking openssl and thinking you are done.
Rather than putting the rest of us at risk by putting zero value on the security of anything you make and guessing someone [a spook] will do it for you, why dont you recommend everyone takes some time out to at least learn the basics.
Yes, there are several "mines" you need to be wary of.
But they all pale into insignificance compared to linking openssl and thinking you are done.
> timing attacks you say. erm, not remotely connected.
Fine, I'll trust your overwhelming evidence instead of the many timing attacks described against RSA implementations.
Fine, I'll trust your overwhelming evidence instead of the many timing attacks described against RSA implementations.
All those attacks are "theoretical". As opposed to actually having been usefull in the wild - FUD (heartbleed otoh.....).
padding -> irrelevent to an RSA public key, everyone has to know the exact modulus and product, even the bad guys (although I recommend keeping public keys relatively secret)
timing -> irrelevent to an RSA public key, there is nothing you can get from how long it takes to decrypt or encrypt even 1024 bits of data (which is more than most impliementations even send), sometimes the exact same data will take 1ms, sometimes 100ms, it depends what else the machine is doing at the time.
padding -> irrelevent to an RSA public key, everyone has to know the exact modulus and product, even the bad guys (although I recommend keeping public keys relatively secret)
timing -> irrelevent to an RSA public key, there is nothing you can get from how long it takes to decrypt or encrypt even 1024 bits of data (which is more than most impliementations even send), sometimes the exact same data will take 1ms, sometimes 100ms, it depends what else the machine is doing at the time.
Something tells me a conspirator wouldn't be required anyways.
The conspirator is there to teach about Defense in Depth. Systems that rely on just 1 "impenetrable" membrane tend to be brittle. Their failure is catastrophically sudden and complete, like glass breaking. Glass is way harder than steel, but steel structures are much more robust, because steel's ductility allows for partial failures and continued resistance/integrity.
Your security is either sound or broken
What about "Defense in depth?"
The conspirator is there to teach about Defense in Depth. Systems that rely on just 1 "impenetrable" membrane tend to be brittle. Their failure is catastrophically sudden and complete, like glass breaking. Glass is way harder than steel, but steel structures are much more robust, because steel's ductility allows for partial failures and continued resistance/integrity.
Your security is either sound or broken
What about "Defense in depth?"
Defense in depth mostly does not work (against willful targeted attacks).
Software is not the real world where things bend. If a barrier is sane, it will stand, nobody will break it. But no matter how many non-sane barriers you add, you can not turn them into a sane one. The only thing you will achieve is to increase your system complexity and get more bugs as a result.
Software is not the real world where things bend. If a barrier is sane, it will stand, nobody will break it. But no matter how many non-sane barriers you add, you can not turn them into a sane one. The only thing you will achieve is to increase your system complexity and get more bugs as a result.
only one encryption scheme works against "willful targeted attacks" - the one time pad (and even that has potential for going wrong)
All the others are about making the effort required greater than the will to get there in favor of usability.
All the standard libraries stop at "miniscule effort for a state".
And states aren't even "the big boys" anymore.
All the others are about making the effort required greater than the will to get there in favor of usability.
All the standard libraries stop at "miniscule effort for a state".
And states aren't even "the big boys" anymore.
[deleted]
Can any crypto people here on HN verify that this gets it right?
I'm the author of the book "Implementing SSL" (https://books.google.com/books?id=LfsC03f8oGsC&printsec=fron..., which covers a lot of the same ground as the link) and consider myself a crypto person - although I haven't had a chance to read every word and check through it with a fine tooth comb, based on what I've had a chance to review, it appears very accurate and very well laid out.
A nitpick: When discussing hash function properties (p. 111), his definition of property #1 ("[it should be hard to] modify a message without changing the hash") is, to me, equivalent to second-preimage resistance ("[Given] a message m, it should be difficult to find another message m' with the same hash"), which he discusses as a separate property under prop #3.
In the explanation of prop #1, he makes it clear that he is talking about the "avalanche effect," so it might be better to say that explicitly in the definition.
Disclaimer: I'm not a crypto person, so it's better if the author comments on this as well.
I think the book is great, by the way, especially the writing style and the choice of what to focus on. Thanks, lvh, for doing this.
In the explanation of prop #1, he makes it clear that he is talking about the "avalanche effect," so it might be better to say that explicitly in the definition.
Disclaimer: I'm not a crypto person, so it's better if the author comments on this as well.
I think the book is great, by the way, especially the writing style and the choice of what to focus on. Thanks, lvh, for doing this.
For knowing lvh, he usually knows what he's talking about ;) I looked at his book a while ago and it looked pretty good to me.
Looks interesting, but I can't open it with Adobe Reader on my Android.
Applied Cryptography is also one of the free advanced courses on Udacity:
https://www.udacity.com/course/applied-cryptography--cs387
https://www.udacity.com/course/applied-cryptography--cs387
I'm really disappointed that (9.4) Elliptic curve cryptography is still under TODO.
If anyone is interested in ECC, ars has a pretty good introduction [0].
[0] : http://arstechnica.com/security/2013/10/a-relatively-easy-to...
If anyone is interested in ECC, ars has a pretty good introduction [0].
[0] : http://arstechnica.com/security/2013/10/a-relatively-easy-to...
You can also mail Sean Devlin to get Set 8 of the Crypto Challenges, which cover ECC. Finding the right place to mail I'll leave as an exercise for the reader.
Quick question, I had apparently Pinboarded this in March 2014. I see the PDF is still pre-release. Has anything changed with this, or is it kinda just coming up again because of recent political climate.
I'm fine either way, just curious if this has changed drastically from what I had looked at previously.
I'm fine either way, just curious if this has changed drastically from what I had looked at previously.
Paragraph "Development" (page 14 in the current version) answers your question:
"The entire Crypto 101 project is publicly developed on GitHub under the crypto101 organization, including this book: https://github.com/crypto101/book "
[...]
"The copy of this book that you are reading right now is based on the git commit with hash 3f89ec3 , also known as 0.4.0-22-g3f89ec3"
Then, looking at the commits, yes the book changed a lot since 2014: https://github.com/crypto101/book/commits/master
"The entire Crypto 101 project is publicly developed on GitHub under the crypto101 organization, including this book: https://github.com/crypto101/book "
[...]
"The copy of this book that you are reading right now is based on the git commit with hash 3f89ec3 , also known as 0.4.0-22-g3f89ec3"
Then, looking at the commits, yes the book changed a lot since 2014: https://github.com/crypto101/book/commits/master
[deleted]
Hi! I'm the author.
I should stop calling it a pre-release. There's no useful "done" marker. It includes most of the material I wanted to talk about, and has for a while. You should update the PDF though; as the other commenter mentioned, it does get updated :)
I should stop calling it a pre-release. There's no useful "done" marker. It includes most of the material I wanted to talk about, and has for a while. You should update the PDF though; as the other commenter mentioned, it does get updated :)
> There's no useful "done" marker.
Well, maybe when there are no more "TODOs" ;)
Well, maybe when there are no more "TODOs" ;)
That doesn't work too well when they get added faster than they get removed; which is what I mean by "no useful done marker". There's an issue tracker now instead of TODOs in the document (not to say that all of them have been removed) -- I don't think that _actually_ improves anything except optics :)
Thanks for this my guy! Maybe I'm telling on myself here, but I get the impression that your average developer doesn't know much about security outside of the basic (sql injection/cross site scripting)
Why is the url a hash? Also I'm getting privacy error on chrome mobile.
Thanks for this, seems pretty useful.
Thanks for this, seems pretty useful.
Hi! I'm the author. It's hosted on a CDN; I don't choose that URL. Sorry about the privacy error; could you take a screenshot and submit a ticket on https://www.github.com/Crypto101/book? Thanks!
I put this pdf on my phone and read through interesting sections over a vacation involving long flights. It's a very nicely written text that you can read over a few days with some basic computer-science/mathematical background.
markild(3)
good work, lvh
Quite seriously, this is exactly what the tech world needs - personally, I know that in terms of understanding of crypto I'm streets ahead of the average Joe, but orders of magnitude behind people who actually know the field. I'm certain I'm far from alone in that set, but the way the world's going means that we with the generalised technical know-how have a moral impetus to bring the rest of the world up to speed with the whys and wherefores.