Wow, this is really a great resource. I wish it had been available a few years back when I took the free online version of CS231n. The hardest part (for me, anyway) was the long-forgotten Calculus needed for backprop. Especially as applied to matrices. I struggled at the time to find accessible explanations of many of the matrix operations, and you seem to have it all laid out here. Thank you.
Adding my own anecdote, there seems to be some kind of problem with high I/O loads on High Sierra.
One of my projects involves indexing thousands of URLs. During development I use a large (80GB+) local cache of a few million files. Running the cache code (~100 lines of simple Golang) totally brings my iMac (3.8GHz Core i5, 24GB) to its knees. CPU shows as mostly idle & memory pressure is low, yet the machine can't redraw windows properly after about 5GB of I/O to local SSD.
Completely frustrating. I moved the code over to an older (2010!) Mac running Ubuntu and it handles it without breaking a sweat.
Paranoid patient question: when one goes for an MRI, is there any kind of scan for foreign metal objects? I'm sure they ask if you have any devices implanted, but do they actually check for the unexpected? What would happen if you had one of these little bristles and you didn't know it?
I think the concern is more that a sentience that is many orders of magnitude more intelligent than humans simply won't care about us. How do you feel about ants?
I like to build something that I've programmed before in another language. Maybe this seems redundant. But if you have the problem fresh in your mind, it leads to asking the kinds of (Googleable) questions that get you very quickly to a base-level understanding of the new language. It also sheds light on the areas where the language is better/worse at specific things than the lang you are used to working with.
A month ago I was in the same place- I would start reading a short blog posting on RNNs/ConvNets/etc., and within 2-3 paragraphs my eyes would glaze over from the math and other foreign terminology. Frustrating. To try and fix this I am "auditing" the Stanford course on ConvNets: http://cs231n.stanford.edu/syllabus.html
I'm about 2/3 done with the homeworks, and I understand this stuff now. I'll never be a data scientist, but I know enough to implement these networks on my own, and to understand blog posts like this. It's a lot of work for one course, much more than I remember from my own undergrad years. I had to revisit Calculus & Linear Algebra too. But if you're genuinely interested in this stuff you can pick it up.
I'm really looking forward to no longer having to figure out which project I have to axe to keep my "small" plan under the 10 repo maximum. That was always annoying.
People are knocking this guy for not being an expert and maybe getting some details wrong. Maybe it's a little bit like watching a non-programmer stumble their way through a blog post about learning to program- experienced programmers may cringe a bit.
But I really appreciate these kinds of write-ups: he declares his non-expertise up-front, and then proceeds to document his understanding as he goes along. There's something useful about this kind of blog post for non-experts.
I'm working my way through Karpathy's writeup on RNNs (http://karpathy.github.io/2015/05/21/rnn-effectiveness). I've mechanically translated his Python to Go, and even managed to make it work. But I still don't entirely understand the math behind it. Now obviously Karpathy IS an expert, but despite his extremely well-written blog post, a lot of it is still somewhat impenetrable to me ("gradient descent"? I took Linear Algebra oh, about 25 years ago). So sometimes it's nice to see
other people who are a bit bewildered by things like tanh(), yet still press on and try to understand the overall process.
And FWIW I had the same reaction as the author when I started toying around with neural nets- it's shocking how small the hidden layer can be and still do useful stuff. It seems like magic, and sometimes you have to run through it step-by-step to understand it.
My comment wasn't meant to knock C++; it was to point out that Linus' opposition to the language is well-documented, and an official git-rewrite in C++ is very unlikely.
I doubt it. I did Java for over a decade, but jumped at the opportunity to use Go with its statically compiled binaries. The JVM is great, but being tied to it is kind of a hassle. Being able to hand a small binary to someone and say "here, run this" with no worry about dependencies- it's a beautiful thing.
I dread tax time every year. I'm self-employed, so I don't exactly expect it to be easy, but I generally end up spending 3-4 full days working on my return. Not including the meticulous record keeping I amortize over the year with a simple spreadsheet.
Every time I think I've finally got it figured out, there's always some weird new situation that throws a wrench in things. Common things like buying/selling a house, moving to a new state or locality, death of a parent, etc. I honestly don't know how most people ever manage to do it correctly, much less optimally.
This year it was 2 different banks that got IRA reporting wrong. Last year, Turbotax "interview mode" would not let me enter some crucial figure correctly. Previous year it was clients not sending 1099s (and agonizing over whether to just report manually or wait for the form). Two years before that it was a RITA (local tax authority) screwup.
I tried hiring a local CPA to take away some of the pain, but he ended up making a $5K mistake, and it took hours of my time to correct. I've had a good accountant in the past, so I know they can make a huge difference, but they are very hard to find.
If we can digitally describe something as a sequence of bytes, and we stack those bytes end-to-end, can we not say that the bytes together form a (very large) integer, and that the integer already appears in the set of natural numbers?
You know I think there is this phenomenon of "listening in the wrong language". As an American in France, sometimes I'd meet people and they'd want to try out their English with me. Every once in a while I'd meet someone and be totally confused by their French, and I'd have to ask for help. "Dude, he's speaking English to you". Then we'd both feel bad.
For a while now I've been thinking about something like this: once a month I put some amount of money in my "browser account". Maybe it's $25, or $10, maybe sometimes it's nothing at all. This money gets automatically distributed to the sites I visit, potentially subject to a white/black list that I control. Payments automated, cryptographically secure. I never see an ad or run ad-related JS. And the sites that I support get my patronage as directly as possible. Edit: oh, and somehow my privacy is magically preserved. Small detail. :-)
Maybe that's too close to the "public radio" model, and would never work at scale, but I'd pay for it.
Particularly for "small" languages like Go and C, there is a benefit to sitting down with a comprehensive reference and reading it cover-to-cover. A few hours/days spent in quiet study will pay dividends over just searching Google/Stackoverflow for how to fix whatever your current problem is.