Look -- as long as "data scientist" is a sexy job title, a lot of different jobs are going to claim they fall under that umbrella. I have an applied math background, and I'm fine with scientific computing, but I have much less experience with databases. I'm a very different candidate than a software engineer who took a machine learning course. Maybe in a few years we'll have more intelligent language for making those distinctions.
It shouldn't be surprising or bad news that some "data scientists" have deeper knowledge than others. We're going through a quantitative revolution -- many fields and industries are nearly untouched by statistical analysis/machine learning, and so there's a lot of low-hanging fruit in going from "nothing" to "something." Even somebody who only knows a little can add value at these margins. But, of course, that won't be true forever -- look at quantitative finance, which is very competitive and requires a lot of education, because the low-hanging fruit was picked in the 90's.
There's room in this world for the statistician, the mathematician, the database engineer, the AI guy, the data visualization expert, the codemonkey who knows a few ML methods, etc.
I'm kind of the villain of this piece -- a relatively novice coder who expects to apply for programming-heavy jobs in the future.
Here's why I think I'm not actually a villain:
1. I'm not under any illusions that people should pay me mega-bucks because I can program at all. I haven't hit my 10,000 hours yet; closer to 1000. You should hire me because I'm a mathematician who can program, not because I'm a hacker genius. Often, being an "X who can program" is much more valuable than just an X. It means you can execute your ideas yourself; and it means you have a clear concept of which ideas can be executed computationally and which cannot.
2. I couldn't just "quit programming" any more than I could quit writing or quit reading. I could resolve to, but I wouldn't last long. I've found that when something makes my brain happy, but isn't technically "my job," I'm much better off following instinct and doing it anyway, rather than forcing myself never to have any side projects.
3. You're worried I'll contribute to the volume of bad code in the world? Well, just because there are more experienced programmers than me doesn't mean I'm dishonest or stupid. I make a point to be very frank about what I can and can't do at the moment. I've found that the stuff I can do is quite valuable to people. I don't try to bluff my way into projects I'm obviously not qualified to contribute to.
There's a difference between optimizing for current conditions, and changing future conditions.
Optimizing for current conditions: right now, a lot of the best job applicants have degrees from top-tier schools, and if you need to hire someone today, using education as a job filter is a reasonable choice.
Changing future conditions: there are plausible arguments that universities should not have a monopoly on education and credentialing, and promoting alternatives to a college degree may be a reasonable long-term advocacy and philanthropic goal.
The real story is: what's going on with this new hedge fund? Is the focus on "conclusions that are fundamentally correct but missed by most of the world" an actual strategy or a contrarian buzzword?
A good deal of experimental research is a cartel on resources and data. The "open access" movement in biology is not what programmers would think of as open; you can get access to many "open" data sets only by application, and you pretty much need to be an academic biologist to be granted an application. Science would be much stronger if there was a norm of truly sharing data. But restricting access is in the interest of each individual researcher who wants to maintain his/her relative prestige advantage.
Violence against women and violence against men are different. Men are more likely to be the victims of violent crimes, and especially more likely to be attacked by strangers. Violence against women is more likely to be rape or domestic violence. Men are more likely to wind up in something you could describe as a "fight." Depending on context, you could say that men are in more danger than women (because most violence is man vs. man) or that women are in more danger than men (because a lot of violence against women comes from familiar figures like family members or boyfriends, and because self-defense is often harder for women.)
I don't think "snake oil" is the right paradigm here. In ML/AI, lots of honest researchers are wrong; being a scientist who's wrong doesn't make you a criminal.
That said: Hawkins' principles are very different both from what the brain does, and from what the state of the art in machine learning does. My impression is that HTM's attempt to be too general and assume too little about the problem.
For vision in particular, most successful computer vision algorithms (as well as what we know about the visual cortex's mechanisms) make extensive use of information related to the fact that the image is an image. That is: edges are probably more likely to be continuous than broken; locally constant curvature is more likely than not; textures and colors usually continue over the surface of an object; objects occlude other objects; etc. Brains and effective computer vision algorithms hard-code a lot of information about the nature of the problem they're solving. Hawkins wants to bypass that, and I think it's probably too ambitious an aspiration.
Then again, if he makes it, more power to him.
I don't think we should be prejudiced against someone who comes from the tech industry and wrote a popular book. It's certainly not "snake oil" -- it seems to be a good-faith attempt to solve an important problem. I think the odds are against it working, but that's not a moral condemnation.
Read the paper; HTM's don't seem to do better than other object recognition algorithms at recognizing shapes, especially because there are visual properties it ignores (curvature, global topological properties, etc.) The accuracy for the picture datasets are only 60-70%. What's interesting about HTM is its generality. I can't judge whether it would be good for the Grok prediction engine, but I know more about image recognition and you definitely don't want to use it for that.
1. Track the things you want to accomplish. Joe's Goals is good for that. If your problem is working enough hours a day, track hours.
2. To-Do lists. I use ToodleDo religiously.
3. Block distracting internet sites. I use SelfControl (for Mac.)
4. Quit your biggest distractions altogether. (For me, that's webforums. I deleted all my webforum accounts.)
Systems are great if it's just forcing yourself to do stuff you're really going to need to do. In a way, that's the easy part. The hard part: when you chronically avoid doing something, often it's because you genuinely don't want to do it, and the right answer is to find a way to stop doing it entirely. One of the biggest improvements in my recent life was to identify certain kinds of work that made me miserable, and make a plan so that I'd never have to do them again. Discipline is a good thing for work that's a little hard and boring but still worthwhile to you; but discipline can't get you through work that you really hate, at least not for long.
The lowest stress, highest paying per hour job I can think of is tutoring high school students. $50 an hour is normal, $100 is possible in affluent neighborhoods if you have a good pitch about why you're worth it; I have even heard of $500 an hour, but that was a math PhD student who found a remarkably rich Manhattan family.
This is an example of something I've been wondering about for years.
We have cryptography and cryptanalysis. But while we have machine learning, we don't really have an established science of "anti-learning" -- creating examples that are hard to classify correctly, in order to thwart snoops.
Oh, I think so too. For a while that was what I wanted to work on, except that I figured Whitetruffle were too good to compete with.
One critical part of a business like that is connections with the recruiting industry, because you're essentially asking employers to let you serve as their recruiter, and you need to build trust. You can't just build the site and wait for people to come. Whitetruffle got that part right: their founder is a former recruiter.
I'm well aware of the drawbacks of academic science. I'm aware that some of the best progress happens outside of academia. It's not about that; it's about resources.
I've been doing some consulting for a company that's working on improving medical diagnosis (among other things.) I'm trying to build risk models for diseases based on genetic data. The problem is that this data isn't freely available. "Open" access data is usually only "open" in the sense that you can apply for access, and you usually have to be a biology professor to be considered. It's not like the tech world at all. We're used to open source. They're not.
If you want to do science, you need access to resources. You need a lab, or you need experimental data. It's really hard to do that outside of academia. Heck, it's hard to read journal articles outside of academia.
Going to school, for all its hassles, is essentially free access to resources. You don't have to be narrowly academia-minded to get an education -- I'm getting a math PhD, but I'm philosophically closer to the tech industry and I don't plan to work in academia. Grad school isn't a tribal identity, it's an opportunity to get what you value out of it.
www.whitetruffle.com is the best I've found so far.
It matches based on skills and a few other attributes (like location and whether you have a US visa.) I've talked to them, since this is a subject close to my heart and I'm curious, and they're planning on adding a more detailed questionnaire and a little machine learning.
Right now they're restricted to New York and the Bay Area and they're focused on engineering positions, mostly at startups. But they do a pretty darn good job. I'm on there, and I get a job offer every few weeks or so, many from excellent companies.
It's a "The way to Tara is via Holyhead" kind of thing.