I'm currently taking a graduate class in Applied Cryptography and will definitely use this as another resource for reference. Love that they've opened it up and made it free.
I use a Jupyter notebook to explore how a Bayesian might compare two products on Amazon with the goal of finding the probability that one product is better than another.
That makes sense. I'm coming from the angle of applied ML where solutions need to solve a business problem rather than advance the field of ML. In consulting many problems can't be solved well without a labeled dataset and in lieu of one, less credible data scientists will claim they can solve it in an unsupervised manner.
I didn't mean to make it sound incidental although I do see your point. Just wanted to chime in with how important having a labeled dataset is for a successful ML project.
I don't disagree with your point, but the unsupervised aspect of NLP typically isn't useful on its own. Usually it's a form of pre-training to help supervised models perform better with less data.
From Google in 2018:
"One of the biggest challenges in natural language processing (NLP) is the shortage of training data. Because NLP is a diversified field with many distinct tasks, most task-specific datasets contain only a few thousand or a few hundred thousand human-labeled training examples. However, modern deep learning-based NLP models see benefits from much larger amounts of data, improving when trained on millions, or billions, of annotated training examples. To help close this gap in data, researchers have developed a variety of techniques for training general purpose language representation models using the enormous amount of unannotated text on the web (known as pre-training). The pre-trained model can then be fine-tuned on small-data NLP tasks like question answering and sentiment analysis, resulting in substantial accuracy improvements compared to training on these datasets from scratch."
I basically agree with this rule. I find that my colleagues who overly hype unsupervised approaches typically don't have much experience working on ML problems without labeled data. My suspicion of this comes from the fact that whenever I give a talk on ML I always have a wealth of personal experience to draw on for examples. My colleagues almost always reuse slides from projects they never worked on.
Time will tell, but as a machine learning engineer, when you see results this good it's more probable that a mistake was made. They could be reporting the training error on an overfit model or data leakage could be occuring due to an improper train-test spilt of the data.
Also, it is definitely appropriate to use the term AI in this case. AI is not a technical term so it's really in the eye of the beholder, but I think it's safe to say that ML is a subset of AI. Perhaps people are conflating AI with AGI?
I appreciate this thoughtful and detailed reply. I was thinking all those things in my head while reading as well, but couldn't bring myself to invest the time to address them all. I got the impression that the author wasn't someone who has a lot of experience building real-world predictive models otherwise they'd appreciate the trade-offs that need to be made sometimes to get something that works well and can be debugged/interpreted without too much trouble. Of course this isn't to say we shouldn't be striving to develop more interpretable solutions, but I don't think this paper is very helpful to due to its lack of rigor and straw-man tactics.
I'm excited to see a couple lectures with Swift. All the work to add interoperability with Python and make swift-jupyter is very appreciated and feels like it's Xmas in June.