I think we are missing the bigger point here. Licensing only matters on things that take real effort or money to produce. Who will care about licenses on software when software is free and infinite? It would be like licensing each ounce of water on Earth.
The `try_roots` example here is actually a _counterexample_ to the author's main argument. They explicitly ignore the "negative discriminant" case. What happens if we consider it?
If we take their "parse" approach, then the types of the arguments a, b, and c have to somehow encode the constraint `b^2 - 4ac >= 0`. This would be a total mess--I can't think of any clean way to do this in Rust. It makes _much_ more sense to simply return an Option and do the validation within the function.
In general, I think validation is often the best way to solve the problem. The only counterexample, which the author fixates on in the post, is when one particular value is constrained in a clean, statically verifiable way. Most of the time, validation is used to check (possibly complex) interactions between multiple values, and "parsing" isn't at all convenient.
To optimize for fast nearest neighbors, I chose 256 dims. Notably, this actually hurt some of the pre-training classification losses pretty severely compared to 2k dims, so it definitely has a quality cost.
The site uses cosine distance. The code itself implements Euclidean distance, but I decided to normalize the vectors last minute out of FUD that some unusually small vectors would appear as neighbors for an abnormal number of examples.
You definitely highlighted a shortcoming of the feature vector model in this case. Indeed it's quite a small model trained on a single Mac for about a week, so it's not very "smart".
I'd expect that this is a problem that could be solved by using larger off the shelf models for image similarity. For this project, I thought it would be cooler to train the model end-to-end myself, but doing so has negative consequences for sure.
I think it would be a useful feature. For the sake of being a fun project, I didn't use CLIP because I only wanted to use models that I trained myself on a single Mac. However, to make this more useful, text search would be quite helpful.
Yup, it's a small model I trained on my Mac mini! The model itself just classifies product attributes like keywords, price, retailer, etc. The features it learns are then used as embeddings
> I asked Nicole Avena, a professor of neuroscience at Mount Sinai who studies sugar addiction, if she believed it could be possible for food companies to engineer, intentionally or not, compounds that would make GLP-1 drugs less effective. Avena told me it was plausible.
Never really thought about this before. The food industry is a virus and current weight loss drugs are the best vaccine we have, but it'll forever be an arms race.
OP here. Interesting, I had not researched the used car market enough to know about the price inflation. Can you think of any other sites that have more realistic prices?
These are good ideas! It should be possible to test how photographic quality correlates with price, though one complication is that I train with data augmentation (a commonly used technique to make better use of the training data). Some of the augmentations such as random cropping and color jitter might already stimulate some level of bad photography.
OP here. Yeah, it's quite odd that the median price prediction doesn't fall into the most likely bucket, even when that bucket has more than a 50% confidence. I'm not sure how to interpret this. One theory: maybe the median prediction head hasn't converged yet and naturally has a bias towards lower values due to the network's initialization.
I'll know more in a week or two once all the models have converged, and can update here!
It's definitely not just a regularizer in my case, because the gap appears even before a single epoch. The gap does also appear for two very different model architectures.
One explanation is that price labels are super noisy. If there is enough noise in the primary labels, you could imagine that adding in the more predictable target variables could help reduce gradient noise and speed up training. That's my current hypothesis, but I'm very open to others. If I had more time I'd try to do more experiments on this.