Good catch on the disclosure, I edited my original comment to reflect this fact.
On the topic of vector search, Milvus is another great vector database - it's open source and we provide single-line startup scripts via `docker-compose` in addition to installation via apt & yum (https://milvus.io/docs/install_standalone-docker.md). There are also no restrictions on the number of vectors that users can store. Internally, we've successfully scaled Milvus to handle billion+ vectors, while many of our users have stored hundreds of millions of vectors in a production environments as well.
Great article. We used something very similar to help implement simlarity search at Yahoo a couple years back (https://yahooresearch.tumblr.com/post/158115871236/introduci...). We were using a indexing strategy called Locally Optimized Product Quantization, which worked great in terms of query times but required a training procedure which made successive inserts fairly inefficient.
Thankfully, we have a much wider variety of indexing options these days (https://milvus.io/docs/index.md) in addition to powerful vector databases (https://zilliz.com/learn/what-is-vector-database). I'm glad to see the barrier to entry for semantic image retrieval becoming lower and lower as ML infrastructure matures.
The crux of the problem is not Luna itself, but the Anchor Protocol (19.5% yield), which enticed users to burn Luna by investing in UST through a, for lack of a better phrase, Ponzi scheme. I get it - people like decentralization. But without some form of regulation, either by the broader crypto community itself or by governments around the world, situations like the LUNA/UST collapse will keep happening.
It just seems unlikely that the algorithm would be open-sourced right after a deal for Twitter is agreed upon (but before it actually goes through). I've never seen a buyout of this scale done by an individual, but I imagine the SEC and several other parties will need to be involved.
At the minimum, I would make a private Github repo first, add all relevant commits, and then make it public once there's actually content.
Even as an ML-focused graphics-less GPU, this is great. If this can be prototyped on an FPGA, it would be even better. Using block RAM for shared memory and built-in PCIe and DDR IP blocks should help speed things up considerably.
It unfortunately wouldn't be very cost-effective for training ML models, but it would take things a step closer to actual tape-out (if some organization has the $$$ for it).
I haven't tried SimCLR, but I did try face embedding models trained with contrastive and triplet loss. For applications where precision is the key metric, I do agree that these loss functions are much better overall.
If discovery or recall is what you're after, a generic image classification model trained with binary cross-entropy might be better. For example, performing reverse image search on a photo of a German Shepherd should always return images of GSheps in the first N pages, but showing other dog breeds in later pages and possibly even cats after that would be a desirable feature for many search/retrieval solutions. An embedding model trained with contrastive loss might have this behavior to a certain extent, but a model based on BCE should be better.
One quick note: k-d trees are great for indexing low-dimensional data, but for high-dimensional embeddings they tend to be a poor indexing choice since you'll end up visiting more nodes in the tree than you'd like. I found [1] to be a great overview of different indexing types for high-dimensional vectors and the advantages of each.
> ViT models are outperforming CNNs in terms of computational efficiency and accuracy, achieving highly competitive performance in tasks like image classification, object detection, and semantic image segmentation.
Since then, this has been show to be untrue. Using more modern training techniques along with depthwise convolutions (https://arxiv.org/abs/2201.03545) results in equal if not better performance on vision tasks. Improved training methodologies have also been shown to boost the accuracy of ResNet50 - an 6-year-old pure convolutional architecture - on ImageNet-1k by over 5% (https://arxiv.org/abs/2110.00476).
Pure ViTs are also more difficult to train when compared with traditional convnets, although this has since then been somewhat remedied by Swin (https://arxiv.org/abs/2103.14030).
Hikaru's win in the Grand Prix was great, but the Candidates is far different. For example, Alireza Firouzja - another member of the upcoming Candidates cycle - has been MIA for a while, likely due to the insane amount of time he is putting into Candidates prep.
Since Hikaru has far fewer recent classical matches than the other upcoming Candidates participants (with the exception of current world #2 Ding Liren, who still needs to finish 30 games before May), he'll have an edge when it comes to preparation. However, unless he prepares like a madman, he'll be at most a wildcard candidate, perhaps beating a favorite or two but unlikely to win it all.
I'll personally be rooting for Ding and Fabi. I like Hikaru as well, but I unfortunately just don't see him beating Magnus in a World Chess Championship.
> That said, Hacker News readers shouldn't be surprised that people use jargon to demonstrate membership of an in-group or keep away outsiders.
To say that HN readers use specific keywords or acronyms is inaccurate, given the diversity of topics that make it to the front page. However, the discussions I see take place here are definitely of much higher overall quality than the average comments thread on other sites.
I'm curious what an optimal strategy for playing such a complex game like this is (if it even exists). I also wonder how much of a role intuition plays, i.e. the ability for chess masters to "feel" whether or not positions are winning or losing.
We're a small group of folks working on an open-source project for generating embedding vectors. If you're interested in machine learning or semantic search, we welcome you to join us!
A concise but not-completely-accurate way I explain it to a lot of pure software folks is that every line of HDL code executes "simultaneously". This can sometimes help them wrap their heads around Verilog/VHDL a bit better.
On the topic of vector search, Milvus is another great vector database - it's open source and we provide single-line startup scripts via `docker-compose` in addition to installation via apt & yum (https://milvus.io/docs/install_standalone-docker.md). There are also no restrictions on the number of vectors that users can store. Internally, we've successfully scaled Milvus to handle billion+ vectors, while many of our users have stored hundreds of millions of vectors in a production environments as well.