Orange Words. My hobby project, a hacker news search system. It was initially created by hand and now I use AI augmented development. It's a good low risk environment for experimenting.
Ah, so this is why I suddenly got a bunch of email.
Hey all, site owner here. Thanks for the visits and all the fun stories! I really miss this era of computing. Feel free to let me know if you have something that should be added to the site.
Writing code that runs down hole or otherwise connects back to the real world would be fun. Maybe I should pickup firmware skills. Good luck with your hiring!
For Vespa there's a managed version hosted by the Vespa company in their cloud environment, and then the open source version is easily run locally or in any environment of your choosing. It takes some attention to detail, but it's quite flexible. I have a long running single node instance on an Intel NUC, but I've also run more complex cluster variations across different cloud environments.
Unrelated to the core topic, I really enjoy the aesthetic of their website. Another similar one is from Fixie.ai (also, interestingly, one of their customers).
For me, it was a bit different, and it comes from a perspective that's a blend of cognitive science and computer science:
Complex systems can be created through the composition of simple processes which are easily explained or modeled. Sometimes there are mysterious emergent properties in the overall system, even when we can explain the components. Other times, through investigation / science / engineering, we are finally able to explain the entire system. It might lose a little of the magic or mystery as a result, but the system itself didn't change. Instead our perspective and understanding changed.
On that note, until we can fully explain some of the workings of our own minds, I'm reluctant to write off "just predicting the next token" as an unimportant process. It's one way to explain LLM inference simply, but it doesn't eliminate the importance. It also doesn't account for as-yet unexplained things which may be happening as a part of training.
Scaling up language models has been shown to predictably improve performance and sample efficiency on a wide range of downstream tasks. This paper instead discusses an unpredictable phenomenon that we refer to as emergent abilities of large language models. We consider an ability to be emergent if it is not present in smaller models but is present in larger models. Thus, emergent abilities cannot be predicted simply by extrapolating the performance of smaller models. The existence of such emergence implies that additional scaling could further expand the range of capabilities of language models.
Version history (for relevant dates):
[v1] Wed, 15 Jun 2022 17:32:01 UTC (59 KB)
[v2] Wed, 26 Oct 2022 05:06:24 UTC (88 KB)
There can be multiple reasons for this[0], including but not limited to:
* The people or industry have low tolerance or fear around risk of false positives
* The industry is centered around billable hours and has no incentive for automation
* The engineers or people perceive ML as this obscure/difficult thing
I'd say the incentives and risks have hindered lots of legal adoption (this is what I observed while working in legaltech for instance). Insurance sounds similar, but I'm less familiar and assume they are coming along more quickly.
[0] I agree with minimaxir's point, that it's a bad assumption to think few teams use basic ML functionality. This will become even more true as emergent tech such as zero shot classification with LLMs becomes more commoditized.
As a general rule (for now), you'll get the best search result for your dev time, with straight ahead BM25 via a js lib.
In terms of overhead, with lower doc counts there's not much overhead with embeddings and knn/ann. Imagine 384 floats per doc or whatever embedding size. At scale it becomes more problematic and less comparable.
With all that said, messing around with vector ops and WASM sounds more fun :)
This is fun to see, and for a seriously deserving team!
Vespa really is an impressive platform. I've been working with search platforms for a while, starting with FAST Instream, MarkLogic, Elastic, Weaviate, and now Vespa. As others have noted, Vespa has a deep technical lineage. You can see a lot of the hard earned lessons in the design, as well as improvements in the flaws of the spiritual predecessors (FAST et. al).
The history alone makes the project interesting. But then, in the last couple of years (or maybe earlier), they started introducing all the fundamentals for supporting deep integration of tensors and a custom HNSW implementation. Whether accident or planning, this was a nice strategic move. They were leagues ahead, just in time for the popularization of learned embeddings.
Now they have the best combination of traditional lexical search, semantic search with embeddings, and the combination of the two in hybrid search. That's without even getting to all the functionality in the multi-phase ranking, hosted ML models, and document processing engine.
Obviously I'm biased (we use Vespa at work for enterprise search products), but I can't recommend this engine/platform enough.
Orange Words. My hobby project, a hacker news search system. It was initially created by hand and now I use AI augmented development. It's a good low risk environment for experimenting.