Great project! Love the library+db approach. Some questions:
1. How much work is it to add bindings for new languages?
2. I know you provide conductor as a service. What are my options for workflow recovery if I don't have outbound network access?
3. Considering this came out of https://dbos-project.github.io/, do you guys have plans beyond durable workflows?
Shameless plug - for folks who don't want to take on the work of model selection, on-demand scaling of model serving, scaling the vector database for search set size and query throughput, we built a service that hides all this behind a simple API [1]. The example in [1] is for images, but here is quick-start for text [2].
The easiest and likely most effective method may be to compute vector embeddings using a sentence transformer model, and find nearest neighbors among these vectors for all articles in the set. The distance between the nearest vectors will give you a degree of similarity between the articles. You'll need to figure out some thresholds on these distances to figure out what are near copies vs different articles on the same story. There are efficient methods to find approximate nearest neighbors among a large set of these vectors - available as both OSS and SaaS. Faiss [1], ScaNN [2], and Pinecone [3] are some examples.
This is one of the methods mentioned in the article. I don't have implementation experience with the other string distance measures in the article (under "normalized string" in the table), except for Q-grams. Compared to the above method Q-grams don't scale as well and are not as robust because it doesn't encapsulate an understanding of the semantics of the text.
Ha, I agree that software engineering is too hard for ML engineers, and even for software engineers like myself who have been doing it for 20 years like
zcw100 said :).
Author of the blog post here. It was definitely written from my narrow viewpoint and experience. Our goal is to make more solutions accessible to software developers and instinct was the same as yours - a lot of ML can be within the realm of engineering (even small / one-person teams) and that there are accidental complexities standing in the way of wider use. Our solution (AutoML+SaaS) def doesn't work for every situation. I'm curious to hear more of your thoughts on how ML can be made more accessible to Eng (and vice versa).
Author of the blog post here - it's very cool to see this on HN!
I wrote this as someone who considers himself a half-decent software engineer trying to use ML for a side project and feeling frustrated by all the effort and "accidental complexity" involved. Why focus on software engineers and ML in this post/rant/company? Because "software is eating the world" and having ML be more accessible to software engineers will broaden the range of problems they can solve.
Thanks for all the comments - I acknowledge all/most of the criticisms as valid. A SaaS/AutoML solution won't work for everyone and definitely not for every problem, and it won't be the only answer to making ML more approachable.
There are a continuum of offerings in this space. Some have lots of custom control of the training pipeline and deployment, and on the other side, things like RoboFlow that try to make it easy / hide the complexity. We consider ourselves even further to the “hide complexity” side, since we try several deep networks automatically vs. making you choose, re-train automatically, abstract-away non-essential ML jargon, etc. In addition, we don't limit ourselves to only vision - we'd like to be the one stop shop for ML as a service. We also have developer-friendly pricing with quick and easy signup.
Thank you! We do think that "model export" is important, but we're still working out how to do it in the most seamless and non-ML-expert friendly way. Do you have a use-case and target hardware in mind?
Thanks you for the kind words and feedback! You basically went through most of the UI flow that we designed for. You're spot-on about testing new classifiers - answering the question "Can ML even help with my problem?" is much easier with Nyckel and prototyping and rapid iteration starts with that.
Our goal is to be cost-competitive, even for bigger projects. Given how early we are, our pricing structure is still being worked on, especially for high-volume.
Integrating with labeling solutions is in our roadmap. In the meantime, our API should enable any data/labeling integrations.
Named entity recognition is also in the roadmap. Would love to hear more about your use-case and we can give you access to the beta when ready.
Thanks for trying us out. We just added a beta for classifying tabular inputs (a mixture of text and numbers) - this may be of interest to you. We have seen some people use our platform to detect stock market trends. Let us know how it goes and reach out if we can help (george at nyckel dot com).
Desired Output is what you tagged it as. Function Output is what the model predicted.
We tried to make the lingo developer-friendly. We think of models as functions that transform inputs to outputs. Instead of writing code to do so, as developers usually do, you train the function by providing desired outputs to sample inputs.
Thanks for trying us out and for the feedback! I agree that our filters are a little confusing right now and we're working on fixing it. In the meantime, here are a couple of filters you could try:
- To see all cases where the model disagrees with your annotation: Function Output = Disagrees, Desired Output = Any.
- To see the least confident predictions from the model: Function Output = Any, Desired Output = Any, Sort By = Least Confident Prediction.
Your idea us helping you pick a confidence threshold is a good one. We'll get that into our near-term roadmap.
Agreed that they would be convenient. We are looking at both those options. There are devils in the details like seamlessly taking advantage of available hardware acceleration.
Would love to talk more about your use case so we prioritize the right things for model export. Drop me a line (george at nyckel dot com).
We think the number of use-cases for ML is going to grow drastically as 1) ML state-of-the-art continues to get better; and 2) Developers realize how accessible it can be. More problems will be solved by a "machine-learned function" rather than imperatively coded functions. Taking it one step at a time, in 5-10 years we'd love to be the default place the developers go to solve their ML problems. This would mean that we go beyond image/text classification to a broader set of input and output types.
Thanks for the feedback on the UI! None of us are UI experts and we agree that it isn't great. We're working to make it better.
1. How much work is it to add bindings for new languages? 2. I know you provide conductor as a service. What are my options for workflow recovery if I don't have outbound network access? 3. Considering this came out of https://dbos-project.github.io/, do you guys have plans beyond durable workflows?