Openlayer (YC S21) | Remote or onsite (US, CA, NYC) | Full-Time | Customer Engineer
Openlayer is solving the AI reliability problem. We are looking for stellar customer engineers to join the team and help make customers happy. Think air-traffic control for our customers: you will be in charge of building requested features, integrations, and fixing bugs.
We are hiring in SF, NYC, and open to remote. Our team is currently 6 people, but we are very actively growing and adding more headcount. Those with experience in Python, Next.js, Docker, and cloud compute systems are encouraged to apply!
Openlayer (YC S21) | Remote or onsite (US, CA, NYC) | Full-Time | Senior Software Engineer, Product
Openlayer is solving the AI reliability problem. We are looking for stellar frontend engineers to join the team and help build the platform that next-gen teams use to build the most powerful AI systems.
We are hiring in SF, NYC, and open to remote. Our team is currently 6 people, but we are very actively growing and adding more headcount. Those with experience in Next.js, React, and TypeScript are encouraged to apply!
Most of us get how crucial AI evals are now. The thing is, almost all the eval platforms we've seen are clunky and don't have any product cohesion. There's too much manual setup and adaptation needed, which breaks developers' workflows.
That's why we're releasing a simpler workflow.
If you're using GitHub, you only need to add two files to the repo (one config + one script). Then, connect your repo to Openlayer and define must-pass tests for your AI system. Once integrated, every commit triggers these tests automatically on Openlayer, ensuring continuous evaluation without extra effort.
We offer 100+ tests (and are always adding more), including custom tests. We're language-agnostic, and you can customize the workflow using our CLI and REST API.
As a final note, you can leverage the same setup to monitor your live AI systems after you deploy them. It's just a matter of setting some env vars in your staging/prod environments, and your Openlayer tests will run on top of your live data and send alerts if they start failing.
Compared to Galileo, we offer a more comprehensive suite of evals that support more tasks than LLMs and NLP.
We offer more features around error and subpopulation analysis, versioning, running evals during development, and collaboration. Through what (I believe) is a more clean and simple DevEx and UI!
re: Lilac, there’s some intersect w/r/t dataset exploration, but we have more evals than the ones they offer.
More than data quality, we give insights into data drift and model performance and let you set up expectations and get alerts on whether they fail during development and production. + distinct in some of the ways described above
We’re really happy to see more tools and platforms in this space. Definitely a big uptick since we started 3 years ago, w the advent of gen ai this is all top of mind (and deservedly so).
You can upload just the predictions of the model (and whatever metadata you want to track), so in that sense any format is supported.
If you want to unlock explainability for your tabular classification or regression, or text classification models, you can upload the actual model binary. We support a bunch of frameworks out-of-the-box, but you can use any architecture through our custom upload.
We noticed that the industry is laser-focused on tackling feature drift after production, but we spotted a gap. Most ML teams are wrestling with model validation even before anything is deployed in production. We also noticed that post-deployment analysis sometimes misses the mark, lacking components like identifying underperforming cohorts or giving actionable insights. This leads to a barrage of alerts and the inevitable alert fatigue.
We decided to start Openlayer to offer a more holistic solution that helps teams from the ML development and experiment tracking phase to more advanced tasks like monitoring and fairness. We established a strong baseline with this launch and are now building several features on top.
We work with both startups and enterprises across a range of task types!
Some common ones are fraud and churn detection for financial institutions or e-commerce sites (both tabular classification examples). It's very important for these types of tasks in particular to guard against biases and false negatives, so they use us to set up wide test nets that help give them assurance that their models are working properly before they hit production (and to monitor them post-deployment).
Another example is Zuma (https://www.getzuma.com), a startup building an AI-driven chatbot that uses us to track their experiments and improve the accuracy of their NLP intent classification model.
Of course, we're also building out support for evaluating LLMs. Because this is an open problem, we've been spending a lot of time interviewing people in the space who are building these models (please reach out if this is you!).
It’s important to note a significant percentage of these deaths are cases in which other substances with trace amounts of fentanyl are consumed unbeknownst to the user. Dealers often use the same scales, which is one risk factor for cross-contaminating supply.
My mother is a physician, she just last night told me about a case she saw over the weekend in which a young 20-something nearly OD’d on fentanyl from taking ecstasy. She survived, but with life-altering brain trauma rendering her unable to remember who she is. She needs tubes for her food supply, and a ventilator to breathe.
We have a Next.js website with a blog section that's powered by Sanity. Problem is, it's pretty finicky to install basic components that we like to use (which looks like Payload comes with out of the box) + communicating with Sanity via groq is clunky with Typescript (and possibly causing some SEO issues).
How simple is it to install Payload in an existing Next.js app to power basically just the /blog/* subdirectory?
Great idea! Definitely something I could see being useful in industry.
Have you thought about making this work for more things than ArXiv? Could be interesting to prepend Smort to any URL and create “rooms” for annotating the web (other static content like Wikipedia, or even temporal content like Youtube videos). But maybe that’s too wide of a scope, just got me thinking :)
Openlayer is solving the AI reliability problem. We are looking for stellar customer engineers to join the team and help make customers happy. Think air-traffic control for our customers: you will be in charge of building requested features, integrations, and fixing bugs.
We are hiring in SF, NYC, and open to remote. Our team is currently 6 people, but we are very actively growing and adding more headcount. Those with experience in Python, Next.js, Docker, and cloud compute systems are encouraged to apply!
https://www.ycombinator.com/companies/openlayer/jobs/yIE9WI3...