How's it like working with schools/universities as a startup? I've always heard edtech can be a slow, bureaucratic sales cycle (and maybe not a high willingness to pay?).
I know a few different companies who ultimately moved out of the education market completely or just try to leverage their education traction as a beachhead to other markets. It sounds like you're focused on the education market - what's your take?
Can you explain a bit more about this? Are you building a profile of what I write and who I write it to? Fetching relevant examples and passing that through some (cloud?) LLM when writing the email?
I'm definitely curious on the technicals but there is also a bit of a trust element here - both on trusting that my email (likely some of my most sensitive data) is handled with care and trust that the actual responses are phrased well.
How are you handling the formatting and tone part of the email so that it doesn't sound like AI? I've tried to use AI tools for email multiple times but always end up significantly editing or rewriting the email myself.
Very cool, congrats on the launch! What's your plan for when one of the larger players like ElevenLabs or Google adds support for these languages? I would guess the reason why they haven't is because they don't see a large opportunity. How are you thinking about it?
This is really interesting, definitely going to give it a try! Seems fun but are you seeing people actually needing to make lots of videos like this? What's your vision - how does this become really big?
I know it's not unbiased, hence the quotes and why it is stated more as an observation of the how it can be difficult to find. Like most tools, this has flaws but I find it still useful because it presents questions to reflect on e.g. potential contradictions in logic and reasoning. I don't know of another way to get an analysis of inner thoughts easily.
Love the idea! It's hard to get an "unbiased" outside perspective, especially on more personal, inner thoughts. Will definitely try this out, thanks for sharing.
Why do you think Anthropic has such a large system prompt then? Do you have any data or citable experience suggesting that the prompting isn't that important? Genuinely curious as we are debating at my workplace on how much investment into prompt engineering is worth it so any additional data points would be super helpful.
After this experience, do you think you'll ever build anything in the future with Electron? When do you think Electron is actually the right choice (if at all)?
I was confused by this as ChatGPT launched in Nov 2022 and had tens of millions of users by end of 2022.
> And though when we started our business in 2023 (ChatGPT wasn’t out yet), you could begin to feel that something like that was possible in a way it wasn’t before.
Are there any approaches today that you've found are at least mostly reliable? Bonus points if it is somewhat clear/easy/predictable to know when it isn't or won't be.
We use human evaluation but that is naturally far from scalable, which has especially been a problem when working on more complicated workflows/chains where changes can have a cascading effect. I've been encouraging a lot of dev experimentation on my team but would like to get a more consistent eval approach so we can evaluate and discuss changes with more grounded results. If all of these metrics are low confidence, they become counterproductive since people easily fall into the trap of optimizing the metric.
Where can I learn more detail about the metrics you support and how they work?
I tried multiple other solutions but kept running into the problem that occasionally the framework would give me some score/evaluation of an LLM response that didn't make any sense, and there was minimal information about how it came up with the score. Often, I'd end up digging into the implementation of the framework to find the underlying evaluation prompt or classifier only to realize that the metric name is confusing or results are low confidence. I'm more cautious about using these tools now and look more deeply at how they work so that I can assess grading quality before relying on them to identify problematic outputs (e.g. hallucinations).
I know a few different companies who ultimately moved out of the education market completely or just try to leverage their education traction as a beachhead to other markets. It sounds like you're focused on the education market - what's your take?