I recently wrote a post outlining our method to reduce hallucinations in LLM agents by leveraging a verified semantic cache. The approach pre-populates the cache with verified question-answer pairs, ensuring that frequently asked questions are answered accurately and consistently without invoking the LLM unnecessarily.
The key idea lies in dynamically determining how queries are handled:
- Strong matches (≥80% similarity): Responses are directly served from the cache.
- Partial matches (60–80% similarity): Verified answers are used as few-shot examples to guide the LLM.
- No matches (<60% similarity): The query is processed by the LLM as usual.
This not only minimizes hallucinations but also reduces costs and improves response times.
If the user asks such a question, your agent should not invoke the RAG at all, but simply answer from the history. You need to focus on your orchestration step.
Search for ReAct agents, can build using either LangGraph or Bedrock Agents.
Skeptical that this will be a "good" experience for everyone involved considering how generic AI openers/responses are, but also hopeful that it can reduce friction for some.
Not an expert by any means but streaming HQ video is pretty expensive (even more so for live content), seems like the only providers that can do so profitably are YouTube and Netflix. I'm sure a big reason for that is the engineering (esp. CDN)
Hmm interesting, didn't realize that data sovereignty requirements were so stringent. Wonder how other cloud providers are doing in this sense considering GPU shortages across the board.
I'm confused, what's expensive about it? It's a serverless pay per token model?
Do you mean specifically the Bedrock Knowledgebase/RAG -- that uses serverless OpenSearch which costs at minimum $200ish/month bc it doesn't scale to zero?
Also shouting out Continue.dev for vscode users.
I set it up yesterday, open-source version of Cursor.
(not affiliated, I tried to setup Avante but I'm a neovim noob and have skill issues)
Just set it up 2 days ago. I fully agree with his take. Cursor does a lot to reduce the headache of copy pasting results from ChatGPT/Claude.
Makes coding with AI much faster and less of a chore. You still need to know what you're doing as the models are fantastic at boilerplate but often get things wrong. But it's significantly sped up iteration for me, and hence translates to more fun.
WITCH is the acronym for Indian tech consulting companies for WiPro, InfoSys, Tata Consultancy Services, C something, H something. They're stereotyped as being cheap and low-quality.
Not agreeing/disagreeing here, just stating author's intent.
Do publications to my BigCo employer's engineering blog count towards an O-1 application? Or is it just Research Papers/citations to formal journals that matter?
What are some other common ways to build a portfolio towards an O-1 application other than research citations or founding a startup?
https://aws.amazon.com/blogs/opensource/using-strands-agents...