Just out of curiosity, did you set the reasoning effort to high?
I've seen big jumps in performance between medium and high reasoning. Not that this is will solve your particular issue, but curious to see what settings you had.
Edit: I gave this a shot. My settings were temperature = 0.00 (not sure if these models respect temp settings but did it just in case), I also set reasoning effort to high. I'm using the groq-hosted version.
This was my prompt
"You have a fox, a chicken and some grain. The chicken will eat the grain if left alone with it. The fox is vegetarian, for religious reasons, and will also eat the grain if left alone with it. You need to get all across a river, but your boat can only take one at a time. How do you proceed?"
This was the response:
"*Solution – 7 moves*
| Move | What you take | What’s left on the left bank | What’s on the right bank |
| 7 | *Take the grain* across | – | *Farmer + Fox + Chicken + Grain* |
*Explanation*
- The grain is the only thing that can be eaten, and it cannot be left alone with either the fox or the chicken.
- So the grain must always be with the farmer whenever any other animal is on the same side.
The sequence above never leaves the grain with the fox or the chicken without the farmer, and after the seventh move all three items are safely on the far side."
I wish I could find a link, but I remember a study that argued that large bureaucracies can actually impede authoritarian governments concentrating power.
The idea being that when there are so many levers to pull and a disjointed system managing them, it makes it effectively impossible for a small group to effectively wield power. It’s like a buffer against concentrating power into a single individual.
Not that I’m arguing for endless bloat to the US government, that comes with its own problems. I agree we need to rein it in.
But I think there is a freedom-centric argument for a slightly larger government bureaucracy than is strictly necessary.
Or thinking about it in reverse, the bureaucracy is currently preventing the executive branch from just doing whatever it wants. I know Congress and the Supreme Court should act as blocks, but to paraphrase Stalin how much infantry do they have?
A slow moving bureaucratic executive can act as a buffer against ineffective other branches.
Or for those that may support the current administration consolidating power, what if the tides turn? What if in 4 years whoever the liberal villain du jour is takes power? Are we making it so that AOC is the most powerful president in history?
I think the issue is that this is not "dismantling the wheels of established power", so much as it is centralizing and increasing the power of the executive branch.
It's not getting rid of all these bureaucrats in DC and giving power back to the people. It's getting rid any sort of independence and removing the barriers to centralizing power under Trump so that he can grab even more power and control.
And just to be pedantic, Trump received 49% of the vote and Kamala receive 48%. And that's of people who voted. He received 77 million out 244 million of the voting-eligible population, or around 31%.
There may be a plurality of people who want the Executive branch under Trump to consolidate power, but it's not the majority.
We do this as well with a lot of success. It’s cool to see others kinda independently coalescing around this solution.
What we find really effective is at content ingestion time, we prepend “decorator text” to the document or chunk. This incorporates various metadata about the document (title, author(s), publication date, etc).
Then at query time, we generate a contextual hypothetical document that matches the format of the decorator text.
We add hybrid search (BM25 and rerank) to that, also add filters (documents published between these dates, by this author, this type of content, etc). We have an LLM parameterize those filters and use them as part of our retrieval step.
I've tried a few things that seem to work. The first works pretty much perfectly, but adds quite a bit of latency to the final response. The second isn't perfect, but it's like 95% there
1 - the first option is to break this in to three prompts. The first prompt is either write a brief version, an outline of the full response, or even the full response. The second prompt is a validator, so you pass the output of the first to a prompt that says "does this follow the instructions. Return True | False." If True, send it to a third that says "Now rewrite this to answer the user's question." If False, send it back to the first with instructions to improve the response. This whole process can mean it takes 30 seconds or longer before the streaming of the final answer starts.
There are plenty of variations on the above process, so obviously feel free to experiment.
2 - The second option is to have instructions in your main prompt that says "Start each response with an internal dialogue wrapped in <thinking> </thinking> tags. Inside those tags first describe all of the rules you need to follow, then plan out exactly how you will respond to the user while following those rules."
Then on your frontend have the UI watch for those tags and hide everything between them from the user. This method isn't perfect, but it works extremely well in my experience. And if you're using a model like gpt-4o or claude 3.5 sonnet, it makes it really hard to make a mistake. This is the approach we're currently going with.
I think those people can, and often times should, be involved in the sales process. Espcially for big ticket B2B sales. But really those people just need to be available for demos, technical questions, etc.
If those people had to do their jobs, plus manage pipelines, plus BDR work, plus chasing leads that went cold, plus negotiating over contract language and price. . . well you get the point.
For big ticket B2B stuff, you really need a sales person (sorry Account Executive) running point. It doesn't mean that's the only person you talk to, or really even the person you talk to the most, but an AE needs to own getting the sale closed.
I've seen big jumps in performance between medium and high reasoning. Not that this is will solve your particular issue, but curious to see what settings you had.
Edit: I gave this a shot. My settings were temperature = 0.00 (not sure if these models respect temp settings but did it just in case), I also set reasoning effort to high. I'm using the groq-hosted version.
This was my prompt
"You have a fox, a chicken and some grain. The chicken will eat the grain if left alone with it. The fox is vegetarian, for religious reasons, and will also eat the grain if left alone with it. You need to get all across a river, but your boat can only take one at a time. How do you proceed?"
This was the response:
"*Solution – 7 moves*
| Move | What you take | What’s left on the left bank | What’s on the right bank |
|------|--------------|----------------------------|------------------------|
| 1 | *Take the grain* across | Fox, Chicken | *Farmer + Grain* |
| 2 | *Return alone* | *Farmer + Fox + Chicken* | Grain |
| 3 | *Take the fox* across | Chicken | *Farmer + Fox + Grain* |
| 4 | *Bring the grain back* | *Farmer + Grain + Chicken* | Fox |
| 5 | *Take the chicken* across | Grain | *Farmer + Chicken + Fox* |
| 6 | *Return alone* | *Farmer + Grain* | Chicken + Fox |
| 7 | *Take the grain* across | – | *Farmer + Fox + Chicken + Grain* |
*Explanation*
- The grain is the only thing that can be eaten, and it cannot be left alone with either the fox or the chicken. - So the grain must always be with the farmer whenever any other animal is on the same side.
The sequence above never leaves the grain with the fox or the chicken without the farmer, and after the seventh move all three items are safely on the far side."