Also this seems plain wrong. Input token caching has now idea whether you @include the file or copy the contents into the prompt. That is handled entirely by opencode and, all else being equal, has no bearing on the cache ability of a trace.
> Our cache hit rate sits at 85.7%, which saves us an estimated five figures compared to what we would pay at full input token pricing. This is partially thanks to the shared context file optimisation — sub-reviewers reading from a cached context file rather than each getting their own copy of the MR metadata, but also by using the exact same base prompts across all runs, across all merge requests.
This blog post is full of small inconsistencies that make it read like a low quality SEO piece.
> We also extract a shared context file (shared-mr-context.txt) from the coordinator's prompt and write it to disk. Sub-reviewers read this file instead of having the full MR context duplicated in each of their prompts. This was a deliberate decision, as duplicating even a moderately-sized MR context across seven concurrent reviewers would multiply our token costs by 7x.
No, it would not, because neither is the prompt of the subagent 100% of its token usage, nor will the "shared-mr-context.txt" which is then being read have a size of zero compared to the creation of this shared context.
> You don't need seven concurrent AI agents burning Opus-tier tokens to review a one-line typo fix in a README.
Yeah, well you wouldn't have anyways. Earlier in the post it says that Opus is "exclusively for the Review Coordinator".
Wouldnt the margin be higher? All other models being moved from unquantized to quantized would lower their performance, while bonsai stays. I get what you see if it was in regards to score/modelsize, but not for absolute performance
I definitely do this, along with the compulsion sometimes to tell the agent how a problem was fixed in the end, when investigating myself after the model failing to do so. Just common courtesy after working on something together. Let’s rationalize this as giving me an opportunity to reflect and rubberduck the solution.
Regarding not just telling „try again“: of course you are right to suggest that applying human cognition mechanisms to llm is not founded on the same underlying effects.
But due to the nature of training and finetuning/rf I don’t think it is unreasonable that instructing to do backwards reflection could have a positive effect. The model might pattern match this with and then exhibit a few positive behaviors. It could lead it doing more reflection within the reasoning blocks and catch errors before answering, which is what you want. These will have attention to the question of „what caused you to make this assumption“, also, encouraging this behavior. Yes, both mechanisms are exhibited through linear forward going statical interpolation, but the concept of reasoning has proven that this is an effective strategy to arrive at a more grounded result than answering right away.
Lastly, back to anthro. it shows that you, the user, is encouraging of deeper thought an self corrections. The model does not have psychological safety mechanisms which it guards, but again, the way the models are trained causes them to emulate them. The RF primes the model for certain behavior, I.e. arriving at answer at somepoint, rather than thinking for a long time. I think it fair to assume that by „setting the stage“ it is possible to influence what parts of the RL activate.
While role-based prompting is not that important anymore, I think the system prompts of the big coding agents still have it, suggesting some, if slight advantage, of putting the model in the right frame of mind. Again, very sorry for that last part, but anthro. does seem to be a useful analogy for a lot of concepts we are seeing (the reason for this being in the more far of epistemological and philosophical regions, both on the side of the models and us)
Would staying at an LTS version instead of running my production workloads on the bleeding edge also be free-riding, because I am depriving the community of my testing?
Getting to know the views and values of your date is not a weird thing to do on the first date. If it’s a question that annoys them, they should consider why.
I like it, putting established knowledge in skill form is not difficult or great innovation, but definitely useful and worthy to share. I would critique one or two things: For skills like this, which are intended to be shared and but at the same time not complete "frameworks", I think skills should be atomic. For this i would split it into the technical writing guidelines (tone, grammar, etc) and the workflow (drafts, review, the specific folders).
Agents should be able to discover and load both without a problem, but it makes it easier for other people to pick-and-chose.
Secondly, I'd combine one/two into one markdown file each. I don't want the reviewer / writer to selectively read those, since a review always needs to apply all of them. I get that the goal of scoping the review procedure into 10 individual steps is to create more focus on each task by giving it its own procedure step, but in my experience doing small focused steps like that will lead to much longer review times and worst case a very fragmented text, because small edits are applied on top of each other, without considering the big picture. A recent LLM with sufficient reasoning should be able to apply all rules in one go.
I have been using parakeet TDT v3 with just 0.6B params and its insanely fast (feels instant, even on M1 Air). The accuracy is all I could ask for - I dont see the benefit of a much larger 4B model?
Not knocking your app, but asking before your app seems very focused on one model, while others allow the user to pick according to their needs.
In legal and public opinion distributions and authorship might not be looked at with such a technical lens, especially in a country trying to ban encrypted communications. A muddying between the two could easily be constructed intentionally, or unintentionally by ignorance of executive and judicial powers.
In Chapter D.7 they describe: "The complex reflection in water is interpreted by the network as a distant mountain, therefore the water surface is broken."
This is really interesting to me because the model would have to encode the reflection as both the depth of the reflecting surface (for texture, scattering etc) as well as the "real depth" of the reflected object. The examples in Figure 11 and 12 already look amazing.
While the situation in Europe is, of course, diverse, the culture in Germany (which is often looked to as a thought leader in the eu) is strictly to disallow and discredit any substantial criticism of the „only democracy in the Middle East“. Usually under a blanket accusation of antisemitism. Especially due to germanys history, this claim can be brought forward and be spread in a news cycle without substantive scrutiny. The fact that conflating the State of Israel with jewish people could be met with the same accusation is largely ignored.
If you are interested in the topic on a high level I suggest the following starting points
United Nations: „UN experts urge Germany to halt criminalisation and police violence against Palestinian solidarity activism“ as well as numerous statements by amnesty international or the Wikipedia Section „Restrictions on Pro-Palestinian expression“ of the article „Censorship in Germany“
In the post they described that they observed errors happening in their testing env, but decided to ignore because they were rolling out a security fix. I am sure there is more nuance to this, but I don’t know whether that makes it better or worse
> Our cache hit rate sits at 85.7%, which saves us an estimated five figures compared to what we would pay at full input token pricing. This is partially thanks to the shared context file optimisation — sub-reviewers reading from a cached context file rather than each getting their own copy of the MR metadata, but also by using the exact same base prompts across all runs, across all merge requests.