You may have heard of Programmatic Tool Calling from anthropic or Code Mode from cloudflare for calling mcp servers. I have been working on an open source version of this and recently put together some basic docs, and thought this page would be a good one to share.
There are plenty of other features to explore, but this one alone has resulted in significant improvements in the speed of execution in many tasks including minecraft playing agents, computer use agents, and for querying and aggregating information from my telemetry systems in a lightweight low risk way.
have built in this space which led me to develop a minizinc mcp server [0] for scheduling bocce tournaments [1]. scheduling with constraints is a np hard problem and it makes sense people struggle. tools exist to solve this problem but they are complex and hard to use for non technical folks, and even technical folks. am hoping a tool like this can bridge the gap and would like to bring it to your awareness if you aren't already thinking about the problem this way :)
edit: after reading a bit more of description looks like yall are taking a similar approach, kudos!
as an infrastructure engineer the idea of being able to train computer use agents without provisioning infrastructure sounds amazing!
a common use case i run into is i want to be able to configure corporate vpn software on windows machines. is there a link for a getting started guide i could try this out with?
ran into this when writing agents to fix unit tests. often times they would just give up early so i started writing the verifiers directly into the agent's control flow and this produced much more reliable results. i believe claude code has hooks that do something similar as well.
From 1:14:55-1:15:20, within the span of 25 seconds, the way Demis spoke about releasing all known sequences without a shred of doubt was so amazing to see. There wasn't a single second where he worried about the business side of it (profits, earnings, shareholders, investors) —he just knew it had to be open source for the betterment of the world.
Gave me goosebumps. I watched that on repeat for more than 10 times.
I opt for the greedy strategy in most game play scenarios for pretty much the reasons you described here. I was considering making a mod to perform this action for me and was looking for a more "correct" solution but greedy is way simpler and just as effective for most cases.