Vessel primarily uses DOM content analysis to identify interactive elements on the page which then indicates which sort of tools might be relevant to surface to the agent. URL pattern is the obvious next indicator, and prior agent action is where it gets REALLY interesting imo because that's where you start building a predictive model of what the agent needs next rather than just reacting to whatever the current state is.
Your schema work sounds pretty interesting - any links? I'd be curious to check it out!
This is a really good question; currently Vessel uses a flat registry, but the design is specifically oriented toward solving the hallucination problem.
The way that I manage that in Vessel is contextual surfacing. This way, the model doesn't necessarily have 40+ tools to choose from at any given time, but rather is focused toward a subset of tools that are applicable given the current page context.
Speed and efficiency are my number 1 priority with this browser so this framework may change/shift as time goes on, but this is an approach that I'm particularly interested in exploring.
Your schema work sounds pretty interesting - any links? I'd be curious to check it out!