Thanks! I agree with your sentiment for a lot of basic mundane tasks, but there are a number of tasks that exist today that are very high value yet still mundane and requires manual work.
Examples include form filling, sales prospecting, lead enrichment, or even just keeping track of prices of important things.
Over time, we do expect the cost of tokens on these models to decrease drastically. Powerful vision models are still relatively new compared to other generic LLM models for text. Definitely a lot of room for optimizations that we expect will come quickly!
For the WebArena result, we actually used a mixture of models checking each other's work and evaluating in real time. We found the verifications to be really effective in producing accurate results. Feel free to take a look at our architectural blog post to learn more in detail: https://blog.withmeka.com/introducing-meka-an-open-source-fr...
Unfortunately, we didn't try it out with open source models, but you are welcome to pull the repo and try with any model that has good visual grounding! (I heard UI-TARS and the latest Qwen visual model are quite good)
We explored using a containerized VM that exposed agentic controls in the open source version, but generally found that the cloud-based solutions were much faster to get started and easier to work with. Our repo contains adapters that work with several of the most popular cloud-hosted VM-as-a-service infra providers.
Definitely would be happy to be wrong and missed something here!
IMO, the combination of having an "evaluator model" at the end to verify if the intent of the task was complete, and using multiple models that look over each other's work in every step was helpful - lots of human organization analogies there, like "trust but verify" and pair programming. Memory management was also very key.
Hi there, I'm Edward, one of the co-founders. The OS that the agent operates in is a fresh confined environment, and not a company or personal computer.
Looks pretty neat, but does this support sites with HTTPS or certificate pinning? A simple proxy is easy, but that's been the annoying part in the past
After an Egyptian protest in internet petition form, I'm glad the Canadian government had some sense in them to hold on to a future in tech innovation in Canada.
I can only imagine what kind of immediate effect this would have had on tech startups.
Examples include form filling, sales prospecting, lead enrichment, or even just keeping track of prices of important things.
Over time, we do expect the cost of tokens on these models to decrease drastically. Powerful vision models are still relatively new compared to other generic LLM models for text. Definitely a lot of room for optimizations that we expect will come quickly!