My team works on automatic environment generation for RL post-training. One of our projects is using coding agents to build web clones for BUAs/CUAs.
We tested Gemini, Claude Code, GLM, and Codex using our harness on their abilities to recreate a Slack workspace and benchmarked their performance.
Saw a variety of results:
- *Gemini 3 Pro:* Achieved the highest visual score (0.91 SSIM) but lacked interactive functionality.
- *Claude Opus 4.6:* Developed the most complete application, balancing full interactivity with consistent self-correction.
- *GLM-5:* Produced the best code architecture but reached a plateau in visual improvement.
- *GPT-5.3 Codex:* Initialized quickly but entered a five-hour "scaling spiral" that failed to yield further progress.
Next, we’re planning:
- More web apps for cloning and benchmarking across the models
- More functionality (the trajectory didn’t include full Slack features)
- Better scoring for functionality (easier to catch Gemini’s mistake)
After doing few experiments, I think that having Agents work on browser for all tasks wouldn't be best due to many factors like token cost, safety, etc. But browser/computer can be a tool that the agent can be alongside MCPs to complete tasks that requires interaction with such modalities.
Most current web agent benchmarks focus on single-tab tasks (e.g., 'go to Gmail and star this email'). We found that frontier models that score highly on those tasks (like in WebArena) often fall apart when they have to coordinate context across 2+ applications. We built a simulated environment with scenarios and deterministic verifiers to see why.
It's an interesting article and I agree with some points you brought up here. But here are some of them to which I don't agree to
1. Evals are used throughout the article in the sense of LLM benchmarking, but this is not the point. One could effectively evaluate any AI system by building custom evals.
2. The purpose of evals is to help devs systematically improve their AI systems (at least how we look at it) not any of the ones listed in your article. It's not a one-time thing, it's a practice like the scientific method.
We tested Gemini, Claude Code, GLM, and Codex using our harness on their abilities to recreate a Slack workspace and benchmarked their performance.
Saw a variety of results:
- *Gemini 3 Pro:* Achieved the highest visual score (0.91 SSIM) but lacked interactive functionality. - *Claude Opus 4.6:* Developed the most complete application, balancing full interactivity with consistent self-correction. - *GLM-5:* Produced the best code architecture but reached a plateau in visual improvement. - *GPT-5.3 Codex:* Initialized quickly but entered a five-hour "scaling spiral" that failed to yield further progress.
Next, we’re planning:
- More web apps for cloning and benchmarking across the models - More functionality (the trajectory didn’t include full Slack features) - Better scoring for functionality (easier to catch Gemini’s mistake)
Repo: https://github.com/vibrantlabsai/cloning-bench
Blog post: https://vibrantlabs.com/blog/pa-bench