I took this product for a spin and it's remarkably accurate considering it's a new product. I see a lot of opportunity for this to fly off the shelf for builders.
I'm Ben — I built UCP Playground (https://ucpplayground.com), the testing tool mentioned in the post. I can confirm the Saturday night debugging session. We were chatting while patching things live on both sides for hours.
A few things I can add from the Playground side:
On schema quality > model capability: This is the biggest takeaway from 180+ sessions across 11 models. We have a leaderboard tracking completion rates per model, and the variance between stores is larger than the variance between models. A clean store schema makes even weaker models succeed. A messy schema makes even Claude and GPT fail. The store is the bottleneck, not the LLM.
On variant resolution: The agents that skip get_product_details and try to cart directly almost always fail. They guess variant IDs from search results, hit type errors, and drop off. Llama 3.3 70B's higher success rate comes down to it reliably calling the details endpoint first — it follows the tool sequence more faithfully than models that try to shortcut.
On the three transports: UCP Playground tests MCP, REST, and ECP (embedded checkout via iframe). The REST fallback turned out to be critical — MCP connections to WooCommerce stores can be flaky (OOM on unbounded queries, rate limits), so the agent silently falls back to REST for that tool call and continues. Without that, a lot of sessions would just die at search.
On store instructions: One thing we surface in Playground that merchants don't always realize — stores can inject response_instructions into MCP tool responses. These are behavioral prompts that tell the agent how to present products. We extract and display them per tool call. It's worth auditing what your store is telling agents to do.
If anyone wants to see what a real agent session looks like, Playground lets you share replays as public links. Happy to answer questions about the testing/observability side.
Spot on. I see UCP manifests as the "Trust Contract" that defines what is possible, but you're right—contract fulfillment in a non-deterministic environment is where things get messy.
My goal with UCP Checker is to solve the first-order problem: "Is this even a valid endpoint?" You're describing the critical second-order problem: preventing an agent from being hijacked via Indirect Prompt Injection once it actually fetches that content.
I’ve been thinking about this separation of concerns a lot. Ideally, we need a layered approach:
Static Layer (UCP Checker): Validates the schema, capabilities, and reachability.
Runtime Layer: A proxy or sidecar that scans fetched content for "ignore previous instructions" patterns or credential exfiltration attempts before the LLM processes it.
I’d love to hear if you think that "Runtime Guardrail" should live on the merchant side (e.g., a "UCP Shield" gateway) or if it's strictly the responsibility of the Agent/Model provider to sanitize inputs?
I’ve been frustrated by how opaque agentic commerce is right now. Shopify and Google launched the Universal Commerce Protocol (UCP), but for a developer, it's often a black box in production. You might have your manifest at /.well-known/ucp, but is Gemini actually reading it? Or is it getting silently dropped by a legacy firewall rule?
I built this extension as a lightweight debugger for my own agentic workflows. It’s like Waze for UCP—as we browse, the community is crowdsourcing a map of agent-ready domains. We’ve verified 508 domains so far.
Key features for devs:
Zero-Click Workflow: Uses badge icons to show status instantly (Green = ready, Gray = not detected).
Instant Manifest Debugging: Surfaces HTTP status, response times, and version metadata for UCP manifests.
Gemini Readiness: Checks specifically if the Gemini agent (and others like GPTBot) can reach the store or if they are blocked by robots.txt/WAF rules.
Privacy-First: No tracking, no data selling, and minimal permissions (activeTab only).
You can try it out without an account or signup. I'd love to hear what other signals you'd want to see in the debugger, or if keeping it this lightweight is best!