Show HN: Ocarina – Automate and test MCP servers from YAML, no LLM(github.com)

2 pointsby msradam14 hari yang lalu1 comments

Show HN: Ocarina – Automate and test MCP servers from YAML, no LLM

https://github.com/msradam/ocarina

Hi all. As someone who has spent years working with Ansible and other automation frameworks, the recent MCP boom has me fascinated. People are creating nice, typed, LLM-readable (and thus human-readable) interfaces for their servers, over a standardized protocol that exposes both tools and resources. I wanted to see if I could create a way to run scripts against these servers directly, with no AI in the loop.

Ocarina lets you inspect MCP server characteristics and write Rondos (the equivalent of Ansible playbooks) that express tool calls in yaml so you can play them step-by-step without needing to drive with an LLM.

Here's what a Rondo looks like:

  keys:
    owner: acme
    repo: api
  server:
    command: npx
    args: [-y, "@modelcontextprotocol/server-github"]
  rondo:
    - name: recent commits
      tool: list_commits
      args:
        owner: "{{owner}}"
        repo: "{{repo}}"
      grab: ".0.sha"
      echo: latest_sha
    - name: commit detail
      tool: get_commit
      args:
        owner: "{{owner}}"
        repo: "{{repo}}"
        sha: "{{latest_sha}}"
      expect:
        contains: "feat"

I have started using Ocarina myself to test and validate the MCP servers I am creating at my job, and I am interested in the broader field of how we can maximize the use of the massive MCP ecosystem. The README includes links to repos you can clone and try right away (and a little Blender demo ;])

Happy Saturday!

3 comments

kerlenton12 hari yang lalu

The deterministic MCP server testing without any LLM in the loop is actually pretty useful for CI. However, the issue that I run into consistently is a level above: the server can do just fine with all the scripted calls but fail in real practice as the model does not even try to use the tool or uses it with some unexpected arguments. This is an issue of behavior that cannot really be described using the YAML file as you write the call yourself.

Are there any means to create cases from real sessions or everything is written by hand?

msradam10 hari yang lalu

Dworf13 hari yang lalu