I appreciate your time checking it out! I've used and keep using DSPy a lot for work, and I felt I was missing a limb in my Rails-related projects. Let me know if you have any thoughts or feedback, every person has a different perspective and I always learn something new.
Thanks for sharing your experience! I know there's many of us out there dabbling with LLMs and some solid businesess built on Ruby, lurking in the background without publishing much.
Your single-tool approach is a solid starting point. As it grows, you might hit context window limits and find the prompt getting unwieldy. Things like why is this prompt choking on 1.5MB of JSON coming from this other API/Tool?
When you look at systems like Codex CLI, they run at least four separate LLM subsystems: (1) the main agent prompt, (2) a summarizer model that watches the reasoning trace and produces user-facing updates like "Searching for test files...", (3) compaction and (4) a reviewer agent. Each one only sees the context it needs. Like a function with their inputs and outputs. Total tokens stay similar, but signal density per prompt goes up.
DSPy.rb[0] enables this pattern in Ruby: define typed Signatures for each concern, compose them as Modules/Prompting Techniques (simple predictor, CoT, ReAct, CodeAct, your own, ...), and let each maintain its own memory scope. Three articles that show this:
- "Ephemeral Memory Chat"[1] — the Two-Struct pattern (rich storage vs. lean prompt context) plus cost-based routing between cheap and expensive models.
- "Evaluator Loops"[2] — decompose generation from evaluation: a cheap model drafts, a smarter model critiques, each with its own focused signature.
- "Workflow Router"[3] — route requests to the right model based on complexity, only escalate to expensive LLMs when needed.
And since you're already using RubyLLM, the dspy-ruby_llm adapter lets you keep your provider setup while gaining the decomposition benefits.
Thanks for coming to my TED talk. Let me know if you need someone to bounce ideas off.
Maintainer of DSPy.rb here. The key difference is the level of abstraction:
RubyLLM gives you a clean API for LLM calls and tool definitions. You're still writing prompts and managing conversations directly.
DSPy.rb treats prompts as functions with typed signatures. You define inputs/outputs and the framework handles prompt construction, JSON parsing, and structured extraction. Two articles that might help:
1. "Building Your First ReAct Agent" shows how to build tool-using agents with type-safe tool definitions [0].
2. "Building Chat Agents with Ephemeral Memory" demonstrates context engineering patterns (what the LLM sees vs. what you store), cost-based routing between models, and memory management [1].
The article's approach (RubyLLM + single tool) works great for simple cases. DSPy.rb shines when you need to decompose into multiple specialized modules with different concerns. Some examples: separate signatures for classification vs. response generation, each optimized independently with separate context windows and memory to maintain.
Would love to learn how dspy.rb is working for you!
Note that RubyLLM and DSPy.rb aren't mutually exclusive (`gem 'dspy-ruby_llm'`) adapter gives us access to a TON of providers.
I’ve been leaning hard into Sorbet runtime types for DSPy.rb[0] and finding real value. T::Struct at API boundaries, typed props for config, runtime validation where data enters the system.
For generating (with LLMs) API clients and CLIs it’s especially useful—define the shape once, get validation at ingress/egress for free.
You’re right they’re different models. The path would be Falcon + async-job-adapter-active_job. I am exploring eliminating worker processes, so jobs run as fibers in the same process, yielding during I/O.
Both model prompts as functions. BAML is a DSL - write .baml files, generate code, get validated structured outputs.
DSPy is a programming paradigm. I like to look at it like the MVC for the Web. You define Signatures[0]: typed contracts governing the relationship between your models and your app. Signatures model prompts as functions too, without leaving Ruby. Then compose them into modules (Predict, ChainOfThought, ReAct, your own). The framework can automatically optimize prompts based on your metrics.
DSPy.rb brings the DSPy paradigm's tooling (optimizers, evaluation loops) to Ruby. Comes with OpenTelemetry OOTB. It also borrows BAML's schema format for 85% token savings vs JSON Schema in complex signatures. [1]
Everyone is talking about prompt, context, and harness engineering -and I agree they are good ways to frame how to build workflows and agents- this is just programming really.
Nice work. Thanks for sharing it! I've been thinking about using something like this for LLM agent workflows - the outbound action pattern would work well for tool calls that need to wait on external APIs.
I'm working on DSPy.rb [1] and this could pair nicely for multi-step reasoning chains.
I work with DSPy in Python and felt it was missing in the Ruby ecosystem.
So I started https://github.com/vicentereig/dspy.rb: a composable, type-safe version built for Rubyists who want to design and optimize prompts, and reuse LLM pipelines without leaving their language of choice. Working with DSPy::Signatures reminds me a bit of designing a db schema with an ORM.
It’s still early, but it already lets you define structured modules, instrument them in Langfuse, wire them up like functional components, and experiment with signature optimization. All in plain Ruby.