Crunchy Bridge's Ruby Back End: Sorbet, Tapioca, and Parlour(crunchydata.com)
crunchydata.com
Crunchy Bridge's Ruby Back End: Sorbet, Tapioca, and Parlour
https://www.crunchydata.com/blog/crunchy-bridges-ruby-backend-sorbet-tapioca-and-parlour-generated-type-stubs
28 comments
> I've worked in some big Rails/Ruby code bases and usually any mess was the result of someone who didn't know the language writing a bunch of code alone with little to no code review or guidance.
This! 100% This is why code review and strict linters( i.e. Rubocop) are an absolute must for any large ruby code base.
This! 100% This is why code review and strict linters( i.e. Rubocop) are an absolute must for any large ruby code base.
Yeah, that sort of thing is always nice. I think just generally being aware of these issues and having a Ruby person around goes a long way.
Ruby code based that I’ve worked on that were a mess had nothing to do with review or guidance and everything to do with delivering as quickly as possible.
It’s inherently a part of Ruby, a feature one might say, that you can probably spin up a full functioning web application from Front End to Database in hours. This type of speed setup and launch inhibits not even thinking through design principles or other people coming back to read or work on the code.
Furthermore, because everything can be an object you end up writing boiler plate checks for nil values all over the place or don’t and ignore those cases that later crash the entire process running.
Most of these issues are not prevalent in strongly typed languages because, other than custom objects, what you define is what you get.
If speed is your only priority, say a startup, I would never just go throwing untyped Ruby code around because it’s gonna cost you in the long run trying to get others to work on your code.
It’s inherently a part of Ruby, a feature one might say, that you can probably spin up a full functioning web application from Front End to Database in hours. This type of speed setup and launch inhibits not even thinking through design principles or other people coming back to read or work on the code.
Furthermore, because everything can be an object you end up writing boiler plate checks for nil values all over the place or don’t and ignore those cases that later crash the entire process running.
Most of these issues are not prevalent in strongly typed languages because, other than custom objects, what you define is what you get.
If speed is your only priority, say a startup, I would never just go throwing untyped Ruby code around because it’s gonna cost you in the long run trying to get others to work on your code.
I agree that in a lot of cases, moving fast is what causes the mess, and I see that as a tradeoff sometimes.
I've written a LOT of ruby over the years, and honestly bugs related to objects being the wrong type were in the minority. Yes, something popping up as nil in a weird place did happen, but not that often and it's usually easy to fix.
I've written a LOT of ruby over the years, and honestly bugs related to objects being the wrong type were in the minority. Yes, something popping up as nil in a weird place did happen, but not that often and it's usually easy to fix.
Also I’ll always hate ActiveAdmin for the rest of my life.
> easy to write Ruby code and it less you do almost anything
Types do help in that this would be visible as it's happening with the parameter list and allowed types list getting longer and longer or uses of untyped.
A similar thing with highly overloaded types could happen in type-inferred languages but compile errors catch when expectations don't match.
Types do help in that this would be visible as it's happening with the parameter list and allowed types list getting longer and longer or uses of untyped.
A similar thing with highly overloaded types could happen in type-inferred languages but compile errors catch when expectations don't match.
I would definitely agree in a typed language, and I see how identifying this "smell" early could be nice in Ruby. However you can do that with validations too and get basically the same results. If your validations are overflowing the page, maybe you're doing too much.
For something like Ruby types feel bolted on and for the trouble I'm not sure the payoff is worth it personally.
But this is all just my opinion, clearly a lot of smart folks a Shopify disagree.
For something like Ruby types feel bolted on and for the trouble I'm not sure the payoff is worth it personally.
But this is all just my opinion, clearly a lot of smart folks a Shopify disagree.
It is bolted on, and personally I might not use it either. But at scale it makes a big difference.
The other difference is that I can not add a validation and the code passes existing tests and runs. Typecheck won't pass without making the messy longer type sig or writing the untyped smell.
The other difference is that I can not add a validation and the code passes existing tests and runs. Typecheck won't pass without making the messy longer type sig or writing the untyped smell.
I always hear the at scale claim, but have never seen it even in larger code based. The academic literature also seems far from conclusive. I get why people like types, but I think they’re a bit oversold.
The literature I read was about comparable defect rates between different programming languages producing comparable functionality.
I hadn't read anything about productivity of different languages and team sizes--if you have any references I'd be interested in reading them.
One story I did read was about how Twitter switched to Scala (and later Java IIRC) mainly for the static typing and not for performance or other reasons.
I don't understand the not seeing it in a large codebase point. I work on a large Ruby code base that started without type annotations. Since I joined, we're now all adding type annotations for everyone's benefit, so they definitely exist. The reason for not wholly rewriting such a large codebase in a static typed language should generally be considered a no go. Many devs complained at the time and now most (if not all, I don't know of exceptions) now appreciate having the type checking and visible annotations for reading. I don't work on front-end but I expect that there was a some of that too in the React TypeScript codebases.
I hadn't read anything about productivity of different languages and team sizes--if you have any references I'd be interested in reading them.
One story I did read was about how Twitter switched to Scala (and later Java IIRC) mainly for the static typing and not for performance or other reasons.
I don't understand the not seeing it in a large codebase point. I work on a large Ruby code base that started without type annotations. Since I joined, we're now all adding type annotations for everyone's benefit, so they definitely exist. The reason for not wholly rewriting such a large codebase in a static typed language should generally be considered a no go. Many devs complained at the time and now most (if not all, I don't know of exceptions) now appreciate having the type checking and visible annotations for reading. I don't work on front-end but I expect that there was a some of that too in the React TypeScript codebases.
[deleted]
Im not sure if Crystal had hit it's 1.0 release by the time they selected ruby for this project, other than a ruby-like repl, it seems like it hits all the marks
We use Crystal a lot too at Crunchy Bridge. It is used for all of the on-instance programs, the per-region monitoring service, as well as the customer CLI. I'm a big fan of Crystal, and wrote the postgres driver for the language :)
However a main reason we used Ruby was for the production REPL which lets us manage our entire server fleet with like 1/3 of the staff we'd need otherwise.
However a main reason we used Ruby was for the production REPL which lets us manage our entire server fleet with like 1/3 of the staff we'd need otherwise.
Amen to that. We reached $2M ARR as 3 engineers at Missive [1]. As the CTO, I can tell life wouldn’t quite be the same without the ability to save the day in the production console at the speed of light.
[1] https://missiveapp.com/
[1] https://missiveapp.com/
> production REPL which lets us manage our entire server fleet with like 1/3 of the staff
I'm intrigued. Are you talking about the `rails console` REPL? How did that allow you to manage your fleet with 1/3 staff?
I'm intrigued. Are you talking about the `rails console` REPL? How did that allow you to manage your fleet with 1/3 staff?
thank willl.
When I investigate in Crystal I found your github and your Posgres driver.
When I investigate in Crystal I found your github and your Posgres driver.
I've said this before (and been downvoted for it). Crystal is such a phenomenal language in that it brings a HEAVY spirit of Ruby (some Ruby stuff even runs unmodified) yet has wonderful type-safety. It's also compiled and relatively fast. It seems like such a win/win.
I work on Crunchy Bridge. Mostly, the Ruby parts discussed.
I personally find Sorbet more marginal in its benefits than perhaps conveyed here, but the Parlour-Tapioca additions are key parts of making it practical. I'd recommend those to anyone looking to use Sorbet. Sorbet has been moderately expensive to implement and maintain, definitely not entirely a set-and-forget piece of infrastructure (compare: Sequel, which is), and periodically someone has to re-familiarize themselves with the details to address changes in Sorbet or Tapioca (somewhat less often, these days, perhaps?).
It also has some unfortunate interaction (for load performance) with the lazy loading in Zeitwerk (also a good program), since you will tend to load files that you refer to types in only. And you can't really use Dependabot or similar if you need an rbi generation step, so someone needed to write a custom github action. Plus we needed to stumble around a bit with rbi generation (parlour and tapioca, which this post hopefully invites you to use). Also, the stack traces, as instrumented by the sorbet runtime, are pretty ugly, and I get to filter through those every single time. And it broke pry's show-source, which I've had to learn to live without. Code reloading is also a lot more busted than it otherwise needs to be. This kind of friction definitely adds up.
My favorite part of sorbet is catching unbound symbol typos extremely quickly, often simple ones local to a file. That would probably be the 80/20 for me, if a simpler program to replace Sorbet's value to me were to be integrated.
Probably people who really like LSP-style features will have a more positive assessment of sorbet than I convey here. Though I have lsp enabled, I don't find most of its features amazingly compelling, so, perhaps I am the odd one. I am perhaps overly influenced by xcscope.el as close to my ideal. Emacs's projectile, ivy, and counsel projects are a partial substitute, along with the hippie-expand feature.
In my opinion, the 100% branch coverage briefly mentioned has more to do with the tenor of developing on the project. This was an adapted practice from Sequel and Roda, which are Jeremy Evans projects, which I have long admired in their methodology: Evans makes a huge scope of features look so natural to write and maintain, and I think the test coverage is part of that. My previous work at Citus Cloud also implemented a similar technique, based on line coverage, so I've been doing something like this for a number of years.
Brandur, who wrote this, has more experience with large, fluctuating Ruby code bases that slide in cogency. I tend to think that the high coverage requirements carries more weight in how I experience daily development life than Sorbet does.
I personally find Sorbet more marginal in its benefits than perhaps conveyed here, but the Parlour-Tapioca additions are key parts of making it practical. I'd recommend those to anyone looking to use Sorbet. Sorbet has been moderately expensive to implement and maintain, definitely not entirely a set-and-forget piece of infrastructure (compare: Sequel, which is), and periodically someone has to re-familiarize themselves with the details to address changes in Sorbet or Tapioca (somewhat less often, these days, perhaps?).
It also has some unfortunate interaction (for load performance) with the lazy loading in Zeitwerk (also a good program), since you will tend to load files that you refer to types in only. And you can't really use Dependabot or similar if you need an rbi generation step, so someone needed to write a custom github action. Plus we needed to stumble around a bit with rbi generation (parlour and tapioca, which this post hopefully invites you to use). Also, the stack traces, as instrumented by the sorbet runtime, are pretty ugly, and I get to filter through those every single time. And it broke pry's show-source, which I've had to learn to live without. Code reloading is also a lot more busted than it otherwise needs to be. This kind of friction definitely adds up.
My favorite part of sorbet is catching unbound symbol typos extremely quickly, often simple ones local to a file. That would probably be the 80/20 for me, if a simpler program to replace Sorbet's value to me were to be integrated.
Probably people who really like LSP-style features will have a more positive assessment of sorbet than I convey here. Though I have lsp enabled, I don't find most of its features amazingly compelling, so, perhaps I am the odd one. I am perhaps overly influenced by xcscope.el as close to my ideal. Emacs's projectile, ivy, and counsel projects are a partial substitute, along with the hippie-expand feature.
In my opinion, the 100% branch coverage briefly mentioned has more to do with the tenor of developing on the project. This was an adapted practice from Sequel and Roda, which are Jeremy Evans projects, which I have long admired in their methodology: Evans makes a huge scope of features look so natural to write and maintain, and I think the test coverage is part of that. My previous work at Citus Cloud also implemented a similar technique, based on line coverage, so I've been doing something like this for a number of years.
Brandur, who wrote this, has more experience with large, fluctuating Ruby code bases that slide in cogency. I tend to think that the high coverage requirements carries more weight in how I experience daily development life than Sorbet does.
Crunchy Bridge sounds like a new Intel chipset name.
Something I've always wondered is if there's any reason to have so many different typing engines. TypeScript, for example, seems feature complete enough to provide a mechanism to type any language. (I'm almost sure this can be proven.)
The real challenge instead would be to design the syntax extensions for typing so they feel natural.
`sig { returns(Blah) }` feels ugly.
The real challenge instead would be to design the syntax extensions for typing so they feel natural.
`sig { returns(Blah) }` feels ugly.
Infer kind of does this, it has multiple frontends and provides linting capabilities
https://fbinfer.com
https://fbinfer.com
Big fan of Sequel. Haven’t ever tried out the type checking stuff for Ruby but I might give it a look after reading this.
Doesn't the article make the whole thing sound like an incredibly painful experience?
Sorbet is really good. Makes refactoring really easy.
Ruby has a lot of ways to express the type of information you're passing around. The problem is when you don't validate things coming from users, APIs, and the database, or if you muddy you state in some overburdened god-object. I think the problem is that its so easy to write Ruby code and it less you do almost anything you want, so it is very easy to make a mess. Types may or may not help.
I've worked in some big Rails/Ruby code bases and usually any mess was the result of someone who didn't know the language writing a bunch of code alone with little to no code review or guidance.