Finding relationships between Ruby’s top packages and their dependencies(shopify.engineering)
shopify.engineering
Finding relationships between Ruby’s top packages and their dependencies
https://shopify.engineering/relationships-between-ruby-top-100-packages-dependencies
47 comments
I'm pretty convinced Ruby and Python's package ecosystem has most of the same problems of JavaScript's but it's just not talked about as much. This article definitely has a lot of similarities to npm's MFA implementation and security concerns that have come up.
Not true at all, at least for Ruby.
Most larger React projects have _far more_ dependencies_ than an equivalent Ruby project. You could chalk it up to a few things I think:
* the JS ecosystem is just much larger - more devs (especially inexperienced ones) means more code and more copy/pasting solutions that rely on `npm install` from the web.
* JS has just not been as stable and mature as Ruby or Python until the past couple of years (thanks to ES5/ES6), especially if you consider the browser acceptance of latest ES.
* The general explosion of NPM packages that do dumb, trivial shit - see https://www.npmjs.com/package/is-odd and https://www.davidhaney.io/npm-left-pad-have-we-forgotten-how... for two examples.
* There are at least four major tools to manage dependencies in JS: npm, pnpm, yarn, and yarn 2. This often means you end up with multiple ways teams are handling JS deps in their projects, with all the non-essential complexity that comes with it. In Ruby there is just one way - Bundler. I don't even want to talk about python.
* There are more leading frameworks to choose from in JS, which means you have libraries that support all of them, doing things in slightly different ways. Consider a legacy Angular app that updated to React in 2018 and is updating to Svelte or whatever now. If the team hasn't been very disciplined in updating and removing all transitive dependencies through the upgrades, you will inevitably end up with a ton of stuff (is `react-date-wrangler-jobber` used? I still see references to it, but is that page live? who knows!?) in your bundle hanging around and cluttering up your build.
I've been updating things across our main React/Rails product at $current_job. Most major dependencies were out of date by 1-2 years. Updating Ruby has been *so much* easier, whereas the React / npm stuff often ends up in a rats nest of conflicting dependencies. And this is after we made a major effort in our build and packaging setup across the board.
I do think it is getting better in the npm world, slowly. Ever so slowly.
Most larger React projects have _far more_ dependencies_ than an equivalent Ruby project. You could chalk it up to a few things I think:
* the JS ecosystem is just much larger - more devs (especially inexperienced ones) means more code and more copy/pasting solutions that rely on `npm install` from the web.
* JS has just not been as stable and mature as Ruby or Python until the past couple of years (thanks to ES5/ES6), especially if you consider the browser acceptance of latest ES.
* The general explosion of NPM packages that do dumb, trivial shit - see https://www.npmjs.com/package/is-odd and https://www.davidhaney.io/npm-left-pad-have-we-forgotten-how... for two examples.
* There are at least four major tools to manage dependencies in JS: npm, pnpm, yarn, and yarn 2. This often means you end up with multiple ways teams are handling JS deps in their projects, with all the non-essential complexity that comes with it. In Ruby there is just one way - Bundler. I don't even want to talk about python.
* There are more leading frameworks to choose from in JS, which means you have libraries that support all of them, doing things in slightly different ways. Consider a legacy Angular app that updated to React in 2018 and is updating to Svelte or whatever now. If the team hasn't been very disciplined in updating and removing all transitive dependencies through the upgrades, you will inevitably end up with a ton of stuff (is `react-date-wrangler-jobber` used? I still see references to it, but is that page live? who knows!?) in your bundle hanging around and cluttering up your build.
I've been updating things across our main React/Rails product at $current_job. Most major dependencies were out of date by 1-2 years. Updating Ruby has been *so much* easier, whereas the React / npm stuff often ends up in a rats nest of conflicting dependencies. And this is after we made a major effort in our build and packaging setup across the board.
I do think it is getting better in the npm world, slowly. Ever so slowly.
My opinion is that the standard library of JS is so underwhelming that a high % of installs on NPM are for things that the standard library _should_ give you, but doesn't. Date/time, currency, csv, arbitrary precision decimals, int/bigint math, the entirety of lodash, etc. Give JS a fitting standard library, and I'd bet you see the dependency trees shrink by a factor of 10.
> If the team hasn't been very disciplined in updating and removing all transitive dependencies through the upgrades, you will inevitably end up with a ton of stuff (is `react-date-wrangler-jobber` used? I still see references to it, but is that page live? who knows!?) in your bundle hanging around and cluttering up your build.
Ah yep - last week I removed 25 (!) completely unused libraries from a package.json file.
> the React / npm stuff often ends up in a rats nest of conflicting dependencies
At a previous job we started with Webpack 3, right before Webpack 4 was released, and even months later a bunch of stuff still didn't work with Webpack 4.
Ah yep - last week I removed 25 (!) completely unused libraries from a package.json file.
> the React / npm stuff often ends up in a rats nest of conflicting dependencies
At a previous job we started with Webpack 3, right before Webpack 4 was released, and even months later a bunch of stuff still didn't work with Webpack 4.
> * JS has just not been as stable and mature as Ruby or Python until the past couple of years (thanks to ES5/ES6), especially if you consider the browser acceptance of latest ES.
+1. I see way more of my js packages deprecated and renamed than ruby. There hasn't been a huge backwards compatibility issue in ruby since like ~2.7 (ruby 3 and 3.1 were easy), but hunting down which version of node the npm package works on is a huge pain.
+1. I see way more of my js packages deprecated and renamed than ruby. There hasn't been a huge backwards compatibility issue in ruby since like ~2.7 (ruby 3 and 3.1 were easy), but hunting down which version of node the npm package works on is a huge pain.
As it happens, folks from RubyGems, PyPI, npm and others meet regularly to discuss common problems and share ideas: https://github.com/ossf/wg-securing-software-repos
That's not very surprising. I think what's surprising is the perception of RubyGems, PyPI, etc is much better than npm's when there are common problems. There's an element of suffering from success but some of the problems have been around in RubyGems and PyPI before Node/npm took off and those were seen as the hot technologies.
My experience is that the average npm-based app has significantly more (and smaller) dependencies than a similarly-sized Ruby or Python app, even before factoring in that npm permits using multiple versions of the same dependency at once. This isn't necessarily a bad thing, but it results in a somewhat different set of issues.
I'd suggest there's a correlation between the size of those package repositories and how much they get talked about. npm has several times the package count of PyPI and RubyGems together, and I don't think that can be accounted for just because it's common to make "micro" JavaScript packages.
I am the person screenshotted who was confidently wrong. It was definitely a useful exercise, in my opinion.
[deleted]
Aside from Shopify and others from that era, what modern SaaS offerings are choosing Rails (or Ruby) for their main app? I had one last peer still using Rails, and since Heroku's been on the ropes they're out.
Other than DHH adherents, is anybody still excited about Rails? To draw an even smaller circle, is anybody still excited about Ruby when Rails isn't involved?
Other than DHH adherents, is anybody still excited about Rails? To draw an even smaller circle, is anybody still excited about Ruby when Rails isn't involved?
> is anybody still excited about Ruby when Rails isn't involved?
Far more excited than I am about JS when the browser isn't involved.
> what modern SaaS offerings are choosing Rails
Github, Airbnb, Shopify, Constant Contact, SlideShare, CrunchBase, Bloomberg, Yammer, GitLab, ZenDesk, Hashicorp, Hulu, Netflix.
Far more excited than I am about JS when the browser isn't involved.
> what modern SaaS offerings are choosing Rails
Github, Airbnb, Shopify, Constant Contact, SlideShare, CrunchBase, Bloomberg, Yammer, GitLab, ZenDesk, Hashicorp, Hulu, Netflix.
GitHub is actively transitioning away from Rails. Netflix is mostly Java / Node.js, they are not a significant Rails shop. Also, OP said "modern", probably to mean not companies that chose their tech stack a decade ago. New SaaS these days trend towards Next.js / Node (Supabase, Substack, Notion, Planetscale, Railway, Secureframe, Newfront, etc.), some Django, the occasional Rails or Phoenix.
Supabase wrote their backend in Elixir. AFIK, their offering started as essentially “various stuff common in Elixir systems, as a service for JS devs”.
More here: https://thinkingelixir.com/podcast-episodes/073-elixir-at-su...
More here: https://thinkingelixir.com/podcast-episodes/073-elixir-at-su...
Yes, some portion of their backend is Elixir/Phoenix: https://github.com/supabase/realtime
but most of their current stack and active development, frontend and backend, is Next.js, and not the Rails/Phoenix fullstack way: https://github.com/supabase/supabase
but most of their current stack and active development, frontend and backend, is Next.js, and not the Rails/Phoenix fullstack way: https://github.com/supabase/supabase
I just did a full write up on the Architecture becaase we have this question often: https://supabase.com/docs/architecture#architecture
> most of their current stack and active development, frontend and backend, is Next.js
This is only the Dashboard. We dogfood Supabase to build Supabase, so we use the tools in the doc above
> most of their current stack and active development, frontend and backend, is Next.js
This is only the Dashboard. We dogfood Supabase to build Supabase, so we use the tools in the doc above
No better clarification than the author! In my mind, I was thinking of just the web framework UI-for-CRUD comparison, where Next.js is supplanting what Rails is typically used for (like your Dashboard / GUI, and most of the core webapps for the previous 2010 Rails co's), but fair enough, you're probably leveraging a lot more of your own PostgREST, triggers, functions, & webhooks than a typical backend Next shop.
> GitHub is actively transitioning away from Rails
Cite?
Cite?
https://www.theregister.com/2018/08/16/github_rails_microsof...
"GitHub is about a third of the way through an architectural change that began last year. The company is moving away from Ruby on Rails toward a more heterogeneous, composable infrastructure. Ruby still has a place at GitHub – Lambert referred to the company as a Ruby shop, but he said there's more Go, Java and even some Haskell being deployed for services."
https://github.blog/2021-03-16-improving-large-monorepo-perf...
"We’re also in the middle of a significant refactoring effort, doing our part to decompose GitHub’s famous Ruby monolith, and writing a new microservice in Go"
"GitHub is about a third of the way through an architectural change that began last year. The company is moving away from Ruby on Rails toward a more heterogeneous, composable infrastructure. Ruby still has a place at GitHub – Lambert referred to the company as a Ruby shop, but he said there's more Go, Java and even some Haskell being deployed for services."
https://github.blog/2021-03-16-improving-large-monorepo-perf...
"We’re also in the middle of a significant refactoring effort, doing our part to decompose GitHub’s famous Ruby monolith, and writing a new microservice in Go"
He referred to the company as "a Ruby shop" so maybe that's a bit overstated?
No major company completely overhauls their tech stack overnight. I originally stated that they're transitioning away from Ruby so I don't think that's overstated, they are. Just like Twitter, LinkedIn, Soundcloud, Grab, Parse, Twitch, and Deliveroo have done before them, it's not unique.
It's 20 years ahead of Perl, but same fate. It doesn't have a niche besides web w/ Rails, which javascript/typescript is now dominating. It's the slowest and has the most well-known scale problems. The dev ecosystem is almost unanimously moving towards static types, a la Typescript and MyPy. It's symbols are not as transferable like more popular C-based langs: Javascript, Java, Python, Go. The only point devs will routinely point to in favor of it is "well, it just feels fun and nice to work with", while actually StackOverflow survey data shows Rust, Typescript, Python all leagues ahead in "loved", while Ruby is in majority dissatisfaction land right next to the monstrosities of C++ and Scala.
It's 20 years ahead of Perl, but same fate. It doesn't have a niche besides web w/ Rails, which javascript/typescript is now dominating. It's the slowest and has the most well-known scale problems. The dev ecosystem is almost unanimously moving towards static types, a la Typescript and MyPy. It's symbols are not as transferable like more popular C-based langs: Javascript, Java, Python, Go. The only point devs will routinely point to in favor of it is "well, it just feels fun and nice to work with", while actually StackOverflow survey data shows Rust, Typescript, Python all leagues ahead in "loved", while Ruby is in majority dissatisfaction land right next to the monstrosities of C++ and Scala.
* Ruby has static types (Sorbet)
* Most of the alternatives still don't come close to Rails for web app productivity--that's more than a gee it feels fun
* The pendulum of JS apps is swinging away from pointless SPA's and more toward hotwire and live-updating apps
* Ruby is as fast or faster than Python on most points
* Most of the alternatives still don't come close to Rails for web app productivity--that's more than a gee it feels fun
* The pendulum of JS apps is swinging away from pointless SPA's and more toward hotwire and live-updating apps
* Ruby is as fast or faster than Python on most points
I would have used Rails for the current project I'm on, but for one thing:
Data science.
In the past (around 2010 to about 2018) I did the web stuff in Rails and communicated through the database for the ML stuff I needed from Python, but even though Rails is better than Django, these days Django is good enough and it isn't worth the tooling and the employee dual lingual requirements to justify both.
It sucks and I wish I could use Ruby more. I agree Ruby and Rails are both better than Python and Django on their own, but the scientific computing part of Python is just so phenomenal that there isn't really anything to do about it anymore other than to accept it.
Ruby is like Elivish. It's poetic and beautiful and consistent where it counts, full of symbolism and grandeur, but in a world of dwarves, humans, and orcs it just isn't useful enough to justify making an entire team learn it.
Data science.
In the past (around 2010 to about 2018) I did the web stuff in Rails and communicated through the database for the ML stuff I needed from Python, but even though Rails is better than Django, these days Django is good enough and it isn't worth the tooling and the employee dual lingual requirements to justify both.
It sucks and I wish I could use Ruby more. I agree Ruby and Rails are both better than Python and Django on their own, but the scientific computing part of Python is just so phenomenal that there isn't really anything to do about it anymore other than to accept it.
Ruby is like Elivish. It's poetic and beautiful and consistent where it counts, full of symbolism and grandeur, but in a world of dwarves, humans, and orcs it just isn't useful enough to justify making an entire team learn it.
I still see a lot of Rails contract work out there, roughly as many roles as ever before.
I've thought about switching to Elixir work but I just don't see the demand there yet. There's 2 orders of magnitude more Rails roles than there are Elixir jobs. That might change with time hopefully.
I can only speak for myself but I think Rails 7 is the most exciting release for years.
Hotwire, Stimulus, et al are finally a compelling reason to do frontend work in Rails again vs the last few years of using it mainly as an API for JavaScript SPA frameworks.
Ruby 3 finally provides a decent solution for async IO (with fully parallel ractors a work in progress).
Tooling is also much improved over a few years ago with Sorbet and Treesitter AST based stuff.
Rails might be boring but boring gets the job done.
I've thought about switching to Elixir work but I just don't see the demand there yet. There's 2 orders of magnitude more Rails roles than there are Elixir jobs. That might change with time hopefully.
I can only speak for myself but I think Rails 7 is the most exciting release for years.
Hotwire, Stimulus, et al are finally a compelling reason to do frontend work in Rails again vs the last few years of using it mainly as an API for JavaScript SPA frameworks.
Ruby 3 finally provides a decent solution for async IO (with fully parallel ractors a work in progress).
Tooling is also much improved over a few years ago with Sorbet and Treesitter AST based stuff.
Rails might be boring but boring gets the job done.
Can only agree. Rails 7 with Hotwire, Turbo, Stimulus is very much a different experience than earlier versions. The fact that Rails is still evolving and changing in such big strides makes me think that it could stay relevant longer than people might think.
Been using Rails since 1.2. Still excited. Rails 7 + Hotwire + ESBuild = <3
Once I add bootstrap and simple_form, the amount of code I don't have to write in a Rails application is just incredible.
And yes, I've used other things on and off my entire career. React is a love/hate relationship. Happy to punt it when working on a small project.
Once I add bootstrap and simple_form, the amount of code I don't have to write in a Rails application is just incredible.
And yes, I've used other things on and off my entire career. React is a love/hate relationship. Happy to punt it when working on a small project.
I've been trying out Alpine† instead of Vue/React/whatevs and I'm liking it.
Also, I'm using Propshaft couple with Importmap instead of Hotwire & Turborails and so on
† https://alpinejs.dev/
Also, I'm using Propshaft couple with Importmap instead of Hotwire & Turborails and so on
mkdir foo
cd foo
bundle init
# edit Gemfile to set the Rails you want: main, edge, etc.
bundle
# plus whatever other options you normally choose when you run `new'
bundle exec rails new --skip-hotwire -a propshaft .
bundle install
bundle binstubs bundler
bin/bundle add importmap-rails
bin/rails importmap:install
bin/importmap pin @rails/ujs alpinejs bootstrap
† https://alpinejs.dev/
> To draw an even smaller circle, is anybody still excited about Ruby when Rails isn't involved?
I love Ruby, and I'm excited about where it is and where it's going. It's my go-to language for just about everything. I write automations with it, API clients with it, little just-for-me programs with it, little experiments with it.
Ruby is a beautiful, delightful object oriented language and I wouldn't want to work with anything else right now (especially not Javascript).
I love Ruby, and I'm excited about where it is and where it's going. It's my go-to language for just about everything. I write automations with it, API clients with it, little just-for-me programs with it, little experiments with it.
Ruby is a beautiful, delightful object oriented language and I wouldn't want to work with anything else right now (especially not Javascript).
I discovered Ruby in 2001. It's been my go-to language ever since. I can write in other languages (mainly C, C++, Python and Haskell), and most certainly read a multitude of other languages, but Ruby just hits the spot.
“Excited” is an interesting word. I still like Return of the Jedi, but I’m not excited about it because it’s old.
The Ruby/Rails community is very mature, the language and the framework are flexible enough for anything, and there are libraries and guides for anything you want to do.
So no, it’s not that exciting now, but it’s still widely used. From my own experience, my company continues to use it for many project because we have tons of our own library code that couldn’t easily be recreated.
The Ruby/Rails community is very mature, the language and the framework are flexible enough for anything, and there are libraries and guides for anything you want to do.
So no, it’s not that exciting now, but it’s still widely used. From my own experience, my company continues to use it for many project because we have tons of our own library code that couldn’t easily be recreated.
I'm pivoting my career away from Rails after 10 years with it. Elixir and Phoenix fills the Rails void and does what Rails does better than Rails with usually better code at ~7x the speed. In my opinion - there's no reason to build a new app in Rails today aside from familiarity.
FWIW, I started with 2.X back in 2007/2008 and I have decided that my current employer is my last role working in a Rails shop.
What do you plan to focus on?
I started my career in embedded and I am strongly considering swinging back towards systems work.
I'm curious what sort of "market" you are finding for paid elixir/phoenix work. Are you a freelancer/consultant?
A customer of mine develops a surprising number of web apps with Rails. REST APIs or full stack apps with Turbo/Hotwire/Stimulus whatever the name is. I've been working with them for six months and I saw and worked on at least five apps at different state of maintenence and delivery, including one started from scratch and completed in three months with a React front end. Maybe there are more productive frameworks but Rails is still a good way to make developers productive.
Are they SaaS apps intended to be used by millions of people? No, maybe only in the thousands or tens of thousands but I think that not many SaaS are used daily by million of people. There are only 24 hours in a day.
Are they SaaS apps intended to be used by millions of people? No, maybe only in the thousands or tens of thousands but I think that not many SaaS are used daily by million of people. There are only 24 hours in a day.
If HN interest is a reasonable proxy, then judging by the number of Ruby stories that flow through here, yes.
“Other than DHH adherents, is anybody still excited about Rails? To draw an even smaller circle, is anybody still excited about Ruby when Rails isn't involved?”
If people use a language/framework and don’t constantly talk about it online, does anyone even use that language/framework?
The non-snarky answer is yes and yes. Source: me, who just attended Euruko, a Ruby conference. Rails is just mature, dependable, and boring, so there’s not as much to talk about compared to X-framework of the week/month.
If people use a language/framework and don’t constantly talk about it online, does anyone even use that language/framework?
The non-snarky answer is yes and yes. Source: me, who just attended Euruko, a Ruby conference. Rails is just mature, dependable, and boring, so there’s not as much to talk about compared to X-framework of the week/month.
Me. I am. I like Ruby. It’s simple and easy to use.
I use it all the time and just don't talk about it, because it's boring and does the job. Why? Because I know how it works, and can make functional things in minutes. It's my go to hammer when I need to build another stupid fucking website with a database. I don't care if it's "slow", I couldn't give less of a shit about the extra 12 cpu cycles every request needs, it is completely irrelevant because I'm not serving traffic to millions of concurrent users.
I think Rails still has a future in the same way C does. Not many are particularly excited about it and it's possibly not the right choice for new projects, but it has a very large existing app base that will continue for a long time.
IMO the biggest issue is the fact Ruby is untyped. Things like updating Rails is an absolute nightmare.
IMO the biggest issue is the fact Ruby is untyped. Things like updating Rails is an absolute nightmare.
Terraform Cloud's web UI and API are Ruby on Rails
> Other than DHH adherents, is anybody still excited about Rails?
I like Rails despite DHH. I'm a part of huge communities where people are still excited for Rails.
It might not be the newest thing but I love working in Rails. Kind of depressive when, every time Ruby comes up here (it's still a lot, which should be a better indicator), this same topic comes up. Not because i'm tied to the language, but that the joy I am having isn't shared.
I like Rails despite DHH. I'm a part of huge communities where people are still excited for Rails.
It might not be the newest thing but I love working in Rails. Kind of depressive when, every time Ruby comes up here (it's still a lot, which should be a better indicator), this same topic comes up. Not because i'm tied to the language, but that the joy I am having isn't shared.
Same.