Shopify now on Rails 5.0. started 12 years ago on 0.5, the First version released(twitter.com)
twitter.com
Shopify now on Rails 5.0. started 12 years ago on 0.5, the First version released
https://twitter.com/tobi/status/819246328211443713
31 comments
Somewhat of a tangent - are there any huge Ruby apps that are not Rails monoliths? All of the big Ruby apps are big Rails monoliths - Cookpad, Github, Shopify. Of course they probably have some services, but you wouldn't call them a "microservices" architecture by any means.
Sounds like Stripe may be a large service oriented system written in Ruby but not Rails. https://www.quora.com/Why-did-Stripe-choose-to-use-Ruby-for-...
Interesting that the bulk of their web interface was written in CoffeeScript at the time of the answer even though they weren't using Rails. I've just never seen a well-known product using CoffeeScript without Rails.
I do wonder if that remains to be the case or if they've moved their web interface to something else. They've released a number of new products the past year, and I'm sure at least one isn't CoffeeScript given the content of the post.
I do wonder if that remains to be the case or if they've moved their web interface to something else. They've released a number of new products the past year, and I'm sure at least one isn't CoffeeScript given the content of the post.
I believe Stripe is Sinatra with a custom rolled ORM. Seem to remember one of the founders posting that...maybe even here on HN.
Not really an app in typical sense but Heroku is mostly Sinatra afaik.
I'd be quite curious how much money keeping Rails up to date has cost Shopify over the years, compared to the benefits.
Compared to a lighter weight framework like Sinatra, Express, etc., it seems that Rails is only a benefit if you were unable to make halfway decent library and architecture choices on your own, or if for some reason you benefit from hiring people who are already familiar with a particular Rails version.
It's very interesting that companies like Shopify and Github have continued paying the upgrade tax with Rails... a tax that is necessarily mainly because of the tight coupling between core components and the assumption of tight coupling included in most of the library ecosystem. Seemingly, Shopify is well beyond the phase where "hey that gem will save us two weeks" is really a relevant factor.
This is not a criticism of Rails, just an expression of surprise that companies like Shopify and Github still consider it a value-add after the upgrade tax.
Compared to a lighter weight framework like Sinatra, Express, etc., it seems that Rails is only a benefit if you were unable to make halfway decent library and architecture choices on your own, or if for some reason you benefit from hiring people who are already familiar with a particular Rails version.
It's very interesting that companies like Shopify and Github have continued paying the upgrade tax with Rails... a tax that is necessarily mainly because of the tight coupling between core components and the assumption of tight coupling included in most of the library ecosystem. Seemingly, Shopify is well beyond the phase where "hey that gem will save us two weeks" is really a relevant factor.
This is not a criticism of Rails, just an expression of surprise that companies like Shopify and Github still consider it a value-add after the upgrade tax.
I'm not familiar with the rails ecosystem. Is there a particular nuance with Rails that gives it a more challenging than usual upgrade path? Or is it just that because it is such a comprehensive framework, each upgrade is subject to a much larger surface area of changes to consider?
Here's the guide for upgrading a Rails app, covering versions 3.0 up to 5.0:
http://guides.rubyonrails.org/upgrading_ruby_on_rails.html
As you can see, changes in Rails will sometimes dictate changes in your app. Sometimes there are flags that can be set (or not set) to defer some changes or alter default behaviour.
http://guides.rubyonrails.org/upgrading_ruby_on_rails.html
As you can see, changes in Rails will sometimes dictate changes in your app. Sometimes there are flags that can be set (or not set) to defer some changes or alter default behaviour.
Github is well known for NOT paying the upgrade tax, or at least deferring it much later than others do.
I've personally shepherded a 100kloc Rails app from 1.2 in 2007 through 4.2 in 2016, so I can offer some perspective.
First, I think you overestimating the upgrade tax of Rails, and underestimating both the value-add of Rails and the upgrade tax of any other library.
In my experience, the most painful upgrades just as often come from random gems (or gem interactions), or ruby itself. You aren't going to avoid this pain by using Sinatra. If you choose only low-level libraries and you roll everything yourself, then yes, you can potentially avoid this pain, but now you're reinventing a lot of wheels.
It's not just a matter of a gem saving two weeks, it goes to the very heart of the buy-vs-build question. Assuming the same quality of work, upgrading a well-maintained gem is going to be less work than maintaining the functionality yourself. The conventions and tight coupling of Rails may run counter to architectural decisions you want to make, but on the flip side they prevent a huge amount of bike-shedding (I didn't appreciate this until I jumped onto a legacy Django project and discovered a different flavor of chaos). Assuming you have the chops to design a better architecture from scratch, and assuming you put forth the effort to achieve a similar level of polish to what you get with large community-driven open-source frameworks like Rails, you still have to document and enforce your style guide and architectural decisions, and train your engineering staff on them in a way that you get for free with every Rails dev. On balance if you are building typical web applications Rails gives a ton of value, and if you are using a non-trivial subset of features then the upgrade tax is probably lower than curating your own stack, and much lower if you are re-implementing parts of the provided stack from scratch.
The important question is not really Rails vs Sinatra, but rather the stability of specific libraries. In that regard I used to consider ruby the most unstable and I would recommend against using ruby for anyone that cares about maintenance cost. However over the last few years it seems like Ruby has stabilized significantly (Rails 4.0 => 5.0 was dramatically easier than Rails 2.0 => 3.0), and Node has really taken up the mantle of moving-fastest-and-breaking-the-most-things.
My general feeling is Rails is pretty effective for any significantly sized web backend, even if it's just an API, as long as it's talking to some kind of relational database and make use of Rails RESTful routing conventions. If you're talking microservices then I would prefer Sinatra, but at that point I would probably consider Go or Node or Elixir for the improved performance profile that likely would be a more primary consideration if I was going down that route in the first place.
First, I think you overestimating the upgrade tax of Rails, and underestimating both the value-add of Rails and the upgrade tax of any other library.
In my experience, the most painful upgrades just as often come from random gems (or gem interactions), or ruby itself. You aren't going to avoid this pain by using Sinatra. If you choose only low-level libraries and you roll everything yourself, then yes, you can potentially avoid this pain, but now you're reinventing a lot of wheels.
It's not just a matter of a gem saving two weeks, it goes to the very heart of the buy-vs-build question. Assuming the same quality of work, upgrading a well-maintained gem is going to be less work than maintaining the functionality yourself. The conventions and tight coupling of Rails may run counter to architectural decisions you want to make, but on the flip side they prevent a huge amount of bike-shedding (I didn't appreciate this until I jumped onto a legacy Django project and discovered a different flavor of chaos). Assuming you have the chops to design a better architecture from scratch, and assuming you put forth the effort to achieve a similar level of polish to what you get with large community-driven open-source frameworks like Rails, you still have to document and enforce your style guide and architectural decisions, and train your engineering staff on them in a way that you get for free with every Rails dev. On balance if you are building typical web applications Rails gives a ton of value, and if you are using a non-trivial subset of features then the upgrade tax is probably lower than curating your own stack, and much lower if you are re-implementing parts of the provided stack from scratch.
The important question is not really Rails vs Sinatra, but rather the stability of specific libraries. In that regard I used to consider ruby the most unstable and I would recommend against using ruby for anyone that cares about maintenance cost. However over the last few years it seems like Ruby has stabilized significantly (Rails 4.0 => 5.0 was dramatically easier than Rails 2.0 => 3.0), and Node has really taken up the mantle of moving-fastest-and-breaking-the-most-things.
My general feeling is Rails is pretty effective for any significantly sized web backend, even if it's just an API, as long as it's talking to some kind of relational database and make use of Rails RESTful routing conventions. If you're talking microservices then I would prefer Sinatra, but at that point I would probably consider Go or Node or Elixir for the improved performance profile that likely would be a more primary consideration if I was going down that route in the first place.
I was at a dinner last year for the Shopify conference and a dev asked Tobi about a new update to the Rails templating engine and why Shopify hadn't adopted it yet. I was blown away to hear Tobi talking how he personally figured out the extra hardware costs of implementing the update. He is one of the few tech CEOs I am confident has balanced the costs regarding sticking with Rails. That said would love to hear his thoughts on this.
What are those two templating engines?
I don't know about the other, but Shopify has its own templating engine called Liquid, which allows their users to create themes etc fairly easily.
https://shopify.github.io/liquid/
https://shopify.github.io/liquid/
Thanks! I suppose the other one is plain ERB.
The advantage of ERB is that it's plain Ruby and there is no need to learn another language. For the same reason in the Python world I prefer web2py's use of Python than Django's custom language. It reminds me of the days of Strut's <c:choose> tags and all the others.
Interesting that Liquid is faster that ERB. All the surrounding HTML tags being the same, it means that the translated Liquid-to-Ruby code is faster than native Ruby code, which is strange. But maybe Liquid is faster at handling the HTML part of the page. I googled for some data but I couldn't find anything.
However I can understand why Shopify doesn't want to put full Ruby into the hands of customers. Some nasty guy could have used Shopify's CPUs to make some computations for free. Apparently GitHub pages also use Liquid, because of Jekyll and maybe for the same underlying reason.
The advantage of ERB is that it's plain Ruby and there is no need to learn another language. For the same reason in the Python world I prefer web2py's use of Python than Django's custom language. It reminds me of the days of Strut's <c:choose> tags and all the others.
Interesting that Liquid is faster that ERB. All the surrounding HTML tags being the same, it means that the translated Liquid-to-Ruby code is faster than native Ruby code, which is strange. But maybe Liquid is faster at handling the HTML part of the page. I googled for some data but I couldn't find anything.
However I can understand why Shopify doesn't want to put full Ruby into the hands of customers. Some nasty guy could have used Shopify's CPUs to make some computations for free. Apparently GitHub pages also use Liquid, because of Jekyll and maybe for the same underlying reason.
I've been using liquid for a long time (heavy Jekyll user), and it is pretty good at what it does. Shopify actually lets their enterprise customers (Shopify Plus) to run actual ruby code (which runs a very limited mruby setup) to help process orders.
https://github.com/Shopify/scripts-public
https://github.com/Shopify/scripts-public
I've personally worked on several Rails apps that have been through Rails versions 2 up to 5.
Most of the time (if you have good tests) the upgrade process is fairly painless, and has been becoming easier with more recent releases.
As mentioned by others, the biggest pain is usually from third party gems (e.g. I have one app I only just moved to Rails 5 because it used a non-standard database adapter that has only just released a version compatible with Rails 5).
Going through your whole app at upgrade time is also an excellent excuse to have a broad look over the codebase and pay off some technical debt.
Most of the time (if you have good tests) the upgrade process is fairly painless, and has been becoming easier with more recent releases.
As mentioned by others, the biggest pain is usually from third party gems (e.g. I have one app I only just moved to Rails 5 because it used a non-standard database adapter that has only just released a version compatible with Rails 5).
Going through your whole app at upgrade time is also an excellent excuse to have a broad look over the codebase and pay off some technical debt.
> it seems that Rails is only a benefit if you were unable to make halfway decent library and architecture choices on your own, or if for some reason you benefit from hiring people who are already familiar with a particular Rails version.
As far as I can tell you just described almost everyone. I'm a web dev contractor that has jumped around between a few dozen code bases in the past few years, often to clean things up. My experience has been that it doesn't seem matter if the code was written by a team of MIT grads or outsourced to china. With exactly 1 exception, all the home grown framework code bases were in much worse shape, often with huge performance issues.
As to why it's so hard for people to mash together their own sets of dependencies in a halfway maintainable way, I have no idea.
As far as I can tell you just described almost everyone. I'm a web dev contractor that has jumped around between a few dozen code bases in the past few years, often to clean things up. My experience has been that it doesn't seem matter if the code was written by a team of MIT grads or outsourced to china. With exactly 1 exception, all the home grown framework code bases were in much worse shape, often with huge performance issues.
As to why it's so hard for people to mash together their own sets of dependencies in a halfway maintainable way, I have no idea.
From a contractor point of view, joining a team working with a standardized architecture means that you are productive from day one. I don't like custom projects with custom architectures and directory layouts. It slows down everybody and IMHO it's not cost effective for the customer. In the Rails world I could accept Trailblazer which is at least a bit established. A custom Sinatra application, no thanks. I gave it a try and ended up with a half assed clone of Rails or Django, messier and less pleasant to work with. One shouldn't underestimate the polish of frameworks after years of development. I rather pay the upgrade tax than the extra amount of work to build everything from the ground up.
That's a LOT of code in the controllers. If I had to guess, it sounds like they're putting business logic in their controllers, which is a fairly well-established anti-pattern.
How's Shopify's platform performing these days?
How's Shopify's platform performing these days?
Excellently.
They have 452 classes in controllers and 41374 LOC. It's an average of 91.5 lines of Ruby per controller, not so many for the usual 7 scaffolded methods and some extras like strong params.
About performances, I always thought that writing logic in controllers, models or lib was a matter of readability and ease of maintenance.
Does it make a difference to the CPU if 50 lines of code run in a controller or in a model or in a file in lib or in a gem?
Maybe do 452 controllers slow down the router?
About performances, I always thought that writing logic in controllers, models or lib was a matter of readability and ease of maintenance.
Does it make a difference to the CPU if 50 lines of code run in a controller or in a model or in a file in lib or in a gem?
Maybe do 452 controllers slow down the router?
My holiday project was to upgrade a Rails project I helped create to Rails 5. The BadgeApp [0] is used to certify open source projects as satisfying the Core Infrastructure Initiative's Best Practices Badge. It is open source, so you can see the diff [1] if you're interested. The upgrade took me several hours, and was greatly assisted by our 99% test coverage. Biggest line count change was from needing to explicitly specify params in tests. The biggest annoyance was that Fastly's gem still doesn't support Rails 5, so I needed to fork it [2].
I too am hugely appreciative of the Rails ecosystem.
[0] https://github.com/linuxfoundation/cii-best-practices-badge/ [1] https://github.com/linuxfoundation/cii-best-practices-badge/... [2] https://github.com/fastly/fastly-rails/pull/67
Here are the app statistics:
I too am hugely appreciative of the Rails ecosystem.
[0] https://github.com/linuxfoundation/cii-best-practices-badge/ [1] https://github.com/linuxfoundation/cii-best-practices-badge/... [2] https://github.com/fastly/fastly-rails/pull/67
Here are the app statistics:
+----------------------+--------+--------+---------+---------+-----+-------+
| Name | Lines | LOC | Classes | Methods | M/C | LOC/M |
+----------------------+--------+--------+---------+---------+-----+-------+
| Controllers | 687 | 476 | 8 | 59 | 7 | 6 |
| Helpers | 184 | 134 | 0 | 21 | 0 | 4 |
| Models | 711 | 470 | 5 | 55 | 11 | 6 |
| Mailers | 160 | 115 | 3 | 12 | 4 | 7 |
| Javascripts | 405 | 308 | 0 | 35 | 0 | 6 |
| Libraries | 369 | 281 | 0 | 0 | 0 | 0 |
| Tasks | 369 | 281 | 0 | 0 | 0 | 0 |
| Controller tests | 605 | 513 | 6 | 53 | 8 | 7 |
| Helper tests | 56 | 45 | 2 | 7 | 3 | 4 |
| Model tests | 398 | 309 | 9 | 37 | 4 | 6 |
| Mailer tests | 71 | 54 | 4 | 5 | 1 | 8 |
| Integration tests | 614 | 446 | 10 | 25 | 2 | 15 |
+----------------------+--------+--------+---------+---------+-----+-------+
| Total | 4629 | 3432 | 47 | 309 | 6 | 9 |
+----------------------+--------+--------+---------+---------+-----+-------+Dan: Thanks again for your work (I also work on BadgeApp)!
One other important change due to the Rails 5 upgrade was that because jquery-turbolinks doesn't support Rails 5 [0], we eventually had to drop both jquery-turbolinks and turbolinks. If you're using jquery-turbolinks, you may also need to drop both jquery-turbolinks and turbolinks. Turbolinks doesn't work correctly with many JavaScript libraries (causing no end of subtle bugs), and in our case the performance boost turbolinks is supposed to provide, so turbolinks just wasn't worth the trouble anyway. Good riddance.
Rails 5 has some nice additions. I particularly appreciate that Rails 5 has two new security hardening mechanisms to reduce the risk of CSRF security vulnerabilities: per-form CSRF tokens and origin-checking CSRF mitigation. We use a number of mechanisms to make our application secure [1].
The application doesn't have many LOC, but we do work to keep it that way. We try to reuse code where it makes sense, and always try to keep it simple. Still, it doesn't take many lines to create a useful app, and that's a good thing.
[0] https://github.com/kossnocorp/jquery.turbolinks/issues/56 [1] https://github.com/linuxfoundation/cii-best-practices-badge/...
One other important change due to the Rails 5 upgrade was that because jquery-turbolinks doesn't support Rails 5 [0], we eventually had to drop both jquery-turbolinks and turbolinks. If you're using jquery-turbolinks, you may also need to drop both jquery-turbolinks and turbolinks. Turbolinks doesn't work correctly with many JavaScript libraries (causing no end of subtle bugs), and in our case the performance boost turbolinks is supposed to provide, so turbolinks just wasn't worth the trouble anyway. Good riddance.
Rails 5 has some nice additions. I particularly appreciate that Rails 5 has two new security hardening mechanisms to reduce the risk of CSRF security vulnerabilities: per-form CSRF tokens and origin-checking CSRF mitigation. We use a number of mechanisms to make our application secure [1].
The application doesn't have many LOC, but we do work to keep it that way. We try to reuse code where it makes sense, and always try to keep it simple. Still, it doesn't take many lines to create a useful app, and that's a good thing.
[0] https://github.com/kossnocorp/jquery.turbolinks/issues/56 [1] https://github.com/linuxfoundation/cii-best-practices-badge/...
If I can bother you with a dumb question, how do you produce these statistics?
bundle exec rake stats
Also, for discovery of a bunch more goodies:
bundle exec rake -T
bundle exec rake -T
I wonder if Shopify have any Ruby performance or Rails Scaling issues to share?
And the last thing is Rails just prove to be popular and works well for ( and may be only ) paid SaaS. It allows you to quickly iterate your product until you find a product / market fit. And once you have generate a revenue stream, the Server / Software Scaling Cost becomes relatively minor compared to Human Cost. But for those who are on a freeminum model and depends on Ads, Cost of running Ruby Rails may quickly out pace your budget before you have large audience to generate revenue.
1. https://speakerdeck.com/a_matsuda/the-recipe-for-the-worlds-...