Site's been down a lot, and I'm not fond of the campaign tracking stuff in the original URL, so here's the text:
Moving to Go
Posted on September 19, 2012 [by Alari?]
During the last few months we have systematically improved both front-end user interface, and backend server code for the Toggl main time tracking page. It was motivated by the fact that we found it increasingly hard to cope with the steady growth of users and traffic, resulting in serious slowdowns of the system and in some cases even downtime.
The majority of Toggl program code originated from the time when our user base was 10x smaller, so the system needed a major overhaul. There were also several functional shortcomings, for example entries were not refreshed automatically when changes were made elsewhere (e.g. mobile phone). Also we had no offline support of any kind on the web.
Our situation in May this year
It took 5-7 seconds to load a time tracking page in Toggl, quite often even up to 20 seconds and more. Most of the time was spent by the server to compile the necessary dataset of time entries, but also related data.
The whole backend was based on Ruby on Rails, we used Ruby 1.8 at that time.
User interface could not be used offline.
Javascript code was bloated and contained a lot of unnecessary code (for example we had multiple date parsing/formatting libraries, etc)
The loading and parsing of Javascript started to block our UI because of its size.
Backend API calls were not optimized for the purpose, we requested too much data which strained both the database and bandwidth.
New implementation
After careful consideration we decided to re-use the offline-enabled time tracking code that is also used in our mobile apps and in Toggl Desktop, only retaining the visual of the existing time tracking page. So basically we decided to replace the whole backend code of that page.
We had run some experiments with the Go programming language (http://golang.org/) before, and decided that we should re-implement some parts of our backend with this new platform. So far, it has paid off, as the development process was fast, and deployment surprisingly simple. The resulting code is a big improvement in terms of speed. We’ll continue to replace our backend code with Go.
Secondly we implemented a Redis-backed (http://redis.io/) WebSocket server in Go to enable realtime synchronization between different Toggl clients – so your time entries, projects etc. would be updated automatically if you use Toggl on multiple devices.
Thirdly we added HTML5 manifest and local storage to support offline usage of Toggl. This served also as the speed enabler. Offline was already implemented with our mobile interface m.toggl.com, so we reused a lot of that.
In frontend Javascript code, we’re moving to Backbone.js (http://backbonejs.org/). As the amount of Javascript code is increasing fast, this library enables to structure it better. For parsing, formatting and manipulating dates in Javascript, we’ve moved to Moment.js (http://momentjs.com/) It has simplified our code a lot.
Another update we made was to upgrade to Ruby 1.9. The upgrade gave the system another speed boost.
Finally we spent time measuring HTML/CSS/JS load times and optimizing the milliseconds there. Our goal was to get the load time to under 1 second, or even faster. While the tracking page now loads faster, it’s still a work in progress as we’re doing too many API requests when loading the timer. Also, we’re still using Javascript libraries that are quite bloated – for example for the sidebar report charts.
We still like Ruby on Rails a lot, and at the moment, will continue using it for serving user interface. Go will be used together with PostgreSQL and Redis for backend data crunching and efficient API calls.
Incremental launch
These changes encompassed several risks, as there were a lot of potential critical bugs associated. That’s why we decided to roll the update out incrementally. We started in early July, and slowly and cautiously added new users until we had approximately 10% using it. This amount of users gave us enough feedback on system stability and bugs. After 4 weeks it had stabilized enough to start rolling it out more aggressively. By now all users have been converted to the new version.
As mentioned before, speed is an important feature in Toggl. Robustness and speed is something you, our users, keep telling us if we ask what are the most important things you need from Toggl. We have gained some valuable lessons with the latest upgrade, and will continue implementing those also in other parts of Toggl.
Eh, you're right in theory, but in practice not really.
There's on the order of a 1,000-1,000,000x difference or more between a Numpy version of what OkCupid does, and what their C++ does, even if you use the same algorithm.
Similarly, when we recently farmed out a string similarity algorithm from Python to Java, with a more expensive and sophisticated approach in the latter, it ended up being conservatively ~10,000x faster.
I love Python and use it for...just about everything really, but until you've actually optimized software at a systems level you really have no idea what you're talking about.
Proper algorithms and data structures are necessary but not sufficient.
It's a pity higher-level languages don't offer better escape hatches when you need them.
I can just about guarantee you they're abusing lensing to make it viable. (Not that that's a bad thing, WoW is the same way.)
More impressive is the fact that the server can support 1,800 concurrent gaming-grade low latency clients regardless of lensing. That's really hard to do. These guys should be getting paid to implement low-latency protocols and servers.
I could see it being a lot like Palestine, with a one-state or two-state solution.
The two-state solution would simply have "China Facebook" and "Rest of the world Facebook" with all that implies.
The one-state solution would have a "Whole World Facebook" with each gov't (China) being given open season to sensor anything posted by one of their nationals.
I could see either approach having a lot of merit. Exposing the Chinese users to the rest of the world as much as possible has some benefit, but so does limiting the degree to which the situation in China is accepted.
By making a separate Chinese facebook, you're signaling to other authoritarian gov'ts that they're the exceptional/unpleasant case and would otherwise not be accommodated.
>How do you know grove.io was dropped for the new hot thing?
Because that's what happened to her last six projects?
Are you completely incapable of inductive reasoning or are you being obtuse so that you can make the total unaccountability of SV founders a moral crusade?
>Sentry is also open source, but for very different reasons than what you describe. I treat it like part of the business value, but that also means that many people can simply run their own
Then there's no issue in the hypothetical scenario I laid out.
I'm a Sentry user (of the hosted service) because it's open sourced. I wouldn't have given you a seconds notice otherwise and I doubt I'm alone in that.
Why be so defensive when you're clearly not in a position to be guilty of leaving all your customers up the creek?
What's with the hagiography? Culver made things people came to depend on, then those things disappeared.
I don't see how this is any different than picking up women and telling them you think they're serious girlfriend material in a context (business services) where that's the expected standard, then dropping them for the new hot thing.
Why is it suddenly acceptable to do this just because they called themselves a startup?
The Google API I was using had a 6 month lead time, no recommendations for graceful transition, no offer of data dumps, and the final shutdown date was never communicated.
I don't demand anything of them, but they won't create a dependency where I previously had none and expect to be rewarded with money for it.
I am making founders aware of this attitude so that they can
1. Realize that there's a problem
2. Distinguish themselves among their competitors by addressing this concern
I'm not aiming to prescribe solutions, just describing the heuristics I presently rely on to determine if a service is likely to become a liability. I always do a fluid cost-benefit analysis beyond the black-and-white I described in my top comment.
That you believed what I said had anything to do with "demanding" free labor of anyone is telling and indicates a defensive posture on the subject.
Not a surprising reaction to have, given that Sentry (your errors-as-a-service project) depends on people trusting you not to just shut it down tomorrow.
Site's been down a lot, and I'm not fond of the campaign tracking stuff in the original URL, so here's the text:
Moving to Go
Posted on September 19, 2012 [by Alari?]
During the last few months we have systematically improved both front-end user interface, and backend server code for the Toggl main time tracking page. It was motivated by the fact that we found it increasingly hard to cope with the steady growth of users and traffic, resulting in serious slowdowns of the system and in some cases even downtime.
The majority of Toggl program code originated from the time when our user base was 10x smaller, so the system needed a major overhaul. There were also several functional shortcomings, for example entries were not refreshed automatically when changes were made elsewhere (e.g. mobile phone). Also we had no offline support of any kind on the web.
Our situation in May this year
It took 5-7 seconds to load a time tracking page in Toggl, quite often even up to 20 seconds and more. Most of the time was spent by the server to compile the necessary dataset of time entries, but also related data. The whole backend was based on Ruby on Rails, we used Ruby 1.8 at that time. User interface could not be used offline. Javascript code was bloated and contained a lot of unnecessary code (for example we had multiple date parsing/formatting libraries, etc) The loading and parsing of Javascript started to block our UI because of its size. Backend API calls were not optimized for the purpose, we requested too much data which strained both the database and bandwidth. New implementation
After careful consideration we decided to re-use the offline-enabled time tracking code that is also used in our mobile apps and in Toggl Desktop, only retaining the visual of the existing time tracking page. So basically we decided to replace the whole backend code of that page.
We had run some experiments with the Go programming language (http://golang.org/) before, and decided that we should re-implement some parts of our backend with this new platform. So far, it has paid off, as the development process was fast, and deployment surprisingly simple. The resulting code is a big improvement in terms of speed. We’ll continue to replace our backend code with Go.
Secondly we implemented a Redis-backed (http://redis.io/) WebSocket server in Go to enable realtime synchronization between different Toggl clients – so your time entries, projects etc. would be updated automatically if you use Toggl on multiple devices.
Thirdly we added HTML5 manifest and local storage to support offline usage of Toggl. This served also as the speed enabler. Offline was already implemented with our mobile interface m.toggl.com, so we reused a lot of that.
In frontend Javascript code, we’re moving to Backbone.js (http://backbonejs.org/). As the amount of Javascript code is increasing fast, this library enables to structure it better. For parsing, formatting and manipulating dates in Javascript, we’ve moved to Moment.js (http://momentjs.com/) It has simplified our code a lot.
Following Google Page Speed (https://developers.google.com/speed/pagespeed/) tips, we’ve started to use a Javascript loader to reduce resource blocking. At the moment, we’re using LABjs (http://labjs.com/).
Another update we made was to upgrade to Ruby 1.9. The upgrade gave the system another speed boost.
Finally we spent time measuring HTML/CSS/JS load times and optimizing the milliseconds there. Our goal was to get the load time to under 1 second, or even faster. While the tracking page now loads faster, it’s still a work in progress as we’re doing too many API requests when loading the timer. Also, we’re still using Javascript libraries that are quite bloated – for example for the sidebar report charts.
We still like Ruby on Rails a lot, and at the moment, will continue using it for serving user interface. Go will be used together with PostgreSQL and Redis for backend data crunching and efficient API calls.
Incremental launch
These changes encompassed several risks, as there were a lot of potential critical bugs associated. That’s why we decided to roll the update out incrementally. We started in early July, and slowly and cautiously added new users until we had approximately 10% using it. This amount of users gave us enough feedback on system stability and bugs. After 4 weeks it had stabilized enough to start rolling it out more aggressively. By now all users have been converted to the new version.
As mentioned before, speed is an important feature in Toggl. Robustness and speed is something you, our users, keep telling us if we ask what are the most important things you need from Toggl. We have gained some valuable lessons with the latest upgrade, and will continue implementing those also in other parts of Toggl.