Marko vs. React: Performance Benchmark(github.com)
github.com
Marko vs. React: Performance Benchmark
https://github.com/patrick-steele-idem/marko-vs-react
10 comments
This comparison benchmark was not designed to answer all of those questions. Perhaps just the following: "Does it scale well?"
The conclusion from these tests is that React currently does not scale well on the server (at least for this type of page) due to high CPU and memory usage while requests per second were low for a very modest page of 100 search results items. This is not surprising since the focus of React has been on the client-side performance, but it is important because many projects are adopting React as an "isomorphic" solution.
Hopefully the community and the authors of React can use this information to make React better. There is always something that can be learned from benchmarks like this.
React does do a good job, but I also think Marko+Marko Widgets will be a much better fit for certain types of applications since it also includes a very strong UI component model and it is lighter. If nothing else, hopefully others can learn something from looking at the code and seeing two different approaches.
The conclusion from these tests is that React currently does not scale well on the server (at least for this type of page) due to high CPU and memory usage while requests per second were low for a very modest page of 100 search results items. This is not surprising since the focus of React has been on the client-side performance, but it is important because many projects are adopting React as an "isomorphic" solution.
Hopefully the community and the authors of React can use this information to make React better. There is always something that can be learned from benchmarks like this.
React does do a good job, but I also think Marko+Marko Widgets will be a much better fit for certain types of applications since it also includes a very strong UI component model and it is lighter. If nothing else, hopefully others can learn something from looking at the code and seeing two different approaches.
I'm sure it would help if these tests used the minified version of React on the server.
Also this test could be more efficiently written. First (in SearchResults.jsx), setState accepts a callback in its second parameter, you don't need componentDidUpdate() or this.doneCallback (which is triggering more GC). Second in SearchResultsItem.jsx, you have an empty componentDidMount() function, you should get rid of that. Also this is super optimization, but it's also important that you think deeply about the creation of objects, therefore creating potential garbage that has to be cleaned up after short usage. In SearchResultsItem.jsx, you should just go with a className (className={"search-results-item" + (this.state.purchased && ' user-has-purchased')}) instead of a style object, which becomes useless if you don't have a purchase.
By giving each SearchResultsItem their own state management, you're making it more dirty than you has to be, it should just be a simple list item. Typically in this type of app (Search), the SearchResultsItem component would be stateless (no this.state, getInitialState). When someone clicks handleBuyButtonClick, that communicates the change to the store, which then flows down to SearchResults, which "re-renders" the list of SearchResultsItems.
There are probably a lot more optimizations I could go over, but in short conclusion React's performance is more than the library, it is a mode of thinking.
Also this test could be more efficiently written. First (in SearchResults.jsx), setState accepts a callback in its second parameter, you don't need componentDidUpdate() or this.doneCallback (which is triggering more GC). Second in SearchResultsItem.jsx, you have an empty componentDidMount() function, you should get rid of that. Also this is super optimization, but it's also important that you think deeply about the creation of objects, therefore creating potential garbage that has to be cleaned up after short usage. In SearchResultsItem.jsx, you should just go with a className (className={"search-results-item" + (this.state.purchased && ' user-has-purchased')}) instead of a style object, which becomes useless if you don't have a purchase.
By giving each SearchResultsItem their own state management, you're making it more dirty than you has to be, it should just be a simple list item. Typically in this type of app (Search), the SearchResultsItem component would be stateless (no this.state, getInitialState). When someone clicks handleBuyButtonClick, that communicates the change to the store, which then flows down to SearchResults, which "re-renders" the list of SearchResultsItems.
There are probably a lot more optimizations I could go over, but in short conclusion React's performance is more than the library, it is a mode of thinking.
Thanks for the feedback zghst. Here's my response to your comments:
> I'm sure it would help if these tests used the minified version of React on the server.
Minification sometimes makes runtime performance worse due to the tricks that minifiers use to make code smaller. Even if it did help, it really only makes sense to run this benchmark against the unmodified react package installed from npm. If someone wants to look into make the react module run faster by modifying the source then that would be great, but I consider that outside the scope of this benchmark.
> First (in SearchResults.jsx), setState accepts a callback in its second parameter, you don't need componentDidUpdate() or this.doneCallback (which is triggering more GC).
Didn't notice that `setState` supported a callback. I made the change and it did not make any noticeable change in the benchmark (it only removed the need for a few extra assignments). NOTE: Adding a "this.doneCallback" is not trigging more GC because there is only one done callback every being created and it is common across both the Marko benchmark and the React benchmark.
> Second in SearchResultsItem.jsx, you have an empty componentDidMount() function, you should get rid of that.
Removing the empty function did make any noticeable difference in the numbers but I removed it anyway for those who want to re-run the tests on their own.
> In SearchResultsItem.jsx, you should just go with a className (className={"search-results-item" + (this.state.purchased && ' user-has-purchased')}) instead of a style object, which becomes useless if you don't have a purchase.
I used styles specifically because most React guides recommend using inline styles (not CSS class names). Also, as part of the benchmark I wanted to test behavior and I defined the behavior to be: "When the user clicks on the "Buy Now" button the search results item should turn yellow". I attempted to implement this in the most appropriate way for both Marko and React and I think I did it in a fair way.
> By giving each SearchResultsItem their own state management, you're making it more dirty than you has to be, it should just be a simple list item. Typically in this type of app (Search), the SearchResultsItem component would be stateless (no this.state, getInitialState). When someone clicks handleBuyButtonClick, that communicates the change to the store, which then flows down to SearchResults, which "re-renders" the list of SearchResultsItems.
What you are suggesting is the Flux way of updating the data store and view and I think that is a great approach. However, if I were to update this benchmark to do things in the Flux way then it would complicate the benchmark and it would make the React test slower given that it would try to re-render all of the search results items instead of just re-rendering the one search results item that was modified.
> I'm sure it would help if these tests used the minified version of React on the server.
Minification sometimes makes runtime performance worse due to the tricks that minifiers use to make code smaller. Even if it did help, it really only makes sense to run this benchmark against the unmodified react package installed from npm. If someone wants to look into make the react module run faster by modifying the source then that would be great, but I consider that outside the scope of this benchmark.
> First (in SearchResults.jsx), setState accepts a callback in its second parameter, you don't need componentDidUpdate() or this.doneCallback (which is triggering more GC).
Didn't notice that `setState` supported a callback. I made the change and it did not make any noticeable change in the benchmark (it only removed the need for a few extra assignments). NOTE: Adding a "this.doneCallback" is not trigging more GC because there is only one done callback every being created and it is common across both the Marko benchmark and the React benchmark.
> Second in SearchResultsItem.jsx, you have an empty componentDidMount() function, you should get rid of that.
Removing the empty function did make any noticeable difference in the numbers but I removed it anyway for those who want to re-run the tests on their own.
> In SearchResultsItem.jsx, you should just go with a className (className={"search-results-item" + (this.state.purchased && ' user-has-purchased')}) instead of a style object, which becomes useless if you don't have a purchase.
I used styles specifically because most React guides recommend using inline styles (not CSS class names). Also, as part of the benchmark I wanted to test behavior and I defined the behavior to be: "When the user clicks on the "Buy Now" button the search results item should turn yellow". I attempted to implement this in the most appropriate way for both Marko and React and I think I did it in a fair way.
> By giving each SearchResultsItem their own state management, you're making it more dirty than you has to be, it should just be a simple list item. Typically in this type of app (Search), the SearchResultsItem component would be stateless (no this.state, getInitialState). When someone clicks handleBuyButtonClick, that communicates the change to the store, which then flows down to SearchResults, which "re-renders" the list of SearchResultsItems.
What you are suggesting is the Flux way of updating the data store and view and I think that is a great approach. However, if I were to update this benchmark to do things in the Flux way then it would complicate the benchmark and it would make the React test slower given that it would try to re-render all of the search results items instead of just re-rendering the one search results item that was modified.
Not necessarily addressing the speed portion, but if you're going to give a key to a repeated item in React, it is best to use the index instead of a unique identifier(if the overall DOM structure does not change much between renders) so that React does not destroy and recreate each item on tree change.
funkiee, good catch on that issue. I just tried using the current index during iteration, instead of the actual unique ID of the search results item and that sped things up a lot. In fact, now React and Marko perform almost exactly on par for client-side rendering. Server-side rendering of React components is still very slow in comparison. I will update the README with the new results.
On a related note, the React docs were very misleading when they stated the following:
"In practice, finding a key is not really hard. Most of the time, the element you are going to display already has a unique id. When that's not the case, you can add a new ID property to your model or hash some parts of the content to generate a key." Source: http://facebook.github.io/react/docs/reconciliation.html#key...
I tried following their advice and used the unique ID of the search results record and it turns out that was the wrong thing to do. It was better to use the index of the current iteration and I think that should always be the recommended thing to do given the performance difference.
On a related note, the React docs were very misleading when they stated the following:
"In practice, finding a key is not really hard. Most of the time, the element you are going to display already has a unique id. When that's not the case, you can add a new ID property to your model or hash some parts of the content to generate a key." Source: http://facebook.github.io/react/docs/reconciliation.html#key...
I tried following their advice and used the unique ID of the search results record and it turns out that was the wrong thing to do. It was better to use the index of the current iteration and I think that should always be the recommended thing to do given the performance difference.
The next sentence in your quote has some importance too, "Remember that the key only has to be unique among its siblings, not globally unique."
I agree there could be a little more clarity in that block of text, but I think the trade-offs section describes it well. Additionally, if there are any nodes in the tree that have a chance of being the same item between renders, you're going to benefit immensely from implementing a `shouldComponentUpdate` in your `SearchResultsItem.jsx` file.
React certainly doesn't win on every performance level, and if you try to compete on performance you're eventually going to lose. A careful developer could implement this list with no framework at all and have performance that beats both.
In the end, it's more about the mental model that best fits you and your team. I find the composite UI of React where I am building every component of other components in Javascript without a templating language to get in my way to be best suited to me.
If Marko works well for your team and you find it easy to get new developers on board and contributing code then that's awesome, keep up the good work.
I agree there could be a little more clarity in that block of text, but I think the trade-offs section describes it well. Additionally, if there are any nodes in the tree that have a chance of being the same item between renders, you're going to benefit immensely from implementing a `shouldComponentUpdate` in your `SearchResultsItem.jsx` file.
React certainly doesn't win on every performance level, and if you try to compete on performance you're eventually going to lose. A careful developer could implement this list with no framework at all and have performance that beats both.
In the end, it's more about the mental model that best fits you and your team. I find the composite UI of React where I am building every component of other components in Javascript without a templating language to get in my way to be best suited to me.
If Marko works well for your team and you find it easy to get new developers on board and contributing code then that's awesome, keep up the good work.
funkiee, would you be interested in opening a Github issue or submitting a Pull Request if you believe that is the right thing to do? I'll give it a try in the meantime, but it would be great to have a public discussion on that so others don't make the same mistake (if it does indeed help). Thanks!
It re-creates items even when they have unique identifiers? Is that a bug?
It's as intended. If the ID in a list of 100 changes, and that ID is the key, React is going to assume the tree is different during reconciliation. If you were to use an index, like 0, on pagination the key is still 0 and as such the DOM nodes will be reused. http://facebook.github.io/react/docs/reconciliation.html See #2 in the Trade-offs section.
Really good questions to ask and statements to reassert:
* Does is scale well? * Will this framework / library help me in my career forward? * How hard is it to implement features? * Is the framework / library enjoyable to work in? * Does the framework / library lend itself to allowing elegant solutions? * Does the framework / library provoke a thoughtful and/or refreshing pattern or style in your code? * Is the framework / library sometimes blocking solutions to problems? * Can a new team member readily understand this, explore and contribute after a day or so of training? * How easy is it collaborating with others? * This framework / library is simple and understandable enough to get started quickly * I am intrigued by this framework / library. * There is a lot of support and resources * There aren't a lot of grey areas
As far as I know, React knocks these out of the park.