The latest test that we have, dubbed "Fortunes", does server-side templating, so we (or a generous community member) could add a JSF implementation for that test.
Thanks for the kind words. We're very interested in adding additional tests, this round even includes a new test dubbed "Fortunes" which does in fact do server-side templating. We have an open github issue[1] asking for the community's input for just this sort of thing, and we'd love to have your feedback included.
I want to give a quick thank you to everyone who has contributed thus far. Being able to show frameworks that span this many different languages and platforms is really an amazing achievement that we only we able to accomplish with the help of the community. For the frameworks that are still missing, we're not done yet, continue submitting pull requests and we'll get them in.
Correct, because we chose MySQL as the database backend, both Django and Flask are punished for not having a connection pool. We've received requests to do the test using Postgres which does have a connection pool, and we hope to eventually get to that.
There will most certainly be a Round 3, and 4 and on and on as long as we continue to have feedback from the community. I can guarantee that we'll have a Lua test in there, but if we receive a pull request, we'll include it.
We're looking into moving to unicorn as the community has suggested. If anyone is interested in setting up Puma or Goliath, we'd be interested in testing those out as well.
"I like these comparisons. It's unlikely that anyone is going to pick a framework for their project based on just this alone."
This is exactly right, performance is one of many items to consider when building your own website, and while we're aiming to have a very comprehensive and useful set of performance data, we would agree that other factors need to be taken into account.
The short answer is that at some point, we kept adding more and more tests on our own, which kept the blog from going out, and we finally decided to have a cutoff date and let the community continue adding tests that they were interested in. I think adding a database test to Netty (as well as Go for that matter) would be a great addition.
We've had a lot of requests Mono, and are actually very interested in trying this out. We've been hesitant to move forward only because we're unsure how stable the C# community believes Mono to be. We hope to have that test at some point though, but the fastest way to get a test included is to issue a pull request. So if anyone is interested in trying this out, let us know and we can help guide the process.
We have one pull request for Onion, which is c based. We're working with the author to work through some issues, but we're also excited to see how it will stack up.
Since we moved to a different benchmarking tool for round 2, we had to re-run all the tests, which is a time consuming process, especially on the EC2 hardware. Now that we've gotten that out of the way, we have an easier time running and processing results for individual tests. So stay tuned!