Benchmarks are like single match scorelines: interesting, but rarely comparable. We built a public ELO score leaderboard for analytics engines that treats every public benchmark (TPC-DS, TPC-H, SSB, vendor blogs, community posts - even the messy ones) as a “match.”
Why ELO? Because comparing databases is more like ranking football clubs over a season than timing a single sprint. Creating a relative robust and relative score is what counts.
We are currently preparing a submission for JOSE and are looking for feedback but also more people who are interested in joining in and refining the template
Together with www.linkedin.com/in/aaron-cul... I have created a template for making LLMs (from different vendors and even self hosted ones) easily accessible to researchers - including advanced document RAG with docling.
As some others already posted: There is more to copying data than just moving it. It is about observability. A lot of companies have created their own frameworks.
Did you ever want to find an alternative to cloud data PaaS/SaaS vendor lock-in?
In my blog post, I show how a 0-cost switch between local, Databricks and AWS EMR environments can be achieved to save cost by using dagster. Additionally, software engineering best practices around testability, and producing DRY code are enhanced.
Check out my deep dive and let’s discuss the potential impacts on our projects!
Why ELO? Because comparing databases is more like ranking football clubs over a season than timing a single sprint. Creating a relative robust and relative score is what counts.