HackerTrans
TopNewTrendsCommentsPastAskShowJobs

geoHeil

no profile record

Submissions

Show HN: ELO-Ranking Analytics Engines

data-inconsistencies.datajourney.expert
6 points·by geoHeil·11 mesi fa·1 comments

[untitled]

1 points·by geoHeil·11 mesi fa·0 comments

LLM-in-a-Box: A Templated, Self-Hostable Framework for Generative AI

github.com
3 points·by geoHeil·12 mesi fa·3 comments

Getting Better at Data Engineering

georgheiler.com
3 points·by geoHeil·anno scorso·0 comments

Mix data SaaS workloads (Databricks vs. EMR) to save money by using dagster

georgheiler.com
7 points·by geoHeil·2 anni fa·2 comments

comments

geoHeil
·11 mesi fa·discuss
Benchmarks are like single match scorelines: interesting, but rarely comparable. We built a public ELO score leaderboard for analytics engines that treats every public benchmark (TPC-DS, TPC-H, SSB, vendor blogs, community posts - even the messy ones) as a “match.”

Why ELO? Because comparing databases is more like ranking football clubs over a season than timing a single sprint. Creating a relative robust and relative score is what counts.
geoHeil
·12 mesi fa·discuss
We are currently preparing a submission for JOSE and are looking for feedback but also more people who are interested in joining in and refining the template
geoHeil
·12 mesi fa·discuss
Together with www.linkedin.com/in/aaron-cul... I have created a template for making LLMs (from different vendors and even self hosted ones) easily accessible to researchers - including advanced document RAG with docling.
geoHeil
·anno scorso·discuss
Nice. What about tracking the learning process and comparing against the peer group/ class?
geoHeil
·anno scorso·discuss
As some others already posted: There is more to copying data than just moving it. It is about observability. A lot of companies have created their own frameworks.

One example I find useful for a lot of usecases is dagster. You can define resources to encapsulate complexity https://docs.dagster.io/guides/build/external-resources in fact with components build on top of custom DSLs https://docs.dagster.io/guides/labs/components

At Magenta/Telekom we are bulding on https://georgheiler.com/event/magenta-data-architecture-25/ - you can follow along with this template here https://github.com/l-mds/local-data-stack/ you may find these examples useful to understand how to use dagster/graph-based data pipeliens @scale
geoHeil
·2 anni fa·discuss
Did you ever want to find an alternative to cloud data PaaS/SaaS vendor lock-in? In my blog post, I show how a 0-cost switch between local, Databricks and AWS EMR environments can be achieved to save cost by using dagster. Additionally, software engineering best practices around testability, and producing DRY code are enhanced.

Check out my deep dive and let’s discuss the potential impacts on our projects!

Big thanks also to my co-author Hernan.