HackerTrans
TopNewTrendsCommentsPastAskShowJobs

huy

no profile record

Submissions

What We Know and Don't Know About Analytics Engineering

holistics.io
2 points·by huy·قبل 5 سنوات·0 comments

The rise of analytics engineers and what we know about it

holistics.io
1 points·by huy·قبل 6 سنوات·0 comments

A Short Story About SQL’s Biggest Rival

holistics.io
256 points·by huy·قبل 6 سنوات·123 comments

comments

huy
·قبل سنتين·discuss
Congrats on the launch!

I think here's a few players in this space (dev-friendly BI tool) already: - Holistics.io - Lightdash - Hashboard

These tools all allow analysts to use both/either a local/cloud IDE to write analytics logic, and check in to Git version control.

How do you plan to differentiate with them?
huy
·قبل 4 سنوات·discuss
> While it looks really nice, hard to imagine it's quicker to hand draw this than reverse it out of your existing schema

They have an option to take your SQL dump text and convert it back to DBML which is the DSL they use to render.
huy
·قبل 5 سنوات·discuss
If you're new to the BI space, this book should give you a pretty good high-level overview: https://www.holistics.io/books/setup-analytics/
huy
·قبل 5 سنوات·discuss
If you’re using Chartio and looking to migrate, here are a few alternatives you can consider. These are mostly cloud-based tool that is SQL-native and sends your written SQL queries to your SQL data-warehouse.

1/ Redash: write SQL gets charts, similar to Chartio (although a bit less polished as it was developed by a small team). Open-source or Cloud-based. Downside is they recently get acquired by Databricks, so I’m not sure how that will affect their product roadmap. Support for Google Sheets as well.

2/ Superset: also write SQL gets charts. They’re both open-source (Apache Incubated), and being supported by a commercial company behind (Preset). Their visualization library looks pretty diverse too.

3/ Metabase: This is also based on SQL. The difference is that they put a simple modeling layer in between your DW and SQL, so that they provide a simple ‘ask question’ interface for business users to get data, it’s quite simple though so if business users want more sophisticated questions they’ll need to resort back to SQL. Their UI is pretty sleek. Both open source and cloud-based (pricing starts from $100) with a commercial company behind.

4/ Holistics: Similar to Metabase, where they provide a modeling layer and self-service capability to non-technical layer. Difference is their modeling layer seems more sophisticated than that of Metabase. They also support the T in ELT as part of modeling. Support for Google Sheets as well. Their UI looks well-polished. Cloud-based only, with pricing starting from $100. They’re also working on Git version-controlling your SQL/modeling code so that it’s easier to track change/audit, this might help prevent vendor lock-in and ease future migration.

5/ Mode: Write SQL gets charts (similar to Redash/Superset). They’re a lot more polished than Redash, with a focus towards doing Data Science (native Python/R support, Notebook, etc). Their Free plan supports up to 5 users, but beyond that you will need to talk to their sales people to get quoted pricing. I heard it can get over $1k/month based with additional user pricing on top.

6/ Looker: probably best in-class in term of SQL-based data modeling, but very expensive.

7/ There are also other tools which I don’t think fit in the SQL-native narrative, like AWS Quicksight, Google Data Studio, PowerBI, Tableau and Qlik.

P/s: These 2 posts give a fairly good framework on how to look/evaluate different BI tools. This is a pretty crowded space with many different paradigms baked in over many years, so it might be a bit confusing for new people looking into the field

- https://www.holistics.io/books/setup-analytics/navigating-th...

- https://www.holistics.io/books/setup-analytics/the-arc-of-ad...
huy
·قبل 6 سنوات·discuss
There are so many such solutions, that It depends on your use cases.

If you're looking for high-level dashboarding/monitoring that connects directly with your sources (REST, 3rd-party apps like Facebook, Adwords, etc), then tools in the realm of Klipfolio, Databox, Grow.com will suit you well. If you're looking for visualization tools that connect to your data-warehouse/SQL database, then BI tools like Holistics, Metabase, Redash, etc will likely work for you. And of course there's tools that can support both kinds of sources, like PowerBI and Google Data Studio.

P/s: This post does a good job at laying down a framework for evaluating these sort of BI tools: https://www.holistics.io/books/setup-analytics/navigating-th...
huy
·قبل 6 سنوات·discuss
For those who're curious, the tool to build the database diagrams is dbdiagram.io
huy
·قبل 6 سنوات·discuss
I counted 6.

Fivetran, dbt, Preset (Superset/Airflow), Sisu, Imply and Databricks.

Though, as someone who's in this space a while, I think they did a decently fair job at articulating the 'modern' data infrastructure landscape.
huy
·قبل 6 سنوات·discuss
I think you have a point, but there are more nuiances than that.

There are typically 2 types of data to collect: Transactional data and behavioural data.

Most transactional data, due to their important nature, are already generated and captured by the production applications. Since the logic is coded by application engineer, it's usually hard to get this data wrong. These data are then ETL-ed (or EL-ed) over to a DW, as described by the article.

For behavioural data, this is where your statement will most apply to. This is where tools like Snowplow, Posthog, Segment, etc come in to set up the proper event data collection engine. This is also where it's important to "collect data properly", as these kinds of event data changes structure fast, and hard to keep track over time. I'd admit this space (data collection management) is still nascent, with only tools like iterative.ly on the market.
huy
·قبل 6 سنوات·discuss
For those who're interested in learning more about the history and evolution of data infrastructure/BI - basically why and how it has come to this stage - check out this short guidebook [1] that my colleagues and I put together a few months back.

It goes into details how much relevance the practices of the past (OLAP, Kimball's modeling) has with the current changes in by the cloud era (MPP, cheap storage/compute, etc). Chapter 4 will be most interesting for HN audience: It walks through the different waves of data adoption ever since BI was invented in the 60-70s.

https://holistics.io/books/setup-analytics/
huy
·قبل 6 سنوات·discuss
I know of three (all OSS)

* https://www.posthog.com/ (more mixpanel alternative, can auto-track events too)

* https://snowplowanalytics.com/

* https://rudderstack.com/ (explicitly say it's Segment alternative)
huy
·قبل 6 سنوات·discuss
There's also dbdiagram.io that focuses exclusively on database diagrams. Nice thing about it is you can just write simple DSL code and they render the diagrams automatically. The downside is that they don't support collaborative editing yet.