A guide to product analytics tools for startups(satchel.com)
satchel.com
A guide to product analytics tools for startups
https://satchel.com/web-analytics/
28 comments
Is this dated? GA4 has switched to entirely event-driven framework for measurement.
So all the products recommended by Satchel ar YC products, huh
[deleted]
We use event-based analytics. A super easy way to do it is to create an AWS Lambda function that you can hit using a Rest API. A simple `console-log` statement with JSON.stringify will log it to Cloudwatch logs. And bam, you can create basic charts and stuff with [Log Insights](https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/CWL...). Up to 5Gb in the free tier. Easy dashboards. All data is in AWS (so less tools to keep up with)
Tradeoff - their query language is poor, and I really want to do cohort analysis. But you can export as CSV, or easily push to Datadog.
Tradeoff - their query language is poor, and I really want to do cohort analysis. But you can export as CSV, or easily push to Datadog.
"Paid Plan Affordability" for Heap is listed as "noticeably better" (green), but on Heap's website, they either have a limited free tier, or a business plan "Starting at $12,000/year". Is that what's considered affordable for a startup? This whole website seems like a shill for YC companies.
If I can give an advice to the author. The structure of the article is not very good.
There is no best product analytics tool, the 3 products presented are all great products. Instead of saying "this product is the best". You should do,
- If you are in this case, use this product
- In this case use this other one.
- ...
If you are just a startup wanting to go into product analytics, any of this product will be fine
There is no best product analytics tool, the 3 products presented are all great products. Instead of saying "this product is the best". You should do,
- If you are in this case, use this product
- In this case use this other one.
- ...
If you are just a startup wanting to go into product analytics, any of this product will be fine
Great writeup here but I have a question on whether Product Analytics is at all appropriate for Startups. I'm incredibly biased (this problem is why I built Chartio), but I think these tools end up being quite bulky, expensive and a non-agile way to do analytics on your product. I feel these products are best suited for later stage companies trying with great tracking detail to improve a funnel or engagement metric, not simply monitor usage.
The majority of the important things for a startup are usually already being tracked with timestamps in the database (new signups, new users, churned users, new todo items, etc). If a key metric is not directly tracked, there's usually a good enough proxy available somewhere in the database. A Busineess Intelligence product, or a SQL to chart tool is much more applicable and affordable in the startup stage.
There are likely going to be responses here "But what about..." and I'll premptively respond to those with the question: Could that be answered/solved almost just as well with a query to the database, without sacrificing the agility, extra setup and data collection time, and expense overhead? Those are all very costly things for a startup.
The majority of the important things for a startup are usually already being tracked with timestamps in the database (new signups, new users, churned users, new todo items, etc). If a key metric is not directly tracked, there's usually a good enough proxy available somewhere in the database. A Busineess Intelligence product, or a SQL to chart tool is much more applicable and affordable in the startup stage.
There are likely going to be responses here "But what about..." and I'll premptively respond to those with the question: Could that be answered/solved almost just as well with a query to the database, without sacrificing the agility, extra setup and data collection time, and expense overhead? Those are all very costly things for a startup.
(I used to work at Heap.)
I think the post is spot on and it shows the Satchel team put did a ton of research into the post. There are two comments I would make on top of the post. First, know if the advice applies to you. If you are pre-product market fit it's probably too early to think about event based analytics. If you have a small number of users and are able to talk with all of them, you will get much more meaningful data getting to know them than if you were to set up product analytics. You probably don't have enough users to get meaningful data from product analytics anyways.
Second, while the autotrack functionality at Heap is fantastic, what I saw was that a significant portion of Heap's customers were not able to use it. This primarily happened because in addition to using Heap's autotrack to collect data, a lot of Heap's customers were also using Segment to collect and route the data between different tools. This created two different sources of truth for the data and Segment usually wound up winning. For that reason, I left Heap ~18 months ago to start Freshpaint (freshpaint.io). Freshpaint is an autotrack based alternative to Segment, allowing you to autotrack data and feed that same dataset into all your different tools. That way you get all the advantages of autotrack without needing to maintain two sources of truth.
I think the post is spot on and it shows the Satchel team put did a ton of research into the post. There are two comments I would make on top of the post. First, know if the advice applies to you. If you are pre-product market fit it's probably too early to think about event based analytics. If you have a small number of users and are able to talk with all of them, you will get much more meaningful data getting to know them than if you were to set up product analytics. You probably don't have enough users to get meaningful data from product analytics anyways.
Second, while the autotrack functionality at Heap is fantastic, what I saw was that a significant portion of Heap's customers were not able to use it. This primarily happened because in addition to using Heap's autotrack to collect data, a lot of Heap's customers were also using Segment to collect and route the data between different tools. This created two different sources of truth for the data and Segment usually wound up winning. For that reason, I left Heap ~18 months ago to start Freshpaint (freshpaint.io). Freshpaint is an autotrack based alternative to Segment, allowing you to autotrack data and feed that same dataset into all your different tools. That way you get all the advantages of autotrack without needing to maintain two sources of truth.
Another autotracking tool is https://useitbetter.com which combines quant analytics with heatmapping and a/b testing.
I believe Mixpanel also offered autotracking for a while, but it didn't work out for some reason? It's probably difficult to mix pricing for manual tagging nad autotracking since the latter produce massive amounts of data.
I believe Mixpanel also offered autotracking for a while, but it didn't work out for some reason? It's probably difficult to mix pricing for manual tagging nad autotracking since the latter produce massive amounts of data.
The suggested website gives me a white page? Need to unblock my content blocker to get it loading. Not a great start imho
Honestly this is a pretty weak overview of the landscape, apparently limited only to the top YC analytics companies and published by a YC company.
There are tons of options out there, including many self-hosted solutions. You'll have to do further research beyond this article, however.
There are tons of options out there, including many self-hosted solutions. You'll have to do further research beyond this article, however.
For example, Matomo... larger list here
https://github.com/onurakpolat/awesome-analytics
I mean... have you ever used Matomo? It isn't good. I checked it out earlier this year.
I’m always suspicious of articles like these that keep bringing up one company that fits their criteria. That tells me the criteria was constructed around a specific offering, not the other way around.
I know PostHog is only lightly mentioned in the article, but I would highly recommend checking them out! [0] (They're also open-source and provide a self-hosted solution. [1])
[0] https://posthog.com/
[1] https://github.com/posthog/posthog
[0] https://posthog.com/
[1] https://github.com/posthog/posthog
Yeah they do Fullstory-type stuff, Heap-type stuff, plus they have an SDK in most popular languages (which no other company has.) If I didn't already set my stuff up before I had heard of them, I would have gone with them. They have the best parts of other tools.
Unless their stuff doesn't work well. I only spent a couple of hours so I might not have noticed UX issues that would have been painful to deal with.
Unless their stuff doesn't work well. I only spent a couple of hours so I might not have noticed UX issues that would have been painful to deal with.
Contrary to the main theme of the article, Google Analytics does in fact offer event based analytics in addition to page view analytics.
When using event based goals, Google Analytics removes the ability to visualize funnels.
In my day-to-day this is a big drawback.
In my day-to-day this is a big drawback.
I was also befuddled by this. I was using event tracking with GA at least 3 years ago I think, so it was difficult for me to take this article seriously as it hammers on this point repeatedly
Google analytics v4 taking it another level also.
I’ve implemented both amplitude and mixpanel for event tracking on Django apps.
I currently have a project that uses both auto JS and python-based track() calls using mixpanel.
I’ve found people commonly block js analytics and it isn’t safe to rely on for user behavior.
For example, you can instrument react hooks, but you really aught to instrument the api calls they rely on or they will not get captured in many circumstances.
I also have found mixpanel’s anon / known user ID merge to be poorly documented and the community support to be kind of slow and did not get the sense the folks answering had not personally done the implementations being discussed.
One of the biggest things I wanted out of event tracking was to build event bases web hooks to trigger drip-contact events.
Mixpanel quietly deprecates web hooks a few months ago and now wants you to use partners that process your data and do this for you.
I currently have a project that uses both auto JS and python-based track() calls using mixpanel.
I’ve found people commonly block js analytics and it isn’t safe to rely on for user behavior.
For example, you can instrument react hooks, but you really aught to instrument the api calls they rely on or they will not get captured in many circumstances.
I also have found mixpanel’s anon / known user ID merge to be poorly documented and the community support to be kind of slow and did not get the sense the folks answering had not personally done the implementations being discussed.
One of the biggest things I wanted out of event tracking was to build event bases web hooks to trigger drip-contact events.
Mixpanel quietly deprecates web hooks a few months ago and now wants you to use partners that process your data and do this for you.
Out of curiosity, what category is your product in? I don't think i've found a huge amount of people block js analytics, but thats probably due to the categories of products I've worked in.
This new product is a consumer social / video concept.
Doesn’t just about every extension-based ad blocker block all js analytics discussed in this post?
Have you done backup backend instrumentation if particular events to see definitively that you’re getting duplicate events as often as you think?
One thing I noticed is that without the belt and suspenders approach it really was not obvious what I was missing.
I also care a lot about people who run ad blockers because I consider them to be more savvy and thus more likely to influence the behavior of others.
Doesn’t just about every extension-based ad blocker block all js analytics discussed in this post?
Have you done backup backend instrumentation if particular events to see definitively that you’re getting duplicate events as often as you think?
One thing I noticed is that without the belt and suspenders approach it really was not obvious what I was missing.
I also care a lot about people who run ad blockers because I consider them to be more savvy and thus more likely to influence the behavior of others.
Disclosure: I work at Mixpanel
We updated our packaging/pricing a few weeks ago. Our Free plan allows for 100k MTUs now (100x more volume), and our Growth plan is cheaper. https://mixpanel.com/pricing/
We wanted to make a big improvement in both these categories for customers:
- "How far you can get without paying"
- "Paid Plan Affordability"
Just an FYI since that change happened since this Satchel review was published in May.
We updated our packaging/pricing a few weeks ago. Our Free plan allows for 100k MTUs now (100x more volume), and our Growth plan is cheaper. https://mixpanel.com/pricing/
We wanted to make a big improvement in both these categories for customers:
- "How far you can get without paying"
- "Paid Plan Affordability"
Just an FYI since that change happened since this Satchel review was published in May.
Hey there!
We're kicking the tires on Mixpanel - currently we're prepping for an alpha release, and there are a few things about the dashboards I wish either don't exist, or are not easy to find:
1. I wish I had more options for visualizations. Fot instance, at this scale, it would be really nice to have a scatter plot of events at exactly the time they occur, rather than by the hour
2. I really wish I could customize the graphs more - like if I have a 3 day view, I would rather group the events by the hour rather than the day, and it seems like I can't do that.
3. Is there a place to just query all the events, and get them in a table form? Like I would love to just be able to get all of a particular event in a certain time period, and then look through the properties and exact timings one by one.
Anyway, just some casual observations from a potential user.
We're kicking the tires on Mixpanel - currently we're prepping for an alpha release, and there are a few things about the dashboards I wish either don't exist, or are not easy to find:
1. I wish I had more options for visualizations. Fot instance, at this scale, it would be really nice to have a scatter plot of events at exactly the time they occur, rather than by the hour
2. I really wish I could customize the graphs more - like if I have a 3 day view, I would rather group the events by the hour rather than the day, and it seems like I can't do that.
3. Is there a place to just query all the events, and get them in a table form? Like I would love to just be able to get all of a particular event in a certain time period, and then look through the properties and exact timings one by one.
Anyway, just some casual observations from a potential user.
Wow that's a game changer