Snowplow CEO here. We haven't used Jitsu before but are very familiar with Segment. It looks like Jitsu sits in the Segment product family, along with Rudderstack: basically a Customer Data Platform bundle of simple JSON event tracking, Fivetran-style transactional/SaaS data ingest, and then relaying of data out to various SaaS endpoints plus cloud DWs.
Snowplow started at the same time as Segment (2012) but has evolved along a separate tech tree. Micro-service architecture, cloud native, using Kinesis or Cloud Pub-Sub as the data transit, enrichment framework plus a Confluent-style schema registry supporting very rich and versioned JSON Schema-based event payloads. We are built by and for data platform teams; our open-source behavioral data engine doesn't have a UI (our commercial Behavioral Data Platform does). Hosted trial here https://try.snowplowanalytics.com/
Definitely room for both product families in the market! I'm sure Jitsu will do great.
Back in the early years of Snowplow, we adopted a similar approach for our support of 3rd party webhooks[1] as an event stream source supported natively by Snowplow (https://github.com/snowplow/snowplow). (The important background here is that all events and entities flowing through Snowplow are described using versioned JSON Schema).
We built a kind of Maven Central for 3rd party webhooks' schemas[2], and got to work adding schemas for various popular webhooks, e.g. Sendgrid and Pingdom. But, we never met a single SaaS vendor who was interested in 'adopting' the JSON Schemas for their webhook, let alone publishing versioned JSON Schemas for their whole API. This meant that at Snowplow we have stayed on the hook for keeping these vendors' webhook schema definitions up to date in our registry.
It's a real tragedy of the commons - oceans of developer time wasted on tedious low-leverage work (updating API client code) so that each SaaS vendor can 'move fast and break things'. The ultimate irony is that if these vendors were to adopt, publish and respect versioned schema definitions internally, using something like OpenAPI, they would see huge productivity gains (think enhanced CI/CD testing, auto-gen of client code etc).
I just wanted to jump in and say the same thing. I had the hardback of the Cuckoo's Egg as a teenager in the 90s. Huge inspiration to me and I have worked in and around tech ever since. Thank you.
He may think he doesn't have an 'HR department', but whoever holds the pen on his holiday policy, equipment policy, training & development policy, maternity & family friendly rights policy, expenses policy - well, there's his HR department.
Hey dylz - not yet. Historically Snowplow has been engineered in a way that is idiomatic to each host cloud - in other words, it runs using the AWS / GCP services that a data platform team would use if they were building a behavioral data pipeline from scratch on that cloud.
This said, we are steadily working to refactor all our components to be more generic, and already we run a lot of the Snowplow components on k8s for our customers.
Snowplow has had a strong following around HNers ever since we launched back in 2012. The truth is though that Snowplow open-source is still a complex project to stand up, and a lot of technology teams still aren't aware of the power of having all their own behavioral data (events from webapps, mobile etc) in their own data warehouse or lake, and available in real-time streams (Kinesis and GC Pub/Sub).
With this background, I am delighted to be able to Show HN our new Try Snowplow experience. Under lockdown last year, the team decided to go back to basics and build a new version of Snowplow called Try Snowplow; that version is now in GA.
Try Snowplow is a small version of the Snowplow technology that can be setup quickly and easily - normally under 15 mins. Like all versions of Snowplow, it runs in your own cloud environment.
If you are still puzzled why you want your own behavioral data in your own warehouse, check out our library of use cases here: https://snowplowanalytics.com/use-cases/
Any questions just shout! Happy to answer anything Snowplow-related, and thanks again for Hacker News' support of Snowplow over the years.
I chortled when I saw the health warning against railway-oriented programming. Reading that post deeply influenced (and continues to influence) the architecture of Snowplow (https://github.com/snowplow/snowplow) and I titled a chapter of Event Streams in Action 'Railway-Oriented Processing' (https://www.manning.com/books/event-streams-in-action). Thanks Scott.