We needed a way to detect ingestion anomalies across 200+ microservices without building a separate ML stack. It turns out BigQuery already tracks every row written in WRITE_API_TIMELINE, and its built-in AI.DETECT_ANOMALIES (powered by Google’s TimesFM) can handle the detection.
I think it is great stuff you are building, especially for data integration of external sources that don’t offers webhooks or other pub/sub integration options. But is Airbyte really Open Source? I’m not questioning the reasons for moving to a source available license due to how AWS treated open source, but I think it is a bit of false marketing of Airbyte referring to it as open source. Sure, some pieces are MIT but the core is Elastic License 2.0 from what I can see. https://airbyte.io/blog/a-new-license-to-future-proof-the-co...