It's a side-project from our consultancy work. We're two deep technologists and so far entertaining the notion that we're very bad at (product) sales. But we're trying to learn that now.
The basic idea is to make Regex-scans so fast/cheap that "a metric" can be anything numeric in the text and "tracing" is useless because you can just log (and filter) more things. Turns out Regex at >200GB/s solves a lot of problems.
Metric cardinality explosion is immediately a non-issue, histograms have arbitrary resolution, and you can get from histogram pixels back to the underlying logs. And no need to instrument everything thrice for logs, metrics and traces.
The next big feature I'm aiming for is needle-in-a-haystack searches. The data block headers support it already, but the scan engine doesn't yet use it.
Nah, it's fine. Storage of raw logs is pretty cheap (and I think this is widely assumed). For querying, two problems arise:
1. Query latency, i.e. we need enough CPUs to quickly return a result. This is solved by horizontal scaling. All the idle time can be amortized across customers in the SaaS setting (not everyone is looking at the same time).
2. Query cost, i.e. the total amount of CPU time (and other resources) spent per data scanned must be reasonable. This ultimately depends on the speed of the regex engine. We're currently at $0.05/TB scanned. And metric queries on multi-TB datasets can usually be sampled without impacting result quality much.
The point that the trinity of logs, metrics and traces wastes a lot of engineering effort to pre-select the right metrics (and labels) and storage (by having too many information triplicate), is a good one.
> We believe raw data based approach will transform how we use observability data and extract value from it.
Yep. We have built quuxLogging on the same premise, but with more emphasis on "raw": Instead of parsing events (wide or not), we treat it fundamentally as a very large set of (usually text) lines and optimized hard on the querying-lots-of-text part. Basically a horizontally scaled (extremely fast) regex engine with data aggregation support.
Having a decent way to get metrics from logs ad-hoc completely solves the metric cardinality explosion.
It's a side-project from our consultancy work. We're two deep technologists and so far entertaining the notion that we're very bad at (product) sales. But we're trying to learn that now.