fat-apple · HackerTrans

We’re truly sorry that you felt let down! You raise a great point that is worth clarifying though. All logs, metrics and future traces are stored in S3/GCS (and yes metrics are stored in TSDB format). While this does not allow a single query language to ask questions across all these sources in one query (yet), it is absolutely possible to build what Datadog is with a new user interface. To go even further, now that it’s all in one place (S3/GCS) other technologies can be leveraged to create a higher level query language across everything, such as PrestoDB (one would have to build backends for it), or even a new dedicated open source columnar store.

fat-apple

há 5 anosdiscuss

Thanks for the great feedback, definitely understand where you're coming from and we'll keep that in mind moving forward! Very much appreciate the constructive criticism!

fat-apple

há 5 anosdiscuss

Thank you! We like to simplify but not obfuscate!

fat-apple

há 5 anosdiscuss

Thank you! Yes, design is very near and dear to our hearts! If you’re interested in giving me some early feedback on our UX, email me [email protected].

fat-apple

há 5 anosdiscuss

Thanks for your support! We’re still early in our journey when compared to the featureset Datadog has built over time, but we definitely aspire to bridge the best of Datadog and open source. Based on your experience, what sorts of features would you prioritize to get there?

fat-apple

há 5 anosdiscuss

Thanks for your feedback! As with many in the industry, we are trying our best to figure this out.

Our intention is to be really transparent with how we build and price software, which is why our commercial features will also be public in our repo, but commercially licensed. Transparency is critical in our opinion.

This is the model we’ve seen work for other highly impactful software projects.

We’ve created a ticket to track our addition of commercial code to our repo: https://github.com/opstrace/opstrace/issues/319

fat-apple

há 5 anosdiscuss

Curious to learn more about the tradeoffs you’ve found between Cortex and Victoria Metrics and why you could be drawn to one more than the other?

fat-apple

há 5 anosdiscuss

Thanks for that feedback - we've got a lot of work to do to make that more clear! We're still early in our journey so we’re not there yet, but we’re moving fast. We're working on a new collaborative UI for interacting with your data in a way that solves a lot of problems we've witnessed with current monitoring UIs (let me know if you want more detail). It's in early development and we haven't released it yet, so while Opstrace does have a UI now, it's currently limited to system management (adding/removing users and tenants). For interacting with data, we currently ship a Grafana instance per tenant. The roadmap has some basic information about this (might not be something you stumbled across). Let me know if I can clarify anything else. https://opstrace.com/docs/references/roadmap

fat-apple

há 5 anosdiscuss

We certainly don't want to create an adverse incentive where you would consider limiting the number of devs who had access to the monitoring system. There are trade-offs but we think that per-seat pricing (like GitLab, GitHub) actually does make it much easier to budget and plan for monitoring spend. Generally, a headcount plan is more predictable than the data your application generates. For example, a single engineer can add (and maybe should be adding) far more metrics and logs to their applications to monitor it correctly. They should not also be worried about breaking the budget when doing so. Does this make sense to you - what do you think?

fat-apple

há 5 anosdiscuss

Yes you can do this today. You cannot send this directly to Opstrace via grpc yet, so you would need to use the Prometheus remote write exporter (https://github.com/open-telemetry/opentelemetry-collector/bl...) with the open telemetry collector. Right now this means only metrics are supported but we will be working on traces as well. Check out the point about tracing on our roadmap https://opstrace.com/docs/references/roadmap

fat-apple

há 5 anosdiscuss

:-) I have talked about the subject in this comment thread: https://news.ycombinator.com/item?id=25991764

fat-apple

há 5 anosdiscuss

We’re about to release a Datadog compatible API so you can point your Datadog agent at Opstrace instead (stay tuned for the blog post). Our goal is to be able to tell you exactly how much data the agent is sending and how much that is costing you (and for example what services/containers are responsible for the bulk of the cost). Here’s a list of the PRs: https://github.com/opstrace/opstrace/pulls?q=is%3Apr+is%3Acl...

fat-apple

há 5 anosdiscuss

Currently you can only deploy to AWS and GCP, but we do intend to extend support to on-prem/dedicated servers in due course (see https://news.ycombinator.com/item?id=25992237). Until now we’ve been focussing completely on building a scalable, reliable product by standing on the shoulders of these cloud providers where we can take advantage of services like S3, RDS, and elastic compute.

We've done a deep dive into the cost model for metrics and posted more about it here: https://opstrace.com/blog/pulling-cost-curtain-back. We are still working on a full cost analysis for logs - I'd be happy to send it to you once we have it (feel free to email me [email protected] to chat about your use case). Our goal is to be super transparent (see https://news.ycombinator.com/item?id=25992081) with cost and to have a page on our website that helps someone determine what to expect (probably some sort of calculator with live data). Our UI will also show you exactly what your system is currently costing you with some breakdown for teams or services so you know who/what is driving your monitoring cost. We're doing user testing on our to-be-released UI now and would love to have people like yourself give us early feedback (since you mentioned the BigQuery interface).

fat-apple

há 5 anosdiscuss

Great questions!

(1) As it stands today, you can already use https://vector.dev/docs/reference/sinks/prometheus_remote_wr... to write metrics directly to our Prometheus API. You can also use https://vector.dev/docs/reference/sinks/loki/ to send your logs to our Loki API. Vector is very cool in our opinion and we’d love to see if there is more we can do with it. What are your thoughts?

(2) As for cost, our super early experiments (https://opstrace.com/blog/pulling-cost-curtain-back) indicate that ingesting 1M active series with 18-month retention is less than $30 per day. It is a very important topic and we've already spent quite a bit of time on exploring this. Our goal is to be super transparent (something you don’t get with SaaS vendors like Datadog) by adding a system cost tab in the UI. Clearly, the cost depends on the specific configuration and use case, i.e. on parameters such as load profile, redundancy, and retention. A credible general answer would come in the shape of some kind of formula, involving some of these parameters -- and empirically derived from real-world observations (testing, testing, testing!). For now, it's fair to say that we're in the observation phase -- from here, we'll certainly do many optimizations specifically towards reducing cost, and we'll also focus on providing good recommendations (because as we all know cost is just one dimension in a trade-off space). We're definitely excited about the idea of providing users useful, direct insight into the cost (say, daily cost) of their specific, current Opstrace setup (observation is key!). We've talked a lot about "total cost of ownership" (TCO) in the team.

fat-apple

há 5 anosdiscuss

Mat here (Seb's Cofounder). Great question. We are not only building a piece of infrastructure but a complete product with its own UI and features, rather than a standalone API. Our customer is the end-user more than the person wanting to build on top of it. GitLab and others have shown that when you do that the probability of being forked or just resold goes down drastically.

comments