> What about ZeroMQ over UDP? A remote time service. Pulling from a git repo.
>
> No really, I'm asking because I'm trying to figure out how to deal with all this in my tests right now, and I honestly have no idea. But I doubt Codeparrot is going to be able to intercept and intelligently mock everything.
In theory, *everything* can be recorded and replayed; that's the magic of patching-in at the application layer. *Any* function call can be recorded, and, replayed.
In practice, we support whatever is demanded by our customers. If people are ready to pay for something, we are happy to build support for it.
Thanks Leonid! Your vote of confidence means a lot.
OTel for go requires user code changes. Languages that allow monkey-patching (java, js, python, etc.).
> I wonder if it can be done purely using OpenTelementry (e.g. you depend on typical OTel setup), and then read the data directly from OTel DB.
OTel doesn't work out of the box. OTel usually doesn't collect request or response for any network or db call. 90% of my time is spent on extending the individual agents' code; so that they can collect additional required information, and perform "replay".
> 2. Requests don't happen in a vacuum. They likely have data dependencies on prior requests. I recommend some way of sampling sessions rather than individual requests. Replaying the 3rd request in a series of 6 is likely just going to be exercising failure paths.
This is so true. I ran into this problem when I was trying to implement a kind of sampling that limited total RPS.
Thanks for reminding. I am definitely looking into more sophisticated methods of sampling.
> I didn't see any explanation on how results are determined. I think it's important to surface those types of details on the website. I'm not going to watch the video on it in hopes of learning.
> 1. what is the type of service instrumentation needed to capture the data? Wonder why this is needed when typically the data is already captured in an APM log? The instrumentation might add performance and security concerns.
Implementation is very similar to an APM log. So the same performance and security concerns apply. We are working on giving both at the same time (Automated tests, and APM), to reduce overhead.
> 2. what is the sampling logic to capture the traffic? It might compromise the fidelity of the test data and give a false sense of test accuracy.
It is random sampling. I feel, 1M or 10M randomly sampled requests should cover all cases.
> 3. what is the duration of data capture? Is it a week's or month's or quarterly data? Meeting 90% coverage on a week's production sample data will provide a false metric.
I was thinking 1 week should be enough. Maybe we will have to add some custom sampling logic for lesser frequency calls (like monthly crons).
> 4. can it faithfully handle data privacy and customer anonymization? This is critical for API's dealing with PCI and other sensitive data.
Yes. Additionally, for compliance, we offer a self-hosted solution- Our code runs on your servers and no data ever leaves your cloud / on-prem.
Starting with "I'm just curious", is a good way to make sure I'm asking.
"asking" can often come across as "questioning", especially in workplace chat (like on slack). As a manager I don't want to give this vibe to colleagues (reportees, peers, and seniors).