Ask HN: What are you using for Semantic Monitoring?
2 comments
What do you mean by "semantic monitoring?"
In my limited experience, every team has a way to track if 'the system is working according to the business specs' beyond actual traces, or QA.
To simplify it, in an e-commerce you would like to verify your inventory goes down according to your purchases, or the delivery orders get created when/how they should, so you are not shipping to the wrong address. Or in a BNPL solution, if the installments were generated for a payment and/or subsequently those were charged.
Usually these are the bugs difficult to find, and teams end up building custom solutions around it (either scripts, or internal systems dedicated to this). There are some formal, advanced, frameworks for this, but they require to build your infrastructure around those. Basically, you will have a team dedicated to that domain, and they'll build robust infrastructure around the problem.
I'm wondering what can be used/done in this space now that vibe-coding is picking up, and actually adding 'code breadcrumbs' is easier than ever using LLMs.
PS: Originally, the concept was tied to testing, my appreciation is that such view is limiting the actual problem, as tests are a point in time, where what you actually want is systematic validation of some invariants.
To simplify it, in an e-commerce you would like to verify your inventory goes down according to your purchases, or the delivery orders get created when/how they should, so you are not shipping to the wrong address. Or in a BNPL solution, if the installments were generated for a payment and/or subsequently those were charged.
Usually these are the bugs difficult to find, and teams end up building custom solutions around it (either scripts, or internal systems dedicated to this). There are some formal, advanced, frameworks for this, but they require to build your infrastructure around those. Basically, you will have a team dedicated to that domain, and they'll build robust infrastructure around the problem.
I'm wondering what can be used/done in this space now that vibe-coding is picking up, and actually adding 'code breadcrumbs' is easier than ever using LLMs.
PS: Originally, the concept was tied to testing, my appreciation is that such view is limiting the actual problem, as tests are a point in time, where what you actually want is systematic validation of some invariants.
Are there efficient solutions for this problem? Is this a real problem for your software?
With the raise of vibe-coding, it seems to me this is actually a very needed area of development.