HackerTrans
TopNewTrendsCommentsPastAskShowJobs

tybug

no profile record

Submissions

Property-Based Testing

tybug.dev
2 points·by tybug·上個月·0 comments

Swarm Testing

tybug.dev
2 points·by tybug·上個月·0 comments

Property-based testing is about to rule the (software) world

tybug.dev
2 points·by tybug·5 個月前·0 comments

comments

tybug
·3 個月前·discuss
We plan to rewrite hypothesis in rust and expose an FFI through that! This is a medium term plan (months, not weeks or years), but we're acutely aware that relying on a python component is not a long-term solution.
tybug
·3 個月前·discuss
Yes! I just wrote up documentation for the protocol earlier this week: https://hegel.dev/reference/protocol.

In reality, we hope to provide more guidance than this to people who want to write their own language frontend. This protocol reference doesn't talk about the realities of [hegel-core](https://github.com/hegeldev/hegel-core) and how to invoke it, for example.

We intend to write a "How to write your own Hegel library" how-to guide. You can subscribe to this issue to get notified when we write that: https://github.com/hegeldev/website/issues/3.

If you're eager, pointing your favorite LLM at https://hegel.dev/reference/protocol + https://github.com/hegeldev/hegel-rust and asking it to write you one for your language of choice should be enough to get you started!
tybug
·3 個月前·discuss
I didn't expect to see Hegel when opening up HN today! Feel free to ask any questions about it. We released hegel-go earlier this week, and plan to release hegel-cpp sometime next week, so look forward to that :)
tybug
·4 個月前·discuss
To put it on the record: my position is current models can't get us there, and neither can the next iteration of models, but in two model iterations this will be worth doing. There's a lot of fiddly details in Hypothesis that are critical to get right. You can get a plausible 80% port with agents today but find they've structured it in a way to make it impossible to get to 100%.
tybug
·4 個月前·discuss
`from_type` just supports a bunch more things than rust ever can due to the flexibility of python's type system. `from_type(object)` is amazing, for example, and not something we can write in rust.
tybug
·4 個月前·discuss
Yep, `#[derive(DefaultGenerator)]` and `generators::default<T>()` are the right tools here.

This is one of the areas we've dogfooded the least, so we'd definitely be happy to get feedback on any sharp corners here!

I think `from_type` is one of Hypothesis's most powerful and ergonomic strategies, and that while we probably can't get quite to that level in rust, we can still get something that's pretty great.
tybug
·4 個月前·discuss
I actually think there's another angle here where PBT helps, which wasn't explored in the blog post.

That angle is legibility. How do you know your AI-written slop software is doing the right thing? One would normally read all the code. Bad news: that's not much less labor intensive as not using AI at all.

But, if one has comprehensive property-based tests, they can instead read only the property-based tests to convince themselves the software is doing the right thing.

By analogy: one doesn't need to see the machine-checked proof to know the claim is correct. One only needs to check the theorem statement is saying the right thing.
tybug
·4 個月前·discuss
As possibly the one community on earth where it's actually better to post the code than the blog post: TL;DR this is a universal property-based testing protocol (https://github.com/hegeldev/hegel-core) and family of libraries (https://github.com/hegeldev/hegel-rust, more to come later).

I've talked with lots of people in the PBT world who have always seen something like this as the end goal of the PBT ecosystem. It seemed like a thing that would happen eventually, someone just had to do it. I'm super excited to actually be doing it and bringing great PBT to every and any language.

It doesn't hurt that this is coming right as great PBT in every language is suddenly a lot more important thanks to AI code!
tybug
·8 個月前·discuss
Appreciate the thoughts <3. I do think there might be stronger examples we could choose. Possibly json encode/decode..
tybug
·8 個月前·discuss
(Hypothesis maintainer here) If you have recommendations for a better example on the front page, I'd love to hear them! (I mean this entirely genuinely and non-sarcastically; I agree sorting can give misleading ideas, but it is also concise and well understood by every reader).
tybug
·8 個月前·discuss
(Hypothesis maintainer here)

Yup, a standard test suite just doesn't run for long enough for coverage guidance to be worthwhile by default.

That said, coverage-guided fuzzing can be a really valuable and effective form of testing (see eg https://hypofuzz.com/).