But it's not clear that using Sonnet or any other LLM as a "grader" would result in the same improvement. For objective grading, you could be sure that the additional adaptive support is helping. For subjective things like writing style, literature, poetry, you end up with whatever Sonnet thinks is good (and randomly so).
It still could be better for students, but it's not obvious that it would be (or maybe not as strongly?).
Do you have a larger study planned for the Fall? It definitely seems promising.
I'm curious how well you feel this worked because the subject was Statistics (objective grading) versus something more subjective like Civics or Literature.
This is nice work, but I found the bug finding example to be weird:
> One such bug was in the sign function for zigzag decoding of the datrs/varinteger library. On input Std.U64.MAX, the expression (value + 1) overflowed, causing crashes in debug mode and silent corruption in release mode—an edge case that testing and fuzzing would typically miss.
In what way would this boundary condition case be considered something that "testing [...] would typically miss"? It's certainly something that bad tests would miss or not think about, but I find that (a) careful people and (b) ML coding systems are actually really good at "oh, I should test the extreme values". Especially for things that parse user input.
I'm curious if they found other bugs that were more interesting, but found them too hard to explain quickly.
> Some of the newly added features may come as a surprise to those of you who keep a close track of ZLUDA development. Most of them were previously explicitly outside ZLUDA's roadmap. There has been a change of plans. ZLUDA development is no longer commercially funded, so it's back to being my weekend project. This means that the priority is no longer what makes commercial sense, but what I find the most entertaining. That's why the sudden addition of textures, PhysX and better Windows support.
Outside of the time commitment that full-time support would offer, I generally think focusing on amusement is a good life strategy.
Technically, we did RTSL before Ingo and then Matt did their stuff at Intel. LLVM wasn't particularly usable in 2007, but Austin and I did try it (this isn't in the paper!). A few years later, I did an actual LLVM JIT for OSL, but we didn't have enough coherence for vectorized shading at the time at Sony.
But none of it was surprising. The original RenderMan shading language was "vectorized" and it even used SIMD instructions on modern processors to run the "interpreter loops". That is, a single "color add" in RSL might have looked like:
for (int i=0; i < grid_points; i++) {
out_color[i] += foo;
}
and the inner part there could use vector instructions. That just isn't nearly enough to get useful wins.
The point of ISPC et al. was to give people CUDA-like easy mode for "trust me, just vectorize the whole thing and deal with the masking for me". It goes beyond a hardcoded shading language (easier target!) though didn't reach as far as CUDA with complex structs / C++ capabilities.
I'm not trying to convince you to stay (I work for neither anymore!), just wanted to note that you can technically request a waiver. I'm not sure how this works in practice though. Like, if you want to leave Athena and move to something on-premise is that enough to have just that workload? Maybe!
Edit: I also didn't follow this at the time, but the AWS wording suggests that the "EU Data Act" is also involved.
Yeah, but that missing context is super important.
If they want it for local dev work, that's pretty different from wanting a high-performance air gapped object store without rewriting clients.
They seem to know what they're doing (having complained about a methodology problem in MinIO), and yet don't personally want to throw their hat in the ring not maybe pay anyone...
"Designing AI for Disruptive Science" is a bit market-ey, but "AI Risks 'Hypernormal' Science" is just a trimmed section heading "Current AI Training Risks Hypernormal Science".