I recently came across the existence of Penrose string diagram after curiously and naively researching the existence of index free notation for linear algebra. This seems to be a very interesting paper in the very same category of things I'd love to study and learn about but probably won't ever find the time!
Is a paper that publishes a 0.01% improvement of something at the cost of 5 times more power really an improvement? I believe that every single computer science measurement metric should have Joules or Watts in the denominator. If you are training a model I want to see performance per total energy consumed. If you are measuring inference accuracy, measure PER WATT.
I've always been a bit confused by the apparent tendency of the computer science field to mostly ignore energy and power. We are too often satisfied with the idea that software and programs exist in a perfect whiteboard world of xkcd 505 abstract compute.
I work on differentiable geometric optics with PyTorch. Seeing a list like this is really illustrative of the power that PyTorch provides when you start considering it like a general purpose GPU-enabled state of the art numerical optimization framework.
One thing I wonder is why no one has made a fork of PyTorch yet that removes all the API surface that doesn't produce GPU friendly code. Make dtype and device arg mandatory without defaults, remove in place operations that trigger a CPU sync, etc. This would increase confidence that written code will run on the GPU and pass torch.export() on the first try.
Thanks! Arbitrary precision arithmetic is definitely something I'd like to learn more about, yeah. Haven't had time to study it so much yet unfortunately.
Interesting! I'm not familiar with IEEE 1788. The TypeScript library (not-so-float) that I wrote which powers the calculator uses the JS Number type which is double precision IEEE 754. Outward rounding is not supported by JS so I used a bit level manipulation hack by casting to TypedArray [0] to implement the equivalent of C's nextafter() function. Otherwise I mostly followed Hickey & van Emden paper which is really delightful [1]. The real hard work is actually generating all the test cases. Good luck getting 100% test coverage on interval division!
I don't handle it, ahah. You are right that if you take any classical numerical computing algorithm and replace the floating point reals by interval unions, most of the time the number of intervals in the unions in each of your variables will grow very fast. This is one of the problems of unions and as far as I'm aware it's a topic of active academic research.
Yeah it's super interesting. Like you said, I learned that the IEEE 754 spec actually requires that complete implementations of floating point numbers expose a way to programmatically choose the rounding mode. As far as I know only C allows you to do that, and even then it depends on hardware support. For JS I had to use ugly typedarray casts. Which kinda only accidentally work due to endianess. But technically there should be an API for it!
There's other unused stuff in IEEE 754 like that: the inexact bit or signaling NaNs!
It's possible to support that but it makes the code very very much more complicated. I've decided early on to not support it. Would be a cool addition though!
I wonder if we could design a programming language specifically for teaching CS, and have a way to hard-exclude it from all LLM output. Kinda like anti virus software has special strings that are not viruses but trigger detections for testing.
This would probably require cooperation during model training, but now that I think of it, is there adversarial research on LLM? Can you design text data specifically to mess with LLM training? Like what is the 1MB of text data that if I insert it into the training set harms LLM training performance the most?
I've had a lot of enjoyment flipping the agentic workflow around: code manually and ask the agent for code review. Keeps my coding skills and knowledge of the codebase sharp, and catches bugs before I commit them!
Author here. Outward rounding to combat precision issues is what interval arithmetic is most known for (try 0.1+0.2 with "full precision mode" enabled), but that's really a shame in my opinion. Outward rounding is cool, but the "inclusion property", as it's known in research papers, works at every scale! This is what enables things like:
50 * (10 + [-1, 1])
[450, 550]
which is lovely, I think. Adding the union layer to it enables even cooler things, like the true inverse of the square function. Did you know it's not sqrt? Try 'sqinv(64)'.
I made interval calculator actually mostly as a way to test my implementation of interval union arithmetic [0], which I needed for another project: a backwards updating spreadsheet [1][2].
Working with tensor datatypes in numerical computing, I've been wondering if it would be possible to somehow add an extra dimension to tensors that would serve as the "floating point precision" dimension, instead of a data type. After all why couldn't the bit depth be one of the tensor dims? Maybe it would be possible to implement arbitrary floating point precision that way?
This is very interesting! I made an interval arithmetic calculator a few months ago [0]. I wonder if you could combine the two techs to make something even more powerful.
You'd be surprised. It really depends on how you define the problem and what your goal is. My goal with bidicalc what to find ONE solution. This makes the problem somewhat possible since when there are an infinity of solution, the goal is just to converge to one. For example solving 100 = X + Y with both X and Y unknown sounds impossible in general, but finding one solution is not so difficult. The idea is that any further constraint that would help choose between the many solutions should be expressed by the user in the spreadsheet itself, rather than hardcoded in the backwards solver.
> What kind of problems can you solve backwardly?
This is the weakness of the project honestly! I made it because I was obsessed with the idea and wanted it to exist, not because I was driven by any use case. You can load some premade examples in the app, but I haven't found any killer use case for it yet. I'm just glad it exists now. You can enter any arbitrary DAG of formulas, update any value, input or output, and everything will update upstream and downstream from your edit and remain valid. That's just extremely satisfying to me.
I also implemented a spreadsheet last year [0] in pure TypeScript, with the fun twist that formulas also update backwards. While the backwards root finding algorithm was challenging, I also found it incredibly humbling to discover how much complexity there is in the UX of the simple spreadsheet interface. Handling selection states, reactive updates, detecting cycles of dependency and gracefully recovering from them is a massive state machine programming challenge! Very fun project with a lot of depth!
I myself didn't hand roll my own parser but used Ohm-js [1] which I highly recommend if you want to parse a custom language in Javascript or TypeScript.
> One way of doing this is to keep track of all dependencies between the cells and trigger updates when necessary. Maintaining a dependency graph would give us the most efficient updates, but it’s often an overkill for a spreadsheet.
On that subject, figuring out the efficient way to do it is also a large engineering challenge, and is definitely not overkill but absolutely required for a modern spreadsheet implementation. There is a good description of how Excel does it in this famous paper "Build systems a la carte" paper, which interestingly takes on a spreadsheet as a build system [2].
Mindshare death is a very large overstatement given the massive amount of legacy C++ out there that will be maintained by poor souls for year to come. But you are right, there used to be a great language hiding within C++ if the committee ever dared to break backwards compat. But even if they did it now it would be too late and they'd just end up with a worse Rust or Zig.