I feel like what happens is first they release a giant model. Then they start optimizing the model to increase inference speed and reduce costs.
But then they introduce bugs leading to lots of complaints. Usually the complaints are about models being dumbed down but almost always the labs say they are doing no such thing.
So I am thinking if we believe the labs then they have a very error prone optimization effort going on.
Document parsing is top of my mind lately because in some of the areas we work on the bottleneck is starting to become being able to query documents the same way one queries an api.
I keep thinking the most obvious analogue is we need some way to represent documents the same way we can represent structured data in parquet. Parquet allows easy range bases queries and there is so much tooling built around Arrow.
But for documents I keep hitting a wall to figure out what the right abstractions are. Parquet allows filterable metadata. But what such metadata is there for documents. Then there is the arbitrrariness of chunking, vectorization.
If we could just do this in a 2 step process where every document to process can be represented in a parquet like data format then I think we will atleast have the semblance of a solution.
I see Netflix pumping out tech articles but can't help but notice how much worse the UI experience is getting. Video erroring out, general slowness etc.
I am looking forward to testing out these for agent automation. I kept trying to figure out what the ideal infrastructure for this purpose is and right now from just reading about it, micro VM's seem ideal.
But this is all hypothetical until I get to test it out.
Not to take anything away from the work which is indeed useful but I found the article bizarre because of what references were used and what were omitted.
The author talks about Intermediate Representation and then goes on about biology without mentioning even once compilers. Things like LLVM IR and ML IR literally have the term IR in them and what the author is trying to do seems closer in spirit to those than anything else.
Then the actual modeling in the IR was done with the concept of Blocks. Which seems very similar to the concept of Blocks as used in Notion. And yet no reference to it either.
> re-factoring a big repo of decades old fortran+C cod
Having been in academia in the past and now in software I can say with a lot of certainty that this will take a lot more upfront work than otherwise.
Academic code does not have a lot of structure. And usually lacks a lot in terms of tests. While AI is best when it can mimic patterns as well as there are tests to target.
So you will probably need to budget a few weeks to establish good patters, docs as well as testing patterns before you can seriously make it really do what you want it to do.
I love using Homebrew but I wish there was more support for pinning. I recently setup a new remote VM and tried to use a Brewfile for my setup. Turns out I cannot pin Neovim and so had to force upgrade my setup to 0.12.
I am more used to uv than pixi or mise so it would be an easier addition to my workflow.
However I do think it would probably be nicer if this kind of approach used conda packages as a source of truth. So kind of like pixi but without pixi!
I always feel like I want to use something like this, but then realize my NeoVim setup + tmux + Ghostty is good enough and I am not ready to learn a whole another system for modest gains.
The friction I have currently is obviously things like port forwarding, in app browser etc.
I keep thinking to try out cmux but haven't gotten around to it.
Funny that its hosted on vercel. Probably because its employee driven rather than top down. Saves all the bureaucracy to get someone to sign a budget item to buy a domain.