Just a passerby, but wanted to say thanks for your work. Ory services are a delight and I was excited to see them spring up years ago and even more excited to see them continue to be developed and put to good use!
It is surprising for me to see these features finally being added to Node after such a long time. Especially so when I remember reading discussion after discussion about how something like this wasn't possible. I touched on this in a blog post some time ago [1]. Glad Node is catching up.
I have been wondering about the use of diffusion techniques for text generation, it is nice to see Google release a model that, seemingly, validates some thoughts I had.
Most folks I have seen experimenting with AI are either using a paid service or running high-grade hardware (even if consumer-level). The best I have in my current repertoire is a 5700XT and am not able to upgrade from that yet. The limitation, though, has at least also given some more significant insights into the shortcomings of current models.
Model sizes have gotten quite large and coherence seems to mostly have scaled with the density of a model, leaving the smaller models useful for only smaller tasks. Context size is also extremely important from my experiments with long-running dialogues and agent sessions, but a smaller GPU simply cannot fit a decent model and enough context at the same time. I do wonder if diffusion techniques will allow for a rebalancing of this density-to-coherence connection, letting smaller models produce chunks of coherent text even if limited by context. From my viewpoint it seems it will. Mixed tool call + response outputs also have the potential to be better.
Speed is also another problem I, and everyone else, has had with modern LLMs. The nature of cycling around the input with a new additional output each time is time consuming. On an older GPU with no AI-specific hardware it is an eternity! Being able to at least track 0-100% progress state would be an improvement from the current solution. At the moment one must simply wait for the LLM to decide to stop (or hit the max number of inference tokens). I am hopeful that, even on lower-end GPUs, a diffusion model will perform slightly better.
This does now beg several questions. If we are processing noise, where does the noise come from? Is there a good source of noise for LLMs/text specifically? Is the entire block sized beforehand or is it possible to have variable length in responses?
Lix is excellent. It is already faster (parsing), safer (better defaults, removed footguns), and easier to use (better errors, etc) than Nix. If anyone wants to get started using Nix then I highlight recommend you install Lix from the link in the parent comment.
For those unfamiliar with the game I would highly recommend it if you are interested in CRPG games with excellent writing. There is a lot of text in this game, but with the most recent version of the game most of it is voice acted. Many lines will stick with you later. It's rare to not be taken by something in the game, as expansive as it is.
For a more general description of the game: you are a detective, you must solve the case, and your fractured psyche will not let you do it alone.
The same is true for me on Safari. I've tried with and without AdGuard and the experience on plenty of sites that I just want to get a quick tidbit from is awfully painful.
Pikuma is excellent! I have enjoyed going through the videos on the YouTube channel to help understand some 3d graphics rendering techniques. Everything that I have seen has been as easy to digest as this post. Highly recommend checking out the rest of the catalogue if this kind of content is interesting to you.
As someone who grew up with iPhones coming out around the time I was in middle school and apps producing noises like this were used as pranks in class I have to ask that people don't do this. The sound is painful.
I went through it a few times in a day and felt confident enough to be able to solve the Morse puzzles on my own in Keep Talking and Nobody Explodes when playing with friends. The site made it pretty easy to pick up!
This has been a tough one to figure out. I do not think there is a great answer here. GitHub has a large network of people and the barrier to contributing there is fairly low. Cost is also subsidized greatly by GitHub. Nixpkgs being one of the largest and most active repositories on the entire platform would certainly strain any self-hosted or otherwise unprepared service. However, the benefits of a FOSS git forge with commitments to federation and values similar to this project also exist. Perhaps the answer here is to begin on GitHub to ease initial contributions and concerns over cost and to later transition to something like Codeberg.
This page was created by me. I have tried to capture what I feel are important values that are hopefully shared by others who are leaving or have left Nix. With the values, goals, and roadmap clearly stated, others can join to contribute. Part of the reason I felt this was important was to make sure that anyone who would want to contribute did not feel like there would be some rug pull moment. I wanted to outline exactly what was going to happen and how it was going to be better than what we have currently with Nix so that we can build that vision together.
Though, this is not an effort that can be undertaken by one person. My goal is publishing this page was to create a set of values, goals, and a roadmap for people (like myself) who would otherwise be leaving the Nix ecosystem due to its shortcomings. These values, goals, and roadmap can then be used for alignment to actually make it a reality. I worried that if these things weren't specified up front that people would instead continue arguing in circles about past problems rather than building a solution.
The last example reminds me of the once-proposed html imports feature. It was being pushed by Google back when Polymer was a thing and would've given us actual web components delivered as html documents with full support for html, css, and js just like we use normally. If I remember correctly there were a few issues with the approach that needed to be resolved and nobody stepped up to make it a reality. So instead we got the custom elements spec years later which only implemented a small percentage of what made html imports useful.