Slap the gpus in a car and offset the cost of ownership by supplying the grid for GPU power on the go. Either get paid in rebates or tokens. Contribute to a distributed training/inferencing network.
Anything to close Pandora's box. "They" liked the eras they could control the communications, and therefore the narrative. Boomers on their last legs, question is, will the future undo the unjustness that was forced upon them? Restore the rungs of the ladders that were removed, so they could have a chance too? Or are they going to stay in the fear narrative, and make this tragedy worse?
A self-contained CUDA inference engine for LiquidAI/LFM2.5-8B-A1B (hybrid conv + GQA-attention MoE, 8.5B params, 1B active) targeting a single RTX 3060 (12 GB) using flash-decoding. MIT license.
I wonder if "battelites" might be profitable. Like an pay-per-usage energy grid in space with battery backup that can beam power around to other satellites that might not have easy access to power, or have their power grids temporarily obstructed.
Terminal-based chat application powered by *locally installed* llama.cpp, featuring an auto-managed server backend, reasoning modes, and 7 built-in filesystem tools for interactive AI assistance with a focus on only allowing read only agentic access to system and only offline-focused agentic commands (for now). Powered by the g023/g023-Qwen3.5-9B-GGUF:IQ2_M model.
I've started creating https://github.com/g023/localmodelrouter/ which offers Ollama like functionality but as a single .py file with minimal dependencies and more focus on letting llama.cpp handle the dirty work.
HarnessHarvester generates executable Python harnesses from natural language task descriptions, executes them in a sandboxed environment, reviews them with multi-faceted LLM judges, and repairs failures using branching strategies. It includes two autonomous modes: autolearn (continuous discovery loop) and autoimprove (iterative enhancement of existing harnesses). This concept is designed to be an offline first harness/scaffolding builder where you get the harness instead of some remote api.
A single file, python based, minimal/recognizable dependencies, turboquant playground, barebones af, with some easy to access globals to experiment with at top of 'run_tquant.py'. Test model is a 1.77B model that I altered by duplicating a layer in a Qwen3 1.7B model. Probably work fine with the regular Qwen3 1.7B model as well, but for right now I'm just working with my surgically altered one while I work on the script.
I had some issues in the original, but had to jump away for a bit here to do some backups (weak). Anyways, I updated to make the necessary fixes, and also made some more tweaking values at top to play with and dialed in the params for the more this specific model a bit more. I will start testing with some other models here as my next step in this little experiment. Thanks for the interest. Feel free to try latest version and run the interactive mode to chat it up with the model and get a feedback on the results as you go. If you have any suggestions, let me know. I'm trying to keep this one as barebones as possible to make it easier for others to port to other languages, or integrate into other uses more easily.
edit: just added Mirostat v2 to clean up repetitive output from the model
A sophisticated multi-level reasoning engine with agentic memory, tool integration, and user control modes. Built as a (primarily) single-file Python program using vanilla Python and local LLM integration. Powered by Ollama API and utilizes a 1.77B Qwen3 variant that has layer 21 duplicated. Different layers of memory for agentic processes. Has ability to oversee command execution or yolo mode. MIT Licensed.
I'm thinking that the bubble will be the vortex caused by an abundance of power that becomes freely available locally due to the AI datacenters moving to space.
Something about the modern day fails to match the feelings of when MUDs were in their prime. With text you can describe so much more than a picture can paint. You can visualize a smell, a taste, or a feeling in text, but it doesn't translate well when you have graphics painting your imagination for you.