Just pushed a major update: a full translation layer is now live. You can serve any HTML file and automatically translate it on the fly with a local LLM. No client JS, no cloud, no bullshit. Looking for testers, bug hunters, and feedback from anyone running Apache or interested in offline-first web tech. Supports 46 languages with reliable LLM coverage (curated for quality, not just quantity).
I’ve been building this in my spare time over the past few days. mod_muse-ai is an Apache module written in C that lets you serve HTML pages generated by an AI model (local or remote). It reads `.ai` files, merges in a system prompt and a layout prompt, and sends the request to either a local Ollama instance or any OpenAI API-compatible endpoint (like LM Studio, Mistral, etc.), then streams the HTML response back to the browser.
The goal is to eliminate the need for a separate backend for AI-generated content. All feedback welcome—especially on the architecture and config design.
It offloads to system memory, but since there are "only" 3 Billion active parameters, it works surprisingly well.
I've been able to run models that are up to 29GB in size, albeit very, very slow on my system with 32GB RAM.
I have an RTX 3070 with 8GB VRAM and for me Qwen3:30B-A3B is fast enough. It's not lightning fast, but more than adequate if you have a _little_ patience.
I've found that Qwen3 is generally really good at following instructions and you can also very easily turn on or off the reasoning by adding "/no_think" in the prompt to turn it off.
The reason Qwen3:30B works so well is because it's a MoE. I have tested the 14B model and it's noticeably slower because it's a dense model.