Show HN: WebGPU enables local LLM in the browser – demo site with AI chatandreinwald.github.io145 points·by andreinwald·il y a 12 mois·54 comments
andreinwald·il y a 12 mois·discussIt works on small Llama-3.2-1B model, specially for less powerfull GPU devices
andreinwald·il y a 12 mois·discussModel used: Llama 3.2 1B (small). Quality hould be similar with running Ollama app with same small model.
andreinwald·il y a 12 mois·discussOn my Android device works pretty fast.But keep in mind that it's small Llama-3.2-1B model, specially for less powerfull GPU.