HackerTrans
TopNewTrendsCommentsPastAskShowJobs

andreinwald

no profile record

Submissions

Show HN: WebGPU enables local LLM in the browser – demo site with AI chat

andreinwald.github.io
145 points·by andreinwald·12 miesięcy temu·54 comments

comments

andreinwald
·12 miesięcy temu·discuss
It works on small Llama-3.2-1B model, specially for less powerfull GPU devices
andreinwald
·12 miesięcy temu·discuss
Model used: Llama 3.2 1B (small). Quality hould be similar with running Ollama app with same small model.
andreinwald
·12 miesięcy temu·discuss
On my Android device works pretty fast.

But keep in mind that it's small Llama-3.2-1B model, specially for less powerfull GPU.
andreinwald
·12 miesięcy temu·discuss
Yes, its very similar to Ollama app, and Llama-3.2-1B model used
andreinwald
·12 miesięcy temu·discuss
Llama 3.2 1B
andreinwald
·12 miesięcy temu·discuss
Also here https://caniuse.com/webgpu
andreinwald
·12 miesięcy temu·discuss
Demo site is asking before download