I'm on android so I've used termux+ollama, but if you don't want to set that up in a terminal or want a GUI pocketpal AI is a really good app for both android and iOS. It let's you run hugging face models.
I've run qwen3 4B on my phone, it's not the best but it's better than old gpt-3.5. It also does have a reasoning mode, and in reasoning mode it's better than the original gpt-4 and rhe original gpt-4o, but not the latest gpt-4o. I get usable speed, but it's not really comparable to most cloud hosted models.
Do you think someone will distill this or quantize it further than the current 4-bit from OpenAI so it could run on less than 16gb RAM? (The 20b version). To me, something like 7-8B with 1-3B active would be nice as I'm new to local AI and don't have 16gb RAM.
Does anyone think people will distill this model? It is allowed. I'm new to running open source llms, but I've run qwen3 4b and phi4-mini on my phone before through ollama in termux.