Simple on-device iOS AI demo app of TinyStories-1M(apps.apple.com)
apps.apple.com
Simple on-device iOS AI demo app of TinyStories-1M
https://apps.apple.com/us/app/tinystories-on-device/id6451497115
4 comments
yea, I was also surprised at the lack of apps. I guess we can take a shot at llama-7b, maybe a base model and a storytelling fine tune. we'll have to look into which compiler is best, I do want to tinker with the mlc toolchain. will post to HN when we get it working!
MLC and llama.cpp both have performant metal ports, and mlc has a demo ios app!
For story finetunes, I am out of the loop... You have to dig through HF by date to find them, for instance:
https://huggingface.co/zarakiquemparte/hermesboros-limarp-7b
https://huggingface.co/models?p=1&sort=modified&search=7b
You can also look through what people are hosting on Kobold Horde (a storytelling/rp community) ignoring the old pygmalion/opt models: https://lite.koboldai.net/#
Also, you should at least host an instruct model instead of vanilla llama 7B.
For story finetunes, I am out of the loop... You have to dig through HF by date to find them, for instance:
https://huggingface.co/zarakiquemparte/hermesboros-limarp-7b
https://huggingface.co/models?p=1&sort=modified&search=7b
You can also look through what people are hosting on Kobold Horde (a storytelling/rp community) ignoring the old pygmalion/opt models: https://lite.koboldai.net/#
Also, you should at least host an instruct model instead of vanilla llama 7B.
Hey all,
After reading the TinyStories paper (using small language models of 1m to 33m parameter size to generate short stories), me and a friend decided to port it as a test to see what performance of tiny LMs look like on mobile devices. The stories it generates aren't great or anything, it's really just a toy tech demo to play with. We used HuggingFace's exporters library to do this.
You can try it out here: https://apps.apple.com/us/app/tinystories-on-device/id645149...
(we are not affiliated with MS research or the authors)
After reading the TinyStories paper (using small language models of 1m to 33m parameter size to generate short stories), me and a friend decided to port it as a test to see what performance of tiny LMs look like on mobile devices. The stories it generates aren't great or anything, it's really just a toy tech demo to play with. We used HuggingFace's exporters library to do this.
You can try it out here: https://apps.apple.com/us/app/tinystories-on-device/id645149...
(we are not affiliated with MS research or the authors)
7B in particular should be fast and usable on most iOS devices. 13B should work on 8GB of RAM. Both are quite coherent and "smart."
Theres a tragic gap between these optimized implementations and mobile apps, for some reason.