Getting ROCm working was... an adventure. We documented the entire (painful) journey in a detailed blog post because honestly, nothing went according to plan. If you've ever wrestled with ROCm setup for ML, you'll probably relate to our struggles.
The good news? Everything works smoothly now! We'd love for you to try it out and see what you think.
One of the maintainers here. Yes this is a big part of our plans. In addition to our plugin system which allows arbitrary Python scripts, we will soon publish how to add decorators to any existing script which can be run externally but logged into Transformer Lab. So you could do training anywhere but trigger evals in the app, for example.
Apple MLX is a game changer for what is possible in Local LLM development for everyone. Getting this to work as a single application that "just works" across platforms has been one of the hardest engineering problems we've ever worked on, but we're determined to get it right.
The functionality in Transformer Lab comes from plugins. Plugins are just Python scripts behind the scenes. So anything that can be done in Python can be done as a plugin.
Right now we have export plugins for going to GGUF, MLX and LlamaFile but if you know a good library for exporting to TensorRT, let's make a plugin for this! (Feel free to join our Discord if you want help)
But this article from 1990 (may be outdated) states that a misunderstanding of the public is thinking that scrap mostly comes from old devices or planes. But these types of things last long and don't get recycled enough to provide enough volume. This seems to say that the bulk of scrap comes as a byproduct of the smelting process:
The following article from 2004 however says that recycled Ti comes from 10% old scrap (old recycled parts we normally think of as recycled metal) and 90% "new scrap" which is partly wasted Ti generated when making parts. For example in 2004, the aerospace industry used 12000t of Ti but wasted 10000t of that.
Some of the tools offer a path to doing tool use (fetching URLs and doing things with them) or RAG (searching your documents). I think Oobabooga https://github.com/oobabooga/text-generation-webui offers the latter through plugins.
You can chat with the models directly in the same way you can chat with GPT 3.5.
Many of the opensource tools that run these models let you also edit the system prompt, which lets you tweak their personality.
The more advanced tools let you train them, but most of the time, people are downloading pre-existing models and using them directly.
If you are training models, it depends what you are doing. Finetuning an existing pre-trained model requires lots of examples but you can often do a lot with, say, 1000 examples in a dataset.
If you are training a large model completely from scratch, then, yes, you need tons of data and very few people are doing that on their local machines.
If you're able to purchase a separate GPU, the most popular option is to get an NVIDIA RTX3090 or RTX4090.
Apple Mac M2 or M3's are becoming a viable option because of MLX https://github.com/ml-explore/mlx . If you are getting an M series Mac for LLMs, I'd recommend getting something with 24GB or more of RAM.
If you are looking for a downloadable app as an alternative to Oobabooga / Sillytavern, we built https://github.com/transformerlab/transformerlab-app as a way to easily download, interact, and even train local llms.
The good news? Everything works smoothly now! We'd love for you to try it out and see what you think.