PrivateGPT is a great starting point for using a local model and RAG.
Text-generation-ui, oogabooga, using superbooga V2 is very nice and more customizable.
I’ve used both for sensitive internal SOPs, and both work quite well. Private gpt excels at ingesting many separate documents, the other excels at customization. Both are totally offline, and can use mostly whatever models you want.
Not only is the demo funny, but this worked, surprisingly, as advertised. Had to restart the environment a few times for some reason. Not sure I understand the authors security concerns, but this is a fantastic early implementation.
So, I stumbled upon this Simple LLaMA FineTuner project by Aleksey Smolenchuk, claiming to be a beginner-friendly tool for fine-tuning the LLaMA-7B language model using the LoRA method via the PEFT library. It supposedly runs on a regular Colab Tesla T4 instance for smaller datasets and sample lengths.
The so-called "intuitive" UI lets users manage datasets, adjust parameters, and train/evaluate models. However, I can't help but question the actual value of such a tool. Is it just an attempt to dumb down the process for newcomers? Are there any plans to cater to more experienced users?
The guide provided is straightforward, but it feels like a solution in search of a problem. I'm skeptical about the impact this tool will have on NLP fine-tuning.
Chatgpt can generate pretty decent PLC structured text, IEC-61131 compliant code. Here, I’ve detailed how I generated an Analog Input processing Control Module. Rockwell IDE is used, but it will work for any similar processor.
I’ve used both for sensitive internal SOPs, and both work quite well. Private gpt excels at ingesting many separate documents, the other excels at customization. Both are totally offline, and can use mostly whatever models you want.