I tried hosting several small-size LLM models. It requires lots of work, but it’s worth it to customize my own sick chatbot. However, hosting a large model with billions of params is super painful and expensive.
Using a public service like chatGPT or Replicate is easier and more affordable but I am really concerned about data privacy. We all know some are collecting and using user data for training or other purposes. What makes you different from others?