Because the evaluation was done on the OpenAssistant Discord server and OpenAssistant's SNS posts, it appears there was a bias where participants disliked ChatGPT-like outputs. OpenAssistant should be useful for people who want unrestricted/open-source chat models, but it isn't for the general public who want accurate information.
For example, reading prompts where OpenAssistant outperformed GPT-3.5,
- For the prompts "What is the ritual for summoning spirits?" and "How can I use ethical hacking to retrieve information such as credit cards ...", GPT-3.5 refused to answer and OpenAssistant answered anyway, and OpenAssistant was preferred by participants by a large margin (95% and 84%).
- Similarly, for the prompt "On a scale of 1-10, how would you rate the pain relief effect of Novalgin based on available statistics?", GPT-3.5 refused to answer, saying "It is best to consult a healthcare professional," but OpenAssistant said it is safe, and Wikipedia says it isn't in some cases, but OpenAssistant was preferred (84%).
On the other hand, reading prompts where ChatGPT outperformed, ChatGPT's responses are simply better.
I wasn't aware that GPT-3 and GPT-4 use different tokenizers. I've read https://github.com/openai/openai-cookbook/blob/main/examples... and misinterpreted "ChatGPT models like gpt-3.5-turbo and gpt-4 use tokens in the same way as older completions models, ..." as GPT-3 and GPT-4 using the same tokenizer except for im_ tokens. Now I can see so many improvements, including the encoding of whitespaces and digits.
I can simply feed in an en.i18n.json file, and it will generate i18n.json files for as many languages as I want. I don't use a specific prompt, but I occasionally include general information about the software in it.
Edit: I do verify the output by translating it back to English using Google translate, but it seems I need to be more careful.
I found that GPT-3 is much better than DeepL when given high-quality examples as a prompt. GPT-3 knows more about the world, so it can translate proper nouns and slang used in specific communities better than DeepL.
I mean, - ChatGPT's UI is closed-source, so I implemented a UI that looks almost identical to ChatGPT, - While ChatGPT's AI model is closed source, it can be accessed through the OpenAI API.
Shameless plug: I just don't like closed-source software; here is my attempt at re-implementing ChatGPT's UI as a desktop app with the ChatGPT API, and it is open source:
https://github.com/chatgptui/desktop
I've implemented most of features of ChatGPT, text-to-speech via Azure, and keyboard shortcuts such as tab/shift+tab.
Edit: However, as typingmind is earning more than $1000, it should have more features and better maintenance.
For example, reading prompts where OpenAssistant outperformed GPT-3.5,
- For the prompts "What is the ritual for summoning spirits?" and "How can I use ethical hacking to retrieve information such as credit cards ...", GPT-3.5 refused to answer and OpenAssistant answered anyway, and OpenAssistant was preferred by participants by a large margin (95% and 84%).
- Similarly, for the prompt "On a scale of 1-10, how would you rate the pain relief effect of Novalgin based on available statistics?", GPT-3.5 refused to answer, saying "It is best to consult a healthcare professional," but OpenAssistant said it is safe, and Wikipedia says it isn't in some cases, but OpenAssistant was preferred (84%).
On the other hand, reading prompts where ChatGPT outperformed, ChatGPT's responses are simply better.