devetec · HackerTrans

5 năm trướcdiscuss

If you learn programming from Stack Overflow and Github, and then repeat something that you learned over your time at reading, that's not just copying text. That's having learned the text. You could say the human brain is mashing together several different code examples, taking some text from each.

devetec

5 năm trướcdiscuss

It has been cleared up! https://towardsdatascience.com/the-most-important-supreme-co...

devetec

5 năm trướcdiscuss

You can train models on copyrighted materials. https://towardsdatascience.com/the-most-important-supreme-co...

devetec

5 năm trướcdiscuss

Distributed GPT training doesn't really work (sadly). Maybe someone will get a good solution to that, though.

devetec

5 năm trướcdiscuss

There is a FOSS version of GPT-3 being worked on by EleutherAI, but they are only at 6 billion parameters so far, while the largest model of GPT-3 is 175 billion (more parameters is better (usually)). EleutherAI is getting more compute from CoreWeave to actually train the GPT-3-like FOSS model, so that's something to look forward to eventually :). Github Copilot uses Codex though, which seems to be a GPT model trained on just code by OpenAI. It wouldn't be too hard to train a FOSS version of Codex on the open-source code of Github and other sources.

comments