Almost everyone I know in the dev community agree that both generative art and code is fine. Most people don't care about copyright, to say the least. At least the people I know in real life and chat with online.
Only thing I've seen was artists on Reddit and Twitter getting angry about the AI art.
Abstract: The widespread adoption of large language models (LLMs) makes it important to recognize their strengths and limitations. We argue that in order to develop a holistic understanding of these systems we need to consider the problem that they were trained to solve: next-word prediction over Internet text. By recognizing the pressures that this task exerts we can make predictions about the strategies that LLMs will adopt, allowing us to reason about when they will succeed or fail. This approach - which we call the teleological approach - leads us to identify three factors that we hypothesize will influence LLM accuracy: the probability of the task to be performed, the probability of the target output, and the probability of the provided input. We predict that LLMs will achieve higher accuracy when these probabilities are high than when they are low - even in deterministic settings where probability should not matter. To test our predictions, we evaluate two LLMs (GPT-3.5 and GPT-4) on eleven tasks, and we find robust evidence that LLMs are influenced by probability in the ways that we have hypothesized. In many cases, the experiments reveal surprising failure modes. For instance, GPT-4's accuracy at decoding a simple cipher is 51% when the output is a high-probability word sequence but only 13% when it is low-probability. These results show that AI practitioners should be careful about using LLMs in low-probability situations. More broadly, we conclude that we should not evaluate LLMs as if they are humans but should instead treat them as a distinct type of system - one that has been shaped by its own particular set of pressures.
Criticisms of Elon Musk are not solely based on ideological differences, but also on concerns regarding his behavior, management style, and the working conditions at his companies, which some believe warrant scrutiny irrespective of his personal beliefs. He just doesn't feel like a good person and isn't relatable. This is why I think people hate him.
I suspect they are trying to compete in the console market without having a console per se. AFAIK the iPhone 15 Pro has 1/3rd of the TFLOPs of the Playstation 5. They also have a smooth TV-connectivity system.
Most people who have consoles in the US already have an iPhone in their pocket. Now you just need a controller, some kind of dock for the phone, and pay off a lot of devs to make games for the platform to get the ball rolling.
"With FLASHATTENTION (Dao et al., 2022), there is negligible GPU memory overhead as we increase the sequence length and we observe around 17% speed loss when increasing the sequence length from 4,096 to 16,384 for the 70B model."
"For the 7B/13B models, we use learning rate 2e−5 and a cosine learning rate schedule with 2000 warm-up steps. For the larger 34B/70B models, we find it important to set a smaller learning rate (1e−5) to get monotonically decreasing validation losses."
"In the training curriculum ablation study, models trained with a fixed context window of 32k from scratch required 3.783 × 10^22 FLOPs and achieved performance metrics like 18.5 F1 on NarrativeQA, 28.6 F1 on Qasper, and 37.9 EM on Quality."
"Continual pretraining from short context models can easily save around 40% FLOPs while imposing almost no loss on performance."
"Through early experiments at the 7B scale, we identified a key limitation of LLAMA 2’s positional encoding (PE) that prevents the attention module from aggregating information of distant tokens. We adopt a minimal yet necessary modification on the RoPE positional encoding (Su et al., 2022) for long-context modeling – decreasing the rotation angle."
Pretty exciting stuff. Getting close to GPT-4 hopefully soon!
Ashamed to say but I do all these things on my main OSs for ~15 years and I never had any problem (IIRC). Yes, I'm not huge on security and I realize what this could do, but I figure the chance is as big as getting it through a channel I don't expect.