Even if something wasn't written by ChatGPT, I'm supposing ChatGPT's influence on writing has been so strong that (1) any typical reader of text on the internet will have ingested a lot of ChatGPT writing habits in their own writing, and (2) that any reader of ChatGPT text is so habituated to those writing habits that even non-ChatGPT generated text appears similar to ChatGPT generated text.
Has anyone figured out why anyone would bother adopting the google 'open knowledge format'?
Normally I expect a set of tooling to be build on top of any open format. Value-adds and interoperability. Instead I just see a way to organize markdown files.
I strongly recommend it if one's able. It's a bit more stable than a quickly evolving ML/DL/AI ecosystem or frontend ecosystem. The skills are more durable. It repays deep investment and knowledge.
It allows you to straddle both the distributed systems and services domain and the ML domain.
ML systems problems are extremely interesting since they require extremes of compute, storage, network, and latency, in very different parts of the model lifecycle. Its unique problem is the scarcity and cost of hardware accelerators.
I've worked eleven years in the space and rarely have had the desire to leave.
Pytorch is widely accepted as the de facto ML framework in both research and industry. TensorFlow comes second in industry. Jax is hardly used at all, but uses the same backend as TensorFlow.
Triton is a python-like language to define ML math operations that run efficiently on hardware accelerators like GPUs or TPUs. OpenAI open sourced it. If there's a particular math operation you have a unique need for in your model, and it hasn't already been implemented by some other library, and it's important for efficiency, you'd probably write it in triton these days. It'll be compiled to an intermediate representation, then to an efficient runtime.
The course linked deals with "MLSys", or "ml systems". That means using GPUs and other hardware accelerators efficiently to run ML math operations on one or more computers.
95% of working ML engineers will never need to write Triton, and will be more than satisfied with PyTorch. Many more ML engineers will, nevertheless, write Triton code, because it is interesting, fun, easy, and people are impressed when you tell them you did.
Hosting pytorch models efficiently is currently awkward, because there's no clear winner in the ecosystem. ONNX is a way of representing model graphs in a framework-agnostic way. Other systems can interpret ONNX graphs to do inference. So sometimes, when someone wants to host a pytorch model, they turn it into an ONNX model and run it with an efficient runtime on CPUs or GPUs.
You're right that "assumes" might be misleading. Maybe "implies" would be more correct.
The point the author's trying to make is that if we state in a paper something like "the LLM understands, believes, thinks, ..." then we're supposing an intelligence much like our conception of a human intelligence. It's a form of 'begging the question' -- assuming what you're trying to prove.
It is not quite a fair argument, just because we don't have a precise vocabulary around how to talk about the activity of LLMs that doesn't involve making these loose analogies. Except for philosophers and people engaging in this kind of "is it truly intelligent or no" conversation, being imprecise in this way doesn't necessary have any cost, but is just a convenient way to avoid developing a jargon.
Yes. There's nothing essentially new this latest round of AI has unturned that philosophers haven't turned over decades (or more ago). Nothing stopped philosophers supposing even a functionally perfect simulacrum of human intelligence, and getting technologically closer to it doesn't.
The real effect of the latest round of AI has been inducing software engineers to be pretend-philosophers as they're approaching this set of questions for the first time -- and are having a very hard time engaging given their enthusiasm for technology.
And the 'argument', which is a funny way to recast the chinese room argument, which has also been discussed to death.
And you're also assuming any kind of position other than your own dogma -- that AI has Intelligence In Its Name and Humans Have Intelligence Therefore AI Has Human-Like Intelligence -- is based on some religious belief in the specialness of humans instead of pointing out where this analogy between Intelligence in its two senses breaks down.
Amodei is convinced he's abraham prefiguring AI's christ. Very different than Altman's cold power-seeking. You can always trust someone who's selfish, since they'll always do whatever's in their benefit at all times.
This is why 'agents' are the solution for these companies. Token spending goes through the roof. As long as a human is in the loop needing to read or review at human speed, that's a ceiling on how many tokens per user they can generate.
From the other end, I've seen this go wrong a couple ways:
When I'm doing it: I can go on way too long trying to consider way too much, when really, putting down some code and reading it and writing it myself would give me a better understanding.
When others are doing it: they can get very entrenched in a certain way of thinking, and are sure it's correct because of their AI conversations. Some context or data point was missing from their conversations with the AI.
Why do those experiences indicate the presence or non-presence of an afterlife?
This claim from Ayer -- how do we make the leap from these experiences existing to being evidence of a life after consciousness?
> On the face of it, these experiences, on the assumption that the last one was veridical, are rather strong evidence that death does not put an end to consciousness
I came away with a very different conclusion, which is that the fact that such “bad” software can be so resoundingly successful for a business, yet be so odious to experienced human reviewers, means that it was the right engineering choice to go fast, rather than “do things right” by emphasizing code quality.
What good would it truly be if a 3K line function is split into 8 modules? It’ll be neater and more comprehensible to a human reader. More debuggable, definitely.
But given the business problem the have: winner takes all of a massive market, first mover wins, — the right move is to throw the usual rulebook about quality software out the window, and double down on the bets of the company, that AI will make human code engineering less and less necessary very quickly.
It turned out incredibly well despite the “bad” engineering — which in this case, I really count as good engineering.