"Just when you thought it was over... we’re introducing Gemini 2.0 Flash Thinking, a new experimental model that unlocks stronger reasoning capabilities and shows its thoughts.
The model plans (with thoughts visible), can solve complex problems with Flash speeds, and more ..."
Simran Arora: "Join us for a livestream this Thursday, Halloween/Diwali, and join our channel on the GPU Mode Discord server to hang out with us/get involved:"
Jeremy makes compelling arguments. Here are some more mundane corollaries:
It is only a matter of time before you and your company are affected by the pending regulations. In the future, almost all software products will be using AI models, in the same way that most software products use the Internet today, whereas they did not in the 1990s.
Imagine you had to license Oracle software because MySQL or PostgresSQL could not offer certain capabilities, or are less capable because of regulation.
Now also imagine that your products have to agree with the political world view of either Sundar Pichai or Elon Musk. And if you need capabilities only present in one of the commercial alternatives, you don't even that that choice.
The comments will inform the drafting of regulations on open weight models under the Biden executive order on AI using his powers under the Defense Production Act.
"By enabling the use of a single high-precision base model accompanied by multiple 1-bit deltas, BitDelta dramatically reduces GPU memory requirements by more than 10x, which can also be translated to enhanced generation latency in multi-tenant settings."
Everything you say makes sense. Training is definitely more compute intensive than inference.
Training is both memory throughput and compute constrained. Much research in speeding up training goes into optimizing HBM to SRAM communication. The equivalent for your chips would be communication from the SRAM of one chip to the SRAM of another, where it sounds like your architecture has a major memory throughput advantage over GPUs. So I assume you don't have a proportional compute advantage?
By the way, it's great to see a non von Neumann architecture showing a major performance advantage in a real world application. And your chips are conceptually equivalent to chiplets; you should have a major cost advantage on bleeding edge process nodes if you scale up manufacturing. Overall very impressive!
https://x.com/OfficialLoganK/status/1869789822384255300