antinucleon

3 lata temudiscuss

Yes

3 lata temudiscuss

Mojo is trying to create a new language to solve the problem, and specialized for CPU. We are using a more pragmatic way to solve GPU AI computation problem.

antinucleon

3 lata temudiscuss

We haven't compared yet.

antinucleon

3 lata temudiscuss

We developed AITemplate majorly for Meta's focus at that time, eg Ads/Ranking need. For HippoML is startup we are building for Generative AI. HippoML is not using AITemplate.

antinucleon

3 lata temudiscuss

It is actually non-trivial to get GPU run fast, especially on SoC with strong CPU like M2.

antinucleon

3 lata temudiscuss

Hippo is faster than AITemplate, and supports more generative models. We haven't compared vs TVM, but for absolute token/s on M2 Max, Hippo is able to run decoding on LLAMA with datacenter level GPUs performance (with other SW).

antinucleon

3 lata temudiscuss

We will disclose more details very soon.

antinucleon

3 lata temudiscuss

Yes. We support >= 1bit <= 16bit models out of box for various of models.

antinucleon

3 lata temudiscuss

AITemplate's original designer is here. We quit Meta in January and start HippoML (https://hippoml.com/). We just disclosed our new engine's performance on LLM: https://blog.hippoml.com/large-language-model-inference-from... On Apple M2 Max our new engine encode/decode is 13.8X/2.4X faster than llama.cpp

antinucleon

Submissions

The Great Wave Has Arrived (Memo from GLM CEO Jie Tang)

What Muse Spark 1.1 Taught Us About Enterprise Agent Architecture

Stop Waiting for a Bigger Context Window

Show HN: INT21 – Self-Improving PTX Kernel Factory

7 days of autonomous agent search outperformed FlashAttention-4 and CUDNN

From Node.js/Python to PTX: The first AI framework generated by AI agents

PetaFLOPS Inference Era: 1 Pflops Attention, and Preliminary End-to-End Results

Unified DataCenter and Local Foundation Model Serving: Beyond Docker Way

Super AI Creativity App Run with Local GPU on Windows/Linux/MacOS

comments