Kimi K2.5 on 2 Mac Studios (<$20K) running at 24 TPStwitter.com6 ポイント·投稿者 alexandercheema·5 か月前·4 コメント
Clustering Nvidia DGX Spark and M3 Ultra Mac Studio for 4x Faster LLM Inferencetwitter.com8 ポイント·投稿者 alexandercheema·9 か月前·1 コメント
Clustering Nvidia DGX Spark and M3 Ultra Mac Studio for 4x Faster LLM Inferenceblog.exolabs.net5 ポイント·投稿者 alexandercheema·9 か月前·0 コメント
alexandercheema·3 か月前·議論lol bro there is already a big OSS project called exo: https://github.com/exo-explore/exo
alexandercheema·9 か月前·議論Appreciate you checking back so often. We have some exciting plans. Keep checking and it won't be long before something pops up :)
alexandercheema·9 か月前·議論Yes, these models are mostly compute-bound so benefit even more from the compute on the DGX Spark.
alexandercheema·9 か月前·議論Blog author here. Actually, no. The model can be streamed into the DGX Spark, so we can run prefill of models much larger than 128GB (e.g. DeepSeek R1) on the DGX Spark. This feature is coming to EXO 1.0 which will be open-sourced soonTM.