I'm working on inference.club, a distributed inference network for consumer hardware. Sign up with GitHub, get an API key, run an agent on your home network that registers your local inference resources with inference.club, set permissions for who can use your services, try out models in the playground and use the API. So far it supports the following models:
- LLMs (any OpenAI compatible API, vLLM, LM Studio, etc.)
- image gen + image edit (flux klein)
- text to speech (magpie, dia with voice cloning)
- speech to text (OpenAI audio transcriptions + riva compatible)
- image to textured 3d model (trellis2)
- image+text to video (ltx2.3-gguf)
- text to music (acestep)
currently it is just me and Claude vibing. While using Fable 5 moved all of my local inference services to k3s across 3 RTX 4090 PCs and my DGX Spark, now I can just tell Claude/Hermes/etc. to start and stop services.
inference.club is built with Tailscale's tsnet library. It is sort of like an OpenRouter built for different types of local AI models. inference.club also lets you showcase and share generated content. For example here is 90 seconds disco funk track generated by acestep: https://inference.club/s/Vxm6ozW24oBs_JGbPcq7tA
I was inspired by AI Horde, and wanted to see if I could build something that could support all of the model modalities that I use for generating short-form AI slop content on local hardware. This is also similar to Hugging Face Spaces, but running on consumer hardware with a common API. I've been watching the quality of local AI inference making massive improvements in quality and performance, and I want to make it easier for people to try "local AI" even if they don't have a GPU.
I'm working on Waywo, a semantic search engine for "What are you working on?" threads powered by new features in Redis 8 like the vector set data structure for semantic search.
Waywo will help users quickly digest hundreds of project descriptions, explore similar projects, deduplicate projects across threads from previous posts, visualize a graph of all projects, and more! I'll be documenting my approach to building this with coding Agents like cursor and Gemini CLI
I'm building Waywo for the Redis Hackathon on DEV.to that is running from now until August 10! Follow me on DEV/GitHub/X (@briancaffey) to see how this project turns out!
- LLMs (any OpenAI compatible API, vLLM, LM Studio, etc.) - image gen + image edit (flux klein) - text to speech (magpie, dia with voice cloning) - speech to text (OpenAI audio transcriptions + riva compatible) - image to textured 3d model (trellis2) - image+text to video (ltx2.3-gguf) - text to music (acestep)
currently it is just me and Claude vibing. While using Fable 5 moved all of my local inference services to k3s across 3 RTX 4090 PCs and my DGX Spark, now I can just tell Claude/Hermes/etc. to start and stop services.
inference.club is built with Tailscale's tsnet library. It is sort of like an OpenRouter built for different types of local AI models. inference.club also lets you showcase and share generated content. For example here is 90 seconds disco funk track generated by acestep: https://inference.club/s/Vxm6ozW24oBs_JGbPcq7tA
I was inspired by AI Horde, and wanted to see if I could build something that could support all of the model modalities that I use for generating short-form AI slop content on local hardware. This is also similar to Hugging Face Spaces, but running on consumer hardware with a common API. I've been watching the quality of local AI inference making massive improvements in quality and performance, and I want to make it easier for people to try "local AI" even if they don't have a GPU.