For new open weights models, will you need to adapt model code and optimization for your inference engine by hand?
It's true that BS=1 is king when it comes to agentic workflows, however these kinds of system serve multiple requests concurrently with dynamic batching. Do you think it will scale as well ?
As ADHD dev that thought about something similar (but never worked on it) I'd love to see the code. Please make a post when you intend to release it as opensource, it seems super interesting
Hi everyone! I'm Christian, one of the few software engineers from San Marino ! First learner then engineer with 4 YoE. Currently working at Corsi.it, an e-learning platform based in Italy. Here I had the pleasure to work in many different areas: mobile, then web as a fullstack, then a bit of infra. Would love to find a place to specialise more as a backend dev, or even as a SRE.
My other passions include tinkering a bit with hardware and SBCs, some light pentesting with Flipper Zero, code puzzles and writing programs for stuff I need or out of spite.
I'm always amazed by all the brilliant people here in HN and I feel this is the right place to find challenging and fun opportunities!
For new open weights models, will you need to adapt model code and optimization for your inference engine by hand?
It's true that BS=1 is king when it comes to agentic workflows, however these kinds of system serve multiple requests concurrently with dynamic batching. Do you think it will scale as well ?
Any plans to release it open source?
Congratz again for the release