Sirus (Qax) is 90% built with LLM coding agents (mix).
I should say that I'm deeply involved in the process - architecting and reviewing (and sometimes revising) to ensure the code is clean and aligned with my preferences.
one approach to ensuring purity of work is staging good changes before asking the agent to make new ones. that way, stashing/deleting poor quality work isn't too expensive and doesn't cost you the current progress.
The comments in this post strongly validate the need for reliable video processing and understanding with VLMs.
While you can use Gemini or other local VLMs, the real challenge is token efficiency, accuracy, and coverage. For example, how do you make a VLM “watch” a 2-hour or 4GB video without losing context or meaning?
Video transcript alone can be sufficient for basic workflows needing no visual context. But when deep contextual understanding is required, e.g., self-driving, security analysis, warehouse tracking, etc., you’ll need more advanced methods like keyframe sampling, clipping, chunking, and shots+transcript.
Two approaches for passing binary files, say a 2-hour lecture video, to a VLM:
1. Send it whole: accurate, but slow to encode and process.
2. Keyframe extraction: fast, but could miss what matters.
There's no right or wrong approach here. It all depends on your use case and acceptable trade-offs.
Find useful strategies for encoding binary files in the post below. Enjoy!