Probably a bit niche at the moment really. The only people interested in that are us nerds, and the product segment is very adhoc - especially for the local crowd where an epyc, with a bunch of pcie riders and some 3090s on a steel frame is considered optimal
Ampere make some big 32-128 core server ARM chips that are somewhere around Zen4c ish in performance, although they aren't commonly used outside of the server context. But they do at least exist.
AFAIK youtube will stretch the player window to match the aspect ratio of the source media, lots of cinematic content that's a wider than normal (21:9 I think?) ratio that youtube adjusts the player window to fit around without black bars.
They won't ever squash or stretch video though, so this means the original uploader stretched the 4:3 content to 16:9 at some point before upload
I would love to have a shit load of small (27B dense. 35B MoE) agents running locally and looking at and ingesting every bit of data about me, my life and what I get up to see what sort of correlations it finds. Give a coding agent access to a data lake of events and let it build up its own analytics tooling to extract and draw out information from that data, and present it to me as daily/weekly/monthly summaries.
Its dbt inspired stream ETL tool (or maybe just the TL?), it currently just has a dev mode that does RabbitMQ to local Parque files while I'm getting the core of it to a place I'm happy with.
It runs SQL models against the incoming messages and outputs the results to one or more output tables. Has a local WAL so you can tune it to have sensible sized output files (or not, if you need regular updates but at the expense of query perf.)
Planning on adding Protobuf messages, Kafka as a source and S3 and Iceberg tables as sinks this week.
Lightly inspired by a some projects at work where a lot of time and effort was spent doing this and resulted in something not very reusable without a lot of refactor work. Feel like the stream -> data lake pattern should be something that is just SQL + Config, same way dbt is for transformations within a data warehouse.
No plans on adding any cross message joins or aggregations as that would require cross worker communications and I explicitly want to keep the workers stateless (minus the WAL of course)
Would really appreciate any feedback on the core concept, esp. if this is something you'd actually use in prod (if it were finished!) Not sure if there is something that does this already that I don't know about, or if this genuinely fills some sort of hole in the exisitng tooling
And you know Nvidia can't be constent with one format for FLOPs within a single graph, 1,000,000x faster but comparing FP32 to FP8 or NVFP4 and acting like it's the same.
settings.json -> global config
Env vars -> settings different to your global for a specific project
Slash commands / chat keywords -> need to change a setting mid chat
Wish they gave some numbers for total GPU hours to train this model, seems comparatively tiny when compared to LLMs so interested to know how close this is to something trainable by your average hobbyist/university/small lab
An LLM observability SDK that let's you store pre and post request metadata with every call in as lightweight an SDK as possible.
Stores to S3 in batched JSON files, so can easily plug into existing tooling like DuckDB for analysis.
It's designed to answer questions like; "how do different user tiers of my services rate this two different models and three different systems prompts?". You can capture all the information required to answer this in the SDK and do some queries over the data to get the answers.
Tbh, for most companies/orgs the cost/complexity of multi region just isn't worth it.
The cost of a work days worth of downtime is rarely enough to justify the expense of trying to deploy across multiple regions or clouds.
Esp if you are public facing and not internal. You just go 'well everyone else was down to because of aws' and your customers just go 'ah okay fair enough'