Sure, local coding is clearly _possible_, but it's not practical for most people. I've yet to see a reliable setup, if you have one, I'd love to see.
> creating plans, using subagents and compactions
Yes, these are all things that Claude Code does for you. However, for the thought loop issue, these are not the fixes. The canonical fix is to limit the number of thought tokens (llama.cpp's `--reasoning-budget`) or try to mess with the various penalty parameters. In any case, it's not a solved problem as far as I can tell.
I've been working with local models for the past year. There's so many possibilities, but I don't think coding is one. Coding requires so many layers beyond inference; I spent so much time trying to replicate what Claude Code does end to end locally. Understanding all the layers and keeping up with the advancements feels like a slog. Even this article messes up and misunderstands what some of the settings are doing. Qwen in particular seems to work at first, then often gets stuck in thought loops when used for actual work.
However, text-to-speech, speech-to-text, and non-code LLM use cases are so useful to have local, and don't require big hardware.
Having a universal reliable inference engine interface, I think, is the big unlock that needs to happen before app devs can ship these features.
Personal concrete use case: meeting recording app. This uses Parakeet + Qwen to create local transcriptions and post-cleanup, respectively.
Right now this app has to download and manage all these models, then bundle an inference engine to run them. It's a lot of code that probably should belong to the OS, or at least a standard interface.
While apps can offload some of this to llama.cpp or a similar process over http, that's another set of setup for the user to do before they can have a useful app.
Anyway, if you're getting started on a Mac, I'd suggest trying out oMLX (https://github.com/jundot/omlx) before messing with llama.cpp. In particular they have community benchmarks so you can see what kind of performance you're likely to get: https://omlx.ai/benchmarks. I wished each one had more configuration details though.
I've been using this keyboard on and off for a while now. I've always switched back to gboard, however this update made me convert full time. It's really good.
There are a few issues, like it randomly capitalizes words in the middle of sentences. Also, it doesn't seem to take context into account when suggesting words, so words that clearly wouldn't follow the last word will often show up.
It's not as good as gboard yet, but close enough that I'm going to stick with it.
Note that if you have a more powerful device, you can get larger models for voice and larger dictionaries from their site. They make a noticeable difference.
There are still a few unsolved problems that require tuning for specific applications. Applications that own the video call have a much easier time, they have access to each individual audio stream. Applications like this, however, have to deal with overlapping voices from a single stream. If it's trying to attribute each utterance to an individual, separating the voices is tough, or can lead to confusing transcripts. There are many little problems like this which make it a tough problem in real world usage. Domain specific terms, or proper nouns is another source of inaccuracy.
Nice! I really like how many variations on this idea are coming out. MacWhisper used to be great, but is kinda of a buggy mess now.
I'm making my own, for personal use. I did a survey of many and they all (that I could find) skip the fundamentals.
The major issues that I've run into:
- Crash recovery. Most of these apps are incredibly buggy and crash all the time, taking the recorded audio with them. Macwhisper is incredibly bad at this.
- Disk space. Many of these apps save wav files to disk. After a few hours of meetings, you may end up with gigabytes eaten.
- Microphone bleed. People don't always use headphones, the system mic will pick up the speaker sounds, causing duplicate (approximately) transcriptions.
I've yet to find a solution that handles all these correctly, let alone having high quality transcriptions.
UUIDs are way over used. There is almost always a better key to use, usually a bigint for databases. If you're making some kind of leaderless distributed data store, then maybe, but even then there are other ID sharding strategies I'd go for first depending on the constraints.
For a single database, bigints are smaller and faster, with less footguns.
UUIDs can be nice for an opaque public ID, however I'd still prefer something like a Sqid for space and usability.
Every time I see something like this (turn function calls into a network call), I reflect fondly on the list of fallacies in distributed computing [0]. These are issues that largely have to be handled in an application-specific way, and cannot be solved in the general case.
This list alone has saved me many late debugging nights, just by not making or using systems that ignore the list.
But in my experience, webp is better enough that the whole file loads around the same time the jpg progressive loading kicks in. Given that progressive jpgs are larger than non progressive (so not a 'free' feature), jpg is just a waste of bandwidth at this point.
As always, it really depends on what the source is. Often images are created with some software like Photoshop, would you commit the psd file to git? If you're a photographer, would you commit 20mb+ raw image files? Might make sense for a few images, but git is just not the right solution for binary data in general. Every modification has to duplicate the entire file. This makes working with the repo unpleasant very quickly.
In general, I recommend people back up binary files to cloud storage, like S3, and only commit optimized, deployment ready assets to git. There's also GitLFS, but it's clucky to use.
This post greatly over simplifies how many issues come up optimizing images. Also, the Github workflow doesn't commit the optimized images back to git, so this would have to run before packaging and would have to run on every image, not just new images added.
Ideally, images are compressed _before_ getting committed to git. The other issue is that compression can leave images looking broken. Any compressed image should be verified before deploying. Using lossless encoders is safer. However, even then, many optimizers will strip ICC profile data which will make colors look off or washed out (especially if the source is HDR).
Finally, use webp. It's supported everywhere and doesn't have all the downsides of png and jpg. It's not worth it to deploy these older formats anymore. Jpgxl is ever better, but support will take a while.
Anyway, I made an ImageOptim clone that supports webp encoding a while ago[0]. I usually just chuck any images in there first, then commit them.
After my dad died from cancer in 2018, I saw first hand the resulting loneliness and the lack of resources available. Being an engineer, I figured I might as well try to solve it, at least for some people. In 2020, I started a non-profit for small support groups[0]. We're small, and I've been mostly funding it myself, but it's growing. The main issue is we don't have the resources to cover every topic, so it's not for everyone (yet). Happy to chat if you have feedback, email is in my profile.
The WIP code was committed with the expectation that very few people would see it because it was not linked anywhere in the main readme. It's a calculated risk, so that the code wouldn't get out of date with main. The risk changed when their LLM (wrongly) decided to elevate it to users before it was ready.
It's clear DeepWiki is just a sales funnel for Devin, so all of this is being done in bad faith anyway. I don't expect them to care much.
I dunno, it seems to be real excited about a VS Code extension that doesn't exist and isn't mentioned in the actual documentation. There's just too many factual errors to list.
I took a look at a project I maintain[0], and wow. It's so wrong in every section I saw. The generated diagrams make no sense. The text sections take implementation details that don't matter and present them to the user like they need to know them. It's also outdated.
I hope actual users never see this. I dread thinking about having to go around to various LLM generated sites to correct documentation I never approved of to stop confusing users that are tricked into reading it.
> creating plans, using subagents and compactions
Yes, these are all things that Claude Code does for you. However, for the thought loop issue, these are not the fixes. The canonical fix is to limit the number of thought tokens (llama.cpp's `--reasoning-budget`) or try to mess with the various penalty parameters. In any case, it's not a solved problem as far as I can tell.