Wedeo – a Rust Rewrite of FFmpeg(github.com)
github.com
Wedeo – a Rust Rewrite of FFmpeg
https://github.com/sharifhsn/wedeo
31 comments
> It provides no additional features compared to FFmpeg, and despite incorporating FFmpeg's assembly code, is significantly slower.
What a fantastic use of tokens. The author isn't even reviewing the code so you can't really even call it a study.
> This codebase is AI-generated. Written by Claude (Anthropic) via Claude Code, directed and reviewed by a human.
I suppose “review” is misleading, since I’m not actually reading the code, I’m simply asking Claude to explain what the code is doing. I’ll change that so it’s expressly clear that none of this code is actually human-reviewed.
These things may not be true in the future. I just meant to describe the current state of the project. With Rust being a modern language and the repository being AI-optimized, I wouldn't be surprised if it grew to have more features. And I hope that Rust can close the gap on performance, it would be a great testament to Rust's usefulness for high performance software.
Worth mentioning https://github.com/yazaldefilimone/ffmpreg, another rust rewrite of ffmpeg (apparently from scratch, not translated)
I found this project, but I’m not surprised that it doesn’t have much progress. Doing this kind of Rust rewrite is an absolutely gargantuan undertaking that I don’t think it would be feasible without AI. It’s not as if FFmpeg is some crufty, slow code that would seriously benefit from it.
Why even keep working on this by hand for months when an AI can just write it fully featured in a week?
Because it can't. I'm aware of only two projects that have attempted this.
1. Anthropic's C compiler. This is a buggy, unoptimized, unwieldy mess. One that can compile the Linux kernel, which is no mean feat. But it is not "fully featured." And it had an incredible array of tests to start with.
2. Cursor's web browser. This is, flatly, a trainwreck. It's built on top of massive dependencies that do incredible amounts of heavy lifting despite claims to the contrary, and never seems to have ever produced a successful build.
1. Anthropic's C compiler. This is a buggy, unoptimized, unwieldy mess. One that can compile the Linux kernel, which is no mean feat. But it is not "fully featured." And it had an incredible array of tests to start with.
2. Cursor's web browser. This is, flatly, a trainwreck. It's built on top of massive dependencies that do incredible amounts of heavy lifting despite claims to the contrary, and never seems to have ever produced a successful build.
The tests are the real clincher here. That’s the main reason that this project has made so much progress: the FATE suite highlights specific bugs to fix and tracks regressions.
As for optimization, that seems to be more of a question of effort than whether it’s possible. I was able to take down the performance gap on Rust vs C (without Assembly) from 10x to 1.5x through detailed profiling and iterative improvements with Claude.
It also looks like the Anthropic C compiler was built from scratch. By contrast, `wedeo` was directly based on FFmpeg’s existing code. Going by spec and test suite only would have taken a lot longer, and the quality would have been significantly lower.
As for optimization, that seems to be more of a question of effort than whether it’s possible. I was able to take down the performance gap on Rust vs C (without Assembly) from 10x to 1.5x through detailed profiling and iterative improvements with Claude.
It also looks like the Anthropic C compiler was built from scratch. By contrast, `wedeo` was directly based on FFmpeg’s existing code. Going by spec and test suite only would have taken a lot longer, and the quality would have been significantly lower.
Anthropic's C compiler also had an incredible test suite available, and was trained on compiler codebases. It might have been built "from scratch" but compilers are incredibly well-trodden ground.
And to contrast, Cursor's browser explicitly reviewed Servo's architecture before setting out, and still wound up like that.
And to contrast, Cursor's browser explicitly reviewed Servo's architecture before setting out, and still wound up like that.
The difference between using general knowledge and summaries and doing code-for-code rewrites cannot be understated. When creating `wedeo`, despite DIRECTLY reviewing FFmpeg's code, Claude would frequently make mistakes that would have to be fixed. I would constantly have it go and refer back to the ground truth FFmpeg source implementation. It did eventually work, but only after several rounds of reviews, and often dead-end investigations. Being able to directly inspect (and in some cases modify for extra debug output) FFmpeg at any given time was invaluable, and I doubt any rewrite would succeed without having that.
To be clear, `wedeo` took a few weeks to get to this point, and it is by no means fully featured. Even with LLMs, it will take a few months to reach that point with me as the sole human contributor.
A rust rewrite that's GPL and isn't MIT or Apache licensed? I'm shocked.
It’s generated by an AI so the "author" doesn’t care about the license.
I don’t know if I would say I don’t “care” about the license. I want to respect the FFmpeg license, and considering that my project is a straight ripoff, it would be dishonest to not have the same license. Whether AI-generated code can even be legally licensed or copyrighted is a separate question that hasn’t been decided by courts. I’ll happily comply with whatever experts recommend.
Good answer. Asking an AI coding agent to port a C codebase to Rust is clearly and obviously creating a derivative work. You need to follow the terms of the LGPL license, and your Rust fork of FFMPEG must be licensed as LGPL. Videolan holds the copyright to FFMPEG, and that includes derivative works. LGPL allows you to do exactly what you're doing.
Note that if you convert x264, x265 or any of the other GPL v2 libraries, your Rust version of those libraries will be GPL v2. When builds of FFMPEG are created with GPL v2 libraries, the entire build becomes GPL v2 (which has a more restrictive copyleft provision than LGPL).
Good to know! I have licensed as LGPL so I think that covers it. Rewriting x264 and x265 are in scope so I’ll keep that in mind. I will design the application as well so that those can easily be excluded from compilation to get a pure LGPL build.
It incorporates code directly from FFmpeg and FFmpeg is LGPL.
After playing around with Claude Code for a bit, rewriting some Python tooling in Rust to great effect, I was interested in pushing the boundaries of what LLMs could do in terms of rewriting projects in Rust. The result is `wedeo`.
For those unfamiliar, [FFmpeg](https://www.ffmpeg.org/) is "a complete, cross-platform solution to record, convert and stream audio and video". It is one of the most powerful and impressive pieces of open-source software and is the underlying infrastructure for a ton of A/V software. It is also, like the Linux kernel, written purely in C and Assembly.
There are a couple reasons for why FFmpeg is a good candidate for this kind of project. The source code is incredibly high quality, being battle-tested and having contributions from the best experts in the area. A/V encoding and decoding is also rigorously described by specification, which is easily consumed by LLMs. The project's nature of being a bunch of codecs bundled together in a convenient interface made it easy to rewrite incrementally. C, being a systems programming language, maps nicely to Rust, and the assembly could be directly ported over.
The major contribution `wedeo` currently has is an implementation of the H.264 decoder, which is around 30,000 lines of code for the scalar Rust implementation. It doesn't support some complex features like interlacing or 10-bit, but it should support 99% of H.264 encoded video.
There has not been any kind of performance optimization done (except porting some assembly over and basic multithreading), so `wedeo` is much slower than FFmpeg. I expect the gap to close somewhat over time, but I doubt that even the most optimized Rust could beat FFmpeg's high quality C and Assembly.
In order to get this project to the point where it can actually play video properly, I have utilized some existing libraries. The intention is for the codebase to be pure Rust, so any functionality which only exists as a non-Rust library will have to be rewritten. `symphonia` is used for audio, `rav1d` and `rav1e` for AV1, and `wgpu` and `winit` for the player.
I have only confirmed that this works on MacOS M-series, so I would welcome testing on other machines.
There is a lot missing from `wedeo`, in particular any video codecs besides H.264 and AV1, as well as H.264 encoding. I will be working on this as a side project, but I would also welcome contributions on these. And if anyone is interested in taking on an active role as maintainer, I'd gladly hand the reins over.
# On AI Slop
Although I can read and write Rust, 100% of the code in `wedeo` is AI-generated and I have not directly reviewed a single line of it, other than asking Claude to fix bugs and explain parts of it. It is intended as an experiment in pure LLM usage.
I'm aware that this community and many other technical spaces online have been overwhelmed by "AI slop", bloated projects that have LOC that run in the tens of thousands, and I can see why many people might interpret `wedeo` that way. However, the amount of code in this project is similar to the equivalent amount in FFmpeg.
I also think that this project is at least interesting in that this has never been attempted before, and I want to see where it can be taken. I think it's at least a better use of tokens than typical AI slop.
Excited to see what the community thinks!
P.S. I just saw [this tweet](https://x.com/FFmpeg/status/2039115531744334180) from FFmpeg. Ironic, but I assure you this is not an April Fools' joke.
For those unfamiliar, [FFmpeg](https://www.ffmpeg.org/) is "a complete, cross-platform solution to record, convert and stream audio and video". It is one of the most powerful and impressive pieces of open-source software and is the underlying infrastructure for a ton of A/V software. It is also, like the Linux kernel, written purely in C and Assembly.
There are a couple reasons for why FFmpeg is a good candidate for this kind of project. The source code is incredibly high quality, being battle-tested and having contributions from the best experts in the area. A/V encoding and decoding is also rigorously described by specification, which is easily consumed by LLMs. The project's nature of being a bunch of codecs bundled together in a convenient interface made it easy to rewrite incrementally. C, being a systems programming language, maps nicely to Rust, and the assembly could be directly ported over.
The major contribution `wedeo` currently has is an implementation of the H.264 decoder, which is around 30,000 lines of code for the scalar Rust implementation. It doesn't support some complex features like interlacing or 10-bit, but it should support 99% of H.264 encoded video.
There has not been any kind of performance optimization done (except porting some assembly over and basic multithreading), so `wedeo` is much slower than FFmpeg. I expect the gap to close somewhat over time, but I doubt that even the most optimized Rust could beat FFmpeg's high quality C and Assembly.
In order to get this project to the point where it can actually play video properly, I have utilized some existing libraries. The intention is for the codebase to be pure Rust, so any functionality which only exists as a non-Rust library will have to be rewritten. `symphonia` is used for audio, `rav1d` and `rav1e` for AV1, and `wgpu` and `winit` for the player.
I have only confirmed that this works on MacOS M-series, so I would welcome testing on other machines.
There is a lot missing from `wedeo`, in particular any video codecs besides H.264 and AV1, as well as H.264 encoding. I will be working on this as a side project, but I would also welcome contributions on these. And if anyone is interested in taking on an active role as maintainer, I'd gladly hand the reins over.
# On AI Slop
Although I can read and write Rust, 100% of the code in `wedeo` is AI-generated and I have not directly reviewed a single line of it, other than asking Claude to fix bugs and explain parts of it. It is intended as an experiment in pure LLM usage.
I'm aware that this community and many other technical spaces online have been overwhelmed by "AI slop", bloated projects that have LOC that run in the tens of thousands, and I can see why many people might interpret `wedeo` that way. However, the amount of code in this project is similar to the equivalent amount in FFmpeg.
I also think that this project is at least interesting in that this has never been attempted before, and I want to see where it can be taken. I think it's at least a better use of tokens than typical AI slop.
Excited to see what the community thinks!
P.S. I just saw [this tweet](https://x.com/FFmpeg/status/2039115531744334180) from FFmpeg. Ironic, but I assure you this is not an April Fools' joke.
FFMPEG is a media processing framework. Of course, it is normally built with the many encoders, decoders and filters that do the actual media processing work. Those components are separate open source projects. The performance critical part of FFMPEG is in the component encoders, decoders and filters, not the framework that sets up the media processing graph, gets it running and monitors it. It could be useful or interesting to have a Rust port of FFMPEG (for developers who might want to contribute new features). I wouldn't bother trying to port component encoders, decoders or filters from C to Rust. There is no need, and no benefit. They are mostly older and stable. Rust will never be faster than C, even if you link all the hand-optimized assembly code.
The challenge of maintaining a Rust fork of FFMPEG will be to stay current with changes to FFMPEG. LLMs are very good at converting from one language to another, so it might be feasible, especially if you can automate it. But I would definitely stick to using Rust only for the top layer of FFMPEG.
I don’t really think there’s any actual benefit to `wedeo` existing, beyond the novelty of having pure Rust media processing software and perhaps being easier to use as a library. Like I said, this is mostly an experiment in LLM’s ability to rewrite software in Rust. FFmpeg is one of the most high quality pieces of open source software out there, I seriously doubt that LLMs on their own could eclipse it, even with all of Rust’s benefits. So I definitely understand what you’re saying but it’s not really my motivation.
As for the updating, this is something that I’ve been considering but it’s obviously not in scope until `wedeo` is fully featured. But yes, this is something that LLMs should be able to do quite easily, it could even trigger on every commit.
As for the updating, this is something that I’ve been considering but it’s obviously not in scope until `wedeo` is fully featured. But yes, this is something that LLMs should be able to do quite easily, it could even trigger on every commit.
The benefit is that you and other Rust developers could potentially implement improvements to FFMPEG. Again, if you stick to porting only the top layer of FFMPEG - the media processing framework with the CLI - you can still keep the rest as C and link it all together in a build. Your Rust version (and any improvements implemented by other Rust developers who contribute to wedeo) can easily be converted back to C so that patches could be offered back upstream to the main FFMPEG project.
Have you taken a look at the various test harnesses for video encoding/decoding, or consider publishing a standalone one yourself? I get a bit-for-bit rewrite can have some advantage but it's hard to know if the AI implementation actually covers all of the edges cases.
I’m using FFmpeg’s own FATE suite, which is intended to poke at those edge cases, as well as JVT vectors. I’d love to get a more robust set of weird files similar to what symphonia has, though, so if anyone has any resources I’d appreciate it.
Thank you for being up-front in disclaiming that this project is AI-written, both here and in the Github page. I really appreciate the transparency.
> 100% of the code in `wedeo` is AI-generated
Thanks but no thanks
Thanks but no thanks
What's the unsafe situation like in Wedeo?
It’s used for multithreading and also for assembly loading. It’s not checked by anything other than Claude itself, so that would definitely be a strong candidate for manual review. I did have Claude ingest Rust Atomics & Locks by Mara Bos to help guide the implementation for concurrency, at least.