unlord · HackerTrans

há 11 mesesdiscuss

> People don’t develop video codecs for fun like they do with software. And the reason is that it’s almost impossible to do without support from the industry.

As someone who lead an open source team (of majority volunteers) for nearly a decade at Mozilla, I can tell you that people do work on video codecs for fun, see https://github.com/xiph/daala

Working with fine people from Xiph.Org and the IETF (and later AOM) on royalty free formats Theora, Opus, Daala and AV1 was by far the most fun, interesting and fulfilling work I've had as professional engineer.

unlord

ano passadodiscuss

Someone asked about benchmarks, so I ran these just now to bring data to the discussion:

  nathan@arm1:~/git/rav1d.new/target$ hyperfine --warmup 2 "release/dav1d -q -i ~/Chimera-AV1-8bit-1920x1080-6736kbps.ivf -o /dev/null --threads 8"
  Benchmark 1: release/dav1d -q -i ~/Chimera-AV1-8bit-1920x1080-6736kbps.ivf -o /dev/null --threads 8
    Time (mean ± σ):     31.532 s ±  1.971 s    [User: 244.512 s, System: 1.644 s]
    Range (min … max):   28.498 s … 34.270 s    10 runs

  nathan@arm1:~/git/dav1d.new/build$ hyperfine --warmup 2 "tools/dav1d -q -i ~/Chimera-AV1-8bit-1920x1080-6736kbps.ivf -o /dev/null --threads 8"
  Benchmark 1: tools/dav1d -q -i ~/Chimera-AV1-8bit-1920x1080-6736kbps.ivf -o /dev/null --threads 8
    Time (mean ± σ):     29.696 s ±  2.308 s    [User: 230.507 s, System: 1.479 s]
    Range (min … max):   26.618 s … 35.105 s    10 runs

It shows that as of this moment rav1d is (31.532 - 29.696)/29.696 * 100 = 6.2% slower to decode this Netflix test sequence. Note, this is an improvement from the (32.775 - 29.696)/29.696 * 100 = 10.4% when Prossimo posted a bounty for improving rav1d [1]:

  nathan@arm1:~/git/rav1d.old/target$ hyperfine --warmup 2 "release/dav1d -q -i ~/Chimera-AV1-8bit-1920x1080-6736kbps.ivf -o /dev/null --threads 8"
  Benchmark 1: release/dav1d -q -i ~/Chimera-AV1-8bit-1920x1080-6736kbps.ivf -o /dev/null --threads 8
    Time (mean ± σ):     32.775 s ±  2.694 s    [User: 254.120 s, System: 1.659 s]
    Range (min … max):   28.847 s … 37.606 s    10 runs

None of this is particularly new, I reported this gap in performance over a year ago [2]. Here are some questions for HN:

Even if rav1d closes the gap, who will ship rav1d over dav1d? More than 85% of dav1d is hand written assembly that rav1d takes as-is and calls via unsafe blocks. Is this really memory safety?

The dav1d improvements must continuously be backported to rav1d, see the 82 closed PR's here [3]. Who is going to pay for this maintenance in perpetuity?

VideoLAN fuzzes dav1d and fixes bugs extremely quickly, typically less than 24h. The rav1d developers are Rust experts, but not codec experts. Is it reasonable to expect the same support?

[1] https://www.memorysafety.org/blog/rav1d-perf-bounty/

[2] https://github.com/memorysafety/rav1d/issues/804

[3] https://github.com/memorysafety/rav1d/pulls?q=backport+is%3A...

unlord

há 2 anosdiscuss

It is actually quite a bit more misleading. I was not able to reproduce these numbers on Zen2 hardware, see https://people.videolan.org/~unlord/dav1d_6tap.png. I spoke with the slide author and he confirmed he was using an -O0 debug build of the checkasm binary.

What's more, the C code is running an 8-tap filter where the SIMD for that function (in all of SSSE3, AVX2 and AVX512) is implemented as 6-tap. Last week I posted MR !1745 (https://code.videolan.org/videolan/dav1d/-/merge_requests/17...) which adds 6-tap to the C code and brings improved performance to all platforms dav1d supports.

This, of course, also closes the gap in these numbers but is a more accurate representation of the speed-up from hand-written assembly.

comments