Gianni Rosato

Introducing oavif

2025-10-12T00:00:00+00:00

oavif is a new approach to target quality encoding in image compression, designed around smarter convergence strategies and quicker scoring to be as fast as possible.

</picture> </div>

Why?</h2>
Target quality encoding is one of the highest impact use cases for image compression. A target quality encoder framework aims to produce an image encoded at a particular quality set by the user according to some metric or visual quality index. This kind of encoder framework is useful for a variety of users, ranging from small website owners to content delivery networks pushing vast quantities of image data through the Web.
The value of target quality encoding is perceptual consistency. If I rely entirely on my encoder's internal quality index (often set with a "q" parameter), I may not get outputs of perfectly consistent quality when using the same "q" across different images. Relying on a metric that represents the viewer's experience is the solution to this; targeting a representative score within this metric will ensure you always receive an optimally encoded file that never undershoots and ruins image quality, and never overshoots and wastes data.
oavif</a> is a tool to do target quality encoding extremely quickly. There are three core components to a target quality encoding framework: the metric, the encoder, and the convergence algorithm. oavif aims to leverage or improve the state of the art in all three categories.
Source Code</a> </div>
I think this use case has been neglected because it sits in an awkward spot, stuck between encoder development and content deployment. Considering image encoders and powerful metrics are fast, it is easy to take them for granted and build inefficient frameworks around them. Slow frameworks waste valuable resources; processing images is expensive. I built oavif with the same approach I've adopted when building encoders, where every CPU cycle counts.
Metric</h2>
There are good metrics and bad metrics in the context of what humans care about in images. PSNR</a> is a bad metric; targeting a PSNR score doesn't mean anything to users, because images at the same PSNR could look completely different. However, PSNR is very fast, and faster metrics lend themselves more favorably to target quality encoding.
SSIMULACRA2</a> correlates highly with subjective human quality ratings, but it is comparatively slow compared to simpler metrics. I set out to remedy this with fssimu2</a>, a faster implementation that uses almost 40% less memory. This is what oavif uses, and it makes computing the in-loop metric much faster compared to the reference library.
Testing on a 4k test image</a> against a distorted sample</a> for an average time to score across 8 runs, Butteraugli (a perceptual metric from the libjxl</a> project) took 2455ms, while the reference SSIMULACRA2 implementation took 1162ms. fssimu2 takes 631.9ms. Testing was done on my M2 MacBook Air using `hyperfine</code></a>.`
``
`Encoder</h2>`AVIF is a capable Web image format. oavif uses libaom</a> (via libavif</a>) because it is the best open-source image encoder available relative to its speed. I worked on improving AVIF encoding</a> in 2024 during my work on SVT-AV1-PSY</a>. Google (with help from Julio Barba</a>) later adopted this work and advanced it further in libaom. It is now used by some websites you may know, such as The Guardian</a>. Aside from speed, encoder consistency</a> is valuable in the context of target quality encoding (I'll explain more about why later). In fact, a perfectly consistent encoder would be able to eliminate the need for targeting entirely since the encoder's user-configurable Q would map perfectly to some perceptual index. libaom has had engineering effort go into encoder consistency, which is a valuable thing. Convergence</h2> A simple convergence loop looks like this: Decode input, pass to encoder</li> Decode encoder output & compare to input with metric</li> If we hit the target metric score, finish; otherwise, repeat prev. step with modified settings</li> </ul> The most important part here is how we decide to modify our settings. This is the convergence algorithm that allows us to search for the best encoder Q. The easiest way to do this is with binary search</a>, and some more recent implementations have utilized clever interpolation</a> using past data to inform the next guess based on the fact that we know encoder Q and target score are likely correlated. oavif takes inspiration from both of these, adding predictive modeling alongside error-informed search space correction to improve search times significantly. To understand why, we'll walk through each stage of the implementation. Binary Search</h3> This testing was done using the Daala subset2</a> image dataset. Importantly, I only used this dataset for validation; oavif was not designed around this specific dataset in any way. When testing, the oavif configuration was left at defaults; only the convergence implementation was modified. The threshold for meeting the target score is ±2.0 in oavif by default, and the default target score is 80.0 as measured by fssimu2 because it is a reasonable "high fidelity" target. Everyone with some algorithms background will start with binary search. Set your bounds for encoder Q to 0..100, and divide the range in half each time you test. In oavif, a pure binary search implementation at default settings yields the following results: Average encoding time: 467.95 ms ± 94.64 Average passes: 3.20 ± 0.45 (max: 4 min: 2)</code></pre>Interpolation</h3> Interpolation-based target quality searches by iteratively probing, measuring, and narrowing the search interval just like binary search. The difference is that it tries to model the score-vs-quantizer curve with interpolation (linear, quadratic, etc) as more data is accumulated. This should theoretically reduce the number of necessary encodes, and can start with standard binary search when there is not enough data to interpolate with. Metric score vs encoder Q is generally (though not perfectly) a mostly monotonic curve. Interpolation-based inverse estimation uses the measured points to approximate that curve and solve for the quantizer that would produce the target score. Higher-order methods use more shape information and are thus theoretically more accurate. Adding linear and quadratic interpolation support to oavif, we see a small reduction in the average number of passes on subset2: Average encoding time: 468.98 ms ± 97.28 Average passes: 3.12 ± 0.39 (max: 4 min: 2)</code></pre> This is a 2.5% improvement. We still need a minimum of two passes to accurately target. Predictive Modeling</h3> This feature uses an exponential curve trained on the gb82 image dataset</a> with libaom (at speed 9, 10-bit, 4:4:4 chroma). The curve looks like this: Based on this, we can write some very simple code to predict a Q value from the target score: fn predictQFromScore(tgt: f64) u32 { const q = 6.83 @exp(0.0282 tgt); return @intFromFloat(@min(100.0, @round(q))); }</code></pre> This yields the biggest average improvement in this testing so far, decreasing average pass count by 56.4% versus interpolation search and 57.5% versus binary search. Average encoding time: 218.33 ms ± 114.16 Average passes: 1.36 ± 0.78 (max: 3 min: 1)</code></pre> The gb82 image set is fairly low-resolution mixed photographic content, while Daala subset2 is medium-resolution photographic content with less variation. The fact that the model generalizes so well is exciting. It is at this stage that encoder consistency becomes important. A more consistent encoder will diverge from our model's predictions less frequently, and theoretically result in a faster target quality loop. Error Bounds</h3> Because our initial predictions tend to be so accurate, we can use them to aggressively narrow our search space without incurring too much risk of a search space collapse. The basis of this is that utilizing plain binary search with prediction is often unreliable. Let's say we would like to target score=80, and our model predicts we need Q=65. We score 82.38. Now we are forced to search (0..65), which is worse than if we had just avoided prediction in the first place (our search space would be 50..100 in that case). This is in spite of the fact that our prediction was very close to the target. oavif uses the distance from the target to its advantage: const abs_err = @abs(e.t.score - o.score_tgt); if (pass == 0) { const err_bound: u32 = @intFromFloat(@ceil(abs_err) * 4.0); if (e.t.score - o.score_tgt > 0) { hi_bound = e.q; lo_bound = if (e.q > err_bound) e.q - err_bound else 0; } else { lo_bound = e.q; hi_bound = @min(100, e.q + err_bound); } }</code></pre> In this case, the error was 2.38; @ceil()</code> brings this to 3, and we multiply by 4 because the midpoint of the new range tends to be very close to the target value based on my testing. The performance improves in kind: Average encoding time: 194.50 ms ± 69.89 Average passes: 1.18 ± 0.39 (max: 2 min: 1)</code></pre> This costs 13.2% fewer passes than interpolation + prediction, and 63.1% fewer passes than binary search. The minimum number of passes necessary in a naive binary search or interpolation-informed search is now the maximum number of passes we need to converge on the target on Daala subset2. You'll also notice the standard deviation went down due to the fact that the ceiling has been lowered. Architecture</h2> We've made it work and we've made it good, so now we can make it fast. oavif is written in Zig, and uses available high-performance C decoder libraries for handling inputs and decoding AVIF in the convergence loop. All image I/O during convergence is done in memory, and a buffer is kept of our latest encode to write to a file if we meet the target in the search space. Efforts have gone into making oavif comparable to libavif's avifenc</code> in terms of features as well. It supports high bit depth I/O, ICC profile handling for most formats, user-configurable encoder settings, and better defaults (until tune=iq becomes the libaom default in libavif). Future Directions</h2> Architecturally, it would be trivial to keep a history of buffers active and always pick from the history, even if our loop doesn't converge on the target. I opted to avoid this for now because it dramatically increases memory usage, but if I receive widespread feedback that memory is unimportant I'll consider an implementation. In its current state, we hit the in-loop buffer the vast majority of the time anyway. I think the future of this kind of workflow is far more accurate predictive modeling. I believe it is possible to improve what I've done if we provide details about the source image as another term in the equation (like variance or entropy) and train our prediction mechanism on this additional data. I'm optimistically convinced this could result in a very high success rate for one-shot targeting. I'm looking forward to seeing more target quality workflows taking advantage of smarter targeting. If you've made it this far, thanks for reading, and enjoy oavif!

The Multimedia Renaissance 2025-08-18T00:00:00+00:00 The best open source video and image compression algorithms have improved significantly over the past year, and developer tools have miraculously kept up with this rapid pace. There has never been a better time for video. </picture> </div> Purpose</h2> When you work on anything, I think it is worth thinking about why you are working on it aside from whether or not it provides some instant gratification. Thinking about what the core purpose of video and image compression might be, it is easy to land on a simple conclusion: smaller images and (especially) videos make the internet cheaper. Video makes up the vast majority of internet bandwidth, so small gains add up. However, this prompts further questioning; what might the end game be for compression? For the average person, what is the deliverable derived from compression technology indistinguishable from magic? I believe this final goal is beautiful multimedia accessible anywhere. In other words, an artist's vision is not served half-baked to any sub-section of their audience. Compression doesn't just make the internet cheaper, it also makes it more beautiful. Timeline</h2> The most revolutionary compression technology has been able to unlock new experiences for people creating and experiencing art. If you look at the history of video, this is illustrated by certain major leaps made over time. Every breakthrough represents a widening of the pipeline between artist and audience. Let's look at the timeline. Video begins as something you experience at a theater; people travel to see certain videos at specific times, and then maybe never again. Video doesn't look that great, and for a while, it doesn't have sound or color.</li> Analog broadcast television lets people see certain videos at specific times anywhere that has the right equipment. At some point, video gets color.</li> VCRs enable time-shifted viewing; you can see certain videos at almost any time.</li> Digital optical discs improve upon the last step by providing better looking videos.</li> The world of digital internet video emerges through peer-to-peer downloads. Now you can see pretty much any video anywhere after waiting for a download. Regardless of your internet connection, a smaller file means a faster download.</li> Streaming platforms make on-demand web video real; now you can watch pretty much any video anywhere with a good internet connection almost instantly.</li> Smartphones proliferate; smarter compression and streaming technology means we can do what we did in the last step on most internet connections, allowing people to access virtually any video almost anywhere on Earth.</li> Today, beautiful video is no longer exclusive to Blu-ray; high-end streaming services provide access to high-fidelity video anywhere with decent internet. Additionally, live video platforms let anyone show lots of people a window into their life from almost anywhere.</li> </ol> Through the combined efforts of better internet and better compression, we could see more breakthroughs in the future. Emerging immersive formats (360° video, VR/AR) and cloud gaming are opportunities for more innovation, and could unlock compelling art with enough accessibility. For now, it is worth acknowledging that we've come a really long way. Present Day</h2> The past two years have been incredibly consequential for video. I would argue this is the strongest the ecosystem has been in over a decade from a technological standpoint. The best open-source video encoder in the world is beginning to show signs of maturity, web-first image compression had its first major breakthrough since JPEG, and life has become a lot easier for developers. Let's walk through each of these stories. Video</h2> SVT-AV1 is the best open-source video encoder in the world, according to a lot of metrics. Despite this, it is still not the most mature (that title goes to x264). For SVT-AV1 to fully supersede x264, it needs to always be better; it cannot just be mostly better. Metrics say we are already here, but subjective testing tells a different story. One year ago (August 2024), the predominant narrative around AV1 was that it was "blurry". Time and energy were invested in swaying this perception, but it ultimately still held. To address this, I had started the SVT-AV1-PSY project earlier that year with the stated goal of building and testing research-grade features for subjective quality based on user feedback. In August 2025, this perception has mostly been shattered. A number of the aforementioned research-grade features have been upgraded to production-grade, with more progress happening every day. Additionally, the number of features has ballooned; SVT-AV1-PSY saw a number of additional releases, and as the project was put to rest, a few new forks emerged to perpetuate the effort. Utilities for using video encoders have improved dramatically as well; the popular chunked encoding script called Av1an saw a development resurgence to reinvigorate some older features, and other tools with similar functionality arrived on the scene to provide more options to users. The implications of these dramatic improvements are numbered. As an enthusiast, it is no longer as easy to avoid producing good-looking AV1 video. SVT-AV1 is arguably more mature than x265, and x264 is the final available contender. Considering it takes the better part of a decade to develop a robust video encoder, the fact that so much is in motion right now is exciting news. Images</h2> In August 2024, the image compression story was looking unfortunate compared to video. WebP failed to deliver exceptional compression improvements over modern JPEG encoders, and AVIF was looking promising but underwhelming. It suffered from the same "blurriness" as AV1, but it was far more severe in a world where modern image encoders tend to be well-optimized for human perception. The best standard by far was JPEG XL, which had been removed from Chrome, effectively killing any chance of ubiquitous use of the format on the web. While all of this held true, in August Julio Barba and I were working on image-focused enhancements for SVT-AV1-PSY. We announced our results publicly, and while they were promising, the implementation was limited to our community-supported video encoder. In 2025, Julio has worked diligently with Google to bring our work to libaom, the reference implementation of AV1. Due to its more complete feature set, it already has a number of image-specific performance considerations, so it was a perfect fit. The new improvements are wrapped into Tune IQ in libaom, and websites like The Guardian are already benefiting from its vastly improved consistency and compression gains. Before Tune IQ, AVIF would occasionally lose to JPEG; this is no longer true. Developers</h2> In August 2024, everyone's favorite metric is SSIMULACRA2. It is incredibly accurate to human visual perception, and helped guide certain decisions made during SVT-AV1-PSY development. It had a couple of issues, though. The reference implementation in C doesn't support videos</li> The Rust implementation ssimulacra2_rs is not fast</li> The Zig implementation is decently fast, but still not as fast as VMAF, SSIM or PSNR</li> </ul> Enter Vship, an SSIMULACRA2 and Butteraugli implementation that uses the GPU. Better developer tools enable new development paradigms, and Vship's 10–100x speed improvement opened the doors for encoder testing frameworks like PSY-EX metrics</code> that allow for streamlined encoder benchmarking automation. My blog post on comparing video encoders would have been a lot longer and more complex in 2024, but now testing an encoder's convex hull is as simple as running a single Bash script to call PSY-EX metrics tooling powered by Vship. What's Next</h2> I have a lot of hope that AV2 will rally intense community efforts in open source like AV1 did. There is a lot of performance left on the table with a standard as complex as AV2, and I hope in the next decade it will be properly realized. For now, I think we should celebrate what has happened so far. AV1 currently features: An innovative open-source decoder (dav1d) that has outperformed everyone's initial expectations for AV1</li> A performant, competitive reference encoder that is still seeing developments right now, for both images and videos</li> A competitive, performant open-source production encoder that has driven dedicated psychovisual feature implementations</li> An independent encoder in Rust</li> Some proprietary solutions that offer extra value (features, performance, quality, etc) to select clients</li> Vibrant online compression communities that are drawn to the field out of pure passion</li> </ul> In the past year, we've seen: Major advances in maturity in the world's best open-source video encoder</li> The first image compression implementation available on any browser that meaningfully surpasses JPEG</li> Faster, easier-to-use developer tools than ever before</li> </ul> I'm excited for what the next year of development has to offer. Comparing Video Encoders 2025-03-15T00:00:00+00:00 Comparing video encoders involves using synthetic metrics to assess visual quality, helping generate data that can be visualized. Plotting encoder performance in different ways helps determine the best encoder for specific implementation needs. </picture> </div> Why?</h2> Comparing video encoders isn't hard; in fact, it is usually quite easy. However, it is very often done incorrectly. That's a bit of an oversimplification. Comparing video encoders extremely well is rather difficult, and it is the focus of a lot of impactful research that aims to produce metrics that can properly assess how good a video looks to our eyes. The human eye is very complex, and guiding compression algorithms to care about the human visual system can get very interesting (I wrote a bit about perceptual color encoding in JPEG</a> with the XYB colorspace used in JPEG XL, it can be very cool stuff). This article is more about what we can do now with the tools that we have, regardless of the metric we're interested in. Many people, including the SVT-AV1 team, make use of PSNR</a>, SSIM</a>, and VMAF</a>, but today we're going to be (mainly) focusing on XPSNR</a>, a perceptual metric by Fraunhofer HHI that is readily available in FFmpeg 7.1. Now that we have established the problem space, we can talk about: The tools we have available to compute metrics in a useful way</li> How we can use them to compare video encoders</li> </ul> Tools</h2> A helpful toolbox of various scripts is provided by the metrics</code> utility by the Psychovisual Experts Group</a>. You can find the code via this GitHub link</a>. This tool lets us compute some image-focused metrics that we will use for video, and Weighted XPSNR, a video metric based on XPSNR that includes chroma information (officially, XPSNR is recommended to be luma-only). Comparisons</h2> There are three "tiers" of comparisons, each involving a bit more data than the last: Comparing a single video to another single video</li> Comparing a series of encoders in terms of compression efficiency</li> Comparing a series of encoders in terms of overall efficiency</li> </ul> We'll start with simple two-video comparisons. Comparing Two Videos</h2> XPSNR is what we call a full-reference distortion metric, which means we compare a distorted video to its source to get a score. Since we're encoding a source video with a video encoder, we can compare the source and the encode with scores.py</code>: ./scores.py [source] [encode]</code> You can also use encode.py</code> to encode the video for you. Either one will give us various statistics for the metrics we have available to us. Given we used the GPU for computation of SSIMULACRA2/Butteraugli (more on that in a second), you'll get something like this output: SSIMULACRA2 scores for every 1 frame: Average: 75.22395 Std Deviation: 3.19206 10th Pctile: 70.52215 Butteraugli scores for every 1 frame: Distance: 0.80522 Max Distance: 0.97927 XPSNR scores: XPSNR Y: 34.80490 XPSNR U: 38.48910 XPSNR V: 37.42110 W-XPSNR: 35.61793</code></pre> You'll notice that this is a lot more than just a single data point. We're just supposed to compare two videos and get a number for how the encode looks, right? Ideally, yes, but with the imperfect tools we have, we must do the best we can. SSIMULACRA2</a> The average, or the arithmetic mean, is simple; now we know how our frames look, on average, according to an image metric.</li> The harmonic mean could pull our average down toward the lower scores present in our per-frame score dataset that we're interpreting. This is theoretically a bit more informative than the average, as our eyes are going to be more sensitive to variability in the video's fidelity, so we make consistency desirable by favoring the lower-scoring frames. Note that we haven't reported the harmonic mean with SSIMULACRA2 here, as SSIMULACRA2 is capable of producing negative scores which are incompatible with the harmonic mean.</li> The standard deviation tells us more about the video's consistency.</li> The 10th percentile lets us know how our least desirable frames are scoring.</li> </ul> So, lots of ways to try to make an image fidelity metric useful for video. Butteraugli</a> The way we use Butteraugli in metrics</code>, we use 3pnorm, which weighs and averages certain parts of the frame, leaning toward more noticeable differences. So for our use case: The "Distance" is the average of per-frame 3pnorm scores</li> The "Max Distance" is the maximum 3pnorm score we saw in a frame</li> </ul> And finally, Weighted XPSNR, or W-XPSNR. This is kind of simple: Y XPSNR is "real" XPSNR, luma-only</li> U & V XPSNR are for chroma</li> W-XPSNR extrapolates mean square error from each of the three scores and computes a weighted average favoring luma, then computes back to the dB units that PSNR-derived metrics (and others) use for reporting fidelity.</li> </ul> So, Weighted XPSNR is just a weighted average for luma and chroma scores that aims to fairly favor luma since that is what our eyes care most about. Comparing Compression Efficiency</h2> Now we have scores for one video. But, what size is it? How does it compare to another video from another encoder that's a slightly different size with slightly different scores? You can interpret this subjectively, like saying your 1.74MB video at XPSNR 34.03 from Encoder A feels like a better option than a 1.81MB video that scores 34.21 from Encoder B, but how can we know for sure? The best way we can do this is by looking at a curve that plots size-to-score for a series of clips, which is meant to allow us to see which encoder (or configuration) achieves the best compression efficiency. Here's a plot comparing various SVT-AV1 speed settings: You can see that despite the fact that Preset 4 & Preset 2 produce smaller files at each CRF level, they are not the most efficient presets, because Preset 0 displays the best compression efficiency according to the curve. Each one of these curves came from an invocation of stats.py</code> that provided us with the data we wanted. Here's an example of how to use stats.py</code>: ./stats.py \ -i source.mkv \ -q "20 21 24 26 30" \ -o ./stats.csv \ -g 4 \ svtav1 -- --preset 8 --tune 2</code></pre> This encodes source.mkv</code> at 5 CRF values, then outputs the results to stats.csv</code> which include metrics and encode time. We use 4 GPU threads, and we pass a couple of options to SVT-AV1. We picked our 5 CRF values by choosing our bounds and the number of values we want, according to a formula (in Python): min_q + (max_q - min_q) * ((step / (q_steps - 1)) ** 1.5) for step in range(q_steps)</code> Rounding our results to integers (necessary with current SVT-AV1) gave us 5 CRF values between 20 and 30, according to my input. We use this formula to focus more of our data points on higher fidelity encodes, where the difference in filesize may be larger for smaller differences in fidelity. This is more helpful when working with much higher fidelity than we care about here, but it is a good thing to remember, because we want a curve with less data points to look more like one with a greater number of data points. Now, we can compare encoders by generating multiple curves. We have the data, and we can use plot.py</code> with our data inputs for a simple plot. For a hobbyist use case, this may be a fine place to stop. If it encodes in reasonable time, and it is closer to the upper left on the curve, it may satisfy you to use the more compression efficient encoder. But, what about at production scale, where you care more about time? Comparing Overall Efficiency</h2> Before moving on, consider what we need for this graph: A value representing the metric difference between two curves</li> A value representing the time difference between the encoder/configuration used for each curve</li> A graph that compares these two for various steps on a curve, which we can use to compare other curves for other encoders</li> </ul> That final encoder curve describes an encoder's overall efficiency according to a given metric. Now, let's explore how to gather each value. BD-Rate (Bjontegaard Delta Rate) is a way to compare the efficiency of two curves. It answers the question: "For the same quality level, how much more or less data does method B need compared to method A?" If you stopped at the end of the previous section and ran plot.py</code>, you'd notice that it provides BD-Rate numbers for each stats file you provided it, relative to the first stats file. So, if the BD-Rate is -20% between A & B, it means the second method needs 20% less data to achieve the same quality as the first method, which is a good improvement. plot.py</code> writes these BD-Rate values to a CSV, along with the average time computed across the encodes in each stats file. Now, your next plot.py</code> invocation (for the next stats files belonging to the next encoder) needs to use the previous worst stats file as the first argument in order to compute BD-Rates relative to the encoder you're now comparing against. You'll get another CSV output. Here's an example result, comparing SVT-AV1 v3.0.0 to SVT-AV1-PSY v2.3.0-B: BD-Rates computed relative to SVT-AV1-PSY v2.3.0-B's Preset 10, which is why that data point has a BD-Rate of 0%. You can see that SVT-AV1 v3.0.0 is able to produce smaller BD-Rates relative to v2.3.0-B in less time, so it would be considered the more efficient encoder, according to W-XPSNR. Again, even though Preset 10 in SVT-AV1 v3.0.0 has a worse BD-Rate than Preset 10 in SVT-AV1-PSY v2.3.0-B, the time difference means that along the curve SVT-AV1 v3.0.0 is more efficient overall. Conclusion</h2> That's all for now. I hope you found this blog post helpful in understanding how to compare encoders, because it is a crucial part of encoder development and can help you make an informed decision about which encoder to use for your specific needs. Happy encoding! Sponsor Me on GitHub Sponsors</h3> Help support my open source efforts - a little goes a long way! Sponsor</a> The End of SVT-AV1-PSY 2024-11-26T00:00:00+00:00 SVT-AV1-PSY, a community fork of the SVT-AV1 video encoder, was created to enhance perceptual quality and ease of use for AV1 encoding. I co-created and currently maintain the project. Our new goal is to merge our features into SVT-AV1. Why?</h2> If you aren't familiar with codecs or AV1, I'm not sure why you're on my site. Nevertheless, here's some background: AV1 is a modern video codec, meant to make videos smaller and better looking</li> Unlike H.264 and H.265, AV1 is royalty-free, meaning you don't have to pay to use it</li> There are three very popular AV1 encoders that can be used to produce AV1 video: libaom, rav1e, and SVT-AV1</li> </ul> Encoders</h3> There are three main AV1 encoders, as mentioned above: libaom from Google, rav1e from xiph, and SVT-AV1 from Intel, Netflix, Meta, and others. Historically, SVT-AV1 has been noteworthy for its impressive speed and parallelism, while libaom was known for its great quality and class-leading still image performance (AV1 has an image format associated with it called AVIF). Forks</h3> The libaom team at Google is largely insulated from community influence; this means a random individual submitting some new code has a low chance of it being accepted. This led to a community culture of forking, with the first noteworthy fork being aom-av1-psy</a> by BlueSwordM. The goal of this fork was to enhance libaom (aomenc) with better perceptual quality. This means that the encoder would be able to make better decisions about what to keep and what to throw away according to our eyes, instead of chasing metric scores that, while important, don't always reflect the characteristics of the human visual system. BlueSwordM retired aom-av1-psy in September of 2022 to focus on other work. Perceptual fidelity is not a new concept; encoders like x264 and x265 have a number of perceptually focused features like "psy-rd" and "psy-trellis" (you can read more about these elsewhere). x264 and x265 are very mature encoders for older formats, so mainstream AV1 encoders haven't had the same level of development in this area. This helped aom-av1-psy become very popular among AV1 enthusiasts, and its legacy lives on in the form of two newer forks: aom-av1-lavish</a> (retired as of June 2024) and aom-psy101</a> (still maintained at the time of writing). Usually, forks were maintained by a single individual – all of the forks I've mentioned so far have been or currently are. Coding isn't the only thing that goes into maintaining a project, so a solo dev is often responsible for documentation, community management, releases, bug reports, feature requests, testing, and more. This puts pressure on a singular developer to manage a rather complex project with a lot of eyes on it, which is difficult and has often led to project maintenance issues. All of this had been swirling around in my head at the time, because compiling aom-av1-lavish by collecting the latest experimental patches floating around Discord was kind of difficult and I was keenly interested in what a new project isolated from the aom-av1-psy legacy might look like. This brings us to SVT-AV1-PSY. How It Started</h2> I had entertained little discussions with friends about what kind of potential SVT-AV1 might have in 2023. The community often doesn't care much about speed versus perceptual quality, so many weren't that invested in seeing what could come out of SVT-AV1 as the aomenc forks were (and have always been) quite good. The rav1e encoder already had a perceptual lean, so many thought the path forward was clearly to invest as much community time as possible into rav1e. I thought this would be a herculean effort that would likely not pay off, and I was convinced that SVT-AV1 had something to offer with the incredible year over year speed improvements we kept on seeing. However, I never once thought of forking. Coincidentally, at this point a perceptual quality-related issue</a> appeared in the SVT-AV1 GitLab repository. This piqued my attention immediately, and the initial results that Variance Boost was producing looked extremely promising. While the author, Julio Barba, was sharing patch files to communicate with the dev team, excited community members were manually patching the source code and building the encoder to test it out. I was one of those people, and I was very impressed with the results, but the patching workflow was cumbersome and reminded me of my current woes with aom-av1-lavish. So, I hastily put together a GitHub repository with the patch applied to the SVT-AV1 source code alongside some minor tweaks from an SVT-AV1 fork maintained by BlueSwordM. I called it SVT-AV1-PSY, named after aom-av1-psy. Julio ended up appearing across the various AV1 Discord servers run by community members, and BlueSwordM was able to get in touch with me about how I was integrating and testing his work. I offered to maintain the project while Julio and BlueSwordM would act as the primary contributors, so they could focus on code while I dealt with everything else. We made our first release in February of 2024, and the project began to gain traction from there. How It's Going</h2> Although I didn't intend to do much programming to start, it was too tantalizing, and I had just begun learning C in October of 2023 so I thought I would try my hand. In the end, everyone on the dev team ended up contributing code, and we also attracted the attention of Clybius – the maintainer of aom-av1-lavish – who stopped working on his aom-av1-psy fork to join us. Trix, a noteworthy encoder known for their work with animation, also joined the team and provided us with a lot of valuable feedback through their extensive testing in the SVT-AV1 Deep Dive</a> blog series. I could not have asked for a better group of people to work with. The amount I have learned through them, especially Julio, has been immeasurable, and I wouldn't be nearly as competent as I am now without them. I am very grateful for their time and effort, and I am proud of what we have accomplished together. Here's a short list of what has been achieved with SVT-AV1-PSY: We've authored over 20 highly effective features, aimed at improving perceptual quality and ease of use</li> We have produced a total of 13 releases at the time of writing, introducing our own release cadence to help push finished features to users faster</li> We've built a community of over 500 members on the AV1 for Dummies</a> Discord server, where discussion mainly revolves around SVT-AV1-PSY and related projects</li> We've racked up over 250 stars on GitHub at the time of writing (more than SVT-AV1! Though that's likely due to GitHub vs GitLab)</li> We have a website</a> for showing off our results, where we notably demonstrate our AVIF performance</a>. At its inception, AVIF with SVT-AV1-PSY was the best in the world by a wide margin; libaom is competitive now due to the introduction of our breakthrough changes into libaom</li> We've recently forged a relationship with the incredibly responsive and talented mainline SVT-AV1 team, and we are working to get our features merged into the mainline encoder so everyone can enjoy them</li> </ul> SVT-AV1-PSY is the most popular community fork of a major AV1 encoder, and I am very proud of what we have accomplished. I am excited to see where we go from here, and I am grateful for the opportunity to have worked with such talented and dedicated individuals. I am also grateful for the community that has supported us, and I hope we can continue to provide you even more exciting compression technology in the future. How It Ended (?)</h2> Full disclosure, SVT-AV1-PSY isn't "ending," per se – that was just some clickbait. We are in the process of getting as many of our features as possible merged into the mainline SVT-AV1 encoder, and we are working closely with the mainline team to see that through. What's happening next is that we are slowly rolling back our involvement in SVT-AV1-PSY. Due to life, work, and other extenuating circumstances, both myself and Julio are going to be taking a step back from heavily contributing to the project for the time being. I am still going to be around to help out as the maintainer, but we are going to be less active in the day-to-day operations of the project. As of January 2025, I'm going to be stepping back from authoring perceptual quality and performance changes for reasons I will disclose soon; my involvement will mostly consist of maintaining the project and helping out with the mainline integration going forward through December. Hopefully this isn't a disappointing announcement; I think it is incredibly exciting that the community is in a position to contribute to mainline SVT-AV1 directly, and a fork only serves to duplicate efforts on that front. SVT-AV1-PSY was always meant to be a temporary project, and started as a playground for testing new features that may not have been production-ready immediately. I am very proud of what we have accomplished, and I am excited to see what the future holds for SVT-AV1. Next Steps</h2> I am a big believer in the idea that breakthrough compression performance enables brand-new applications for multimedia tech, and compression doesn't solely exist to make video files smaller for the benefit of the corporate bottom line. Saying good compression only exists to save X% bandwidth is akin to saying post production only exists to drive X% more sales; post production and visual art are creative and expressive, and compression is a tool to enable that creativity in ways that we likely can't imagine yet. I am humbled to be a part of a community that is pushing the boundaries of what is possible with video compression, and to have contributed my drop in the bucket with SVT-AV1-PSY. I am excited to see where things go from here, and I'm hopefully not leaving the multimedia compression world anytime soon. Thank you to Julio, BlueSwordM, and Clybius for all of your help, and thank you to the community for your support. I am excited to see what the future holds! Sponsor Me on GitHub Sponsors</h3> Help support my open source efforts - a little goes a long way! Sponsor</a> Stupidly Tiny Images 2024-10-07T00:00:00+00:00 How tiny can very tiny banner images get? Why?</h2> My friend Haylin</a> introduced me to 88x31 images</a> and I completely fell in love with them. She told me that this weekend I should add some to my site, so I ended up making a couple of custom ones and peering with friends on this site's home page. Fun times. However, there was a catch. It bothered me that these images were all PNGs and (shudders) GIFs straight out of the 1990s & early 2000s. I get that's the aesthetic, but I don't think it is paramount that we stick with older formats. I've been involved in the multimedia compression scene donating my time as an AV1 framework and encoder developer for a bit now, and I'd consider myself to be somewhat well-versed in digital multimedia and data compression. Recently in the image compression space, I've been focused on making lossy AVIF performant on larger, mostly photographic images</a> that you'd commonly find on the Web. The 88x31s are thus super far out of my comfort zone, so I was excited to see what I could accomplish with "off-the-shelf" tools in a day or two. Level 1: PNG</h2> The easiest question to ask ourselves here is "How can we make the existing 88x31s smaller without any compromises whatsoever?" The answer is simple: PNG optimization. PNG is (in my opinion) an extremely bloated spec compared to formats like QOI</a> that achieve the same goals at much lower complexity. The benefit of this is that there is usually a lot of room for improvement in any standard PNG bitstream you encounter online. I'm going to ignore animated GIFs and APNG's for now and just look as still images - we can start with twelve 88x31s (archive</a>) pulled straight from matdoes.dev/buttons</a>. We'll cover animations later. We have ourselves a combined dataset of GIFs and PNGs here. originals/9.png: 1339 bytes originals/4.gif: 3058 bytes originals/5.gif: 845 bytes originals/7.gif: 2206 bytes originals/12.png: 389 bytes originals/3.gif: 1099 bytes originals/linux_2.png: 1052 bytes originals/6.png: 825 bytes originals/firefox_now.gif: 2355 bytes originals/linux_1.gif: 583 bytes originals/11.gif: 667 bytes originals/10.gif: 2523 bytes Total size: 16941 bytes Number of items: 12 Average size: 1411 bytes</code></pre> We're weighing in at an average of 1411 bytes per image. That's not too bad, but we can do better even with the most basic, low-effort PNG encoding. Using magick -quality 0</code> we can get some PNGs out of the static GIFs. Here's the effect on the filesize: png_unoptim/9.png: 1339 bytes png_unoptim/12.png: 389 bytes png_unoptim/linux_1.png: 721 bytes png_unoptim/firefox_now.png: 2352 bytes png_unoptim/11.png: 752 bytes png_unoptim/linux_2.png: 1052 bytes png_unoptim/10.png: 2458 bytes png_unoptim/4.png: 3137 bytes png_unoptim/5.png: 957 bytes png_unoptim/7.png: 1621 bytes png_unoptim/6.png: 825 bytes png_unoptim/3.png: 1213 bytes Total size: 16816 bytes Number of items: 12 Average size: 1401 bytes</code></pre> Even low effort, completely unoptimized PNG beats GIF here on average. GIF sucks, please don't use GIF. Now let's see what happens if we use ect -9 --mt-deflate -strip</code> on the PNGs: png_ect9/9.png: 1326 bytes png_ect9/12.png: 369 bytes png_ect9/linux_1.png: 458 bytes png_ect9/firefox_now.png: 2050 bytes png_ect9/11.png: 495 bytes png_ect9/linux_2.png: 1029 bytes png_ect9/10.png: 2182 bytes png_ect9/4.png: 2765 bytes png_ect9/5.png: 706 bytes png_ect9/7.png: 1316 bytes png_ect9/6.png: 808 bytes png_ect9/3.png: 1006 bytes Total size: 14510 bytes Number of items: 12 Average size: 1209 bytes</code></pre> It doesn't seem like a crazy difference, but considering how little flexibility we have with such tiny images, it is pretty impressive that we're looking at the same exact format and the same image data. Let's take it a step further with ect -9999 -strip</code>: png_ect9/9.png: 1324 bytes png_ect9/12.png: 368 bytes png_ect9/linux_1.png: 454 bytes png_ect9/firefox_now.png: 2048 bytes png_ect9/11.png: 492 bytes png_ect9/linux_2.png: 1027 bytes png_ect9/10.png: 2179 bytes png_ect9/4.png: 2761 bytes png_ect9/5.png: 706 bytes png_ect9/7.png: 1311 bytes png_ect9/6.png: 806 bytes png_ect9/3.png: 1003 bytes Total size: 14479 bytes Number of items: 12 Average size: 1206 bytes</code></pre> Look at that, a teensy bit more off. This is pretty much right up against the ceiling of what PNG can do for us without lossy techniques like palette compression. PNG Summary</h3> TL;DR, convert your GIFs to PNGs and run them through ect -9 --mt-deflate -strip</code>. With images this tiny, it is super fast, and you'll shave off some bits at zero cost to compatibility or fidelity. But what if we want to go further? Level 2: Lossless WebP</h2> I know WebP has a bit of a reputation on the Web as having been the format that was shoved down our throats by Google and didn't really ever become a new standard for anything. I've talked in the past about lossy WebP being barely competitive with JPEG despite being far newer, but overall I can't say I dislike WebP as much as the next person. There are some important things to consider with WebP: WebP is mostly talked about as a lossy format, but it can be lossless.</li> WebP's lossless mode is really impressive, and consistently outperforms the most optimized PNGs.</li> WebP's animation support is pretty complete, especially compared to the mess that is GIF and the relatively obscure APNG. Discord recently supports animated WebP by default.</li> </ul> I'm going to go in guns blazing with max-effort bruteforce WebP lossless via cwebp</code>. I have a script</a> that tries every encoding effort level from 0 to 9 and picks the smallest one, as effort 9 isn't always the smallest for some reason. Here's the result: webp_l/5.webp: 670 bytes webp_l/linux_2.webp: 700 bytes webp_l/9.webp: 1114 bytes webp_l/4.webp: 2456 bytes webp_l/3.webp: 954 bytes webp_l/firefox_now.webp: 1724 bytes webp_l/12.webp: 296 bytes webp_l/11.webp: 456 bytes webp_l/10.webp: 1866 bytes webp_l/7.webp: 1182 bytes webp_l/linux_1.webp: 370 bytes webp_l/6.webp: 744 bytes Total size: 12532 bytes Number of items: 12 Average size: 1044 bytes</code></pre>WebP Summary</h3> An additional meaningful improvement with a bare compatibility cost and still no loss of fidelity whatsoever. This is the power of WebP's lossless mode. It's almost as significant as the gap between PNG and GIF. WebP is worth using for something like this. It is fair to have hesitation about adopting a slightly less common format, but WebP is supported across all major browsers. If your favorite app doesn't support WebP, it isn't WebP's fault at this point; it is a royalty free format that has its merits, and shouldn't be bastardized an account of its shaky introduction. But what if you don't care at all about compatibility, and you just want the smallest possible file size without fidelity loss? Step 3: JPEG XL</h2> JPEG XL is an incredibly cool format. I have plenty of additional JPEG XL coverage on my blog, and sites like the community-run JPEG XL site</a> exist if you'd like more information, but the TL;DR is that JPEG XL is pretty much great at everything. It is a royalty-free, modern format that is designed to be the best of all worlds, and it is a format that I am very excited about. Unfortunately, Google's Chromium team notoriously rejected JPEG XL from the Chromium browser engine, so it is not supported in Chrome or any browser that uses Chromium. This is a huge bummer. JPEG XL is supported in Safari and throughout the Apple ecosystem, as well as in browsers like Waterfox and Thorium, so it is fair game to use JPEG XL images on your site with an appropriate fallback. For our purposes, we're just going to look at JPEG XL's lossless compression with cjxl 0.11.0</code>. We'll be using the hidden super high effort mode via cjxl -d 0 -e 11 --allow_expert_options</code> to see how low we can go: jxl_l/11.jxl: 417 bytes jxl_l/10.jxl: 2394 bytes jxl_l/linux_2.jxl: 740 bytes jxl_l/12.jxl: 279 bytes jxl_l/linux_1.jxl: 369 bytes jxl_l/firefox_now.jxl: 2505 bytes jxl_l/9.jxl: 1216 bytes jxl_l/3.jxl: 846 bytes jxl_l/7.jxl: 1244 bytes jxl_l/6.jxl: 773 bytes jxl_l/4.jxl: 3403 bytes jxl_l/5.jxl: 590 bytes Total size: 14776 bytes Number of items: 12 Average size: 1231 bytes</code></pre> ...Hm. Not exactly what was promised. The reality is that JPEG XL's lossless mode is best with larger images, and images with >8 BPC. Especially for HDR, JPEG XL significantly outperforms high bit depth PNG, and WebP doesn't support >8 BPC. For tiny images like this, it is clearly not the best choice given what we're seeing here. The one saving grace we have at our disposal with JPEG XL is the super useful tools it makes available to us to do slightly lossy compression. We can do this by lowering the bit depth the encoder uses, or allowing JPEG XL's modular lossless compression to utilize a lossy palette. PNG and WebP are capable of similar tricks as well (like PNG's palette compression mode and WebP's "near lossless" compression), but JPEG XL does it better via cjxl in my opinion. Through adding --modular_lossy_palette --modular_palette_colors=0</code>: jxl_lossy/11.jxl: 638 bytes jxl_lossy/10.jxl: 1677 bytes jxl_lossy/linux_2.jxl: 946 bytes jxl_lossy/12.jxl: 281 bytes jxl_lossy/linux_1.jxl: 608 bytes jxl_lossy/firefox_now.jxl: 1589 bytes jxl_lossy/9.jxl: 1248 bytes jxl_lossy/3.jxl: 996 bytes jxl_lossy/7.jxl: 1403 bytes jxl_lossy/6.jxl: 773 bytes jxl_lossy/4.jxl: 1583 bytes jxl_lossy/5.jxl: 746 bytes Total size: 12488 bytes Number of items: 12 Average size: 1040 bytes</code></pre> This is our best result yet, albeit a lossy one. Visual inpection will allow you determine if this is too lossy for you, in which case it may be best to stick with WebP. JPEG XL Summary</h3> Overall, JPEG XL results seem mixed at this scale, and we've been driven to resorting to slightly lossy compression which isn't a great showing for the codec. However, if you look closely, there are some images that are the smallest with JPEG XL... jxl_l/11.jxl: 417 bytes jxl_l/12.jxl: 279 bytes jxl_l/linux_1.jxl: 369 bytes jxl_l/3.jxl: 846 bytes jxl_l/6.jxl: 773 bytes jxl_l/5.jxl: 590 bytes</code></pre> Exactly half the pictures were smallest with cjxl. The reference library for JPEG XL (called libjxl) is still pre-1.0, so maybe the bitstream has latent expressivity in its modular lossless mode that we've yet to see realized. The creators of the format have mentioned that peak bitstream expressivity has yet to be reached with libjxl especially as it pertains to lossy encoding, so only time will tell. Mixing Codecs</h2> By mixing our favorable results from WebP and JPEG XL, we minify our 88x31s to the maximum degree: mix/11.jxl: 417 bytes mix/12.jxl: 279 bytes mix/linux_2.webp: 700 bytes mix/9.webp: 1114 bytes mix/linux_1.jxl: 369 bytes mix/4.webp: 2456 bytes mix/firefox_now.webp: 1724 bytes mix/3.jxl: 846 bytes mix/10.webp: 1866 bytes mix/6.jxl: 773 bytes mix/5.jxl: 590 bytes mix/7.webp: 1182 bytes Total size: 12316 bytes Number of items: 12 Average size: 1026 bytes</code></pre> This gives us, with no fidelity loss, our smallest average size yet. This is a great result, and it shows that inspecting your outputs with compression is almost always worth doing. Animation</h2> For animated images, we have three major options: GIF</li> APNG</li> Animated WebP</li> </ul> I'll spare you the misery - Animated WebP is the best choice for animated images. It's the smallest, and it's supported in all major browsers. GIF is supported everywhere, but it's the largest. APNG encoding with FFmpeg: ffmpeg -y -i {} -plays 0 {.}.apng</code> Animated WebP encoding with FFmpeg: ffmpeg -y -i {} -pix_fmt bgra -c:v libwebp_anim -lossless 1 {.}.webp</code> Using this image</a>: du -h upallnight.* 104K upallnight.apng 104K upallnight.gif 72K upallnight.webp</code></pre> In a bit more detail: stat -f '%z' upallnight.* 106127 # APNG 106384 # GIF 70712 # WebP</code></pre> I know single image comparisons are flawed, but I've done more than just this one and the results are consistent. APNG can often make GIFs a bit smaller, but WebP crushes it here. You can optimize APNG further, but there isn't (usually) a compatibility benefit - animated WebP is the way to go. My Approach</h2> The 88x31s on my homepage are a combination of a number of different coding techniques. I prioritized getting the maximum fidelity out of JPEG XL, and used lossless WebP as a fallback. My personal 88x31 is a lossy palette JPEG XL, while the JPEG XL community button is a lossy VarDCT JPEG XL. I quantized the bit depth more aggressively with some images at very little fidelity loss, and others I was able to use full lossless. Doing each of your images individually is a great way to get the best results, but can be time consuming. Everything animated is animated WebP, as animated JPEG XL isn't supported on Safari so the majority of people seeing my JPEG XL animations wouldn't see anything but a still frame. Conclusion</h2> My final thoughts: For lossless compression, WebP consistently outperformed PNG and GIF, offering meaningful size reductions at a minimal compatibility tradeoff.</li> JPEG XL showed mixed results at this small scale but was able to achieve the smallest file sizes for about half the images using lossless mode. Its slightly lossy palette mode produced the smallest average size overall, but it is still lossy so you may want to use with caution.</li> Using WebP for some images and JPEG XL for others yielded the best overall lossless compression results.</li> For animated images, WebP is clearly superior to both GIF and APNG in terms of file size while still being lossless (lossy WebP is quite good, too; it is a video codec under the hood, after all).</li> Carefully evaluating each image individually with different codecs and settings is the key to achieving the best results.</li> </ul> Even for tiny, simple images, modern compression techniques can yield meaningful improvements over legacy formats. It's worth taking the time to experiment with different codecs and settings to find the best balance of quality and file size for your images. You can argue it isn't practical because the images are already so tiny, and you'd probably be right, but this is more about doing it for sport. So, mess around a bit and see what you can do - it was a lot of fun for me, and as the "compression guy," I can't be caught dead using unoptimized images anywhere :) Peer with me: <a href="https://giannirosato.com" target="_blank"> <picture> <source srcset="https://giannirosato.com/static/images/88x31/gb82_88x31.jxl" type="image/jxl" > <img src="https://giannirosato.com/static/images/88x31/gb82_88x31.webp" alt="gianni" width="88" height="31" > </picture> </a></code></pre>Sponsor Me on GitHub Sponsors</h3> Help support my open source efforts - a little goes a long way! Sponsor</a> QOI Encoding in Zig 2024-03-13T00:00:00+00:00 Writing a fast and simple QOI encoder; illustrated with code examples, a spec breakdown, and step by step implementation instructions. Note: the code for this project on GitHub has changed, but all of my commentary here remains valid. Why?</h2> I started writing C as my first programming language in October. While I have other projects labelled under the programming tag on this site, a friend was the lead developer on both, and I was just becoming familiar with Python; I never truly learned the concepts until later, as I hadn't needed to until I took over development for those projects. I had dabbled with Bash scripts, but other than that, I have had very limited programming exposure until recently, though I've had extensive exposure to technology. Becoming familiar with C quickly led me to discover Zig in November. Around the same time, I stumbled upon the QOI (Quite OK Image Format) lossless image format built around speed and simplicity. The simplicity part was what caught my eye; I'm a new programmer, and writing a QOI encoder would simultaneously allow me to get down and dirty with manipulating data on a low level without getting into the weeds of a more complicated spec like that of PNG. Here's the project link, before we get too carried away: GitHub Link</a> As soon as I got started with Zig, I already felt it was cool; Rust felt (and continues to feel) intimidating as I was becoming acquainted with C. "Higher-level" languages don't interest me very much. Full disclosure: I'm not going to go through everything here, and I may get some terminology details wrong. Please reach out to correct me if you notice anything. Zig</h2> Zig is a programming language designed to be the modern replacement for C. I won't go over every advantage Zig brings to the table over other languages, but according to Zig's official site</a>, Zig has some unique advantages over languages like C++, D, and Rust: No hidden control flow or allocations: Zig code is explicit about function calls and memory allocation, so readability is improved and the programmer has more control.</li> First-class support for no standard library: Zig can be used without any standard library, making it friendly for bare-metal and high-performance development.</li> A portable language for libraries: Zig aims to be a new portable language for libraries, interacting well with C code while introducing safety and better language design.</li> A package manager and build system: Zig comes with a capable build system and package manager, useful even for existing C/C++ projects.</li> Simplicity: Zig avoids unclear features like macros and operator overloading found in other languages.</li> Tooling: Zig provides a comprehensive toolchain with cross-compilation (which is very cool), C/C++ support, resource compiler, and more, packaged in a single archive.</li> </ol> Zig positions itself as a simpler, more explicit, more portable language than C++, D, and Rust while offering more powerful tooling, which is all fine and dandy, but I'm not super familiar with Rust, D, or C++. Compared to C, Zig's additional safety features like safety-checked undefined behavior and the defer</code> keyword to prevent issues one could normally have with free()</code> in C are what make Zig special and interesting to me. Its focus on simplicity and safety makes it a compelling choice for systems programming tasks, including working with multimedia codecs. QOI Overview</h2> QOI is very cool because it is extremely simple while still being relatively efficient for a lossless image codec. A QOI file consists of a 14-byte header, followed by any number of data “chunks” and an 8-byte end marker. (spec</a>) </blockquote> QOI supports two colorspaces: Linear RGB & sRGB with linear alpha. These do not affect the way pixels are encoded. Pixels (well, "chunks" according to the spec, but I find it more intuitive to call them pixels as that is how they are interpreted and represented on a screen) have either three or four channels; they are either RGB or RGBA. The QOI philosophy is that everything is byte-aligned, making it trivial to stream bytes to a decoder one at a time and have the decoder produce a usable output. QOI's lossless compression takes place via six main pixel types: SPEC_NAME</code> ("Name I Made Up"): Description from spec. I'm only making up a name so I can reference it elsewhere more easily. </li> QOI_OP_RGB</code> ("RGB Pixels"): Full RGB pixel value using 8 bits for each of the red, green, and blue channels. The alpha channel is 255 in RGB images, and remains unchanged from the previous pixel. </li> QOI_OP_RGBA</code> ("RGBA Pixels"): Full RGBA pixel value using 8 bits for each of the red, green, blue, & alpha channels. </li> QOI_OP_DIFF</code> ("Diff Pixels"): The difference between the current pixel and the previous pixel for the red, green, and blue channels are stored using 2 bits for each channel. The differences are stored with a bias of 2 and wrap (so 1 minus 2 would be 255). The alpha channel remains unchanged. </li> QOI_OP_LUMA</code> ("Luma Pixels"): These pixels encode the green channel difference from the previous pixel using 6 bits, and then encode the red and blue channel differences relative to the green channel difference using 4 bits each. This allows for more efficient encoding of small color changes. The alpha channel remains unchanged. </li> QOI_OP_RUN</code> ("Run Pixels"): These are the simplest, encoding a run-length of pixels that are identical to the previous pixel. The run length is stored using 6 bits with a bias of -1, allowing for runs of 1 to 62 pixels. </li> QOI_OP_INDEX</code> ("Hashed Pixels"): These are stored by referencing a previously seen pixel value from a rolling array of 64 recent pixel values by using a simple hash on each pixel as it is identified. If another pixel matches a previously seen hash value in the array, the index of the referenced pixel is stored. </li> </ol> That's pretty much it! The entire spec is one page</a> if you'd like a bit more detail. Amazingly, QOI trades blows with libpng's PNG encoding while being much faster</a> on average with the reference encoder given the test corpus the author used in that benchmark. Explaining the Codebase</h2> Now that I've done the requisite amount of shilling, it is time to get into the real content. I want this to be a thorough walk-through of my decisions writing this encoding implementation, so you can take inspiration from my work on your own if you are also new to Zig. Again, the source code for my project can be found at this git repository</a>. The program starts by importing the necessary standard library modules and defining the necessary data structures. The QoiEnum</code> enum, for instance, defines the different opcodes used in the QOI format, such as QOI_OP_RGB</code> and QOI_OP_RGBA</code>: const QoiEnum = enum(u8) { QOI_OP_RGB = 0xFE, QOI_OP_RGBA = 0xFF, QOI_OP_INDEX = 0x00, QOI_OP_DIFF = 0x40, QOI_OP_LUMA = 0x80, QOI_OP_RUN = 0xC0, };</code></pre> These are fundamental to the QOI format's compression techniques, which we'll explore in a bit. Next, we have the QoiPixel</code> union, which represents a single pixel's color values. By using a union, we can access the pixel's channels (red, green, blue, alpha) individually through the channels</code> array or the vals</code> struct. We can also access them as a single unsigned 32-bit integer through concatenated_pixel_values</code>: const QoiPixel = extern union { vals: extern struct { red: u8, green: u8, blue: u8, alpha: u8, }, channels: [4]u8, concatenated_pixel_values: u32, };</code></pre> Important to note here is that the extern</code> keyword defines the union (and the enclosed struct) as C ABI compatible data structures. In Zig's standard unions without the extern</code> keyword, trying to access inactive fields of the union is safety-checked undefined behavior. Your code will compile successfully, but if you compiled in Safe mode (which you probably did by default with zig build</code>) you'll get some runtime errors accompanied by a stack trace. I enjoy the fact that Zig allows you to use extern union</code>s to essentially get different "viewpoints" on how to access the underlying data in memory while maintaining a different native paradigm; it is a testament to the language's ability to espouse flexibility while still providing a feeling of familiarity. The QoiDesc</code> struct holds essential information about the image being encoded, including its dimensions, color channels, and colorspace, in order to write the header for the encoded file. This struct provides methods for assigning our instance of QoiDesc</code> & writing the QOI header to the output file: const QoiDesc = struct { width: u32 = 0, height: u32 = 0, channels: u8 = 0, colorspace: u8 = 0, fn qoiSetEverything(w: u32, h: u32, ch: u8, c: u8) QoiDesc { return QoiDesc{ .width = w, .height = h, .channels = ch, .colorspace = c }; } fn writeQoiHeader(self: QoiDesc, dest: *[14]u8) void { // ... (implementation omitted for brevity) } };</code></pre> Struct methods in Zig are not super different from normal functions. According to Zig's website: Struct methods are not special, they are only namespaced functions that you can call with dot syntax. (1</a>) </blockquote> Using these within the scope of struct definitions is cool because it makes the codebase nice and neat. You can also call methods with dot syntax, which feels tidy: // Declare variable `desc` providing width, height, channels, colorspace. // We can call this method using dot syntax on the type: var desc = QoiDesc.qoiSetEverything(width, height, channels, colorspace); // Now, we use dot syntax on the `desc` variable to call the // `writeQoiHeader` method, where `desc` is passed to the method as `self`: desc.writeQoiHeader(qoi_file[0..14]);</code></pre> You don't need to do foo->bar</code> like in C, either; Zig will do that automatically if you just do foo.bar</code>. I like this! The QoiEnc</code> struct encapsulates the encoding logic. It maintains a pixel buffer, a run counter, and various other state variables necessary for the encoding process. The qoiEncInit</code> method initializes the encoder with the image description & an output buffer: const QoiEnc = struct { buffer: [64]QoiPixel, prev_pixel: QoiPixel, pixel_offset: usize, len: usize, data: [*]u8, offset: [*]u8, run: u8, pad: u24, fn qoiEncInit(self: *QoiEnc, desc: QoiDesc, data: [*]u8) !void { // ... (implementation omitted for brevity) } // ... (other methods omitted for brevity, but there are a bunch) };</code></pre> Now, remember the types of pixels allowed in QOI that we enumerated earlier? The QoiEnc</code> struct provides separate methods for encoding each of these, including qoiEncRun</code>, qoiEncLuma</code>, qoiEncIndex</code>, qoiEncFullColor</code>, and qoiEncDifference</code>. Let's take a closer look at the qoiEncRun</code> method, which handles run-length encoding: fn qoiEncRun(self: *QoiEnc) void { const tag: u8 = @intFromEnum(QoiEnum.QOI_OP_RUN) | (self.run - 1); self.run = 0; self.offset[0] = tag; self.offset += 1; }</code></pre> This is a "Run pixel" which is the simplest kind of pixel conceptually within QOI, in my opinion. This method writes a tag byte to the output buffer, indicating a run of identical pixels. The tag byte consists of the QOI_OP_RUN</code> opcode combined with the run length minus one via a logical or. The qoiEncLuma</code> method, on the other hand, handles encoding "Luma Pixels": fn qoiEncLuma(self: *QoiEnc, green_diff: i8, dr_dg: i8, db_dg: i8) void { const green_diff_biased: u8 = @intCast(green_diff + 32); const dr_dg_biased: u8 = @intCast(dr_dg + 8); const db_dg_biased: u8 = @intCast(db_dg + 8); const tags = [2]u8{ @intFromEnum(QoiEnum.QOI_OP_LUMA) | green_diff_biased, dr_dg_biased << 4 | db_dg_biased }; for (tags, 0..) |tag, i| self.offset[i] = tag; self.offset += tags.len; }</code></pre> This method calculates the differences between the red, green, and blue channels of the current pixel and the previous pixel. It then encodes these differences using a luma tag and bias values, as we already explained when we talked about QOI. Encoding "Diff Pixels" with qoiEncDifference</code> is a similar story: fn qoiEncDifference(enc: *QoiEnc, red_diff: i32, green_diff: i32, blue_diff: i32) void { const green_diff_biased: u8 = @intCast(green_diff + 2); const red_diff_biased: u8 = @intCast(red_diff + 2); const blue_diff_biased: u8 = @intCast(blue_diff + 2); const tag: u8 = @intFromEnum(QoiEnum.QOI_OP_DIFF) | red_diff_biased << 4 | green_diff_biased << 2 | blue_diff_biased; enc.offset[0] = tag; enc.offset += 1; }</code></pre> The bit manipulations for properly writing to enc.offset</code> felt a bit intimidating, and to be honest, they still do & probably always will. Thinking about the actual structure of the pixels being written to the file, it begins to make a bit more sense thinking about the fact that the pixels are stored in order of red's difference, then green's difference, then blue's difference, with a tag preceding them; the entire pixel is a single byte, the differences are just being ordered within that byte. Now, let's look at encoding "RGB Pixels" & "RGBA Pixels" with qoiEncFullColor</code>: fn qoiEncFullColor(enc: *QoiEnc, px: QoiPixel, channels: u8) void { var s: u3 = 0; if (channels > 3) s = 5 else s = 4; const tags: [5]u8 = if (channels > 3) .{ @intFromEnum(QoiEnum.QOI_OP_RGBA), px.vals.red, px.vals.green, px.vals.blue, px.vals.alpha, } else .{ @intFromEnum(QoiEnum.QOI_OP_RGB), px.vals.red, px.vals.green, px.vals.blue, undefined, }; for (tags[0..s], 0..s) |tag, i| enc.offset[i] = tag; enc.offset += s; }</code></pre> Here, we have some trademark "Zigginess". Let's walk through it: The method takes a pointer to a QoiEnc, which is essentially "self"</li> A QoiPixel, and</li> the number of channels. Within the method:</li> We declare a variable s</code> which is a mere 3 bits</li> We're making tags</code> an array of 5 unsigned 8-bit integers, and assigning different values to the array depending on the number of channels.</li> </ul> Let's stop here for a moment. As you can see, these if</code> expressions are being used more like ternary expressions would be in C. The assignment to tags</code> is being done via an anonymous list literal. These confused me greatly when first learning Zig, but it makes a bit more sense looking at a more concise example: // Initialize an array with array literal syntax. Compiler infers size const arr_a = [_]i6{6, 7, 8}; // Initialize an array with an anonymous list literal. Compiler cannot // infer size, as it isn't aware of the anonymous list literal const arr_b: [3]i6 = .{6, 7, 8};</code></pre> Now we can observe the for</code> loop at the end of the function block. I like the way Zig handles for loops, essentially leaving traditional C-style i++</code> iteration for while</code> loops. What we're essentially doing in this loop is slicing tags</code> from index 0 to the index before s</code>. A slice is a pointer and a length. The difference between an array and a slice is that the array's length is part of the type and known at compile-time, whereas the slice's length is known at runtime. Both can be accessed with the len</code> field. (2</a>) </blockquote> We're also keeping track of 0 through the integer just preceding s</code> with 0..s</code>. The values are captured with |tag, i|</code>, where we use i</code> to assign values with tag</code> to elements in enc.offset</code>. Finally, we add s</code> to enc.offset</code>, as s</code> represents the number of bytes we need, and enc.offset</code> is a u8 multi-pointer. This felt pretty cool, to have for</code> loops and if</code> statements so gracefully doing my bidding based on little s</code>. How about we take a look at encoding "Hashed Pixels" next, now that we're getting the hang of things? fn qoiEncIndex(enc: *QoiEnc, index_pos: u8) void { const tag: u8 = @intFromEnum(QoiEnum.QOI_OP_INDEX) | index_pos; enc.offset[0] = tag; enc.offset += 1; }</code></pre> This is pretty straightforward. We calculate the hash itself outside of the function with const index_pos: u6 = @truncate(cur_pixel.vals.red *% 3 +% cur_pixel.vals.green *% 5 +% cur_pixel.vals.blue *% 7 +% cur_pixel.vals.alpha *% 11);</code>. The *%</code> & +%</code> just mean wrapping addition/multiplication, which we understand from the QOI pixel descriptions. We need our numbers to wrap around. The built-in @truncate</code> helps too, as it ensures we don't overflow a u6</code>; it truncates bits from an integer type resulting in a smaller integer type, so it is taking the low 6 bits of our hash result because there are only 64 entries in our index table. The enc.offset</code> is incremented by 1, because the pixel is a single byte. That's all the pixels! Now, the qoiEncodeChunk</code> function is where the magic happens. It iterates over the input pixel data, compares each pixel with the previous one, and chooses the optimal encoding operation based on the differences between the pixels. The encoded data is then written to the output buffer using the respective encoding methods: fn qoiEncodeChunk(desc: *QoiDesc, enc: *QoiEnc, qoi_pixel_bytes: [*]u8) void { var cur_pixel: QoiPixel = undefined; // ... (omitted for brevity. this is just deciding "alpha or no alpha?" // and writing relevant data to the current pixel.) const index_pos: u6 = @truncate(cur_pixel.vals.red *% 3 +% cur_pixel.vals.green *% 5 +% cur_pixel.vals.blue *% 7 +% cur_pixel.vals.alpha *% 11); if (qoiComparePixel(cur_pixel, enc.prev_pixel, desc.channels)) { // Handle "run pixel" encoding // ... } else { if (enc.run > 0) { enc.qoiEncRun(); } if (qoiComparePixel(enc.buffer[index_pos], cur_pixel, 4)) { // Handle "hash pixel" encoding // ... } else { enc.buffer[index_pos] = cur_pixel; // Handle various encoding operations based on pixel differences // ... } } enc.prev_pixel = cur_pixel; enc.pixel_offset += 1; // Handle padding at the end of the image once it is time to do so // ... }</code></pre> You can see the index_pos</code> hash calculation in there, where it happens, and what triggers it. Hopefully, everything else is clear, too. Alongside the others, this function demonstrates Zig's ability to work with low-level data structures and perform bit-level operations. Main Function</h2> Now that we understand everything we can do to encode a QOI image, let's look at the main</code> function. This main</code> function serves as a great example of Zig's clean and readable syntax as well as QOI's simplicity. It parses command-line arguments, handles file I/O, and orchestrates the encoding process while staying concise. Error handling is easy in Zig, with the error union !void</code> ensuring any errors are dealt with as they are returned. I'd like to expand on error unions more, but I'll save that for another article. pub fn main() !void { // Get allocator var gpa = std.heap.GeneralPurposeAllocator(.{}){}; const allocator = gpa.allocator(); defer _ = gpa.deinit(); // Parse args into string array (error union needs 'try') const args = try std.process.argsAlloc(allocator); defer std.process.argsFree(allocator, args); if (std.mem.eql(u8, args[1], "-h") or std.mem.eql(u8, args[1], "--help") or args.len < 4 or args.len > 4 or args[1].len < 1) { _ = try printHelp(); return; } var width: u32 = undefined; var height: u32 = undefined; var channels: u8 = undefined; const colorspace: u8 = try parseInt(u8, args[3], 10); print("Opening {s} ... ", .{args[1]}); const file = try std.fs.cwd().openFile(args[1], .{ .mode = .read_only }); defer file.close(); const bytes_read = try file.readToEndAlloc(allocator, std.math.maxInt(usize)); defer allocator.free(bytes_read); var offset: usize = 0; if (std.mem.eql(u8, bytes_read[0..2], "P7")) { print("file is a PAM\n", .{}); offset = try parsePamHeader(bytes_read[0..72], &width, &height, &channels); } else { print("\n\x1b[31mInvalid Input: Input file does not appear to be a compatible PAM.\x1b[0m\n", .{}); print("If your PAM input contains comments in the header, please strip them.\n", .{}); return error.InvalidInput; } const image_size: usize = width * height * channels; if (image_size > bytes_read.len - offset) { print("\x1b[31mInvalid Input: Image size is larger than the file size.\x1b[0m\n", .{}); return error.InvalidInput; } var desc = QoiDesc.qoiSetEverything(width, height, channels, colorspace); const qoi_file_size = @as(usize, desc.width) * @as(usize, desc.height) * (@as(usize, desc.channels) + 1) + 14 + 8 + @sizeOf(usize); var qoi_file = try allocator.alloc(u8, qoi_file_size); defer allocator.free(qoi_file); print("Writing {s} ... ", .{args[2]}); desc.writeQoiHeader(qoi_file[0..14]); var pixel_seek: [*]u8 = bytes_read[offset..].ptr; var enc: QoiEnc = undefined; try enc.qoiEncInit(desc, qoi_file.ptr); while (!(enc.pixel_offset >= enc.len)) { qoiEncodeChunk(&desc, &enc, pixel_seek); pixel_seek += desc.channels; } const used_len = @intFromPtr(enc.offset) - @intFromPtr(enc.data); const outfile = try std.fs.cwd().createFile(args[2], .{ .truncate = true }); defer outfile.close(); _ = try outfile.writeAll(enc.data[0..used_len]); print("\x1b[32mSuccess!\x1b[0m\n\tOriginal:\t{d} bytes\n\tCompressed:\t{d} bytes ", .{ image_size + offset, used_len }); if ((image_size + offset) > used_len) { const used_len_flt: f64 = @floatFromInt(used_len); const image_size_flt: f64 = @floatFromInt(image_size + offset); const percent_dec: f64 = 100.0 - ((used_len_flt / image_size_flt) * 100.0); print("(\x1b[33m{d:.2}%\x1b[0m smaller)\n", .{percent_dec}); } else { print("\n", .{}); } }</code></pre> That's it, the entire main</code> function. It could probably be made even more concise with some code golfing, but for now, I'm happy with how readable it is even without comments. Coming from any other low-level language background, I think this function is relatively easy to parse. This is a big argument for Zig, in my opinion - transitioning C projects is not a massive burden if you have interoperability as well as a codebase that feels familiar to long-time C developers. Even as a relatively inexperienced C developer, Zig was trivial to pick up. Conclusion</h2> Exploring QOI encoding in Zig has been an incredibly rewarding ordeal. Not only did this experience allow me to dive into the intricacies of simple lossless image compression, but it also showcased to me the power and versatility of Zig. Full disclosure, I may have gotten some details incorrect here, so corrections may be imminent. I'll add a note to the very top of this page if I make any larger corrections to the article. If you're a programmer seeking a modern systems programming language, I highly recommend giving Zig a try. If you have any questions, comments, or concerns, feel free to reach out to me via my socials on this site's homepage. Thanks for reading! Thank you to Cancername for their help with the QOI encoding implementation! Freestanding QOI Encoder in Zig: GitHub Link</a> (no code comments yet, coming soon). Lossless Data Compression 2023-10-02T00:00:00+00:00 Comparing the efficacy of various lossless data compression algorithms on text data, comparing time to compress vs output size. The algorithms tested include ZIP, 7-zip, XZ, Brotli, Zstandard, and zpaq. Lossless Data Compression</h1> While I've covered image compression a decent amount in these sorts of blog posts, I wanted to dive into generic lossless data compression a bit more to see what this interesting realm of compression has to offer. Inspired by the many other existing lossless data compression comparisons, I wanted to approach this with visualizations similar to the ones I provide in my image codec comparisons. Because lossless data compression doesn't discard data, these graphs plot time vs. compressed size, allowing you to compare the various compression steps of lossless data compression algorithms relative to the time taken to compress/decompress. Methodology & Algorithms</h1> I want to make it clear upfront that I did not thoroughly test every single available compression algorithm under the sun, but instead hand-picked a couple that I found interesting. Notably not present are gzip & bzip2; gzip has an interesting history, but its use of Deflate & LZ77 makes it pretty much identical to ZIP for all intents & purposes. Bzip2 is interesting, but due to its age & effectiveness compared to more modern compression algorithms, I found it less interesting than more modern options. It also decompresses slower than the more space-efficient XZ. Feel free to pique my interest in these two algorithms if you're interested in seeing them included. I must provide the usual disclaimer as well that my testing is non-scientific & shouldn't inform your use of one technology over another. That being said, here are the formats I tested: ZIP (via 7zip)</li> 7zip</li> XZ</li> Brotli</li> Zstandard</li> zpaq</li> </ul> & here are the parameters I used for compression, often after archiving to a .tar: ZIP: 7zzs a -bso0 -tzip -mx[level] "out" "input"</code></li> 7zip: 7zzs a -bso0 -mx[level] "out" "input"</code></li> XZ: xz -q -q -T0 -k -[level] "input" "out"</code></li> Brotli: brotli -q [level] "input" -o "out"</code></li> Zstandard: zstd -q --ultra -[level] -T0 "input" -o "out"</code></li> zpaq: zpaq a "out" "input" -m[level]</code></li> </ul> Finally, here are my system specs: CPU: 16-core (8-mt/8-st) 13th Gen Intel Core i7-13700K (-MST AMCP-) speed/min/max: 888/800/5300:5400:4200 MHz Kernel: 6.5.5-zen1-1-zen x86_64</code></pre> If you'd like to do a deeper dive into my methodology, I have my script compression-plotter</code> available on GitHub</a>. Given you have all the dependencies, you are free to run it yourself to cross-reference any results you come up with against mine. I used a 1.0GB generic Wikipedia download for this test, which makes it text-specific. Now, let's dive into which algorithms I chose & what they do. ZIP (Deflate)</h3> Most modern variants of the tried & true ZIP use the Deflate compression algorithm, which is also famously used in PNG. Deflate acts as a combination of LZ77 lossless coding & Huffman coding, where it can first use LZ77 to find patterns in the data & reduce redundancy. This is followed by using Huffman coding to assign smaller bit values to patterns found more frequently by LZ77. Considering ZIP came out in 1989, I don't expect it to be the strongest performer on this benchmark. I'm interested to see how 7zip's supposedly improved ZIP encoder fares against the other standards, though; as the 7zip website's homepage claims: "For ZIP and GZIP formats, 7-Zip provides a compression ratio that is 2-10 % better than the ratio provided by PKZip and WinZip." ZIP is noteworthy for its nearly universal compatibility. "Traditional ZIP" (compression method 8) is limited to 4 GB, though most ZIP compressors use Deflate64(tm) (compression level 9 in the ZIP specification) to bypass this limitation. 7zip</h3> Using the static 7zip binary for Linux, I benchmarked 7zip not initially very interested in its compression capabilities. However, as I continued to use it, I was impressed by its usability & sane defaults; 7zip compression & decompression both thread rather effectively, which makes the format feel very fast to work with even if system & user time tell a slightly different story. It is supported just fine on my Arch Linux installation with GNOME, and it works on macOS by default. To open a 7zip (.7z) archive on Windows 10, you need the well-known 7-Zip utility</a>. 7zip is mostly based on LZMA & LZMA2, though there is a lot more going on</a> if you look deeper. LZMA2 is an improved version of LZMA, which itself is based on LZ77. XZ</h3> XZ can only compress one file at a time, so making a tar archive of the files you'd like to compress (if there are multiple) is necessary when using XZ. The XZ format itself is an improvement on LZMA, allowing for preprocessing filters similar to 7zip to increase the resulting archive's compression ratio. I've been able to decompress .tar.xz archives on macOS & Linux just fine, but Windows 10 needs 7-Zip once again. Brotli</h3> Brotli was designed by Google, & is commonly used as a compression format on the Web. It was released in late 2013, & it is commonly used on the Web for content delivery. It is a core part of the .woff2 Web Open Font Format, allowing web fonts to be smaller when sent to users as part of a website. It is not very common to pass around .tar.br files, so it is perfectly acceptable that such files aren't really compatible anywhere. Brotli is almost universally compatible across the Web, being supported by as much as 96% of the world wide web's users. Brotli is based on LZ77 & Huffman coding, much like ZIP. It also uses context modeling to allow the use of multiple Huffman trees for the same alphabet in the same block; without getting into the weeds, this essentially means that based on the context of the data being compressed, it can be compressed more efficiently especially if it contains multiple different kinds of data. Brotli was co-authored & partially developed by Jyrki Alakuijala, who also worked on JPEG XL & jpegli. JPEG XL's metadata information can be uncompressed or Brotli-compressed. Zstandard</h3> Zstandard is a compression algorithm by Facebook known for its extremely fast decompression speeds. It was released in early 2015 and is used in a variety of different contexts. It was designed to perform similarly to older Deflate-based compression algorithms like ZIP or gzip while being overall faster. In practice, it is said to compress similarly to pure LZMA while being much faster. While .tar.zst archives aren't going to be very popular to find on the Internet & elsewhere, it is already a very popular tool for compression in the world of open-source software. It has been integrated into both the FreeBSD kernel & the Linux kernel and is available as a filesystem compression method for the btrfs, squashfs, bcachefs, & OpenZFS filesystems. All Arch Linux packages are compressed at zstd level 20, allowing Arch packages to be decompressed 14 times faster than when Arch used XZ at the cost of an average 0.8% filesize increase. It is popular in the game emulation scene as well, as many game file formats for emulating console games support zstd compression. The ZIP file format actually supports Zstandard in compression level 93 since version 6.3.8 which was published in 2020. Content encoding using zstd is supported since Chromium 118 behind an experimental flag, meaning it might compete with Brotli on the web in the future. Apple's LZFSE algorithm is purportedly similar to Zstandard compression level 6. ZPAQ</h3> I know much less about ZPAQ, but from what I can glean it uses a multitude of different compression algorithms to try to achieve the best size-to-compression-time ratio possible while producing the smallest possible archives without much concern given to decompression performance. On the official ZPAQ website</a>, it looks like it is designed for "realistic backups that have a lot of duplicate files and a lot of already compressed files." What I find very cool about ZPAQ is that it is an incremental journaling archiver, meaning you can add files to an existing archive based on if they were changed or not which reduces the time needed to wait for a new backup to finish. If other tools here are capable of this, I have not seen it advertised, but this is particularly cool for ZPAQ since it is so focused on compression ratio & in practice, this kind of feature may reduce the burden imposed by long compression times. ZPAQ archives aren't handled by default on my Linux installation in Nautilus. Results</h1> Compression</h3> Here are my compression performance results, measured in real time. This graph uses a logarithmic horizontal scale factor: My personal takeaways from this test: This is very much not fair to Brotli, as the reference encoder can only use 1 thread at a time, disadvantaging it in real clock-on-the-wall time comparisons like this one. Wait for the next test</li> Zstandard is indeed competitive with xz & 7zip, although it isn't quite as good. Higher effort ZPAQ pulls far ahead of everything</li> ZIP is, predictably, behind the pack by a good amount. Can't say I'm surprised :P</li> XZ extreme vs. normal doesn't seem to have a meaningful effect on</li> Low-effort Zstandard is really fast, and competes with ZIP despite being much faster. Mission accomplished?</li> </ul> Here are my compression performance results, measured in user + system time (the time the computer uses across its resources, added up). This graph uses a logarithmic horizontal scale factor: My personal takeaways from this test: Not sure why I omitted ZPAQ from this test. Either way, we know from the previous test it probably would have looked the same considering it is multithreaded</li> Now, Brotli looks much more competitive. Not sure if allowing the other encoders to use multiple threads hampers their performance at all, though I don't know why it would</li> This makes zstd look much more competitive with XZ & 7zip</li> ZIP still lags behind. Using this older Deflate algorithm isn't going to save you any compute resources compared to modern options like Brotli & zstd</li> </ul> Decompression</h3> Evaluating any of these compression standards for practical use means you must take decompression into account. Let's look at the first batch of results for real time, where I tested decompression performance for the lowest & highest levels of compression effort for each standard. Once again, this graph uses a logarithmic horizontal scale factor: My personal takeaways from this test: Yikes, not looking good for ZPAQ in this test. It definitely showed its strengths before as an archival format, but as anything else, it doesn't look viable</li> The gap between XZ & 7zip is pretty astonishing. I'd like to see in the next test if this has to do with 7zip taking advantage of multiple threads more effectively, but either way, 7zip clearly has the better user experience</li> I'm once again left wondering how much better Brotli will perform when given the opportunity to level the playing field with the other standards</li> ZIP still isn't good</li> Considering decompression performance is one of Zstandard's highlights, 7zip makes a strong case for being a better choice if Zstandard can't compete in the next test</li> That third pink X next to the one closest to the bottom for XZ is decompression performance for the "extreme" version of the highest compression level. Not much difference, again</li> </ul> Finally, here are the user + sys time results for decompression. This should tell us a more complete story about why 7zip is so fast in real time. This graph uses a logarithmic horizontal scale factor: My personal takeaways from this test: ZPAQ is still not great</li> The gap has closed a lot between 7zip & XZ, showing that under the hood, 7zip is really just a more advanced XZ with more thoughtfully designed utilities optimized for desktop use where a decoder can comfortably make use of multi-threading on powerful systems</li> ZIP isn't that bad compared to 7zip & XZ! This is the first real win ZIP pulls out, in my opinion, considering weaker systems may see performance more similar to this test for real time decompression than my previous test</li> Brotli is good, but zstd is clearly in a class by itself. Now it makes a lot more sense why decompression performance was emphasized so much, and why zstd is the optimal choice for filesystem compression</li> </ul> Conclusion</h1> My conclusion overall is that there really are different scenarios where each format shines. I'll explain for each format: Firstly, ZPAQ is going to be my choice for computer backups if I ever do them (I often don't). The incremental option alongside the incredible compression ratio makes it the best option for this use case, in my opinion, given you'll be willing to allow it to take its sweet time as it decompresses. For a backup format, I don't think that's the biggest deal in the world anyway. 7zip seems like the best option for general desktop use, and if you're distributing files, providing .7z</code> archives might not be a terrible idea if you're looking to phase out ZIP to save server space & bandwidth while maintaining compatibility. The last holdout for support is Android, as Windows 11 can now decompress 7zip archives. 7zip's widespread support as well as its good performance earn it this position. Did you know even iOS can preview the contents of 7zip archives? XZ is a bit of a black sheep here in a world where 7zip exists. 7zip & XZ are both open source, too. Tell me if XZ has some hidden strengths I'm missing. Brotli as a replacement for simpler algorithms on the Web for content delivery makes complete sense. It decompresses fast & compresses well, and can be parallelized using other implementations to optimize real time encode performance. In the meantime, servers are more concerned with user & system time, so those graphs are likely more salient when considering Brotli performance where it is applicable. Zstandard is what I'd consider to be the future of all of the above categories. In certain scenarios, if it takes off as a content delivery format, I could see it replacing Brotli if the benefits of super-fast & super-light decode improve the responsiveness of web pages & are worth sacrificing a bit of compression ratio. When using the much higher effort settings, it actually beats Brotli on all accounts. I'd be happy to pass around .tar.zst</code> archives in the future to replace 7zip & ZIP, making speedy decode a reality on systems of all different measures of compute performance. I'm glad Zstandard is making its way as a great format to use in the FOSS community for innovative new implementations of compression like filesystem compression & accessible package compression that doesn't tax your CPU when decoding, but I'd personally like to see its use cases expand in the future. ZIP's Deflate implementation is very clearly useful for one thing: compatibility. I'm glad the Deflate patent has expired, otherwise I'd be much more disappointed in ZIP's standardization as the default compression archive format in most situations. Personally, for the time being, I'm just glad we didn't unintentionally standardize around uncompressed archives, as I'd be willing to bet many don't know ZIPs are actually compressed. What do you think of my results? Is there anything I missed or could have done better? Should I include bzip2 or some other algorithm next time? Let me know. Thank you for reading! XYB JPEG 2023-07-16T00:00:00+00:00 A brand new JPEG implementation from Google – called jpegli – supports the use of a perceptual colorspace to improve compression efficiency. How does it fare against other codecs, like WebP and AVIF? XYB JPEG</h1> This is going to be a shorter post because I have done this kind of visual quality evaluation three times now, so if you're not sure what you're reading, please look at my Image Codec Comparison</a> post from March & my Mini Image Codec Comparison</a> from just last month. This may be the last one of these kinds of posts for a while unless something interesting can be wrung out of another SSIMULACRA2 plot. Maybe with comparing speeds next time? The reason I'm making this post is that I just discovered that the ssimulacra2</code> binary that comes with libjxl can properly decode & benchmark XYB JPEG encoded with cjpegli</code> & I wanted to test it for myself. The source images & results are published here, but first I think it is worth explaining in greater detail what the XYB color space is. Perceptual Color</h2> A perceptual color space is a color space designed to work around certain assumptions about the human visual system. One of these assumptions is trichromacy, which implies we can perceptually understand three independent color channels; the response from each channel can be attenuated or bolstered by the stimulus it receives, which allows us to see a myriad of different colors due to different wavelengths of light in the visual spectrum interacting at different intensities. We assume trichromacy in humans due to the presence of three types of cones in our eyes: one that receives green light, another that receives red, & a final cone that can perceive blue light. These are the three additive primaries in our trichromatic visual system. Our cones & their respective responses are illustrated in the graph below. Wikimedia Commons. CC BY 3.0 If you were to place two coordinates in a perfect perceptual color space, the distance between the coordinates should map perfectly to our perception of their color difference. The problem is color perception is non-Euclidian, & this is an impossible problem to completely solve. W. David Wright & John Guild were up to the challenge, however, & built the CIE XYZ colorspace to map more perfectly to our human visual system than prior attempts. Instead of a red, green, & blue channel, XYZ specifies a luminance channel Y (like the commonly used YCbCr) & chromaticity channels Z ("quasi-equal" to blue) & X (another chromaticity coefficient based on psychovisual testing by Wright & Guild; "a mix of the three CIE RGB curves chosen to be nonnegative"). If you want to try to visualize it, here's an example of our visual gamut under D65 illumination visualized within the XYZ color space: xyz_visual</a> Wikimedia Commons. CC BY-SA 4.0 XYB Color</h3> Now, that's the XYZ color space. XYB</a> is an LMS-based color space. LMS stands for long, medium, and short, representing the response our cones have to different visible spectra, & is newer than XYZ. The same general principle of modeling colors based on our cone response still applies to both, though, & I think talking about XYZ is the best way to explain perceptual color on a basic level. The labels on the cones_response SVG above show our L, M, & S cones. A 3x3 matrix operation can be applied to LMS to produce XYB, & you will find that XYB justifies allocating fewer bits to the S channel (blue) due to the lower density of blue cones in our eyes</a>. This allows perceptual quantization of color values that map better to our visual system, spending bits where we can see them instead of where they aren't salient to us. JPEG XL already uses XYB internally. If you're interested in how jpegli works, see my Mini Image Codec Comparison</a>. Methodology</h2> Here are the encoders I used for this test: cjpegli from the libjxl repos, in 4:4:4 RGB, 4:4:4 XYB, 4:2:2 XYB, & 4:2:0 XYB modes</li> mozjpeg mozjpeg version 4.1.3 (build 20230612)</code></li> cwebp cwebp 1.3.1 | libsharpyuv: 0.2.1</code></li> cjxl cjxl v0.9.0 2a6f1f2c [AVX2]</code></li> avifenc via aom-av1-lavish, latest git (Endless_Merging branch)</li> </ul> And here are their parameters: cjpegli input -q [quality] N/A/--xyb --chroma_subsampling=[444/422/420] output.jpg</code></li> cjpeg -q [quality] input > [output.jpg]</code></li> cwebp -m 6 -q [quality] input -o output.webp</code></li> cjxl input output.jxl -q [quality] -e 6 -p</code></li> avifenc -c aom -s 8 -j 12 -d 10 -y 444 --min 1 --max 63 -a end-usage=q -a cq-level=[quality] -a tune=ssim [input] [output.avif]</code></li> </ul> I'm also using the standard ssimulacra2</code> binary instead of ssimulacra2_rs</code> because it seems to be much faster & can decode both JPEG & JXL natively. Faster AVIF & JXL encoding</h3> Last blog post, Jon Sneyers suggested I use more realistic encoding speeds for cjxl</code> & avifenc</code> as opposed to slower, more thorough speeds to test what people might realistically end up using outside hobbyist work where codec nerds are eager to wait an excruciatingly long time for encoding to finish. Besides, I already have plenty of material published regarding high-effort cjxl</code> & avifenc</code>, so let's see what they can both do with a little speed this time around. One thing to note is that I'm using 12 threads to encode AVIF despite having a 24-thread system. It seems that allowing avifenc to use all available threads harms encoding performance a bit on my machine, as was the case in my previous blog post. At speed 8, 12 threads seemed to outperform 8, so I decided to specify -j 12</code> to give AVIF encoding its best shot for speed. I decided on JXL effort 6 & AVIF speed 8 based on the following speed test results on my laptop with a different image not present in this corpus. While AVIF is still slower, they both seem to line up well enough here. avifenc</code> is better for overall user time & much better for system time in this configuration, but cjxl</code> wins when it comes down to how much real time it takes to perform the benchmark which correlates to how fast it actually is between hitting the figurative "start" & "stop" buttons. Because this is likely due to cjxl</code> utilizing system resources better, I won't fault it for using more user & system time versus avifenc. Log scale: Here are some speed benchmarks for each, using the parameters I specified earlier. Parameter modifications are disclosed. Benchmarked with hyperfine</code>, tested on the first image in the corpus (test1.png). hyperfine --warmup 5 --runs 20 "command"</code> zsh inxi -v CPU: 16-core (8-mt/8-st) 13th Gen Intel Core i7-13700K (-MST AMCP-) speed/min/max: 1512/800/5400 MHz Kernel: 6.4.3-zen1-1-zen x86_64 Up: 5h 5m Mem: 6.95/31.12 GiB (22.3%) Storage: 6.83 TiB (2.6% used) Procs: 523 Shell: Zsh inxi: 3.3.28</code> avifenc speed 8 j12 182.4 kB % hyperfine --warmup 5 --runs 20 "avifenc -c aom -s 8 -j 12 -d 10 -y 444 --min 1 --max 63 -a end-usage=q -a cq-level=21 -a tune=ssim test1.png out.avif"</code> Benchmark 1: avifenc -c aom -s 8 -j 12 -d 10 -y 444 --min 1 --max 63 -a end-usage=q -a cq-level=21 -a tune=ssim test1.png out.avif Time (mean ± σ): 249.9 ms ± 6.2 ms [User: 397.2 ms, System: 19.5 ms] Range (min … max): 243.4 ms … 271.7 ms 20 runs </blockquote> cjxl e6 180.9 kB % hyperfine --warmup 5 --runs 20 "cjxl test1.png out.jxl -d 1.0 -e 6 -p -v"</code> Benchmark 1: cjxl test1.png out.jxl -d 1.0 -e 6 -p -v Time (mean ± σ): 192.1 ms ± 2.3 ms [User: 641.1 ms, System: 666.6 ms] Range (min … max): 188.7 ms … 199.7 ms 20 runs </blockquote> cwebp m6 177.2 kB % hyperfine --warmup 5 --runs 20 "cwebp -m 6 -q 90 test1.png -o out.webp"</code> Benchmark 1: cwebp -m 6 -q 90 test1.png -o out.webp Time (mean ± σ): 342.9 ms ± 1.0 ms [User: 337.4 ms, System: 4.5 ms] Range (min … max): 341.0 ms … 344.9 ms 20 runs </blockquote> cjpeg (mozjpeg) 184.1 kB % hyperfine --warmup 5 --runs 20 "cjpeg -q 80 test1.png > out.jpeg"</code> Benchmark 1: cjpeg -q 80 test1.png > out.jpeg Time (mean ± σ): 196.7 ms ± 1.2 ms [User: 191.2 ms, System: 5.1 ms] Range (min … max): 195.3 ms … 199.7 ms 20 runs </blockquote> cjpegli 4:4:4 rgb 184.8 kB % hyperfine --warmup 5 --runs 20 "cjpegli test1.png -d 1.4 --chroma_subsampling=444 out_444.jpeg"</code> Benchmark 1: cjpegli test1.png -d 1.4 --chroma_subsampling=444 out_444.jpeg Time (mean ± σ): 139.8 ms ± 0.8 ms [User: 132.0 ms, System: 7.5 ms] Range (min … max): 138.3 ms … 141.5 ms 20 runs </blockquote> cjpegli 4:4:4 xyb 179.5 kB % hyperfine --warmup 5 --runs 20 "cjpegli test1.png --xyb -d 1.4 --chroma_subsampling=444 out_444-xyb.jpeg"</code> Benchmark 1: cjpegli test1.png --xyb -d 1.4 --chroma_subsampling=444 out_444-xyb.jpeg Time (mean ± σ): 156.7 ms ± 0.8 ms [User: 148.7 ms, System: 7.6 ms] Range (min … max): 155.4 ms … 158.3 ms 20 runs </blockquote> cjpegli 4:2:2 xyb 178.9 kB % hyperfine --warmup 5 --runs 20 "cjpegli test1.png --xyb -d 0.9 --chroma_subsampling=422 out_422-xyb.jpeg"</code> Benchmark 1: cjpegli test1.png --xyb -d 0.9 --chroma_subsampling=422 out_422-xyb.jpeg Time (mean ± σ): 149.8 ms ± 0.8 ms [User: 141.9 ms, System: 7.5 ms] Range (min … max): 148.4 ms … 151.8 ms 20 runs </blockquote> cjpegli 4:2:0 xyb 186.4 kB % hyperfine --warmup 5 --runs 20 "cjpegli test1.png --xyb -d 0.5 --chroma_subsampling=420 out_420-xyb.jpeg"</code> Benchmark 1: cjpegli test1.png --xyb -d 0.5 --chroma_subsampling=420 out_420-xyb.jpeg Time (mean ± σ): 147.8 ms ± 0.7 ms [User: 139.8 ms, System: 7.6 ms] Range (min … max): 146.5 ms … 149.0 ms 20 runs </blockquote> Photographic Images</h2> Here are some lossy previews of the four images that were tested. These will load properly on a browser supporting JXL, and on other browsers will fall back to XYB JPEG. Here are the test results for each image! This was an incredibly shocking first showing for XYB JPEG, outperforming other JPEGs, WebP, & even AVIF at the critical high-fidelity range. There's some weirdness going on with mixing chroma subsampling & XYB color for some reason, with an incredibly low quality ceiling that is hit very quickly. For 4:4:4, this is impressive to see XYB JPEG come second only to JXL. Here, once again, XYB JPEG impresses by surpassing AVIF entirely at medium & higher fidelity while putting the other JPEG encoders to shame. I am floored by how well this performs, although JXL remains a better option overall. This is more what I'd expect out of AVIF, as it seems to be suffering in these other tests due to the faster preset we're using. I'm still impressed with XYB JPEG's ability to surpass WebP at 75+ SSIMU2. Pretty similar to before, XYB JPEG handily beats WebP while still being outperformed by AVIF. I'm still wowed by the fact that the ancient JPEG is still even reasonably competitive with AVIF, but seeing how much better JXL does throughout these tests, it is clear the modern formats still have hope. Extrapolations</h2> On average, seeing such a good showing from XYB JPEG has inspired me to start using the format more often. Anywhere ICC profiles are supported along with JPEG, XYB JPEG will be supported. I'm interested to see how it'd perform on a bigger dataset against higher effort JXL & AVIF, but for now, I'm satisfied with its performance enough to argue for its adoption. The one snag is ICC color management support, although I've yet to run into a modern browser that doesn't pass these tests</a>. You can try it for yourself if you like, I'm not sure which browsers don't support ICCv2 color management, but those would not deal with XYB JPEG properly. Inspiration & Sources: en.wikipedia.org/wiki/CIE_1931_color_space#Definition_of_the_CIE_XYZ_color_space</a> raphlinus.github.io/color/2021/01/18/oklab-critique.html</a> https://en.wikipedia.org/wiki/LMS_color_space#Image_processing</a> observablehq.com/@mattdesl/perceptually-smooth-multi-color-linear-gradients</a> Thanks for reading! Sponsor Me on GitHub Sponsors</h3> Help support my open source efforts - a little goes a long way! Sponsor</a> Better JPEG Encoding From Google 2023-06-14T00:00:00+00:00 JPEG is old, but a new perceptually optimized encoder just became available from Google called jpegli. Like the mozjpeg encoder aimed to outperform older JPEG implementations, jpegli aims to push the format to its limits in terms of compression efficiency. Why?</h2> JPEG is a really neat codec. It has served us super well for a number of years. JPEG's ingeniously simple design that divides an image into 8x8 pixel blocks, separates chroma (color) from luma (luminance) into the YCbCr color space (you can also use RGB), applies the DCT, quantizes the DCT coefficients, & passes the data on to the decoder makes JPEG images very easy to read and write nowadays. They look good, too; in fact, they are looking better and better as their encoding implementations (somehow) continue to improve; MozJPEG improved quality relative to libjpeg-turbo & older implementations, especially at low to medium fidelity; jpegli promises to further improve over MozJPEG, potentially across the board. JPEG also still has features that many modern codecs don't have that are very useful, like support for progressive decode which allows parts of the image to be sent as the image data is being transferred. Blurhashes</a> are very popular for their ability to send some data before the entire image, and progressive decode enhances this even further. WebP, HEIC, & AVIF don't support progressive decode*, although JPEG XL does. Outside of features, JPEG supposedly became competitive with WebP in coding efficiency when MozJPEG released. This is contrary to the narrative that WebP is 25%-34% smaller</a> than JPEG (which, if you read the article, isn't the conclusion here; Google specifically says "We observed that the average WebP file size is 25%-34% smaller compared to JPEG file size at equivalent SSIM index" which is important because SSIM is a rather outdated metric at this point & largely isn't indicative of perceptual quality. They also tested libjpeg, not mozjpeg). Could jpegli be so efficient that it results in WebP being entirely outcompeted by a codec that's nearly twenty years older? How Jpegli Works</h2> Jpegli's gains are largely due to better adaptive quantization based on the heuristics used by JPEG XL. Projects like Guetzli</a> have achieved better JPEG compression through similar means, but have been really slow to work with. It is outlined in the readme for Guetzli that it is not fast: Guetzli uses a significant amount of CPU time. You should count on using about 1 minute of CPU per 1 MPix of input image. </blockquote> Another smart jpegli trick is usage of the XYB colorspace, which is perceptually more in line with the human visual system compared to RGB. We haven't tested that today because SSIMULACRA2.1 doesn't understand what it is looking at when given XYB JPEGs (even properly transcoded to PNG, there's some sort of harmful color shift), but JPEG XL uses the XYB color space by default for lossy image coding and the results are clearly incredible for coding efficiency. The way jpegli handles XYB color in a JPEG image is by applying an ICC color profile that maps the existing JPEG color channels to XYB. This actually has the potential to increase the bit depth of the image, which could allow 10 bit JPEGs in the future. I'm excited to see XYB JPEG continue to improve via jpegli, but for now we're just going going to use libjxl's included cjpegli</code> binary to test some photographic images. Methodology (Photographic)</h2> Here are the encoders I used for this test: cjpegli from the libjxl repos, in 4:4:4, 4:2:2, & 4:2:0 chroma subsampling modes</li> mozjpeg mozjpeg version 4.1.1 (build 20230217)</code></li> cwebp cwebp 1.3.0 | libsharpyuv: 0.2.0</code></li> cjxl via cjxl v0.9.0 e2fe7bad [AVX2]</code></li> avifenc via aom-av1-lavish, latest git (Endless_Merging branch)</li> </ul> And here are their parameters: cjpegli input -q [quality] --chroma_subsampling=[444/422/420] output.jpg</code></li> cjpeg -q [quality] input > [output.jpg]</code></li> cwebp -m 6 -q [quality] input -o output.webp</code></li> cjxl input output.jxl -q [quality] -e 8</code></li> avifenc -c aom -s 4 -j 8 -d 10 -y 444 --min 1 --max 63 -a end-usage=q -a cq-level=[quality] -a tune=ssim [input] [output.avif]</code></li> </ul> Here are some speed benchmarks for each, using the parameters above. Parameter modifications are disclosed. Benchmarked with hyperfine</code>, tested on the first image in the corpus: hyperfine --warmup 2 --runs 20 "command"</code> inxi -v CPU: 16-core (8-mt/8-st) 13th Gen Intel Core i7-13700K (-MST AMCP-) speed/min/max: 1386/800/5400 MHz Kernel: 6.3.7-zen1-1-zen x86_64 Up: 29m Mem: 5633.8/31868.7 MiB (17.7%) Storage: 6.83 TiB (20.4% used) Procs: 520 Shell: Zsh inxi: 3.3.27</code> cjpegli -q 90, --chroma_subsampling=444: 156.1 kB Time (mean ± σ): 80.1 ms ± 0.7 ms [User: 75.7 ms, System: 4.2 ms] Range (min … max): 79.1 ms … 81.4 ms 20 runs </blockquote> </li> cjpegli -q 90, --chroma_subsampling=422: 145.4 kB Time (mean ± σ): 74.3 ms ± 0.7 ms [User: 70.2 ms, System: 3.9 ms] Range (min … max): 73.2 ms … 76.0 ms 20 runs </blockquote> </li> cjpegli -q 90, --chroma_subsampling=420: 132.2 kB Time (mean ± σ): 70.6 ms ± 0.5 ms [User: 66.3 ms, System: 4.1 ms] Range (min … max): 69.8 ms … 71.5 ms 20 runs </blockquote> </li> mozjpeg -q 86: 154.8 kB Time (mean ± σ): 105.0 ms ± 0.9 ms [User: 101.6 ms, System: 3.0 ms] Range (min … max): 103.5 ms … 107.6 ms 20 runs </blockquote> </li> cwebp -q 90: 130.7 kB Time (mean ± σ): 194.4 ms ± 1.4 ms [User: 190.5 ms, System: 3.6 ms] Range (min … max): 191.8 ms … 197.1 ms 20 runs </blockquote> </li> cjxl -q 92: 134.0 kB Time (mean ± σ): 768.1 ms ± 7.0 ms [User: 1068.8 ms, System: 398.4 ms] Range (min … max): 758.6 ms … 784.1 ms 20 runs </blockquote> </li> avifenc cq-level=11: 155.5 kB Time (mean ± σ): 1.500 s ± 0.008 s [User: 3.670 s, System: 0.016 s] Range (min … max): 1.493 s … 1.529 s 20 runs </blockquote> </li> avifenc cq-level=11 -j all: 155.5 kB Time (mean ± σ): 1.571 s ± 0.132 s [User: 3.778 s, System: 0.019 s] Range (min … max): 1.495 s … 1.975 s 20 runs </blockquote> </li> </ul> Using all available threads with avifenc would routinely produce slightly worse results, and Hyperfine threw up an error every time suggesting some results were appearing to be outliers. This is why I lowered the worker count to 8 for the quality benchmark. I found this anomalous behavior interesting as it highlights avifenc's trouble with scaling that stems from issues with the AVIF image format. I would wager a guess that this scaling issue is because of AVIF's use of certain intra-frame coding techniques like directional prediction that share data between blocks, theoretically improving coding efficiency but reducing parallelization & worsening generation loss. If I recall correctly, JPEG XL specifically avoided intra coding techniques like these in order to parallelize effectively (and improve resilience to generation loss). Photographic Images</h2> Here are some lossy previews of the four images that were tested. These will load properly on a browser supporting JXL, and on other browsers will be transcoded to PNG. And, here are the test results for each image! It is worth disclosing that in my analyses, I prefer to look at the 60-70+ range that I consider most useful. Others may disagree, so please take my rankings for what they are: my opinions. The first image favors jpegli quite a bit here over mozjpeg. The medium-high fidelity range from 60-80 is kind of a tie between all three chroma subsampling modes & WebP, while mozjpeg lags behind. It is worth noting that WebP's overall quality ceiling is lower here, with a max score of around 90. My picks: JXL</li> AVIF</li> jpegli 4:2:0</li> WebP</li> jpegli 4:4:4</li> jpegli 4:2:2</li> mozjpeg</li> </ol> The simple lines and large sections of color in this image remind me of a nonphotographic image, and therefore favor AVIF & WebP which are derived from video codecs (AV1 & VP8 respectively). Although it looks like 4:4:4 jpegli does well at scores >95, that medium-high fidelity range is dominated by mozjpeg. WebP's quality ceiling is really low here, and it appears incapable of hitting a score of 90 which is where SSIMULACRA2.1 starts to consider an image perceptually lossless. My picks: AVIF</li> JXL</li> WebP</li> mozjpeg</li> jpegli 4:4:4</li> jpegli 4:2:0</li> jpegli 4:2:2</li> </ol> This one looks like a toss-up the whole time between jpegli, mozjpeg, and webp, with AVIF pulling ahead & JXL pulling even further ahead. Jpegli 4:4:4 is consistent all the way across, while mozjpeg starts to drop off at higher fidelity & WebP sadly hits its quality ceiling just before cresting the 90 mark while still maintaining strong coding efficiency throughout the rest of the quality scale where it is present. My picks: JXL</li> AVIF</li> jpegli 4:4:4</li> WebP</li> mozjpeg</li> jpegli 4:2:0</li> jpegli 4:2:2</li> </ol> This is the closest of all. I am once again disappointed that WebP is outperformed by jpegli 4:4:4 at around 83+ & can't seem to score higher than a 90, while jpegli 4:4:4 & mozjpeg cruise up to around 95. That extra headroom is important for photography & lots of use cases outside of web delivery, which is why it is important to me that a codec is able to perform throughout the quality spectrum without bumping its head too early. Here we see AVIF & JXL behaving similarly to my Image Codec Comparison</a> blog post, with strong low fidelity performance from AVIF & better high fidelity results coming out of JXL. My picks: JXL</li> AVIF</li> jpegli 4:4:4</li> WebP</li> mozjpeg</li> jpegli 4:2:0</li> jpegli 4:2:2</li> </ol> Extrapolations</h2> First, if you don't know what chroma subsampling is, I highly recommend you check out Master of Zen's explainer</a>. It is a more detailed explanation, & it thoroughly covers the effects of chroma subsampling. It is worth noting that I didn't try every mozjpeg tune here, and I have seen the SSIM & MS-SSIM tunes work well compressing photographic images. I also didn't test every chroma subsampling mode that mozjpeg has to offer, and allowed mozjpeg to subsample chroma automatically - this automatic selection gives it a theoretical advantage over each individual jpegli chroma subsampling mode, but wouldn't allow the encoder to display its breadth of competence like jpegli here. In the end, I wouldn't call this a loss for mozjpeg anyhow - it is clearly not obsolete in any case, and the only scenario in which it clearly lost was the first image test. The overall average & three image average results make a compelling case for mozjpeg overall. While four images isn't anything significant to extrapolate any real conclusions about these encoders, we can see some clear differences between the different encoders. In particular, I think the notion that WebP is noticeably more efficient than JPEG is almost entirely negated by modern JPEG encoders like mozjpeg & now jpegli. WebP is especially weak outside of the Web, where higher fidelity is more precious. Stepping back from my personal preference for high fidelity, it is very impressive how consistently well jpegli 4:2:0 is able to do when the target is <50-60. This is usually WebP's territory (ignoring AVIF & JXL for a moment) relative to JPEG, but 4:2:0 jpegli is closer than mozjpeg to WebP on every image except the second one. 4:2:2 jpegli seems like it doesn't have much of a use at all. Should I Use It?</h2> I would say yes, given you: Cannot use AVIF or JXL.</li> Can't use WebP, or if you can, it compresses worse.</li> Prefer progressive decode & can't use JXL</li> Have A/B tested against mozjpeg at the very least, & I'd recommend testing against other cjpegli chroma subsampling modes as well.</li> Like the results more with your eyes, which are more important than metrics.</li> </ul> I use metrics because it'd be really difficult to articulate my visual preferences here and extrapolate objective data based on them. On your own, try to use your eyes as much as possible, and train yourself to look for artifacts & recognize what they are; mosquito noise, ringing, blocking, and banding are all terms you should be familiar with. Now that Apple announced support for JXL</a>, hopefully the industry tide on the Web will shift toward the new image codec & we'll see such incredible market penetration that we don't need to use JPEG anymore. For now, this isn't a reality, and JPEG's compatibility remains unmatched; given the circumstances, I'd recommend that you of course use JXL when possible. When you must use a JPEG, it wouldn't be unwise to leverage the plethora of existing JPEG encoding tools to your advantage when you can. Thanks for reading! *AVIF kinda supports progressive decode if you ship an animated AVIF with a single super lossy frame that loads before the actual image. There are other tricks you can use as well, and more information about this is outlined in this Autocompressor blog post</a>. Sponsor Me on GitHub Sponsors</h3> Help support my open source efforts - a little goes a long way! Sponsor</a> Who Has the Best Hardware AV1 Encoder? 2023-04-15T00:00:00+00:00 This article compares Nvidia's NVENC encoder to Intel's QSV encoder for AV1, as well as for other codecs. Who will win? Please be aware that this page contains JPEG XL images. In unsupported browsers, images may not load properly. Why?</h2> Nobody really knows which hardware accelerated AV1 encoder is better right now. There are other resources available, but they don't use good metrics. Everyone's favorite bad, obsolete, easily tricked metric is VMAF, which was good for a time but simply isn't good anymore. You can check out this Twitter thread</a> to see what I'm talking about; these aren't edge cases either. You can produce a "better quality" video according to VMAF using Contrast Adaptive Sharpening filters which inherently do not improve video fidelity in any way. Other metrics are PSNR, SSIM, MS-SSIM, SSIMULACRA, Butteraugli, & SSIMULACRA2. PSNR doesn't accurately represent our visual system as humans, SSIM was good for its time but is now old and also provides inconsistent/misleading results, & MS-SSIM does better but also doesn't correlate as well with our visual systems as newer metrics. The distance-based Butteraugli metric isn't bad, but SSIMULACRA2 is currently considered the gold standard for a visual quality metric. You can read more about why this metric is great here</a>, but SSIMULACRA2 (ssimu2) is, in my opinion, the only way to really measure subjective visual quality right now. The scores it produces correlate best with my eyes, and many others agree. Here's how SSIMULACRA2 assesses quality: Returns a score in range -inf..100, which correlates to subjective visual quality scores as follows: 30 = low quality. This corresponds to the p10 worst output of mozjpeg -quality 30.</li> 50 = medium quality. This corresponds to the average output of cjxl -q 40 or mozjpeg -quality 40, or the p10 output of cjxl -q 50 or mozjpeg -quality 60.</li> 70 = high quality. This corresponds to the average output of cjxl -q 65 or mozjpeg -quality 70, p10 output of cjxl -q 75 or mozjpeg -quality 80.</li> 90 = very high quality. Likely impossible to distinguish from the original when viewed at 1:1 from a normal viewing distance. This corresponds to the average output of mozjpeg -quality 95 or the p10 output of cjxl -q 95.</li> </ul> </blockquote> In my Image Codec Comparison</a> benchmark, I said "Because of the irrelevance of negative SSIMULACRA2 scores, I stopped the vertical axis on the graph(s) at 0." I still maintain this logic was correct for the image benchmark, but with livestreams, it isn't unusual to see a low bitrate stream due to a poor network connection on either a streamer's end or your own. This test is also more about how the encoders perform at certain target bitrates, as mentioned above, and since the goal isn't necessarily always to target a certain quality like it is with images, I think some of the less relevant data (eg target 500kb/s) is still important to include because of how it frames the more useful data (eg the 2000-6000kb/s range popular with streamers). Anyway, I've seen testing from many using VMAF, and their conclusions are varied relative to which hardware encoder is the best. These tests</a> are more thorough than mine, but aren't accompanied by much explanation or any VAAPI results for Linux. They also use VMAF, so everything must be taken with a grain of salt. I still think they're useful in judging the quality of many hardware encoders that I haven't tested here (like AMD's stuff) so I'll still recommend you go take a look if you're interested. So, what did we find in our own testing? Methodology (Gaming Corpus)</h2> In order to understand who uses hardware encoders and where, look no further than livestreaming. Livestreaming can be done via OBS, and you have a choice to decide between CBR, VBR, & CQP encoding. Many choose to opt for CBR or VBR, since your connection is most likely the bottleneck & you can target a certain bitrate. I'd personally opt for VBR so that the encoder can intelligently allocate more or less bits when necessary while still targeting a specific bitrate on average. This also allows us to get better overall quality per bit, so the viewer can see a higher quality stream while using less data on average. It is more reliable for data usage & bandwidth than CQP which targets quality, & more efficient than CBR which will give every scene the same amount of bits no matter its content. With help from a couple friends online (namely BlacKnight</a> & Maik</a>), I tested hardware & software encoders from target 500kb/s to target 8500kb/s in increments of 500. This gave us 16 data points per encoder per clip to work with. I used two clips from DERF's Test Media - namely the CSGO clip & the Minecraft clip under "Gaming" - and transcoded them to h264 with crf 1</code> to provide the encoders with near mathematically lossless video that they could effectively decode. The loss introduced by reencoding at such high quality is miniscule & should have a nearly undetectable effect on the benchmark results. Here's a screenshot from the CSGO clip: & here's one from the Minecraft clip: All hardware encoders were on their highest effort preset (-compression_level 7) for optimal quality. It didn't have a massive effect on speed in my testing. The encoders tested for the CSGO clip were the following: h264 (UHD770 VAAPI)</li> h264 (Arc A770 VAAPI)</li> HEVC (UHD770 VAAPI)</li> HEVC (Arc A770 VAAPI)</li> HEVC (Arc A770 VAAPI, 10 bit)</li> VP9 (UHD770 VAAPI)</li> x264 (slower preset)</li> SVT-AV1 (preset 6, 10 bit)</li> SVT-AV1 (preset 8)</li> SVT-AV1 (preset 8, 10 bit)</li> Nvenc AV1 (RTX 4090)</li> Nvenc AV1 (RTX 4090, 10 bit)</li> QSV AV1 (Arc A770)</li> QSV AV1 (Arc A770, 10 bit)</li> </ul> The encoders tested for the Minecraft clip were the following: x265 (medium preset, 10 bit)</li> SVT-AV1 (preset 4, 10 bit)</li> SVT-AV1 (preset 6, 10 bit)</li> SVT-AV1 (preset 8, 10 bit)</li> Nvenc AV1 (RTX 4090, 10 bit)</li> QSV AV1 (Arc A770, 10 bit)</li> </ul> You may be wondering why there is such an emphasis on 10 bit encoding despite the 8 bit source. In my testing, even just with my eyes, encoding to 10 bit video with most lossy video codecs (HEVC & AV1 in particular) yields better coding efficiency than encoding to 8 bit. I can't tell you exactly why, but I'm sure others on the AV1 Community Discord</a> can. You may also be wondering about the lack of AMD numbers. I have a Radeon Rx 6600xt & myself and others found it very difficult to properly test AMD's hardware encoders. You can look at the results I linked above that report VMAF numbers - the massive discrepancy in AMD's AV1 performance compared to other AV1 encoders should be enough to let you know how good AMD's results are. Results</h2> Finally, we come to the graphs. It is also worth noting that the default GOP size for the hardware encoders is around 300 frames; leaving it on "auto" in FFmpeg generally yielded slightly better results by a few fractions of a point in ssimu2, so that is what we did. CSGO</h3> Here is a graph for target bitrate (VBR) with the CSGO clip: And a log scale version that we can use to see discrepancies better: This is kind of a lot to unpack, as there are a considerable number of overlapping lines here. I've attached a CSV file here</a> so that you can visualize this data on your own in any way you please, but I think we can come to a couple interesting conclusions given this current visualization. For this clip: There are clear, marked differences in coding efficiency depending on the encoder even if the codec is the same. So, just because your video is h264 doesn't mean that is was encoded as efficiently as every other h264 video. You can see here that the Arc A770's h264 encoder yields better quality per bit than the h264 encoder that's part of the iGPU on my 13700k. The fixed function hardware Intel is deploying for these two implementations is distinct, and Arc's is clearly better. The software x264 encoder that runs directly on the CPU is certainly slower, but you can see how that pays off in spades.</li> SVT-AV1 is in a class by itself. Preset 8 dominates the hardware encoders across the board by a not insignificant margin, & Preset 6 pulls ahead even further. While hardware encoders are very fast and a gem for streaming, software encoders still take home the efficiency crown.</li> NVENC AV1 is more efficient than QSV AV1 on Arc, even if the advantage is slight. Even when comparing 8 bit NVENC to 10 bit QSV, NVENC still manages to pull ahead ever so slightly.</li> </ul> If you came here to rub salt in Arc users' wounds, I wouldn't say the advantage is significant enough to warrant purchasing a 40-series card over an Arc card solely for encoding especially considering the price discrepancy. However, the advantage is still there, and shouldn't be ignored. My takeaway is if you have more money than you know what to do with, don't buy an Arc GPU for encoding if you have a 40-series card from Nvidia already. Spend it on a nice bowl of ramen instead (or a couple nice bowls of ramen, realistically). Minecraft</h3> Let's see what else we can find looking at the Minecraft results: Here's another log scale version: This one is more stripped down, & everything is easier to see. Here are my takeaways: Now that we have x265 in the mix, we can see NVENC beat it across the board & QSV came close. This is pretty impressive, as x265 medium isn't a fast setting by any stretch of the imagination.</li> SVT-AV1 preset 4's performance leaves us with some clues about our final conclusion, seeing as software AV1 encoding (even with SVT-AV1, which doesn't produce as good quality per bit as AOM encoder forks like aom-av1-lavish) continues to pull ahead significantly the more time & effort we allow it to use. It may be possible to stream using SVT-AV1 preset 8 on higher end systems right now, but more likely you'd be using preset 9 or 10, which is where I think SVT-AV1 is probably on par with hardware encoders. As CPUs become more powerful, this will become easier, & it is clear that dedicated streaming hardware may continue to provide value for streamers who can afford it.</li> </ul> Conclusion</h2> Who Won?</h3> Nvidia's hardware acceleration for AV1 video encoding is superior in these two clips. It is a tiny margin, and it is important to judge subjective visual quality for yourself when evaluating tiny discrepancies like this, but SSIMULACRA2 says that Nvidia has the better solution. I don't think that's the most important takeaway here, though, considering our peek at the potential software encoders have. Software vs hardware?</h3> Software encoders are run on the CPU without any fixed function hardware designed around any specific codec. Hardware accelerated video encoding takes place on specialized ASICs that are attached to most consumer GPUs, & encoding with hardware acceleration means the encoder can also make use of a GPU's highly parallel compute to further accelerate certain video encoding functions. Developing these ASICs is expensive, and dedicating valuable silicon real estate on GPUs to such features must prove to be well worth a company's time and money. Nvidia & Intel both seem to agree that AV1 has enough of a future to make it financially sane to put fast dedicated hardware in the hands of every consumer who has one of their latest GPUs, which is pretty cool. The issue regarding efficiency is that this fixed function hardware is designed to be fast above all else, & implementing less important codec features into this hardware can inflate costs without helping efficiency meaningfully. In software, while slower, encoders can pick and choose to enable/disable whatever features they like based on user specified parameters or internal defaults for each preset. This makes the use cases for both solutions very different. When should I use hardware encoders?</h3> As I mentioned earlier, livestreaming is where hardware accelerated video encoding is most popular. Nvidia has turned their NVENC hardware into a differentiating feature because of the popularity of livestreaming, & as sites like Twitch & YouTube roll out AV1 for the masses, livestreamers will appreciate having a powerful hardware implementation for the more efficient codec to stream with given their hardware readily supports it. Hardware encoding can also be great for screen recording, capturing clips in games, and trancoding media on the fly for a media server or something similar. When speed is the priority over coding efficiency, hardware encoders are a fantastic choice. Here are some places where you probably shouldn't use hardware encoders: Transcoding media you legally own for later consumption (unless you are burdened by a time limit or it is too painful to encode at less than real time)</li> Re-encoding home video that needs to take up less space somewhere or needs to be shared easily</li> Producing a final copy of a video project of some sort to be shared with others</li> Illegally releasing pirated media that has been transcoded to achieve a smaller filesize for viewer convenience (I cannot recommend this regardless)</li> </ul> You should never transcode something that has already been transcoded unless you have to - it is best to search for the source. If you have a BluRay collection that you transcoded to h264 ten years ago, don't go re-encoding it to AV1 now - find the source & do it over, else you risk preserving artifacts from the original encode. And do it with a software encoder. If you're exporting a video for a project or something similar, it may be worth encoding to a lossless or near lossless file & reencoding as efficiently as possible later when you can afford the time so you can optimize for quality per bit. SVT-AV1 is great for that sweet spot balance of speed & efficiency for AV1 video. Rav1e, my personal favorite software AV1 encoder, isn't the best, but it is great for high fidelity & should never segfault. aom-av1-lavish</a> is the reigning champ for coding efficiency in a software encoder. Use at speed 4 or lower for excellent results; they will be results you'll have to wait very long for, though. Tools like av1an</a> can help. You can try my SVT-AV1 GUI, Aviator</a>, if you'd like to mess around with SVT-AV1 without using the command line. rAV1ator</a> it its cousin, powered by av1an & rav1e instead of SVT-AV1 & FFmpeg. Please give them a look if you're interested in getting started with AV1. Once you'd like to dive down the rabbit hole, feel free to look into this article</a> for a detailed guide written by a talented encoder. Thanks for reading! Photographic Corpus (lossy) 2023-03-16T00:00:00+00:00 A collection of the images I used for the blog post published on the same day as this one, lossily compressed with JPEG XL. All images are licensed under CC BY-SA 4.0. Image Codec Comparison 2023-03-16T00:00:00+00:00 The current debate surrounding AVIF versus JPEG XL needs some third-party quantitative data to support arguments on either side. I've compiled a dataset of 27 high-resolution photographic images to compare performance across them all at a wide fidelity range. Why?</h2> To start things off, I have been a fan of the AVIF image format since I heard about it, not long after I started learning about the AV1 video codec in 2020. I figured the fact that it was backed by the Alliance for Open Media, was the natural successor to WebP, and was based on a rather powerful video codec that was rapidly growing in popularity was great for users & organizations alike to easily understand what the technology stood for, where it came from, and what benefits it provided over the existing lossy image king, JPEG. Good codecs are open source codecs, so the fact that AVIF was open source was a big deal to me as well. It is also already widely supported</a> at the time of writing. I thought unequivocally that AVIF would be the future of images on the web when I first learned about it. I don't think I was entirely wrong, but I hadn't yet heard of JPEG XL. JPEG XL</a> (JXL for short) is an image codec by the JPEG committee (yes, the JPEG committee). Initially I hadn't thought much of the codec because the compression efficiency was "the same as AVIF," so I kind of wrote it off, as I'm sure many uninformed people have & continue to do. Upon doing more research, my position flipped and I found myself deeply invested in the future of JXL. JXL's fascinating features include the highly efficient variable-blocksize discrete cosine transform (which is superior to JPEG's DCT), usage of the XYB colorspace which more accurately represents the response our cones in our eyeballs have to different wavelengths of light, & a "modular" mode for efficient lossless or near-lossless coding. It is also used while encoding lossy JPEG XL images, but I don't know much about it & I won't pretend to. Currently the reference encoder also supports patch detection, noise synthesis, and will (probably) eventually support spline detection as it is present in the codec specification. JPEG XL also supports lossless JPEG transcoding that reduces the filesize of JPEGs which in my opinion is a game-changing feature. That alone could be a revolutionary codec by itself. Progressive decode is also a standout feature of JPEG XL, and I found this article</a> very interesting regarding that. Some more interesting JXL articles & resources: www.roboleary.net/webdev/2023/03/06/next-web-image-format-not-jpegxl.html</a> tonisagrista.com/blog/2023/jpegxl-vs-avif</a> cloudfour.com/thinks/on-container-queries-responsive-images-and-jpeg-xl</a> motionmill.com/2023/02/google-stopt-jpex-xl-gebruiken</a> ds.jpeg.org/whitepapers/jpeg-xl-whitepaper.pdf</a> cloudinary.com/blog/the-case-for-jpeg-xl</a> Also worth noting is the format's most outspoken pioneer Jon Sneyers</a> (co-author of the JPEG XL spec) as well as other noteworthy developers are extremely active within the community and value the feedback of the people who are passionate about their work. This should not be as unique as it is, but it is incredibly valuable to have one's voice heard so easily & frequently by developers who actually care. I commend Sneyers & the other core JXL devs for this. Anyway, TL;DR: JXL has a lot more to offer than just good compression. But, I still wish to know; how does it stack up against other image formats regarding lossy compression & lossless compression? Methodology (Photographic Corpus)</h2> In order to produce an unbiased test to determine how good JPEG XL is relative to my needs, I took 27 photos (mostly consisting of landscape/architecture photography) around my seaside hometown using my Sony a5000 at 20.1mp, edited them in Darktable</a>, exported them as 16-bit sRGB PNGs (AdobeRGB had some</a> problems</a>), & finally I ran a script written by my friend RootAtCali (check out their cool site</a>) to determine each image format's SSIMULACRA2 scores for each quality level available via the format's lossy coding option. I'll be certain to make my lossless sources available via an edit to this post, & they'll be linked at the top. In the meantime, here's another page</a> with full resolution lossy JXL photos encoded at -d 1.0</code>. The images are under CC-BY-SA 4.0. These tests were run using the SSIMULACRA2</a> visual quality metric via ssimulacra2_rs</a>. This metric is designed to model human vision far better than VMAF, SSIM, PSNR, & other less effective alternatives. Via the readme on Github: Returns a score in range -inf..100, which correlates to subjective visual quality scores as follows: 30 = low quality. This corresponds to the p10 worst output of mozjpeg -quality 30.</li> 50 = medium quality. This corresponds to the average output of cjxl -q 40 or mozjpeg -quality 40, or the p10 output of cjxl -q 50 or mozjpeg -quality 60.</li> 70 = high quality. This corresponds to the average output of cjxl -q 65 or mozjpeg -quality 70, p10 output of cjxl -q 75 or mozjpeg -quality 80.</li> 90 = very high quality. Likely impossible to distinguish from the original when viewed at 1:1 from a normal viewing distance. This corresponds to the average output of mozjpeg -quality 95 or the p10 output of cjxl -q 95.</li> </ul> </blockquote> Because of the irrelevance of negative SSIMULACRA2 scores, I stopped the vertical axis on the graph(s) at 0. For context, here's an overview of the image formats tested: JPEG, the reigning champ. has been used for decades. Supports progressive decode</li> WebP, based on VP8, an open-source video codec.</li> JPEG XL, the designated successor to JPEG. Designed by the JPEG committee.</li> AVIF, based on AV1, a modern open-source video codec.</li> </ul> Here's a helpful chart</a> provided by Cloudinary. Notably absent are JPEG 2000 (J2K) & HEIC. Considering JXL is J2K's successor & J2K never saw widespread adoption, I'm not sure including it helps anybody. HEIC is being excluded because it sucks, & also is notably not royalty free. Here are the encoders tested for each format: JPEG via the mozjpeg encoder</li> JPEG via the mozjpeg encoder, tuned for MS-SSIM, using the 'Tuned for MS-SSIM on Kodak image set' quantization table & arithmetic coding</li> JPEG via the elusive & mysterious jpegli encoder (more on this later)</li> WebP via cwebp</li> JPEG XL via the libjxl reference encoder implementation, cjxl</li> AVIF via aom-av1-lavish, tuned for SSIM</li> </ul> Here's the encoder parameters I used, along with the encoder version (in the same order as above) cjpeg -q [quality] "input" > "output.jpg"</code> | mozjpeg version 4.1.1 (build 20230217) cjpeg -q [quality] -quant-table 2 -tune-ms-ssim -arithmetic "input" > "output.jpg"</code> | mozjpeg version 4.1.1 (build 20230217) benchmark_xl --input=[input] --codec=jpeg:enc-jpegli:rgb:q[quality] --save_compressed --output_dir=[outdir]</code> | cjxl v0.9.0 c4927fbf cwebp -m 6 -q [quality] "input" -o [output.webp]</code> | 1.3.0 libsharpyuv: 0.2.0 cjxl "input" "output.jxl" -q [quality] -e 7 --brotli_effort 11</code> | cjxl v0.9.0 c4927fbf avifenc -c aom -s 6 -j 16 -d 10 -y 444 --min 1 --max 63 -a end-usage=q -a cq-level=[quality] -a tune=ssim "input" "output.avif"</code> | AOMedia Project AV1 Encoder Psy v3.6.0 (default) WebP is at maximum effort & the JPEG encoders were left stock in terms of effort. cjxl is on effort 7 (the internal default, so I didn't really need to specify it, & brotli_effort</code> is only for lossless iirc), & we encode avif at speed 6 because it correlates best with JXL's default speed (& is also rav1e's default speed internally although we're not using rav1e). Aviator</a>, based on SVT-AV1, consciously defaults to speed 6 for AV1 video encoding, as does rav1e-based rAV1ator</a>. Benchmarking with this image dataset gave us a total of just below 18,000 data points before averaging. For the graph, I calculated the arithmetic mean of the bpp & SSIMULACRA2 score for each quality step for each encoder. Here are the results: Now, hold your breath, conclusions come at the end! Methodology (Non-photographic)</h2> This is a shorter test with less going on, but still interesting & worth bringing up. I'm using this</a> image to test non-photographic encoding using the same parameters as above. Here are the results: While this is only one image, I have universally seen AVIF take a rather large lead with every non-photographic image I've tested. Lossless? Animation?</h2> Stay tuned! While both AVIF & JXL (as well as WebP) support animation, I'd like to cover that in a different post. I'd like to learn more about how AVIF & JXL work relative to lossless image coding as well before I tackle a test like that. Conclusion</h2> Where does JPEG XL win?</h3> It is clear to me that JPEG XL is going to be my choice for exporting my amateur photographs. It performs exceptionally well at higher quality, outperforming AVIF & the others by a significant margin. AVIF is stronger as quality decreases, but AVIF is further weakened by the fact that encoding using more threads generally hurts efficiency & encoding an AVIF image is generally slower than encoding a JXL image. As for the other plethora of features JXL has to offer, AVIF can barely compete. AVIF is also hampered by its lower maximum image dimensions compared to competing codecs. Where does AVIF win?</h3> AVIF wins at low to medium (maybe low-medium) quality. This is great for content devilery on the web, as images don't always need to be high fidelity or retain a ton of high-frequency detail to maintain their appeal; something that looks pleasing and resembles the source can be more desirable. Despite the fact that AVIF lacks progressive decode, its superior coding efficiency at lower quality somewhat makes up for this in my opinion. AVIF is also the winner for our non-photographic image, and if you trust that image isn't an edge case, it opens up some cool opportunities for sharing digital art as JXL doesn't have an encoder available with spline detection to compete right now. Being based on the AV1 video codec, AVIF is allegedly easier to implement as well; the codec's widespread adoption sees it already achieving a degree of success on the Web. How about the others?</h3> WebP isn't very good in my opinion, plumetting at higher quality & being soundly defeated by AVIF at lower quality. JPEG clearly still has potential even years later, and MS-SSIM tuning seems to do the encoder some favors with the photographic dataset. The jpegli encoder borrows some of JXL's fancy coding techniques to improve JPEG quality even further, especially at higher quality. I didn't even test jpegli with the superior XYB colorspace (as this proved hard to do), which to my eyes nets jpegli an even better quality improvement over standard jpeg encoders. There's a cool Twitter thread</a> about this. According to Jyrki Alakuijala</a>, "jpegli supports more than 8 bits per channel (around 10.5 bits) and can codify HDR dynamics (HLG/PQ/XYB/etc.) in the old '8-bit' format." This seems extremely promising. Takeaway</h3> I want a web where both AVIF & JPEG XL can exist, and developers decide which format to use for its merits. Google's Chromium team has rejected JXL support for Chrome, and Mozilla remains "neutral," on JXL, keeping a flag in Firefox Nightly to enable partial support. The Chromium team shouldn't have the absolute authority to shoot down would-be standards like this, as it seems they are the sole decision-maker in this space & all others must follow their lead. In my opinion, JPEG XL & AVIF have fundamentally different strengths which lend them to different use cases. Considering JXL has been endorsed by Facebook, Adobe, Intel and the Video Electronics Standards Association, The Guardian, Flickr and SmugMug, Shopify, the Krita Foundation, Serif Ltd, Gaia Sky, and many more, the market is most certainly interested. My current optimistic hope is that JXL takes off outside the web among professionals working with tools like the Adobe suite or alternatives, and camera manufacturers, smartphone OEMS, and others take notice & begin to think about JXL more seriously. The benefits cannot be ignored, and it is (in my opinion) the only image format that is in every way superior to JPEG & offers a concrete future for the many existing JPEGs on the Web & beyond. While this is a non-scientific test, please share this post if you'd like to see JPEG XL get more attention for its merits. AVIF is already widely supported, I don't need to fight for it; JPEG XL still has a war to be won, and I'd like anyone who wants better images & a better Web to be on JXL's side as well as AVIF's, and to understand that JPEG XL needs our assistance as users to gain widespread adoption. Here's what I'd recommend doing right now: Using Chrome or another Chromium-based browser? Consider switching to Thorium</a>. It is available for nearly every platform, including Windows, Linux, macOS, and Android, and has complete JXL support. It will sync with your Google account just like Chrome will. It is also faster</a>. Using Firefox? Not a bad choice. If your workflow permits, switching to Firefox Nightly for casual browsing & enabling the JXL flag for partial JXL support by searching 'jxl' in about:config</code> is not a bad option. I'm attached to Firefox myself, & using the flag on my phone works great for most things that don't employ an alpha channel or HDR. If you'd like to take a look at some beautiful JXL images, you can head here</a>. You can test your browser's JXL support here</a>. Remember, the Web may be the last platform without a proprietor, and we need to keep it that way. Sponsor Me on GitHub Sponsors</h3> Help support my open source efforts - a little goes a long way! Sponsor</a> Browser Benchmarks 2023-02-28T00:00:00+00:00 Let's compare browser performance across a variety of architectures, operating systems, and browsers. Which browser is the most performant? Why?</h2> The web browser Thorium</a> claims to improve upon Chromium's (and by extension, Chrome's) browser performance significantly. I think Thorium's a really neat browser, & I commend their continued support for the fantastic JPEG XL codec when most seem to have abandoned it. This isn't to disprove or undermine their browser benchmark results in any way, I simply wanted to do my own testing & report back my results as a means of letting myself know how my different browsers perform. I am a Firefox user right now, but that's always subject to change, although right now I'm not particularly partial to Chromium-based browsers. Methodology</h2> I currently use the Firefox Flatpak on my desktop & laptop, Firefox on my iPad (being WebKit based, I didn't feel it was useful to test it), & I just recently switched from Bromite to Firefox Nightly on my Pixel 7 for its JPEG XL support & timely updates (where Bromite hasn't been updated since Chromium 108). For this test, I benchmarked: Chromium 110.0.5481.177 on my 13700k/6600xt</li> Thorium 109.0.5414.120 on my 13700k/6600xt</li> Firefox 110 (Build ID 20230214051806) on my 13700k/6600xt</li> Safari on my iPad Mini 6 (iOS 16.3)</li> Chromium 109.0.5414.117 on my Pixel 7 (CalyxOS)</li> Firefox Nightly 112.0a1 (Build ID 20230226214053) on my Pixel 7 (CalyxOS)</li> </ul> And, here are my system specs: ➜ ~ inxi -v CPU: 16-core (8-mt/8-st) 13th Gen Intel Core i7-13700K (-MST AMCP-) speed/min/max: 1041/800/5400 MHz Kernel: 6.1.12-zen1-1-zen x86_64 Up: 4d 17h 40m Mem: 8923.9/31871.1 MiB (28.0%) Storage: 6.83 TiB (19.2% used) Procs: 557 Shell: Zsh inxi: 3.3.25</code> Although I was on the previous kernel release when these tests were completed, a version of kernel 6.1. I don't think that matters too much here, but I thought it may be important to note. I didn't enable any flags for any of the browsers except one to enable JXL support on Firefox Nightly, so all were kept almost completely stock. Anyhow, let's get into the results: Speedometer 2.1</h3> Here are my Speedometer 2.1 results: Via browserbench.org</a>: "Speedometer is a browser benchmark that measures the responsiveness of Web applications. It uses demo web applications to simulate user actions such as adding to-do items." Here we see Thorium pull ahead quite a bit, & it's a consistent theme that Thorium takes the lead when it comes to JavaScript stuff. My iPad follows in a closer second than anticipated considering its running a mobile chip, followed by my desktop's other browsers & finally the Pixel. It is important to note that the WebKit team produces this benchmark along with MotionMark & JetStream. MotionMark 1.2</h3> Here are my MotionMark 1.2 results: Via browserbench.org</a>: "MotionMark is a graphics benchmark that measures a browser’s capability to animate complex scenes at a target frame rate. More details</a> about the benchmark are available. Bigger scores are better. For accurate results, please take your browser window full screen, or rotate your device to landscape orientation." This is where we see Firefox take the lead for once on my desktop, but it is negated by my iPad, which somehow obliterates my desktop-class Radeon GPU. I'm not entirely sure how to interpret these results, but I guess Firefox's WebRender really uses the GPU well & Safari does even better. JetStream 2</h3> Here are my JetStream 2 results: Via browserbench.org</a>: "JetStream 2.1 is a JavaScript and WebAssembly benchmark suite focused on the most advanced web applications. It rewards browsers that start up quickly, execute code quickly, and run smoothly. For more information, read the in-depth analysis</a>. Bigger scores are better." Once again, the iPad has a fantastic showing. The Pixel surprisingly gained a lot of ground here on Chromium, & Thorium continues to show steady & consistent gains over Chromium. BaseMark Web 3.0</h3> Here are my BaseMark Web 3.0 results: Via basemark.com</a>: "Basemark Web 3.0 is a comprehensive web browser performance benchmark that tests how well your mobile or desktop system can use web based applications. This benchmark includes various system and graphic tests that use the web recommendations and features. After running the benchmark you will see how your system performed compared to other systems and browsers in Basemark Power Board</a>. Basemark Web 3.0 measures real-world client-side performance to detect browser bottlenecks." This basically shows the same results as before, with Firefox falling further behind that I'd expect. I don't think that in everyday use it feels much slower, and actually I think the "feeling" of speed is more pronounced going from Chromium to Thorium versus Firefox to Chromium. The iPad once again kills it, which is also not consistent with my "feeling" when it comes to browsing the web on that device. The Pixel's results are relatively in line with how I've tangibly experienced the device's web performance. WebXPRT 4</h3> Here are my WebXPRT 4 results: Via principledtechnologies.com</a>: "WebXPRT 4 is a browser benchmark that compares the performance of almost any web-enabled device. It contains HTML5, JavaScript, and WebAssembly-based scenarios created to mirror the tasks you do every day: Photo Enhancement, Organize Album Using AI, Stock Option Pricing, Encrypt Notes and OCR Scan using WASM, Sales Graphs, and Online Homework. Use WebXPRT to see exactly how well different devices handle real-world tasks." My interpretation of WebXPRT is that it is a more applicable (albeit more complex) indicator of web performance, and doesn't necessarily mean anything relative to the "snappy" feeling you get when loading simple web pages. I'm surprised to see Firefox pull ahead here, even if it's just by a tiny bit, while the iPad's results finally feel more realistic to me. The Pixel's results are basically the same as always, although Firefox Nightly actually ekes out a win here. Mozilla Kraken</h3> Here are my Mozilla Kraken results (important that I be thorough in saying lower is better here): Via wiki.mozilla.org</a>: "Kraken is a JavaScript performance benchmark created by Mozilla</a> that measures the speed of several different test cases extracted from real-world applications and libraries." Ultimately, this feels very similar to Speedometer & I honestly kind of doubt Firefox's win is very noteworthy here. The iPad continues to perform really well, & the Pixel's Firefox & Chromium results feel closer to one another. Maybe Mozilla is optimizing JS performance to score well on this benchmark, as it is their benchmark, after all. Conclusion</h2> Here's the geometric mean of all the test results: I calculated the geometric mean using the reciprocal of each Kraken benchmark result because for that test lower was better, and apparently the geometric mean is sensitive to values close to zero, so keep that in mind. If there's a better way to do this (like normalizing the values with min-max or median normalization) let me know. Here's the geometric mean of all the test results besides MotionMark: Just because I really doubt the iPad is that performant. I'm open to being corrected, though. Here's the number of first place finishes each browser recieved: I will leave you, the audience, to come to your own conclusions here. Overall, I really like Thorium. Let me know if you'd like another one of these with certain special flags, different benchmarks, browser compatibility, or different versions of the same browser. This took me around 3 days to put together. Thanks for reading! Aviator Versus Handbrake; Visual Quality Assessment 2023-02-12T00:00:00+00:00 In this test, we will compare the latest Aviator release to the latest Handbrake release in terms of visual quality per bit. Note: rAV1ator</a> isn't compared in this test because it is based on different underlying technologies that would be hard to compare directly to Aviator & Handbrake, especially for very short clips. Overview</h1> This blog post illustrates a comparison between Aviator 0.2.0 & Handbrake 1.6.1 using two sources that illustrate different use cases; the first being the 2160p crowd_run.y4m</code> source from Derf's Test Media</a> & the other being a short clip of an animated sequence from a popular animated show. Conditions</h2> This is a relevant comparison because both Aviator</a> & Handbrake</a> use the SVT-AV1 encoder under the hood, and both can encode videos with 10 bit color (default for Aviator) which should increase the visual quality of the output. Because of the similarity of the two programs when outputting AV1 video, I was initially under the impression that upon beginning the test I'd be incapable of discerning a difference between the two based on how well they scored. I was mistaken. Aviator settings: - Resolution: 3840x2160 for crowd_run, 1920x1080 for animation - CRF: 25-50 (10-60 for the expanded set) - Speed: 6 for crowd_run, 4 for animation - Container: MKV Handbrake settings: - Format: Matroska - Resolution Limit: None (defaults to same as Aviator) - All Filters Off - Video Encoder: AV1 10-bit (SVT) - Framerate: Same as Source - RF: 25-50 (10-60 for the expanded set) - Speed: 6 for crowd_run, 4 for animation Notes</h2> While encoding, both used a similar amount of RAM under load but Handbrake used 10gb in the background even after the encode had completed for each trial with crowd_run. This meant that I had to restart it every time I wanted to encode another iteration of the test.</li> Handbrake failed to copy the subtitle track by default for the animated source, which can be a problem when encoding any kind of animation where the audio isn't in your native language.</li> Handbrake's "Tune" dropdown explains in the tooltip that it can "improve efficiency for particular source characteristics" without specifying such characteristics, so this was left alone. Its dropdown had one option, which was to tune for PSNR. I know PSNR is not an accurate gauge of visual quality, but the average user may not because there is no tooltip explaining what this acronym means.</li> The results for the animation tests were calculated using the overall bitrate of the file, not just the video portion; this means that for Aviator, the subtitle track was factored into that overall bitrate. This shouldn't have hurt the results very much, but offers a minute disadvantage to Aviator.</li> Film grain synthesis was not used on the crowd_run source, despite the presence of grain. While Aviator offers this functionality, Handbrake does not, and most visual quality metrics don't understand the benefits of grain synthesis anyway despite the improvement to visual quality offered by the feature for sources with grain present.</li> </ul> Results</h1> These tests were run using the SSIMULACRA2</a> visual quality metric via ssimulacra2_rs</a>. This metric is designed to model human vision far better than VMAF, SSIM, PSNR, & other less effective alternatives. First, the crowd_run results from CRF 25 through 50. This is a clear and decisive victory for Aviator by a larger margin than I initially thought possible between the two utilities, considering that they're both using 10 bit SVT-AV1. Aviator's out-of-the box tuning for visual quality has paid off. It is important to note that these are incredibly high bitrates. The scene is very complex, and the video being 2160p50 makes it require a lot of bits to achieve a watchable level of visual quality. SVT-AV1 defaluts to CRF 35 internally, while Aviator defaults to CRF 32 & Handbrake defaults to RF 30. The results above were done in increments of 5 from CRF/RF 25 through 50 (25, 30, 35, etc). In order to get the bigger picture, I tested a wider quality range from 10 through 60 that dips into the realm of impracticality a bit given the obscene bitrate approached by lower CRF/RF values & the relatively low quality image produced with higher CRF. Here's the expanded results, with CRF 10 through 60. Here, we see the two become relatively the same at lower & higher quality. While Aviator has a tiny advantage at lower quality, Handbrake looks to take the lead by an almost imperceptible margin at higher quality. For this source, because the bitrate skyrockets beyond CRF/RF 25 & the quality plummets beyond CRF/RF 50, I would consider this a win for Aviator in the range I'd consider usable that encompasses the default quality levels for Aviator, Handbrake, & SVT-AV1's stock behavior. For the animation test, CRF 25 through 50 were tested. This sees less of a performance delta than the more lifelike crowd_run source, and also sees bitrates reach an acceptable level that would be more common to see from an animated 1080p24 source. Aviator sees an advantage here still, with the gap widening at slightly higher bitrates. Conclusion</h1> It is clear that Aviator's prioritization of visual quality performance has paid off, even with SSIMULACRA2 being a synthetic benchmark. It appears that Handbrake is only worth using when dipping below CRF/RF 20, but when fine detail preservation at very high bitrate is a priority it may be worth using another codec (which Handbrake will offer you the option to use, seeing its diverse selection of codecs besides AV1). Even then, the quality difference is minute & may vary between sources. Aviator is the undisputed AV1 champ between the two, and appears to win in situations where AV1 is most useful. Grain Synth</h2> While it is hard to benchmark, it is worth mentioning that Aviator supports film grain synthesis ("Grain Synth") while Handbrake does not. This can improve the visual quality of any source with grain present by removing it & reapplying a synthesized version at decode time. This allows the encoder to spend less bits compressing grain (which is notoriously difficult to compress) and instead apply it artificially with little to no discernible difference to the viewer. The crowd_run source (or any live action source that hasn't been heavily denoised) has grain present, and while SSIMULACRA2 doesn't totally understand the benefits of grain synthesis, it is clear to even the untrained eye the advantage it offers. Sources</h2> If you'd like the encoded clips I used to run this test, I will give them to you for crowd_run. Otherwise, feel free to replicate the crowd_run segment for yourself or the animation segment using another animated source of your choosing. Thanks for reading! Aviator 0.2.0 2023-02-11T00:00:00+00:00 Aviator aims to enable simple, accessible AV1 encoding for everyone with a GUI powered by a fast AV1 encoder. Why?</h2> First, you can install here</a>. To enable simple, accessible AV1 encoding for the masses via Flathub. See my post about rAV1ator here</a> for the rav1e + av1an based version of this utility. Aviator has used SVT-AV1 & FFmpeg for a while, but has been missing a couple key features including a progress bar, a stop encode button, and more buttons on the header bar then just the close button. I've been glad to make this release available on Flathub, but I think it has surfaced some serious issues with how Flathub allows you to build applications. rAV1ator vs Aviator: Flathub</h2> There's an open issue</a> regarding exactly my complaint here, which is that Flathub doesn't allow --share=network</code> in the build-args. This places an undue burden on the developer to jump through extra hoops to make "reproducible builds," which I could guess there is potential to do with crypto instead of putting it on the dev. I champion Flatpak because of its superb user experience and seamless integration with the Linux desktop, but from a developer standpoint it is a nightmare of quirks & workarounds that require a depth of experience that many simply do not possess. Even if I did have such experience, I'm not sure how I'd be able to ship rAV1ator on Flathub with such a hurdle. Aviator is great, and I'd easily recommend to anyone interested in video encoding. But I'd like it to be the best I can make it, and Flathub is simply not allowing this to happen. Hopefully with enough complaining we'll be able to fix things, but the light at the end of the tunnel doesn't seem to be approaching very quickly. rAV1ator 1.0 2023-02-08T00:00:00+00:00 rAV1ator, a powerful AV1 encoding GUI, sees its 1.0 release. Edit (Feb 11, 2023): This version of Aviator is now called rAV1ator & is distributed outside of Flathub. Instructions to install are below, & the page has been edited to reflect the sitation. Edit 2 (Mar 8, 2024): rAV1ator is not to be confused with rAV1ator CLI, which is a different tool based on different underlying technologies for the command line. It does not share any "DNA" with this article's rAV1ator. What is rAV1ator?</h2> Manifesto</h3> rAV1ator</a> was designed to make AV1 encoding accessible to the average person, but with the intention to teach someone who is a total beginner the ins & outs of encoding AV1 videos. You can install it by running flatpak --user remote-add --no-gpg-verify project-volo https://giannirosato.com/repo && flatpak --user install project-volo net.natesales.rAV1ator</code>. I don't need to make the pitch for the open source video codec AV1 - you can find that here</a> - but currently I believe a large issue barring passionate interest in AV1 is how hard it is getting started. There are tools out there that make it easy to use the SVT-AV1 encoder with a GUI, but I think there's a lack of other alternatives that are more flexible and use cutting-edge technology while remaining easy to understand (especially as it pertains to Linux where many are familiar with the command line & don't see the need for a GUI). Enter rAV1ator. How does it work?</h2> rav1e + av1an</h3> Behind the scenes, rAV1ator uses the Rust-based rav1e</a> encoder which has seen decent increases in speed recently and is improving more every day. We decided to use rav1e in order to give users a memory-safe AV1 encoder implementation that prioritizes visual quality & 'just works' for the most part. Specifically, we are using BlueSwordM's fork which has been tuned to improve visual quality. Alongside rav1e, we are using a custom av1an</a> fork to split videos into multiple chunks that can be encoded in parallel to increase speed & provide more options to the user. av1an also allows us to use chroma noise with our grain synthesis option which is discussed two sections down. We've configured av1an to spawn chunks automatically, which it does based on your CPU core count & RAM quantity. Each worker is allocated one thread to account for systems that may not have multithreading. This default provides a usable balance between not aggressively hogging system resources & still performing well. That being said, AV1 encoding is heavy and will tax your system. Video</h3> The first screen is unassuming, with sane defaults that respect the fact that a user might not know exactly what they are messing with right out of the box. Helpful tooltips are provided upon hovering over almost anything, with detailed descriptions of each function. Resolution will match the video source, & otherwise defaults to 1536x864. rav1e's quantizer is set to default to 100 internally, but I believe 80 is a better option for the sake of providing better quality so that is what we use as a default. Speed 6 is a bit on the faster side for my taste, but not cripplingly slow on most systems. Grain synthesis is disabled by default, as it is not always desirable. Grain Synthesis</h3> AV1 provides a unique feature in the form of grain synthesis, which has largely been reserved for use in the command line for those who know its benefits. Grain synthesis aims to provide natural film grain to a video without hurting compression efficiency by removing grain from the source & reapplying it at decode time as a filter. As explained in the in-app tooltip, film grain levels are equivalent to ISO noise from a camera. 1 would be ISO 100, 24 would be ISO 2400, continuing up to 64. Because of av1an, we are also able to apply this noise as chroma noise which increases its realism. Audio</h3> The audio screen provides more sane defaults; these include compression via the highly efficient & open source Opus codec, an option to downmix the output to stereo from a larger number of channels, and a toggle for variable bit rate. By default, the audio bitrate will be pulled from the source. If this isn't detected, it will default to 48kbps which is admittedly rather low but ultimately not something I've heard many complaints about. I think it provides a reasonable level of quality for more general applications, like screen recordings or smartphone videos. Downmixing to stereo uses ffmpeg's default -ac 2</code> flag which I've found to be decent and reliable despite discarding the subwoofer channel(s). Output</h3> The output screen has a file selector with two container options that are clearly explained with a tooltip, and a big blue 'Encode' button that eagerly invites you to start encoding. Upon pressing the blue button, the progress bar will indicate to you the status of the encode alongside a percentage. Eventually, you will get a notification when the encode is complete & the progress bar will reflect the status of your encode with a message just in case you have Do Not Disturb on or missed the notification by accident. If the av1an process does not complete successfully or you manually halt your encode with the Stop Encode button, you will be met with a different notification that the status of the progress bar's text will math. That's great, but I'm on Windows</h2> Well, I'm sorry for you, and you should reconsider your choice of operating system. That being said, there is a great Windows AV1 encoding GUI called nmkoder</a> that you should definitely try out. You're missing feature X, Y, Z</h2> Right now, I'm going to let this release fester for a bit before diving back into development. I would still love to see your suggestions for improvement over at the github page</a> where you can put in issues for whatever you'd like to see us do in the future. Regarding hardware encoding: I did my research</a>. There's a time and a place. Special Thanks</h2> Many thanks to Nate</a> for walking me through development since I began learning python as my first real language a couple weeks ago, and thanks to the people who are a part of the AV1 community for their passion toward multimedia codecs that keeps our videos crispy, our storage unconstrained, & our CPUs toasty. Particular thanks to Blue</a> & MasterOfZen</a> for their tools that I use in & out of this project and are indispensible to the community. Enjoy rAV1ator, happy flying! ✈️ </a>

Conclusion</h2>

Conclusion</h2>

Sources</h2>
If you'd like the encoded clips I used to run this test, I will give them to you for crowd_run. Otherwise, feel free to replicate the crowd_run segment for yourself or the animation segment using another animated source of your choosing. Thanks for reading!</p>

That's great, but I'm on Windows</h2>
Well, I'm sorry for you, and you should reconsider your choice of operating system. That being said, there is a great Windows AV1 encoding GUI called nmkoder</a> that you should definitely try out.</p>

Gianni Rosato

Introducing oavif

The Multimedia Renaissance

Comparing Video Encoders

The End of SVT-AV1-PSY

Stupidly Tiny Images

QOI Encoding in Zig

Lossless Data Compression

XYB JPEG

Better JPEG Encoding From Google

Who Has the Best Hardware AV1 Encoder?

Photographic Corpus (lossy)

Image Codec Comparison

Browser Benchmarks

Aviator Versus Handbrake; Visual Quality Assessment

Aviator 0.2.0

rAV1ator 1.0

Gianni Rosato

Introducing oavif

Convergence</h2> A simple convergence loop looks like this:</p> Decode input, pass to encoder</li> Decode encoder output & compare to input with metric</li>

The Multimedia Renaissance

Comparing Video Encoders

Why?</h2> Comparing video encoders isn't hard; in fact, it is usually quite easy. However, it is very often done incorrectly.</p>

Comparisons</h2> There are three "tiers" of comparisons, each involving a bit more data than the last:</p> Comparing a single video to another single video</li> Comparing a series of encoders in terms of compression efficiency</li>

Comparing Overall Efficiency</h2> Before moving on, consider what we need for this graph:</p> A value representing the metric difference between two curves</li> A value representing the time difference between the encoder/configuration used for each curve</li>

Sponsor Me on GitHub Sponsors</h3> Help support my open source efforts - a little goes a long way!</p> Sponsor</a></p>

The End of SVT-AV1-PSY

Why?</h2> If you aren't familiar with codecs or AV1, I'm not sure why you're on my site. Nevertheless, here's some background:</p>

Sponsor Me on GitHub Sponsors</h3> Help support my open source efforts - a little goes a long way!</p>

Stupidly Tiny Images

JPEG XL Summary</h3> Overall, JPEG XL results seem mixed at this scale, and we've been driven to resorting to slightly lossy compression which isn't a great showing for the codec. However, if you look closely, there are some images that are the smallest with JPEG XL...</p>

Mixing Codecs</h2> By mixing our favorable results from WebP and JPEG XL, we minify our 88x31s to the maximum degree:</p>

Animation</h2> For animated images, we have three major options:</p>

Conclusion</h2> My final thoughts:</p>

Sponsor Me on GitHub Sponsors</h3> Help support my open source efforts - a little goes a long way!</p>

QOI Encoding in Zig

Lossless Data Compression

XYB JPEG

Better JPEG Encoding From Google

Who Has the Best Hardware AV1 Encoder?

Conclusion</h2>

Photographic Corpus (lossy)

Image Codec Comparison

Conclusion</h2>

Browser Benchmarks

Aviator Versus Handbrake; Visual Quality Assessment

Sources</h2> If you'd like the encoded clips I used to run this test, I will give them to you for crowd_run. Otherwise, feel free to replicate the crowd_run segment for yourself or the animation segment using another animated source of your choosing. Thanks for reading!</p>

Aviator 0.2.0

rAV1ator 1.0

Sponsor Me on GitHub Sponsors</h3>
Help support my open source efforts - a little goes a long way!</p>
Sponsor</a></p>

Sponsor Me on GitHub Sponsors</h3>
Help support my open source efforts - a little goes a long way!</p>
Sponsor</a></p>

Sponsor Me on GitHub Sponsors</h3>
Help support my open source efforts - a little goes a long way!</p>
Sponsor</a></p>

Sources</h2>
If you'd like the encoded clips I used to run this test, I will give them to you for crowd_run. Otherwise, feel free to replicate the crowd_run segment for yourself or the animation segment using another animated source of your choosing. Thanks for reading!</p>