Transparent AVIF images work in Firefox now!
I’ve finished the AVIF image transparency support in Firefox recently, in BMO 1654462.
The AVIF support can be enabled by toggling the
false by default). As far as I know, it’s the last piece of the main AVIF work that has to be done. Hopefully the
image.avif.enabled would be set to
true by default soon.
To be honest, I am just lucky enough to pick this topic to implement the last piece.
Most of the hard works are done by our excellent
mp4parse-rust developers and the original AVIF decoder author. They give me great advices and help me to achieve this work.
Before v.s. After
|Before (w/ green background)||After (w/ green background)|
|I420 w/ I709|
|I444 w/ I709|
|I444 w/ Identity|
Test page is here
How the AVIF image decoding work
The steps to render an AVIF image to the screen from the raw data are as follows:
- Extract the AVIF image data from the mp4 container.
- The AVIF image is stored in the mp4 format, so we need to get its data first
- we use
mp4parse-rustto parse the AVIF image in Firefox
- Decode the AVIF image to Y-Cb-Cr-Alpha data by Dav1d or AOM decoder
- Convert the Y-Cb-Cr-Alpha data into RGBA data
- The conversion here is the critical part of this project.
- Render the decoded RGBA data to the screen
- This part is relatively easy since Firefox already has APIs to do this job for all other image formats or videos.
The biggest challenge in this project is to convert the Y-Cb-Cr-Alpha data into RGBA data correctly. This conversion is the only part that we don’t exactly know how to do in Firefox.
To know how to convert the Y-Cb-Cr-Alpha data into RGBA data, first thing I do is to look at the third-party library used in Firefox to convert the Y-Cb-Cr to RGB data. The third-party library is libyuv. Fortunately, I saw one function that can transform the Y-Cb-Cr-Alpha into RGBA data if the Y-Cb-Cr’s data layout is I420. There are many different layouts of the Y-Cb-Cr’s data, such as I444 or I422. Therefore, the problem is narrowed down to convert the Y-Cb-Cr-Alpha into RGBA data if Y-Cb-Cr’s data layout is not I420.
The next task is to look at the function that converts Y-Cb-Cr-Alpha to RGBA data for the I420 layout to see how the conversion is implemented and evaluate if it’s possible to apply its manner to other forms.
The answer is yes. The conversion for the I420 layout is a 3-step process:
- It converts the Y-Cb-Cr to RGB data and fills the RGB data into the RGBA buffer.
- It injects the Alpha data into the RGBA buffer.
- It does the “pre-attenuate” if the caller requests it.
The last two steps are unrelated to the Y-Cb-Cr ‘s layout. To apply the same approach to the non-I420 layout, the only thing I need to figure out is how to convert the Y-Cb-Cr to RGB data. The problem now is narrowed down to the conversion for non-I420 layout. Fortunately, the
libyuv library already provides such functions. All I need to do is call the operations depends on the Y-Cb-Cr ‘s layout, then fill the alpha data and do the “pre-attenuate” if required. As a result, the conversion works as follows:
YCbCrA_to_RGBA: if the layout is I420 format call libyuv::I420AlphaToARGB else convert the Y-Cb-Cr data to R-G-B data via YCbCr_to_RGBA32 fill the Alpha data to the RGBA32 buffer attenuate RGBA data YCbCr_to_RGBA32: switch layout: case I400: call libyuv::I400ToARGB case I420: switch color-space: case BT601: call libyuv::I420ToARGB case BT709: call libyuv::H420ToARGB case BT2020: call libyuv::U420ToARGB case I422: switch color-space: case BT601: call libyuv::I422ToARGB case BT709: call libyuv::H422ToARGB case BT2020: call libyuv::U422ToARGB case I444: switch color-space: case Identity: call GBRPlanarToARGB case BT601: call libyuv::I444ToARGB case BT709: call libyuv::H444ToARGB case BT2020: call libyuv::U444ToARGB
To see what the actual code is, please read the code here
2020 is a year trapping people in a specific zone and make us feel stuck in life. I am glad I have an opportunity to embrace a new challenge and explore a field I don’t have experience in at the end of 2020.
This is my first work in the graphic area, and I enjoy it. It reminds me that we can always try something new in life.
2020 doesn’t stop me from keeping advancing myself. I am confident that the next year and the following years won’t stop me either. This won’t be the last time I try something different. I look forward to writing a new story in my life!
2021, I am coming!