ARM Community: ARM Unveils Details of ASTC Texture Compression at HPG Conference - Part 2 - ARM Community

In part one of this two-part post, I wrote about the first part of Tom Olson's paper from High Performance Graphics 2012. We saw how Bounded Integer Sequence Encoding allows ASTC to have a finely graded tradeoff between the roles of bits in a block. In this part, we will learn about the other techniques that contribute to ASTC’s flexibility and quality.

Partitioning the Color Space
Sometimes, the colors in a block do not fit neatly along one line. In an image of a red ball resting on green grass, blocks at the edge of the ball will may contain some green pixels and some red pixels. In this case, the encoder will recognize this, partition the texels into two sets, and assign two separate sets of color endpoints, one for the reds and one for the greens. It will then assign each texel to one of the two color partitions.

Instead of storing an extra bit for each pixel, an index is stored, which looks up a partition pattern in a large table. As there are 2048 patterns to choose from, the chances are that there will be a good match somewhere in the table.

In fact, the table is not stored as a table, but instead generated procedurally from a function. This saves a lot of space, as the table would otherwise be quite large. It can also be generalized to define 2-partition, 3-partition or 4-partition patterns with virtually no extra space. (There is another advantage which we will come back to later.)

A different color endpoint mode can be chosen for each partition, if this would be more efficient.

Dual Plane Mode
The features described so far work well when the channels in the texture are well correlated with each other, as is usually the case in images of read objects. However, we don’t always use textures just for surface color.

In the case of masked textures, which contain transparent “holes”, the alpha (transparency) channel often has little correlation with the colors in the image. Similarly, for normal maps, two texture channels are often interpreted as the X and Y coordinates of a surface normal at a particular point, and these two exhibit less correlation than actual images.

In this case, ASTC allows the encoder to store two color weights per texel, and then apply the second weight to a selected channel from the image. So, in the mask case, we would use weight 0 to interpolate the RGB components, and weight 1 to interpolate alpha.

144 Texels Into 128 Bits Doesn’t Go
For the larger block sizes, it is not possible to dedicate even 1 bit to every texel weight. The way around this is traditionally to omit values from the weight grid (for example in a checkerboard pattern), and reconstruct the missing weights by summing the nearest existent values.

Andrew Pomianowski, from our partners AMD, contributed a valuable insight into this process. We can allow the weight grid to be independent in size from the texel grid, and interpolate the weights bilinearly to calculate the effective weight at each texel position. This is actually easier to implement than traditional infill mechanisms, as there are no special rules for edge texels, and it produces good results for any combination of grid size and texel footprint.

So, for smoothly changing areas of an image, it is often possible to encode a coarse weight grid, which allows use of finer quantization on the color endpoints or the weights themselves. This is another example of the tradeoffs available in the block encoding.

Dynamic Range
Most existing texture compression methods encode images which have low dynamic range. Each of the red, green and blue components for a texel lie in the range 0..1, and the values are linearly spaced.

Modern content may also make use of high dynamic range (HDR) content, where the brightness of components can go higher than 1.

ASTC allows encoding of HDR images, with a greatly extended dynamic range and near-logarithmic interpolation of color values to match the human visual system. At 8 bits per pixel, it is comparable to the current de facto HDR encoding scheme, BC6H.

Unlike BC6H, however, it offers HDR as another independent choice in its palette of options. Content creators have exactly the same range of color formats and bit rates as they do with LDR images. Even at lower bit rates, the results are stil impressive:



Adaptive Encoding
We have already seen that ASTC allows different bit tradeoffs in each block, but the dynamic adaptation goes a lot further than that.

Of all the options available: block footprint size, number of partitions, partition index, color endpoint modes, quantization levels, weight grid size, single or dual plane, and choice of LDR/HDR, only the block footprint is global. Everything else can vary from block to block.

This means that grey areas in a color image will be encoded using the more efficient luminance-only data, opaque areas of an RGBA image will be encoded without spending bits on the alpha channel, and smooth areas will be encoded with fewer (but more accurate) color weights. This freedom of choice increases the quality of the image as a whole, which would otherwise have to be encoded using the same worst-case parameters for every block.

While this is undoubtedly good for quality, it brings with it a problem of speed. With the large number of choices available for each parameter, the encoder has a vast configuration space in which to search when encoding a block. Doing this naively would lead to very long encoding times.

Developing good heuristics, search algorithms and error metrics has been the main thrust of encoder technology development and using ARM’s encoding tools, the user can now trade off encoding time against quality according to their needs.

World-Beating Performance
Having described all of this, the results speak for themselves. Despite using some bits in each block to indicate the encoding mode, ASTC performs as well as, and usually much better than, the currently available texture compression schemes.

• At 2 bits per pixel, it outperforms PVRTC by more than 2dB.
• At 3.56 bits per pixel, ASTC does better than PVRTC and DXT1 at 4 bits per pixel by around 1.5dB. It also beats ETC2 at 4 bits per pixel by about 0.7dB. This is despite a 10% bit rate disadvantage.
• At 8 bits per pixel for HDR, ASTC and BC6H are roughly equivalent, both producing very good mPSNR ratios of 40-50 dB in the tested images.

For comparison, it is worth noting that differences as small as 0.25dB are visible to the human eye. The HPG paper goes into detail on the measurement techniques, and the significance of the results.

3D Textures
One thing that Tom does not discuss in his paper, for reasons of space, is ASTC’s support for 3D textures. The partition function generalizes nicely to 3D, allowing ASTC to define a cuboidal block footprint from 3x3x3 texels up to 6x6x6.

The advantage of a true 3D encoder, as opposed to separately encoding 2D slices of a 3D image, is that it can take advantage of similarity between the slices. When comparing the two approaches, our experiments show a significant increase in measured quality for true 3D encoding at the same bit rate.

Allowing flexible bit rate selection for 3D images is revolutionary. Until now, their expense means that use has been very rare. ASTC does not just support using 3D images in the mobile space, it enables them, handing content developers a whole new technique to use. Once they get their hands on this new tool, who knows what innovative applications they will discover?

Conclusion
ASTC is all about giving content developers what they want – a wide range of supported color formats, together with fine control over the space/time/quality tradeoff. And it does all of this while comfortably outperforming the best-in-class LDR compression schemes, and matching current best performance for HDR.

With ASTC, I think it’s no exaggeration to say that ARM has created the biggest leap forward in texture compression technology for years.

Sean Ellis is part of the Technical Staff in ARM's Media Processing Division. He has been working with 3D graphics since 1988, including on ARM's original machine, the Archimedes. He was a key player in the specification of the Java Mobile 3D Graphics standards (M3G and M3G2) and is currently working to define the architecture of the next generation of ARM GPUs. He is also involved in patent work, helping to protect ARM's innovations in the multimedia arena.