TexNN: Fast Texture Encoding Using Neural Networks

Abstract

We present a novel deep learning‐based method for fast encoding of textures into current texture compression formats. Our approach uses state‐of‐the‐art neural network methods to compute the appropriate encoding configurations for fast compression. A key bottleneck in the current encoding algorithms is the search step, and we reduce that computation to a classification problem. We use a trained neural network approximation to quickly compute the encoding configuration for a given texture. We have evaluated our approach for compressing the textures for the widely used adaptive scalable texture compression format and evaluate the performance for different block sizes corresponding to 4 × 4, 6 × 6 and 8 × 8. Overall, our method (TexNN) speeds up the encoding computation up to an order of magnitude compared to prior compression algorithms with very little or no loss in the visual quality.

Image and Video Coding Techniques for Ultra-low Latency

“…Pratapa et al [115] took a neural network approach to compress textures in the ASTC format. The most computationally intensive parts (partition selection and color endpoint approximation) are oloaded to the training phase.…”

“…Deep learning has been used to accelerate complex steps of existing algorithms [85,90,107,115]. While the computational cost of deep learning inference is very high, it might still be lower than some parts of modern complex formats such as ASTC or VVC.…”

Performance of Texture Compression Algorithms in Low-Latency Computer Vision Tasks

“…The ASTC encoding had usually been considered too slow for even real-time purposes. However, in [26], the authors replace the complex configuration search by a NN inference, accelerating the computation by up to 10×. To compare with other types of compression methods, in [17] the authors evaluate the decoding speed of a JPEG XS decoder, including memory transfer times.…”