DXT compression - part 2 - The basics

Welcome to part 2 of the DXT compression series. In this series I go over the techniques and results used to compress Firefall's texture data as they are discovered and implemented. Red 5 Studios has graciously allowed me to post about this work publicly with the intention that peer review and group process will end up with something better overall, not only just for Red 5 but for others in the industry as well. So please do comment and suggest improvements if you have ideas or thoughts on the matter :) 

The Data Set

Firefall - Orbital Comm Tower

The data set for these tests are all the textures in the Orbital Comm Tower map in Firefall. I chose this subset of the entire game as it is many times smaller to work with and its compression ratios appear to be pretty close to the entire game.

In total it is 2.6gb of textures split into 128x128 tiles to be used later with Virtual Texturing.

Firefall's texture data is separated into 3 DXT5s containing the usual suspects of diffuse, alpha, normal, specular exponent, material, etc..

First I intend to thoroughly go over the basics and find the optimal settings for LZMA with DXT compression. Then in future posts go over straight forward data transformations and block re-ordering. My goal is 1-3 posts per week, time permitting.

---

The Baseline

First off, lets look at what the baseline is. Straight up compressing the data with LZMA gives the following results (measured in bpp) with various compression levels. Default parameters for each level are used. 

Vertical axis is bits per pixel (bpp). Horizontal axis is LZMA compression level.

The optimal compression level for LZMA appears to be 5 in terms of cost/performance at 2.37bpp.  Likely due to the use of the normal compression algorithm vs the fast algorithm on levels 0-4 (which gives 2.46bpp). The improvement of level 5 over levels 0-4 is about 3.66%. I'll take it.

LZMA has some knobs that you can turn to further tweak performance.

• The dictionary size in bytes
  
• lc - The number of literal context bits (high bits of previous literal).
  
• lp - The number of literal pos bits (low bits of current position for literals).
  
• pb - The number of pos bits (low bits of current position).
  
• fb - Word size (the number of fast bytes).
  

The LZMA dictionary size appears to have no effect on the compression ratios.

Setting lp to 4 (16 bytes, the size of a DXT5 block) and lc to 0 further improves compression performance to 6.5% (over level 0's 2.46bpp) to 2.30bpp.

Setting pb to 4 (16 bytes, the size of a DXT5 block) also improves compression performance to 7.32% (over level 0's 2.46bpp) to 2.28bpp.

The fb setting appears to have no effect on compression ratios. 

So, for just straight up LZMA compression, we are talking about a compression ratio of 14:1 vs 4:1 for straight DXT5. 

Next up is various data transposes and optimal LZMA settings for that.