Fabian's BitKnit is coming to Oodle. BitKnit is a pretty unique LZ; it makes clever use of the properties of RANS to hit a space-speed tradeoff point that nothing else does. It gets close to LZMA compression levels (sometimes more, sometimes less) while being more like zlib speed.
LZNA and LZNIB are also much improved. The bit streams are the same, but we found some little tweaks in the encoders & decoders that make significant difference. (5-10%, but that's a lot in compression, and they were already world-beating, so the margin is just bigger now). The biggest improvement came from some subtle issues in the parsers.
As usual, I'm trying to be as fair as possible to the competition. Everything is run single threaded. LZMA and LZHAM are run at max compression with context bits at their best setting. Compressors like zlib that are just not even worth considering are not included, I've tried to include the strongest competition that I know of now. This is my test of "slowies" , that is, all compressors set at high (not max) compression levels. ("oohc" is Oodle Optimal1 , my compression actually goes up quite a bit at higher levels, but I consider anything below 2 mb/s to encode to be just too slow to even consider).
|
|
|
|
The raw data : ("game test set")
by ratio:
oohcLZNA : 2.88:1 , 5.3 enc mb/s , 135.0 dec mb/s
lzma : 2.82:1 , 2.9 enc mb/s , 43.0 dec mb/s
oohcBitKnit : 2.76:1 , 6.4 enc mb/s , 273.3 dec mb/s
lzham : 2.59:1 , 1.8 enc mb/s , 162.9 dec mb/s
oohcLZHLW : 2.38:1 , 4.2 enc mb/s , 456.3 dec mb/s
zstdhc9 : 2.11:1 , 29.5 enc mb/s , 558.0 dec mb/s
oohcLZNIB : 2.04:1 , 11.5 enc mb/s , 1316.4 dec mb/s
by encode speed:
zstdhc9 : 2.11:1 , 29.5 enc mb/s , 558.0 dec mb/s
oohcLZNIB : 2.04:1 , 11.5 enc mb/s , 1316.4 dec mb/s
oohcBitKnit : 2.76:1 , 6.4 enc mb/s , 273.3 dec mb/s
oohcLZNA : 2.88:1 , 5.3 enc mb/s , 135.0 dec mb/s
oohcLZHLW : 2.38:1 , 4.2 enc mb/s , 456.3 dec mb/s
lzma : 2.82:1 , 2.9 enc mb/s , 43.0 dec mb/s
lzham : 2.59:1 , 1.8 enc mb/s , 162.9 dec mb/s
by decode speed:
oohcLZNIB : 2.04:1 , 11.5 enc mb/s , 1316.4 dec mb/s
zstdhc9 : 2.11:1 , 29.5 enc mb/s , 558.0 dec mb/s
oohcLZHLW : 2.38:1 , 4.2 enc mb/s , 456.3 dec mb/s
oohcBitKnit : 2.76:1 , 6.4 enc mb/s , 273.3 dec mb/s
lzham : 2.59:1 , 1.8 enc mb/s , 162.9 dec mb/s
oohcLZNA : 2.88:1 , 5.3 enc mb/s , 135.0 dec mb/s
lzma : 2.82:1 , 2.9 enc mb/s , 43.0 dec mb/s
-----------------------------------------------------------------
Log opened : Fri Dec 18 17:56:44 2015
total : oohcLZNIB : 167,495,105 ->81,928,287 = 3.913 bpb = 2.044 to 1
total : encode : 14.521 seconds, 3.39 b/kc, rate= 11.53 M/s
total : decode : 0.127 seconds, 386.85 b/kc, rate= 1316.44 M/s
total : encode+decode : 14.648 seconds, 3.36 b/kc, rate= 11.43 M/s
total : oohcLZHLW : 167,495,105 ->70,449,624 = 3.365 bpb = 2.378 to 1
total : encode : 40.294 seconds, 1.22 b/kc, rate= 4.16 M/s
total : decode : 0.367 seconds, 134.10 b/kc, rate= 456.33 M/s
total : encode+decode : 40.661 seconds, 1.21 b/kc, rate= 4.12 M/s
total : oohcLZNA : 167,495,105 ->58,242,995 = 2.782 bpb = 2.876 to 1
total : encode : 31.867 seconds, 1.54 b/kc, rate= 5.26 M/s
total : decode : 1.240 seconds, 39.68 b/kc, rate= 135.04 M/s
total : encode+decode : 33.107 seconds, 1.49 b/kc, rate= 5.06 M/s
total : oohcBitKnit : 167,495,105 ->60,763,350 = 2.902 bpb = 2.757 to 1
total : encode : 26.102 seconds, 1.89 b/kc, rate= 6.42 M/s
total : decode : 0.613 seconds, 80.33 b/kc, rate= 273.35 M/s
total : encode+decode : 26.714 seconds, 1.84 b/kc, rate= 6.27 M/s
total : zstdhc9 : 167,495,105 ->79,540,333 = 3.799 bpb = 2.106 to 1
total : encode : 5.671 seconds, 8.68 b/kc, rate= 29.53 M/s
total : decode : 0.300 seconds, 163.98 b/kc, rate= 558.04 M/s
total : encode+decode : 5.971 seconds, 8.24 b/kc, rate= 28.05 M/s
total : lzham : 167,495,105 ->64,682,721 = 3.089 bpb = 2.589 to 1
total : encode : 93.182 seconds, 0.53 b/kc, rate= 1.80 M/s
total : decode : 1.028 seconds, 47.86 b/kc, rate= 162.86 M/s
total : encode+decode : 94.211 seconds, 0.52 b/kc, rate= 1.78 M/s
total : lzma : 167,495,105 ->59,300,023 = 2.832 bpb = 2.825 to 1
total : encode : 57.712 seconds, 0.85 b/kc, rate= 2.90 M/s
total : decode : 3.898 seconds, 12.63 b/kc, rate= 42.97 M/s
total : encode+decode : 61.610 seconds, 0.80 b/kc, rate= 2.72 M/s
-------------------------------------------------------
