10-14-10 - Image Comparison Part 3 - JPEG vs AIC

Testing Bilsen's AIC (AIC is a subset of H264 Intra without the good encoder of x264) :

Bilsen's AIC doesn't have the crippling low quality colorspace problem of x264, but JPEG just kills it on both metrics. Note that I use jpegflatnosub for rmse and jpeg default options for the perceptual metric.

I think we've already debunked the claims that JPEG is "easy to beat" or "not competitive with modern codecs" or that the "H264 Intra Predictors are a big advantage". (granted AIC is not the best modern codec by a long shot).

I should fill in some more details before I go further.

All the tests so far have been on one image, I made it with my camera by taking a RAW photo and scaling & cropping it down from 4000x3000 down to 1920x1200 to reduce noise and improve chroma resolution. The image is called "my_soup" (maybe I'll post it somewhere for download). I will at some point run some tests on a bunch of images, because it's a bad idea to test on just one.

As I said before, the JPEG I'm using is just IJG , but I am losslessly recompressing the JPEGs with PAQ. I also tried the old JPEG -arith , and I found it's about half way between jpeg-huff and jpeg-PAQ, so I believe this is roughly a fair way of making the JPEG entropy coder back end "modern". I haven't really tried to optimize the JPEG encoding at all, for example there might be better quant matrices, or better options to give to IJG, and obviously you could easily add an unblock on the outside, etc. Without any of that stuff, JPEG is already competitive.

I should also take this chance to state the caveat : MS-SSIM-SCIELAB is in no way a proof of visual superiority. It's the best analytic metric I have handy that is pretty close to visual quality, but the only test we have for real visual quality at the moment is to look at the output with your own eyes.

The jpeg results are made like this :

jpegtest.bat :

c:\util\jpeg8b\cjpeg -dct float -optimize -quality %2 -outfile %1.jpg %1
paq8o8 -6 %1.jpg
call d %1.jpg*
c:\util\jpeg8b\djpeg -dct float -bmp -dither none -outfile de.bmp %1.jpg
namebysize de.bmp %1.jpg.paq8o8 jpeg_test_ .bmp

jpegtests.bat :

call jpegtest %1 5
call jpegtest %1 10
call jpegtest %1 15
call jpegtest %1 20
call jpegtest %1 25
call jpegtest %1 30
call jpegtest %1 40
call jpegtest %1 50
call jpegtest %1 60
call jpegtest %1 65
call jpegtest %1 70
call jpegtest %1 75
call jpegtest %1 80
call jpegtest %1 85
call jpegtest %1 90
call jpegtest %1 95
call jpegtest %1 100
call dele jpg_test\*
call mov jpeg_test_* jpg_test\
imdiff %1 jpg_test -c
call zr imdiff.csv jpg_imdiff.csv
transposecsv jpg_imdiff.csv jpg_trans.csv