In this test we use common test files to provide additional benchmark data for all compressors including those that did not qualify for the main benchmark.
| Program | Ver | Arguments | WR |
|---|---|---|---|
| paq | 8p | -5 | 9.786 |
| lpaq | 8 | 7 | 8.334 |
| zpaq | 1.00 | cmax.cfg | 7.918 |
| ppmonstr | Jr1 | -m600 -o16 | 7.443 |
| ash | 0.7b1 | /m512 | 7.387 |
| ccm | 1.30c | x 5 | 7.370 |
| epm | r9 | 7.365 | |
| cmm4 | 0.1e | 75 | 7.348 |
| bit | 0.7 | -p=4 | 7.230 |
| durilca | 0.5 | -m384 -o8 | 7.121 |
| nanozip | 0.07a | -m.5g -cc | 7.015 |
| bee | 0.79 | -m3 -d7 | 6.791 |
| bwmonstr | 0.02 | 6.599 | |
| rzm | 0.07h | 6.593 | |
| bwtmix | 1s | c1150 | 6.437 |
| enc | 0.15 | 6.399 | |
| m1 | x2-0.6 | 6 enwik7.txt | 6.356 |
| blizzard | 0.24b | c 100000000 | 6.294 |
| m03 | 0.2a1 | 100000000 | 6.288 |
| rings | 1.5c | 5 | 6.282 |
| grzipii | 0.24 | -p -b8m -m1 | 6.240 |
| bcm | 0.09 | -b118 | 6.233 |
| ppmd | sh8 | /m590 /o16 | 6.222 |
| bbb | 1 | cfm100q | 6.196 |
| bma | 1.35b | -mx -m52m | 6.194 |
| bsc | 2.2.0 | -p -m3 | 6.190 |
| mnzip | 0 | 5 | 6.174 |
| dark | 0.51 | p-b118m | 6.130 |
| paf | 03a | -r1 | 6.114 |
| lzpxj | 1.2h | e | 6.103 |
All compressors that pass the qualification without an error are tested here with known test files to provide additional benchmark data. Another reason to run benchmarks with these files is that some experts may prefer to see the results for these files despite the problems that arise using them.
The files used here are often part of collections (like Calgary Corpus) that sample various types of data attempting to providing broad evaluation over a range of data types. These test files are flawed for various reasons that include:
Using these files may work for developing a compressor (evaluating changes to algorithms), but the files suit poorly for comparing 2 or more compressors. We can observe that the main Compression Ratings benchmark by large does not suffer from any of these points.
To address the point 4. We have not included some files that fit into this category especially. Because of 1. it follows that the measured timings are skewed for programs that slow down as the size of input grows and compression ratio is difficult to measure (also because of 3.) as the decoder program size takes large portition of the output size. To address this, we do not count the size of decompressor program in this test (the size column lists the raw compressed file size) and we introduce "weighted compression ratio" (WR) [2] that is compression ratio with decompressor size counted in, except that we discount 90% of the decompressor size, but no more than 85000 bytes (roughly 90% of the median uncompressed decompression program size). The 90% discount is an arbitrary number to limit random effect from including the decompressor size to the results for small files. We limit the discount to the median because the purpose is to attempt lessen this effect, but not to eliminate the decompressor size (such as static dictionaries) completely. No ratings are provided in these tests because of the results require interpretation even to establish compressor performance for these specific files. (We cannot make conclusions about compressors strenght over certain types of files because of 2.)
[1] Files are too special (performance doesn't translate to other data) that small tweaks to a program may improve compression ratio dramatically (but have no effect in other files). Or files that are already compressed or contain errors (like book1).
[2] WR=sizeof(input)/(sizeof(output)+sizeof(decompressor)-0.9*min(95kb,sizeof(decompressor)))