In this test we compare Burrows-Wheeler based compressors. We especially try to compare such compressors that do not use preprocessing.
We define BWT compressor to be a program that applies BWT transformation before compression. When decompressing the program applies inverse BWT transformation after decompressing the data.
We define BWT compressor without preprocessing to be such that it does not apply any kind of transformation to the data before the context sorting phase of Burrows-Wheeler transform.
For this test we include compressors that do not qualify for the main benchmark and run compressors with only a single configuration for maximum compression. It follows that some programs have faster modes of operation. See the main benchmarks for exhaustive comparison of such modes. We do not assign ratings and no size for decoder program is considered. Authors are welcome to write feedback if they find their programs are missing or tested incorrectly.
We have no decisive evidence whether a program uses preprocessing or not without the source code. Hence we must use undisclosed arbitrary qualifying methods to choose the set of programs for this test.
Brief comments about the programs which are considered for the test.
dc 0.98b, bssc 0.95a, ba 1.01b, bma 1.35b, ybs 0.3f |
Closed source programs that use preprocessing. We try disabling the preprocessing by using the provided switches. |
dark 0.51, grzipii 0.24, bsc 2.2.0, szip 1.12, ecp 0.e.s |
The programs are open source and have switches to disable preprocessing. |
imp 1.12 |
The program's decoder is open source. The program does not use preprocessing but it switches from BWT to LZ if BWT sorting is problematic. This probably happens in this test only with the enwik9. |
zzip 0.36c, bzip 0.21, bzip2 1.0.5, mnzip 0, grzip 0.7.3, bred 3 |
Programs are open source and use preprocessing. ZZIP has a maximum blocksize of 15 MB. |
rings 1.5c |
It appears the program is using ST, not BWT. The program apparently uses preprocessing that cannot be disabled. |
blizzard 0.24b, sbc 0.970r3, gca 0.9k |
Closed source programs that are very likely using preprocessing. No switches are provided to disable it. |
m99 2.21, m03 0.2a1, uhbc 1.0, bcm 0.11, mcomp 2.00, m03exp 5.2, bwmonstr 0.02, bwic 0, x1 0.95a, qlfc 6.6 |
It is likely that preprocessing is not used in these programs. |
bwtzip 0, chile 0.5, abc 2.4, bwc 0.99d, bbb 1, mar 0, bwtmix 1 |
Programs are open source and are not using preprocessing. |
The following open source programs are not using preprocessing: bbb, bwtzip, chile, abc, bwc, dark, bwmmix. Because we do not wish to discard all other compressors, we include them anyway with hopes that they do not use heavy preprocessing for the test files. We discard rings and sbc since they do not fit well to the concept of BWT without preprocessing. We would have reasons to discard other compressors such as blizzard, imp, zzip, but we hope their preprocessing is rather inefficient for our test files and we include them anyway.
The programs are set up with maximum memory setting that works in the test machine. For bma the following switches are used in addition to the ones listed in the tables: '-rv- -pe- -t- -d- -l- -z- -tx-'. We run bma with -sif0 because otherwise the program attempts to choose the best compression method that may lead of confusing results especially with the transformation test.
Update 2010 spring: some of the very old compressors no longer run on the test platform: ba, x1, uhbc, m03exp.
Program | Version | Space for a block |
---|---|---|
bwmonstr | 0.02 | ~0.6n |
bbb | 1 | 1.25n (+4n) |
bwtmix | 1 | 5n |
bcm | 0.11 | 5n |
bsc | 2.2.0 | 5n |
blizzard | 0.24b | 6n |
m03 | 0.2a1 | 6n |
uhbc | 1.0 | 8n |
dark | 0.51 | 5n |
m99 | 2.21 | 6n |
qlfc | 6.6 | 6n |
m03exp | 5.2 | 12n? |
chile | 0.5 | 7n |
bma | 1.35b | 12n |
bssc | 0.95a | 9n |
ybs | 0.3f | 9n |
bred | 3 | 5n |
grzipii | 0.24 | 10n |
zzip | 0.36c | 12n |
grzip | 0.7.3 | 10n? |
abc | 2.4 | 10n? |
bwc | 0.99d | 8n |
bwtzip | 0 | ~50n |
ecp | 0.e.s | 5n? |
szip | 1.12 | 5n |
gca | 0.9k | ? |
bzip | 0.21 | 8n? |
bzip2 | 1.0.5 | 8n |
imp | 1.12 | 5n |
x1 | 0.95a | 5n? |
Most programs use 5 to 10n blocksize memory. bbb 1 and bwmonstr 0.02 work with less than 5n memory. bbb uses 1.25n blocksize memory, but requires 4n space for temporary files and runs out of disk space on the test platform with 500MB block, so we use 334MB block instead. bwmonstr uses less than 0.6n blocksize memory. The table beside gives an estimated memory requirements for each compressor compressing enwik9.
We will use special test files for this test: enwik9 for a large test file. A smaller 5 000 000 byte sample of enwik9 to attempt get as many programs as possible to compress the file using a single block. We also add book1 and obj2 (from the Calgary Corpus) and a sample of Chinese Wikipedia (zhwik8). Finally a random file DejaVuSans.ttf from Application1.
Program | Ver | Arguments | Size | % | C.Time | % | D.Time | % | C.M | D.M |
---|---|---|---|---|---|---|---|---|---|---|
bwmonstr | 0.02 | 160 468 597 | 16.0 | 63790.09 | 390 | 53012.55 | 322 | 592 | 587 | |
bbb | 1 | cb333333334 | 172 179 824 | 17.2 | 2201.36 | 100 | 1380.86 | 100 | 446 | 406 |
bwtmix | 1 | c1250 | 178 510 043 | 17.9 | 1924.28 | 100 | 824.00 | 100 | 618 | 618 |
bsc | 2.4.5 | -pt -m3 -b120 | 179 910 090 | 18.0 | 149.22 | 174 | 45.91 | 308 | 605 | 605 |
bcm | 0.11 | -b118 | 180 484 787 | 18.0 | 458.06 | 100 | 331.92 | 100 | 594 | 594 |
m03 | 0.2a1 | 100000000 | 180 893 111 | 18.1 | 526.25 | 100 | 394.20 | 100 | 579 | 576 |
blizzard | 0.24b | c 100000000 | 182 780 264 | 18.3 | 377.67 | 100 | 296.12 | 100 | 575 | 574 |
mcomp | 2.00 | -M112M -mw | 183 146 631 | 18.3 | 257.80 | 100 | 241.66 | 100 | 566 | 565 |
dark | 0.51 | p-b112mrf | 185 516 042 | 18.6 | 242.20 | 100 | 275.56 | 100 | 564 | 564 |
m99 | 2.21 | -m 96m | 186 635 795 | 18.7 | 365.38 | 100 | 303.12 | 100 | 591 | 581 |
qlfc | 6.6 | 100000000 | 190 040 642 | 19.0 | 527.52 | 100 | 201.98 | 100 | 576 | 576 |
bma | 1.35b | -sif0 -mx -m52m | 195 271 574 | 19.5 | 711.22 | 100 | 211.06 | 100 | 595 | 379 |
bssc | 0.95a | -b16383aeflz | 201 678 505 | 20.2 | 291.48 | 100 | 120.73 | 100 | 139 | 82 |
ybs | 0.3f | -op -m16m | 202 110 510 | 20.2 | 314.88 | 100 | 134.61 | 100 | 146 | 82 |
bred | 3 | -w0 -M100 -m729 | 204 611 934 | 20.5 | 366.89 | 100 | 174.53 | 100 | 480 | 480 |
grzipii | 0.24 | -m1 -b8m -p | 207 894 667 | 20.8 | 204.75 | 100 | 151.22 | 100 | 84 | 44 |
zzip | 0.36c | -mx -15m | 211 182 117 | 21.1 | 349.67 | 100 | 191.83 | 100 | 183 | 79 |
grzip | 0.7.3 | 211 659 211 | 21.2 | 230.25 | 100 | 157.80 | 100 | 81 | 41 | |
abc | 2.4 | 213 717 700 | 21.4 | 279.34 | 100 | 264.50 | 100 | 46 | 57 | |
bwtzip | 0 | 12000000 | 220 678 813 | 22.1 | 1594.38 | 100 | 271.45 | 100 | 578 | 71 |
ecp | 0.e.s | -b127 | 226 202 000 | 22.6 | 820.17 | 100 | 183.81 | 100 | 64 | 76 |
gca | 0.9k | 240 007 964 | 24.0 | 209.70 | 100 | 95.81 | 100 | 23 | 10 | |
bzip | 0.21 | 251 879 486 | 25.2 | 358.64 | 100 | 135.36 | 100 | 9 | 7 | |
bzip2 | 1.0.5 | 253 977 891 | 25.4 | 126.31 | 100 | 49.05 | 100 | 8 | 5 | |
imp | 1.12 | -2 | 257 928 866 | 25.8 | 108.34 | 99 | 25.91 | 100 | 7 | 7 |
chile | 0.5 | -b=90000 | disqualified: verify error | |||||||
mar | 0 | -a bwt | disqualified: verify error | |||||||
mnzip | 0 | 5 | disqualified: verify error | |||||||
dc | 0.98b | -cprleab15200 | disqualified: verify error |
Program | Ver | Arguments | Size | % | C.Time | % | D.Time | % | C.M | D.M |
---|---|---|---|---|---|---|---|---|---|---|
bwmonstr | 0.02 | 1 169 155 | 23.4 | 306.09 | 389 | 279.05 | 316 | 33 | 33 | |
bwtmix | 1 | c1250 | 1 193 190 | 23.9 | 6.34 | 100 | 4.01 | 100 | 44 | 44 |
m03 | 0.2a1 | 100000000 | 1 194 946 | 23.9 | 2.42 | 100 | 1.89 | 100 | 31 | 30 |
bcm | 0.11 | -b118 | 1 203 330 | 24.1 | 1.69 | 99 | 1.44 | 100 | 26 | 26 |
bsc | 2.4.5 | -pt -m3 -b120 | 1 207 968 | 24.2 | 0.48 | 274 | 0.23 | 400 | 26 | 26 |
blizzard | 0.24b | c 100000000 | 1 217 457 | 24.3 | 1.25 | 100 | 1.20 | 100 | 31 | 31 |
bbb | 1 | cb333333334 | 1 219 528 | 24.4 | 8.14 | 100 | 6.25 | 100 | 328 | 13 |
grzipii | 0.24 | -m1 -b8m -p | 1 221 219 | 24.4 | 0.84 | 100 | 0.78 | 100 | 38 | 27 |
abc | 2.4 | 1 224 006 | 24.5 | 1.39 | 100 | 1.48 | 100 | 45 | 54 | |
ybs | 0.3f | -op -m16m | 1 226 146 | 24.5 | 1.26 | 100 | 0.66 | 100 | 45 | 25 |
mcomp | 2.00 | -M112M -mw | 1 229 820 | 24.6 | 1.03 | 100 | 1.03 | 100 | 27 | 27 |
bssc | 0.95a | -b16383aeflz | 1 230 195 | 24.6 | 1.26 | 100 | 0.55 | 100 | 56 | 31 |
mnzip | 0 | 5 | 1 230 994 | 24.6 | 1.94 | 100 | 0.55 | 100 | 77 | 29 |
dc | 0.98b | -cprleab15200 | 1 235 265 | 24.7 | 0.80 | 98 | 0.80 | 98 | 32 | 26 |
grzip | 0.7.3 | 1 236 193 | 24.7 | 1.19 | 100 | 0.84 | 100 | 52 | 25 | |
dark | 0.51 | p-b112mrf | 1 241 866 | 24.8 | 0.80 | 100 | 1.06 | 100 | 26 | 26 |
m99 | 2.21 | -m 96m | 1 244 333 | 24.9 | 1.70 | 100 | 1.50 | 100 | 40 | 32 |
qlfc | 6.6 | 100000000 | 1 259 286 | 25.2 | 1.30 | 100 | 0.78 | 100 | 31 | 30 |
bma | 1.35b | -sif0 -mx -m52m | 1 263 225 | 25.3 | 2.03 | 100 | 0.91 | 100 | 55 | 32 |
chile | 0.5 | -b=90000 | 1 268 941 | 25.4 | 24.42 | 100 | 23.03 | 100 | 119 | 31 |
zzip | 0.36c | -mx -15m | 1 275 293 | 25.5 | 1.39 | 100 | 0.94 | 100 | 59 | 26 |
bwtzip | 0 | 12000000 | 1 316 961 | 26.3 | 5.41 | 100 | 1.36 | 100 | 231 | 31 |
bred | 3 | -w0 -M100 -m729 | 1 336 870 | 26.7 | 0.91 | 100 | 0.61 | 100 | 26 | 25 |
mar | 0 | -a bwt | 1 350 357 | 27.0 | 4.73 | 100 | 0.66 | 100 | 35 | 30 |
ecp | 0.e.s | -b127 | 1 352 985 | 27.1 | 1.88 | 100 | 0.91 | 100 | 26 | 30 |
gca | 0.9k | 1 363 546 | 27.3 | 0.94 | 98 | 0.52 | 97 | 16 | 10 | |
bzip | 0.21 | 1 442 558 | 28.9 | 1.83 | 100 | 0.70 | 100 | 9 | 7 | |
bzip2 | 1.0.5 | 1 454 779 | 29.1 | 0.58 | 100 | 0.25 | 100 | 8 | 5 | |
imp | 1.12 | -2 | 1 459 323 | 29.2 | 0.45 | 100 | 0.16 | 100 | 7 | 7 |
Program | Ver | Arguments | Size | % | C.Time | % | D.Time | % | C.M | D.M |
---|---|---|---|---|---|---|---|---|---|---|
bwmonstr | 0.02 | 21 834 050 | 21.8 | 6121.00 | 390 | 5635.95 | 316 | 94 | 94 | |
bwtmix | 1 | c1250 | 22 458 892 | 22.5 | 171.70 | 100 | 83.50 | 100 | 499 | 499 |
m03 | 0.2a1 | 100000000 | 22 521 598 | 22.5 | 51.47 | 100 | 41.64 | 100 | 577 | 576 |
bsc | 2.4.5 | -pt -m3 -b120 | 22 567 048 | 22.6 | 13.12 | 191 | 4.89 | 321 | 481 | 481 |
bbb | 1 | cb333333334 | 22 743 533 | 22.7 | 184.24 | 100 | 139.44 | 100 | 362 | 127 |
bcm | 0.11 | -b118 | 22 751 408 | 22.8 | 43.52 | 100 | 32.38 | 100 | 481 | 481 |
blizzard | 0.24b | c 100000000 | 22 945 129 | 22.9 | 35.09 | 100 | 29.17 | 100 | 574 | 572 |
mcomp | 2.00 | -M112M -mw | 22 957 472 | 23.0 | 24.22 | 100 | 22.64 | 100 | 482 | 482 |
m99 | 2.21 | -m 96m | 23 463 768 | 23.5 | 36.78 | 100 | 32.88 | 100 | 586 | 577 |
dark | 0.51 | p-b112mrf | 23 487 459 | 23.5 | 23.20 | 100 | 26.38 | 100 | 481 | 480 |
qlfc | 6.6 | 100000000 | 23 604 163 | 23.6 | 39.58 | 100 | 19.50 | 100 | 575 | 575 |
chile | 0.5 | -b=90000 | 24 176 109 | 24.2 | 519.12 | 100 | 467.34 | 100 | 619 | 531 |
bma | 1.35b | -sif0 -mx -m52m | 24 731 676 | 24.7 | 54.86 | 100 | 23.39 | 100 | 583 | 328 |
bssc | 0.95a | -b16383aeflz | 25 166 003 | 25.2 | 25.89 | 100 | 11.67 | 100 | 139 | 82 |
ybs | 0.3f | -op -m16m | 25 502 206 | 25.5 | 26.50 | 100 | 13.33 | 100 | 146 | 82 |
bred | 3 | -w0 -M100 -m729 | 25 570 771 | 25.6 | 29.12 | 100 | 16.08 | 100 | 480 | 480 |
dc | 0.98b | -cprleab15200 | 25 787 905 | 25.8 | 19.55 | 100 | 15.92 | 99 | 93 | 77 |
grzipii | 0.24 | -m1 -b8m -p | 25 975 623 | 26.0 | 19.25 | 100 | 16.66 | 100 | 60 | 44 |
grzip | 0.7.3 | 26 210 904 | 26.2 | 23.44 | 100 | 17.62 | 100 | 79 | 40 | |
zzip | 0.36c | -mx -15m | 26 312 987 | 26.3 | 28.03 | 100 | 19.17 | 100 | 92 | 77 |
abc | 2.4 | 26 804 047 | 26.8 | 26.23 | 100 | 27.89 | 100 | 46 | 56 | |
bwtzip | 0 | 12000000 | 27 498 056 | 27.5 | 149.19 | 100 | 27.66 | 100 | 548 | 71 |
ecp | 0.e.s | -b127 | 27 745 648 | 27.7 | 43.95 | 100 | 18.19 | 100 | 64 | 76 |
gca | 0.9k | 29 536 149 | 29.5 | 18.86 | 100 | 9.91 | 99 | 23 | 10 | |
bzip | 0.21 | 30 254 459 | 30.3 | 34.05 | 100 | 14.12 | 100 | 9 | 7 | |
imp | 1.12 | -2 | 30 507 730 | 30.5 | 8.33 | 99 | 3.00 | 97 | 7 | 7 |
bzip2 | 1.0.5 | 30 541 167 | 30.5 | 11.20 | 100 | 5.05 | 100 | 8 | 5 | |
mar | 0 | -a bwt | disqualified: verify error | |||||||
mnzip | 0 | 5 | disqualified: verify error |
Program | Ver | Arguments | Size | % | C.Time | % | D.Time | % | C.M | D.M |
---|---|---|---|---|---|---|---|---|---|---|
bwmonstr | 0.02 | 204 844 | 26.6 | 37.61 | 384 | 27.95 | 345 | 31 | 31 | |
bwtmix | 1 | c1250 | 208 208 | 27.1 | 0.88 | 100 | 0.61 | 100 | 24 | 24 |
bcm | 0.11 | -b118 | 209 771 | 27.3 | 0.20 | 100 | 0.19 | 99 | 6 | 6 |
m03 | 0.2a1 | 100000000 | 210 483 | 27.4 | 0.36 | 100 | 0.27 | 100 | 7 | 6 |
blizzard | 0.24b | c 100000000 | 212 130 | 27.6 | 0.16 | 100 | 0.12 | 100 | 7 | 6 |
bsc | 2.4.5 | -pt -m3 -b120 | 213 118 | 27.7 | 0.06 | 402 | 0.03 | 403 | 17 | 17 |
ybs | 0.3f | -op -m16m | 213 162 | 27.7 | 0.11 | 100 | 0.06 | 100 | 8 | 5 |
bbb | 1 | cb333333334 | 213 162 | 27.7 | 1.24 | 100 | 0.73 | 100 | 327 | 8 |
abc | 2.4 | 213 952 | 27.8 | 0.20 | 100 | 0.17 | 100 | 20 | 22 | |
bssc | 0.95a | -b16383aeflz | 214 618 | 27.9 | 0.16 | 99 | 0.05 | 98 | 37 | 6 |
dc | 0.98b | -cprleab15200 | 215 353 | 28.0 | 0.11 | 85 | 0.11 | 85 | 7 | 6 |
grzipii | 0.24 | -m1 -b8m -p | 215 377 | 28.0 | 0.11 | 100 | 0.09 | 100 | 14 | 5 |
dark | 0.51 | p-b112mrf | 215 512 | 28.0 | 0.11 | 100 | 0.11 | 99 | 6 | 5 |
mnzip | 0 | 5 | 216 572 | 28.2 | 0.19 | 100 | 0.05 | 100 | 41 | 6 |
m99 | 2.21 | -m 96m | 217 267 | 28.3 | 0.25 | 100 | 0.20 | 100 | 15 | 7 |
mcomp | 2.00 | -M112M -mw | 217 403 | 28.3 | 0.16 | 100 | 0.14 | 89 | 7 | 7 |
gca | 0.9k | 217 757 | 28.3 | 0.14 | 89 | 0.09 | 83 | 12 | 8 | |
bma | 1.35b | -sif0 -mx -m52m | 219 094 | 28.5 | 0.20 | 100 | 0.09 | 99 | 10 | 7 |
qlfc | 6.6 | 100000000 | 219 108 | 28.5 | 0.16 | 100 | 0.06 | 100 | 6 | 6 |
grzip | 0.7.3 | 219 325 | 28.5 | 0.16 | 99 | 0.11 | 99 | 16 | 5 | |
zzip | 0.36c | -mx -15m | 220 746 | 28.7 | 0.17 | 99 | 0.11 | 99 | 10 | 5 |
chile | 0.5 | -b=90000 | 222 513 | 28.9 | 3.67 | 100 | 3.48 | 100 | 94 | 6 |
bwtzip | 0 | 12000000 | 228 395 | 29.7 | 0.78 | 100 | 0.17 | 100 | 37 | 6 |
bzip | 0.21 | 230 247 | 30.0 | 0.30 | 100 | 0.11 | 98 | 8 | 6 | |
bzip2 | 1.0.5 | 232 598 | 30.3 | 0.09 | 100 | 0.03 | 97 | 7 | 5 | |
bred | 3 | -w0 -M100 -m729 | 233 224 | 30.3 | 0.11 | 99 | 0.05 | 100 | 5 | 5 |
mar | 0 | -a bwt | 234 435 | 30.5 | 1.05 | 100 | 0.05 | 98 | 7 | 6 |
imp | 1.12 | -2 | 235 075 | 30.6 | 0.08 | 99 | 0.01 | 100 | 6 | 6 |
ecp | 0.e.s | -b127 | 239 258 | 31.1 | 0.22 | 100 | 0.09 | 100 | 6 | 6 |
Program | Ver | Arguments | Size | % | C.Time | % | D.Time | % | C.M | D.M |
---|---|---|---|---|---|---|---|---|---|---|
bwmonstr | 0.02 | 65 706 | 26.6 | 15.73 | 385 | 17.00 | 299 | 31 | 30 | |
m03 | 0.2a1 | 100000000 | 70 849 | 28.7 | 0.12 | 86 | 0.09 | 100 | 5 | 3 |
bwtmix | 1 | c1250 | 71 700 | 29.1 | 0.28 | 100 | 0.20 | 100 | 21 | 21 |
grzipii | 0.24 | -m1 -b8m -p | 72 350 | 29.3 | 0.03 | 100 | 0.03 | 100 | 11 | 3 |
grzip | 0.7.3 | 72 616 | 29.4 | 0.05 | 98 | 0.03 | 100 | 12 | 2 | |
bcm | 0.11 | -b118 | 72 947 | 29.6 | 0.06 | 100 | 0.05 | 98 | 3 | 3 |
abc | 2.4 | 72 964 | 29.6 | 0.16 | 100 | 0.14 | 100 | 10 | 9 | |
mnzip | 0 | 5 | 73 196 | 29.7 | 0.06 | 100 | 0.01 | 100 | 36 | 3 |
mcomp | 2.00 | -M112M -mw | 73 496 | 29.8 | 0.05 | 98 | 0.05 | 66 | 5 | 4 |
bssc | 0.95a | -b16383aeflz | 73 798 | 29.9 | 0.05 | 98 | 0.01 | 100 | 19 | 3 |
bsc | 2.4.5 | -pt -m3 -b120 | 73 921 | 30.0 | 0.03 | 352 | 0.01 | 413 | 16 | 16 |
blizzard | 0.24b | c 100000000 | 74 449 | 30.2 | 0.05 | 100 | 0.05 | 100 | 4 | 3 |
gca | 0.9k | 74 510 | 30.2 | 0.05 | 67 | 0.03 | 48 | 5 | 4 | |
qlfc | 6.6 | 100000000 | 74 789 | 30.3 | 0.05 | 100 | 0.01 | 100 | 3 | 3 |
chile | 0.5 | -b=90000 | 75 214 | 30.5 | 1.24 | 100 | 1.12 | 100 | 91 | 3 |
ybs | 0.3f | -op -m16m | 75 410 | 30.6 | 0.03 | 97 | 0.01 | 100 | 4 | 3 |
dc | 0.98b | -cprleab15200 | 75 632 | 30.6 | 0.05 | 66 | 0.05 | 67 | 4 | 4 |
dark | 0.51 | p-b112mrf | 75 851 | 30.7 | 0.03 | 100 | 0.03 | 100 | 3 | 3 |
bzip | 0.21 | 76 017 | 30.8 | 0.08 | 100 | 0.03 | 100 | 4 | 3 | |
zzip | 0.36c | -mx -15m | 76 099 | 30.8 | 0.05 | 100 | 0.03 | 100 | 3 | 3 |
m99 | 2.21 | -m 96m | 76 371 | 30.9 | 0.09 | 100 | 0.08 | 79 | 13 | 4 |
bzip2 | 1.0.5 | 76 441 | 31.0 | 0.01 | 100 | 0.01 | 100 | 3 | 3 | |
bbb | 1 | cb333333334 | 76 527 | 31.0 | 0.45 | 100 | 0.33 | 100 | 327 | 8 |
ecp | 0.e.s | -b127 | 77 295 | 31.3 | 0.06 | 100 | 0.03 | 100 | 3 | 3 |
mar | 0 | -a bwt | 77 539 | 31.4 | 4.31 | 100 | 0.01 | 100 | 4 | 3 |
bred | 3 | -w0 -M100 -m729 | 77 692 | 31.5 | 0.03 | 100 | 0.01 | 100 | 3 | 3 |
imp | 1.12 | -2 | 77 949 | 31.6 | 0.01 | 100 | 0.00 | 0 | 3 | 3 |
bma | 1.35b | -sif0 -mx -m52m | 78 116 | 31.6 | 0.06 | 98 | 0.03 | 100 | 5 | 4 |
bwtzip | 0 | 12000000 | 80 133 | 32.5 | 0.22 | 100 | 0.06 | 98 | 13 | 3 |
Program | Ver | Arguments | Size | % | C.Time | % | D.Time | % | C.M | D.M |
---|---|---|---|---|---|---|---|---|---|---|
bwmonstr | 0.02 | 275 070 | 47.3 | 38.62 | 387 | 48.50 | 282 | 31 | 31 | |
bwtmix | 1 | c1250 | 295 475 | 50.8 | 0.70 | 100 | 0.52 | 100 | 23 | 23 |
bcm | 0.11 | -b118 | 299 784 | 51.6 | 0.16 | 100 | 0.14 | 100 | 5 | 5 |
m03 | 0.2a1 | 100000000 | 301 098 | 51.8 | 0.38 | 100 | 0.30 | 100 | 7 | 5 |
mcomp | 2.00 | -M112M -mw | 301 461 | 51.9 | 0.12 | 100 | 0.12 | 87 | 6 | 6 |
blizzard | 0.24b | c 100000000 | 302 335 | 52.0 | 0.12 | 87 | 0.11 | 100 | 6 | 5 |
bsc | 2.4.5 | -pt -m3 -b120 | 302 982 | 52.1 | 0.06 | 350 | 0.05 | 396 | 17 | 17 |
gca | 0.9k | 303 161 | 52.1 | 0.12 | 86 | 0.08 | 79 | 10 | 7 | |
grzipii | 0.24 | -m1 -b8m -p | 306 186 | 52.7 | 0.14 | 100 | 0.14 | 100 | 13 | 5 |
bbb | 1 | cb333333334 | 306 847 | 52.8 | 0.97 | 100 | 0.89 | 100 | 327 | 8 |
grzip | 0.7.3 | 307 189 | 52.8 | 0.17 | 99 | 0.16 | 100 | 15 | 4 | |
mnzip | 0 | 5 | 307 194 | 52.8 | 0.17 | 100 | 0.06 | 98 | 39 | 5 |
bssc | 0.95a | -b16383aeflz | 308 721 | 53.1 | 0.12 | 99 | 0.06 | 100 | 21 | 5 |
qlfc | 6.6 | 100000000 | 312 534 | 53.8 | 0.11 | 99 | 0.08 | 99 | 5 | 5 |
abc | 2.4 | 313 813 | 54.0 | 0.19 | 100 | 0.25 | 100 | 19 | 21 | |
ybs | 0.3f | -op -m16m | 314 127 | 54.0 | 0.09 | 100 | 0.08 | 100 | 7 | 4 |
m99 | 2.21 | -m 96m | 314 236 | 54.1 | 0.33 | 100 | 0.28 | 94 | 15 | 6 |
dark | 0.51 | p-b112mrf | 314 266 | 54.1 | 0.11 | 100 | 0.11 | 99 | 5 | 5 |
dc | 0.98b | -cprleab15200 | 314 677 | 54.1 | 0.11 | 85 | 0.11 | 85 | 6 | 5 |
chile | 0.5 | -b=90000 | 316 771 | 54.5 | 2.92 | 100 | 2.77 | 100 | 93 | 5 |
bzip | 0.21 | 320 646 | 55.2 | 0.22 | 100 | 0.11 | 99 | 6 | 5 | |
zzip | 0.36c | -mx -15m | 320 945 | 55.2 | 0.16 | 100 | 0.12 | 100 | 5 | 4 |
bzip2 | 1.0.5 | 322 165 | 55.4 | 0.08 | 99 | 0.03 | 100 | 6 | 4 | |
bma | 1.35b | -sif0 -mx -m52m | 323 545 | 55.7 | 0.14 | 100 | 0.11 | 100 | 8 | 6 |
bred | 3 | -w0 -M100 -m729 | 326 009 | 56.1 | 0.12 | 100 | 0.06 | 100 | 5 | 4 |
imp | 1.12 | -2 | 327 218 | 56.3 | 0.06 | 100 | 0.01 | 100 | 5 | 5 |
mar | 0 | -a bwt | 327 959 | 56.4 | 11.41 | 100 | 0.06 | 100 | 6 | 5 |
ecp | 0.e.s | -b127 | 329 024 | 56.6 | 0.20 | 100 | 0.12 | 100 | 5 | 5 |
bwtzip | 0 | 12000000 | 337 762 | 58.1 | 0.67 | 100 | 0.20 | 100 | 27 | 5 |
Finally we try to get indicators of possible preprocessing and compressor behavior by applying various transformations to the data before compression. The table below lists compressed sizes for book1 with various transformations added. We use red color if a transformation causes the size to grow compared to the book1 result without transformation and green color if the transformation improves compression.
The arguments used for each program are the same as in other tests.
Prog | Ver | book1 | 2 (rev) | 3 (not) | 4 (xor) | 5 (mul) | 6 (reo) | 7 (dc) |
---|---|---|---|---|---|---|---|---|
bwmonstr | 0.02 | 204 844 | 204 966 | 204 183 | 204 361 | 208 456 | 203 476 | 202 298 |
bwtmix | 1 | 208 208 | 208 714 | 208 076 | 208 119 | 214 138 | 207 179 | 209 081 |
bcm | 0.11 | 209 771 | 210 125 | 209 937 | 209 789 | 215 063 | 208 498 | 210 218 |
m03 | 0.2a1 | 210 483 | 210 681 | 210 362 | 210 399 | 212 304 | 209 791 | 207 109 |
blizzard | 0.24b | 212 130 | 212 076 | 211 988 | 212 317 | 217 176 | 211 596 | 212 840 |
bsc | 2.4.5 | 213 118 | 213 934 | 213 131 | 213 113 | 216 969 | 211 633 | 210 124 |
bbb | 1 | 213 162 | 213 134 | 213 123 | 213 121 | 218 558 | 211 629 | 212 937 |
ybs | 0.3f | 213 162 | 214 333 | 213 178 | 213 144 | 217 332 | 211 628 | 211 025 |
abc | 2.4 | 213 952 | 214 785 | 213 970 | 213 942 | 217 641 | 212 356 | 211 218 |
bssc | 0.95a | 214 618 | 215 733 | 214 581 | 214 461 | 218 668 | 212 930 | 212 577 |
dc | 0.98b | 215 353 | 216 517 | 215 410 | 215 300 | 219 460 | 213 744 | 212 725 |
grzipii | 0.24 | 215 377 | 216 271 | 215 241 | 215 139 | 219 331 | 213 621 | 212 296 |
dark | 0.51 | 215 512 | 214 427 | 215 567 | 215 638 | 218 491 | 214 687 | 213 876 |
mnzip | 0 | 216 572 | 217 187 | 216 585 | 216 597 | 220 542 | 214 973 | 212 074 |
m99 | 2.21 | 217 267 | 218 248 | 217 320 | 217 173 | 221 478 | 215 445 | 213 859 |
mcomp | 2.00 | 217 403 | 217 021 | 217 481 | 217 410 | 222 833 | 215 572 | 215 664 |
gca | 0.9k | 217 757 | 217 634 | 217 806 | 217 879 | 224 099 | 216 278 | 222 022 |
bma | 1.35b | 219 094 | 218 416 | 219 144 | 219 124 | 223 561 | 217 241 | 216 698 |
qlfc | 6.6 | 219 108 | 219 815 | 219 052 | 219 065 | 222 757 | 217 454 | 213 853 |
grzip | 0.7.3 | 219 325 | 220 073 | 219 230 | 219 120 | 223 375 | 217 541 | 214 913 |
zzip | 0.36c | 220 746 | 223 401 | 224 048 | 223 576 | 227 697 | 222 359 | 219 045 |
chile | 0.5 | 222 513 | 224 089 | 222 833 | 222 811 | 227 055 | 221 265 | 217 853 |
bwtzip | 0 | 228 395 | 229 679 | 228 289 | 228 359 | 232 383 | 226 803 | 225 882 |
bzip | 0.21 | 230 247 | 231 512 | 230 267 | 230 180 | 233 749 | 228 843 | 224 172 |
bzip2 | 1.0.5 | 232 598 | 234 538 | 232 568 | 232 559 | 235 851 | 231 416 | 225 420 |
bred | 3 | 233 224 | 231 982 | 233 311 | 233 299 | 236 066 | 232 645 | 228 038 |
mar | 0 | 234 435 | 235 166 | 234 558 | 234 371 | 238 855 | 232 998 | 228 514 |
imp | 1.12 | 235 075 | 236 949 | 235 099 | 234 931 | 238 449 | 233 788 | 228 095 |
ecp | 0.e.s | 239 258 | 238 252 | 239 203 | 238 974 | 244 434 | 237 030 | 233 351 |
The results indicate that no compressor use heavy preprocessing for the original data. gca 0.9k, bcm 0.11, bwtmix 1 and blizzard 0.24 do not benefit from dc preprocessing, which is designed to help BWT compression. This may suggest that these programs may have preprocessing. bwtmix does not, but the program has a case insensitive model, which boosts the performance for text data. zzip uses preprocessing, but dc preprocessing works much better for it. This may be the case with some other compressors like ba.
Here we have percentage differences instead of the absolute sizes for the same data. A sum values are provided to show the sum of size differences for all transforms. Here a large negative number is desired. We also add a mean of absolute percentages for alphabet permutation transformations (this excludes the dc preprocessing). A low number means the model does not see how different contexts are arranged. This may be negative or positive depending whether such property desired or not. Finally we list average gain for each transform. The row also shows the mean for "sum" and "mean" columns.
Prog | Ver | 2 (rev) | 3 (not) | 4 (xor) | 5 (mul) | 6 (reo) | 7 (dc) | sum | mean |
---|---|---|---|---|---|---|---|---|---|
bwmonstr | 0.02 | 0.060 % | -0.323 % | -0.236 % | 1.763 % | -0.668 % | -1.243 % | -0.108 % | 0.610 % |
bwtmix | 1 | 0.243 % | -0.063 % | -0.043 % | 2.848 % | -0.494 % | 0.419 % | 0.485 % | 0.738 % |
bcm | 0.11 | 0.169 % | 0.079 % | 0.009 % | 2.523 % | -0.607 % | 0.213 % | 0.398 % | 0.677 % |
m03 | 0.2a1 | 0.094 % | -0.057 % | -0.040 % | 0.865 % | -0.329 % | -1.603 % | -0.178 % | 0.277 % |
blizzard | 0.24b | -0.025 % | -0.067 % | 0.088 % | 2.379 % | -0.252 % | 0.335 % | 0.410 % | 0.562 % |
bsc | 2.4.5 | 0.383 % | 0.006 % | -0.002 % | 1.807 % | -0.697 % | -1.405 % | 0.015 % | 0.579 % |
bbb | 1 | -0.013 % | -0.018 % | -0.019 % | 2.531 % | -0.719 % | -0.106 % | 0.276 % | 0.660 % |
ybs | 0.3f | 0.549 % | 0.008 % | -0.008 % | 1.956 % | -0.720 % | -1.003 % | 0.130 % | 0.648 % |
abc | 2.4 | 0.389 % | 0.008 % | -0.005 % | 1.724 % | -0.746 % | -1.278 % | 0.016 % | 0.575 % |
bssc | 0.95a | 0.520 % | -0.017 % | -0.073 % | 1.887 % | -0.787 % | -0.951 % | 0.096 % | 0.657 % |
dc | 0.98b | 0.541 % | 0.026 % | -0.025 % | 1.907 % | -0.747 % | -1.220 % | 0.080 % | 0.649 % |
grzipii | 0.24 | 0.415 % | -0.063 % | -0.111 % | 1.836 % | -0.815 % | -1.431 % | -0.028 % | 0.648 % |
dark | 0.51 | -0.503 % | 0.026 % | 0.058 % | 1.382 % | -0.383 % | -0.759 % | -0.030 % | 0.471 % |
mnzip | 0 | 0.284 % | 0.006 % | 0.012 % | 1.833 % | -0.738 % | -2.077 % | -0.113 % | 0.575 % |
m99 | 2.21 | 0.452 % | 0.024 % | -0.043 % | 1.938 % | -0.839 % | -1.569 % | -0.006 % | 0.659 % |
mcomp | 2.00 | -0.176 % | 0.036 % | 0.003 % | 2.498 % | -0.842 % | -0.800 % | 0.120 % | 0.711 % |
gca | 0.9k | -0.056 % | 0.023 % | 0.056 % | 2.912 % | -0.679 % | 1.959 % | 0.702 % | 0.745 % |
bma | 1.35b | -0.309 % | 0.023 % | 0.014 % | 2.039 % | -0.846 % | -1.094 % | -0.029 % | 0.646 % |
qlfc | 6.6 | 0.323 % | -0.026 % | -0.020 % | 1.665 % | -0.755 % | -2.398 % | -0.202 % | 0.558 % |
grzip | 0.7.3 | 0.341 % | -0.043 % | -0.093 % | 1.847 % | -0.813 % | -2.012 % | -0.129 % | 0.628 % |
zzip | 0.36c | 1.203 % | 1.496 % | 1.282 % | 3.149 % | 0.731 % | -0.771 % | 1.182 % | 1.572 % |
chile | 0.5 | 0.708 % | 0.144 % | 0.134 % | 2.041 % | -0.561 % | -2.094 % | 0.062 % | 0.718 % |
bwtzip | 0 | 0.562 % | -0.046 % | -0.016 % | 1.746 % | -0.697 % | -1.100 % | 0.075 % | 0.613 % |
bzip | 0.21 | 0.549 % | 0.009 % | -0.029 % | 1.521 % | -0.610 % | -2.638 % | -0.200 % | 0.544 % |
bzip2 | 1.0.5 | 0.834 % | -0.013 % | -0.017 % | 1.399 % | -0.508 % | -3.086 % | -0.232 % | 0.554 % |
bred | 3 | -0.533 % | 0.037 % | 0.032 % | 1.219 % | -0.248 % | -2.224 % | -0.286 % | 0.414 % |
mar | 0 | 0.312 % | 0.052 % | -0.027 % | 1.885 % | -0.613 % | -2.526 % | -0.153 % | 0.578 % |
imp | 1.12 | 0.797 % | 0.010 % | -0.061 % | 1.435 % | -0.547 % | -2.969 % | -0.223 % | 0.570 % |
ecp | 0.e.s | -0.420 % | -0.023 % | -0.119 % | 2.163 % | -0.931 % | -2.469 % | -0.300 % | 0.731 % |
average gain | 0.265 % | 0.043 % | 0.024 % | 1.955 % | -0.602 % | -1.307 % | 0.063 % | 0.640 % |