Welcome to the most extensive file compression benchmark ever. CompressionRatings.Com tests are pedantic and verifiable. All the software and test material are public.
CompressionRatings.Com is a reference for both professionals and end-users looking for the latest information about the performance of lossless data compression software and technology.
In the frequently updated rating lists each piece of tested software is assigned a rating. The ratings can be used to compare relative performance of various compressors and provide evidence based answers to end-user questions such as: how compressors compare, which compressor is "the best" one, which compressor has "the highest compression ratio", etc.
Continue to the benchmark results...
lzturbo 0.95 submitted by the author.
Hook 1.4 submitted by the author.
LZP2 090420 submitted by the author.
Calculator is a new feature. You can find the link for each test in the summary page. The calculator can be used to calculate optimal compressor for a protocol or used as an alternative custom user rating system. There is a save button which stores the calculator setup into a cookie, so that whenever you click the calculator link your setup will be restored.
7-Zip 9.01a added. See the program page how this version compares to the earlier ones.
PPMd sh8 submitted by the author.
LZP2 (two versions) submitted by the author.
Improvements for program pages. Now the absolute ratio and speed (kB/s) are listed for each configuration. See ZPAQ, 7-Zip, ASH pages for an example. Another improvement is for the qualification results. Gray color is used for ratings below the qualification threshold. Hover mouse over such number to see the difference.
Software added: ASH 0.7 and 0.7b1 submitted by the author.
Software added: LZP2 submitted by the author.
An additional unpacker for lzturbo -59 submitted by the author.
Software added: lzturbo 0.94b submitted by the author.
qpress is now tested also with TAR mode (faster due to the kernel time increasing when accessing multiple files). Also fixed description for ST2rc 8a disqualification.
Change to the qualification system: > 1/3 threshold changed to 1/5 for S.E. and R.E ratings. This results some szip, bwc, ZZIP, bzip2, BZP configurations to qualify. Additional qualification files will be added soon. Another change is that instead of fastest time, now a mean is used for the listed times.
Software added: lzturbo 0.94a submitted by the author.
Software added: Several versions of Mix, ST2rc, fpaq submitted by the author.
Software added: tornado 0.4a.
The tests for these compressors and qpress, lzop, THOR, copy, FastLZ have been run with a new system that repeats compression up to 5 times for each test until 30 seconds time passes. The fastest time is listed. The same is done for the decompression. This ensures the time information is now very accurate. All the rest of the compressors tested before April 8. are now being tested again.
Software added: lzturbo 0.94 with 11 configurations submitted by the author. Disqualified due to the program crashing when launched.
Software added: ASH 0.6b3 submitted by the author. The earlier 0.6b2 was accidentally compiled with wrong rounding mode
.
Software added: PPMd sh7b1 with 16 configurations, ASH 0.6b2 with 3 configurations, Mix 3e21, ST2rc with 4 versions, fpaq 0pv4bmm_o3_xn_ipo submitted by the author along with program descriptions for PPMY and ASH. 12 of the PPMd sh7b1 configurations qualify.
With more and more compressors tested with numerous versions and configurations, you can use the 'Brief' instead of the 'Detailed' tables to see only the best configurations for each compressor. Bios settings for the test machine were changed in order to improve stability. This makes programs run 0.4% slower. All the tests will be run again later once some additional technical improvements have been implemented.
Software added: XWRT 3.2 with 8 configurations submitted by the author. The earlier test with information that the program could compress only XML files was incorrect.
The most requested feature: to be able to sort tables by ratio, time, etc. has now been implemented. You can now click on the column title to sort by it. This works for all the brief and detailed results tables.
Also added: PowerArchiver 2009/5.00b2 and couple of extra configurations for ZZIP.
Software added: PPMY 2.02(3c), 2.02(c3+sse), ASH 0.5, 0.6, PPMd sh6, sh7. PPMd sh variants qualify by performance, but are disqualified as they fail verify check in OS1 test.
Software added: KZIP 2007. Added ST mode configuration for BSSC. Both qualified.
Software added: BIT 0.7, XWRT 3.2, TarsaLZP 2007.8, lzrw. Only lzrw got past the qualifying.
Software added: ecp 0.e.s, Qc 0.050, QazaR 0.0pre5. Older compressors, none of which qualified.
I had forgotten szip, IMP and X1 have a BWT mode. They are now included to the BWT comparison with ecp.
Software added: BMA 1.35b, mnzip 0, bred 3, bzip 0.21, MAr 0, bwic 0, GCA 0.9k, GRZip 0.7.3.
All of these programs with the exception of bred and GRZip fail to qualify. These are BWT compressors and are added to the BWT comparison. The comparison should now include all original BWT programs ever published.
Most of these programs are very old. Two of them are of historical interest: bred and bwic.
bwic is written by the founder of IF. Unfortunately no source or Windows executable is available.
I managed to modify bred to have it compiled and running. I also added various modifications, such as to allow large block sizes. This program is written by David Wheeler, the co-founder of BWT and hence bred was the first program ever to use it. David never lost his talent for writing perfect programs, and one commentator described Wheeler's data compression program as mathematical/computer magic
[The Independent, Dec 22, 2004] Unfortunately a look into the actual code reveals a different picture. The program crashes on compressing Image1 with the default configuration and the quality of the code will make it unlikely anybody will spend time fixing it.
Added extra configurations for M99, GRZipII, mcomp, SBC.
Software added: Flashzip 0.94 submitted by the author. See the program page how this version compares to the earlier ones.
Software added: 7-Zip 4.66a, lzpxj 1.2h.
Software added: Flashzip 0.93c with 6 configurations submitted by the author.
Software added: qpress 0.35b.
Software added: bcm 0.07, Flashzip 0.93b. BWT comparison updated.
This version of Flashzip fails similarly to the earlier version. It runs ok launched manually, but fails under the test environment. Out of the 82 programs currently tested no other program has such problems. This suggests the program may have some hacks and optimizations that backfire. As usual, the authors can contact me and we can try solve any such problems.
Software added: CMM4 0.1e.
Software added: BWMonstr 0.01. BWT comparison updated.
Added two more tests for the extended section: Medical1 and Medical2. These tests consist of medical images from the Lukas Corpus.
I have continued receiving criticism that some compressors are not tested properly. This criticism is only possible if one has not read the documentation or even the 1-min tutorial.
Update: I continue receiving this kind of feedback, yet I'm puzzled how to clarify things any further. I'd be thankful for tips how can the site be made easier to understand.
There is still considerable confusion how the benchmark works. Perhaps a 1-minute tutorial to Compression Ratings is helpful.
Software added: ZPAQ all versions up to to 1.00. See the program page how all the versions compare.
Qualifying2 -file has been replaced, sampled from a new Text1 -file. The new Text1 file has no redundant headers and now it should contain only the pure plaintext material. I have also revised the qualifying. Now a rating higher than 1/5 of the best one will qualify or any other rating performance higher than 1/3 of the best one will qualify as well. The new file and qualification system caused some BWT based compressors like M99 and bcm to qualify. The results are online.
The author of Dark has submitted updated links to his compressor and source code. A broken to the link to the copy program is fixed.
The BWT comparison has triggered responses from people including several authors. Authors have written that their compressor does not use preprocessing. A proof of that cannot consist of an email by the author. Even if the author emails his source code and I'm able to confirm this, others are unable to do so. This site is about evidence based results that can be confirmed and therefore trusted. Secondly a lot of suggestions have been put forward about detecting the presence of preprocessing: xoring, reversing, filtering, etc. I have stated that this does not provide proof, but I have now included such test anyway. Perhaps even without it giving evidence, it may satisfy some readers.
Software added: BWMonstr 0.00.
The program is also added to the BWT comparison.
Software added: M03exp 5.2, bwc 0.99d, ABC 2.4.
Each one is also added to the BWT comparison which has been extended with more test files and compressors.
bcm 0.05 and new configurations for QUAD and balz submitted by the author.
Software added: szip 1.12, FastLZ, bwtzip.
bcm is also added to the BWT comparison. In this comparison we attempt to compare BWT compressors without preprocessing. This comparison will be extended in the next few days.
Software added: BSSC 0.95a, chile 0.5, ZZIP 0.36c, BA 1.01b, BriefLZ 1.04, fpaq 0 and c, barf, flzp, sr, bbb 1, PPMs J. Additional confs. for YBS, Dark, GRZipII, Blizzard.
While analyzing the Text1 file I found it was low quality and contains duplicate books. Unfortunately the Manzini corpus file has similar problems. Therefore I have manually (...) picked and verified each file and constructed a new test file for Text1 using the Gutenberg material. This new file still has some redundant headers. They will be removed later when I have the time. I will also replace the Qualifying2 file once this is done. Added another unusual file for the extended test: Pitches1.
Software added: BIX 1.00b7, UDA 0.301, Stuffit 9.0. Additional confs. for Info-ZIP, SBC.
A new test: Application3. Also went back to the original plan for full size Game1, since the "concerns" were nothing but hysteria from a single person. This aside, thanks to everybody for the feedback, it has been overwhelmingly positive!
Software added: IMP 1.12, JAR 1.02.
New "extended" tests added: Database1, PGN1, FP1. The extended tests are not part of the main corpus and do not have an effect to the overall ratings.
Game1 size is now reverted back to 404 MB due to concerns that there were too much weight given to it. This is still the only real test for the handling of diverse embedded data. There is nothing to stop other changes from happening. I look forward to hearing criticism. Please make sure there is a rational argument behind your criticism, and not that program X is performing badly, therefore we need to make changes. Those who wish for tests on multicore or other platforms can donate the hardware and all the test will be duplicated for the platform.
Software added: Blizzard 0.24, HOOK 1.3c, M99 2.2.1, Ocamyd 1.66, ASH 0.4a, mcomp 2.00. Legacy software: ARJ 3.15, PKZIP 2.50 and 4.00, X1 0.94, Slim 0.23d, RK 1.04, EPM r9, ENC 0.15, DC 0.98b, HA 0.999b, bicom 1.01, QLFC 6.6.
Additional confs. for paq, 7-Zip, ACE, RINGS. Also re-tested SBC Archiver and YBS. This time there was no problems. Perhaps the earlier unstability may have been due to overheating of the test rig?
I have now tested all software that I can think of. Authors may submit software that is missing.
Software added: DURILCA 0.5, QUAD 1.12, GRZipII 0.24. Additional confs. for 7-Zip, RAR, Info-ZIP, gzip, CCM, UHARC, SBC Archiver.
Changes to the corpus: completely new and original Audio1 corpus (202 MB) which replaces the temporary one. This corpus presents an extreme range of different kinds of music samples. Game1 test size has been increased to 688 MB. A new test: Text2 (XML 95 MB).
Software added: Squeez 5.62, Packet 0.90b, Flashzip 0.92b, cabarc 1.00. Additional confs. for 7-Zip, RAR, ACE, CCM (CCMx).
Added some test descriptions and layout changes. Tried to fix some browsers displaying the site wrong. Please let me know if you catch some more visual oddities.
Software added: WinZip 12.0, WinRK 3.1.2, Slug 1.27, Slug 1.24, PPMonstr Jr1. Additional confs. for RAR, ACE, lzop, NanoZip. App2 test repeated for PPMd.
Added copy which bypasses qualification and is not listed in the summary. This will help us find a better way to assign Rating S.E. With 50 percent copy is highest rated in tests that have average compression ratio of less than 2:1. Therefore the Rating S.E. compression is now set to 1/3 instead of 1/2 in hope that this is more useful.
Software added: Slug 1.27b, SBC 0.970r3. Additional confs. for UHARC, FreeArc and THOR.
Software added: RZM 0.07h.
Additional configurations for FreeArc submitted by Bulat Ziganshin. Also additional switches for NanoZip and PPMd.
Change to the corpus: os1 test size is increased from 400 to 500Mbytes. Fixed: one file was missing from the audio1 download description. Later fast compressors will be tested again with multiple runs, so that we improve accuracy of timings. Program infopages now list all tested configurations. Modified the qualification so that a Rating S.E above 1000 qualifies.
Addition: the site still is confusing to navigate. Fixed some of the layout, the site looked wrong on most browsers.
Software added: paq 8p, RINGS 1.5c. Additional switches for UHARC.
Software added: balz 1.15, Dark 0.51, FreeArc 0.50a, PPMd Jr1, lzop 1.01rc1, quicklz 1.31, qpress 0.33, gzip 1.2.4, THOR 0.96a, RAR 3.80, lpaq 8, NanoZip 0.06a, UHARC 0.6b, BZP 0.3, Info-ZIP 2.3, 7-Zip 4.65a, ACE 2.6, CCM 1.30c, bzip2 1.0.5
So far all tests have been run with the default settings only, except for RAR, 7-Zip and Info-ZIP. I call upon authors to submit software configurations. The submited configurations have priority. If an author does not submit configurations for a compressor, they will be choosen and added eventually in arbitrary fashion as the testing queue shortens.
Word about the corpus. There will be changes. The audio set is temporary and it will be replaced in the upcoming months. Other changes will happen, I will also listen to your feedback.
Currently some things are missing: descriptions of test data and clear indicators for qualified software (currently tagged with (!)), compression ratio column, etc.
Please note that the 'qualification' system is experimental and will likely be tuned until a good system is found.
The tests will continue running 24/7 with frequent daily updates.