The Squash library is an abstraction layer for compression algorithms, making it trivial to switch between them… or write a benchmark which tries them all, which is what you see here!
This is the unstable version of the Squash Compression Benchmark. It is run with a recent git checkout of Squash on a single machine (the stable version is currently run on about 10 machines).
This benchmark currently consists of {{datasets.length|number}} datasets, each of which is tested against {{plugins.length}} plugins containing {{codecs.length|number}} codecs at every compression level they offer—the number varies by codec, but there are {{data_points_per_machine|number}} in total, yielding {{data_points_per_machine*datasets.length|number}} different settings.
If you think certain algorithms are always faster, you've got another thing coming! Different CPUs can behave very differently with the same data.
The Squash benchmark is currently run on many of the machines I have access to—this happens to be fairly recent Intel CPUs, and a mix of ARM SBCs. There is an entry in the FAQ with more details.
The default machine is selected randomly.
Note that we do provide access to the raw data if you would prefer to generate your own charts.
Sometimes all you care about is how long something takes to load or save, and how much disk space or bandwidth is used doesn't really matter. For example, if you have a file that would take 1 second to load if uncompressed and you could cut the file size in half by compressing it, as long as decompressing takes less than half a second the content is available sooner than it would have been without compression.
For the presets I have tried to provide typical real-world speeds, not theoretical peaks. This can be significantly less than the advertised speed.
When entering custom values please keep in mind that this uses bytes per second, not bits per second. Also, it uses binary prefixes (1 MiB is 1024 KiB, not 1000).
Well, okay, I'm writing this before I publish the URL, so not so much "frequently asked questions" as "things I thought you might be wondering", but if I called this section "TITYMBW" the first thing I would have to answer would be "WTF is 'TITYMBW'?"
Sure! This benchmark is updated whenever a new version of Squash is released, so you can just subscribe to the squash-annonce mailing list. We send one message to announce the updated version of both the library and the benchmark.
In order to be included the benchmark the software must be supported by Squash. If there is a specific codec you're interested in, please file an issue against Squash.
If the codec is reliable (it has to pass Squash's unit tests), works on Linux, accessible from C or C++, and open source the odds are good I would be willing to write a plugin, or at least merge a pull request.
If the codec doesn't meet all the above critera I still may be willing to accept a pull request for a plugin, but keep in mind that all the machines I run this on are running Linux with various architectures, so even if there is a plugin for a Windows-only, or x86-only, etc., library it probably will not show up for every machine. For example, I would like to add a Windows API plugin, but it will not show up on any of the Linux machines' results. Ditto for OS X.
The benchmark collects the compressed size, compression time, and decompression time. Those are then used to calculate the values used in the benchmark:
uncompressed size ÷ compressed size
uncompressed size ÷ compression time
uncompressed size ÷ decompression time
(2 × uncompressed size) ÷ (compression time + decompression time)
Sizes are presented using binary prefixes—1 KiB is 1024 bytes, 1 MiB is 1024 KiB, and so on.
I would really like to include that data. If you have a good way to capture that data on the C side (in benchmark.c) I would be very happy to merge it, and integrate it into the results.
Please use quixdb/squash-benchmark#2 to discuss this.
CPU time. Wall-clock data is actually captured but not presented. I'm willing to consider pull requests if you can find a good way to display it, as well as a good reason.
That's a tricky question. For one thing it would explode the number of data points (for example, LZMA would have 4 codecs with 9 levels each, multiplied the number the number of cores—that's 288 different configurations on an 8-core machine per dataset). Obviously this would make generating the benchmark even slower, but the main problem is presenting the data in a way which doesn't destroy the usability for people who are interested in single-threaded compression/decompression.
You can. Just click on the label for either axis and it will toggle between linear and logarithmic. I know this isn't obvious—I'd be happy to merge a PR if you can improve that.
I don't want to switch the default because I think that linear is probably better for most people. Logarithmic tends to be better if all you care about is compression ratio, not speed.
I would like to (see quixdb/squash-benchmark-web#2 and quixdb/squash-benchmark-web#3 for discussion). It is probably going to require downloading all the data from all the machines, which is around {{data_points_per_machine*datasets.length*machines.length*66|formatSize:-1}}, though HTTP compression should help (oh, the irony of being stuck with zlib).
No promises, but I am looking for ideas—I'm certainly willing to at least listen to the request. Please file an issue.
Sorry. I am trying to keep conditions as fair and as close to real-world as I can, but I'm certainly willing to discuss any concerns you have with methodology.
The Squash library typically adds very little overhead, but if you have ideas on how to improve the plugin for your library I'm happy to accept patches.
If the issue is performance on an architecture you don't have access to (e.g., ARM), I'm willing to provide SSH access to most of the machines included in this benchmark to people working on open source libraries. If you would like access to one to help you optimize your code just let me know.
Only if I have, or at least have access to, a machine which fits that description.
In general I include what I have available. That tends to be some newish Intel CPUs, some older Intel CPUs that haven't yet found their way to the electronics recycler, and some ARM SBCs. If you would like to donate other hardware I'm willing to add it to the benchmark.
Flags vary a bit since different plugins require different
flags, but as far as performance related flags are
concerned, plugins are currently compiled with -O3
-flto -march=native -mtune=native.
If you are the author of one of the libraries and would like for your plugin to use different flags when compiled as part of Squash, please file an issue. As long as the flags are safe, we will respect your wishes.
I'm sorry, but if you're using an old browser (probably Internet Explorer) you'll have to upgrade. I'm willing to consider patches, but I will not be putting in any effort to make this page display in older browsers unless someone wants to pay me to do it—and it's not something I enjoy, so they would have to pay pretty well.
If your browser is up to date, make sure JavaScript is enabled. Cookies, Flash, etc. aren't necessary, but JS definitely is.
There are different benchmarks, which may or may not be better, depending on what you're interested in. Most benchmarks use command line programs instead of libraries, which is a bit easier so they tend to include many more codecs, but usually don't include nearly as many different machines. Those benchmarks include:
The only other benchmark I'm aware of focusing on libraries is fsbench. It includes more compression codecs (though fewer total options), as well as hash functions (cryptographic and non-cryptographic) and some other cryptographic functions. It is run on a few different machines against a tarball of Silesa with all files truncated to 1 MiB.
If I'm missing any modern, up-to-date benchmarks please let me know.
Of course! The table in the "choose a machine" section includes a link to a CSV which you can import into your favorite spreadsheet application (or at least your least-hated spreadsheet application). If you don't have one yet, LibreOffice Calc is a good choice.
Additionally, you can grab a copy from the the data folder of the squash-benchmark-web git repository.
If you do something interesting with it please let us know! Or, even better, submit a pull request so everyone can benefit from your brilliance!
The data itself is CC0 licensed. That said, we would certainly appreciate attribution.
Some things can be configured by passing parameters in the query string:
For example, your current configuration would be: {{ location }}?dataset={{ dataset }}&machine={{ machine }}&speed={{ calculatedTransferSpeed / 1024 }}&speed-scale={{ speedScale }}.
Note that all fields are optional; you can provide as many or few of them as you like. Also, please be aware that this isn't necessarily stable—we may change the format when adding new features to the benchmark.