Home

Welcome to the lrzip-next wiki!

Here will be expanded documentation and thoughts on design decisions made.

Why lrzip-next?

I have been involved with the lrzip project since 2007, version 0.19! My initial contributions centered on upgrading the LZMA SDK, multi-threading, and assembler integration. Since 2019, I have been making enhancements to lrzip outside of the main branch. The goal was to keep improving lrzip. These enhancements, broadly, fall into these categories.

Significant changes

NEW!

• ZSTD backend compression
• BZIP3 backend compression
• Ability to store an archive Comment (-C)
• 13 Hashing Algoriths (see CURRENT_HASHES file)
• Add AES 256 Encryption
• More secure encryption key derivation (breaks compatibility with earlier versions) using SHAKE128 and SHAKE256 Extendable Output Functions (XOF)
• Improved INFO output which can show Encrypted File info
• SCRYPT Key Derivation
• Smaller Magic Header
• New Assembler Module for Lz Match Finder
• LZMA - Keeping current with the SDK (currently 24.07)

Current Changes

• Revise computational algorithm for setting block sizes that are passed to back-end compressors to favor larger blocks over more threads
• Add pre-compression BCJ (Branch-Call-Jump) and Delta filtering
• Add ability to set LZMA Dictionary sizes
• Add ability to set rzip compression level separately
• Fine tuning ZPAQ compression to pass 3 method parameters to compressor
• Validation of file prior to decompression/testing
• Update magic header for bzip3 and comments
• Substitute libgcrypt functions for separate sources for md5 and sha512 hash functions, and aes 128 bit encryption.
  (This will allow for future bug fixes and possibly using different encryption methods through a standard library.)

Other changes include

• Updating help and info output to make it more clear what options are
• Man page and Readme updates
• Fix Threshold testing so that a value can be used to limit "compressibility" testing by specifying a percent (i.e. -T75)
• Optimize use of memory overhead
• Creation of a dialog-based front-end for lrzip, lrzip-fe which can be pulled as a sub-module
• Precise versioning that reflects git revision and commit (Note: This breaks downloading a zip file from git), git pull/fetch/clone must be used).

Along the way, bugs and CVEs were corrected. Every attempt is made to include fixes and changes to the main branch, but this is not a fork of lrzip anymore. It's a standalone project.

As my fixes and enhancements piled up, lrzip-next diverged to a point that merging the changes with the main branch became impossible.

These pages will explain some of the changes in detail, and why they were made, the intended benefits, with code snippets.

Please feel free to contribute, submit bugs, comment, make suggestions, and help improve lrzip-next. Currently, there are no plans to have any of these changes integrated into the main lrzip repository.