"Fossies" - the Fresh Open Source Software Archive

Member "lrzip-0.641/WHATS-NEW" (5 Mar 2021, 21603 Bytes) of package /linux/misc/lrzip-0.641.tar.xz:


As a special service "Fossies" has tried to format the requested text file into HTML format (style: standard) with prefixed line numbers. Alternatively you can here view or download the uninterpreted source code file. See also the latest Fossies "Diffs" side-by-side code changes report for "WHATS-NEW": 0.640_vs_0.641.

    1 Changelog will be moved to git entirely from this point forward.
    2 
    3 lrzip-0.641
    4 
    5 Critical bugfix for broken lz4 testing which would prevent secondary
    6 compression from being enabled.
    7 
    8 lrzip-0.640
    9 
   10 Numerous bugfixes and build fixes.
   11 lz4 now used for compressibility testing (only) making lz4-dev a build
   12 requirement.
   13 Fixes for handling of corrupt archives without crashing.
   14 Fixes for creating small lzma based archives to stdout.
   15 Incomplete files are now deleted on interrupting lrzip unless the keep-broken
   16 option is enabled.
   17 Version prints to stdout instead of stderr.
   18 
   19 
   20 lrzip-0.631
   21 
   22 Assembler code is back and works with x86_64
   23 
   24 lrzip-0.621
   25 
   26 Substantial speed ups for the rzip stage in both regular and unlimited modes.
   27 Lrzip now supports long command line options.
   28 Proper support for the various forms of TMPDIR environment variables.
   29 More unix portability fixes.
   30 OSX fixes.
   31 Fixed order of lrzip.conf search.
   32 Addressed all warnings created with pedantic compiler settings and clang
   33 Fixes for some stderr messages being swallowed up.
   34 Fixed being unable to decompress to STDOUT when in a non-writable directory.
   35 Changed broken liblrzip callback function API to match lrzip proper.
   36 
   37 
   38 lrzip-0.620
   39 
   40 Fixes display output of lrzip -i for large files greater than one chunk.
   41 Fixes for various failure to allocate memory conditions when dealing with
   42 large files and STDIO.
   43 Fixes for more unix portability.
   44 Fixes for failure to decompress to STDOUT.
   45 
   46 
   47 lrzip-0.616
   48 
   49 Fixes for various issues with -O not working with trailing slashes and
   50 outputting to directories that already exist.
   51 
   52 
   53 lrzip-0.615
   54 
   55 Fixed -O not working on lrztar.
   56 Made it less likely to run out of ram when working with STDIN/OUT.
   57 Fixed running out of ram when using -U on huge files.
   58 Fixed corrupt archives being generated from incompressible data.
   59 Fixed corrupt archives being generated from very small files.
   60 Fixed endianness on various platforms for MD5 calculation to work.
   61 Fixed rare corruption when compressing with lzma from STDIN.
   62 Fixed all blank data being generated when compressing from STDIN on OSX.
   63 Performance micro-optimisations.
   64 Fixed corrupt archive being generated when all the same non-zero bytes exist on
   65 large files.
   66 
   67 
   68 lrzip-0.614
   69 
   70 Fixed lrztar not working.
   71 
   72 
   73 lrzip-0.613
   74 
   75 Fixed the bug where massive files would show an incorrect md5 value on
   76 decompression - this was a bug from the md5 code upstream.
   77 Compressing ultra-small files to corrupt archives was fixed.
   78 Compilation on various other platforms was fixed.
   79 A crash with using -S was fixed.
   80 
   81 
   82 lrzip-0.612
   83 
   84 Updated to a new zpaq library back end which is faster and now supports three
   85 different compression levels, which will be activated at lrzip levels -L 1+, 4+
   86 and 8+. This significantly increases the maximum compression available by lrzip
   87 with -L 9.
   88 The include file Lrzip.h used by liblrzip is now properly installed into
   89 $prefix/include.
   90 
   91 lrzip-0.611
   92 
   93 lrzcat and lrzuntar have been fixed.
   94 The update counter will continue to update even when there is nothing being
   95 matched (like a file full of zeroes).
   96 Numerous optimisations in the rzip stage speeds up the faster compression modes
   97 noticeably.
   98 Checksumming is done in a separate thread during rzip compression for more
   99 compression speed improvements.
  100 
  101 lrzip-0.610
  102 
  103 The new liblrzip library allows you to add lrzip compression and decompression
  104 to other applications with either simple lrzip_compress and lrzip_decompress
  105 functions or fine control over all the options with low level functions.
  106 Faster rzip stage when files are large enough to require the sliding mmap
  107 feature (usually >1/3 of ram) and in unlimited mode.
  108 A bug where multiple files being compressed or decompressed from the one
  109 command line could have gotten corrupted was fixed.
  110 Modification date of the decompressed file is now set to that of the lrzip
  111 archive (support for storing the original file's date would require modifying
  112 the archive format again).
  113 Compilation warning fixes.
  114 Make lrztar work with directories with spaces in their names.
  115 
  116 lrzip-0.608
  117 
  118 Faster rzip stage through use of a selective get_sb function.
  119 The bash completion script is no longer installed by default to not conflict
  120 with distribution bash completion packages.
  121 More compilation fixes for non-linux platforms.
  122 
  123 lrzip-0.607
  124 
  125 A rare case of not being able to decompress archives was fixed.
  126 The lzma library was updated to version 920.
  127 A bash completion script for lrzip was added.
  128 More debugging info was added in maximum verbose mode.
  129 Less messages occur without verbose mode.
  130 FreeBSD and posix compilation fixes were committed.
  131 
  132 lrzip-0.606
  133 
  134 lrzuntar, which broke last version leaving behind an untarred .tar file, is
  135 working properly again.
  136 
  137 lrzip-0.605
  138 
  139 Addition of lrzcat - automatically decompresses .lrz files to stdout.
  140 lrzip and lrunzip will no longer automatically output to stdout due to
  141 addition of lrzcat executable, and to be consistent with gzip.
  142 lrzip progress output will no longer spam the output unless the percentage
  143 has changed.
  144 lrzip now has no lower limit on file sizes it will happily compress and is
  145 able to work with zero byte sized files.
  146 The percentage counter when getting file info on small files will not show
  147 %nan.
  148 The executable bit will not be enabled when compressing via a means that
  149 can't preserve the original permissions (e.g. from STDIN).
  150 
  151 lrzip-0.604
  152 
  153 lrzip will no longer fail with a "resource temporarily unavailable" error
  154 when compressing files over 100GB that require hundreds of threads to
  155 complete.
  156 
  157 lrzip-0.603
  158 
  159 lrzip now supports stdout without requiring the '-o -' option. It detects when
  160 output is being redirected without a filename and will automatically output to
  161 stdout so you can do:
  162  lrunzip patch-2.6.38.4.lrz | patch -p1
  163 Apple builds will not have errors on compressing files >2GB in size which
  164 broke with 0.600.
  165 lrztar will properly support -o, -O and -S.
  166 lrzip.conf file now supports encryption.
  167 lrzip will now warn if it's inappropriately passed a directory as an argument
  168 directly.
  169 
  170 lrzip-0.602
  171 
  172 Fixed wrong symlinks which broke some package generation.
  173 Imposed limits for 32bit machines with way too much ram for their own good.
  174 Disable md5 generation on Apple for now since it's faulty.
  175 Displays full version with -V.
  176 Checks for podman on ./configure
  177 Now builds on Cygwin.
  178 File permissions are better carried over instead of being only 0600.
  179 
  180 lrzip-0.601
  181 
  182 lrzuntar, lrunzip symlinks and the pod-based manpages are installed again.
  183 Configuration clearly shows now that ASM isn't supported on 64bit.
  184 
  185 lrzip-0.600
  186 
  187 Compressing/decompressing to/from STDIN/STDOUT now works without generating
  188 any temporary files. Very large files compressed in this way will be less
  189 efficiently compressed than if the whole solid file is presented to lrzip,
  190 but it is guaranteed not to generate temporary files on compression. 
  191 Decompressing files on a machine with the same amount of ram will also not
  192 generate temporary files, but if a file was generated on a larger ram machine,
  193 lrzip might employ temporary files, but they will not be the full size of the
  194 final file.
  195 Decompression should now be faster as the rzip reconstruction stage is mostly
  196 performed in ram before being written to disk, and testing much faster.
  197 Final file sizes should be slightly smaller as block headers are now also
  198 compressed.
  199 Heavy grade encryption is now provided with the -e option. A combination of
  200 a time scaled multiply hashed sha512 password with random salt followed by
  201 aes128 block encryption of all data, including the data headers, provides for
  202 extremely secure encryption. Passwords up to 500 characters in length are
  203 supported, and the same file encrypted with the same password is virtually
  204 guaranteed to never produce the same data twice. All data beyond the basic
  205 lrzip opening header is completely obscured. Don't lose your password!
  206 Lrzip will not try to malloc a negative amount of ram on smaller ram machines,
  207 preferring to decrease the number of threads used when compressing, and then
  208 aborting to a nominal minimum.
  209 A new build configuration system which should be more robust and provides
  210 neater output during compilation.
  211 lrzip should work again on big endian hardware.
  212 lrztar / lrzuntar will no longer use temporary files.
  213 
  214 lrzip-0.571
  215 
  216 Avoid spurious errors on failing to mmap a file.
  217 Fee space will now be checked to ensure there is enough room for the
  218 compressed or decompressed file and lrzip will abort unless the -f option is
  219 passed to it.
  220 The extra little chunk at the end of every large file should now be fixed.
  221 The file lzma.txt now has unix end-of-lines.
  222 There will be a more accurate summary of what compression window will be used
  223 when lrzip is invoked with STDIN/STDOUT.
  224 STDIN will now be able to show estimated time to completion and percentage
  225 complete once lrzip knows how much file is left.
  226 Temporary files are much less likely to be left lying around.
  227 Less temporary file space will be used when decompressing to stdout.
  228 File checking will not be attempted when it's meaningless (like to stdout).
  229 Times displayed should avoid the nonsense thousands of seconds bug.
  230 
  231 lrzip-0.570
  232 
  233 Multi-threaded performance has been improved with a significant speed-up on
  234 both compression and decompression. New benchmark results have been added to
  235 the README.benchmarks file.
  236 Visual output has been further improved, with an updated help menu and no
  237 unrelated system errors on failure.
  238 lrzip.conf supports the newer options available.
  239 TMP environment is now respected when using temporary files and TMPDIR can be
  240 set in lrzip.conf.
  241 LRZIP=NOCONFIG environment variable setting can be used to bypass lrzip.conf.
  242 The -M option has been removed as the -U option achieves more and has
  243 understandable semantics.
  244 Memory usage should be very tightly controlled on compression now by default,
  245 using the most possible without running out of ram.
  246 Temporary files generated when doing -t from stdin will no longer be left lying
  247 around.
  248 lrzip will no longer stupidly sit waiting to read from stdin/stdout when called
  249 from a terminal without other arguments.
  250 Executable size will be slightly smaller due to stripping symbols by default
  251 now.
  252 The -T option no longer takes an argument. It simply denotes that lzo testing
  253 should be disabled.
  254 Verbose added to -i now prints a lot more information about an lrzip archive.
  255 
  256 lrzip-0.560
  257 
  258 Implemented OSX multi-threading by converting all semaphores to pthread_mutexes.
  259 Converted the integrity checking to also use md5 hash checking. As a bonus it
  260 is still backwardly compatible by still storing the crc value, and yet is
  261 faster on large files than the old one. On decompression it detects whether
  262 the md5 value has been stored and chooses what integrity checking to use.
  263 Implemented the -H feature which shows the md5 hash value on compression and
  264 decompression. It is also shown in max verbose mode.
  265 Added information about what integrity testing will be used in verbose mode,
  266 and with the -i option.
  267 Added the -c option which will perform a hash check on the file generated on
  268 disk on decompression, comparing it to that from the archive to validate the
  269 decompressed file.
  270 Modified lrzip to delete broken or damaged files when lrzip is interrupted or
  271 the file generated fails an integrity test.
  272 Added the -k keep option to keep broken or damaged files.
  273 Case reports of corruption have been confirmed to NOT BE DUE TO LRZIP.
  274 
  275 lrzip-0.552
  276 
  277 Fixed a potential silent corruption bug on decompression.
  278 Fixed compilation on freebsd.
  279 Fixed failures on incompressible blocks with bzip2 or gzip.
  280 Fixed osx failing to work. It does not support threaded compression or
  281 decompression but should work again.
  282 
  283 lrzip-0.551
  284 
  285 Compressing from stdin should be unbroken again.
  286 Compression values returned at the end of stdin work.
  287 lzma failing to compress a block will not cause a failure.
  288 
  289 lrzip-0.550
  290 
  291 Speed up compression on large files that take more than one pass by overlapping
  292 work on successive streams, thus using multiple CPUs better.
  293 Fix for failures to decompress large files. Decompression will be slightly
  294 slower but more reliable.
  295 Faster lzma compression by default, less prone to memory failures, but at slight
  296 compression cost.
  297 Recover from multithreaded failures by serialising work that there isn't enough
  298 ram to do in parallel.
  299 Revert the "smooth out spacing" change in 0.544 as it slowed things down instead
  300 of speeding them up.
  301 Larger compression windows are back for 32 bits now that memory usage is kept
  302 under better control.
  303 Fixed some memory allocation issues which may have been causing subtle bugs.
  304 
  305 lrzip-0.544
  306 
  307 Hopefully a fix for corrupt decompression on large files with multiple stream 0
  308 entries.
  309 Fix for use under uclibc.
  310 Fix for memory allocation errors on large files on 32 bits.
  311 Smooth out spacing of compression threads making better use of CPU on compress
  312 and decompress.
  313 Fix for using -U on ultra-small files.
  314 Use bzip2 on blocks that lzma fails to compress to make sure they are still
  315 compressed.
  316 
  317 lrzip-0.543
  318 
  319 A fix for when large files being decompressed fail with multithreaded
  320 decompression.
  321 Slight speedup on multithreaded workloads by decreasing the nice value of the
  322 main process compared to the back end threads as it tends to be the rate
  323 limiting component.
  324 Fixed lzma compression windows being set way too small by default.
  325 
  326 lrzip-0.542
  327 
  328 Lrzip will now try to select sane defaults for memory usage in cases where the
  329 virtual memory heavily overcommits (eg. Linux) as this seriously slows down
  330 compression.
  331 For compression windows larger than 2/3 ram, lrzip will now use a sliding mmap
  332 buffer for better performance.
  333 The progress output is more informative in max verbose mode, and will no longer
  334 do more passes than it estimates.
  335 32 bit machines should be able to use slightly larger windows.
  336 The sliding mmap not working on 2nd pass onwards has been fixed which should
  337 speed up the slowdown of death.
  338 
  339 lrzip-0.540
  340 
  341 MASSIVE MULTITHREADING on the decompression phase. Provided there are enough
  342 chunks of data in the archived file, lrzip will use as many threads as there
  343 are CPUs for the backend decompression. Much like the multithreading on the
  344 compression side, it makes the slower compression algorithms speed up the most.
  345 Fixed output from being scrambled and consuming a lot of CPU time on threaded
  346 zpaq compression.
  347 Further fixes to ensure window sizes work on 32 bit machines.
  348 Be more careful about testing for how much ram lrzip can use.
  349 Minor build warning fixes.
  350 Minor tweaks to screen output.
  351 Updated benchmarks.
  352 
  353 lrzip-0.530
  354 
  355 MASSIVE MULTITHREADING on the compression phase. Lrzip will now use as many
  356 threads as you have CPU cores for the back end compression, and even continue
  357 doing the rzip preprocessing stage as long as it can which the other threads
  358 continue. This makes the slower compression algorithms (lzma and zpaq) much
  359 faster on multicore machines, to the point of making zpaq compression almost
  360 as fast as single threaded lzma compression.
  361 -p option added to allow you to specify number of processors to override the
  362 built-in test, or if you wish to disable threading.
  363 -P option to not set permissions has now been removed since failing to set
  364 permissions is only a warning now and not a failure.
  365 Further improvements to the progress output.
  366 Updated benchmarks and docs.
  367 
  368 lrzip-0.520
  369 
  370 Just changed version numbering back to 2 point.
  371 
  372 lrzip-0.5.2
  373 
  374 Fixed the Darwin build again.
  375 Fix the corner case of big ram usage on 32 bit zpaq failing due to the
  376 compression window not being limited by limiting zpaq to 600MB windows on 32
  377 bits as well.
  378 Some previous failures now only induce warnings.
  379 Improved progress output.
  380 
  381 lrzip-0.5.1
  382 
  383 Fixed the build on Darwin.
  384 Rewrote the rzip compression phase to make it possible to use unlimited sized
  385 windows now, not limited by ram. Unfortunately it gets progressively slower in
  386 this mode the bigger the file gets but you can compress a file of any size as
  387 one big compression window with it using the new -U option. Suggest you try
  388 the new improved -M mode first or in combination. See the docs for more
  389 information.
  390 Changed the memory selection system to simply find the largest reasonable sized
  391 window and use that by default instead of guessing the window size.
  392 Setting -M now only affects the window size, trying to find the largest
  393 unreasonably sized window that will still work.
  394 The default compression level is now 9 and affects the rzip compression stage
  395 as well as the backend compression.
  396 Fixed some potential failures during compression.
  397 Improved screen output with more reporting in verbose mode, and chunk size
  398 percentage update.
  399 Fixed file size reporting on compressed files generated from stdin.
  400 Changed to 3 point releases in case we get more than 9 subversions ;)
  401 
  402 lrzip-0.50
  403 
  404 Rewrote the file format to be up to 5% more compact and slightly faster.
  405 Made the memory initialisation much more robust, with attempted fallback
  406 to still work even when initial settings fail.
  407 Updated a lot of the stdin code.
  408 The most common scenario of compression from stdin now works without
  409 temporary files.
  410 Lots more meaningful warnings if failure occurs.
  411 May be able to decompress files on 32 bit machines that were compressed on 64
  412 bit machines with >2GB windows now if there is enough ram.
  413 
  414 lrzip-0.46
  415 
  416 Added lrzuntar which works the same as lrztar -d.
  417 
  418 Con Kolivas
  419 May 2010
  420 
  421 lrzip-0.45
  422 
  423 Added docs for lrztar and lrunzip.
  424 Added distclean and maintainer-clean make targets.
  425 Created git repo: http://github.com/ckolivas/lrzip
  426 
  427 Con Kolivas
  428 March 2010
  429 
  430 lrzip-0.44
  431 
  432 Added an lrztar wrapper to compress / decompress whole directories (finally).
  433 Added -i option to give information about a compressed file.
  434 
  435 lrzip-0.43
  436 
  437 Darwin support updated. Should build on OSX v10.5+
  438 Finally, stdin/stdout support.
  439 Test archive integrity support.
  440 ZPAQ support in config files.
  441 
  442 lrzip-0.42
  443 
  444 ZPAQ compression update now shows which rzip stream it's currently compressing
  445 making the update more useful. It also doesn't update unnecessarily with every
  446 byte compressed which was slowing it down a LOT.
  447 
  448 lrzip-0.41
  449 
  450 ZPAQ compression backend! ZPAQ is from the family of "paq" compressors that
  451 have some of the best compression ratios around, but at the cost of extremely
  452 long compression and equally long decompression times. This can be enabled
  453 with the -z option and makes lrzip archives made with this not backwardly
  454 compatible.
  455 
  456 lrzip-0.40
  457 
  458 Compression windows should be limited by available ram now on 64bit. The limit
  459 on 32bit is still 2GB.
  460 The compression advantages on large files on 64bit machines with large ram
  461 should be substantially better.
  462 The file format is no longer compatible with earlier versions of lrzip.
  463 Support for decompressing older formats is present, but all new files will
  464 be generated in the new format.
  465 Minor speedups.
  466 Decompression should no longer stall at 4GB boundaries for extended periods
  467 making decompression much faster on files >4GB in size.
  468 Documentation and benchmark updates galore.
  469 
  470 lrzip-0.31
  471 
  472 The window size limit is now 2GB on both 32bit and 64bit. While it appears to be
  473 smaller than the old windows, only 900MB was being used on .30 even though it
  474 claimed to use more. This can cause huge improvements in the compression of very
  475 large files. Flushing of data to disk between compression windows was
  476 implemented to minimise disk thrashing of read vs write.
  477 
  478 Con Kolivas
  479 November 2009
  480 
  481 lrzip-0.30
  482 
  483 -P option to not set permissions on output files allowing you to write to
  484 braindead filesystems (eg fat32).
  485 Probably other weird and wonderful bugs have been introduced.
  486 
  487 Con Kolivas
  488 November 2009
  489 
  490 lrzip-0.24 has updated functionality
  491 
  492 FEATURE ENHANCEMENTS
  493 lrzip.conf file may be used to set default parameters.
  494 Omit conf using environment: LRZIP=NOCONFIG lrzip.....
  495 LRZIP environment variable may be used in the future
  496 to store certain types of parameters.
  497 
  498 LZMA SDK has been upgraded to version 4.63. This
  499 version fixes some problems certain users observed,
  500 and is much simpler using a C-only wrapper
  501 interface.
  502 
  503 lrzip now is able to compute an ETA for completion.
  504 In order to do this, the file must be larger than
  505 one compression window in size. That is, is the
  506 compression window is 500MB, and the file is 1GB,
  507 then after the first pass, an ETA will be computed.
  508 If the file is smaller, then no estimate can be made.
  509 
  510 lrzip is now able to compute MB/s transfer speeds
  511 for both compression and decompression.
  512 
  513 CLEANUPS
  514 Some file cleanups have been done.
  515 
  516 Peter Hyman
  517 January 2009
  518 pete@peterhyman.com
  519 
  520 lrzip-0.22 update
  521 FEATURE ENHANCEMENTS
  522 
  523 -g option. Now supports gzip compression. Very fast!
  524 Expanded dictionary buffer size in lzma compressor.
  525 Variable, expanded dictionary size buffer in both lzma
  526  compressor and decompressor.
  527 Improved output during compression when using -vv.
  528 Multi-threading supprt when using multiple processors
  529  or dual core processors when using lzma compression.
  530  This results in a nearly 2x speed improvement.
  531 Assembler module support to speed up CRC checking.
  532 Improvements in autotools usage, system detection
  533  and Makefile enhancements.
  534 Lrzip now has a timer that will print total time
  535  at the end of a compression or decompression if
  536  -q command line option is not used.
  537 
  538 BUG FIX!!!
  539 
  540 Even though lrzip uses a compression threshold to
  541 prevent the lzma compressor from getting data that
  542 may not be compressible, there was still a possibility
  543 that lrzip could hang. This was because a data chunk
  544 could contain an uncompressible segment and if the
  545 lzma compressor got it, it would hang.
  546 
  547 THANKS TO LASSE COLLIN for uncovering the error in
  548 the lzma wrapper code that was causing the hangup.
  549 
  550 January 2008
  551 Peter Hyman
  552 pete@peterhyman.com