1 2XZ Utils Release Notes 3====================== 4 55.2.1 (2015-02-26) 6 7 * Fixed a compression-ratio regression in fast mode of LZMA1 and 8 LZMA2. The bug is present in 5.1.4beta and 5.2.0 releases. 9 10 * Fixed a portability problem in xz that affected at least OpenBSD. 11 12 * Fixed xzdiff to be compatible with FreeBSD's mktemp which differs 13 from most other mktemp implementations. 14 15 * Changed CPU core count detection to use cpuset_getaffinity() on 16 FreeBSD. 17 18 195.2.0 (2014-12-21) 20 21 Since 5.1.4beta: 22 23 * All fixes from 5.0.8 24 25 * liblzma: Fixed lzma_stream_encoder_mt_memusage() when a preset 26 was used. 27 28 * xzdiff: If mktemp isn't installed, mkdir will be used as 29 a fallback to create a temporary directory. Installing mktemp 30 is still recommended. 31 32 * Updated French, German, Italian, Polish, and Vietnamese 33 translations. 34 35 Summary of fixes and new features added in the 5.1.x development 36 releases: 37 38 * liblzma: 39 40 - Added support for multi-threaded compression. See the 41 lzma_mt structure, lzma_stream_encoder_mt(), and 42 lzma_stream_encoder_mt_memusage() in <lzma/container.h>, 43 lzma_get_progress() in <lzma/base.h>, and lzma_cputhreads() 44 in <lzma/hardware.h> for details. 45 46 - Made the uses of lzma_allocator const correct. 47 48 - Added lzma_block_uncomp_encode() to create uncompressed 49 .xz Blocks using LZMA2 uncompressed chunks. 50 51 - Added support for LZMA_IGNORE_CHECK. 52 53 - A few speed optimizations were made. 54 55 - Added support for symbol versioning. It is enabled by default 56 on GNU/Linux, other GNU-based systems, and FreeBSD. 57 58 - liblzma (not the whole XZ Utils) should now be buildable 59 with MSVC 2013 update 2 or later using windows/config.h. 60 61 * xz: 62 63 - Fixed a race condition in the signal handling. It was 64 possible that e.g. the first SIGINT didn't make xz exit 65 if reading or writing blocked and one had bad luck. The fix 66 is non-trivial, so as of writing it is unknown if it will be 67 backported to the v5.0 branch. 68 69 - Multi-threaded decompression can be enabled with the 70 --threads (-T) option. 71 72 - New command line options in xz: --single-stream, 73 --block-size=SIZE, --block-list=SIZES, 74 --flush-timeout=TIMEOUT, and --ignore-check. 75 76 - xz -lvv now shows the minimum xz version that is required to 77 decompress the file. Currently it is 5.0.0 for all supported 78 .xz files except files with empty LZMA2 streams require 5.0.2. 79 80 * xzdiff and xzgrep now support .lzo files if lzop is installed. 81 The .tzo suffix is also recognized as a shorthand for .tar.lzo. 82 83 845.1.4beta (2014-09-14) 85 86 * All fixes from 5.0.6 87 88 * liblzma: Fixed the use of presets in threaded encoder 89 initialization. 90 91 * xz --block-list and --block-size can now be used together 92 in single-threaded mode. Previously the combination only 93 worked in multi-threaded mode. 94 95 * Added support for LZMA_IGNORE_CHECK to liblzma and made it 96 available in xz as --ignore-check. 97 98 * liblzma speed optimizations: 99 100 - Initialization of a new LZMA1 or LZMA2 encoder has been 101 optimized. (The speed of reinitializing an already-allocated 102 encoder isn't affected.) This helps when compressing many 103 small buffers with lzma_stream_buffer_encode() and other 104 similar situations where an already-allocated encoder state 105 isn't reused. This speed-up is visible in xz too if one 106 compresses many small files one at a time instead running xz 107 once and giving all files as command-line arguments. 108 109 - Buffer comparisons are now much faster when unaligned access 110 is allowed (configured with --enable-unaligned-access). This 111 speeds up encoding significantly. There is arch-specific code 112 for 32-bit and 64-bit x86 (32-bit needs SSE2 for the best 113 results and there's no run-time CPU detection for now). 114 For other archs there is only generic code which probably 115 isn't as optimal as arch-specific solutions could be. 116 117 - A few speed optimizations were made to the SHA-256 code. 118 (Note that the builtin SHA-256 code isn't used on all 119 operating systems.) 120 121 * liblzma can now be built with MSVC 2013 update 2 or later 122 using windows/config.h. 123 124 * Vietnamese translation was added. 125 126 1275.1.3alpha (2013-10-26) 128 129 * All fixes from 5.0.5 130 131 * liblzma: 132 133 - Fixed a deadlock in the threaded encoder. 134 135 - Made the uses of lzma_allocator const correct. 136 137 - Added lzma_block_uncomp_encode() to create uncompressed 138 .xz Blocks using LZMA2 uncompressed chunks. 139 140 - Added support for native threads on Windows and the ability 141 to detect the number of CPU cores. 142 143 * xz: 144 145 - Fixed a race condition in the signal handling. It was 146 possible that e.g. the first SIGINT didn't make xz exit 147 if reading or writing blocked and one had bad luck. The fix 148 is non-trivial, so as of writing it is unknown if it will be 149 backported to the v5.0 branch. 150 151 - Made the progress indicator work correctly in threaded mode. 152 153 - Threaded encoder now works together with --block-list=SIZES. 154 155 - Added preliminary support for --flush-timeout=TIMEOUT. 156 It can be useful for (somewhat) real-time streaming. For 157 now the decompression side has to be done with something 158 else than the xz tool due to how xz does buffering, but this 159 should be fixed. 160 161 1625.1.2alpha (2012-07-04) 163 164 * All fixes from 5.0.3 and 5.0.4 165 166 * liblzma: 167 168 - Fixed a deadlock and an invalid free() in the threaded encoder. 169 170 - Added support for symbol versioning. It is enabled by default 171 on GNU/Linux, other GNU-based systems, and FreeBSD. 172 173 - Use SHA-256 implementation from the operating system if one is 174 available in libc, libmd, or libutil. liblzma won't use e.g. 175 OpenSSL or libgcrypt to avoid introducing new dependencies. 176 177 - Fixed liblzma.pc for static linking. 178 179 - Fixed a few portability bugs. 180 181 * xz --decompress --single-stream now fixes the input position after 182 successful decompression. Now the following works: 183 184 echo foo | xz > foo.xz 185 echo bar | xz >> foo.xz 186 ( xz -dc --single-stream ; xz -dc --single-stream ) < foo.xz 187 188 Note that it doesn't work if the input is not seekable 189 or if there is Stream Padding between the concatenated 190 .xz Streams. 191 192 * xz -lvv now shows the minimum xz version that is required to 193 decompress the file. Currently it is 5.0.0 for all supported .xz 194 files except files with empty LZMA2 streams require 5.0.2. 195 196 * Added an *incomplete* implementation of --block-list=SIZES to xz. 197 It only works correctly in single-threaded mode and when 198 --block-size isn't used at the same time. --block-list allows 199 specifying the sizes of Blocks which can be useful e.g. when 200 creating files for random-access reading. 201 202 2035.1.1alpha (2011-04-12) 204 205 * All fixes from 5.0.2 206 207 * liblzma fixes that will also be included in 5.0.3: 208 209 - A memory leak was fixed. 210 211 - lzma_stream_buffer_encode() no longer creates an empty .xz 212 Block if encoding an empty buffer. Such an empty Block with 213 LZMA2 data would trigger a bug in 5.0.1 and older (see the 214 first bullet point in 5.0.2 notes). When releasing 5.0.2, 215 I thought that no encoder creates this kind of files but 216 I was wrong. 217 218 - Validate function arguments better in a few functions. Most 219 importantly, specifying an unsupported integrity check to 220 lzma_stream_buffer_encode() no longer creates a corrupt .xz 221 file. Probably no application tries to do that, so this 222 shouldn't be a big problem in practice. 223 224 - Document that lzma_block_buffer_encode(), 225 lzma_easy_buffer_encode(), lzma_stream_encoder(), and 226 lzma_stream_buffer_encode() may return LZMA_UNSUPPORTED_CHECK. 227 228 - The return values of the _memusage() functions are now 229 documented better. 230 231 * Support for multithreaded compression was added using the simplest 232 method, which splits the input data into blocks and compresses 233 them independently. Other methods will be added in the future. 234 The current method has room for improvement, e.g. it is possible 235 to reduce the memory usage. 236 237 * Added the options --single-stream and --block-size=SIZE to xz. 238 239 * xzdiff and xzgrep now support .lzo files if lzop is installed. 240 The .tzo suffix is also recognized as a shorthand for .tar.lzo. 241 242 * Support for short 8.3 filenames under DOS was added to xz. It is 243 experimental and may change before it gets into a stable release. 244 245 2465.0.8 (2014-12-21) 247 248 * Fixed an old bug in xzgrep that affected OpenBSD and probably 249 a few other operating systems too. 250 251 * Updated French and German translations. 252 253 * Added support for detecting the amount of RAM on AmigaOS/AROS. 254 255 * Minor build system updates. 256 257 2585.0.7 (2014-09-20) 259 260 * Fix regressions introduced in 5.0.6: 261 262 - Fix building with non-GNU make. 263 264 - Fix invalid Libs.private value in liblzma.pc which broke 265 static linking against liblzma if the linker flags were 266 taken from pkg-config. 267 268 2695.0.6 (2014-09-14) 270 271 * xzgrep now exits with status 0 if at least one file matched. 272 273 * A few minor portability and build system fixes 274 275 2765.0.5 (2013-06-30) 277 278 * lzmadec and liblzma's lzma_alone_decoder(): Support decompressing 279 .lzma files that have less common settings in the headers 280 (dictionary size other than 2^n or 2^n + 2^(n-1), or uncompressed 281 size greater than 256 GiB). The limitations existed to avoid false 282 positives when detecting .lzma files. The lc + lp <= 4 limitation 283 still remains since liblzma's LZMA decoder has that limitation. 284 285 NOTE: xz's .lzma support or liblzma's lzma_auto_decoder() are NOT 286 affected by this change. They still consider uncommon .lzma headers 287 as not being in the .lzma format. Changing this would give way too 288 many false positives. 289 290 * xz: 291 292 - Interaction of preset and custom filter chain options was 293 made less illogical. This affects only certain less typical 294 uses cases so few people are expected to notice this change. 295 296 Now when a custom filter chain option (e.g. --lzma2) is 297 specified, all preset options (-0 ... -9, -e) earlier are on 298 the command line are completely forgotten. Similarly, when 299 a preset option is specified, all custom filter chain options 300 earlier on the command line are completely forgotten. 301 302 Example 1: "xz -9 --lzma2=preset=5 -e" is equivalent to "xz -e" 303 which is equivalent to "xz -6e". Earlier -e didn't put xz back 304 into preset mode and thus the example command was equivalent 305 to "xz --lzma2=preset=5". 306 307 Example 2: "xz -9e --lzma2=preset=5 -7" is equivalent to 308 "xz -7". Earlier a custom filter chain option didn't make 309 xz forget the -e option so the example was equivalent to 310 "xz -7e". 311 312 - Fixes and improvements to error handling. 313 314 - Various fixes to the man page. 315 316 * xzless: Fixed to work with "less" versions 448 and later. 317 318 * xzgrep: Made -h an alias for --no-filename. 319 320 * Include the previously missing debug/translation.bash which can 321 be useful for translators. 322 323 * Include a build script for Mac OS X. This has been in the Git 324 repository since 2010 but due to a mistake in Makefile.am the 325 script hasn't been included in a release tarball before. 326 327 3285.0.4 (2012-06-22) 329 330 * liblzma: 331 332 - Fix lzma_index_init(). It could crash if memory allocation 333 failed. 334 335 - Fix the possibility of an incorrect LZMA_BUF_ERROR when a BCJ 336 filter is used and the application only provides exactly as 337 much output space as is the uncompressed size of the file. 338 339 - Fix a bug in doc/examples_old/xz_pipe_decompress.c. It didn't 340 check if the last call to lzma_code() really returned 341 LZMA_STREAM_END, which made the program think that truncated 342 files are valid. 343 344 - New example programs in doc/examples (old programs are now in 345 doc/examples_old). These have more comments and more detailed 346 error handling. 347 348 * Fix "xz -lvv foo.xz". It could crash on some corrupted files. 349 350 * Fix output of "xz --robot -lv" and "xz --robot -lvv" which 351 incorrectly printed the filename also in the "foo (x/x)" format. 352 353 * Fix exit status of "xzdiff foo.xz bar.xz". 354 355 * Fix exit status of "xzgrep foo binary_file". 356 357 * Fix portability to EBCDIC systems. 358 359 * Fix a configure issue on AIX with the XL C compiler. See INSTALL 360 for details. 361 362 * Update French, German, Italian, and Polish translations. 363 364 3655.0.3 (2011-05-21) 366 367 * liblzma fixes: 368 369 - A memory leak was fixed. 370 371 - lzma_stream_buffer_encode() no longer creates an empty .xz 372 Block if encoding an empty buffer. Such an empty Block with 373 LZMA2 data would trigger a bug in 5.0.1 and older (see the 374 first bullet point in 5.0.2 notes). When releasing 5.0.2, 375 I thought that no encoder creates this kind of files but 376 I was wrong. 377 378 - Validate function arguments better in a few functions. Most 379 importantly, specifying an unsupported integrity check to 380 lzma_stream_buffer_encode() no longer creates a corrupt .xz 381 file. Probably no application tries to do that, so this 382 shouldn't be a big problem in practice. 383 384 - Document that lzma_block_buffer_encode(), 385 lzma_easy_buffer_encode(), lzma_stream_encoder(), and 386 lzma_stream_buffer_encode() may return LZMA_UNSUPPORTED_CHECK. 387 388 - The return values of the _memusage() functions are now 389 documented better. 390 391 * Fix command name detection in xzgrep. xzegrep and xzfgrep now 392 correctly use egrep and fgrep instead of grep. 393 394 * French translation was added. 395 396 3975.0.2 (2011-04-01) 398 399 * LZMA2 decompressor now correctly accepts LZMA2 streams with no 400 uncompressed data. Previously it considered them corrupt. The 401 bug can affect applications that use raw LZMA2 streams. It is 402 very unlikely to affect .xz files because no compressor creates 403 .xz files with empty LZMA2 streams. (Empty .xz files are a 404 different thing than empty LZMA2 streams.) 405 406 * "xz --suffix=.foo filename.foo" now refuses to compress the 407 file due to it already having the suffix .foo. It was already 408 documented on the man page, but the code lacked the test. 409 410 * "xzgrep -l foo bar.xz" works now. 411 412 * Polish translation was added. 413 414 4155.0.1 (2011-01-29) 416 417 * xz --force now (de)compresses files that have setuid, setgid, 418 or sticky bit set and files that have multiple hard links. 419 The man page had it documented this way already, but the code 420 had a bug. 421 422 * gzip and bzip2 support in xzdiff was fixed. 423 424 * Portability fixes 425 426 * Minor fix to Czech translation 427 428 4295.0.0 (2010-10-23) 430 431 Only the most important changes compared to 4.999.9beta are listed 432 here. One change is especially important: 433 434 * The memory usage limit is now disabled by default. Some scripts 435 written before this change may have used --memory=max on xz command 436 line or in XZ_OPT. THESE USES OF --memory=max SHOULD BE REMOVED 437 NOW, because they interfere with user's ability to set the memory 438 usage limit himself. If user-specified limit causes problems to 439 your script, blame the user. 440 441 Other significant changes: 442 443 * Added support for XZ_DEFAULTS environment variable. This variable 444 allows users to set default options for xz, e.g. default memory 445 usage limit or default compression level. Scripts that use xz 446 must never set or unset XZ_DEFAULTS. Scripts should use XZ_OPT 447 instead if they need a way to pass options to xz via an 448 environment variable. 449 450 * The compression settings associated with the preset levels 451 -0 ... -9 have been changed. --extreme was changed a little too. 452 It is now less likely to make compression worse, but with some 453 files the new --extreme may compress slightly worse than the old 454 --extreme. 455 456 * If a preset level (-0 ... -9) is specified after a custom filter 457 chain options have been used (e.g. --lzma2), the custom filter 458 chain will be forgotten. Earlier the preset options were 459 completely ignored after custom filter chain options had been 460 seen. 461 462 * xz will create sparse files when decompressing if the uncompressed 463 data contains long sequences of binary zeros. This is done even 464 when writing to standard output that is connected to a regular 465 file and certain additional conditions are met to make it safe. 466 467 * Support for "xz --list" was added. Combine with --verbose or 468 --verbose --verbose (-vv) for detailed output. 469 470 * I had hoped that liblzma API would have been stable after 471 4.999.9beta, but there have been a couple of changes in the 472 advanced features, which don't affect most applications: 473 474 - Index handling code was revised. If you were using the old 475 API, you will get a compiler error (so it's easy to notice). 476 477 - A subtle but important change was made to the Block handling 478 API. lzma_block.version has to be initialized even for 479 lzma_block_header_decode(). Code that doesn't do it will work 480 for now, but might break in the future, which makes this API 481 change easy to miss. 482 483 * The major soname has been bumped to 5.0.0. liblzma API and ABI 484 are now stable, so the need to recompile programs linking against 485 liblzma shouldn't arise soon. 486 487