1.\" $NetBSD: vndcompress.1,v 1.14 2014/01/22 06:18:17 riastradh Exp $ 2.\" 3.\" Copyright (c) 2013 The NetBSD Foundation, Inc. 4.\" All rights reserved. 5.\" 6.\" This documentation is derived from text contributed to The NetBSD 7.\" Foundation by Taylor R. Campbell. 8.\" 9.\" Redistribution and use in source and binary forms, with or without 10.\" modification, are permitted provided that the following conditions 11.\" are met: 12.\" 1. Redistributions of source code must retain the above copyright 13.\" notice, this list of conditions and the following disclaimer. 14.\" 2. Redistributions in binary form must reproduce the above copyright 15.\" notice, this list of conditions and the following disclaimer in the 16.\" documentation and/or other materials provided with the distribution. 17.\" 18.\" THIS SOFTWARE IS PROVIDED BY THE NETBSD FOUNDATION, INC. AND CONTRIBUTORS 19.\" ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED 20.\" TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR 21.\" PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE FOUNDATION OR CONTRIBUTORS 22.\" BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR 23.\" CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF 24.\" SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS 25.\" INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN 26.\" CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) 27.\" ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE 28.\" POSSIBILITY OF SUCH DAMAGE. 29.\" 30.Dd January 21, 2014 31.Dt VNDCOMPRESS 1 32.Os 33.Sh NAME 34.Nm vndcompress , 35.Nm vnduncompress 36.Nd compress and uncompress disk images in cloop2 format 37.Sh SYNOPSIS 38.Nm 39.Op Fl c 40.Op Fl rR 41.Op Fl b Ar blocksize 42.Op Fl k Ar checkpoint-blocks 43.Op Fl l Ar length 44.Op Fl p Ar partial-offset 45.Op Fl w Ar winsize 46.Ar image 47.Ar compressed-image 48.Op Ar blocksize 49.Nm vnduncompress 50.Op Fl d 51.Op Fl w Ar winsize 52.Ar compressed-image 53.Ar image 54.Sh DESCRIPTION 55The 56.Nm 57utility compresses disk images in cloop2 format, which the kernel's 58.Xr vnd 4 59device can interpret as read-only disk devices using the 60.Fl z 61option to 62.Xr vnconfig 8 . 63.Pp 64By default, 65.Nm vndcompress 66compresses an image, and 67.Nm vnduncompress 68uncompresses an image, but the 69.Fl c 70and 71.Fl d 72options can control whether either utility compresses or decompresses. 73.Pp 74The following options are available for the compression operation: 75.Bl -tag -width indent 76.It Fl b Ar blocksize 77Set the compression block size to 78.Ar blocksize , 79which must be a multiple of 512 and must be no more than 4294966784, or 800xfffffe00. 81(On 32-bit systems, the limit may be smaller, limited by the available 82virtual address space.) 83.Pp 84For compatibility with the old version of 85.Nm , 86the compression block size may instead be specified at the end of the 87command line. 88.It Fl k Ar checkpoint-blocks 89Write a checkpoint after every 90.Ar checkpoint-blocks 91blocks of output. 92If interrupted, 93.Nm 94can restart at the last checkpoint with the 95.Fl r 96option. 97You can also request a checkpoint at any time by sending 98.Dv SIGUSR2 99to the 100.Nm 101process. 102.It Fl l Ar length 103Specify the length of the input, so that the input may be a pipe, 104socket, or FIFO. 105Otherwise, the input must know its size, i.e. must have its size 106reported in 107.Fa st_size 108by 109.Xr fstat 2 . 110.It Fl p Ar partial-blocks 111Stop after writing 112.Ar partial-blocks 113blocks of output. 114This option is mainly useful for automatic testing of restarts. 115.It Fl R 116With the 117.Fl r 118option, if restarting fails, then abort right now and don't touch the 119output file. 120.It Fl r 121Try to restart a partial compression from the last checkpoint. 122If restarting fails, and the 123.Fl R 124option is not specified, then truncate the output file and start 125compression afresh. 126Restarting may fail for various reasons: if there have been no 127checkpoints, or if the output file has been corrupted in some easily 128recognizable ways, or if the input file's size has changed, or if the 129block size does not match, and so on. 130.El 131.Pp 132The following option is available for both compression and 133decompression: 134.Bl -tag -width indent 135.It Fl w Ar winsize 136Use an in-memory window of 137.Ar winsize 138entries into the table of compressed block offsets. 139If 140.Ar winsize 141is zero, 142.Nm 143will use memory proportional to the number of blocks in the 144uncompressed image, namely 64 bits or 8 bytes per block. 145If 146.Ar winsize 147is nonzero, 148.Nm 149will use memory proportional to 150.Ar winsize , 151and independent of the size of the uncompressed image. 152.Pp 153A nonzero 154.Ar winsize 155requires the compressed image to be a seekable file, which compression 156requires anyway, in order to record the offsets of compressed blocks 157once they are compressed and written, but which is a limitation for 158decompression. 159Thus, decompressing from a pipe is incompatible with a nonzero 160.Ar winsize . 161.Pp 162By default, 163.Nm 164uses a fixed window size, unless decompressing with nonseekable input. 165.El 166.Sh EXIT STATUS 167.Ex -std 168.Sh EXAMPLES 169Compress an ISO 9660 CD-ROM image: 170.Bd -literal -offset indent 171# vndcompress cdrom.iso cdrom.izo 172.Ed 173.Pp 174Send a 59 GB disk partition over a local network with netcat (don't do 175this over the internet!): 176.Bd -literal -offset indent 177# nc 10.0.1.2 12345 < /dev/rcgd1h 178.Ed 179.Pp 180Receive it and save it compressed on another host, showing a progress 181bar interactively, restarting if possible, and taking a checkpoint 182every 16 MB (i.e., every 256 compression blocks, which are 64 KB by 183default): 184.Bd -literal -offset indent 185# nc -l 12345 | progress -l 59g \e 186 vndcompress -l 59g -k 256 -r /dev/stdin disk.cloop2 187.Ed 188.Pp 189If the process is interrupted and you rewire your network and disks so 190that the receiving host now has the disk you want to image, you can 191start up where you left off, using the 192.Fl R 193option to keep 194.Nm 195from clobbering the partial transfer if anything goes wrong: 196.Bd -literal -offset indent 197# vndcompress -l 59g -k 256 -rR /dev/rcgd1h disk.cloop2 198.Ed 199.Pp 200Mount the disk with 201.Xr vnd 4 , 202assuming your kernel was built with the 203.Dv VND_COMPRESSION 204option enabled: 205.Bd -literal -offset indent 206# vnconfig -z vnd0 disk.cloop2 207# mount /dev/vnd0d /mnt 208.Ed 209.Sh SIGNALS 210.Nm 211responds to the following signals: 212.Bl -tag -width indent 213.It Dv SIGINFO , Dv SIGUSR1 214Report progress to standard error. 215.It Dv SIGUSR2 216Write a checkpoint to disk now. 217.El 218.Sh FORMAT 219The cloop2 format consists of a header, an offset table, and a sequence 220of compressed blocks. 221The header is described by the following packed structure: 222.Bd -literal -offset indent 223struct cloop2_header { 224 char cl2h_magic[128]; 225 uint32_t cl2h_blocksize; 226 uint32_t cl2h_n_blocks; 227} __packed; 228.Ed 229.Pp 230The 231.Fa cl2h_magic 232field is an arbitrary sequence of 128 bits, the 233.Fa cl2h_blocksize 234field is a big-endian integer number of bytes per compression block, 235and the 236.Fa cl2h_n_blocks 237field is a big-endian integer number of the compression blocks in the 238image. 239.Pp 240The offset table is a sequence of one more than 241.Fa cl2h_n_blocks 242big-endian 64-bit integers specifying the offset of each compression 243block relative to the start of the file. 244The extra offset specifies the end of the last compression block, which 245may be truncated if the uncompressed image's size is not a multiple of 246the compression block size. 247.Sh SEE ALSO 248.Xr vnd 4 , 249.Xr vnconfig 8 250.Sh HISTORY 251The 252.Nm 253command first appeared in 254.Nx 3.0 . 255It was rewritten to be more robust, to support restarting partial 256transfers, and to support bounded memory usage in 257.Nx 7.0 . 258