1.\" $NetBSD: vndcompress.1,v 1.10 2013/05/04 14:34:13 riastradh Exp $ 2.\" 3.\" Copyright (c) 2013 The NetBSD Foundation, Inc. 4.\" All rights reserved. 5.\" 6.\" This documentation is derived from text contributed to The NetBSD 7.\" Foundation by Taylor R. Campbell. 8.\" 9.\" Redistribution and use in source and binary forms, with or without 10.\" modification, are permitted provided that the following conditions 11.\" are met: 12.\" 1. Redistributions of source code must retain the above copyright 13.\" notice, this list of conditions and the following disclaimer. 14.\" 2. Redistributions in binary form must reproduce the above copyright 15.\" notice, this list of conditions and the following disclaimer in the 16.\" documentation and/or other materials provided with the distribution. 17.\" 18.\" THIS SOFTWARE IS PROVIDED BY THE NETBSD FOUNDATION, INC. AND CONTRIBUTORS 19.\" ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED 20.\" TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR 21.\" PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE FOUNDATION OR CONTRIBUTORS 22.\" BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR 23.\" CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF 24.\" SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS 25.\" INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN 26.\" CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) 27.\" ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE 28.\" POSSIBILITY OF SUCH DAMAGE. 29.\" 30.Dd April 27, 2013 31.Dt VNDCOMPRESS 1 32.Os 33.Sh NAME 34.Nm vndcompress , 35.Nm vnduncompress 36.Nd compress and uncompress disk images in cloop2 format 37.Sh SYNOPSIS 38.Nm 39.Op Fl c 40.Op Fl rR 41.Op Fl k Ar checkpoint-blocks 42.Op Fl l Ar length 43.Op Fl p Ar partial-offset 44.Op Fl s Ar blocksize 45.Ar image 46.Ar compressed-image 47.Op Ar blocksize 48.Nm vnduncompress 49.Op Fl d 50.Ar compressed-image 51.Ar image 52.Sh DESCRIPTION 53The 54.Nm 55utility compresses disk images in cloop2 format, which the kernel's 56.Xr vnd 4 57device can interpret as read-only disk devices using the 58.Fl z 59option to 60.Xr vnconfig 8 . 61.Pp 62By default, 63.Nm vndcompress 64compresses an image, and 65.Nm vnduncompress 66uncompresses an image, but the 67.Fl c 68and 69.Fl d 70options can control whether either utility compresses or decompresses. 71.Pp 72The following options are available for the compression operation: 73.Bl -tag -width indent 74.It Fl k Ar checkpoint-blocks 75Write a checkpoint after every 76.Ar checkpoint-blocks 77blocks of output. 78If interrupted, 79.Nm 80can restart at the last checkpoint with the 81.Fl r 82option. 83You can also request a checkpoint at any time by sending 84.Dv SIGUSR2 85to the 86.Nm 87process. 88.It Fl l Ar length 89Specify the length of the input, so that the input may be a pipe, 90socket, or FIFO. 91Otherwise, the input must know its size, i.e. must have its size 92reported in 93.Fa st_size 94by 95.Xr fstat 2 . 96.It Fl p Ar partial-blocks 97Stop after writing 98.Ar partial-blocks 99blocks of output. 100This option is mainly useful for automatic testing of restarts. 101.It Fl R 102With the 103.Fl r 104option, if restarting fails, then abort right now and don't touch the 105output file. 106.It Fl r 107Try to restart a partial compression from the last checkpoint. 108If restarting fails, and the 109.Fl R 110option is not specified, then truncate the output file and start 111compression afresh. 112Restarting may fail for various reasons: if there have been no 113checkpoints, or if the output file has been corrupted in some easily 114recognizable ways, or if the input file's size has changed, or if the 115block size does not match, and so on. 116.It Fl s Ar blocksize 117Set the compression block size to 118.Ar blocksize , 119which must be a multiple of 512 and must be no more than 4294966784, or 1200xfffffe00. 121(On 32-bit systems, the limit may be smaller, limited by the available 122virtual address space.) 123.Pp 124For compatibility with the old version of 125.Nm , 126the compression block size may instead be specified at the end of the 127command line. 128.El 129.Sh EXIT STATUS 130.Ex -std 131.Sh EXAMPLES 132Compress an ISO 9660 CD-ROM image: 133.Bd -literal -offset indent 134# vndcompress cdrom.iso cdrom.izo 135.Ed 136.Pp 137Send a 59 GB disk partition over a local network with netcat (don't do 138this over the internet!): 139.Bd -literal -offset indent 140# nc 10.0.1.2 12345 < /dev/rcgd1h 141.Ed 142.Pp 143Receive it and save it compressed on another host, showing a progress 144bar interactively, restarting if possible, and taking a checkpoint 145every 16 MB (i.e., every 256 compression blocks, which are 64 KB by 146default): 147.Bd -literal -offset indent 148# nc -l 12345 | progress -l 59g \e 149 vndcompress -l 59g -k 256 -r /dev/stdin disk.cloop2 150.Ed 151.Pp 152If the process is interrupted and you rewire your network and disks so 153that the receiving host now has the disk you want to image, you can 154start up where you left off, using the 155.Fl R 156option to keep 157.Nm 158from clobbering the partial transfer if anything goes wrong: 159.Bd -literal -offset indent 160# vndcompress -l 59g -k 256 -rR /dev/rcgd1h disk.cloop2 161.Ed 162.Pp 163Mount the disk with 164.Xr vnd 4 , 165assuming your kernel was built with the 166.Dv VND_COMPRESSION 167option enabled: 168.Bd -literal -offset indent 169# vnconfig -z vnd0 disk.cloop2 170# mount /dev/vnd0d /mnt 171.Ed 172.Sh SIGNALS 173.Nm 174responds to the following signals: 175.Bl -tag -width indent 176.It Dv SIGINFO , Dv SIGUSR1 177Report progress to standard error. 178.It Dv SIGUSR2 179Write a checkpoint to disk now. 180.El 181.Sh FORMAT 182The cloop2 format consists of a header, an offset table, and a sequence 183of compressed blocks. 184The header is described by the following packed structure: 185.Bd -literal -offset indent 186struct cloop2_header { 187 char cl2h_magic[128]; 188 uint32_t cl2h_blocksize; 189 uint32_t cl2h_n_blocks; 190} __packed; 191.Ed 192.Pp 193The 194.Fa cl2h_magic 195field is an arbitrary sequence of 128 bits, the 196.Fa cl2h_blocksize 197field is a big-endian integer number of bytes per compression block, 198and the 199.Fa cl2h_n_blocks 200field is a big-endian integer number of the compression blocks in the 201image. 202.Pp 203The offset table is a sequence of one more than 204.Fa cl2h_n_blocks 205big-endian 64-bit integers specifying the offset of each compression 206block relative to the start of the file. 207The extra offset specifies the end of the last compression block, which 208may be truncated if the uncompressed image's size is not a multiple of 209the compression block size. 210.Sh SEE ALSO 211.Xr vnd 4 , 212.Xr vnconfig 8 213.Sh HISTORY 214The 215.Nm 216command first appeared in 217.Nx 3.0 . 218It was rewritten to be more robust and to support restarting partial 219transfers in 220.Nx 7.0 . 221.Sh BUGS 222The amount of memory consumed by 223.Nm 224and 225.Nm vnduncompress 226is proportional to the number of compression blocks they process, 227because they store the whole offset table in memory at once. 228Instead, they should use a limited-size window into the offset table. 229