xref: /netbsd-src/external/bsd/file/dist/doc/file.1 (revision 1b9578b8c2c1f848eeb16dabbfd7d1f0d9fdefbd)
1.\"	$NetBSD: file.1,v 1.7 2011/05/13 21:15:34 christos Exp $
2.\"
3.\" $File: file.man,v 1.94 2011/04/20 19:08:44 christos Exp $
4.Dd April 20, 2011
5.Dt FILE 1
6.Os
7.Sh NAME
8.Nm file
9.Nd determine file type
10.Sh SYNOPSIS
11.Nm
12.Bk -words
13.Op Fl bchiklLNnprsvz0
14.Op Fl Fl apple
15.Op Fl Fl mime-encoding
16.Op Fl Fl mime-type
17.Op Fl e Ar testname
18.Op Fl F Ar separator
19.Op Fl f Ar namefile
20.Op Fl m Ar magicfiles
21.Ar
22.Ek
23.Nm
24.Fl C
25.Op Fl m Ar magicfiles
26.Nm
27.Op Fl Fl help
28.Sh DESCRIPTION
29This manual page documents version 5.07 of the
30.Nm
31command.
32.Pp
33.Nm
34tests each argument in an attempt to classify it.
35There are three sets of tests, performed in this order:
36filesystem tests, magic tests, and language tests.
37The
38.Em first
39test that succeeds causes the file type to be printed.
40.Pp
41The type printed will usually contain one of the words
42.Em text
43(the file contains only
44printing characters and a few common control
45characters and is probably safe to read on an
46.Dv ASCII
47terminal),
48.Em executable
49(the file contains the result of compiling a program
50in a form understandable to some
51.Tn UNIX
52kernel or another),
53or
54.Em data
55meaning anything else (data is usually
56.Dq binary
57or non-printable).
58Exceptions are well-known file formats (core files, tar archives)
59that are known to contain binary data.
60When modifying magic files or the program itself, make sure to
61.Em "preserve these keywords" .
62Users depend on knowing that all the readable files in a directory
63have the word
64.Dq text
65printed.
66Don't do as Berkeley did and change
67.Dq shell commands text
68to
69.Dq shell script .
70.Pp
71The filesystem tests are based on examining the return from a
72.Xr stat 2
73system call.
74The program checks to see if the file is empty,
75or if it's some sort of special file.
76Any known file types appropriate to the system you are running on
77(sockets, symbolic links, or named pipes (FIFOs) on those systems that
78implement them)
79are intuited if they are defined in the system header file
80.In sys/stat.h .
81.Pp
82The magic tests are used to check for files with data in
83particular fixed formats.
84The canonical example of this is a binary executable (compiled program)
85.Dv a.out
86file, whose format is defined in
87.In elf.h ,
88.In a.out.h
89and possibly
90.In exec.h
91in the standard include directory.
92These files have a
93.Dq "magic number"
94stored in a particular place
95near the beginning of the file that tells the
96.Tn UNIX
97operating system
98that the file is a binary executable, and which of several types thereof.
99The concept of a
100.Dq "magic"
101has been applied by extension to data files.
102Any file with some invariant identifier at a small fixed
103offset into the file can usually be described in this way.
104The information identifying these files is read from the compiled
105magic file
106.Pa /usr/share/misc/magic.mgc ,
107or the files in the directory
108.Pa /usr/share/misc/magic
109if the compiled file does not exist.
110In addition, if
111.Pa $HOME/.magic.mgc
112or
113.Pa $HOME/.magic
114exists, it will be used in preference to the system magic files.
115.Pp
116If a file does not match any of the entries in the magic file,
117it is examined to see if it seems to be a text file.
118ASCII, ISO-8859-x, non-ISO 8-bit extended-ASCII character sets
119(such as those used on Macintosh and IBM PC systems),
120UTF-8-encoded Unicode, UTF-16-encoded Unicode, and EBCDIC
121character sets can be distinguished by the different
122ranges and sequences of bytes that constitute printable text
123in each set.
124If a file passes any of these tests, its character set is reported.
125ASCII, ISO-8859-x, UTF-8, and extended-ASCII files are identified
126as
127.Dq text
128because they will be mostly readable on nearly any terminal;
129UTF-16 and EBCDIC are only
130.Dq character data
131because, while
132they contain text, it is text that will require translation
133before it can be read.
134In addition,
135.Nm
136will attempt to determine other characteristics of text-type files.
137If the lines of a file are terminated by CR, CRLF, or NEL, instead
138of the Unix-standard LF, this will be reported.
139Files that contain embedded escape sequences or overstriking
140will also be identified.
141.Pp
142Once
143.Nm
144has determined the character set used in a text-type file,
145it will
146attempt to determine in what language the file is written.
147The language tests look for particular strings (cf.
148.In names.h )
149that can appear anywhere in the first few blocks of a file.
150For example, the keyword
151.Em .br
152indicates that the file is most likely a
153.Xr troff 1
154input file, just as the keyword
155.Em struct
156indicates a C program.
157These tests are less reliable than the previous
158two groups, so they are performed last.
159The language test routines also test for some miscellany
160(such as
161.Xr tar 1
162archives).
163.Pp
164Any file that cannot be identified as having been written
165in any of the character sets listed above is simply said to be
166.Dq data .
167.Sh OPTIONS
168.Bl -tag -width indent
169.It Fl b , Fl Fl brief
170Do not prepend filenames to output lines (brief mode).
171.It Fl C , Fl Fl compile
172Write a
173.Pa magic.mgc
174output file that contains a pre-parsed version of the magic file or directory.
175.It Fl c , Fl Fl checking-printout
176Cause a checking printout of the parsed form of the magic file.
177This is usually used in conjunction with the
178.Fl m
179flag to debug a new magic file before installing it.
180.It Fl e , Fl Fl exclude Ar testname
181Exclude the test named in
182.Ar testname
183from the list of tests made to determine the file type.
184Valid test names are:
185.Bl -tag -width compress
186.It apptype
187.Dv EMX
188application type (only on EMX).
189.It ascii
190Various types of text files (this test will try to guess the text
191encoding, irrespective of the setting of the
192.Sq encoding
193option).
194.It encoding
195Different text encodings for soft magic tests.
196.It tokens
197Looks for known tokens inside text files.
198.It cdf
199Prints details of Compound Document Files.
200.It compress
201Checks for, and looks inside, compressed files.
202.It elf
203Prints ELF file details.
204.It soft
205Consults magic files.
206.It tar
207Examines tar files.
208.El
209.It Fl F , Fl Fl separator Ar separator
210Use the specified string as the separator between the filename and the
211file result returned.
212Defaults to
213.Sq \&: .
214.It Fl f , Fl Fl files-from Ar namefile
215Read the names of the files to be examined from
216.Ar namefile
217(one per line)
218before the argument list.
219Either
220.Ar namefile
221or at least one filename argument must be present;
222to test the standard input, use
223.Sq -
224as a filename argument.
225.It Fl h , Fl Fl no-dereference
226option causes symlinks not to be followed
227(on systems that support symbolic links).
228This is the default if the environment variable
229.Dv POSIXLY_CORRECT
230is not defined.
231.It Fl i , Fl Fl mime
232Causes the file command to output mime type strings rather than the more
233traditional human readable ones.
234Thus it may say
235.Sq text/plain; charset=us-ascii
236rather than
237.Dq ASCII text .
238In order for this option to work,
239.Nm
240changes the way
241it handles files recognized by the command itself (such as many of the
242text file types, directories etc), and makes use of an alternative
243.Dq magic
244file.
245(See the
246.Sx FILES
247section, below).
248.It Fl Fl mime-type , Fl Fl mime-encoding
249Like
250.Fl i ,
251but print only the specified element(s).
252.It Fl k , Fl Fl keep-going
253Don't stop at the first match, keep going.
254Subsequent matches will be
255have the string
256.Sq "\[rs]012\- "
257prepended.
258(If you want a newline, see the
259.Fl r
260option.)
261.It Fl l , Fl Fl list
262Print information about the strength of each magic pattern.
263.It Fl L , Fl Fl dereference
264option causes symlinks to be followed, as the like-named option in
265.Xr ls 1
266(on systems that support symbolic links).
267This is the default if the environment variable
268.Ev POSIXLY_CORRECT
269is defined.
270.It Fl l
271Shows sorted patterns list in the order which is used for the matching.
272.It Fl m , Fl Fl magic-file Ar magicfiles
273Specify an alternate list of files and directories containing magic.
274This can be a single item, or a colon-separated list.
275If a compiled magic file is found alongside a file or directory,
276it will be used instead.
277.It Fl N , Fl Fl no-pad
278Don't pad filenames so that they align in the output.
279.It Fl n , Fl Fl no-buffer
280Force stdout to be flushed after checking each file.
281This is only useful if checking a list of files.
282It is intended to be used by programs that want filetype output from a pipe.
283.It Fl p , Fl Fl preserve-date
284On systems that support
285.Xr utime 3
286or
287.Xr utimes 2 ,
288attempt to preserve the access time of files analyzed, to pretend that
289.Nm
290never read them.
291.It Fl r , Fl Fl raw
292Don't translate unprintable characters to \eooo.
293Normally
294.Nm
295translates unprintable characters to their octal representation.
296.It Fl s , Fl Fl special-files
297Normally,
298.Nm
299only attempts to read and determine the type of argument files which
300.Xr stat 2
301reports are ordinary files.
302This prevents problems, because reading special files may have peculiar
303consequences.
304Specifying the
305.Fl s
306option causes
307.Nm
308to also read argument files which are block or character special files.
309This is useful for determining the filesystem types of the data in raw
310disk partitions, which are block special files.
311This option also causes
312.Nm
313to disregard the file size as reported by
314.Xr stat 2
315since on some systems it reports a zero size for raw disk partitions.
316.It Fl v , Fl Fl version
317Print the version of the program and exit.
318.It Fl z , Fl Fl uncompress
319Try to look inside compressed files.
320.It Fl 0 , Fl Fl print0
321Output a null character
322.Sq \e0
323after the end of the filename.
324Nice to
325.Xr cut 1
326the output.
327This does not affect the separator which is still printed.
328.It Fl -help
329Print a help message and exit.
330.El
331.Sh FILES
332.Bl -tag -width /usr/share/misc/magic.mgc -compact
333.It Pa /usr/share/misc/magic.mgc
334Default compiled list of magic.
335.It Pa /usr/share/misc/magic
336Directory containing default magic files.
337.El
338.Sh ENVIRONMENT
339The environment variable
340.Ev MAGIC
341can be used to set the default magic file name.
342If that variable is set, then
343.Nm
344will not attempt to open
345.Pa $HOME/.magic .
346.Nm
347adds
348.Dq Pa .mgc
349to the value of this variable as appropriate.
350However,
351.Pa file
352has to exist in order for
353.Pa file.mime
354to be considered.
355The environment variable
356.Ev POSIXLY_CORRECT
357controls (on systems that support symbolic links), whether
358.Nm
359will attempt to follow symlinks or not.
360If set, then
361.Nm
362follows symlink, otherwise it does not.
363This is also controlled by the
364.Fl L
365and
366.Fl h
367options.
368.Sh SEE ALSO
369.Xr hexdump 1 ,
370.Xr od 1 ,
371.Xr strings 1 ,
372.Xr od 1 ,
373.Xr hexdump 1 ,
374.Xr magic 5
375.Sh STANDARDS CONFORMANCE
376This program is believed to exceed the System V Interface Definition
377of FILE(CMD), as near as one can determine from the vague language
378contained therein.
379Its behavior is mostly compatible with the System V program of the same name.
380This version knows more magic, however, so it will produce
381different (albeit more accurate) output in many cases.
382.\" URL: http://www.opengroup.org/onlinepubs/009695399/utilities/file.html
383.Pp
384The one significant difference
385between this version and System V
386is that this version treats any white space
387as a delimiter, so that spaces in pattern strings must be escaped.
388For example,
389.Bd -literal -offset indent
390\*[Gt]10	string	language impress\ 	(imPRESS data)
391.Ed
392.Pp
393in an existing magic file would have to be changed to
394.Bd -literal -offset indent
395\*[Gt]10	string	language\e impress	(imPRESS data)
396.Ed
397.Pp
398In addition, in this version, if a pattern string contains a backslash,
399it must be escaped.
400For example
401.Bd -literal -offset indent
4020	string		\ebegindata	Andrew Toolkit document
403.Ed
404.Pp
405in an existing magic file would have to be changed to
406.Bd -literal -offset indent
4070	string		\e\ebegindata	Andrew Toolkit document
408.Ed
409.Pp
410SunOS releases 3.2 and later from Sun Microsystems include a
411.Nm
412command derived from the System V one, but with some extensions.
413This version differs from Sun's only in minor ways.
414It includes the extension of the
415.Sq \*[Am]
416operator, used as,
417for example,
418.Bd -literal -offset indent
419\*[Gt]16	long\*[Am]0x7fffffff	\*[Gt]0		not stripped
420.Ed
421.Sh MAGIC DIRECTORY
422The magic file entries have been collected from various sources,
423mainly USENET, and contributed by various authors.
424Christos Zoulas (address below) will collect additional
425or corrected magic file entries.
426A consolidation of magic file entries
427will be distributed periodically.
428.Pp
429The order of entries in the magic file is significant.
430Depending on what system you are using, the order that
431they are put together may be incorrect.
432If your old
433.Nm
434command uses a magic file,
435keep the old magic file around for comparison purposes
436(rename it to
437.Pa /usr/share/misc/magic.orig ) .
438.Sh EXAMPLES
439.Bd -literal -offset indent
440$ file file.c file /dev/{wd0a,hda}
441file.c:   C program text
442file:     ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV),
443          dynamically linked (uses shared libs), stripped
444/dev/wd0a: block special (0/0)
445/dev/hda: block special (3/0)
446
447$ file -s /dev/wd0{b,d}
448/dev/wd0b: data
449/dev/wd0d: x86 boot sector
450
451$ file -s /dev/hda{,1,2,3,4,5,6,7,8,9,10}
452/dev/hda:   x86 boot sector
453/dev/hda1:  Linux/i386 ext2 filesystem
454/dev/hda2:  x86 boot sector
455/dev/hda3:  x86 boot sector, extended partition table
456/dev/hda4:  Linux/i386 ext2 filesystem
457/dev/hda5:  Linux/i386 swap file
458/dev/hda6:  Linux/i386 swap file
459/dev/hda7:  Linux/i386 swap file
460/dev/hda8:  Linux/i386 swap file
461/dev/hda9:  empty
462/dev/hda10: empty
463
464$ file -i file.c file /dev/{wd0a,hda}
465file.c:      text/x-c
466file:        application/x-executable
467/dev/hda:    application/x-not-regular-file
468/dev/wd0a:   application/x-not-regular-file
469
470.Ed
471.Sh HISTORY
472There has been a
473.Nm
474command in every
475.Dv UNIX since at least Research Version 4
476(man page dated November, 1973).
477The System V version introduced one significant major change:
478the external list of magic types.
479This slowed the program down slightly but made it a lot more flexible.
480.Pp
481This program, based on the System V version,
482was written by Ian Darwin
483.Aq ian@darwinsys.com
484without looking at anybody else's source code.
485.Pp
486John Gilmore revised the code extensively, making it better than
487the first version.
488Geoff Collyer found several inadequacies
489and provided some magic file entries.
490Contributions by the
491.Sq \*[Am]
492operator by Rob McMahon,
493.Aq cudcv@warwick.ac.uk ,
4941989.
495.Pp
496Guy Harris,
497.Aq guy@netapp.com ,
498made many changes from 1993 to the present.
4991989.
500.Pp
501Primary development and maintenance from 1990 to the present by
502Christos Zoulas
503.Aq christos@astron.com .
504.Pp
505Altered by Chris Lowth
506.Aq chris@lowth.com ,
5072000: handle the
508.Fl i
509option to output mime type strings, using an alternative
510magic file and internal logic.
511.Pp
512Altered by Eric Fischer
513.Aq enf@pobox.com ,
514July, 2000,
515to identify character codes and attempt to identify the languages
516of non-ASCII files.
517.Pp
518Altered by Reuben Thomas
519.Aq rrt@sc3d.org ,
5202007-2011, to improve MIME support, merge MIME and non-MIME magic,
521support directories as well as files of magic, apply many bug fixes,
522update and fix a lot of magic, improve the build system, improve the
523documentation, and rewrite the Python bindings in pure Python.
524.Pp
525The list of contributors to the
526.Sq magic
527directory (magic files)
528is too long to include here.
529You know who you are; thank you.
530Many contributors are listed in the source files.
531.Sh LEGAL NOTICE
532Copyright (c) Ian F. Darwin, Toronto, Canada, 1986-1999.
533Covered by the standard Berkeley Software Distribution copyright; see the file
534COPYING in the source distribution.
535.Pp
536The files
537.Pa tar.h
538and
539.Pa is_tar.c
540were written by John Gilmore from his public-domain
541.Xr tar 1
542program, and are not covered by the above license.
543.Sh RETURN CODE
544.Nm
545returns 0 on success, and non-zero on error.
546.Sh BUGS
547.Pp
548Please report bugs and send patches to the bug tracker at
549.Pa http://bugs.gw.com/
550or the mailing list at
551.Aq file@mx.gw.com .
552.Sh TODO
553.Pp
554Fix output so that tests for MIME and APPLE flags are not needed all
555over the place, and actual output is only done in one place. This
556needs a design. Suggestion: push possible outputs on to a list, then
557pick the last-pushed (most specific, one hopes) value at the end, or
558use a default if the list is empty. This should not slow down evaluation.
559.Pp
560Continue to squash all magic bugs. See Debian BTS for a good source.
561.Pp
562Store arbitrarily long strings, for example for %s patterns, so that
563they can be printed out. Fixes Debian bug #271672. Would require more
564complex store/load code in apprentice.
565.Pp
566Add syntax for relative offsets after current level (Debian bug #466037).
567.Pp
568Make file -ki work, i.e. give multiple MIME types.
569.Pp
570Add a zip library so we can peek inside Office2007 documents to
571figure out what they are.
572.Pp
573Don't complain when ~/.magic is not compiled.
574.Pp
575Add an option to print URLs for the sources of the file descriptions.
576.Sh AVAILABILITY
577You can obtain the original author's latest version by anonymous FTP
578on
579.Pa ftp.astron.com
580in the directory
581.Pa /pub/file/file-X.YZ.tar.gz .
582