1.\" $OpenBSD: re_format.7,v 1.9 2000/04/18 03:01:33 aaron Exp $ 2.\" 3.\" Copyright (c) 1997, Phillip F Knaack. All rights reserved. 4.\" 5.\" Copyright (c) 1992, 1993, 1994 Henry Spencer. 6.\" Copyright (c) 1992, 1993, 1994 7.\" The Regents of the University of California. All rights reserved. 8.\" 9.\" This code is derived from software contributed to Berkeley by 10.\" Henry Spencer. 11.\" 12.\" Redistribution and use in source and binary forms, with or without 13.\" modification, are permitted provided that the following conditions 14.\" are met: 15.\" 1. Redistributions of source code must retain the above copyright 16.\" notice, this list of conditions and the following disclaimer. 17.\" 2. Redistributions in binary form must reproduce the above copyright 18.\" notice, this list of conditions and the following disclaimer in the 19.\" documentation and/or other materials provided with the distribution. 20.\" 3. All advertising materials mentioning features or use of this software 21.\" must display the following acknowledgement: 22.\" This product includes software developed by the University of 23.\" California, Berkeley and its contributors. 24.\" 4. Neither the name of the University nor the names of its contributors 25.\" may be used to endorse or promote products derived from this software 26.\" without specific prior written permission. 27.\" 28.\" THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND 29.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE 30.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE 31.\" ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE 32.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL 33.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS 34.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) 35.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT 36.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY 37.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF 38.\" SUCH DAMAGE. 39.\" 40.\" @(#)re_format.7 8.3 (Berkeley) 3/20/94 41.\" 42.Dd March 20, 1994 43.Dt RE_FORMAT 7 44.Os 45.Sh NAME 46.Nm re_format 47.Nd POSIX 1003.2 regular expressions 48.Sh DESCRIPTION 49Regular expressions (``RE''s), 50as defined in POSIX 1003.2, come in two forms: 51modern REs (roughly those of 52.Xr egrep 1 ; 531003.2 calls these ``extended'' REs) 54and obsolete REs (roughly those of 55.Xr ed 1 ; 561003.2 ``basic'' REs). 57Obsolete REs mostly exist for backward compatibility in some old programs; 58they will be discussed at the end. 591003.2 leaves some aspects of RE syntax and semantics open; 60`\(dg' marks decisions on these aspects that 61may not be fully portable to other 1003.2 implementations. 62.Pp 63A (modern) RE is one\(dg or more non-empty\(dg 64.Em branches , 65separated by `|'. 66It matches anything that matches one of the branches. 67.Pp 68A branch is one\(dg or more 69.Em pieces , 70concatenated. 71It matches a match for the first, followed by a match for the second, etc. 72.Pp 73A piece is an 74.Em atom 75possibly followed by a single\(dg `*', `+', `?', or 76.Em bound . 77An atom followed by `*' matches a sequence of 0 or more matches of the atom. 78An atom followed by `+' matches a sequence of 1 or more matches of the atom. 79An atom followed by `?' matches a sequence of 0 or 1 matches of the atom. 80.Pp 81A 82.Em bound 83is `{' followed by an unsigned decimal integer, 84possibly followed by `,' 85possibly followed by another unsigned decimal integer, 86always followed by `}'. 87The integers must lie between 0 and RE_DUP_MAX (255\(dg) inclusive, 88and if there are two of them, the first may not exceed the second. 89An atom followed by a bound containing one integer \fIi\fR 90and no comma matches 91a sequence of exactly \fIi\fR matches of the atom. 92An atom followed by a bound 93containing one integer \fIi\fR and a comma matches 94a sequence of \fIi\fR or more matches of the atom. 95An atom followed by a bound 96containing two integers \fIi\fR and \fIj\fR matches 97a sequence of \fIi\fR through \fIj\fR (inclusive) matches of the atom. 98.Pp 99An 100.Em atom 101is a regular expression enclosed in `()' 102(matching a match for the regular expression), 103an empty set of `()' (matching the null string)\(dg, 104a 105.Em "bracket expression" 106(see below), `.' 107(matching any single character), `^' (matching the null string at the 108beginning of a line), `$' (matching the null string at the 109end of a line), a `\e' followed by one of the characters 110`^.[$()|*+?{\e' 111(matching that character taken as an ordinary character), 112a `\e' followed by any other character\(dg 113(matching that character taken as an ordinary character, 114as if the `\e' had not been present\(dg), 115or a single character with no other significance (matching that character). 116A `{' followed by a character other than a digit is an ordinary 117character, not the beginning of a bound\(dg. 118It is illegal to end an RE with `\e'. 119.Pp 120A 121.Em "bracket expression" 122is a list of characters enclosed in `[]'. 123It normally matches any single character from the list (but see below). 124If the list begins with `^', 125it matches any single character 126(but see below) 127.Em not 128from the rest of the list. 129If two characters in the list are separated by `\-', this is shorthand 130for the full 131.Em range 132of characters between those two (inclusive) in the 133collating sequence, 134e.g., `[0-9]' in ASCII matches any decimal digit. 135It is illegal\(dg for two ranges to share an 136endpoint, e.g., `a-c-e'. 137Ranges are very collating-sequence-dependent, 138and portable programs should avoid relying on them. 139.Pp 140To include a literal `]' in the list, make it the first character 141(following a possible `^'). 142To include a literal `\-', make it the first or last character, 143or the second endpoint of a range. 144To use a literal `\-' as the first endpoint of a range, 145enclose it in `[.' and `.]' to make it a collating element (see below). 146With the exception of these and some combinations using `[' (see next 147paragraphs), all other special characters, including `\e', lose their 148special significance within a bracket expression. 149.Pp 150Within a bracket expression, a collating element (a character, 151a multi-character sequence that collates as if it were a single character, 152or a collating-sequence name for either) 153enclosed in `[.' and `.]' stands for the 154sequence of characters of that collating element. 155The sequence is a single element of the bracket expression's list. 156A bracket expression containing a multi-character collating element 157can thus match more than one character, 158e.g., if the collating sequence includes a `ch' collating element, 159then the RE `[[.ch.]]*c' matches the first five characters 160of `chchcc'. 161.Pp 162Within a bracket expression, a collating element enclosed in `[=' and 163`=]' is an equivalence class, standing for the sequences of characters 164of all collating elements equivalent to that one, including itself. 165(If there are no other equivalent collating elements, 166the treatment is as if the enclosing delimiters were `[.' and `.]'.) 167For example, if o and \o'o^' are the members of an equivalence class, 168then `[[=o=]]', `[[=\o'o^'=]]', and `[o\o'o^']' are all synonymous. 169An equivalence class may not\(dg be an endpoint 170of a range. 171.Pp 172Within a bracket expression, the name of a 173.Em "character class" 174enclosed 175in `[:' and `:]' stands for the list of all characters belonging to that 176class. 177Standard character class names are: 178.Pp 179.Bl -item -compact -offset indent 180.It 181alnum digit punct 182.It 183alpha graph space 184.It 185blank lower upper 186.It 187cntrl print xdigit 188.El 189.Pp 190These stand for the character classes defined in 191.Xr ctype 3 . 192A locale may provide others. 193A character class may not be used as an endpoint of a range. 194.Pp 195There are two special cases\(dg of bracket expressions: 196the bracket expressions `[[:<:]]' and `[[:>:]]' match the null string at 197the beginning and end of a word respectively. 198A word is defined as a sequence of 199word characters 200which is neither preceded nor followed by 201word characters. 202A word character is an 203.Em alnum 204character (as defined by 205.Xr ctype 3 ) 206or an underscore. 207This is an extension, 208compatible with but not specified by POSIX 1003.2, 209and should be used with 210caution in software intended to be portable to other systems. 211.Pp 212In the event that an RE could match more than one substring of a given 213string, 214the RE matches the one starting earliest in the string. 215If the RE could match more than one substring starting at that point, 216it matches the longest. 217Subexpressions also match the longest possible substrings, subject to 218the constraint that the whole match be as long as possible, 219with subexpressions starting earlier in the RE taking priority over 220ones starting later. 221Note that higher-level subexpressions thus take priority over 222their lower-level component subexpressions. 223.Pp 224Match lengths are measured in characters, not collating elements. 225A null string is considered longer than no match at all. 226For example, 227`bb*' matches the three middle characters of `abbbc', 228`(wee|week)(knights|nights)' matches all ten characters of `weeknights', 229when `(.*).*' is matched against `abc' the parenthesized subexpression 230matches all three characters, and 231when `(a*)*' is matched against `bc' both the whole RE and the parenthesized 232subexpression match the null string. 233.Pp 234If case-independent matching is specified, 235the effect is much as if all case distinctions had vanished from the 236alphabet. 237When an alphabetic that exists in multiple cases appears as an 238ordinary character outside a bracket expression, it is effectively 239transformed into a bracket expression containing both cases, 240e.g., `x' becomes `[xX]'. 241When it appears inside a bracket expression, all case counterparts 242of it are added to the bracket expression, so that (e.g.) `[x]' 243becomes `[xX]' and `[^x]' becomes `[^xX]'. 244.Pp 245No particular limit is imposed on the length of REs\(dg. 246Programs intended to be portable should not employ REs longer 247than 256 bytes, 248as an implementation can refuse to accept such REs and remain 249POSIX-compliant. 250.Pp 251Obsolete (``basic'') regular expressions differ in several respects. 252`|', `+', and `?' are ordinary characters and there is no equivalent 253for their functionality. 254The delimiters for bounds are `\e{' and `\e}', 255with `{' and `}' by themselves ordinary characters. 256The parentheses for nested subexpressions are `\e(' and `\e)', 257with `(' and `)' by themselves ordinary characters. 258`^' is an ordinary character except at the beginning of the 259RE or\(dg the beginning of a parenthesized subexpression, 260`$' is an ordinary character except at the end of the 261RE or\(dg the end of a parenthesized subexpression, 262and `*' is an ordinary character if it appears at the beginning of the 263RE or the beginning of a parenthesized subexpression 264(after a possible leading `^'). 265Finally, there is one new type of atom, a 266.Em "back reference" : 267`\e' followed by a non-zero decimal digit 268.Em d 269matches the same sequence of characters 270matched by the 271.Em d Ns th 272parenthesized subexpression 273(numbering subexpressions by the positions of their opening parentheses, 274left to right), 275so that (e.g.) `\e([bc]\e)\e1' matches `bb' or `cc' but not `bc'. 276.Sh SEE ALSO 277.Xr regex 3 278.Pp 279POSIX 1003.2, section 2.8 (Regular Expression Notation). 280.Sh BUGS 281Having two kinds of REs is a botch. 282.Pp 283The current 1003.2 spec says that `)' is an ordinary character in 284the absence of an unmatched `('; 285this was an unintentional result of a wording error, 286and change is likely. 287Avoid relying on it. 288.Pp 289Back references are a dreadful botch, 290posing major problems for efficient implementations. 291They are also somewhat vaguely defined 292(does 293`a\e(\e(b\e)*\e2\e)*d' match `abbbd'?). 294Avoid using them. 295.Pp 2961003.2's specification of case-independent matching is vague. 297The ``one case implies all cases'' definition given above 298is current consensus among implementors as to the right interpretation. 299.Pp 300The syntax for word boundaries is incredibly ugly. 301