1=head1 NAME 2X<regular expression> X<regex> X<regexp> 3 4perlre - Perl regular expressions 5 6=head1 DESCRIPTION 7 8This page describes the syntax of regular expressions in Perl. 9 10If you haven't used regular expressions before, a quick-start 11introduction is available in L<perlrequick>, and a longer tutorial 12introduction is available in L<perlretut>. 13 14For reference on how regular expressions are used in matching 15operations, plus various examples of the same, see discussions of 16C<m//>, C<s///>, C<qr//> and C<??> in L<perlop/"Regexp Quote-Like 17Operators">. 18 19 20=head2 Modifiers 21 22Matching operations can have various modifiers. Modifiers 23that relate to the interpretation of the regular expression inside 24are listed below. Modifiers that alter the way a regular expression 25is used by Perl are detailed in L<perlop/"Regexp Quote-Like Operators"> and 26L<perlop/"Gory details of parsing quoted constructs">. 27 28=over 4 29 30=item m 31X</m> X<regex, multiline> X<regexp, multiline> X<regular expression, multiline> 32 33Treat string as multiple lines. That is, change "^" and "$" from matching 34the start of the string's first line and the end of its last line to 35matching the start and end of each line within the string. 36 37=item s 38X</s> X<regex, single-line> X<regexp, single-line> 39X<regular expression, single-line> 40 41Treat string as single line. That is, change "." to match any character 42whatsoever, even a newline, which normally it would not match. 43 44Used together, as C</ms>, they let the "." match any character whatsoever, 45while still allowing "^" and "$" to match, respectively, just after 46and just before newlines within the string. 47 48=item i 49X</i> X<regex, case-insensitive> X<regexp, case-insensitive> 50X<regular expression, case-insensitive> 51 52Do case-insensitive pattern matching. 53 54If locale matching rules are in effect, the case map is taken from the 55current 56locale for code points less than 255, and from Unicode rules for larger 57code points. However, matches that would cross the Unicode 58rules/non-Unicode rules boundary (ords 255/256) will not succeed. See 59L<perllocale>. 60 61There are a number of Unicode characters that match multiple characters 62under C</i>. For example, C<LATIN SMALL LIGATURE FI> 63should match the sequence C<fi>. Perl is not 64currently able to do this when the multiple characters are in the pattern and 65are split between groupings, or when one or more are quantified. Thus 66 67 "\N{LATIN SMALL LIGATURE FI}" =~ /fi/i; # Matches 68 "\N{LATIN SMALL LIGATURE FI}" =~ /[fi][fi]/i; # Doesn't match! 69 "\N{LATIN SMALL LIGATURE FI}" =~ /fi*/i; # Doesn't match! 70 71 # The below doesn't match, and it isn't clear what $1 and $2 would 72 # be even if it did!! 73 "\N{LATIN SMALL LIGATURE FI}" =~ /(f)(i)/i; # Doesn't match! 74 75Perl doesn't match multiple characters in a bracketed 76character class unless the character that maps to them is explicitly 77mentioned, and it doesn't match them at all if the character class is 78inverted, which otherwise could be highly confusing. See 79L<perlrecharclass/Bracketed Character Classes>, and 80L<perlrecharclass/Negation>. 81 82=item x 83X</x> 84 85Extend your pattern's legibility by permitting whitespace and comments. 86Details in L</"/x"> 87 88=item p 89X</p> X<regex, preserve> X<regexp, preserve> 90 91Preserve the string matched such that ${^PREMATCH}, ${^MATCH}, and 92${^POSTMATCH} are available for use after matching. 93 94In Perl 5.20 and higher this is ignored. Due to a new copy-on-write 95mechanism, ${^PREMATCH}, ${^MATCH}, and ${^POSTMATCH} will be available 96after the match regardless of the modifier. 97 98=item a, d, l and u 99X</a> X</d> X</l> X</u> 100 101These modifiers, all new in 5.14, affect which character-set rules 102(Unicode, etc.) are used, as described below in 103L</Character set modifiers>. 104 105=item Other Modifiers 106 107There are a number of flags that can be found at the end of regular 108expression constructs that are I<not> generic regular expression flags, but 109apply to the operation being performed, like matching or substitution (C<m//> 110or C<s///> respectively). 111 112Flags described further in 113L<perlretut/"Using regular expressions in Perl"> are: 114 115 c - keep the current position during repeated matching 116 g - globally match the pattern repeatedly in the string 117 118Substitution-specific modifiers described in 119 120L<perlop/"s/PATTERN/REPLACEMENT/msixpodualgcer"> are: 121 122 e - evaluate the right-hand side as an expression 123 ee - evaluate the right side as a string then eval the result 124 o - pretend to optimize your code, but actually introduce bugs 125 r - perform non-destructive substitution and return the new value 126 127=back 128 129Regular expression modifiers are usually written in documentation 130as e.g., "the C</x> modifier", even though the delimiter 131in question might not really be a slash. The modifiers C</imsxadlup> 132may also be embedded within the regular expression itself using 133the C<(?...)> construct, see L</Extended Patterns> below. 134 135=head3 /x 136 137C</x> tells 138the regular expression parser to ignore most whitespace that is neither 139backslashed nor within a bracketed character class. You can use this to 140break up your regular expression into (slightly) more readable parts. 141Also, the C<#> character is treated as a metacharacter introducing a 142comment that runs up to the pattern's closing delimiter, or to the end 143of the current line if the pattern extends onto the next line. Hence, 144this is very much like an ordinary Perl code comment. (You can include 145the closing delimiter within the comment only if you precede it with a 146backslash, so be careful!) 147 148Use of C</x> means that if you want real 149whitespace or C<#> characters in the pattern (outside a bracketed character 150class, which is unaffected by C</x>), then you'll either have to 151escape them (using backslashes or C<\Q...\E>) or encode them using octal, 152hex, or C<\N{}> escapes. 153It is ineffective to try to continue a comment onto the next line by 154escaping the C<\n> with a backslash or C<\Q>. 155 156You can use L</(?#text)> to create a comment that ends earlier than the 157end of the current line, but C<text> also can't contain the closing 158delimiter unless escaped with a backslash. 159 160Taken together, these features go a long way towards 161making Perl's regular expressions more readable. Here's an example: 162 163 # Delete (most) C comments. 164 $program =~ s { 165 /\* # Match the opening delimiter. 166 .*? # Match a minimal number of characters. 167 \*/ # Match the closing delimiter. 168 } []gsx; 169 170Note that anything inside 171a C<\Q...\E> stays unaffected by C</x>. And note that C</x> doesn't affect 172space interpretation within a single multi-character construct. For 173example in C<\x{...}>, regardless of the C</x> modifier, there can be no 174spaces. Same for a L<quantifier|/Quantifiers> such as C<{3}> or 175C<{5,}>. Similarly, C<(?:...)> can't have a space between the C<(>, 176C<?>, and C<:>. Within any delimiters for such a 177construct, allowed spaces are not affected by C</x>, and depend on the 178construct. For example, C<\x{...}> can't have spaces because hexadecimal 179numbers don't have spaces in them. But, Unicode properties can have spaces, so 180in C<\p{...}> there can be spaces that follow the Unicode rules, for which see 181L<perluniprops/Properties accessible through \p{} and \P{}>. 182X</x> 183 184=head3 Character set modifiers 185 186C</d>, C</u>, C</a>, and C</l>, available starting in 5.14, are called 187the character set modifiers; they affect the character set rules 188used for the regular expression. 189 190The C</d>, C</u>, and C</l> modifiers are not likely to be of much use 191to you, and so you need not worry about them very much. They exist for 192Perl's internal use, so that complex regular expression data structures 193can be automatically serialized and later exactly reconstituted, 194including all their nuances. But, since Perl can't keep a secret, and 195there may be rare instances where they are useful, they are documented 196here. 197 198The C</a> modifier, on the other hand, may be useful. Its purpose is to 199allow code that is to work mostly on ASCII data to not have to concern 200itself with Unicode. 201 202Briefly, C</l> sets the character set to that of whatever B<L>ocale is in 203effect at the time of the execution of the pattern match. 204 205C</u> sets the character set to B<U>nicode. 206 207C</a> also sets the character set to Unicode, BUT adds several 208restrictions for B<A>SCII-safe matching. 209 210C</d> is the old, problematic, pre-5.14 B<D>efault character set 211behavior. Its only use is to force that old behavior. 212 213At any given time, exactly one of these modifiers is in effect. Their 214existence allows Perl to keep the originally compiled behavior of a 215regular expression, regardless of what rules are in effect when it is 216actually executed. And if it is interpolated into a larger regex, the 217original's rules continue to apply to it, and only it. 218 219The C</l> and C</u> modifiers are automatically selected for 220regular expressions compiled within the scope of various pragmas, 221and we recommend that in general, you use those pragmas instead of 222specifying these modifiers explicitly. For one thing, the modifiers 223affect only pattern matching, and do not extend to even any replacement 224done, whereas using the pragmas give consistent results for all 225appropriate operations within their scopes. For example, 226 227 s/foo/\Ubar/il 228 229will match "foo" using the locale's rules for case-insensitive matching, 230but the C</l> does not affect how the C<\U> operates. Most likely you 231want both of them to use locale rules. To do this, instead compile the 232regular expression within the scope of C<use locale>. This both 233implicitly adds the C</l> and applies locale rules to the C<\U>. The 234lesson is to C<use locale> and not C</l> explicitly. 235 236Similarly, it would be better to use C<use feature 'unicode_strings'> 237instead of, 238 239 s/foo/\Lbar/iu 240 241to get Unicode rules, as the C<\L> in the former (but not necessarily 242the latter) would also use Unicode rules. 243 244More detail on each of the modifiers follows. Most likely you don't 245need to know this detail for C</l>, C</u>, and C</d>, and can skip ahead 246to L<E<sol>a|/E<sol>a (and E<sol>aa)>. 247 248=head4 /l 249 250means to use the current locale's rules (see L<perllocale>) when pattern 251matching. For example, C<\w> will match the "word" characters of that 252locale, and C<"/i"> case-insensitive matching will match according to 253the locale's case folding rules. The locale used will be the one in 254effect at the time of execution of the pattern match. This may not be 255the same as the compilation-time locale, and can differ from one match 256to another if there is an intervening call of the 257L<setlocale() function|perllocale/The setlocale function>. 258 259The only non-single-byte locale Perl supports is (starting in v5.20) 260UTF-8. This means that code points above 255 are treated as Unicode no 261matter what locale is in effect (since UTF-8 implies Unicode). 262 263Under Unicode rules, there are a few case-insensitive matches that cross 264the 255/256 boundary. Except for UTF-8 locales in Perls v5.20 and 265later, these are disallowed under C</l>. For example, 0xFF (on ASCII 266platforms) does not caselessly match the character at 0x178, C<LATIN 267CAPITAL LETTER Y WITH DIAERESIS>, because 0xFF may not be C<LATIN SMALL 268LETTER Y WITH DIAERESIS> in the current locale, and Perl has no way of 269knowing if that character even exists in the locale, much less what code 270point it is. 271 272In a UTF-8 locale in v5.20 and later, the only visible difference 273between locale and non-locale in regular expressions should be tainting 274(see L<perlsec>). 275 276This modifier may be specified to be the default by C<use locale>, but 277see L</Which character set modifier is in effect?>. 278X</l> 279 280=head4 /u 281 282means to use Unicode rules when pattern matching. On ASCII platforms, 283this means that the code points between 128 and 255 take on their 284Latin-1 (ISO-8859-1) meanings (which are the same as Unicode's). 285(Otherwise Perl considers their meanings to be undefined.) Thus, 286under this modifier, the ASCII platform effectively becomes a Unicode 287platform; and hence, for example, C<\w> will match any of the more than 288100_000 word characters in Unicode. 289 290Unlike most locales, which are specific to a language and country pair, 291Unicode classifies all the characters that are letters I<somewhere> in 292the world as 293C<\w>. For example, your locale might not think that C<LATIN SMALL 294LETTER ETH> is a letter (unless you happen to speak Icelandic), but 295Unicode does. Similarly, all the characters that are decimal digits 296somewhere in the world will match C<\d>; this is hundreds, not 10, 297possible matches. And some of those digits look like some of the 10 298ASCII digits, but mean a different number, so a human could easily think 299a number is a different quantity than it really is. For example, 300C<BENGALI DIGIT FOUR> (U+09EA) looks very much like an 301C<ASCII DIGIT EIGHT> (U+0038). And, C<\d+>, may match strings of digits 302that are a mixture from different writing systems, creating a security 303issue. L<Unicode::UCD/num()> can be used to sort 304this out. Or the C</a> modifier can be used to force C<\d> to match 305just the ASCII 0 through 9. 306 307Also, under this modifier, case-insensitive matching works on the full 308set of Unicode 309characters. The C<KELVIN SIGN>, for example matches the letters "k" and 310"K"; and C<LATIN SMALL LIGATURE FF> matches the sequence "ff", which, 311if you're not prepared, might make it look like a hexadecimal constant, 312presenting another potential security issue. See 313L<http://unicode.org/reports/tr36> for a detailed discussion of Unicode 314security issues. 315 316This modifier may be specified to be the default by C<use feature 317'unicode_strings>, C<use locale ':not_characters'>, or 318C<L<use 5.012|perlfunc/use VERSION>> (or higher), 319but see L</Which character set modifier is in effect?>. 320X</u> 321 322=head4 /d 323 324This modifier means to use the "Default" native rules of the platform 325except when there is cause to use Unicode rules instead, as follows: 326 327=over 4 328 329=item 1 330 331the target string is encoded in UTF-8; or 332 333=item 2 334 335the pattern is encoded in UTF-8; or 336 337=item 3 338 339the pattern explicitly mentions a code point that is above 255 (say by 340C<\x{100}>); or 341 342=item 4 343 344the pattern uses a Unicode name (C<\N{...}>); or 345 346=item 5 347 348the pattern uses a Unicode property (C<\p{...}>); or 349 350=item 6 351 352the pattern uses L</C<(?[ ])>> 353 354=back 355 356Another mnemonic for this modifier is "Depends", as the rules actually 357used depend on various things, and as a result you can get unexpected 358results. See L<perlunicode/The "Unicode Bug">. The Unicode Bug has 359become rather infamous, leading to yet another (printable) name for this 360modifier, "Dodgy". 361 362Unless the pattern or string are encoded in UTF-8, only ASCII characters 363can match positively. 364 365Here are some examples of how that works on an ASCII platform: 366 367 $str = "\xDF"; # $str is not in UTF-8 format. 368 $str =~ /^\w/; # No match, as $str isn't in UTF-8 format. 369 $str .= "\x{0e0b}"; # Now $str is in UTF-8 format. 370 $str =~ /^\w/; # Match! $str is now in UTF-8 format. 371 chop $str; 372 $str =~ /^\w/; # Still a match! $str remains in UTF-8 format. 373 374This modifier is automatically selected by default when none of the 375others are, so yet another name for it is "Default". 376 377Because of the unexpected behaviors associated with this modifier, you 378probably should only use it to maintain weird backward compatibilities. 379 380=head4 /a (and /aa) 381 382This modifier stands for ASCII-restrict (or ASCII-safe). This modifier, 383unlike the others, may be doubled-up to increase its effect. 384 385When it appears singly, it causes the sequences C<\d>, C<\s>, C<\w>, and 386the Posix character classes to match only in the ASCII range. They thus 387revert to their pre-5.6, pre-Unicode meanings. Under C</a>, C<\d> 388always means precisely the digits C<"0"> to C<"9">; C<\s> means the five 389characters C<[ \f\n\r\t]>, and starting in Perl v5.18, experimentally, 390the vertical tab; C<\w> means the 63 characters 391C<[A-Za-z0-9_]>; and likewise, all the Posix classes such as 392C<[[:print:]]> match only the appropriate ASCII-range characters. 393 394This modifier is useful for people who only incidentally use Unicode, 395and who do not wish to be burdened with its complexities and security 396concerns. 397 398With C</a>, one can write C<\d> with confidence that it will only match 399ASCII characters, and should the need arise to match beyond ASCII, you 400can instead use C<\p{Digit}> (or C<\p{Word}> for C<\w>). There are 401similar C<\p{...}> constructs that can match beyond ASCII both white 402space (see L<perlrecharclass/Whitespace>), and Posix classes (see 403L<perlrecharclass/POSIX Character Classes>). Thus, this modifier 404doesn't mean you can't use Unicode, it means that to get Unicode 405matching you must explicitly use a construct (C<\p{}>, C<\P{}>) that 406signals Unicode. 407 408As you would expect, this modifier causes, for example, C<\D> to mean 409the same thing as C<[^0-9]>; in fact, all non-ASCII characters match 410C<\D>, C<\S>, and C<\W>. C<\b> still means to match at the boundary 411between C<\w> and C<\W>, using the C</a> definitions of them (similarly 412for C<\B>). 413 414Otherwise, C</a> behaves like the C</u> modifier, in that 415case-insensitive matching uses Unicode rules; for example, "k" will 416match the Unicode C<\N{KELVIN SIGN}> under C</i> matching, and code 417points in the Latin1 range, above ASCII will have Unicode rules when it 418comes to case-insensitive matching. 419 420To forbid ASCII/non-ASCII matches (like "k" with C<\N{KELVIN SIGN}>), 421specify the "a" twice, for example C</aai> or C</aia>. (The first 422occurrence of "a" restricts the C<\d>, etc., and the second occurrence 423adds the C</i> restrictions.) But, note that code points outside the 424ASCII range will use Unicode rules for C</i> matching, so the modifier 425doesn't really restrict things to just ASCII; it just forbids the 426intermixing of ASCII and non-ASCII. 427 428To summarize, this modifier provides protection for applications that 429don't wish to be exposed to all of Unicode. Specifying it twice 430gives added protection. 431 432This modifier may be specified to be the default by C<use re '/a'> 433or C<use re '/aa'>. If you do so, you may actually have occasion to use 434the C</u> modifier explicitly if there are a few regular expressions 435where you do want full Unicode rules (but even here, it's best if 436everything were under feature C<"unicode_strings">, along with the 437C<use re '/aa'>). Also see L</Which character set modifier is in 438effect?>. 439X</a> 440X</aa> 441 442=head4 Which character set modifier is in effect? 443 444Which of these modifiers is in effect at any given point in a regular 445expression depends on a fairly complex set of interactions. These have 446been designed so that in general you don't have to worry about it, but 447this section gives the gory details. As 448explained below in L</Extended Patterns> it is possible to explicitly 449specify modifiers that apply only to portions of a regular expression. 450The innermost always has priority over any outer ones, and one applying 451to the whole expression has priority over any of the default settings that are 452described in the remainder of this section. 453 454The C<L<use re 'E<sol>foo'|re/"'/flags' mode">> pragma can be used to set 455default modifiers (including these) for regular expressions compiled 456within its scope. This pragma has precedence over the other pragmas 457listed below that also change the defaults. 458 459Otherwise, C<L<use locale|perllocale>> sets the default modifier to C</l>; 460and C<L<use feature 'unicode_strings|feature>>, or 461C<L<use 5.012|perlfunc/use VERSION>> (or higher) set the default to 462C</u> when not in the same scope as either C<L<use locale|perllocale>> 463or C<L<use bytes|bytes>>. 464(C<L<use locale ':not_characters'|perllocale/Unicode and UTF-8>> also 465sets the default to C</u>, overriding any plain C<use locale>.) 466Unlike the mechanisms mentioned above, these 467affect operations besides regular expressions pattern matching, and so 468give more consistent results with other operators, including using 469C<\U>, C<\l>, etc. in substitution replacements. 470 471If none of the above apply, for backwards compatibility reasons, the 472C</d> modifier is the one in effect by default. As this can lead to 473unexpected results, it is best to specify which other rule set should be 474used. 475 476=head4 Character set modifier behavior prior to Perl 5.14 477 478Prior to 5.14, there were no explicit modifiers, but C</l> was implied 479for regexes compiled within the scope of C<use locale>, and C</d> was 480implied otherwise. However, interpolating a regex into a larger regex 481would ignore the original compilation in favor of whatever was in effect 482at the time of the second compilation. There were a number of 483inconsistencies (bugs) with the C</d> modifier, where Unicode rules 484would be used when inappropriate, and vice versa. C<\p{}> did not imply 485Unicode rules, and neither did all occurrences of C<\N{}>, until 5.12. 486 487=head2 Regular Expressions 488 489=head3 Metacharacters 490 491The patterns used in Perl pattern matching evolved from those supplied in 492the Version 8 regex routines. (The routines are derived 493(distantly) from Henry Spencer's freely redistributable reimplementation 494of the V8 routines.) See L<Version 8 Regular Expressions> for 495details. 496 497In particular the following metacharacters have their standard I<egrep>-ish 498meanings: 499X<metacharacter> 500X<\> X<^> X<.> X<$> X<|> X<(> X<()> X<[> X<[]> 501 502 503 \ Quote the next metacharacter 504 ^ Match the beginning of the line 505 . Match any character (except newline) 506 $ Match the end of the string (or before newline at the end 507 of the string) 508 | Alternation 509 () Grouping 510 [] Bracketed Character class 511 512By default, the "^" character is guaranteed to match only the 513beginning of the string, the "$" character only the end (or before the 514newline at the end), and Perl does certain optimizations with the 515assumption that the string contains only one line. Embedded newlines 516will not be matched by "^" or "$". You may, however, wish to treat a 517string as a multi-line buffer, such that the "^" will match after any 518newline within the string (except if the newline is the last character in 519the string), and "$" will match before any newline. At the 520cost of a little more overhead, you can do this by using the /m modifier 521on the pattern match operator. (Older programs did this by setting C<$*>, 522but this option was removed in perl 5.10.) 523X<^> X<$> X</m> 524 525To simplify multi-line substitutions, the "." character never matches a 526newline unless you use the C</s> modifier, which in effect tells Perl to pretend 527the string is a single line--even if it isn't. 528X<.> X</s> 529 530=head3 Quantifiers 531 532The following standard quantifiers are recognized: 533X<metacharacter> X<quantifier> X<*> X<+> X<?> X<{n}> X<{n,}> X<{n,m}> 534 535 * Match 0 or more times 536 + Match 1 or more times 537 ? Match 1 or 0 times 538 {n} Match exactly n times 539 {n,} Match at least n times 540 {n,m} Match at least n but not more than m times 541 542(If a curly bracket occurs in any other context and does not form part of 543a backslashed sequence like C<\x{...}>, it is treated as a regular 544character. In particular, the lower quantifier bound is not optional, 545and a typo in a quantifier silently causes it to be treated as the 546literal characters. For example, 547 548 /o{4,a}/ 549 550compiles to match the sequence of six characters 551S<C<"o { 4 , a }">>. It is planned to eventually require literal uses 552of curly brackets to be escaped, say by preceding them with a backslash 553or enclosing them within square brackets, (C<"\{"> or C<"[{]">). This 554change will allow for future syntax extensions (like making the lower 555bound of a quantifier optional), and better error checking. In the 556meantime, you should get in the habit of escaping all instances where 557you mean a literal "{".) 558 559The "*" quantifier is equivalent to C<{0,}>, the "+" 560quantifier to C<{1,}>, and the "?" quantifier to C<{0,1}>. n and m are limited 561to non-negative integral values less than a preset limit defined when perl is built. 562This is usually 32766 on the most common platforms. The actual limit can 563be seen in the error message generated by code such as this: 564 565 $_ **= $_ , / {$_} / for 2 .. 42; 566 567By default, a quantified subpattern is "greedy", that is, it will match as 568many times as possible (given a particular starting location) while still 569allowing the rest of the pattern to match. If you want it to match the 570minimum number of times possible, follow the quantifier with a "?". Note 571that the meanings don't change, just the "greediness": 572X<metacharacter> X<greedy> X<greediness> 573X<?> X<*?> X<+?> X<??> X<{n}?> X<{n,}?> X<{n,m}?> 574 575 *? Match 0 or more times, not greedily 576 +? Match 1 or more times, not greedily 577 ?? Match 0 or 1 time, not greedily 578 {n}? Match exactly n times, not greedily (redundant) 579 {n,}? Match at least n times, not greedily 580 {n,m}? Match at least n but not more than m times, not greedily 581 582Normally when a quantified subpattern does not allow the rest of the 583overall pattern to match, Perl will backtrack. However, this behaviour is 584sometimes undesirable. Thus Perl provides the "possessive" quantifier form 585as well. 586 587 *+ Match 0 or more times and give nothing back 588 ++ Match 1 or more times and give nothing back 589 ?+ Match 0 or 1 time and give nothing back 590 {n}+ Match exactly n times and give nothing back (redundant) 591 {n,}+ Match at least n times and give nothing back 592 {n,m}+ Match at least n but not more than m times and give nothing back 593 594For instance, 595 596 'aaaa' =~ /a++a/ 597 598will never match, as the C<a++> will gobble up all the C<a>'s in the 599string and won't leave any for the remaining part of the pattern. This 600feature can be extremely useful to give perl hints about where it 601shouldn't backtrack. For instance, the typical "match a double-quoted 602string" problem can be most efficiently performed when written as: 603 604 /"(?:[^"\\]++|\\.)*+"/ 605 606as we know that if the final quote does not match, backtracking will not 607help. See the independent subexpression 608L</C<< (?>pattern) >>> for more details; 609possessive quantifiers are just syntactic sugar for that construct. For 610instance the above example could also be written as follows: 611 612 /"(?>(?:(?>[^"\\]+)|\\.)*)"/ 613 614Note that the possessive quantifier modifier can not be be combined 615with the non-greedy modifier. This is because it would make no sense. 616Consider the follow equivalency table: 617 618 Illegal Legal 619 ------------ ------ 620 X??+ X{0} 621 X+?+ X{1} 622 X{min,max}?+ X{min} 623 624=head3 Escape sequences 625 626Because patterns are processed as double-quoted strings, the following 627also work: 628 629 \t tab (HT, TAB) 630 \n newline (LF, NL) 631 \r return (CR) 632 \f form feed (FF) 633 \a alarm (bell) (BEL) 634 \e escape (think troff) (ESC) 635 \cK control char (example: VT) 636 \x{}, \x00 character whose ordinal is the given hexadecimal number 637 \N{name} named Unicode character or character sequence 638 \N{U+263D} Unicode character (example: FIRST QUARTER MOON) 639 \o{}, \000 character whose ordinal is the given octal number 640 \l lowercase next char (think vi) 641 \u uppercase next char (think vi) 642 \L lowercase until \E (think vi) 643 \U uppercase until \E (think vi) 644 \Q quote (disable) pattern metacharacters until \E 645 \E end either case modification or quoted section, think vi 646 647Details are in L<perlop/Quote and Quote-like Operators>. 648 649=head3 Character Classes and other Special Escapes 650 651In addition, Perl defines the following: 652X<\g> X<\k> X<\K> X<backreference> 653 654 Sequence Note Description 655 [...] [1] Match a character according to the rules of the 656 bracketed character class defined by the "...". 657 Example: [a-z] matches "a" or "b" or "c" ... or "z" 658 [[:...:]] [2] Match a character according to the rules of the POSIX 659 character class "..." within the outer bracketed 660 character class. Example: [[:upper:]] matches any 661 uppercase character. 662 (?[...]) [8] Extended bracketed character class 663 \w [3] Match a "word" character (alphanumeric plus "_", plus 664 other connector punctuation chars plus Unicode 665 marks) 666 \W [3] Match a non-"word" character 667 \s [3] Match a whitespace character 668 \S [3] Match a non-whitespace character 669 \d [3] Match a decimal digit character 670 \D [3] Match a non-digit character 671 \pP [3] Match P, named property. Use \p{Prop} for longer names 672 \PP [3] Match non-P 673 \X [4] Match Unicode "eXtended grapheme cluster" 674 \C Match a single C-language char (octet) even if that is 675 part of a larger UTF-8 character. Thus it breaks up 676 characters into their UTF-8 bytes, so you may end up 677 with malformed pieces of UTF-8. Unsupported in 678 lookbehind. (Deprecated.) 679 \1 [5] Backreference to a specific capture group or buffer. 680 '1' may actually be any positive integer. 681 \g1 [5] Backreference to a specific or previous group, 682 \g{-1} [5] The number may be negative indicating a relative 683 previous group and may optionally be wrapped in 684 curly brackets for safer parsing. 685 \g{name} [5] Named backreference 686 \k<name> [5] Named backreference 687 \K [6] Keep the stuff left of the \K, don't include it in $& 688 \N [7] Any character but \n. Not affected by /s modifier 689 \v [3] Vertical whitespace 690 \V [3] Not vertical whitespace 691 \h [3] Horizontal whitespace 692 \H [3] Not horizontal whitespace 693 \R [4] Linebreak 694 695=over 4 696 697=item [1] 698 699See L<perlrecharclass/Bracketed Character Classes> for details. 700 701=item [2] 702 703See L<perlrecharclass/POSIX Character Classes> for details. 704 705=item [3] 706 707See L<perlrecharclass/Backslash sequences> for details. 708 709=item [4] 710 711See L<perlrebackslash/Misc> for details. 712 713=item [5] 714 715See L</Capture groups> below for details. 716 717=item [6] 718 719See L</Extended Patterns> below for details. 720 721=item [7] 722 723Note that C<\N> has two meanings. When of the form C<\N{NAME}>, it matches the 724character or character sequence whose name is C<NAME>; and similarly 725when of the form C<\N{U+I<hex>}>, it matches the character whose Unicode 726code point is I<hex>. Otherwise it matches any character but C<\n>. 727 728=item [8] 729 730See L<perlrecharclass/Extended Bracketed Character Classes> for details. 731 732=back 733 734=head3 Assertions 735 736Perl defines the following zero-width assertions: 737X<zero-width assertion> X<assertion> X<regex, zero-width assertion> 738X<regexp, zero-width assertion> 739X<regular expression, zero-width assertion> 740X<\b> X<\B> X<\A> X<\Z> X<\z> X<\G> 741 742 \b Match a word boundary 743 \B Match except at a word boundary 744 \A Match only at beginning of string 745 \Z Match only at end of string, or before newline at the end 746 \z Match only at end of string 747 \G Match only at pos() (e.g. at the end-of-match position 748 of prior m//g) 749 750A word boundary (C<\b>) is a spot between two characters 751that has a C<\w> on one side of it and a C<\W> on the other side 752of it (in either order), counting the imaginary characters off the 753beginning and end of the string as matching a C<\W>. (Within 754character classes C<\b> represents backspace rather than a word 755boundary, just as it normally does in any double-quoted string.) 756The C<\A> and C<\Z> are just like "^" and "$", except that they 757won't match multiple times when the C</m> modifier is used, while 758"^" and "$" will match at every internal line boundary. To match 759the actual end of the string and not ignore an optional trailing 760newline, use C<\z>. 761X<\b> X<\A> X<\Z> X<\z> X</m> 762 763The C<\G> assertion can be used to chain global matches (using 764C<m//g>), as described in L<perlop/"Regexp Quote-Like Operators">. 765It is also useful when writing C<lex>-like scanners, when you have 766several patterns that you want to match against consequent substrings 767of your string; see the previous reference. The actual location 768where C<\G> will match can also be influenced by using C<pos()> as 769an lvalue: see L<perlfunc/pos>. Note that the rule for zero-length 770matches (see L</"Repeated Patterns Matching a Zero-length Substring">) 771is modified somewhat, in that contents to the left of C<\G> are 772not counted when determining the length of the match. Thus the following 773will not match forever: 774X<\G> 775 776 my $string = 'ABC'; 777 pos($string) = 1; 778 while ($string =~ /(.\G)/g) { 779 print $1; 780 } 781 782It will print 'A' and then terminate, as it considers the match to 783be zero-width, and thus will not match at the same position twice in a 784row. 785 786It is worth noting that C<\G> improperly used can result in an infinite 787loop. Take care when using patterns that include C<\G> in an alternation. 788 789Note also that C<s///> will refuse to overwrite part of a substitution 790that has already been replaced; so for example this will stop after the 791first iteration, rather than iterating its way backwards through the 792string: 793 794 $_ = "123456789"; 795 pos = 6; 796 s/.(?=.\G)/X/g; 797 print; # prints 1234X6789, not XXXXX6789 798 799 800=head3 Capture groups 801 802The bracketing construct C<( ... )> creates capture groups (also referred to as 803capture buffers). To refer to the current contents of a group later on, within 804the same pattern, use C<\g1> (or C<\g{1}>) for the first, C<\g2> (or C<\g{2}>) 805for the second, and so on. 806This is called a I<backreference>. 807X<regex, capture buffer> X<regexp, capture buffer> 808X<regex, capture group> X<regexp, capture group> 809X<regular expression, capture buffer> X<backreference> 810X<regular expression, capture group> X<backreference> 811X<\g{1}> X<\g{-1}> X<\g{name}> X<relative backreference> X<named backreference> 812X<named capture buffer> X<regular expression, named capture buffer> 813X<named capture group> X<regular expression, named capture group> 814X<%+> X<$+{name}> X<< \k<name> >> 815There is no limit to the number of captured substrings that you may use. 816Groups are numbered with the leftmost open parenthesis being number 1, etc. If 817a group did not match, the associated backreference won't match either. (This 818can happen if the group is optional, or in a different branch of an 819alternation.) 820You can omit the C<"g">, and write C<"\1">, etc, but there are some issues with 821this form, described below. 822 823You can also refer to capture groups relatively, by using a negative number, so 824that C<\g-1> and C<\g{-1}> both refer to the immediately preceding capture 825group, and C<\g-2> and C<\g{-2}> both refer to the group before it. For 826example: 827 828 / 829 (Y) # group 1 830 ( # group 2 831 (X) # group 3 832 \g{-1} # backref to group 3 833 \g{-3} # backref to group 1 834 ) 835 /x 836 837would match the same as C</(Y) ( (X) \g3 \g1 )/x>. This allows you to 838interpolate regexes into larger regexes and not have to worry about the 839capture groups being renumbered. 840 841You can dispense with numbers altogether and create named capture groups. 842The notation is C<(?E<lt>I<name>E<gt>...)> to declare and C<\g{I<name>}> to 843reference. (To be compatible with .Net regular expressions, C<\g{I<name>}> may 844also be written as C<\k{I<name>}>, C<\kE<lt>I<name>E<gt>> or C<\k'I<name>'>.) 845I<name> must not begin with a number, nor contain hyphens. 846When different groups within the same pattern have the same name, any reference 847to that name assumes the leftmost defined group. Named groups count in 848absolute and relative numbering, and so can also be referred to by those 849numbers. 850(It's possible to do things with named capture groups that would otherwise 851require C<(??{})>.) 852 853Capture group contents are dynamically scoped and available to you outside the 854pattern until the end of the enclosing block or until the next successful 855match, whichever comes first. (See L<perlsyn/"Compound Statements">.) 856You can refer to them by absolute number (using C<"$1"> instead of C<"\g1">, 857etc); or by name via the C<%+> hash, using C<"$+{I<name>}">. 858 859Braces are required in referring to named capture groups, but are optional for 860absolute or relative numbered ones. Braces are safer when creating a regex by 861concatenating smaller strings. For example if you have C<qr/$a$b/>, and C<$a> 862contained C<"\g1">, and C<$b> contained C<"37">, you would get C</\g137/> which 863is probably not what you intended. 864 865The C<\g> and C<\k> notations were introduced in Perl 5.10.0. Prior to that 866there were no named nor relative numbered capture groups. Absolute numbered 867groups were referred to using C<\1>, 868C<\2>, etc., and this notation is still 869accepted (and likely always will be). But it leads to some ambiguities if 870there are more than 9 capture groups, as C<\10> could mean either the tenth 871capture group, or the character whose ordinal in octal is 010 (a backspace in 872ASCII). Perl resolves this ambiguity by interpreting C<\10> as a backreference 873only if at least 10 left parentheses have opened before it. Likewise C<\11> is 874a backreference only if at least 11 left parentheses have opened before it. 875And so on. C<\1> through C<\9> are always interpreted as backreferences. 876There are several examples below that illustrate these perils. You can avoid 877the ambiguity by always using C<\g{}> or C<\g> if you mean capturing groups; 878and for octal constants always using C<\o{}>, or for C<\077> and below, using 3 879digits padded with leading zeros, since a leading zero implies an octal 880constant. 881 882The C<\I<digit>> notation also works in certain circumstances outside 883the pattern. See L</Warning on \1 Instead of $1> below for details. 884 885Examples: 886 887 s/^([^ ]*) *([^ ]*)/$2 $1/; # swap first two words 888 889 /(.)\g1/ # find first doubled char 890 and print "'$1' is the first doubled character\n"; 891 892 /(?<char>.)\k<char>/ # ... a different way 893 and print "'$+{char}' is the first doubled character\n"; 894 895 /(?'char'.)\g1/ # ... mix and match 896 and print "'$1' is the first doubled character\n"; 897 898 if (/Time: (..):(..):(..)/) { # parse out values 899 $hours = $1; 900 $minutes = $2; 901 $seconds = $3; 902 } 903 904 /(.)(.)(.)(.)(.)(.)(.)(.)(.)\g10/ # \g10 is a backreference 905 /(.)(.)(.)(.)(.)(.)(.)(.)(.)\10/ # \10 is octal 906 /((.)(.)(.)(.)(.)(.)(.)(.)(.))\10/ # \10 is a backreference 907 /((.)(.)(.)(.)(.)(.)(.)(.)(.))\010/ # \010 is octal 908 909 $a = '(.)\1'; # Creates problems when concatenated. 910 $b = '(.)\g{1}'; # Avoids the problems. 911 "aa" =~ /${a}/; # True 912 "aa" =~ /${b}/; # True 913 "aa0" =~ /${a}0/; # False! 914 "aa0" =~ /${b}0/; # True 915 "aa\x08" =~ /${a}0/; # True! 916 "aa\x08" =~ /${b}0/; # False 917 918Several special variables also refer back to portions of the previous 919match. C<$+> returns whatever the last bracket match matched. 920C<$&> returns the entire matched string. (At one point C<$0> did 921also, but now it returns the name of the program.) C<$`> returns 922everything before the matched string. C<$'> returns everything 923after the matched string. And C<$^N> contains whatever was matched by 924the most-recently closed group (submatch). C<$^N> can be used in 925extended patterns (see below), for example to assign a submatch to a 926variable. 927X<$+> X<$^N> X<$&> X<$`> X<$'> 928 929These special variables, like the C<%+> hash and the numbered match variables 930(C<$1>, C<$2>, C<$3>, etc.) are dynamically scoped 931until the end of the enclosing block or until the next successful 932match, whichever comes first. (See L<perlsyn/"Compound Statements">.) 933X<$+> X<$^N> X<$&> X<$`> X<$'> 934X<$1> X<$2> X<$3> X<$4> X<$5> X<$6> X<$7> X<$8> X<$9> 935 936B<NOTE>: Failed matches in Perl do not reset the match variables, 937which makes it easier to write code that tests for a series of more 938specific cases and remembers the best match. 939 940B<WARNING>: If your code is to run on Perl 5.16 or earlier, 941beware that once Perl sees that you need one of C<$&>, C<$`>, or 942C<$'> anywhere in the program, it has to provide them for every 943pattern match. This may substantially slow your program. 944 945Perl uses the same mechanism to produce C<$1>, C<$2>, etc, so you also 946pay a price for each pattern that contains capturing parentheses. 947(To avoid this cost while retaining the grouping behaviour, use the 948extended regular expression C<(?: ... )> instead.) But if you never 949use C<$&>, C<$`> or C<$'>, then patterns I<without> capturing 950parentheses will not be penalized. So avoid C<$&>, C<$'>, and C<$`> 951if you can, but if you can't (and some algorithms really appreciate 952them), once you've used them once, use them at will, because you've 953already paid the price. 954X<$&> X<$`> X<$'> 955 956Perl 5.16 introduced a slightly more efficient mechanism that notes 957separately whether each of C<$`>, C<$&>, and C<$'> have been seen, and 958thus may only need to copy part of the string. Perl 5.20 introduced a 959much more efficient copy-on-write mechanism which eliminates any slowdown. 960 961As another workaround for this problem, Perl 5.10.0 introduced C<${^PREMATCH}>, 962C<${^MATCH}> and C<${^POSTMATCH}>, which are equivalent to C<$`>, C<$&> 963and C<$'>, B<except> that they are only guaranteed to be defined after a 964successful match that was executed with the C</p> (preserve) modifier. 965The use of these variables incurs no global performance penalty, unlike 966their punctuation char equivalents, however at the trade-off that you 967have to tell perl when you want to use them. As of Perl 5.20, these three 968variables are equivalent to C<$`>, C<$&> and C<$'>, and C</p> is ignored. 969X</p> X<p modifier> 970 971=head2 Quoting metacharacters 972 973Backslashed metacharacters in Perl are alphanumeric, such as C<\b>, 974C<\w>, C<\n>. Unlike some other regular expression languages, there 975are no backslashed symbols that aren't alphanumeric. So anything 976that looks like \\, \(, \), \[, \], \{, or \} is always 977interpreted as a literal character, not a metacharacter. This was 978once used in a common idiom to disable or quote the special meanings 979of regular expression metacharacters in a string that you want to 980use for a pattern. Simply quote all non-"word" characters: 981 982 $pattern =~ s/(\W)/\\$1/g; 983 984(If C<use locale> is set, then this depends on the current locale.) 985Today it is more common to use the quotemeta() function or the C<\Q> 986metaquoting escape sequence to disable all metacharacters' special 987meanings like this: 988 989 /$unquoted\Q$quoted\E$unquoted/ 990 991Beware that if you put literal backslashes (those not inside 992interpolated variables) between C<\Q> and C<\E>, double-quotish 993backslash interpolation may lead to confusing results. If you 994I<need> to use literal backslashes within C<\Q...\E>, 995consult L<perlop/"Gory details of parsing quoted constructs">. 996 997C<quotemeta()> and C<\Q> are fully described in L<perlfunc/quotemeta>. 998 999=head2 Extended Patterns 1000 1001Perl also defines a consistent extension syntax for features not 1002found in standard tools like B<awk> and 1003B<lex>. The syntax for most of these is a 1004pair of parentheses with a question mark as the first thing within 1005the parentheses. The character after the question mark indicates 1006the extension. 1007 1008The stability of these extensions varies widely. Some have been 1009part of the core language for many years. Others are experimental 1010and may change without warning or be completely removed. Check 1011the documentation on an individual feature to verify its current 1012status. 1013 1014A question mark was chosen for this and for the minimal-matching 1015construct because 1) question marks are rare in older regular 1016expressions, and 2) whenever you see one, you should stop and 1017"question" exactly what is going on. That's psychology.... 1018 1019=over 4 1020 1021=item C<(?#text)> 1022X<(?#)> 1023 1024A comment. The text is ignored. 1025Note that Perl closes 1026the comment as soon as it sees a C<)>, so there is no way to put a literal 1027C<)> in the comment. The pattern's closing delimiter must be escaped by 1028a backslash if it appears in the comment. 1029 1030See L</E<sol>x> for another way to have comments in patterns. 1031 1032=item C<(?adlupimsx-imsx)> 1033 1034=item C<(?^alupimsx)> 1035X<(?)> X<(?^)> 1036 1037One or more embedded pattern-match modifiers, to be turned on (or 1038turned off, if preceded by C<->) for the remainder of the pattern or 1039the remainder of the enclosing pattern group (if any). 1040 1041This is particularly useful for dynamic patterns, such as those read in from a 1042configuration file, taken from an argument, or specified in a table 1043somewhere. Consider the case where some patterns want to be 1044case-sensitive and some do not: The case-insensitive ones merely need to 1045include C<(?i)> at the front of the pattern. For example: 1046 1047 $pattern = "foobar"; 1048 if ( /$pattern/i ) { } 1049 1050 # more flexible: 1051 1052 $pattern = "(?i)foobar"; 1053 if ( /$pattern/ ) { } 1054 1055These modifiers are restored at the end of the enclosing group. For example, 1056 1057 ( (?i) blah ) \s+ \g1 1058 1059will match C<blah> in any case, some spaces, and an exact (I<including the case>!) 1060repetition of the previous word, assuming the C</x> modifier, and no C</i> 1061modifier outside this group. 1062 1063These modifiers do not carry over into named subpatterns called in the 1064enclosing group. In other words, a pattern such as C<((?i)(?&NAME))> does not 1065change the case-sensitivity of the "NAME" pattern. 1066 1067Any of these modifiers can be set to apply globally to all regular 1068expressions compiled within the scope of a C<use re>. See 1069L<re/"'/flags' mode">. 1070 1071Starting in Perl 5.14, a C<"^"> (caret or circumflex accent) immediately 1072after the C<"?"> is a shorthand equivalent to C<d-imsx>. Flags (except 1073C<"d">) may follow the caret to override it. 1074But a minus sign is not legal with it. 1075 1076Note that the C<a>, C<d>, C<l>, C<p>, and C<u> modifiers are special in 1077that they can only be enabled, not disabled, and the C<a>, C<d>, C<l>, and 1078C<u> modifiers are mutually exclusive: specifying one de-specifies the 1079others, and a maximum of one (or two C<a>'s) may appear in the 1080construct. Thus, for 1081example, C<(?-p)> will warn when compiled under C<use warnings>; 1082C<(?-d:...)> and C<(?dl:...)> are fatal errors. 1083 1084Note also that the C<p> modifier is special in that its presence 1085anywhere in a pattern has a global effect. 1086 1087=item C<(?:pattern)> 1088X<(?:)> 1089 1090=item C<(?adluimsx-imsx:pattern)> 1091 1092=item C<(?^aluimsx:pattern)> 1093X<(?^:)> 1094 1095This is for clustering, not capturing; it groups subexpressions like 1096"()", but doesn't make backreferences as "()" does. So 1097 1098 @fields = split(/\b(?:a|b|c)\b/) 1099 1100is like 1101 1102 @fields = split(/\b(a|b|c)\b/) 1103 1104but doesn't spit out extra fields. It's also cheaper not to capture 1105characters if you don't need to. 1106 1107Any letters between C<?> and C<:> act as flags modifiers as with 1108C<(?adluimsx-imsx)>. For example, 1109 1110 /(?s-i:more.*than).*million/i 1111 1112is equivalent to the more verbose 1113 1114 /(?:(?s-i)more.*than).*million/i 1115 1116Starting in Perl 5.14, a C<"^"> (caret or circumflex accent) immediately 1117after the C<"?"> is a shorthand equivalent to C<d-imsx>. Any positive 1118flags (except C<"d">) may follow the caret, so 1119 1120 (?^x:foo) 1121 1122is equivalent to 1123 1124 (?x-ims:foo) 1125 1126The caret tells Perl that this cluster doesn't inherit the flags of any 1127surrounding pattern, but uses the system defaults (C<d-imsx>), 1128modified by any flags specified. 1129 1130The caret allows for simpler stringification of compiled regular 1131expressions. These look like 1132 1133 (?^:pattern) 1134 1135with any non-default flags appearing between the caret and the colon. 1136A test that looks at such stringification thus doesn't need to have the 1137system default flags hard-coded in it, just the caret. If new flags are 1138added to Perl, the meaning of the caret's expansion will change to include 1139the default for those flags, so the test will still work, unchanged. 1140 1141Specifying a negative flag after the caret is an error, as the flag is 1142redundant. 1143 1144Mnemonic for C<(?^...)>: A fresh beginning since the usual use of a caret is 1145to match at the beginning. 1146 1147=item C<(?|pattern)> 1148X<(?|)> X<Branch reset> 1149 1150This is the "branch reset" pattern, which has the special property 1151that the capture groups are numbered from the same starting point 1152in each alternation branch. It is available starting from perl 5.10.0. 1153 1154Capture groups are numbered from left to right, but inside this 1155construct the numbering is restarted for each branch. 1156 1157The numbering within each branch will be as normal, and any groups 1158following this construct will be numbered as though the construct 1159contained only one branch, that being the one with the most capture 1160groups in it. 1161 1162This construct is useful when you want to capture one of a 1163number of alternative matches. 1164 1165Consider the following pattern. The numbers underneath show in 1166which group the captured content will be stored. 1167 1168 1169 # before ---------------branch-reset----------- after 1170 / ( a ) (?| x ( y ) z | (p (q) r) | (t) u (v) ) ( z ) /x 1171 # 1 2 2 3 2 3 4 1172 1173Be careful when using the branch reset pattern in combination with 1174named captures. Named captures are implemented as being aliases to 1175numbered groups holding the captures, and that interferes with the 1176implementation of the branch reset pattern. If you are using named 1177captures in a branch reset pattern, it's best to use the same names, 1178in the same order, in each of the alternations: 1179 1180 /(?| (?<a> x ) (?<b> y ) 1181 | (?<a> z ) (?<b> w )) /x 1182 1183Not doing so may lead to surprises: 1184 1185 "12" =~ /(?| (?<a> \d+ ) | (?<b> \D+))/x; 1186 say $+ {a}; # Prints '12' 1187 say $+ {b}; # *Also* prints '12'. 1188 1189The problem here is that both the group named C<< a >> and the group 1190named C<< b >> are aliases for the group belonging to C<< $1 >>. 1191 1192=item Look-Around Assertions 1193X<look-around assertion> X<lookaround assertion> X<look-around> X<lookaround> 1194 1195Look-around assertions are zero-width patterns which match a specific 1196pattern without including it in C<$&>. Positive assertions match when 1197their subpattern matches, negative assertions match when their subpattern 1198fails. Look-behind matches text up to the current match position, 1199look-ahead matches text following the current match position. 1200 1201=over 4 1202 1203=item C<(?=pattern)> 1204X<(?=)> X<look-ahead, positive> X<lookahead, positive> 1205 1206A zero-width positive look-ahead assertion. For example, C</\w+(?=\t)/> 1207matches a word followed by a tab, without including the tab in C<$&>. 1208 1209=item C<(?!pattern)> 1210X<(?!)> X<look-ahead, negative> X<lookahead, negative> 1211 1212A zero-width negative look-ahead assertion. For example C</foo(?!bar)/> 1213matches any occurrence of "foo" that isn't followed by "bar". Note 1214however that look-ahead and look-behind are NOT the same thing. You cannot 1215use this for look-behind. 1216 1217If you are looking for a "bar" that isn't preceded by a "foo", C</(?!foo)bar/> 1218will not do what you want. That's because the C<(?!foo)> is just saying that 1219the next thing cannot be "foo"--and it's not, it's a "bar", so "foobar" will 1220match. Use look-behind instead (see below). 1221 1222=item C<(?<=pattern)> C<\K> 1223X<(?<=)> X<look-behind, positive> X<lookbehind, positive> X<\K> 1224 1225A zero-width positive look-behind assertion. For example, C</(?<=\t)\w+/> 1226matches a word that follows a tab, without including the tab in C<$&>. 1227Works only for fixed-width look-behind. 1228 1229There is a special form of this construct, called C<\K> (available since 1230Perl 5.10.0), which causes the 1231regex engine to "keep" everything it had matched prior to the C<\K> and 1232not include it in C<$&>. This effectively provides variable-length 1233look-behind. The use of C<\K> inside of another look-around assertion 1234is allowed, but the behaviour is currently not well defined. 1235 1236For various reasons C<\K> may be significantly more efficient than the 1237equivalent C<< (?<=...) >> construct, and it is especially useful in 1238situations where you want to efficiently remove something following 1239something else in a string. For instance 1240 1241 s/(foo)bar/$1/g; 1242 1243can be rewritten as the much more efficient 1244 1245 s/foo\Kbar//g; 1246 1247=item C<(?<!pattern)> 1248X<(?<!)> X<look-behind, negative> X<lookbehind, negative> 1249 1250A zero-width negative look-behind assertion. For example C</(?<!bar)foo/> 1251matches any occurrence of "foo" that does not follow "bar". Works 1252only for fixed-width look-behind. 1253 1254=back 1255 1256=item C<(?'NAME'pattern)> 1257 1258=item C<< (?<NAME>pattern) >> 1259X<< (?<NAME>) >> X<(?'NAME')> X<named capture> X<capture> 1260 1261A named capture group. Identical in every respect to normal capturing 1262parentheses C<()> but for the additional fact that the group 1263can be referred to by name in various regular expression 1264constructs (like C<\g{NAME}>) and can be accessed by name 1265after a successful match via C<%+> or C<%->. See L<perlvar> 1266for more details on the C<%+> and C<%-> hashes. 1267 1268If multiple distinct capture groups have the same name then the 1269$+{NAME} will refer to the leftmost defined group in the match. 1270 1271The forms C<(?'NAME'pattern)> and C<< (?<NAME>pattern) >> are equivalent. 1272 1273B<NOTE:> While the notation of this construct is the same as the similar 1274function in .NET regexes, the behavior is not. In Perl the groups are 1275numbered sequentially regardless of being named or not. Thus in the 1276pattern 1277 1278 /(x)(?<foo>y)(z)/ 1279 1280$+{foo} will be the same as $2, and $3 will contain 'z' instead of 1281the opposite which is what a .NET regex hacker might expect. 1282 1283Currently NAME is restricted to simple identifiers only. 1284In other words, it must match C</^[_A-Za-z][_A-Za-z0-9]*\z/> or 1285its Unicode extension (see L<utf8>), 1286though it isn't extended by the locale (see L<perllocale>). 1287 1288B<NOTE:> In order to make things easier for programmers with experience 1289with the Python or PCRE regex engines, the pattern C<< (?PE<lt>NAMEE<gt>pattern) >> 1290may be used instead of C<< (?<NAME>pattern) >>; however this form does not 1291support the use of single quotes as a delimiter for the name. 1292 1293=item C<< \k<NAME> >> 1294 1295=item C<< \k'NAME' >> 1296 1297Named backreference. Similar to numeric backreferences, except that 1298the group is designated by name and not number. If multiple groups 1299have the same name then it refers to the leftmost defined group in 1300the current match. 1301 1302It is an error to refer to a name not defined by a C<< (?<NAME>) >> 1303earlier in the pattern. 1304 1305Both forms are equivalent. 1306 1307B<NOTE:> In order to make things easier for programmers with experience 1308with the Python or PCRE regex engines, the pattern C<< (?P=NAME) >> 1309may be used instead of C<< \k<NAME> >>. 1310 1311=item C<(?{ code })> 1312X<(?{})> X<regex, code in> X<regexp, code in> X<regular expression, code in> 1313 1314B<WARNING>: Using this feature safely requires that you understand its 1315limitations. Code executed that has side effects may not perform identically 1316from version to version due to the effect of future optimisations in the regex 1317engine. For more information on this, see L</Embedded Code Execution 1318Frequency>. 1319 1320This zero-width assertion executes any embedded Perl code. It always 1321succeeds, and its return value is set as C<$^R>. 1322 1323In literal patterns, the code is parsed at the same time as the 1324surrounding code. While within the pattern, control is passed temporarily 1325back to the perl parser, until the logically-balancing closing brace is 1326encountered. This is similar to the way that an array index expression in 1327a literal string is handled, for example 1328 1329 "abc$array[ 1 + f('[') + g()]def" 1330 1331In particular, braces do not need to be balanced: 1332 1333 s/abc(?{ f('{'); })/def/ 1334 1335Even in a pattern that is interpolated and compiled at run-time, literal 1336code blocks will be compiled once, at perl compile time; the following 1337prints "ABCD": 1338 1339 print "D"; 1340 my $qr = qr/(?{ BEGIN { print "A" } })/; 1341 my $foo = "foo"; 1342 /$foo$qr(?{ BEGIN { print "B" } })/; 1343 BEGIN { print "C" } 1344 1345In patterns where the text of the code is derived from run-time 1346information rather than appearing literally in a source code /pattern/, 1347the code is compiled at the same time that the pattern is compiled, and 1348for reasons of security, C<use re 'eval'> must be in scope. This is to 1349stop user-supplied patterns containing code snippets from being 1350executable. 1351 1352In situations where you need to enable this with C<use re 'eval'>, you should 1353also have taint checking enabled. Better yet, use the carefully 1354constrained evaluation within a Safe compartment. See L<perlsec> for 1355details about both these mechanisms. 1356 1357From the viewpoint of parsing, lexical variable scope and closures, 1358 1359 /AAA(?{ BBB })CCC/ 1360 1361behaves approximately like 1362 1363 /AAA/ && do { BBB } && /CCC/ 1364 1365Similarly, 1366 1367 qr/AAA(?{ BBB })CCC/ 1368 1369behaves approximately like 1370 1371 sub { /AAA/ && do { BBB } && /CCC/ } 1372 1373In particular: 1374 1375 { my $i = 1; $r = qr/(?{ print $i })/ } 1376 my $i = 2; 1377 /$r/; # prints "1" 1378 1379Inside a C<(?{...})> block, C<$_> refers to the string the regular 1380expression is matching against. You can also use C<pos()> to know what is 1381the current position of matching within this string. 1382 1383The code block introduces a new scope from the perspective of lexical 1384variable declarations, but B<not> from the perspective of C<local> and 1385similar localizing behaviours. So later code blocks within the same 1386pattern will still see the values which were localized in earlier blocks. 1387These accumulated localizations are undone either at the end of a 1388successful match, or if the assertion is backtracked (compare 1389L<"Backtracking">). For example, 1390 1391 $_ = 'a' x 8; 1392 m< 1393 (?{ $cnt = 0 }) # Initialize $cnt. 1394 ( 1395 a 1396 (?{ 1397 local $cnt = $cnt + 1; # Update $cnt, 1398 # backtracking-safe. 1399 }) 1400 )* 1401 aaaa 1402 (?{ $res = $cnt }) # On success copy to 1403 # non-localized location. 1404 >x; 1405 1406will initially increment C<$cnt> up to 8; then during backtracking, its 1407value will be unwound back to 4, which is the value assigned to C<$res>. 1408At the end of the regex execution, $cnt will be wound back to its initial 1409value of 0. 1410 1411This assertion may be used as the condition in a 1412 1413 (?(condition)yes-pattern|no-pattern) 1414 1415switch. If I<not> used in this way, the result of evaluation of C<code> 1416is put into the special variable C<$^R>. This happens immediately, so 1417C<$^R> can be used from other C<(?{ code })> assertions inside the same 1418regular expression. 1419 1420The assignment to C<$^R> above is properly localized, so the old 1421value of C<$^R> is restored if the assertion is backtracked; compare 1422L<"Backtracking">. 1423 1424Note that the special variable C<$^N> is particularly useful with code 1425blocks to capture the results of submatches in variables without having to 1426keep track of the number of nested parentheses. For example: 1427 1428 $_ = "The brown fox jumps over the lazy dog"; 1429 /the (\S+)(?{ $color = $^N }) (\S+)(?{ $animal = $^N })/i; 1430 print "color = $color, animal = $animal\n"; 1431 1432 1433=item C<(??{ code })> 1434X<(??{})> 1435X<regex, postponed> X<regexp, postponed> X<regular expression, postponed> 1436 1437B<WARNING>: Using this feature safely requires that you understand its 1438limitations. Code executed that has side effects may not perform 1439identically from version to version due to the effect of future 1440optimisations in the regex engine. For more information on this, see 1441L</Embedded Code Execution Frequency>. 1442 1443This is a "postponed" regular subexpression. It behaves in I<exactly> the 1444same way as a C<(?{ code })> code block as described above, except that 1445its return value, rather than being assigned to C<$^R>, is treated as a 1446pattern, compiled if it's a string (or used as-is if its a qr// object), 1447then matched as if it were inserted instead of this construct. 1448 1449During the matching of this sub-pattern, it has its own set of 1450captures which are valid during the sub-match, but are discarded once 1451control returns to the main pattern. For example, the following matches, 1452with the inner pattern capturing "B" and matching "BB", while the outer 1453pattern captures "A"; 1454 1455 my $inner = '(.)\1'; 1456 "ABBA" =~ /^(.)(??{ $inner })\1/; 1457 print $1; # prints "A"; 1458 1459Note that this means that there is no way for the inner pattern to refer 1460to a capture group defined outside. (The code block itself can use C<$1>, 1461etc., to refer to the enclosing pattern's capture groups.) Thus, although 1462 1463 ('a' x 100)=~/(??{'(.)' x 100})/ 1464 1465I<will> match, it will I<not> set $1 on exit. 1466 1467The following pattern matches a parenthesized group: 1468 1469 $re = qr{ 1470 \( 1471 (?: 1472 (?> [^()]+ ) # Non-parens without backtracking 1473 | 1474 (??{ $re }) # Group with matching parens 1475 )* 1476 \) 1477 }x; 1478 1479See also 1480L<C<(?I<PARNO>)>|/(?PARNO) (?-PARNO) (?+PARNO) (?R) (?0)> 1481for a different, more efficient way to accomplish 1482the same task. 1483 1484Executing a postponed regular expression 50 times without consuming any 1485input string will result in a fatal error. The maximum depth is compiled 1486into perl, so changing it requires a custom build. 1487 1488=item C<(?I<PARNO>)> C<(?-I<PARNO>)> C<(?+I<PARNO>)> C<(?R)> C<(?0)> 1489X<(?PARNO)> X<(?1)> X<(?R)> X<(?0)> X<(?-1)> X<(?+1)> X<(?-PARNO)> X<(?+PARNO)> 1490X<regex, recursive> X<regexp, recursive> X<regular expression, recursive> 1491X<regex, relative recursion> X<GOSUB> X<GOSTART> 1492 1493Recursive subpattern. Treat the contents of a given capture buffer in the 1494current pattern as an independent subpattern and attempt to match it at 1495the current position in the string. Information about capture state from 1496the caller for things like backreferences is available to the subpattern, 1497but capture buffers set by the subpattern are not visible to the caller. 1498 1499Similar to C<(??{ code })> except that it does not involve executing any 1500code or potentially compiling a returned pattern string; instead it treats 1501the part of the current pattern contained within a specified capture group 1502as an independent pattern that must match at the current position. Also 1503different is the treatment of capture buffers, unlike C<(??{ code })> 1504recursive patterns have access to their callers match state, so one can 1505use backreferences safely. 1506 1507I<PARNO> is a sequence of digits (not starting with 0) whose value reflects 1508the paren-number of the capture group to recurse to. C<(?R)> recurses to 1509the beginning of the whole pattern. C<(?0)> is an alternate syntax for 1510C<(?R)>. If I<PARNO> is preceded by a plus or minus sign then it is assumed 1511to be relative, with negative numbers indicating preceding capture groups 1512and positive ones following. Thus C<(?-1)> refers to the most recently 1513declared group, and C<(?+1)> indicates the next group to be declared. 1514Note that the counting for relative recursion differs from that of 1515relative backreferences, in that with recursion unclosed groups B<are> 1516included. 1517 1518The following pattern matches a function foo() which may contain 1519balanced parentheses as the argument. 1520 1521 $re = qr{ ( # paren group 1 (full function) 1522 foo 1523 ( # paren group 2 (parens) 1524 \( 1525 ( # paren group 3 (contents of parens) 1526 (?: 1527 (?> [^()]+ ) # Non-parens without backtracking 1528 | 1529 (?2) # Recurse to start of paren group 2 1530 )* 1531 ) 1532 \) 1533 ) 1534 ) 1535 }x; 1536 1537If the pattern was used as follows 1538 1539 'foo(bar(baz)+baz(bop))'=~/$re/ 1540 and print "\$1 = $1\n", 1541 "\$2 = $2\n", 1542 "\$3 = $3\n"; 1543 1544the output produced should be the following: 1545 1546 $1 = foo(bar(baz)+baz(bop)) 1547 $2 = (bar(baz)+baz(bop)) 1548 $3 = bar(baz)+baz(bop) 1549 1550If there is no corresponding capture group defined, then it is a 1551fatal error. Recursing deeper than 50 times without consuming any input 1552string will also result in a fatal error. The maximum depth is compiled 1553into perl, so changing it requires a custom build. 1554 1555The following shows how using negative indexing can make it 1556easier to embed recursive patterns inside of a C<qr//> construct 1557for later use: 1558 1559 my $parens = qr/(\((?:[^()]++|(?-1))*+\))/; 1560 if (/foo $parens \s+ \+ \s+ bar $parens/x) { 1561 # do something here... 1562 } 1563 1564B<Note> that this pattern does not behave the same way as the equivalent 1565PCRE or Python construct of the same form. In Perl you can backtrack into 1566a recursed group, in PCRE and Python the recursed into group is treated 1567as atomic. Also, modifiers are resolved at compile time, so constructs 1568like (?i:(?1)) or (?:(?i)(?1)) do not affect how the sub-pattern will 1569be processed. 1570 1571=item C<(?&NAME)> 1572X<(?&NAME)> 1573 1574Recurse to a named subpattern. Identical to C<(?I<PARNO>)> except that the 1575parenthesis to recurse to is determined by name. If multiple parentheses have 1576the same name, then it recurses to the leftmost. 1577 1578It is an error to refer to a name that is not declared somewhere in the 1579pattern. 1580 1581B<NOTE:> In order to make things easier for programmers with experience 1582with the Python or PCRE regex engines the pattern C<< (?P>NAME) >> 1583may be used instead of C<< (?&NAME) >>. 1584 1585=item C<(?(condition)yes-pattern|no-pattern)> 1586X<(?()> 1587 1588=item C<(?(condition)yes-pattern)> 1589 1590Conditional expression. Matches C<yes-pattern> if C<condition> yields 1591a true value, matches C<no-pattern> otherwise. A missing pattern always 1592matches. 1593 1594C<(condition)> should be one of: 1) an integer in 1595parentheses (which is valid if the corresponding pair of parentheses 1596matched); 2) a look-ahead/look-behind/evaluate zero-width assertion; 3) a 1597name in angle brackets or single quotes (which is valid if a group 1598with the given name matched); or 4) the special symbol (R) (true when 1599evaluated inside of recursion or eval). Additionally the R may be 1600followed by a number, (which will be true when evaluated when recursing 1601inside of the appropriate group), or by C<&NAME>, in which case it will 1602be true only when evaluated during recursion in the named group. 1603 1604Here's a summary of the possible predicates: 1605 1606=over 4 1607 1608=item (1) (2) ... 1609 1610Checks if the numbered capturing group has matched something. 1611 1612=item (<NAME>) ('NAME') 1613 1614Checks if a group with the given name has matched something. 1615 1616=item (?=...) (?!...) (?<=...) (?<!...) 1617 1618Checks whether the pattern matches (or does not match, for the '!' 1619variants). 1620 1621=item (?{ CODE }) 1622 1623Treats the return value of the code block as the condition. 1624 1625=item (R) 1626 1627Checks if the expression has been evaluated inside of recursion. 1628 1629=item (R1) (R2) ... 1630 1631Checks if the expression has been evaluated while executing directly 1632inside of the n-th capture group. This check is the regex equivalent of 1633 1634 if ((caller(0))[3] eq 'subname') { ... } 1635 1636In other words, it does not check the full recursion stack. 1637 1638=item (R&NAME) 1639 1640Similar to C<(R1)>, this predicate checks to see if we're executing 1641directly inside of the leftmost group with a given name (this is the same 1642logic used by C<(?&NAME)> to disambiguate). It does not check the full 1643stack, but only the name of the innermost active recursion. 1644 1645=item (DEFINE) 1646 1647In this case, the yes-pattern is never directly executed, and no 1648no-pattern is allowed. Similar in spirit to C<(?{0})> but more efficient. 1649See below for details. 1650 1651=back 1652 1653For example: 1654 1655 m{ ( \( )? 1656 [^()]+ 1657 (?(1) \) ) 1658 }x 1659 1660matches a chunk of non-parentheses, possibly included in parentheses 1661themselves. 1662 1663A special form is the C<(DEFINE)> predicate, which never executes its 1664yes-pattern directly, and does not allow a no-pattern. This allows one to 1665define subpatterns which will be executed only by the recursion mechanism. 1666This way, you can define a set of regular expression rules that can be 1667bundled into any pattern you choose. 1668 1669It is recommended that for this usage you put the DEFINE block at the 1670end of the pattern, and that you name any subpatterns defined within it. 1671 1672Also, it's worth noting that patterns defined this way probably will 1673not be as efficient, as the optimizer is not very clever about 1674handling them. 1675 1676An example of how this might be used is as follows: 1677 1678 /(?<NAME>(?&NAME_PAT))(?<ADDR>(?&ADDRESS_PAT)) 1679 (?(DEFINE) 1680 (?<NAME_PAT>....) 1681 (?<ADDRESS_PAT>....) 1682 )/x 1683 1684Note that capture groups matched inside of recursion are not accessible 1685after the recursion returns, so the extra layer of capturing groups is 1686necessary. Thus C<$+{NAME_PAT}> would not be defined even though 1687C<$+{NAME}> would be. 1688 1689Finally, keep in mind that subpatterns created inside a DEFINE block 1690count towards the absolute and relative number of captures, so this: 1691 1692 my @captures = "a" =~ /(.) # First capture 1693 (?(DEFINE) 1694 (?<EXAMPLE> 1 ) # Second capture 1695 )/x; 1696 say scalar @captures; 1697 1698Will output 2, not 1. This is particularly important if you intend to 1699compile the definitions with the C<qr//> operator, and later 1700interpolate them in another pattern. 1701 1702=item C<< (?>pattern) >> 1703X<backtrack> X<backtracking> X<atomic> X<possessive> 1704 1705An "independent" subexpression, one which matches the substring 1706that a I<standalone> C<pattern> would match if anchored at the given 1707position, and it matches I<nothing other than this substring>. This 1708construct is useful for optimizations of what would otherwise be 1709"eternal" matches, because it will not backtrack (see L<"Backtracking">). 1710It may also be useful in places where the "grab all you can, and do not 1711give anything back" semantic is desirable. 1712 1713For example: C<< ^(?>a*)ab >> will never match, since C<< (?>a*) >> 1714(anchored at the beginning of string, as above) will match I<all> 1715characters C<a> at the beginning of string, leaving no C<a> for 1716C<ab> to match. In contrast, C<a*ab> will match the same as C<a+b>, 1717since the match of the subgroup C<a*> is influenced by the following 1718group C<ab> (see L<"Backtracking">). In particular, C<a*> inside 1719C<a*ab> will match fewer characters than a standalone C<a*>, since 1720this makes the tail match. 1721 1722C<< (?>pattern) >> does not disable backtracking altogether once it has 1723matched. It is still possible to backtrack past the construct, but not 1724into it. So C<< ((?>a*)|(?>b*))ar >> will still match "bar". 1725 1726An effect similar to C<< (?>pattern) >> may be achieved by writing 1727C<(?=(pattern))\g{-1}>. This matches the same substring as a standalone 1728C<a+>, and the following C<\g{-1}> eats the matched string; it therefore 1729makes a zero-length assertion into an analogue of C<< (?>...) >>. 1730(The difference between these two constructs is that the second one 1731uses a capturing group, thus shifting ordinals of backreferences 1732in the rest of a regular expression.) 1733 1734Consider this pattern: 1735 1736 m{ \( 1737 ( 1738 [^()]+ # x+ 1739 | 1740 \( [^()]* \) 1741 )+ 1742 \) 1743 }x 1744 1745That will efficiently match a nonempty group with matching parentheses 1746two levels deep or less. However, if there is no such group, it 1747will take virtually forever on a long string. That's because there 1748are so many different ways to split a long string into several 1749substrings. This is what C<(.+)+> is doing, and C<(.+)+> is similar 1750to a subpattern of the above pattern. Consider how the pattern 1751above detects no-match on C<((()aaaaaaaaaaaaaaaaaa> in several 1752seconds, but that each extra letter doubles this time. This 1753exponential performance will make it appear that your program has 1754hung. However, a tiny change to this pattern 1755 1756 m{ \( 1757 ( 1758 (?> [^()]+ ) # change x+ above to (?> x+ ) 1759 | 1760 \( [^()]* \) 1761 )+ 1762 \) 1763 }x 1764 1765which uses C<< (?>...) >> matches exactly when the one above does (verifying 1766this yourself would be a productive exercise), but finishes in a fourth 1767the time when used on a similar string with 1000000 C<a>s. Be aware, 1768however, that, when this construct is followed by a 1769quantifier, it currently triggers a warning message under 1770the C<use warnings> pragma or B<-w> switch saying it 1771C<"matches null string many times in regex">. 1772 1773On simple groups, such as the pattern C<< (?> [^()]+ ) >>, a comparable 1774effect may be achieved by negative look-ahead, as in C<[^()]+ (?! [^()] )>. 1775This was only 4 times slower on a string with 1000000 C<a>s. 1776 1777The "grab all you can, and do not give anything back" semantic is desirable 1778in many situations where on the first sight a simple C<()*> looks like 1779the correct solution. Suppose we parse text with comments being delimited 1780by C<#> followed by some optional (horizontal) whitespace. Contrary to 1781its appearance, C<#[ \t]*> I<is not> the correct subexpression to match 1782the comment delimiter, because it may "give up" some whitespace if 1783the remainder of the pattern can be made to match that way. The correct 1784answer is either one of these: 1785 1786 (?>#[ \t]*) 1787 #[ \t]*(?![ \t]) 1788 1789For example, to grab non-empty comments into $1, one should use either 1790one of these: 1791 1792 / (?> \# [ \t]* ) ( .+ ) /x; 1793 / \# [ \t]* ( [^ \t] .* ) /x; 1794 1795Which one you pick depends on which of these expressions better reflects 1796the above specification of comments. 1797 1798In some literature this construct is called "atomic matching" or 1799"possessive matching". 1800 1801Possessive quantifiers are equivalent to putting the item they are applied 1802to inside of one of these constructs. The following equivalences apply: 1803 1804 Quantifier Form Bracketing Form 1805 --------------- --------------- 1806 PAT*+ (?>PAT*) 1807 PAT++ (?>PAT+) 1808 PAT?+ (?>PAT?) 1809 PAT{min,max}+ (?>PAT{min,max}) 1810 1811=item C<(?[ ])> 1812 1813See L<perlrecharclass/Extended Bracketed Character Classes>. 1814 1815=back 1816 1817=head2 Special Backtracking Control Verbs 1818 1819These special patterns are generally of the form C<(*VERB:ARG)>. Unless 1820otherwise stated the ARG argument is optional; in some cases, it is 1821forbidden. 1822 1823Any pattern containing a special backtracking verb that allows an argument 1824has the special behaviour that when executed it sets the current package's 1825C<$REGERROR> and C<$REGMARK> variables. When doing so the following 1826rules apply: 1827 1828On failure, the C<$REGERROR> variable will be set to the ARG value of the 1829verb pattern, if the verb was involved in the failure of the match. If the 1830ARG part of the pattern was omitted, then C<$REGERROR> will be set to the 1831name of the last C<(*MARK:NAME)> pattern executed, or to TRUE if there was 1832none. Also, the C<$REGMARK> variable will be set to FALSE. 1833 1834On a successful match, the C<$REGERROR> variable will be set to FALSE, and 1835the C<$REGMARK> variable will be set to the name of the last 1836C<(*MARK:NAME)> pattern executed. See the explanation for the 1837C<(*MARK:NAME)> verb below for more details. 1838 1839B<NOTE:> C<$REGERROR> and C<$REGMARK> are not magic variables like C<$1> 1840and most other regex-related variables. They are not local to a scope, nor 1841readonly, but instead are volatile package variables similar to C<$AUTOLOAD>. 1842Use C<local> to localize changes to them to a specific scope if necessary. 1843 1844If a pattern does not contain a special backtracking verb that allows an 1845argument, then C<$REGERROR> and C<$REGMARK> are not touched at all. 1846 1847=over 3 1848 1849=item Verbs that take an argument 1850 1851=over 4 1852 1853=item C<(*PRUNE)> C<(*PRUNE:NAME)> 1854X<(*PRUNE)> X<(*PRUNE:NAME)> 1855 1856This zero-width pattern prunes the backtracking tree at the current point 1857when backtracked into on failure. Consider the pattern C<A (*PRUNE) B>, 1858where A and B are complex patterns. Until the C<(*PRUNE)> verb is reached, 1859A may backtrack as necessary to match. Once it is reached, matching 1860continues in B, which may also backtrack as necessary; however, should B 1861not match, then no further backtracking will take place, and the pattern 1862will fail outright at the current starting position. 1863 1864The following example counts all the possible matching strings in a 1865pattern (without actually matching any of them). 1866 1867 'aaab' =~ /a+b?(?{print "$&\n"; $count++})(*FAIL)/; 1868 print "Count=$count\n"; 1869 1870which produces: 1871 1872 aaab 1873 aaa 1874 aa 1875 a 1876 aab 1877 aa 1878 a 1879 ab 1880 a 1881 Count=9 1882 1883If we add a C<(*PRUNE)> before the count like the following 1884 1885 'aaab' =~ /a+b?(*PRUNE)(?{print "$&\n"; $count++})(*FAIL)/; 1886 print "Count=$count\n"; 1887 1888we prevent backtracking and find the count of the longest matching string 1889at each matching starting point like so: 1890 1891 aaab 1892 aab 1893 ab 1894 Count=3 1895 1896Any number of C<(*PRUNE)> assertions may be used in a pattern. 1897 1898See also C<< (?>pattern) >> and possessive quantifiers for other ways to 1899control backtracking. In some cases, the use of C<(*PRUNE)> can be 1900replaced with a C<< (?>pattern) >> with no functional difference; however, 1901C<(*PRUNE)> can be used to handle cases that cannot be expressed using a 1902C<< (?>pattern) >> alone. 1903 1904=item C<(*SKIP)> C<(*SKIP:NAME)> 1905X<(*SKIP)> 1906 1907This zero-width pattern is similar to C<(*PRUNE)>, except that on 1908failure it also signifies that whatever text that was matched leading up 1909to the C<(*SKIP)> pattern being executed cannot be part of I<any> match 1910of this pattern. This effectively means that the regex engine "skips" forward 1911to this position on failure and tries to match again, (assuming that 1912there is sufficient room to match). 1913 1914The name of the C<(*SKIP:NAME)> pattern has special significance. If a 1915C<(*MARK:NAME)> was encountered while matching, then it is that position 1916which is used as the "skip point". If no C<(*MARK)> of that name was 1917encountered, then the C<(*SKIP)> operator has no effect. When used 1918without a name the "skip point" is where the match point was when 1919executing the (*SKIP) pattern. 1920 1921Compare the following to the examples in C<(*PRUNE)>; note the string 1922is twice as long: 1923 1924 'aaabaaab' =~ /a+b?(*SKIP)(?{print "$&\n"; $count++})(*FAIL)/; 1925 print "Count=$count\n"; 1926 1927outputs 1928 1929 aaab 1930 aaab 1931 Count=2 1932 1933Once the 'aaab' at the start of the string has matched, and the C<(*SKIP)> 1934executed, the next starting point will be where the cursor was when the 1935C<(*SKIP)> was executed. 1936 1937=item C<(*MARK:NAME)> C<(*:NAME)> 1938X<(*MARK)> X<(*MARK:NAME)> X<(*:NAME)> 1939 1940This zero-width pattern can be used to mark the point reached in a string 1941when a certain part of the pattern has been successfully matched. This 1942mark may be given a name. A later C<(*SKIP)> pattern will then skip 1943forward to that point if backtracked into on failure. Any number of 1944C<(*MARK)> patterns are allowed, and the NAME portion may be duplicated. 1945 1946In addition to interacting with the C<(*SKIP)> pattern, C<(*MARK:NAME)> 1947can be used to "label" a pattern branch, so that after matching, the 1948program can determine which branches of the pattern were involved in the 1949match. 1950 1951When a match is successful, the C<$REGMARK> variable will be set to the 1952name of the most recently executed C<(*MARK:NAME)> that was involved 1953in the match. 1954 1955This can be used to determine which branch of a pattern was matched 1956without using a separate capture group for each branch, which in turn 1957can result in a performance improvement, as perl cannot optimize 1958C</(?:(x)|(y)|(z))/> as efficiently as something like 1959C</(?:x(*MARK:x)|y(*MARK:y)|z(*MARK:z))/>. 1960 1961When a match has failed, and unless another verb has been involved in 1962failing the match and has provided its own name to use, the C<$REGERROR> 1963variable will be set to the name of the most recently executed 1964C<(*MARK:NAME)>. 1965 1966See L</(*SKIP)> for more details. 1967 1968As a shortcut C<(*MARK:NAME)> can be written C<(*:NAME)>. 1969 1970=item C<(*THEN)> C<(*THEN:NAME)> 1971 1972This is similar to the "cut group" operator C<::> from Perl 6. Like 1973C<(*PRUNE)>, this verb always matches, and when backtracked into on 1974failure, it causes the regex engine to try the next alternation in the 1975innermost enclosing group (capturing or otherwise) that has alternations. 1976The two branches of a C<(?(condition)yes-pattern|no-pattern)> do not 1977count as an alternation, as far as C<(*THEN)> is concerned. 1978 1979Its name comes from the observation that this operation combined with the 1980alternation operator (C<|>) can be used to create what is essentially a 1981pattern-based if/then/else block: 1982 1983 ( COND (*THEN) FOO | COND2 (*THEN) BAR | COND3 (*THEN) BAZ ) 1984 1985Note that if this operator is used and NOT inside of an alternation then 1986it acts exactly like the C<(*PRUNE)> operator. 1987 1988 / A (*PRUNE) B / 1989 1990is the same as 1991 1992 / A (*THEN) B / 1993 1994but 1995 1996 / ( A (*THEN) B | C ) / 1997 1998is not the same as 1999 2000 / ( A (*PRUNE) B | C ) / 2001 2002as after matching the A but failing on the B the C<(*THEN)> verb will 2003backtrack and try C; but the C<(*PRUNE)> verb will simply fail. 2004 2005=back 2006 2007=item Verbs without an argument 2008 2009=over 4 2010 2011=item C<(*COMMIT)> 2012X<(*COMMIT)> 2013 2014This is the Perl 6 "commit pattern" C<< <commit> >> or C<:::>. It's a 2015zero-width pattern similar to C<(*SKIP)>, except that when backtracked 2016into on failure it causes the match to fail outright. No further attempts 2017to find a valid match by advancing the start pointer will occur again. 2018For example, 2019 2020 'aaabaaab' =~ /a+b?(*COMMIT)(?{print "$&\n"; $count++})(*FAIL)/; 2021 print "Count=$count\n"; 2022 2023outputs 2024 2025 aaab 2026 Count=1 2027 2028In other words, once the C<(*COMMIT)> has been entered, and if the pattern 2029does not match, the regex engine will not try any further matching on the 2030rest of the string. 2031 2032=item C<(*FAIL)> C<(*F)> 2033X<(*FAIL)> X<(*F)> 2034 2035This pattern matches nothing and always fails. It can be used to force the 2036engine to backtrack. It is equivalent to C<(?!)>, but easier to read. In 2037fact, C<(?!)> gets optimised into C<(*FAIL)> internally. 2038 2039It is probably useful only when combined with C<(?{})> or C<(??{})>. 2040 2041=item C<(*ACCEPT)> 2042X<(*ACCEPT)> 2043 2044This pattern matches nothing and causes the end of successful matching at 2045the point at which the C<(*ACCEPT)> pattern was encountered, regardless of 2046whether there is actually more to match in the string. When inside of a 2047nested pattern, such as recursion, or in a subpattern dynamically generated 2048via C<(??{})>, only the innermost pattern is ended immediately. 2049 2050If the C<(*ACCEPT)> is inside of capturing groups then the groups are 2051marked as ended at the point at which the C<(*ACCEPT)> was encountered. 2052For instance: 2053 2054 'AB' =~ /(A (A|B(*ACCEPT)|C) D)(E)/x; 2055 2056will match, and C<$1> will be C<AB> and C<$2> will be C<B>, C<$3> will not 2057be set. If another branch in the inner parentheses was matched, such as in the 2058string 'ACDE', then the C<D> and C<E> would have to be matched as well. 2059 2060=back 2061 2062=back 2063 2064=head2 Backtracking 2065X<backtrack> X<backtracking> 2066 2067NOTE: This section presents an abstract approximation of regular 2068expression behavior. For a more rigorous (and complicated) view of 2069the rules involved in selecting a match among possible alternatives, 2070see L<Combining RE Pieces>. 2071 2072A fundamental feature of regular expression matching involves the 2073notion called I<backtracking>, which is currently used (when needed) 2074by all regular non-possessive expression quantifiers, namely C<*>, C<*?>, C<+>, 2075C<+?>, C<{n,m}>, and C<{n,m}?>. Backtracking is often optimized 2076internally, but the general principle outlined here is valid. 2077 2078For a regular expression to match, the I<entire> regular expression must 2079match, not just part of it. So if the beginning of a pattern containing a 2080quantifier succeeds in a way that causes later parts in the pattern to 2081fail, the matching engine backs up and recalculates the beginning 2082part--that's why it's called backtracking. 2083 2084Here is an example of backtracking: Let's say you want to find the 2085word following "foo" in the string "Food is on the foo table.": 2086 2087 $_ = "Food is on the foo table."; 2088 if ( /\b(foo)\s+(\w+)/i ) { 2089 print "$2 follows $1.\n"; 2090 } 2091 2092When the match runs, the first part of the regular expression (C<\b(foo)>) 2093finds a possible match right at the beginning of the string, and loads up 2094$1 with "Foo". However, as soon as the matching engine sees that there's 2095no whitespace following the "Foo" that it had saved in $1, it realizes its 2096mistake and starts over again one character after where it had the 2097tentative match. This time it goes all the way until the next occurrence 2098of "foo". The complete regular expression matches this time, and you get 2099the expected output of "table follows foo." 2100 2101Sometimes minimal matching can help a lot. Imagine you'd like to match 2102everything between "foo" and "bar". Initially, you write something 2103like this: 2104 2105 $_ = "The food is under the bar in the barn."; 2106 if ( /foo(.*)bar/ ) { 2107 print "got <$1>\n"; 2108 } 2109 2110Which perhaps unexpectedly yields: 2111 2112 got <d is under the bar in the > 2113 2114That's because C<.*> was greedy, so you get everything between the 2115I<first> "foo" and the I<last> "bar". Here it's more effective 2116to use minimal matching to make sure you get the text between a "foo" 2117and the first "bar" thereafter. 2118 2119 if ( /foo(.*?)bar/ ) { print "got <$1>\n" } 2120 got <d is under the > 2121 2122Here's another example. Let's say you'd like to match a number at the end 2123of a string, and you also want to keep the preceding part of the match. 2124So you write this: 2125 2126 $_ = "I have 2 numbers: 53147"; 2127 if ( /(.*)(\d*)/ ) { # Wrong! 2128 print "Beginning is <$1>, number is <$2>.\n"; 2129 } 2130 2131That won't work at all, because C<.*> was greedy and gobbled up the 2132whole string. As C<\d*> can match on an empty string the complete 2133regular expression matched successfully. 2134 2135 Beginning is <I have 2 numbers: 53147>, number is <>. 2136 2137Here are some variants, most of which don't work: 2138 2139 $_ = "I have 2 numbers: 53147"; 2140 @pats = qw{ 2141 (.*)(\d*) 2142 (.*)(\d+) 2143 (.*?)(\d*) 2144 (.*?)(\d+) 2145 (.*)(\d+)$ 2146 (.*?)(\d+)$ 2147 (.*)\b(\d+)$ 2148 (.*\D)(\d+)$ 2149 }; 2150 2151 for $pat (@pats) { 2152 printf "%-12s ", $pat; 2153 if ( /$pat/ ) { 2154 print "<$1> <$2>\n"; 2155 } else { 2156 print "FAIL\n"; 2157 } 2158 } 2159 2160That will print out: 2161 2162 (.*)(\d*) <I have 2 numbers: 53147> <> 2163 (.*)(\d+) <I have 2 numbers: 5314> <7> 2164 (.*?)(\d*) <> <> 2165 (.*?)(\d+) <I have > <2> 2166 (.*)(\d+)$ <I have 2 numbers: 5314> <7> 2167 (.*?)(\d+)$ <I have 2 numbers: > <53147> 2168 (.*)\b(\d+)$ <I have 2 numbers: > <53147> 2169 (.*\D)(\d+)$ <I have 2 numbers: > <53147> 2170 2171As you see, this can be a bit tricky. It's important to realize that a 2172regular expression is merely a set of assertions that gives a definition 2173of success. There may be 0, 1, or several different ways that the 2174definition might succeed against a particular string. And if there are 2175multiple ways it might succeed, you need to understand backtracking to 2176know which variety of success you will achieve. 2177 2178When using look-ahead assertions and negations, this can all get even 2179trickier. Imagine you'd like to find a sequence of non-digits not 2180followed by "123". You might try to write that as 2181 2182 $_ = "ABC123"; 2183 if ( /^\D*(?!123)/ ) { # Wrong! 2184 print "Yup, no 123 in $_\n"; 2185 } 2186 2187But that isn't going to match; at least, not the way you're hoping. It 2188claims that there is no 123 in the string. Here's a clearer picture of 2189why that pattern matches, contrary to popular expectations: 2190 2191 $x = 'ABC123'; 2192 $y = 'ABC445'; 2193 2194 print "1: got $1\n" if $x =~ /^(ABC)(?!123)/; 2195 print "2: got $1\n" if $y =~ /^(ABC)(?!123)/; 2196 2197 print "3: got $1\n" if $x =~ /^(\D*)(?!123)/; 2198 print "4: got $1\n" if $y =~ /^(\D*)(?!123)/; 2199 2200This prints 2201 2202 2: got ABC 2203 3: got AB 2204 4: got ABC 2205 2206You might have expected test 3 to fail because it seems to a more 2207general purpose version of test 1. The important difference between 2208them is that test 3 contains a quantifier (C<\D*>) and so can use 2209backtracking, whereas test 1 will not. What's happening is 2210that you've asked "Is it true that at the start of $x, following 0 or more 2211non-digits, you have something that's not 123?" If the pattern matcher had 2212let C<\D*> expand to "ABC", this would have caused the whole pattern to 2213fail. 2214 2215The search engine will initially match C<\D*> with "ABC". Then it will 2216try to match C<(?!123)> with "123", which fails. But because 2217a quantifier (C<\D*>) has been used in the regular expression, the 2218search engine can backtrack and retry the match differently 2219in the hope of matching the complete regular expression. 2220 2221The pattern really, I<really> wants to succeed, so it uses the 2222standard pattern back-off-and-retry and lets C<\D*> expand to just "AB" this 2223time. Now there's indeed something following "AB" that is not 2224"123". It's "C123", which suffices. 2225 2226We can deal with this by using both an assertion and a negation. 2227We'll say that the first part in $1 must be followed both by a digit 2228and by something that's not "123". Remember that the look-aheads 2229are zero-width expressions--they only look, but don't consume any 2230of the string in their match. So rewriting this way produces what 2231you'd expect; that is, case 5 will fail, but case 6 succeeds: 2232 2233 print "5: got $1\n" if $x =~ /^(\D*)(?=\d)(?!123)/; 2234 print "6: got $1\n" if $y =~ /^(\D*)(?=\d)(?!123)/; 2235 2236 6: got ABC 2237 2238In other words, the two zero-width assertions next to each other work as though 2239they're ANDed together, just as you'd use any built-in assertions: C</^$/> 2240matches only if you're at the beginning of the line AND the end of the 2241line simultaneously. The deeper underlying truth is that juxtaposition in 2242regular expressions always means AND, except when you write an explicit OR 2243using the vertical bar. C</ab/> means match "a" AND (then) match "b", 2244although the attempted matches are made at different positions because "a" 2245is not a zero-width assertion, but a one-width assertion. 2246 2247B<WARNING>: Particularly complicated regular expressions can take 2248exponential time to solve because of the immense number of possible 2249ways they can use backtracking to try for a match. For example, without 2250internal optimizations done by the regular expression engine, this will 2251take a painfully long time to run: 2252 2253 'aaaaaaaaaaaa' =~ /((a{0,5}){0,5})*[c]/ 2254 2255And if you used C<*>'s in the internal groups instead of limiting them 2256to 0 through 5 matches, then it would take forever--or until you ran 2257out of stack space. Moreover, these internal optimizations are not 2258always applicable. For example, if you put C<{0,5}> instead of C<*> 2259on the external group, no current optimization is applicable, and the 2260match takes a long time to finish. 2261 2262A powerful tool for optimizing such beasts is what is known as an 2263"independent group", 2264which does not backtrack (see L</C<< (?>pattern) >>>). Note also that 2265zero-length look-ahead/look-behind assertions will not backtrack to make 2266the tail match, since they are in "logical" context: only 2267whether they match is considered relevant. For an example 2268where side-effects of look-ahead I<might> have influenced the 2269following match, see L</C<< (?>pattern) >>>. 2270 2271=head2 Version 8 Regular Expressions 2272X<regular expression, version 8> X<regex, version 8> X<regexp, version 8> 2273 2274In case you're not familiar with the "regular" Version 8 regex 2275routines, here are the pattern-matching rules not described above. 2276 2277Any single character matches itself, unless it is a I<metacharacter> 2278with a special meaning described here or above. You can cause 2279characters that normally function as metacharacters to be interpreted 2280literally by prefixing them with a "\" (e.g., "\." matches a ".", not any 2281character; "\\" matches a "\"). This escape mechanism is also required 2282for the character used as the pattern delimiter. 2283 2284A series of characters matches that series of characters in the target 2285string, so the pattern C<blurfl> would match "blurfl" in the target 2286string. 2287 2288You can specify a character class, by enclosing a list of characters 2289in C<[]>, which will match any character from the list. If the 2290first character after the "[" is "^", the class matches any character not 2291in the list. Within a list, the "-" character specifies a 2292range, so that C<a-z> represents all characters between "a" and "z", 2293inclusive. If you want either "-" or "]" itself to be a member of a 2294class, put it at the start of the list (possibly after a "^"), or 2295escape it with a backslash. "-" is also taken literally when it is 2296at the end of the list, just before the closing "]". (The 2297following all specify the same class of three characters: C<[-az]>, 2298C<[az-]>, and C<[a\-z]>. All are different from C<[a-z]>, which 2299specifies a class containing twenty-six characters, even on EBCDIC-based 2300character sets.) Also, if you try to use the character 2301classes C<\w>, C<\W>, C<\s>, C<\S>, C<\d>, or C<\D> as endpoints of 2302a range, the "-" is understood literally. 2303 2304Note also that the whole range idea is rather unportable between 2305character sets--and even within character sets they may cause results 2306you probably didn't expect. A sound principle is to use only ranges 2307that begin from and end at either alphabetics of equal case ([a-e], 2308[A-E]), or digits ([0-9]). Anything else is unsafe. If in doubt, 2309spell out the character sets in full. 2310 2311Characters may be specified using a metacharacter syntax much like that 2312used in C: "\n" matches a newline, "\t" a tab, "\r" a carriage return, 2313"\f" a form feed, etc. More generally, \I<nnn>, where I<nnn> is a string 2314of three octal digits, matches the character whose coded character set value 2315is I<nnn>. Similarly, \xI<nn>, where I<nn> are hexadecimal digits, 2316matches the character whose ordinal is I<nn>. The expression \cI<x> 2317matches the character control-I<x>. Finally, the "." metacharacter 2318matches any character except "\n" (unless you use C</s>). 2319 2320You can specify a series of alternatives for a pattern using "|" to 2321separate them, so that C<fee|fie|foe> will match any of "fee", "fie", 2322or "foe" in the target string (as would C<f(e|i|o)e>). The 2323first alternative includes everything from the last pattern delimiter 2324("(", "(?:", etc. or the beginning of the pattern) up to the first "|", and 2325the last alternative contains everything from the last "|" to the next 2326closing pattern delimiter. That's why it's common practice to include 2327alternatives in parentheses: to minimize confusion about where they 2328start and end. 2329 2330Alternatives are tried from left to right, so the first 2331alternative found for which the entire expression matches, is the one that 2332is chosen. This means that alternatives are not necessarily greedy. For 2333example: when matching C<foo|foot> against "barefoot", only the "foo" 2334part will match, as that is the first alternative tried, and it successfully 2335matches the target string. (This might not seem important, but it is 2336important when you are capturing matched text using parentheses.) 2337 2338Also remember that "|" is interpreted as a literal within square brackets, 2339so if you write C<[fee|fie|foe]> you're really only matching C<[feio|]>. 2340 2341Within a pattern, you may designate subpatterns for later reference 2342by enclosing them in parentheses, and you may refer back to the 2343I<n>th subpattern later in the pattern using the metacharacter 2344\I<n> or \gI<n>. Subpatterns are numbered based on the left to right order 2345of their opening parenthesis. A backreference matches whatever 2346actually matched the subpattern in the string being examined, not 2347the rules for that subpattern. Therefore, C<(0|0x)\d*\s\g1\d*> will 2348match "0x1234 0x4321", but not "0x1234 01234", because subpattern 23491 matched "0x", even though the rule C<0|0x> could potentially match 2350the leading 0 in the second number. 2351 2352=head2 Warning on \1 Instead of $1 2353 2354Some people get too used to writing things like: 2355 2356 $pattern =~ s/(\W)/\\\1/g; 2357 2358This is grandfathered (for \1 to \9) for the RHS of a substitute to avoid 2359shocking the 2360B<sed> addicts, but it's a dirty habit to get into. That's because in 2361PerlThink, the righthand side of an C<s///> is a double-quoted string. C<\1> in 2362the usual double-quoted string means a control-A. The customary Unix 2363meaning of C<\1> is kludged in for C<s///>. However, if you get into the habit 2364of doing that, you get yourself into trouble if you then add an C</e> 2365modifier. 2366 2367 s/(\d+)/ \1 + 1 /eg; # causes warning under -w 2368 2369Or if you try to do 2370 2371 s/(\d+)/\1000/; 2372 2373You can't disambiguate that by saying C<\{1}000>, whereas you can fix it with 2374C<${1}000>. The operation of interpolation should not be confused 2375with the operation of matching a backreference. Certainly they mean two 2376different things on the I<left> side of the C<s///>. 2377 2378=head2 Repeated Patterns Matching a Zero-length Substring 2379 2380B<WARNING>: Difficult material (and prose) ahead. This section needs a rewrite. 2381 2382Regular expressions provide a terse and powerful programming language. As 2383with most other power tools, power comes together with the ability 2384to wreak havoc. 2385 2386A common abuse of this power stems from the ability to make infinite 2387loops using regular expressions, with something as innocuous as: 2388 2389 'foo' =~ m{ ( o? )* }x; 2390 2391The C<o?> matches at the beginning of C<'foo'>, and since the position 2392in the string is not moved by the match, C<o?> would match again and again 2393because of the C<*> quantifier. Another common way to create a similar cycle 2394is with the looping modifier C<//g>: 2395 2396 @matches = ( 'foo' =~ m{ o? }xg ); 2397 2398or 2399 2400 print "match: <$&>\n" while 'foo' =~ m{ o? }xg; 2401 2402or the loop implied by split(). 2403 2404However, long experience has shown that many programming tasks may 2405be significantly simplified by using repeated subexpressions that 2406may match zero-length substrings. Here's a simple example being: 2407 2408 @chars = split //, $string; # // is not magic in split 2409 ($whitewashed = $string) =~ s/()/ /g; # parens avoid magic s// / 2410 2411Thus Perl allows such constructs, by I<forcefully breaking 2412the infinite loop>. The rules for this are different for lower-level 2413loops given by the greedy quantifiers C<*+{}>, and for higher-level 2414ones like the C</g> modifier or split() operator. 2415 2416The lower-level loops are I<interrupted> (that is, the loop is 2417broken) when Perl detects that a repeated expression matched a 2418zero-length substring. Thus 2419 2420 m{ (?: NON_ZERO_LENGTH | ZERO_LENGTH )* }x; 2421 2422is made equivalent to 2423 2424 m{ (?: NON_ZERO_LENGTH )* (?: ZERO_LENGTH )? }x; 2425 2426For example, this program 2427 2428 #!perl -l 2429 "aaaaab" =~ / 2430 (?: 2431 a # non-zero 2432 | # or 2433 (?{print "hello"}) # print hello whenever this 2434 # branch is tried 2435 (?=(b)) # zero-width assertion 2436 )* # any number of times 2437 /x; 2438 print $&; 2439 print $1; 2440 2441prints 2442 2443 hello 2444 aaaaa 2445 b 2446 2447Notice that "hello" is only printed once, as when Perl sees that the sixth 2448iteration of the outermost C<(?:)*> matches a zero-length string, it stops 2449the C<*>. 2450 2451The higher-level loops preserve an additional state between iterations: 2452whether the last match was zero-length. To break the loop, the following 2453match after a zero-length match is prohibited to have a length of zero. 2454This prohibition interacts with backtracking (see L<"Backtracking">), 2455and so the I<second best> match is chosen if the I<best> match is of 2456zero length. 2457 2458For example: 2459 2460 $_ = 'bar'; 2461 s/\w??/<$&>/g; 2462 2463results in C<< <><b><><a><><r><> >>. At each position of the string the best 2464match given by non-greedy C<??> is the zero-length match, and the I<second 2465best> match is what is matched by C<\w>. Thus zero-length matches 2466alternate with one-character-long matches. 2467 2468Similarly, for repeated C<m/()/g> the second-best match is the match at the 2469position one notch further in the string. 2470 2471The additional state of being I<matched with zero-length> is associated with 2472the matched string, and is reset by each assignment to pos(). 2473Zero-length matches at the end of the previous match are ignored 2474during C<split>. 2475 2476=head2 Combining RE Pieces 2477 2478Each of the elementary pieces of regular expressions which were described 2479before (such as C<ab> or C<\Z>) could match at most one substring 2480at the given position of the input string. However, in a typical regular 2481expression these elementary pieces are combined into more complicated 2482patterns using combining operators C<ST>, C<S|T>, C<S*> etc. 2483(in these examples C<S> and C<T> are regular subexpressions). 2484 2485Such combinations can include alternatives, leading to a problem of choice: 2486if we match a regular expression C<a|ab> against C<"abc">, will it match 2487substring C<"a"> or C<"ab">? One way to describe which substring is 2488actually matched is the concept of backtracking (see L<"Backtracking">). 2489However, this description is too low-level and makes you think 2490in terms of a particular implementation. 2491 2492Another description starts with notions of "better"/"worse". All the 2493substrings which may be matched by the given regular expression can be 2494sorted from the "best" match to the "worst" match, and it is the "best" 2495match which is chosen. This substitutes the question of "what is chosen?" 2496by the question of "which matches are better, and which are worse?". 2497 2498Again, for elementary pieces there is no such question, since at most 2499one match at a given position is possible. This section describes the 2500notion of better/worse for combining operators. In the description 2501below C<S> and C<T> are regular subexpressions. 2502 2503=over 4 2504 2505=item C<ST> 2506 2507Consider two possible matches, C<AB> and C<A'B'>, C<A> and C<A'> are 2508substrings which can be matched by C<S>, C<B> and C<B'> are substrings 2509which can be matched by C<T>. 2510 2511If C<A> is a better match for C<S> than C<A'>, C<AB> is a better 2512match than C<A'B'>. 2513 2514If C<A> and C<A'> coincide: C<AB> is a better match than C<AB'> if 2515C<B> is a better match for C<T> than C<B'>. 2516 2517=item C<S|T> 2518 2519When C<S> can match, it is a better match than when only C<T> can match. 2520 2521Ordering of two matches for C<S> is the same as for C<S>. Similar for 2522two matches for C<T>. 2523 2524=item C<S{REPEAT_COUNT}> 2525 2526Matches as C<SSS...S> (repeated as many times as necessary). 2527 2528=item C<S{min,max}> 2529 2530Matches as C<S{max}|S{max-1}|...|S{min+1}|S{min}>. 2531 2532=item C<S{min,max}?> 2533 2534Matches as C<S{min}|S{min+1}|...|S{max-1}|S{max}>. 2535 2536=item C<S?>, C<S*>, C<S+> 2537 2538Same as C<S{0,1}>, C<S{0,BIG_NUMBER}>, C<S{1,BIG_NUMBER}> respectively. 2539 2540=item C<S??>, C<S*?>, C<S+?> 2541 2542Same as C<S{0,1}?>, C<S{0,BIG_NUMBER}?>, C<S{1,BIG_NUMBER}?> respectively. 2543 2544=item C<< (?>S) >> 2545 2546Matches the best match for C<S> and only that. 2547 2548=item C<(?=S)>, C<(?<=S)> 2549 2550Only the best match for C<S> is considered. (This is important only if 2551C<S> has capturing parentheses, and backreferences are used somewhere 2552else in the whole regular expression.) 2553 2554=item C<(?!S)>, C<(?<!S)> 2555 2556For this grouping operator there is no need to describe the ordering, since 2557only whether or not C<S> can match is important. 2558 2559=item C<(??{ EXPR })>, C<(?I<PARNO>)> 2560 2561The ordering is the same as for the regular expression which is 2562the result of EXPR, or the pattern contained by capture group I<PARNO>. 2563 2564=item C<(?(condition)yes-pattern|no-pattern)> 2565 2566Recall that which of C<yes-pattern> or C<no-pattern> actually matches is 2567already determined. The ordering of the matches is the same as for the 2568chosen subexpression. 2569 2570=back 2571 2572The above recipes describe the ordering of matches I<at a given position>. 2573One more rule is needed to understand how a match is determined for the 2574whole regular expression: a match at an earlier position is always better 2575than a match at a later position. 2576 2577=head2 Creating Custom RE Engines 2578 2579As of Perl 5.10.0, one can create custom regular expression engines. This 2580is not for the faint of heart, as they have to plug in at the C level. See 2581L<perlreapi> for more details. 2582 2583As an alternative, overloaded constants (see L<overload>) provide a simple 2584way to extend the functionality of the RE engine, by substituting one 2585pattern for another. 2586 2587Suppose that we want to enable a new RE escape-sequence C<\Y|> which 2588matches at a boundary between whitespace characters and non-whitespace 2589characters. Note that C<(?=\S)(?<!\S)|(?!\S)(?<=\S)> matches exactly 2590at these positions, so we want to have each C<\Y|> in the place of the 2591more complicated version. We can create a module C<customre> to do 2592this: 2593 2594 package customre; 2595 use overload; 2596 2597 sub import { 2598 shift; 2599 die "No argument to customre::import allowed" if @_; 2600 overload::constant 'qr' => \&convert; 2601 } 2602 2603 sub invalid { die "/$_[0]/: invalid escape '\\$_[1]'"} 2604 2605 # We must also take care of not escaping the legitimate \\Y| 2606 # sequence, hence the presence of '\\' in the conversion rules. 2607 my %rules = ( '\\' => '\\\\', 2608 'Y|' => qr/(?=\S)(?<!\S)|(?!\S)(?<=\S)/ ); 2609 sub convert { 2610 my $re = shift; 2611 $re =~ s{ 2612 \\ ( \\ | Y . ) 2613 } 2614 { $rules{$1} or invalid($re,$1) }sgex; 2615 return $re; 2616 } 2617 2618Now C<use customre> enables the new escape in constant regular 2619expressions, i.e., those without any runtime variable interpolations. 2620As documented in L<overload>, this conversion will work only over 2621literal parts of regular expressions. For C<\Y|$re\Y|> the variable 2622part of this regular expression needs to be converted explicitly 2623(but only if the special meaning of C<\Y|> should be enabled inside $re): 2624 2625 use customre; 2626 $re = <>; 2627 chomp $re; 2628 $re = customre::convert $re; 2629 /\Y|$re\Y|/; 2630 2631=head2 Embedded Code Execution Frequency 2632 2633The exact rules for how often (??{}) and (?{}) are executed in a pattern 2634are unspecified. In the case of a successful match you can assume that 2635they DWIM and will be executed in left to right order the appropriate 2636number of times in the accepting path of the pattern as would any other 2637meta-pattern. How non-accepting pathways and match failures affect the 2638number of times a pattern is executed is specifically unspecified and 2639may vary depending on what optimizations can be applied to the pattern 2640and is likely to change from version to version. 2641 2642For instance in 2643 2644 "aaabcdeeeee"=~/a(?{print "a"})b(?{print "b"})cde/; 2645 2646the exact number of times "a" or "b" are printed out is unspecified for 2647failure, but you may assume they will be printed at least once during 2648a successful match, additionally you may assume that if "b" is printed, 2649it will be preceded by at least one "a". 2650 2651In the case of branching constructs like the following: 2652 2653 /a(b|(?{ print "a" }))c(?{ print "c" })/; 2654 2655you can assume that the input "ac" will output "ac", and that "abc" 2656will output only "c". 2657 2658When embedded code is quantified, successful matches will call the 2659code once for each matched iteration of the quantifier. For 2660example: 2661 2662 "good" =~ /g(?:o(?{print "o"}))*d/; 2663 2664will output "o" twice. 2665 2666=head2 PCRE/Python Support 2667 2668As of Perl 5.10.0, Perl supports several Python/PCRE-specific extensions 2669to the regex syntax. While Perl programmers are encouraged to use the 2670Perl-specific syntax, the following are also accepted: 2671 2672=over 4 2673 2674=item C<< (?PE<lt>NAMEE<gt>pattern) >> 2675 2676Define a named capture group. Equivalent to C<< (?<NAME>pattern) >>. 2677 2678=item C<< (?P=NAME) >> 2679 2680Backreference to a named capture group. Equivalent to C<< \g{NAME} >>. 2681 2682=item C<< (?P>NAME) >> 2683 2684Subroutine call to a named capture group. Equivalent to C<< (?&NAME) >>. 2685 2686=back 2687 2688=head1 BUGS 2689 2690Many regular expression constructs don't work on EBCDIC platforms. 2691 2692There are a number of issues with regard to case-insensitive matching 2693in Unicode rules. See C<i> under L</Modifiers> above. 2694 2695This document varies from difficult to understand to completely 2696and utterly opaque. The wandering prose riddled with jargon is 2697hard to fathom in several places. 2698 2699This document needs a rewrite that separates the tutorial content 2700from the reference content. 2701 2702=head1 SEE ALSO 2703 2704L<perlrequick>. 2705 2706L<perlretut>. 2707 2708L<perlop/"Regexp Quote-Like Operators">. 2709 2710L<perlop/"Gory details of parsing quoted constructs">. 2711 2712L<perlfaq6>. 2713 2714L<perlfunc/pos>. 2715 2716L<perllocale>. 2717 2718L<perlebcdic>. 2719 2720I<Mastering Regular Expressions> by Jeffrey Friedl, published 2721by O'Reilly and Associates. 2722