1=head1 NAME 2 3perlreapi - Perl regular expression plugin interface 4 5=head1 DESCRIPTION 6 7As of Perl 5.9.5 there is a new interface for plugging and using 8regular expression engines other than the default one. 9 10Each engine is supposed to provide access to a constant structure of the 11following format: 12 13 typedef struct regexp_engine { 14 REGEXP* (*comp) (pTHX_ 15 const SV * const pattern, const U32 flags); 16 I32 (*exec) (pTHX_ 17 REGEXP * const rx, 18 char* stringarg, 19 char* strend, char* strbeg, 20 I32 minend, SV* screamer, 21 void* data, U32 flags); 22 char* (*intuit) (pTHX_ 23 REGEXP * const rx, SV *sv, 24 char *strpos, char *strend, U32 flags, 25 struct re_scream_pos_data_s *data); 26 SV* (*checkstr) (pTHX_ REGEXP * const rx); 27 void (*free) (pTHX_ REGEXP * const rx); 28 void (*numbered_buff_FETCH) (pTHX_ 29 REGEXP * const rx, 30 const I32 paren, 31 SV * const sv); 32 void (*numbered_buff_STORE) (pTHX_ 33 REGEXP * const rx, 34 const I32 paren, 35 SV const * const value); 36 I32 (*numbered_buff_LENGTH) (pTHX_ 37 REGEXP * const rx, 38 const SV * const sv, 39 const I32 paren); 40 SV* (*named_buff) (pTHX_ 41 REGEXP * const rx, 42 SV * const key, 43 SV * const value, 44 U32 flags); 45 SV* (*named_buff_iter) (pTHX_ 46 REGEXP * const rx, 47 const SV * const lastkey, 48 const U32 flags); 49 SV* (*qr_package)(pTHX_ REGEXP * const rx); 50 #ifdef USE_ITHREADS 51 void* (*dupe) (pTHX_ REGEXP * const rx, CLONE_PARAMS *param); 52 #endif 53 REGEXP* (*op_comp) (...); 54 55 56When a regexp is compiled, its C<engine> field is then set to point at 57the appropriate structure, so that when it needs to be used Perl can find 58the right routines to do so. 59 60In order to install a new regexp handler, C<$^H{regcomp}> is set 61to an integer which (when casted appropriately) resolves to one of these 62structures. When compiling, the C<comp> method is executed, and the 63resulting C<regexp> structure's engine field is expected to point back at 64the same structure. 65 66The pTHX_ symbol in the definition is a macro used by Perl under threading 67to provide an extra argument to the routine holding a pointer back to 68the interpreter that is executing the regexp. So under threading all 69routines get an extra argument. 70 71=head1 Callbacks 72 73=head2 comp 74 75 REGEXP* comp(pTHX_ const SV * const pattern, const U32 flags); 76 77Compile the pattern stored in C<pattern> using the given C<flags> and 78return a pointer to a prepared C<REGEXP> structure that can perform 79the match. See L</The REGEXP structure> below for an explanation of 80the individual fields in the REGEXP struct. 81 82The C<pattern> parameter is the scalar that was used as the 83pattern. Previous versions of Perl would pass two C<char*> indicating 84the start and end of the stringified pattern; the following snippet can 85be used to get the old parameters: 86 87 STRLEN plen; 88 char* exp = SvPV(pattern, plen); 89 char* xend = exp + plen; 90 91Since any scalar can be passed as a pattern, it's possible to implement 92an engine that does something with an array (C<< "ook" =~ [ qw/ eek 93hlagh / ] >>) or with the non-stringified form of a compiled regular 94expression (C<< "ook" =~ qr/eek/ >>). Perl's own engine will always 95stringify everything using the snippet above, but that doesn't mean 96other engines have to. 97 98The C<flags> parameter is a bitfield which indicates which of the 99C<msixp> flags the regex was compiled with. It also contains 100additional info, such as if C<use locale> is in effect. 101 102The C<eogc> flags are stripped out before being passed to the comp 103routine. The regex engine does not need to know if any of these 104are set, as those flags should only affect what Perl does with the 105pattern and its match variables, not how it gets compiled and 106executed. 107 108By the time the comp callback is called, some of these flags have 109already had effect (noted below where applicable). However most of 110their effect occurs after the comp callback has run, in routines that 111read the C<< rx->extflags >> field which it populates. 112 113In general the flags should be preserved in C<< rx->extflags >> after 114compilation, although the regex engine might want to add or delete 115some of them to invoke or disable some special behavior in Perl. The 116flags along with any special behavior they cause are documented below: 117 118The pattern modifiers: 119 120=over 4 121 122=item C</m> - RXf_PMf_MULTILINE 123 124If this is in C<< rx->extflags >> it will be passed to 125C<Perl_fbm_instr> by C<pp_split> which will treat the subject string 126as a multi-line string. 127 128=item C</s> - RXf_PMf_SINGLELINE 129 130=item C</i> - RXf_PMf_FOLD 131 132=item C</x> - RXf_PMf_EXTENDED 133 134If present on a regex, C<"#"> comments will be handled differently by the 135tokenizer in some cases. 136 137TODO: Document those cases. 138 139=item C</p> - RXf_PMf_KEEPCOPY 140 141TODO: Document this 142 143=item Character set 144 145The character set semantics are determined by an enum that is contained 146in this field. This is still experimental and subject to change, but 147the current interface returns the rules by use of the in-line function 148C<get_regex_charset(const U32 flags)>. The only currently documented 149value returned from it is REGEX_LOCALE_CHARSET, which is set if 150C<use locale> is in effect. If present in C<< rx->extflags >>, 151C<split> will use the locale dependent definition of whitespace 152when RXf_SKIPWHITE or RXf_WHITE is in effect. ASCII whitespace 153is defined as per L<isSPACE|perlapi/isSPACE>, and by the internal 154macros C<is_utf8_space> under UTF-8, and C<isSPACE_LC> under C<use 155locale>. 156 157=back 158 159Additional flags: 160 161=over 4 162 163=item RXf_SPLIT 164 165This flag was removed in perl 5.18.0. C<split ' '> is now special-cased 166solely in the parser. RXf_SPLIT is still #defined, so you can test for it. 167This is how it used to work: 168 169If C<split> is invoked as C<split ' '> or with no arguments (which 170really means C<split(' ', $_)>, see L<split|perlfunc/split>), Perl will 171set this flag. The regex engine can then check for it and set the 172SKIPWHITE and WHITE extflags. To do this, the Perl engine does: 173 174 if (flags & RXf_SPLIT && r->prelen == 1 && r->precomp[0] == ' ') 175 r->extflags |= (RXf_SKIPWHITE|RXf_WHITE); 176 177=back 178 179These flags can be set during compilation to enable optimizations in 180the C<split> operator. 181 182=over 4 183 184=item RXf_SKIPWHITE 185 186This flag was removed in perl 5.18.0. It is still #defined, so you can 187set it, but doing so will have no effect. This is how it used to work: 188 189If the flag is present in C<< rx->extflags >> C<split> will delete 190whitespace from the start of the subject string before it's operated 191on. What is considered whitespace depends on if the subject is a 192UTF-8 string and if the C<RXf_PMf_LOCALE> flag is set. 193 194If RXf_WHITE is set in addition to this flag, C<split> will behave like 195C<split " "> under the Perl engine. 196 197=item RXf_START_ONLY 198 199Tells the split operator to split the target string on newlines 200(C<\n>) without invoking the regex engine. 201 202Perl's engine sets this if the pattern is C</^/> (C<plen == 1 && *exp 203== '^'>), even under C</^/s>; see L<split|perlfunc>. Of course a 204different regex engine might want to use the same optimizations 205with a different syntax. 206 207=item RXf_WHITE 208 209Tells the split operator to split the target string on whitespace 210without invoking the regex engine. The definition of whitespace varies 211depending on if the target string is a UTF-8 string and on 212if RXf_PMf_LOCALE is set. 213 214Perl's engine sets this flag if the pattern is C<\s+>. 215 216=item RXf_NULL 217 218Tells the split operator to split the target string on 219characters. The definition of character varies depending on if 220the target string is a UTF-8 string. 221 222Perl's engine sets this flag on empty patterns, this optimization 223makes C<split //> much faster than it would otherwise be. It's even 224faster than C<unpack>. 225 226=item RXf_NO_INPLACE_SUBST 227 228Added in perl 5.18.0, this flag indicates that a regular expression might 229perform an operation that would interfere with inplace substituion. For 230instance it might contain lookbehind, or assign to non-magical variables 231(such as $REGMARK and $REGERROR) during matching. C<s///> will skip 232certain optimisations when this is set. 233 234=back 235 236=head2 exec 237 238 I32 exec(pTHX_ REGEXP * const rx, 239 char *stringarg, char* strend, char* strbeg, 240 I32 minend, SV* screamer, 241 void* data, U32 flags); 242 243Execute a regexp. The arguments are 244 245=over 4 246 247=item rx 248 249The regular expression to execute. 250 251=item screamer 252 253This strangely-named arg is the SV to be matched against. Note that the 254actual char array to be matched against is supplied by the arguments 255described below; the SV is just used to determine UTF8ness, C<pos()> etc. 256 257=item strbeg 258 259Pointer to the physical start of the string. 260 261=item strend 262 263Pointer to the character following the physical end of the string (i.e. 264the C<\0>). 265 266=item stringarg 267 268Pointer to the position in the string where matching should start; it might 269not be equal to C<strbeg> (for example in a later iteration of C</.../g>). 270 271=item minend 272 273Minimum length of string (measured in bytes from C<stringarg>) that must 274match; if the engine reaches the end of the match but hasn't reached this 275position in the string, it should fail. 276 277=item data 278 279Optimisation data; subject to change. 280 281=item flags 282 283Optimisation flags; subject to change. 284 285=back 286 287=head2 intuit 288 289 char* intuit(pTHX_ REGEXP * const rx, 290 SV *sv, char *strpos, char *strend, 291 const U32 flags, struct re_scream_pos_data_s *data); 292 293Find the start position where a regex match should be attempted, 294or possibly if the regex engine should not be run because the 295pattern can't match. This is called, as appropriate, by the core, 296depending on the values of the C<extflags> member of the C<regexp> 297structure. 298 299=head2 checkstr 300 301 SV* checkstr(pTHX_ REGEXP * const rx); 302 303Return a SV containing a string that must appear in the pattern. Used 304by C<split> for optimising matches. 305 306=head2 free 307 308 void free(pTHX_ REGEXP * const rx); 309 310Called by Perl when it is freeing a regexp pattern so that the engine 311can release any resources pointed to by the C<pprivate> member of the 312C<regexp> structure. This is only responsible for freeing private data; 313Perl will handle releasing anything else contained in the C<regexp> structure. 314 315=head2 Numbered capture callbacks 316 317Called to get/set the value of C<$`>, C<$'>, C<$&> and their named 318equivalents, ${^PREMATCH}, ${^POSTMATCH} and $^{MATCH}, as well as the 319numbered capture groups (C<$1>, C<$2>, ...). 320 321The C<paren> parameter will be C<1> for C<$1>, C<2> for C<$2> and so 322forth, and have these symbolic values for the special variables: 323 324 ${^PREMATCH} RX_BUFF_IDX_CARET_PREMATCH 325 ${^POSTMATCH} RX_BUFF_IDX_CARET_POSTMATCH 326 ${^MATCH} RX_BUFF_IDX_CARET_FULLMATCH 327 $` RX_BUFF_IDX_PREMATCH 328 $' RX_BUFF_IDX_POSTMATCH 329 $& RX_BUFF_IDX_FULLMATCH 330 331Note that in Perl 5.17.3 and earlier, the last three constants were also 332used for the caret variants of the variables. 333 334 335The names have been chosen by analogy with L<Tie::Scalar> methods 336names with an additional B<LENGTH> callback for efficiency. However 337named capture variables are currently not tied internally but 338implemented via magic. 339 340=head3 numbered_buff_FETCH 341 342 void numbered_buff_FETCH(pTHX_ REGEXP * const rx, const I32 paren, 343 SV * const sv); 344 345Fetch a specified numbered capture. C<sv> should be set to the scalar 346to return, the scalar is passed as an argument rather than being 347returned from the function because when it's called Perl already has a 348scalar to store the value, creating another one would be 349redundant. The scalar can be set with C<sv_setsv>, C<sv_setpvn> and 350friends, see L<perlapi>. 351 352This callback is where Perl untaints its own capture variables under 353taint mode (see L<perlsec>). See the C<Perl_reg_numbered_buff_fetch> 354function in F<regcomp.c> for how to untaint capture variables if 355that's something you'd like your engine to do as well. 356 357=head3 numbered_buff_STORE 358 359 void (*numbered_buff_STORE) (pTHX_ 360 REGEXP * const rx, 361 const I32 paren, 362 SV const * const value); 363 364Set the value of a numbered capture variable. C<value> is the scalar 365that is to be used as the new value. It's up to the engine to make 366sure this is used as the new value (or reject it). 367 368Example: 369 370 if ("ook" =~ /(o*)/) { 371 # 'paren' will be '1' and 'value' will be 'ee' 372 $1 =~ tr/o/e/; 373 } 374 375Perl's own engine will croak on any attempt to modify the capture 376variables, to do this in another engine use the following callback 377(copied from C<Perl_reg_numbered_buff_store>): 378 379 void 380 Example_reg_numbered_buff_store(pTHX_ 381 REGEXP * const rx, 382 const I32 paren, 383 SV const * const value) 384 { 385 PERL_UNUSED_ARG(rx); 386 PERL_UNUSED_ARG(paren); 387 PERL_UNUSED_ARG(value); 388 389 if (!PL_localizing) 390 Perl_croak(aTHX_ PL_no_modify); 391 } 392 393Actually Perl will not I<always> croak in a statement that looks 394like it would modify a numbered capture variable. This is because the 395STORE callback will not be called if Perl can determine that it 396doesn't have to modify the value. This is exactly how tied variables 397behave in the same situation: 398 399 package CaptureVar; 400 use base 'Tie::Scalar'; 401 402 sub TIESCALAR { bless [] } 403 sub FETCH { undef } 404 sub STORE { die "This doesn't get called" } 405 406 package main; 407 408 tie my $sv => "CaptureVar"; 409 $sv =~ y/a/b/; 410 411Because C<$sv> is C<undef> when the C<y///> operator is applied to it, 412the transliteration won't actually execute and the program won't 413C<die>. This is different to how 5.8 and earlier versions behaved 414since the capture variables were READONLY variables then; now they'll 415just die when assigned to in the default engine. 416 417=head3 numbered_buff_LENGTH 418 419 I32 numbered_buff_LENGTH (pTHX_ 420 REGEXP * const rx, 421 const SV * const sv, 422 const I32 paren); 423 424Get the C<length> of a capture variable. There's a special callback 425for this so that Perl doesn't have to do a FETCH and run C<length> on 426the result, since the length is (in Perl's case) known from an offset 427stored in C<< rx->offs >>, this is much more efficient: 428 429 I32 s1 = rx->offs[paren].start; 430 I32 s2 = rx->offs[paren].end; 431 I32 len = t1 - s1; 432 433This is a little bit more complex in the case of UTF-8, see what 434C<Perl_reg_numbered_buff_length> does with 435L<is_utf8_string_loclen|perlapi/is_utf8_string_loclen>. 436 437=head2 Named capture callbacks 438 439Called to get/set the value of C<%+> and C<%->, as well as by some 440utility functions in L<re>. 441 442There are two callbacks, C<named_buff> is called in all the cases the 443FETCH, STORE, DELETE, CLEAR, EXISTS and SCALAR L<Tie::Hash> callbacks 444would be on changes to C<%+> and C<%-> and C<named_buff_iter> in the 445same cases as FIRSTKEY and NEXTKEY. 446 447The C<flags> parameter can be used to determine which of these 448operations the callbacks should respond to. The following flags are 449currently defined: 450 451Which L<Tie::Hash> operation is being performed from the Perl level on 452C<%+> or C<%+>, if any: 453 454 RXapif_FETCH 455 RXapif_STORE 456 RXapif_DELETE 457 RXapif_CLEAR 458 RXapif_EXISTS 459 RXapif_SCALAR 460 RXapif_FIRSTKEY 461 RXapif_NEXTKEY 462 463If C<%+> or C<%-> is being operated on, if any. 464 465 RXapif_ONE /* %+ */ 466 RXapif_ALL /* %- */ 467 468If this is being called as C<re::regname>, C<re::regnames> or 469C<re::regnames_count>, if any. The first two will be combined with 470C<RXapif_ONE> or C<RXapif_ALL>. 471 472 RXapif_REGNAME 473 RXapif_REGNAMES 474 RXapif_REGNAMES_COUNT 475 476Internally C<%+> and C<%-> are implemented with a real tied interface 477via L<Tie::Hash::NamedCapture>. The methods in that package will call 478back into these functions. However the usage of 479L<Tie::Hash::NamedCapture> for this purpose might change in future 480releases. For instance this might be implemented by magic instead 481(would need an extension to mgvtbl). 482 483=head3 named_buff 484 485 SV* (*named_buff) (pTHX_ REGEXP * const rx, SV * const key, 486 SV * const value, U32 flags); 487 488=head3 named_buff_iter 489 490 SV* (*named_buff_iter) (pTHX_ 491 REGEXP * const rx, 492 const SV * const lastkey, 493 const U32 flags); 494 495=head2 qr_package 496 497 SV* qr_package(pTHX_ REGEXP * const rx); 498 499The package the qr// magic object is blessed into (as seen by C<ref 500qr//>). It is recommended that engines change this to their package 501name for identification regardless of if they implement methods 502on the object. 503 504The package this method returns should also have the internal 505C<Regexp> package in its C<@ISA>. C<< qr//->isa("Regexp") >> should always 506be true regardless of what engine is being used. 507 508Example implementation might be: 509 510 SV* 511 Example_qr_package(pTHX_ REGEXP * const rx) 512 { 513 PERL_UNUSED_ARG(rx); 514 return newSVpvs("re::engine::Example"); 515 } 516 517Any method calls on an object created with C<qr//> will be dispatched to the 518package as a normal object. 519 520 use re::engine::Example; 521 my $re = qr//; 522 $re->meth; # dispatched to re::engine::Example::meth() 523 524To retrieve the C<REGEXP> object from the scalar in an XS function use 525the C<SvRX> macro, see L<"REGEXP Functions" in perlapi|perlapi/REGEXP 526Functions>. 527 528 void meth(SV * rv) 529 PPCODE: 530 REGEXP * re = SvRX(sv); 531 532=head2 dupe 533 534 void* dupe(pTHX_ REGEXP * const rx, CLONE_PARAMS *param); 535 536On threaded builds a regexp may need to be duplicated so that the pattern 537can be used by multiple threads. This routine is expected to handle the 538duplication of any private data pointed to by the C<pprivate> member of 539the C<regexp> structure. It will be called with the preconstructed new 540C<regexp> structure as an argument, the C<pprivate> member will point at 541the B<old> private structure, and it is this routine's responsibility to 542construct a copy and return a pointer to it (which Perl will then use to 543overwrite the field as passed to this routine.) 544 545This allows the engine to dupe its private data but also if necessary 546modify the final structure if it really must. 547 548On unthreaded builds this field doesn't exist. 549 550=head2 op_comp 551 552This is private to the Perl core and subject to change. Should be left 553null. 554 555=head1 The REGEXP structure 556 557The REGEXP struct is defined in F<regexp.h>. 558All regex engines must be able to 559correctly build such a structure in their L</comp> routine. 560 561The REGEXP structure contains all the data that Perl needs to be aware of 562to properly work with the regular expression. It includes data about 563optimisations that Perl can use to determine if the regex engine should 564really be used, and various other control info that is needed to properly 565execute patterns in various contexts, such as if the pattern anchored in 566some way, or what flags were used during the compile, or if the 567program contains special constructs that Perl needs to be aware of. 568 569In addition it contains two fields that are intended for the private 570use of the regex engine that compiled the pattern. These are the 571C<intflags> and C<pprivate> members. C<pprivate> is a void pointer to 572an arbitrary structure, whose use and management is the responsibility 573of the compiling engine. Perl will never modify either of these 574values. 575 576 typedef struct regexp { 577 /* what engine created this regexp? */ 578 const struct regexp_engine* engine; 579 580 /* what re is this a lightweight copy of? */ 581 struct regexp* mother_re; 582 583 /* Information about the match that the Perl core uses to manage 584 * things */ 585 U32 extflags; /* Flags used both externally and internally */ 586 I32 minlen; /* mininum possible number of chars in */ 587 string to match */ 588 I32 minlenret; /* mininum possible number of chars in $& */ 589 U32 gofs; /* chars left of pos that we search from */ 590 591 /* substring data about strings that must appear 592 in the final match, used for optimisations */ 593 struct reg_substr_data *substrs; 594 595 U32 nparens; /* number of capture groups */ 596 597 /* private engine specific data */ 598 U32 intflags; /* Engine Specific Internal flags */ 599 void *pprivate; /* Data private to the regex engine which 600 created this object. */ 601 602 /* Data about the last/current match. These are modified during 603 * matching*/ 604 U32 lastparen; /* highest close paren matched ($+) */ 605 U32 lastcloseparen; /* last close paren matched ($^N) */ 606 regexp_paren_pair *swap; /* Swap copy of *offs */ 607 regexp_paren_pair *offs; /* Array of offsets for (@-) and 608 (@+) */ 609 610 char *subbeg; /* saved or original string so \digit works 611 forever. */ 612 SV_SAVED_COPY /* If non-NULL, SV which is COW from original */ 613 I32 sublen; /* Length of string pointed by subbeg */ 614 I32 suboffset; /* byte offset of subbeg from logical start of 615 str */ 616 I32 subcoffset; /* suboffset equiv, but in chars (for @-/@+) */ 617 618 /* Information about the match that isn't often used */ 619 I32 prelen; /* length of precomp */ 620 const char *precomp; /* pre-compilation regular expression */ 621 622 char *wrapped; /* wrapped version of the pattern */ 623 I32 wraplen; /* length of wrapped */ 624 625 I32 seen_evals; /* number of eval groups in the pattern - for 626 security checks */ 627 HV *paren_names; /* Optional hash of paren names */ 628 629 /* Refcount of this regexp */ 630 I32 refcnt; /* Refcount of this regexp */ 631 } regexp; 632 633The fields are discussed in more detail below: 634 635=head2 C<engine> 636 637This field points at a C<regexp_engine> structure which contains pointers 638to the subroutines that are to be used for performing a match. It 639is the compiling routine's responsibility to populate this field before 640returning the regexp object. 641 642Internally this is set to C<NULL> unless a custom engine is specified in 643C<$^H{regcomp}>, Perl's own set of callbacks can be accessed in the struct 644pointed to by C<RE_ENGINE_PTR>. 645 646=head2 C<mother_re> 647 648TODO, see L<http://www.mail-archive.com/perl5-changes@perl.org/msg17328.html> 649 650=head2 C<extflags> 651 652This will be used by Perl to see what flags the regexp was compiled 653with, this will normally be set to the value of the flags parameter by 654the L<comp|/comp> callback. See the L<comp|/comp> documentation for 655valid flags. 656 657=head2 C<minlen> C<minlenret> 658 659The minimum string length (in characters) required for the pattern to match. 660This is used to 661prune the search space by not bothering to match any closer to the end of a 662string than would allow a match. For instance there is no point in even 663starting the regex engine if the minlen is 10 but the string is only 5 664characters long. There is no way that the pattern can match. 665 666C<minlenret> is the minimum length (in characters) of the string that would 667be found in $& after a match. 668 669The difference between C<minlen> and C<minlenret> can be seen in the 670following pattern: 671 672 /ns(?=\d)/ 673 674where the C<minlen> would be 3 but C<minlenret> would only be 2 as the \d is 675required to match but is not actually 676included in the matched content. This 677distinction is particularly important as the substitution logic uses the 678C<minlenret> to tell if it can do in-place substitutions (these can 679result in considerable speed-up). 680 681=head2 C<gofs> 682 683Left offset from pos() to start match at. 684 685=head2 C<substrs> 686 687Substring data about strings that must appear in the final match. This 688is currently only used internally by Perl's engine, but might be 689used in the future for all engines for optimisations. 690 691=head2 C<nparens>, C<lastparen>, and C<lastcloseparen> 692 693These fields are used to keep track of how many paren groups could be matched 694in the pattern, which was the last open paren to be entered, and which was 695the last close paren to be entered. 696 697=head2 C<intflags> 698 699The engine's private copy of the flags the pattern was compiled with. Usually 700this is the same as C<extflags> unless the engine chose to modify one of them. 701 702=head2 C<pprivate> 703 704A void* pointing to an engine-defined 705data structure. The Perl engine uses the 706C<regexp_internal> structure (see L<perlreguts/Base Structures>) but a custom 707engine should use something else. 708 709=head2 C<swap> 710 711Unused. Left in for compatibility with Perl 5.10.0. 712 713=head2 C<offs> 714 715A C<regexp_paren_pair> structure which defines offsets into the string being 716matched which correspond to the C<$&> and C<$1>, C<$2> etc. captures, the 717C<regexp_paren_pair> struct is defined as follows: 718 719 typedef struct regexp_paren_pair { 720 I32 start; 721 I32 end; 722 } regexp_paren_pair; 723 724If C<< ->offs[num].start >> or C<< ->offs[num].end >> is C<-1> then that 725capture group did not match. 726C<< ->offs[0].start/end >> represents C<$&> (or 727C<${^MATCH}> under C<//p>) and C<< ->offs[paren].end >> matches C<$$paren> where 728C<$paren >= 1>. 729 730=head2 C<precomp> C<prelen> 731 732Used for optimisations. C<precomp> holds a copy of the pattern that 733was compiled and C<prelen> its length. When a new pattern is to be 734compiled (such as inside a loop) the internal C<regcomp> operator 735checks if the last compiled C<REGEXP>'s C<precomp> and C<prelen> 736are equivalent to the new one, and if so uses the old pattern instead 737of compiling a new one. 738 739The relevant snippet from C<Perl_pp_regcomp>: 740 741 if (!re || !re->precomp || re->prelen != (I32)len || 742 memNE(re->precomp, t, len)) 743 /* Compile a new pattern */ 744 745=head2 C<paren_names> 746 747This is a hash used internally to track named capture groups and their 748offsets. The keys are the names of the buffers the values are dualvars, 749with the IV slot holding the number of buffers with the given name and the 750pv being an embedded array of I32. The values may also be contained 751independently in the data array in cases where named backreferences are 752used. 753 754=head2 C<substrs> 755 756Holds information on the longest string that must occur at a fixed 757offset from the start of the pattern, and the longest string that must 758occur at a floating offset from the start of the pattern. Used to do 759Fast-Boyer-Moore searches on the string to find out if its worth using 760the regex engine at all, and if so where in the string to search. 761 762=head2 C<subbeg> C<sublen> C<saved_copy> C<suboffset> C<subcoffset> 763 764Used during the execution phase for managing search and replace patterns, 765and for providing the text for C<$&>, C<$1> etc. C<subbeg> points to a 766buffer (either the original string, or a copy in the case of 767C<RX_MATCH_COPIED(rx)>), and C<sublen> is the length of the buffer. The 768C<RX_OFFS> start and end indices index into this buffer. 769 770In the presence of the C<REXEC_COPY_STR> flag, but with the addition of 771the C<REXEC_COPY_SKIP_PRE> or C<REXEC_COPY_SKIP_POST> flags, an engine 772can choose not to copy the full buffer (although it must still do so in 773the presence of C<RXf_PMf_KEEPCOPY> or the relevant bits being set in 774C<PL_sawampersand>). In this case, it may set C<suboffset> to indicate the 775number of bytes from the logical start of the buffer to the physical start 776(i.e. C<subbeg>). It should also set C<subcoffset>, the number of 777characters in the offset. The latter is needed to support C<@-> and C<@+> 778which work in characters, not bytes. 779 780=head2 C<wrapped> C<wraplen> 781 782Stores the string C<qr//> stringifies to. The Perl engine for example 783stores C<(?^:eek)> in the case of C<qr/eek/>. 784 785When using a custom engine that doesn't support the C<(?:)> construct 786for inline modifiers, it's probably best to have C<qr//> stringify to 787the supplied pattern, note that this will create undesired patterns in 788cases such as: 789 790 my $x = qr/a|b/; # "a|b" 791 my $y = qr/c/i; # "c" 792 my $z = qr/$x$y/; # "a|bc" 793 794There's no solution for this problem other than making the custom 795engine understand a construct like C<(?:)>. 796 797=head2 C<seen_evals> 798 799This stores the number of eval groups in 800the pattern. This is used for security 801purposes when embedding compiled regexes into larger patterns with C<qr//>. 802 803=head2 C<refcnt> 804 805The number of times the structure is referenced. When 806this falls to 0, the regexp is automatically freed 807by a call to pregfree. This should be set to 1 in 808each engine's L</comp> routine. 809 810=head1 HISTORY 811 812Originally part of L<perlreguts>. 813 814=head1 AUTHORS 815 816Originally written by Yves Orton, expanded by E<AElig>var ArnfjE<ouml>rE<eth> 817Bjarmason. 818 819=head1 LICENSE 820 821Copyright 2006 Yves Orton and 2007 E<AElig>var ArnfjE<ouml>rE<eth> Bjarmason. 822 823This program is free software; you can redistribute it and/or modify it under 824the same terms as Perl itself. 825 826=cut 827