This is a live mirror of the Perl 5 development currently hosted at https://github.com/perl/perl5
perlreapi: Update as little as possible for 5.14
[perl5.git] / pod / perlreapi.pod
CommitLineData
108003db
RGS
1=head1 NAME
2
3perlreapi - perl regular expression plugin interface
4
5=head1 DESCRIPTION
6
a0e97681
RGS
7As of Perl 5.9.5 there is a new interface for plugging and using other
8regular expression engines than the default one.
9
10Each engine is supposed to provide access to a constant structure of the
11following format:
108003db
RGS
12
13 typedef struct regexp_engine {
3ab4a224 14 REGEXP* (*comp) (pTHX_ const SV * const pattern, const U32 flags);
49d7dfbc 15 I32 (*exec) (pTHX_ REGEXP * const rx, char* stringarg, char* strend,
2fdbfb4d
AB
16 char* strbeg, I32 minend, SV* screamer,
17 void* data, U32 flags);
49d7dfbc 18 char* (*intuit) (pTHX_ REGEXP * const rx, SV *sv, char *strpos,
2fdbfb4d
AB
19 char *strend, U32 flags,
20 struct re_scream_pos_data_s *data);
49d7dfbc
AB
21 SV* (*checkstr) (pTHX_ REGEXP * const rx);
22 void (*free) (pTHX_ REGEXP * const rx);
2fdbfb4d
AB
23 void (*numbered_buff_FETCH) (pTHX_ REGEXP * const rx, const I32 paren,
24 SV * const sv);
25 void (*numbered_buff_STORE) (pTHX_ REGEXP * const rx, const I32 paren,
26 SV const * const value);
27 I32 (*numbered_buff_LENGTH) (pTHX_ REGEXP * const rx, const SV * const sv,
28 const I32 paren);
192b9cd1
AB
29 SV* (*named_buff) (pTHX_ REGEXP * const rx, SV * const key,
30 SV * const value, U32 flags);
31 SV* (*named_buff_iter) (pTHX_ REGEXP * const rx, const SV * const lastkey,
32 const U32 flags);
49d7dfbc 33 SV* (*qr_package)(pTHX_ REGEXP * const rx);
108003db 34 #ifdef USE_ITHREADS
49d7dfbc 35 void* (*dupe) (pTHX_ REGEXP * const rx, CLONE_PARAMS *param);
108003db 36 #endif
108003db
RGS
37
38When a regexp is compiled, its C<engine> field is then set to point at
a0e97681 39the appropriate structure, so that when it needs to be used Perl can find
108003db
RGS
40the right routines to do so.
41
42In order to install a new regexp handler, C<$^H{regcomp}> is set
43to an integer which (when casted appropriately) resolves to one of these
44structures. When compiling, the C<comp> method is executed, and the
45resulting regexp structure's engine field is expected to point back at
46the same structure.
47
48The pTHX_ symbol in the definition is a macro used by perl under threading
49to provide an extra argument to the routine holding a pointer back to
50the interpreter that is executing the regexp. So under threading all
51routines get an extra argument.
52
882227b7 53=head1 Callbacks
108003db
RGS
54
55=head2 comp
56
3ab4a224 57 REGEXP* comp(pTHX_ const SV * const pattern, const U32 flags);
108003db 58
3ab4a224
AB
59Compile the pattern stored in C<pattern> using the given C<flags> and
60return a pointer to a prepared C<REGEXP> structure that can perform
61the match. See L</The REGEXP structure> below for an explanation of
62the individual fields in the REGEXP struct.
63
64The C<pattern> parameter is the scalar that was used as the
65pattern. previous versions of perl would pass two C<char*> indicating
a0e97681 66the start and end of the stringified pattern, the following snippet can
3ab4a224
AB
67be used to get the old parameters:
68
69 STRLEN plen;
70 char* exp = SvPV(pattern, plen);
71 char* xend = exp + plen;
72
73Since any scalar can be passed as a pattern it's possible to implement
74an engine that does something with an array (C<< "ook" =~ [ qw/ eek
75hlagh / ] >>) or with the non-stringified form of a compiled regular
76expression (C<< "ook" =~ qr/eek/ >>). perl's own engine will always
77stringify everything using the snippet above but that doesn't mean
78other engines have to.
108003db 79
a0e97681 80The C<flags> parameter is a bitfield which indicates which of the
c998b245
AB
81C<msixp> flags the regex was compiled with. It also contains
82additional info such as whether C<use locale> is in effect.
108003db
RGS
83
84The C<eogc> flags are stripped out before being passed to the comp
85routine. The regex engine does not need to know whether any of these
3ab4a224 86are set as those flags should only affect what perl does with the
c998b245
AB
87pattern and its match variables, not how it gets compiled and
88executed.
108003db 89
c998b245
AB
90By the time the comp callback is called, some of these flags have
91already had effect (noted below where applicable). However most of
92their effect occurs after the comp callback has run in routines that
93read the C<< rx->extflags >> field which it populates.
108003db 94
c998b245
AB
95In general the flags should be preserved in C<< rx->extflags >> after
96compilation, although the regex engine might want to add or delete
97some of them to invoke or disable some special behavior in perl. The
98flags along with any special behavior they cause are documented below:
108003db 99
c998b245 100The pattern modifiers:
108003db 101
c998b245 102=over 4
108003db 103
c998b245 104=item C</m> - RXf_PMf_MULTILINE
108003db 105
c998b245
AB
106If this is in C<< rx->extflags >> it will be passed to
107C<Perl_fbm_instr> by C<pp_split> which will treat the subject string
108as a multi-line string.
108003db 109
c998b245 110=item C</s> - RXf_PMf_SINGLELINE
108003db 111
c998b245 112=item C</i> - RXf_PMf_FOLD
108003db 113
c998b245 114=item C</x> - RXf_PMf_EXTENDED
108003db 115
c998b245
AB
116If present on a regex C<#> comments will be handled differently by the
117tokenizer in some cases.
108003db 118
c998b245 119TODO: Document those cases.
108003db 120
c998b245 121=item C</p> - RXf_PMf_KEEPCOPY
108003db 122
f0f9b3b8
KW
123=item Character set
124
125The character set semantics are determined by an enum that is contained
126in this field. This is still experimental and subject to change, but
127the current interface returns the rules by use of the in-line function
128C<get_regex_charset(const U32 flags)>. The only currently documented
129value returned from it is REGEX_LOCALE_CHARSET, which is set if
130C<use locale> is in effect. If present in C<< rx->extflags >>
a0e97681 131C<split> will use the locale dependent definition of whitespace under
c998b245 132when RXf_SKIPWHITE or RXf_WHITE are in effect. Under ASCII whitespace
96090e4f 133is defined as per L<isSPACE|perlapi/isSPACE>, and by the internal
c998b245
AB
134macros C<is_utf8_space> under UTF-8 and C<isSPACE_LC> under C<use
135locale>.
108003db 136
f0f9b3b8
KW
137=back
138
139Additional flags:
140
141=over 4
142
108003db
RGS
143=item RXf_UTF8
144
145Set if the pattern is L<SvUTF8()|perlapi/SvUTF8>, set by Perl_pmruntime.
146
c998b245
AB
147A regex engine may want to set or disable this flag during
148compilation. The perl engine for instance may upgrade non-UTF-8
149strings to UTF-8 if the pattern includes constructs such as C<\x{...}>
150that can only match Unicode values.
151
0ac6acae
AB
152=item RXf_SPLIT
153
154If C<split> is invoked as C<split ' '> or with no arguments (which
5137fa37 155really means C<split(' ', $_)>, see L<split|perlfunc/split>), perl will
0ac6acae
AB
156set this flag. The regex engine can then check for it and set the
157SKIPWHITE and WHITE extflags. To do this the perl engine does:
158
159 if (flags & RXf_SPLIT && r->prelen == 1 && r->precomp[0] == ' ')
160 r->extflags |= (RXf_SKIPWHITE|RXf_WHITE);
161
108003db
RGS
162=back
163
c998b245
AB
164These flags can be set during compilation to enable optimizations in
165the C<split> operator.
166
167=over 4
168
0ac6acae
AB
169=item RXf_SKIPWHITE
170
171If the flag is present in C<< rx->extflags >> C<split> will delete
172whitespace from the start of the subject string before it's operated
173on. What is considered whitespace depends on whether the subject is a
174UTF-8 string and whether the C<RXf_PMf_LOCALE> flag is set.
175
176If RXf_WHITE is set in addition to this flag C<split> will behave like
177C<split " "> under the perl engine.
178
c998b245
AB
179=item RXf_START_ONLY
180
181Tells the split operator to split the target string on newlines
182(C<\n>) without invoking the regex engine.
183
184Perl's engine sets this if the pattern is C</^/> (C<plen == 1 && *exp
185== '^'>), even under C</^/s>, see L<split|perlfunc>. Of course a
186different regex engine might want to use the same optimizations
187with a different syntax.
188
189=item RXf_WHITE
190
191Tells the split operator to split the target string on whitespace
192without invoking the regex engine. The definition of whitespace varies
193depending on whether the target string is a UTF-8 string and on
194whether RXf_PMf_LOCALE is set.
195
0ac6acae 196Perl's engine sets this flag if the pattern is C<\s+>.
c998b245 197
640f820d
AB
198=item RXf_NULL
199
a0e97681 200Tells the split operator to split the target string on
640f820d
AB
201characters. The definition of character varies depending on whether
202the target string is a UTF-8 string.
203
204Perl's engine sets this flag on empty patterns, this optimization
a0e97681 205makes C<split //> much faster than it would otherwise be. It's even
640f820d
AB
206faster than C<unpack>.
207
c998b245 208=back
108003db
RGS
209
210=head2 exec
211
49d7dfbc 212 I32 exec(pTHX_ REGEXP * const rx,
108003db
RGS
213 char *stringarg, char* strend, char* strbeg,
214 I32 minend, SV* screamer,
215 void* data, U32 flags);
216
217Execute a regexp.
218
219=head2 intuit
220
49d7dfbc 221 char* intuit(pTHX_ REGEXP * const rx,
108003db 222 SV *sv, char *strpos, char *strend,
49d7dfbc 223 const U32 flags, struct re_scream_pos_data_s *data);
108003db
RGS
224
225Find the start position where a regex match should be attempted,
226or possibly whether the regex engine should not be run because the
227pattern can't match. This is called as appropriate by the core
228depending on the values of the extflags member of the regexp
229structure.
230
231=head2 checkstr
232
49d7dfbc 233 SV* checkstr(pTHX_ REGEXP * const rx);
108003db
RGS
234
235Return a SV containing a string that must appear in the pattern. Used
236by C<split> for optimising matches.
237
238=head2 free
239
49d7dfbc 240 void free(pTHX_ REGEXP * const rx);
108003db
RGS
241
242Called by perl when it is freeing a regexp pattern so that the engine
243can release any resources pointed to by the C<pprivate> member of the
244regexp structure. This is only responsible for freeing private data;
245perl will handle releasing anything else contained in the regexp structure.
246
192b9cd1 247=head2 Numbered capture callbacks
108003db 248
192b9cd1
AB
249Called to get/set the value of C<$`>, C<$'>, C<$&> and their named
250equivalents, ${^PREMATCH}, ${^POSTMATCH} and $^{MATCH}, as well as the
c27a5cfe 251numbered capture groups (C<$1>, C<$2>, ...).
49d7dfbc 252
a0e97681 253The C<paren> parameter will be C<-2> for C<$`>, C<-1> for C<$'>, C<0>
49d7dfbc
AB
254for C<$&>, C<1> for C<$1> and so forth.
255
192b9cd1
AB
256The names have been chosen by analogy with L<Tie::Scalar> methods
257names with an additional B<LENGTH> callback for efficiency. However
258named capture variables are currently not tied internally but
259implemented via magic.
260
261=head3 numbered_buff_FETCH
262
263 void numbered_buff_FETCH(pTHX_ REGEXP * const rx, const I32 paren,
264 SV * const sv);
265
266Fetch a specified numbered capture. C<sv> should be set to the scalar
267to return, the scalar is passed as an argument rather than being
268returned from the function because when it's called perl already has a
269scalar to store the value, creating another one would be
270redundant. The scalar can be set with C<sv_setsv>, C<sv_setpvn> and
271friends, see L<perlapi>.
49d7dfbc
AB
272
273This callback is where perl untaints its own capture variables under
c998b245 274taint mode (see L<perlsec>). See the C<Perl_reg_numbered_buff_fetch>
49d7dfbc
AB
275function in F<regcomp.c> for how to untaint capture variables if
276that's something you'd like your engine to do as well.
108003db 277
192b9cd1 278=head3 numbered_buff_STORE
108003db 279
2fdbfb4d
AB
280 void (*numbered_buff_STORE) (pTHX_ REGEXP * const rx, const I32 paren,
281 SV const * const value);
108003db 282
192b9cd1
AB
283Set the value of a numbered capture variable. C<value> is the scalar
284that is to be used as the new value. It's up to the engine to make
285sure this is used as the new value (or reject it).
2fdbfb4d
AB
286
287Example:
288
289 if ("ook" =~ /(o*)/) {
290 # `paren' will be `1' and `value' will be `ee'
291 $1 =~ tr/o/e/;
292 }
293
294Perl's own engine will croak on any attempt to modify the capture
a0e97681 295variables, to do this in another engine use the following callback
2fdbfb4d
AB
296(copied from C<Perl_reg_numbered_buff_store>):
297
298 void
299 Example_reg_numbered_buff_store(pTHX_ REGEXP * const rx, const I32 paren,
300 SV const * const value)
301 {
302 PERL_UNUSED_ARG(rx);
303 PERL_UNUSED_ARG(paren);
304 PERL_UNUSED_ARG(value);
305
306 if (!PL_localizing)
307 Perl_croak(aTHX_ PL_no_modify);
308 }
309
99d59c4d 310Actually perl will not I<always> croak in a statement that looks
2fdbfb4d
AB
311like it would modify a numbered capture variable. This is because the
312STORE callback will not be called if perl can determine that it
313doesn't have to modify the value. This is exactly how tied variables
314behave in the same situation:
315
316 package CaptureVar;
317 use base 'Tie::Scalar';
318
319 sub TIESCALAR { bless [] }
320 sub FETCH { undef }
321 sub STORE { die "This doesn't get called" }
322
323 package main;
324
c69ca1d4 325 tie my $sv => "CaptureVar";
2fdbfb4d
AB
326 $sv =~ y/a/b/;
327
328Because C<$sv> is C<undef> when the C<y///> operator is applied to it
329the transliteration won't actually execute and the program won't
192b9cd1
AB
330C<die>. This is different to how 5.8 and earlier versions behaved
331since the capture variables were READONLY variables then, now they'll
332just die when assigned to in the default engine.
2fdbfb4d 333
192b9cd1 334=head3 numbered_buff_LENGTH
2fdbfb4d
AB
335
336 I32 numbered_buff_LENGTH (pTHX_ REGEXP * const rx, const SV * const sv,
337 const I32 paren);
338
339Get the C<length> of a capture variable. There's a special callback
340for this so that perl doesn't have to do a FETCH and run C<length> on
192b9cd1 341the result, since the length is (in perl's case) known from an offset
0a3a8dc0 342stored in C<< rx->offs >> this is much more efficient:
2fdbfb4d
AB
343
344 I32 s1 = rx->offs[paren].start;
345 I32 s2 = rx->offs[paren].end;
346 I32 len = t1 - s1;
347
348This is a little bit more complex in the case of UTF-8, see what
349C<Perl_reg_numbered_buff_length> does with
350L<is_utf8_string_loclen|perlapi/is_utf8_string_loclen>.
351
192b9cd1
AB
352=head2 Named capture callbacks
353
354Called to get/set the value of C<%+> and C<%-> as well as by some
355utility functions in L<re>.
356
357There are two callbacks, C<named_buff> is called in all the cases the
358FETCH, STORE, DELETE, CLEAR, EXISTS and SCALAR L<Tie::Hash> callbacks
359would be on changes to C<%+> and C<%-> and C<named_buff_iter> in the
360same cases as FIRSTKEY and NEXTKEY.
361
362The C<flags> parameter can be used to determine which of these
363operations the callbacks should respond to, the following flags are
364currently defined:
365
366Which L<Tie::Hash> operation is being performed from the Perl level on
367C<%+> or C<%+>, if any:
368
f1b875a0
YO
369 RXapif_FETCH
370 RXapif_STORE
371 RXapif_DELETE
372 RXapif_CLEAR
373 RXapif_EXISTS
374 RXapif_SCALAR
375 RXapif_FIRSTKEY
376 RXapif_NEXTKEY
192b9cd1
AB
377
378Whether C<%+> or C<%-> is being operated on, if any.
2fdbfb4d 379
f1b875a0
YO
380 RXapif_ONE /* %+ */
381 RXapif_ALL /* %- */
2fdbfb4d 382
192b9cd1 383Whether this is being called as C<re::regname>, C<re::regnames> or
c998b245 384C<re::regnames_count>, if any. The first two will be combined with
f1b875a0 385C<RXapif_ONE> or C<RXapif_ALL>.
192b9cd1 386
f1b875a0
YO
387 RXapif_REGNAME
388 RXapif_REGNAMES
389 RXapif_REGNAMES_COUNT
192b9cd1
AB
390
391Internally C<%+> and C<%-> are implemented with a real tied interface
392via L<Tie::Hash::NamedCapture>. The methods in that package will call
393back into these functions. However the usage of
394L<Tie::Hash::NamedCapture> for this purpose might change in future
395releases. For instance this might be implemented by magic instead
396(would need an extension to mgvtbl).
397
398=head3 named_buff
399
400 SV* (*named_buff) (pTHX_ REGEXP * const rx, SV * const key,
401 SV * const value, U32 flags);
402
403=head3 named_buff_iter
404
405 SV* (*named_buff_iter) (pTHX_ REGEXP * const rx, const SV * const lastkey,
406 const U32 flags);
108003db 407
49d7dfbc 408=head2 qr_package
108003db 409
49d7dfbc 410 SV* qr_package(pTHX_ REGEXP * const rx);
108003db
RGS
411
412The package the qr// magic object is blessed into (as seen by C<ref
49d7dfbc
AB
413qr//>). It is recommended that engines change this to their package
414name for identification regardless of whether they implement methods
415on the object.
416
192b9cd1 417The package this method returns should also have the internal
d5213412 418C<Regexp> package in its C<@ISA>. C<< qr//->isa("Regexp") >> should always
192b9cd1
AB
419be true regardless of what engine is being used.
420
421Example implementation might be:
108003db
RGS
422
423 SV*
192b9cd1 424 Example_qr_package(pTHX_ REGEXP * const rx)
108003db
RGS
425 {
426 PERL_UNUSED_ARG(rx);
427 return newSVpvs("re::engine::Example");
428 }
429
430Any method calls on an object created with C<qr//> will be dispatched to the
431package as a normal object.
432
433 use re::engine::Example;
434 my $re = qr//;
435 $re->meth; # dispatched to re::engine::Example::meth()
436
f7e71195
AB
437To retrieve the C<REGEXP> object from the scalar in an XS function use
438the C<SvRX> macro, see L<"REGEXP Functions" in perlapi|perlapi/REGEXP
439Functions>.
108003db
RGS
440
441 void meth(SV * rv)
442 PPCODE:
f7e71195 443 REGEXP * re = SvRX(sv);
108003db 444
108003db
RGS
445=head2 dupe
446
49d7dfbc 447 void* dupe(pTHX_ REGEXP * const rx, CLONE_PARAMS *param);
108003db
RGS
448
449On threaded builds a regexp may need to be duplicated so that the pattern
a0e97681 450can be used by multiple threads. This routine is expected to handle the
108003db
RGS
451duplication of any private data pointed to by the C<pprivate> member of
452the regexp structure. It will be called with the preconstructed new
453regexp structure as an argument, the C<pprivate> member will point at
a0e97681 454the B<old> private structure, and it is this routine's responsibility to
108003db
RGS
455construct a copy and return a pointer to it (which perl will then use to
456overwrite the field as passed to this routine.)
457
458This allows the engine to dupe its private data but also if necessary
459modify the final structure if it really must.
460
461On unthreaded builds this field doesn't exist.
462
463=head1 The REGEXP structure
464
465The REGEXP struct is defined in F<regexp.h>. All regex engines must be able to
466correctly build such a structure in their L</comp> routine.
467
468The REGEXP structure contains all the data that perl needs to be aware of
469to properly work with the regular expression. It includes data about
470optimisations that perl can use to determine if the regex engine should
471really be used, and various other control info that is needed to properly
472execute patterns in various contexts such as is the pattern anchored in
473some way, or what flags were used during the compile, or whether the
474program contains special constructs that perl needs to be aware of.
475
882227b7
AB
476In addition it contains two fields that are intended for the private
477use of the regex engine that compiled the pattern. These are the
478C<intflags> and C<pprivate> members. C<pprivate> is a void pointer to
479an arbitrary structure whose use and management is the responsibility
480of the compiling engine. perl will never modify either of these
481values.
108003db
RGS
482
483 typedef struct regexp {
484 /* what engine created this regexp? */
485 const struct regexp_engine* engine;
486
487 /* what re is this a lightweight copy of? */
488 struct regexp* mother_re;
489
490 /* Information about the match that the perl core uses to manage things */
491 U32 extflags; /* Flags used both externally and internally */
492 I32 minlen; /* mininum possible length of string to match */
493 I32 minlenret; /* mininum possible length of $& */
494 U32 gofs; /* chars left of pos that we search from */
495
496 /* substring data about strings that must appear
497 in the final match, used for optimisations */
498 struct reg_substr_data *substrs;
499
c27a5cfe 500 U32 nparens; /* number of capture groups */
108003db
RGS
501
502 /* private engine specific data */
503 U32 intflags; /* Engine Specific Internal flags */
504 void *pprivate; /* Data private to the regex engine which
505 created this object. */
506
507 /* Data about the last/current match. These are modified during matching*/
508 U32 lastparen; /* last open paren matched */
509 U32 lastcloseparen; /* last close paren matched */
510 regexp_paren_pair *swap; /* Swap copy of *offs */
511 regexp_paren_pair *offs; /* Array of offsets for (@-) and (@+) */
512
513 char *subbeg; /* saved or original string so \digit works forever. */
514 SV_SAVED_COPY /* If non-NULL, SV which is COW from original */
515 I32 sublen; /* Length of string pointed by subbeg */
516
517 /* Information about the match that isn't often used */
518 I32 prelen; /* length of precomp */
519 const char *precomp; /* pre-compilation regular expression */
520
108003db
RGS
521 char *wrapped; /* wrapped version of the pattern */
522 I32 wraplen; /* length of wrapped */
523
524 I32 seen_evals; /* number of eval groups in the pattern - for security checks */
525 HV *paren_names; /* Optional hash of paren names */
526
527 /* Refcount of this regexp */
528 I32 refcnt; /* Refcount of this regexp */
529 } regexp;
530
531The fields are discussed in more detail below:
532
882227b7 533=head2 C<engine>
108003db
RGS
534
535This field points at a regexp_engine structure which contains pointers
536to the subroutines that are to be used for performing a match. It
537is the compiling routine's responsibility to populate this field before
538returning the regexp object.
539
540Internally this is set to C<NULL> unless a custom engine is specified in
541C<$^H{regcomp}>, perl's own set of callbacks can be accessed in the struct
542pointed to by C<RE_ENGINE_PTR>.
543
882227b7 544=head2 C<mother_re>
108003db
RGS
545
546TODO, see L<http://www.mail-archive.com/perl5-changes@perl.org/msg17328.html>
547
882227b7 548=head2 C<extflags>
108003db 549
192b9cd1
AB
550This will be used by perl to see what flags the regexp was compiled
551with, this will normally be set to the value of the flags parameter by
c998b245
AB
552the L<comp|/comp> callback. See the L<comp|/comp> documentation for
553valid flags.
108003db 554
882227b7 555=head2 C<minlen> C<minlenret>
108003db
RGS
556
557The minimum string length required for the pattern to match. This is used to
558prune the search space by not bothering to match any closer to the end of a
559string than would allow a match. For instance there is no point in even
560starting the regex engine if the minlen is 10 but the string is only 5
561characters long. There is no way that the pattern can match.
562
563C<minlenret> is the minimum length of the string that would be found
564in $& after a match.
565
566The difference between C<minlen> and C<minlenret> can be seen in the
567following pattern:
568
569 /ns(?=\d)/
570
571where the C<minlen> would be 3 but C<minlenret> would only be 2 as the \d is
572required to match but is not actually included in the matched content. This
573distinction is particularly important as the substitution logic uses the
a0e97681 574C<minlenret> to tell whether it can do in-place substitution which can result in
108003db
RGS
575considerable speedup.
576
882227b7 577=head2 C<gofs>
108003db
RGS
578
579Left offset from pos() to start match at.
580
882227b7 581=head2 C<substrs>
108003db 582
192b9cd1
AB
583Substring data about strings that must appear in the final match. This
584is currently only used internally by perl's engine for but might be
c998b245 585used in the future for all engines for optimisations.
108003db 586
882227b7 587=head2 C<nparens>, C<lasparen>, and C<lastcloseparen>
108003db
RGS
588
589These fields are used to keep track of how many paren groups could be matched
590in the pattern, which was the last open paren to be entered, and which was
591the last close paren to be entered.
592
882227b7 593=head2 C<intflags>
108003db
RGS
594
595The engine's private copy of the flags the pattern was compiled with. Usually
192b9cd1 596this is the same as C<extflags> unless the engine chose to modify one of them.
108003db 597
882227b7 598=head2 C<pprivate>
108003db
RGS
599
600A void* pointing to an engine-defined data structure. The perl engine uses the
601C<regexp_internal> structure (see L<perlreguts/Base Structures>) but a custom
602engine should use something else.
603
882227b7 604=head2 C<swap>
108003db 605
e9105d30 606Unused. Left in for compatibility with perl 5.10.0.
108003db 607
882227b7 608=head2 C<offs>
108003db
RGS
609
610A C<regexp_paren_pair> structure which defines offsets into the string being
611matched which correspond to the C<$&> and C<$1>, C<$2> etc. captures, the
612C<regexp_paren_pair> struct is defined as follows:
613
614 typedef struct regexp_paren_pair {
615 I32 start;
616 I32 end;
617 } regexp_paren_pair;
618
619If C<< ->offs[num].start >> or C<< ->offs[num].end >> is C<-1> then that
c27a5cfe 620capture group did not match. C<< ->offs[0].start/end >> represents C<$&> (or
108003db
RGS
621C<${^MATCH> under C<//p>) and C<< ->offs[paren].end >> matches C<$$paren> where
622C<$paren >= 1>.
623
882227b7 624=head2 C<precomp> C<prelen>
108003db 625
192b9cd1
AB
626Used for optimisations. C<precomp> holds a copy of the pattern that
627was compiled and C<prelen> its length. When a new pattern is to be
628compiled (such as inside a loop) the internal C<regcomp> operator
629checks whether the last compiled C<REGEXP>'s C<precomp> and C<prelen>
630are equivalent to the new one, and if so uses the old pattern instead
631of compiling a new one.
632
633The relevant snippet from C<Perl_pp_regcomp>:
634
635 if (!re || !re->precomp || re->prelen != (I32)len ||
636 memNE(re->precomp, t, len))
637 /* Compile a new pattern */
108003db 638
882227b7 639=head2 C<paren_names>
108003db 640
c27a5cfe 641This is a hash used internally to track named capture groups and their
108003db
RGS
642offsets. The keys are the names of the buffers the values are dualvars,
643with the IV slot holding the number of buffers with the given name and the
644pv being an embedded array of I32. The values may also be contained
645independently in the data array in cases where named backreferences are
646used.
647
c998b245 648=head2 C<substrs>
108003db
RGS
649
650Holds information on the longest string that must occur at a fixed
651offset from the start of the pattern, and the longest string that must
652occur at a floating offset from the start of the pattern. Used to do
653Fast-Boyer-Moore searches on the string to find out if its worth using
654the regex engine at all, and if so where in the string to search.
655
882227b7 656=head2 C<subbeg> C<sublen> C<saved_copy>
108003db 657
c998b245 658Used during execution phase for managing search and replace patterns.
108003db 659
882227b7 660=head2 C<wrapped> C<wraplen>
108003db 661
c998b245 662Stores the string C<qr//> stringifies to. The perl engine for example
ed215d3c 663stores C<(?^:eek)> in the case of C<qr/eek/>.
108003db 664
c998b245
AB
665When using a custom engine that doesn't support the C<(?:)> construct
666for inline modifiers, it's probably best to have C<qr//> stringify to
667the supplied pattern, note that this will create undesired patterns in
668cases such as:
108003db
RGS
669
670 my $x = qr/a|b/; # "a|b"
192b9cd1 671 my $y = qr/c/i; # "c"
108003db
RGS
672 my $z = qr/$x$y/; # "a|bc"
673
192b9cd1
AB
674There's no solution for this problem other than making the custom
675engine understand a construct like C<(?:)>.
108003db 676
882227b7 677=head2 C<seen_evals>
108003db
RGS
678
679This stores the number of eval groups in the pattern. This is used for security
680purposes when embedding compiled regexes into larger patterns with C<qr//>.
681
882227b7 682=head2 C<refcnt>
108003db
RGS
683
684The number of times the structure is referenced. When this falls to 0 the
685regexp is automatically freed by a call to pregfree. This should be set to 1 in
686each engine's L</comp> routine.
687
108003db
RGS
688=head1 HISTORY
689
690Originally part of L<perlreguts>.
691
692=head1 AUTHORS
693
694Originally written by Yves Orton, expanded by E<AElig>var ArnfjE<ouml>rE<eth>
695Bjarmason.
696
697=head1 LICENSE
698
699Copyright 2006 Yves Orton and 2007 E<AElig>var ArnfjE<ouml>rE<eth> Bjarmason.
700
701This program is free software; you can redistribute it and/or modify it under
702the same terms as Perl itself.
703
704=cut