This is a live mirror of the Perl 5 development currently hosted at https://github.com/perl/perl5
Change comments, documentation for new (?^...)
[perl5.git] / pod / perlreapi.pod
CommitLineData
108003db
RGS
1=head1 NAME
2
3perlreapi - perl regular expression plugin interface
4
5=head1 DESCRIPTION
6
a0e97681
RGS
7As of Perl 5.9.5 there is a new interface for plugging and using other
8regular expression engines than the default one.
9
10Each engine is supposed to provide access to a constant structure of the
11following format:
108003db
RGS
12
13 typedef struct regexp_engine {
3ab4a224 14 REGEXP* (*comp) (pTHX_ const SV * const pattern, const U32 flags);
49d7dfbc 15 I32 (*exec) (pTHX_ REGEXP * const rx, char* stringarg, char* strend,
2fdbfb4d
AB
16 char* strbeg, I32 minend, SV* screamer,
17 void* data, U32 flags);
49d7dfbc 18 char* (*intuit) (pTHX_ REGEXP * const rx, SV *sv, char *strpos,
2fdbfb4d
AB
19 char *strend, U32 flags,
20 struct re_scream_pos_data_s *data);
49d7dfbc
AB
21 SV* (*checkstr) (pTHX_ REGEXP * const rx);
22 void (*free) (pTHX_ REGEXP * const rx);
2fdbfb4d
AB
23 void (*numbered_buff_FETCH) (pTHX_ REGEXP * const rx, const I32 paren,
24 SV * const sv);
25 void (*numbered_buff_STORE) (pTHX_ REGEXP * const rx, const I32 paren,
26 SV const * const value);
27 I32 (*numbered_buff_LENGTH) (pTHX_ REGEXP * const rx, const SV * const sv,
28 const I32 paren);
192b9cd1
AB
29 SV* (*named_buff) (pTHX_ REGEXP * const rx, SV * const key,
30 SV * const value, U32 flags);
31 SV* (*named_buff_iter) (pTHX_ REGEXP * const rx, const SV * const lastkey,
32 const U32 flags);
49d7dfbc 33 SV* (*qr_package)(pTHX_ REGEXP * const rx);
108003db 34 #ifdef USE_ITHREADS
49d7dfbc 35 void* (*dupe) (pTHX_ REGEXP * const rx, CLONE_PARAMS *param);
108003db 36 #endif
108003db
RGS
37
38When a regexp is compiled, its C<engine> field is then set to point at
a0e97681 39the appropriate structure, so that when it needs to be used Perl can find
108003db
RGS
40the right routines to do so.
41
42In order to install a new regexp handler, C<$^H{regcomp}> is set
43to an integer which (when casted appropriately) resolves to one of these
44structures. When compiling, the C<comp> method is executed, and the
45resulting regexp structure's engine field is expected to point back at
46the same structure.
47
48The pTHX_ symbol in the definition is a macro used by perl under threading
49to provide an extra argument to the routine holding a pointer back to
50the interpreter that is executing the regexp. So under threading all
51routines get an extra argument.
52
882227b7 53=head1 Callbacks
108003db
RGS
54
55=head2 comp
56
3ab4a224 57 REGEXP* comp(pTHX_ const SV * const pattern, const U32 flags);
108003db 58
3ab4a224
AB
59Compile the pattern stored in C<pattern> using the given C<flags> and
60return a pointer to a prepared C<REGEXP> structure that can perform
61the match. See L</The REGEXP structure> below for an explanation of
62the individual fields in the REGEXP struct.
63
64The C<pattern> parameter is the scalar that was used as the
65pattern. previous versions of perl would pass two C<char*> indicating
a0e97681 66the start and end of the stringified pattern, the following snippet can
3ab4a224
AB
67be used to get the old parameters:
68
69 STRLEN plen;
70 char* exp = SvPV(pattern, plen);
71 char* xend = exp + plen;
72
73Since any scalar can be passed as a pattern it's possible to implement
74an engine that does something with an array (C<< "ook" =~ [ qw/ eek
75hlagh / ] >>) or with the non-stringified form of a compiled regular
76expression (C<< "ook" =~ qr/eek/ >>). perl's own engine will always
77stringify everything using the snippet above but that doesn't mean
78other engines have to.
108003db 79
a0e97681 80The C<flags> parameter is a bitfield which indicates which of the
c998b245
AB
81C<msixp> flags the regex was compiled with. It also contains
82additional info such as whether C<use locale> is in effect.
108003db
RGS
83
84The C<eogc> flags are stripped out before being passed to the comp
85routine. The regex engine does not need to know whether any of these
3ab4a224 86are set as those flags should only affect what perl does with the
c998b245
AB
87pattern and its match variables, not how it gets compiled and
88executed.
108003db 89
c998b245
AB
90By the time the comp callback is called, some of these flags have
91already had effect (noted below where applicable). However most of
92their effect occurs after the comp callback has run in routines that
93read the C<< rx->extflags >> field which it populates.
108003db 94
c998b245
AB
95In general the flags should be preserved in C<< rx->extflags >> after
96compilation, although the regex engine might want to add or delete
97some of them to invoke or disable some special behavior in perl. The
98flags along with any special behavior they cause are documented below:
108003db 99
c998b245 100The pattern modifiers:
108003db 101
c998b245 102=over 4
108003db 103
c998b245 104=item C</m> - RXf_PMf_MULTILINE
108003db 105
c998b245
AB
106If this is in C<< rx->extflags >> it will be passed to
107C<Perl_fbm_instr> by C<pp_split> which will treat the subject string
108as a multi-line string.
108003db 109
c998b245 110=item C</s> - RXf_PMf_SINGLELINE
108003db 111
c998b245 112=item C</i> - RXf_PMf_FOLD
108003db 113
c998b245 114=item C</x> - RXf_PMf_EXTENDED
108003db 115
c998b245
AB
116If present on a regex C<#> comments will be handled differently by the
117tokenizer in some cases.
108003db 118
c998b245 119TODO: Document those cases.
108003db 120
c998b245 121=item C</p> - RXf_PMf_KEEPCOPY
108003db 122
c998b245 123=back
108003db 124
c998b245 125Additional flags:
108003db 126
c998b245 127=over 4
108003db 128
c998b245 129=item RXf_PMf_LOCALE
108003db 130
c998b245 131Set if C<use locale> is in effect. If present in C<< rx->extflags >>
a0e97681 132C<split> will use the locale dependent definition of whitespace under
c998b245
AB
133when RXf_SKIPWHITE or RXf_WHITE are in effect. Under ASCII whitespace
134is defined as per L<isSPACE|perlapi/ISSPACE>, and by the internal
135macros C<is_utf8_space> under UTF-8 and C<isSPACE_LC> under C<use
136locale>.
108003db
RGS
137
138=item RXf_UTF8
139
140Set if the pattern is L<SvUTF8()|perlapi/SvUTF8>, set by Perl_pmruntime.
141
c998b245
AB
142A regex engine may want to set or disable this flag during
143compilation. The perl engine for instance may upgrade non-UTF-8
144strings to UTF-8 if the pattern includes constructs such as C<\x{...}>
145that can only match Unicode values.
146
0ac6acae
AB
147=item RXf_SPLIT
148
149If C<split> is invoked as C<split ' '> or with no arguments (which
5137fa37 150really means C<split(' ', $_)>, see L<split|perlfunc/split>), perl will
0ac6acae
AB
151set this flag. The regex engine can then check for it and set the
152SKIPWHITE and WHITE extflags. To do this the perl engine does:
153
154 if (flags & RXf_SPLIT && r->prelen == 1 && r->precomp[0] == ' ')
155 r->extflags |= (RXf_SKIPWHITE|RXf_WHITE);
156
108003db
RGS
157=back
158
c998b245
AB
159These flags can be set during compilation to enable optimizations in
160the C<split> operator.
161
162=over 4
163
0ac6acae
AB
164=item RXf_SKIPWHITE
165
166If the flag is present in C<< rx->extflags >> C<split> will delete
167whitespace from the start of the subject string before it's operated
168on. What is considered whitespace depends on whether the subject is a
169UTF-8 string and whether the C<RXf_PMf_LOCALE> flag is set.
170
171If RXf_WHITE is set in addition to this flag C<split> will behave like
172C<split " "> under the perl engine.
173
c998b245
AB
174=item RXf_START_ONLY
175
176Tells the split operator to split the target string on newlines
177(C<\n>) without invoking the regex engine.
178
179Perl's engine sets this if the pattern is C</^/> (C<plen == 1 && *exp
180== '^'>), even under C</^/s>, see L<split|perlfunc>. Of course a
181different regex engine might want to use the same optimizations
182with a different syntax.
183
184=item RXf_WHITE
185
186Tells the split operator to split the target string on whitespace
187without invoking the regex engine. The definition of whitespace varies
188depending on whether the target string is a UTF-8 string and on
189whether RXf_PMf_LOCALE is set.
190
0ac6acae 191Perl's engine sets this flag if the pattern is C<\s+>.
c998b245 192
640f820d
AB
193=item RXf_NULL
194
a0e97681 195Tells the split operator to split the target string on
640f820d
AB
196characters. The definition of character varies depending on whether
197the target string is a UTF-8 string.
198
199Perl's engine sets this flag on empty patterns, this optimization
a0e97681 200makes C<split //> much faster than it would otherwise be. It's even
640f820d
AB
201faster than C<unpack>.
202
c998b245 203=back
108003db
RGS
204
205=head2 exec
206
49d7dfbc 207 I32 exec(pTHX_ REGEXP * const rx,
108003db
RGS
208 char *stringarg, char* strend, char* strbeg,
209 I32 minend, SV* screamer,
210 void* data, U32 flags);
211
212Execute a regexp.
213
214=head2 intuit
215
49d7dfbc 216 char* intuit(pTHX_ REGEXP * const rx,
108003db 217 SV *sv, char *strpos, char *strend,
49d7dfbc 218 const U32 flags, struct re_scream_pos_data_s *data);
108003db
RGS
219
220Find the start position where a regex match should be attempted,
221or possibly whether the regex engine should not be run because the
222pattern can't match. This is called as appropriate by the core
223depending on the values of the extflags member of the regexp
224structure.
225
226=head2 checkstr
227
49d7dfbc 228 SV* checkstr(pTHX_ REGEXP * const rx);
108003db
RGS
229
230Return a SV containing a string that must appear in the pattern. Used
231by C<split> for optimising matches.
232
233=head2 free
234
49d7dfbc 235 void free(pTHX_ REGEXP * const rx);
108003db
RGS
236
237Called by perl when it is freeing a regexp pattern so that the engine
238can release any resources pointed to by the C<pprivate> member of the
239regexp structure. This is only responsible for freeing private data;
240perl will handle releasing anything else contained in the regexp structure.
241
192b9cd1 242=head2 Numbered capture callbacks
108003db 243
192b9cd1
AB
244Called to get/set the value of C<$`>, C<$'>, C<$&> and their named
245equivalents, ${^PREMATCH}, ${^POSTMATCH} and $^{MATCH}, as well as the
c27a5cfe 246numbered capture groups (C<$1>, C<$2>, ...).
49d7dfbc 247
a0e97681 248The C<paren> parameter will be C<-2> for C<$`>, C<-1> for C<$'>, C<0>
49d7dfbc
AB
249for C<$&>, C<1> for C<$1> and so forth.
250
192b9cd1
AB
251The names have been chosen by analogy with L<Tie::Scalar> methods
252names with an additional B<LENGTH> callback for efficiency. However
253named capture variables are currently not tied internally but
254implemented via magic.
255
256=head3 numbered_buff_FETCH
257
258 void numbered_buff_FETCH(pTHX_ REGEXP * const rx, const I32 paren,
259 SV * const sv);
260
261Fetch a specified numbered capture. C<sv> should be set to the scalar
262to return, the scalar is passed as an argument rather than being
263returned from the function because when it's called perl already has a
264scalar to store the value, creating another one would be
265redundant. The scalar can be set with C<sv_setsv>, C<sv_setpvn> and
266friends, see L<perlapi>.
49d7dfbc
AB
267
268This callback is where perl untaints its own capture variables under
c998b245 269taint mode (see L<perlsec>). See the C<Perl_reg_numbered_buff_fetch>
49d7dfbc
AB
270function in F<regcomp.c> for how to untaint capture variables if
271that's something you'd like your engine to do as well.
108003db 272
192b9cd1 273=head3 numbered_buff_STORE
108003db 274
2fdbfb4d
AB
275 void (*numbered_buff_STORE) (pTHX_ REGEXP * const rx, const I32 paren,
276 SV const * const value);
108003db 277
192b9cd1
AB
278Set the value of a numbered capture variable. C<value> is the scalar
279that is to be used as the new value. It's up to the engine to make
280sure this is used as the new value (or reject it).
2fdbfb4d
AB
281
282Example:
283
284 if ("ook" =~ /(o*)/) {
285 # `paren' will be `1' and `value' will be `ee'
286 $1 =~ tr/o/e/;
287 }
288
289Perl's own engine will croak on any attempt to modify the capture
a0e97681 290variables, to do this in another engine use the following callback
2fdbfb4d
AB
291(copied from C<Perl_reg_numbered_buff_store>):
292
293 void
294 Example_reg_numbered_buff_store(pTHX_ REGEXP * const rx, const I32 paren,
295 SV const * const value)
296 {
297 PERL_UNUSED_ARG(rx);
298 PERL_UNUSED_ARG(paren);
299 PERL_UNUSED_ARG(value);
300
301 if (!PL_localizing)
302 Perl_croak(aTHX_ PL_no_modify);
303 }
304
99d59c4d 305Actually perl will not I<always> croak in a statement that looks
2fdbfb4d
AB
306like it would modify a numbered capture variable. This is because the
307STORE callback will not be called if perl can determine that it
308doesn't have to modify the value. This is exactly how tied variables
309behave in the same situation:
310
311 package CaptureVar;
312 use base 'Tie::Scalar';
313
314 sub TIESCALAR { bless [] }
315 sub FETCH { undef }
316 sub STORE { die "This doesn't get called" }
317
318 package main;
319
320 tie my $sv => "CatptureVar";
321 $sv =~ y/a/b/;
322
323Because C<$sv> is C<undef> when the C<y///> operator is applied to it
324the transliteration won't actually execute and the program won't
192b9cd1
AB
325C<die>. This is different to how 5.8 and earlier versions behaved
326since the capture variables were READONLY variables then, now they'll
327just die when assigned to in the default engine.
2fdbfb4d 328
192b9cd1 329=head3 numbered_buff_LENGTH
2fdbfb4d
AB
330
331 I32 numbered_buff_LENGTH (pTHX_ REGEXP * const rx, const SV * const sv,
332 const I32 paren);
333
334Get the C<length> of a capture variable. There's a special callback
335for this so that perl doesn't have to do a FETCH and run C<length> on
192b9cd1 336the result, since the length is (in perl's case) known from an offset
0a3a8dc0 337stored in C<< rx->offs >> this is much more efficient:
2fdbfb4d
AB
338
339 I32 s1 = rx->offs[paren].start;
340 I32 s2 = rx->offs[paren].end;
341 I32 len = t1 - s1;
342
343This is a little bit more complex in the case of UTF-8, see what
344C<Perl_reg_numbered_buff_length> does with
345L<is_utf8_string_loclen|perlapi/is_utf8_string_loclen>.
346
192b9cd1
AB
347=head2 Named capture callbacks
348
349Called to get/set the value of C<%+> and C<%-> as well as by some
350utility functions in L<re>.
351
352There are two callbacks, C<named_buff> is called in all the cases the
353FETCH, STORE, DELETE, CLEAR, EXISTS and SCALAR L<Tie::Hash> callbacks
354would be on changes to C<%+> and C<%-> and C<named_buff_iter> in the
355same cases as FIRSTKEY and NEXTKEY.
356
357The C<flags> parameter can be used to determine which of these
358operations the callbacks should respond to, the following flags are
359currently defined:
360
361Which L<Tie::Hash> operation is being performed from the Perl level on
362C<%+> or C<%+>, if any:
363
f1b875a0
YO
364 RXapif_FETCH
365 RXapif_STORE
366 RXapif_DELETE
367 RXapif_CLEAR
368 RXapif_EXISTS
369 RXapif_SCALAR
370 RXapif_FIRSTKEY
371 RXapif_NEXTKEY
192b9cd1
AB
372
373Whether C<%+> or C<%-> is being operated on, if any.
2fdbfb4d 374
f1b875a0
YO
375 RXapif_ONE /* %+ */
376 RXapif_ALL /* %- */
2fdbfb4d 377
192b9cd1 378Whether this is being called as C<re::regname>, C<re::regnames> or
c998b245 379C<re::regnames_count>, if any. The first two will be combined with
f1b875a0 380C<RXapif_ONE> or C<RXapif_ALL>.
192b9cd1 381
f1b875a0
YO
382 RXapif_REGNAME
383 RXapif_REGNAMES
384 RXapif_REGNAMES_COUNT
192b9cd1
AB
385
386Internally C<%+> and C<%-> are implemented with a real tied interface
387via L<Tie::Hash::NamedCapture>. The methods in that package will call
388back into these functions. However the usage of
389L<Tie::Hash::NamedCapture> for this purpose might change in future
390releases. For instance this might be implemented by magic instead
391(would need an extension to mgvtbl).
392
393=head3 named_buff
394
395 SV* (*named_buff) (pTHX_ REGEXP * const rx, SV * const key,
396 SV * const value, U32 flags);
397
398=head3 named_buff_iter
399
400 SV* (*named_buff_iter) (pTHX_ REGEXP * const rx, const SV * const lastkey,
401 const U32 flags);
108003db 402
49d7dfbc 403=head2 qr_package
108003db 404
49d7dfbc 405 SV* qr_package(pTHX_ REGEXP * const rx);
108003db
RGS
406
407The package the qr// magic object is blessed into (as seen by C<ref
49d7dfbc
AB
408qr//>). It is recommended that engines change this to their package
409name for identification regardless of whether they implement methods
410on the object.
411
192b9cd1 412The package this method returns should also have the internal
d5213412 413C<Regexp> package in its C<@ISA>. C<< qr//->isa("Regexp") >> should always
192b9cd1
AB
414be true regardless of what engine is being used.
415
416Example implementation might be:
108003db
RGS
417
418 SV*
192b9cd1 419 Example_qr_package(pTHX_ REGEXP * const rx)
108003db
RGS
420 {
421 PERL_UNUSED_ARG(rx);
422 return newSVpvs("re::engine::Example");
423 }
424
425Any method calls on an object created with C<qr//> will be dispatched to the
426package as a normal object.
427
428 use re::engine::Example;
429 my $re = qr//;
430 $re->meth; # dispatched to re::engine::Example::meth()
431
f7e71195
AB
432To retrieve the C<REGEXP> object from the scalar in an XS function use
433the C<SvRX> macro, see L<"REGEXP Functions" in perlapi|perlapi/REGEXP
434Functions>.
108003db
RGS
435
436 void meth(SV * rv)
437 PPCODE:
f7e71195 438 REGEXP * re = SvRX(sv);
108003db 439
108003db
RGS
440=head2 dupe
441
49d7dfbc 442 void* dupe(pTHX_ REGEXP * const rx, CLONE_PARAMS *param);
108003db
RGS
443
444On threaded builds a regexp may need to be duplicated so that the pattern
a0e97681 445can be used by multiple threads. This routine is expected to handle the
108003db
RGS
446duplication of any private data pointed to by the C<pprivate> member of
447the regexp structure. It will be called with the preconstructed new
448regexp structure as an argument, the C<pprivate> member will point at
a0e97681 449the B<old> private structure, and it is this routine's responsibility to
108003db
RGS
450construct a copy and return a pointer to it (which perl will then use to
451overwrite the field as passed to this routine.)
452
453This allows the engine to dupe its private data but also if necessary
454modify the final structure if it really must.
455
456On unthreaded builds this field doesn't exist.
457
458=head1 The REGEXP structure
459
460The REGEXP struct is defined in F<regexp.h>. All regex engines must be able to
461correctly build such a structure in their L</comp> routine.
462
463The REGEXP structure contains all the data that perl needs to be aware of
464to properly work with the regular expression. It includes data about
465optimisations that perl can use to determine if the regex engine should
466really be used, and various other control info that is needed to properly
467execute patterns in various contexts such as is the pattern anchored in
468some way, or what flags were used during the compile, or whether the
469program contains special constructs that perl needs to be aware of.
470
882227b7
AB
471In addition it contains two fields that are intended for the private
472use of the regex engine that compiled the pattern. These are the
473C<intflags> and C<pprivate> members. C<pprivate> is a void pointer to
474an arbitrary structure whose use and management is the responsibility
475of the compiling engine. perl will never modify either of these
476values.
108003db
RGS
477
478 typedef struct regexp {
479 /* what engine created this regexp? */
480 const struct regexp_engine* engine;
481
482 /* what re is this a lightweight copy of? */
483 struct regexp* mother_re;
484
485 /* Information about the match that the perl core uses to manage things */
486 U32 extflags; /* Flags used both externally and internally */
487 I32 minlen; /* mininum possible length of string to match */
488 I32 minlenret; /* mininum possible length of $& */
489 U32 gofs; /* chars left of pos that we search from */
490
491 /* substring data about strings that must appear
492 in the final match, used for optimisations */
493 struct reg_substr_data *substrs;
494
c27a5cfe 495 U32 nparens; /* number of capture groups */
108003db
RGS
496
497 /* private engine specific data */
498 U32 intflags; /* Engine Specific Internal flags */
499 void *pprivate; /* Data private to the regex engine which
500 created this object. */
501
502 /* Data about the last/current match. These are modified during matching*/
503 U32 lastparen; /* last open paren matched */
504 U32 lastcloseparen; /* last close paren matched */
505 regexp_paren_pair *swap; /* Swap copy of *offs */
506 regexp_paren_pair *offs; /* Array of offsets for (@-) and (@+) */
507
508 char *subbeg; /* saved or original string so \digit works forever. */
509 SV_SAVED_COPY /* If non-NULL, SV which is COW from original */
510 I32 sublen; /* Length of string pointed by subbeg */
511
512 /* Information about the match that isn't often used */
513 I32 prelen; /* length of precomp */
514 const char *precomp; /* pre-compilation regular expression */
515
108003db
RGS
516 char *wrapped; /* wrapped version of the pattern */
517 I32 wraplen; /* length of wrapped */
518
519 I32 seen_evals; /* number of eval groups in the pattern - for security checks */
520 HV *paren_names; /* Optional hash of paren names */
521
522 /* Refcount of this regexp */
523 I32 refcnt; /* Refcount of this regexp */
524 } regexp;
525
526The fields are discussed in more detail below:
527
882227b7 528=head2 C<engine>
108003db
RGS
529
530This field points at a regexp_engine structure which contains pointers
531to the subroutines that are to be used for performing a match. It
532is the compiling routine's responsibility to populate this field before
533returning the regexp object.
534
535Internally this is set to C<NULL> unless a custom engine is specified in
536C<$^H{regcomp}>, perl's own set of callbacks can be accessed in the struct
537pointed to by C<RE_ENGINE_PTR>.
538
882227b7 539=head2 C<mother_re>
108003db
RGS
540
541TODO, see L<http://www.mail-archive.com/perl5-changes@perl.org/msg17328.html>
542
882227b7 543=head2 C<extflags>
108003db 544
192b9cd1
AB
545This will be used by perl to see what flags the regexp was compiled
546with, this will normally be set to the value of the flags parameter by
c998b245
AB
547the L<comp|/comp> callback. See the L<comp|/comp> documentation for
548valid flags.
108003db 549
882227b7 550=head2 C<minlen> C<minlenret>
108003db
RGS
551
552The minimum string length required for the pattern to match. This is used to
553prune the search space by not bothering to match any closer to the end of a
554string than would allow a match. For instance there is no point in even
555starting the regex engine if the minlen is 10 but the string is only 5
556characters long. There is no way that the pattern can match.
557
558C<minlenret> is the minimum length of the string that would be found
559in $& after a match.
560
561The difference between C<minlen> and C<minlenret> can be seen in the
562following pattern:
563
564 /ns(?=\d)/
565
566where the C<minlen> would be 3 but C<minlenret> would only be 2 as the \d is
567required to match but is not actually included in the matched content. This
568distinction is particularly important as the substitution logic uses the
a0e97681 569C<minlenret> to tell whether it can do in-place substitution which can result in
108003db
RGS
570considerable speedup.
571
882227b7 572=head2 C<gofs>
108003db
RGS
573
574Left offset from pos() to start match at.
575
882227b7 576=head2 C<substrs>
108003db 577
192b9cd1
AB
578Substring data about strings that must appear in the final match. This
579is currently only used internally by perl's engine for but might be
c998b245 580used in the future for all engines for optimisations.
108003db 581
882227b7 582=head2 C<nparens>, C<lasparen>, and C<lastcloseparen>
108003db
RGS
583
584These fields are used to keep track of how many paren groups could be matched
585in the pattern, which was the last open paren to be entered, and which was
586the last close paren to be entered.
587
882227b7 588=head2 C<intflags>
108003db
RGS
589
590The engine's private copy of the flags the pattern was compiled with. Usually
192b9cd1 591this is the same as C<extflags> unless the engine chose to modify one of them.
108003db 592
882227b7 593=head2 C<pprivate>
108003db
RGS
594
595A void* pointing to an engine-defined data structure. The perl engine uses the
596C<regexp_internal> structure (see L<perlreguts/Base Structures>) but a custom
597engine should use something else.
598
882227b7 599=head2 C<swap>
108003db 600
e9105d30 601Unused. Left in for compatibility with perl 5.10.0.
108003db 602
882227b7 603=head2 C<offs>
108003db
RGS
604
605A C<regexp_paren_pair> structure which defines offsets into the string being
606matched which correspond to the C<$&> and C<$1>, C<$2> etc. captures, the
607C<regexp_paren_pair> struct is defined as follows:
608
609 typedef struct regexp_paren_pair {
610 I32 start;
611 I32 end;
612 } regexp_paren_pair;
613
614If C<< ->offs[num].start >> or C<< ->offs[num].end >> is C<-1> then that
c27a5cfe 615capture group did not match. C<< ->offs[0].start/end >> represents C<$&> (or
108003db
RGS
616C<${^MATCH> under C<//p>) and C<< ->offs[paren].end >> matches C<$$paren> where
617C<$paren >= 1>.
618
882227b7 619=head2 C<precomp> C<prelen>
108003db 620
192b9cd1
AB
621Used for optimisations. C<precomp> holds a copy of the pattern that
622was compiled and C<prelen> its length. When a new pattern is to be
623compiled (such as inside a loop) the internal C<regcomp> operator
624checks whether the last compiled C<REGEXP>'s C<precomp> and C<prelen>
625are equivalent to the new one, and if so uses the old pattern instead
626of compiling a new one.
627
628The relevant snippet from C<Perl_pp_regcomp>:
629
630 if (!re || !re->precomp || re->prelen != (I32)len ||
631 memNE(re->precomp, t, len))
632 /* Compile a new pattern */
108003db 633
882227b7 634=head2 C<paren_names>
108003db 635
c27a5cfe 636This is a hash used internally to track named capture groups and their
108003db
RGS
637offsets. The keys are the names of the buffers the values are dualvars,
638with the IV slot holding the number of buffers with the given name and the
639pv being an embedded array of I32. The values may also be contained
640independently in the data array in cases where named backreferences are
641used.
642
c998b245 643=head2 C<substrs>
108003db
RGS
644
645Holds information on the longest string that must occur at a fixed
646offset from the start of the pattern, and the longest string that must
647occur at a floating offset from the start of the pattern. Used to do
648Fast-Boyer-Moore searches on the string to find out if its worth using
649the regex engine at all, and if so where in the string to search.
650
882227b7 651=head2 C<subbeg> C<sublen> C<saved_copy>
108003db 652
c998b245 653Used during execution phase for managing search and replace patterns.
108003db 654
882227b7 655=head2 C<wrapped> C<wraplen>
108003db 656
c998b245 657Stores the string C<qr//> stringifies to. The perl engine for example
ed215d3c 658stores C<(?^:eek)> in the case of C<qr/eek/>.
108003db 659
c998b245
AB
660When using a custom engine that doesn't support the C<(?:)> construct
661for inline modifiers, it's probably best to have C<qr//> stringify to
662the supplied pattern, note that this will create undesired patterns in
663cases such as:
108003db
RGS
664
665 my $x = qr/a|b/; # "a|b"
192b9cd1 666 my $y = qr/c/i; # "c"
108003db
RGS
667 my $z = qr/$x$y/; # "a|bc"
668
192b9cd1
AB
669There's no solution for this problem other than making the custom
670engine understand a construct like C<(?:)>.
108003db 671
882227b7 672=head2 C<seen_evals>
108003db
RGS
673
674This stores the number of eval groups in the pattern. This is used for security
675purposes when embedding compiled regexes into larger patterns with C<qr//>.
676
882227b7 677=head2 C<refcnt>
108003db
RGS
678
679The number of times the structure is referenced. When this falls to 0 the
680regexp is automatically freed by a call to pregfree. This should be set to 1 in
681each engine's L</comp> routine.
682
108003db
RGS
683=head1 HISTORY
684
685Originally part of L<perlreguts>.
686
687=head1 AUTHORS
688
689Originally written by Yves Orton, expanded by E<AElig>var ArnfjE<ouml>rE<eth>
690Bjarmason.
691
692=head1 LICENSE
693
694Copyright 2006 Yves Orton and 2007 E<AElig>var ArnfjE<ouml>rE<eth> Bjarmason.
695
696This program is free software; you can redistribute it and/or modify it under
697the same terms as Perl itself.
698
699=cut