This is a live mirror of the Perl 5 development currently hosted at https://github.com/perl/perl5
regexec.c: White-space only; comment only; no code changes
[perl5.git] / pod / perlreapi.pod
CommitLineData
108003db
RGS
1=head1 NAME
2
3perlreapi - perl regular expression plugin interface
4
5=head1 DESCRIPTION
6
a0e97681
RGS
7As of Perl 5.9.5 there is a new interface for plugging and using other
8regular expression engines than the default one.
9
10Each engine is supposed to provide access to a constant structure of the
11following format:
108003db
RGS
12
13 typedef struct regexp_engine {
02c01adb
KW
14 REGEXP* (*comp) (pTHX_
15 const SV * const pattern, const U32 flags);
16 I32 (*exec) (pTHX_
17 REGEXP * const rx,
18 char* stringarg,
19 char* strend, char* strbeg,
20 I32 minend, SV* screamer,
2fdbfb4d 21 void* data, U32 flags);
02c01adb
KW
22 char* (*intuit) (pTHX_
23 REGEXP * const rx, SV *sv,
24 char *strpos, char *strend, U32 flags,
2fdbfb4d 25 struct re_scream_pos_data_s *data);
49d7dfbc
AB
26 SV* (*checkstr) (pTHX_ REGEXP * const rx);
27 void (*free) (pTHX_ REGEXP * const rx);
02c01adb
KW
28 void (*numbered_buff_FETCH) (pTHX_
29 REGEXP * const rx,
30 const I32 paren,
31 SV * const sv);
32 void (*numbered_buff_STORE) (pTHX_
33 REGEXP * const rx,
34 const I32 paren,
2fdbfb4d 35 SV const * const value);
02c01adb
KW
36 I32 (*numbered_buff_LENGTH) (pTHX_
37 REGEXP * const rx,
38 const SV * const sv,
39 const I32 paren);
40 SV* (*named_buff) (pTHX_
41 REGEXP * const rx,
42 SV * const key,
43 SV * const value,
44 U32 flags);
45 SV* (*named_buff_iter) (pTHX_
46 REGEXP * const rx,
47 const SV * const lastkey,
192b9cd1 48 const U32 flags);
49d7dfbc 49 SV* (*qr_package)(pTHX_ REGEXP * const rx);
108003db 50 #ifdef USE_ITHREADS
49d7dfbc 51 void* (*dupe) (pTHX_ REGEXP * const rx, CLONE_PARAMS *param);
108003db 52 #endif
3c13cae6
DM
53 REGEXP* (*op_comp) (...);
54
108003db
RGS
55
56When a regexp is compiled, its C<engine> field is then set to point at
a0e97681 57the appropriate structure, so that when it needs to be used Perl can find
108003db
RGS
58the right routines to do so.
59
60In order to install a new regexp handler, C<$^H{regcomp}> is set
61to an integer which (when casted appropriately) resolves to one of these
62structures. When compiling, the C<comp> method is executed, and the
63resulting regexp structure's engine field is expected to point back at
64the same structure.
65
66The pTHX_ symbol in the definition is a macro used by perl under threading
67to provide an extra argument to the routine holding a pointer back to
68the interpreter that is executing the regexp. So under threading all
69routines get an extra argument.
70
882227b7 71=head1 Callbacks
108003db
RGS
72
73=head2 comp
74
3ab4a224 75 REGEXP* comp(pTHX_ const SV * const pattern, const U32 flags);
108003db 76
3ab4a224
AB
77Compile the pattern stored in C<pattern> using the given C<flags> and
78return a pointer to a prepared C<REGEXP> structure that can perform
79the match. See L</The REGEXP structure> below for an explanation of
80the individual fields in the REGEXP struct.
81
82The C<pattern> parameter is the scalar that was used as the
83pattern. previous versions of perl would pass two C<char*> indicating
a0e97681 84the start and end of the stringified pattern, the following snippet can
3ab4a224
AB
85be used to get the old parameters:
86
87 STRLEN plen;
88 char* exp = SvPV(pattern, plen);
89 char* xend = exp + plen;
90
91Since any scalar can be passed as a pattern it's possible to implement
92an engine that does something with an array (C<< "ook" =~ [ qw/ eek
93hlagh / ] >>) or with the non-stringified form of a compiled regular
94expression (C<< "ook" =~ qr/eek/ >>). perl's own engine will always
95stringify everything using the snippet above but that doesn't mean
96other engines have to.
108003db 97
a0e97681 98The C<flags> parameter is a bitfield which indicates which of the
c998b245
AB
99C<msixp> flags the regex was compiled with. It also contains
100additional info such as whether C<use locale> is in effect.
108003db
RGS
101
102The C<eogc> flags are stripped out before being passed to the comp
103routine. The regex engine does not need to know whether any of these
3ab4a224 104are set as those flags should only affect what perl does with the
c998b245
AB
105pattern and its match variables, not how it gets compiled and
106executed.
108003db 107
c998b245
AB
108By the time the comp callback is called, some of these flags have
109already had effect (noted below where applicable). However most of
110their effect occurs after the comp callback has run in routines that
111read the C<< rx->extflags >> field which it populates.
108003db 112
c998b245
AB
113In general the flags should be preserved in C<< rx->extflags >> after
114compilation, although the regex engine might want to add or delete
115some of them to invoke or disable some special behavior in perl. The
116flags along with any special behavior they cause are documented below:
108003db 117
c998b245 118The pattern modifiers:
108003db 119
c998b245 120=over 4
108003db 121
c998b245 122=item C</m> - RXf_PMf_MULTILINE
108003db 123
c998b245
AB
124If this is in C<< rx->extflags >> it will be passed to
125C<Perl_fbm_instr> by C<pp_split> which will treat the subject string
126as a multi-line string.
108003db 127
c998b245 128=item C</s> - RXf_PMf_SINGLELINE
108003db 129
c998b245 130=item C</i> - RXf_PMf_FOLD
108003db 131
c998b245 132=item C</x> - RXf_PMf_EXTENDED
108003db 133
c998b245
AB
134If present on a regex C<#> comments will be handled differently by the
135tokenizer in some cases.
108003db 136
c998b245 137TODO: Document those cases.
108003db 138
c998b245 139=item C</p> - RXf_PMf_KEEPCOPY
108003db 140
e72ec78c
KW
141TODO: Document this
142
f0f9b3b8
KW
143=item Character set
144
145The character set semantics are determined by an enum that is contained
146in this field. This is still experimental and subject to change, but
147the current interface returns the rules by use of the in-line function
148C<get_regex_charset(const U32 flags)>. The only currently documented
149value returned from it is REGEX_LOCALE_CHARSET, which is set if
e72ec78c
KW
150C<use locale> is in effect. If present in C<< rx->extflags >>,
151C<split> will use the locale dependent definition of whitespace
152when RXf_SKIPWHITE or RXf_WHITE is in effect. ASCII whitespace
96090e4f 153is defined as per L<isSPACE|perlapi/isSPACE>, and by the internal
e72ec78c 154macros C<is_utf8_space> under UTF-8, and C<isSPACE_LC> under C<use
c998b245 155locale>.
108003db 156
f0f9b3b8
KW
157=back
158
159Additional flags:
160
161=over 4
162
0ac6acae
AB
163=item RXf_SPLIT
164
165If C<split> is invoked as C<split ' '> or with no arguments (which
5137fa37 166really means C<split(' ', $_)>, see L<split|perlfunc/split>), perl will
0ac6acae
AB
167set this flag. The regex engine can then check for it and set the
168SKIPWHITE and WHITE extflags. To do this the perl engine does:
169
170 if (flags & RXf_SPLIT && r->prelen == 1 && r->precomp[0] == ' ')
171 r->extflags |= (RXf_SKIPWHITE|RXf_WHITE);
172
108003db
RGS
173=back
174
c998b245
AB
175These flags can be set during compilation to enable optimizations in
176the C<split> operator.
177
178=over 4
179
0ac6acae
AB
180=item RXf_SKIPWHITE
181
182If the flag is present in C<< rx->extflags >> C<split> will delete
183whitespace from the start of the subject string before it's operated
184on. What is considered whitespace depends on whether the subject is a
185UTF-8 string and whether the C<RXf_PMf_LOCALE> flag is set.
186
187If RXf_WHITE is set in addition to this flag C<split> will behave like
188C<split " "> under the perl engine.
189
c998b245
AB
190=item RXf_START_ONLY
191
192Tells the split operator to split the target string on newlines
193(C<\n>) without invoking the regex engine.
194
195Perl's engine sets this if the pattern is C</^/> (C<plen == 1 && *exp
196== '^'>), even under C</^/s>, see L<split|perlfunc>. Of course a
197different regex engine might want to use the same optimizations
198with a different syntax.
199
200=item RXf_WHITE
201
202Tells the split operator to split the target string on whitespace
203without invoking the regex engine. The definition of whitespace varies
204depending on whether the target string is a UTF-8 string and on
205whether RXf_PMf_LOCALE is set.
206
0ac6acae 207Perl's engine sets this flag if the pattern is C<\s+>.
c998b245 208
640f820d
AB
209=item RXf_NULL
210
a0e97681 211Tells the split operator to split the target string on
640f820d
AB
212characters. The definition of character varies depending on whether
213the target string is a UTF-8 string.
214
215Perl's engine sets this flag on empty patterns, this optimization
a0e97681 216makes C<split //> much faster than it would otherwise be. It's even
640f820d
AB
217faster than C<unpack>.
218
c998b245 219=back
108003db
RGS
220
221=head2 exec
222
49d7dfbc 223 I32 exec(pTHX_ REGEXP * const rx,
108003db
RGS
224 char *stringarg, char* strend, char* strbeg,
225 I32 minend, SV* screamer,
226 void* data, U32 flags);
227
8fd1a950
DM
228Execute a regexp. The arguments are
229
230=over 4
231
232=item rx
233
234The regular expression to execute.
235
236=item screamer
237
238This strangely-named arg is the SV to be matched against. Note that the
239actual char array to be matched against is supplied by the arguments
240described below; the SV is just used to determine UTF8ness, C<pos()> etc.
241
242=item strbeg
243
244Pointer to the physical start of the string.
245
246=item strend
247
248Pointer to the character following the physical end of the string (i.e.
249the \0).
250
251=item stringarg
252
253Pointer to the position in the string where matching should start; it might
254not be equal to C<strbeg> (for example in a later iteration of C</.../g>).
255
256=item minend
257
258Minimum length of string (measured in bytes from C<stringarg>) that must
259match; if the engine reaches the end of the match but hasn't reached this
260position in the string, it should fail.
261
262=item data
263
264Optimisation data; subject to change.
265
266=item flags
267
268Optimisation flags; subject to change.
269
270=back
108003db
RGS
271
272=head2 intuit
273
49d7dfbc 274 char* intuit(pTHX_ REGEXP * const rx,
108003db 275 SV *sv, char *strpos, char *strend,
49d7dfbc 276 const U32 flags, struct re_scream_pos_data_s *data);
108003db
RGS
277
278Find the start position where a regex match should be attempted,
279or possibly whether the regex engine should not be run because the
280pattern can't match. This is called as appropriate by the core
281depending on the values of the extflags member of the regexp
282structure.
283
284=head2 checkstr
285
49d7dfbc 286 SV* checkstr(pTHX_ REGEXP * const rx);
108003db
RGS
287
288Return a SV containing a string that must appear in the pattern. Used
289by C<split> for optimising matches.
290
291=head2 free
292
49d7dfbc 293 void free(pTHX_ REGEXP * const rx);
108003db
RGS
294
295Called by perl when it is freeing a regexp pattern so that the engine
296can release any resources pointed to by the C<pprivate> member of the
297regexp structure. This is only responsible for freeing private data;
298perl will handle releasing anything else contained in the regexp structure.
299
192b9cd1 300=head2 Numbered capture callbacks
108003db 301
192b9cd1
AB
302Called to get/set the value of C<$`>, C<$'>, C<$&> and their named
303equivalents, ${^PREMATCH}, ${^POSTMATCH} and $^{MATCH}, as well as the
c27a5cfe 304numbered capture groups (C<$1>, C<$2>, ...).
49d7dfbc 305
c149d39e
DM
306The C<paren> parameter will be C<1> for C<$1>, C<2> for C<$2> and so
307forth, and have these symbolic values for the special variables:
308
309 ${^PREMATCH} RX_BUFF_IDX_CARET_PREMATCH
310 ${^POSTMATCH} RX_BUFF_IDX_CARET_POSTMATCH
311 ${^MATCH} RX_BUFF_IDX_CARET_FULLMATCH
312 $` RX_BUFF_IDX_PREMATCH
313 $' RX_BUFF_IDX_POSTMATCH
314 $& RX_BUFF_IDX_FULLMATCH
315
316Note that in perl 5.17.3 and earlier, the last three constants were also
317used for the caret variants of the variables.
318
49d7dfbc 319
192b9cd1
AB
320The names have been chosen by analogy with L<Tie::Scalar> methods
321names with an additional B<LENGTH> callback for efficiency. However
322named capture variables are currently not tied internally but
323implemented via magic.
324
325=head3 numbered_buff_FETCH
326
327 void numbered_buff_FETCH(pTHX_ REGEXP * const rx, const I32 paren,
328 SV * const sv);
329
330Fetch a specified numbered capture. C<sv> should be set to the scalar
331to return, the scalar is passed as an argument rather than being
332returned from the function because when it's called perl already has a
333scalar to store the value, creating another one would be
334redundant. The scalar can be set with C<sv_setsv>, C<sv_setpvn> and
335friends, see L<perlapi>.
49d7dfbc
AB
336
337This callback is where perl untaints its own capture variables under
c998b245 338taint mode (see L<perlsec>). See the C<Perl_reg_numbered_buff_fetch>
49d7dfbc
AB
339function in F<regcomp.c> for how to untaint capture variables if
340that's something you'd like your engine to do as well.
108003db 341
192b9cd1 342=head3 numbered_buff_STORE
108003db 343
02c01adb
KW
344 void (*numbered_buff_STORE) (pTHX_
345 REGEXP * const rx,
346 const I32 paren,
2fdbfb4d 347 SV const * const value);
108003db 348
192b9cd1
AB
349Set the value of a numbered capture variable. C<value> is the scalar
350that is to be used as the new value. It's up to the engine to make
351sure this is used as the new value (or reject it).
2fdbfb4d
AB
352
353Example:
354
355 if ("ook" =~ /(o*)/) {
ccf3535a 356 # 'paren' will be '1' and 'value' will be 'ee'
2fdbfb4d
AB
357 $1 =~ tr/o/e/;
358 }
359
360Perl's own engine will croak on any attempt to modify the capture
a0e97681 361variables, to do this in another engine use the following callback
2fdbfb4d
AB
362(copied from C<Perl_reg_numbered_buff_store>):
363
364 void
02c01adb
KW
365 Example_reg_numbered_buff_store(pTHX_
366 REGEXP * const rx,
367 const I32 paren,
368 SV const * const value)
2fdbfb4d
AB
369 {
370 PERL_UNUSED_ARG(rx);
371 PERL_UNUSED_ARG(paren);
372 PERL_UNUSED_ARG(value);
373
374 if (!PL_localizing)
375 Perl_croak(aTHX_ PL_no_modify);
376 }
377
99d59c4d 378Actually perl will not I<always> croak in a statement that looks
2fdbfb4d
AB
379like it would modify a numbered capture variable. This is because the
380STORE callback will not be called if perl can determine that it
381doesn't have to modify the value. This is exactly how tied variables
382behave in the same situation:
383
384 package CaptureVar;
385 use base 'Tie::Scalar';
386
387 sub TIESCALAR { bless [] }
388 sub FETCH { undef }
389 sub STORE { die "This doesn't get called" }
390
391 package main;
392
c69ca1d4 393 tie my $sv => "CaptureVar";
2fdbfb4d
AB
394 $sv =~ y/a/b/;
395
396Because C<$sv> is C<undef> when the C<y///> operator is applied to it
397the transliteration won't actually execute and the program won't
192b9cd1
AB
398C<die>. This is different to how 5.8 and earlier versions behaved
399since the capture variables were READONLY variables then, now they'll
400just die when assigned to in the default engine.
2fdbfb4d 401
192b9cd1 402=head3 numbered_buff_LENGTH
2fdbfb4d 403
02c01adb
KW
404 I32 numbered_buff_LENGTH (pTHX_
405 REGEXP * const rx,
406 const SV * const sv,
2fdbfb4d
AB
407 const I32 paren);
408
409Get the C<length> of a capture variable. There's a special callback
410for this so that perl doesn't have to do a FETCH and run C<length> on
192b9cd1 411the result, since the length is (in perl's case) known from an offset
0a3a8dc0 412stored in C<< rx->offs >> this is much more efficient:
2fdbfb4d
AB
413
414 I32 s1 = rx->offs[paren].start;
415 I32 s2 = rx->offs[paren].end;
416 I32 len = t1 - s1;
417
418This is a little bit more complex in the case of UTF-8, see what
419C<Perl_reg_numbered_buff_length> does with
420L<is_utf8_string_loclen|perlapi/is_utf8_string_loclen>.
421
192b9cd1
AB
422=head2 Named capture callbacks
423
424Called to get/set the value of C<%+> and C<%-> as well as by some
425utility functions in L<re>.
426
427There are two callbacks, C<named_buff> is called in all the cases the
428FETCH, STORE, DELETE, CLEAR, EXISTS and SCALAR L<Tie::Hash> callbacks
429would be on changes to C<%+> and C<%-> and C<named_buff_iter> in the
430same cases as FIRSTKEY and NEXTKEY.
431
432The C<flags> parameter can be used to determine which of these
433operations the callbacks should respond to, the following flags are
434currently defined:
435
436Which L<Tie::Hash> operation is being performed from the Perl level on
437C<%+> or C<%+>, if any:
438
f1b875a0
YO
439 RXapif_FETCH
440 RXapif_STORE
441 RXapif_DELETE
442 RXapif_CLEAR
443 RXapif_EXISTS
444 RXapif_SCALAR
445 RXapif_FIRSTKEY
446 RXapif_NEXTKEY
192b9cd1
AB
447
448Whether C<%+> or C<%-> is being operated on, if any.
2fdbfb4d 449
f1b875a0
YO
450 RXapif_ONE /* %+ */
451 RXapif_ALL /* %- */
2fdbfb4d 452
192b9cd1 453Whether this is being called as C<re::regname>, C<re::regnames> or
c998b245 454C<re::regnames_count>, if any. The first two will be combined with
f1b875a0 455C<RXapif_ONE> or C<RXapif_ALL>.
192b9cd1 456
f1b875a0
YO
457 RXapif_REGNAME
458 RXapif_REGNAMES
459 RXapif_REGNAMES_COUNT
192b9cd1
AB
460
461Internally C<%+> and C<%-> are implemented with a real tied interface
462via L<Tie::Hash::NamedCapture>. The methods in that package will call
463back into these functions. However the usage of
464L<Tie::Hash::NamedCapture> for this purpose might change in future
465releases. For instance this might be implemented by magic instead
466(would need an extension to mgvtbl).
467
468=head3 named_buff
469
470 SV* (*named_buff) (pTHX_ REGEXP * const rx, SV * const key,
471 SV * const value, U32 flags);
472
473=head3 named_buff_iter
474
02c01adb
KW
475 SV* (*named_buff_iter) (pTHX_
476 REGEXP * const rx,
477 const SV * const lastkey,
192b9cd1 478 const U32 flags);
108003db 479
49d7dfbc 480=head2 qr_package
108003db 481
49d7dfbc 482 SV* qr_package(pTHX_ REGEXP * const rx);
108003db
RGS
483
484The package the qr// magic object is blessed into (as seen by C<ref
49d7dfbc
AB
485qr//>). It is recommended that engines change this to their package
486name for identification regardless of whether they implement methods
487on the object.
488
192b9cd1 489The package this method returns should also have the internal
d5213412 490C<Regexp> package in its C<@ISA>. C<< qr//->isa("Regexp") >> should always
192b9cd1
AB
491be true regardless of what engine is being used.
492
493Example implementation might be:
108003db
RGS
494
495 SV*
192b9cd1 496 Example_qr_package(pTHX_ REGEXP * const rx)
108003db
RGS
497 {
498 PERL_UNUSED_ARG(rx);
499 return newSVpvs("re::engine::Example");
500 }
501
502Any method calls on an object created with C<qr//> will be dispatched to the
503package as a normal object.
504
505 use re::engine::Example;
506 my $re = qr//;
507 $re->meth; # dispatched to re::engine::Example::meth()
508
f7e71195
AB
509To retrieve the C<REGEXP> object from the scalar in an XS function use
510the C<SvRX> macro, see L<"REGEXP Functions" in perlapi|perlapi/REGEXP
511Functions>.
108003db
RGS
512
513 void meth(SV * rv)
514 PPCODE:
f7e71195 515 REGEXP * re = SvRX(sv);
108003db 516
108003db
RGS
517=head2 dupe
518
49d7dfbc 519 void* dupe(pTHX_ REGEXP * const rx, CLONE_PARAMS *param);
108003db
RGS
520
521On threaded builds a regexp may need to be duplicated so that the pattern
a0e97681 522can be used by multiple threads. This routine is expected to handle the
108003db
RGS
523duplication of any private data pointed to by the C<pprivate> member of
524the regexp structure. It will be called with the preconstructed new
525regexp structure as an argument, the C<pprivate> member will point at
a0e97681 526the B<old> private structure, and it is this routine's responsibility to
108003db
RGS
527construct a copy and return a pointer to it (which perl will then use to
528overwrite the field as passed to this routine.)
529
530This allows the engine to dupe its private data but also if necessary
531modify the final structure if it really must.
532
533On unthreaded builds this field doesn't exist.
534
3c13cae6
DM
535=head2 op_comp
536
537This is private to the perl core and subject to change. Should be left
538null.
539
108003db
RGS
540=head1 The REGEXP structure
541
542The REGEXP struct is defined in F<regexp.h>. All regex engines must be able to
543correctly build such a structure in their L</comp> routine.
544
545The REGEXP structure contains all the data that perl needs to be aware of
546to properly work with the regular expression. It includes data about
547optimisations that perl can use to determine if the regex engine should
548really be used, and various other control info that is needed to properly
549execute patterns in various contexts such as is the pattern anchored in
550some way, or what flags were used during the compile, or whether the
551program contains special constructs that perl needs to be aware of.
552
882227b7
AB
553In addition it contains two fields that are intended for the private
554use of the regex engine that compiled the pattern. These are the
555C<intflags> and C<pprivate> members. C<pprivate> is a void pointer to
556an arbitrary structure whose use and management is the responsibility
557of the compiling engine. perl will never modify either of these
558values.
108003db
RGS
559
560 typedef struct regexp {
561 /* what engine created this regexp? */
562 const struct regexp_engine* engine;
563
564 /* what re is this a lightweight copy of? */
565 struct regexp* mother_re;
566
02c01adb
KW
567 /* Information about the match that the perl core uses to manage
568 * things */
108003db 569 U32 extflags; /* Flags used both externally and internally */
2d608413
KW
570 I32 minlen; /* mininum possible number of chars in */
571 string to match */
572 I32 minlenret; /* mininum possible number of chars in $& */
108003db
RGS
573 U32 gofs; /* chars left of pos that we search from */
574
575 /* substring data about strings that must appear
576 in the final match, used for optimisations */
577 struct reg_substr_data *substrs;
578
c27a5cfe 579 U32 nparens; /* number of capture groups */
108003db
RGS
580
581 /* private engine specific data */
582 U32 intflags; /* Engine Specific Internal flags */
583 void *pprivate; /* Data private to the regex engine which
584 created this object. */
585
02c01adb
KW
586 /* Data about the last/current match. These are modified during
587 * matching*/
82f14494
DM
588 U32 lastparen; /* highest close paren matched ($+) */
589 U32 lastcloseparen; /* last close paren matched ($^N) */
108003db 590 regexp_paren_pair *swap; /* Swap copy of *offs */
02c01adb
KW
591 regexp_paren_pair *offs; /* Array of offsets for (@-) and
592 (@+) */
108003db 593
02c01adb
KW
594 char *subbeg; /* saved or original string so \digit works
595 forever. */
108003db
RGS
596 SV_SAVED_COPY /* If non-NULL, SV which is COW from original */
597 I32 sublen; /* Length of string pointed by subbeg */
02c01adb
KW
598 I32 suboffset; /* byte offset of subbeg from logical start of
599 str */
6502e081 600 I32 subcoffset; /* suboffset equiv, but in chars (for @-/@+) */
108003db
RGS
601
602 /* Information about the match that isn't often used */
603 I32 prelen; /* length of precomp */
604 const char *precomp; /* pre-compilation regular expression */
605
108003db
RGS
606 char *wrapped; /* wrapped version of the pattern */
607 I32 wraplen; /* length of wrapped */
608
02c01adb
KW
609 I32 seen_evals; /* number of eval groups in the pattern - for
610 security checks */
108003db
RGS
611 HV *paren_names; /* Optional hash of paren names */
612
613 /* Refcount of this regexp */
614 I32 refcnt; /* Refcount of this regexp */
615 } regexp;
616
617The fields are discussed in more detail below:
618
882227b7 619=head2 C<engine>
108003db
RGS
620
621This field points at a regexp_engine structure which contains pointers
622to the subroutines that are to be used for performing a match. It
623is the compiling routine's responsibility to populate this field before
624returning the regexp object.
625
626Internally this is set to C<NULL> unless a custom engine is specified in
627C<$^H{regcomp}>, perl's own set of callbacks can be accessed in the struct
628pointed to by C<RE_ENGINE_PTR>.
629
882227b7 630=head2 C<mother_re>
108003db
RGS
631
632TODO, see L<http://www.mail-archive.com/perl5-changes@perl.org/msg17328.html>
633
882227b7 634=head2 C<extflags>
108003db 635
192b9cd1
AB
636This will be used by perl to see what flags the regexp was compiled
637with, this will normally be set to the value of the flags parameter by
c998b245
AB
638the L<comp|/comp> callback. See the L<comp|/comp> documentation for
639valid flags.
108003db 640
882227b7 641=head2 C<minlen> C<minlenret>
108003db 642
2d608413
KW
643The minimum string length (in characters) required for the pattern to match.
644This is used to
108003db
RGS
645prune the search space by not bothering to match any closer to the end of a
646string than would allow a match. For instance there is no point in even
647starting the regex engine if the minlen is 10 but the string is only 5
648characters long. There is no way that the pattern can match.
649
2d608413
KW
650C<minlenret> is the minimum length (in characters) of the string that would
651be found in $& after a match.
108003db
RGS
652
653The difference between C<minlen> and C<minlenret> can be seen in the
654following pattern:
655
656 /ns(?=\d)/
657
658where the C<minlen> would be 3 but C<minlenret> would only be 2 as the \d is
659required to match but is not actually included in the matched content. This
660distinction is particularly important as the substitution logic uses the
a0e97681 661C<minlenret> to tell whether it can do in-place substitution which can result in
108003db
RGS
662considerable speedup.
663
882227b7 664=head2 C<gofs>
108003db
RGS
665
666Left offset from pos() to start match at.
667
882227b7 668=head2 C<substrs>
108003db 669
192b9cd1
AB
670Substring data about strings that must appear in the final match. This
671is currently only used internally by perl's engine for but might be
c998b245 672used in the future for all engines for optimisations.
108003db 673
1cecf2c0 674=head2 C<nparens>, C<lastparen>, and C<lastcloseparen>
108003db
RGS
675
676These fields are used to keep track of how many paren groups could be matched
677in the pattern, which was the last open paren to be entered, and which was
678the last close paren to be entered.
679
882227b7 680=head2 C<intflags>
108003db
RGS
681
682The engine's private copy of the flags the pattern was compiled with. Usually
192b9cd1 683this is the same as C<extflags> unless the engine chose to modify one of them.
108003db 684
882227b7 685=head2 C<pprivate>
108003db
RGS
686
687A void* pointing to an engine-defined data structure. The perl engine uses the
688C<regexp_internal> structure (see L<perlreguts/Base Structures>) but a custom
689engine should use something else.
690
882227b7 691=head2 C<swap>
108003db 692
e9105d30 693Unused. Left in for compatibility with perl 5.10.0.
108003db 694
882227b7 695=head2 C<offs>
108003db
RGS
696
697A C<regexp_paren_pair> structure which defines offsets into the string being
698matched which correspond to the C<$&> and C<$1>, C<$2> etc. captures, the
699C<regexp_paren_pair> struct is defined as follows:
700
701 typedef struct regexp_paren_pair {
702 I32 start;
703 I32 end;
704 } regexp_paren_pair;
705
706If C<< ->offs[num].start >> or C<< ->offs[num].end >> is C<-1> then that
c27a5cfe 707capture group did not match. C<< ->offs[0].start/end >> represents C<$&> (or
c149d39e 708C<${^MATCH}> under C<//p>) and C<< ->offs[paren].end >> matches C<$$paren> where
108003db
RGS
709C<$paren >= 1>.
710
882227b7 711=head2 C<precomp> C<prelen>
108003db 712
192b9cd1
AB
713Used for optimisations. C<precomp> holds a copy of the pattern that
714was compiled and C<prelen> its length. When a new pattern is to be
715compiled (such as inside a loop) the internal C<regcomp> operator
716checks whether the last compiled C<REGEXP>'s C<precomp> and C<prelen>
717are equivalent to the new one, and if so uses the old pattern instead
718of compiling a new one.
719
720The relevant snippet from C<Perl_pp_regcomp>:
721
722 if (!re || !re->precomp || re->prelen != (I32)len ||
723 memNE(re->precomp, t, len))
724 /* Compile a new pattern */
108003db 725
882227b7 726=head2 C<paren_names>
108003db 727
c27a5cfe 728This is a hash used internally to track named capture groups and their
108003db
RGS
729offsets. The keys are the names of the buffers the values are dualvars,
730with the IV slot holding the number of buffers with the given name and the
731pv being an embedded array of I32. The values may also be contained
732independently in the data array in cases where named backreferences are
733used.
734
c998b245 735=head2 C<substrs>
108003db
RGS
736
737Holds information on the longest string that must occur at a fixed
738offset from the start of the pattern, and the longest string that must
739occur at a floating offset from the start of the pattern. Used to do
740Fast-Boyer-Moore searches on the string to find out if its worth using
741the regex engine at all, and if so where in the string to search.
742
6502e081
DM
743=head2 C<subbeg> C<sublen> C<saved_copy> C<suboffset> C<subcoffset>
744
745Used during the execution phase for managing search and replace patterns,
746and for providing the text for C<$&>, C<$1> etc. C<subbeg> points to a
747buffer (either the original string, or a copy in the case of
748C<RX_MATCH_COPIED(rx)>), and C<sublen> is the length of the buffer. The
749C<RX_OFFS> start and end indices index into this buffer.
750
751In the presence of the C<REXEC_COPY_STR> flag, but with the addition of
752the C<REXEC_COPY_SKIP_PRE> or C<REXEC_COPY_SKIP_POST> flags, an engine
753can choose not to copy the full buffer (although it must still do so in
754the presence of C<RXf_PMf_KEEPCOPY> or the relevant bits being set in
755C<PL_sawampersand>). In this case, it may set C<suboffset> to indicate the
756number of bytes from the logical start of the buffer to the physical start
757(i.e. C<subbeg>). It should also set C<subcoffset>, the number of
758characters in the offset. The latter is needed to support C<@-> and C<@+>
759which work in characters, not bytes.
108003db 760
882227b7 761=head2 C<wrapped> C<wraplen>
108003db 762
c998b245 763Stores the string C<qr//> stringifies to. The perl engine for example
ed215d3c 764stores C<(?^:eek)> in the case of C<qr/eek/>.
108003db 765
c998b245
AB
766When using a custom engine that doesn't support the C<(?:)> construct
767for inline modifiers, it's probably best to have C<qr//> stringify to
768the supplied pattern, note that this will create undesired patterns in
769cases such as:
108003db
RGS
770
771 my $x = qr/a|b/; # "a|b"
192b9cd1 772 my $y = qr/c/i; # "c"
108003db
RGS
773 my $z = qr/$x$y/; # "a|bc"
774
192b9cd1
AB
775There's no solution for this problem other than making the custom
776engine understand a construct like C<(?:)>.
108003db 777
882227b7 778=head2 C<seen_evals>
108003db
RGS
779
780This stores the number of eval groups in the pattern. This is used for security
781purposes when embedding compiled regexes into larger patterns with C<qr//>.
782
882227b7 783=head2 C<refcnt>
108003db
RGS
784
785The number of times the structure is referenced. When this falls to 0 the
786regexp is automatically freed by a call to pregfree. This should be set to 1 in
787each engine's L</comp> routine.
788
108003db
RGS
789=head1 HISTORY
790
791Originally part of L<perlreguts>.
792
793=head1 AUTHORS
794
795Originally written by Yves Orton, expanded by E<AElig>var ArnfjE<ouml>rE<eth>
796Bjarmason.
797
798=head1 LICENSE
799
800Copyright 2006 Yves Orton and 2007 E<AElig>var ArnfjE<ouml>rE<eth> Bjarmason.
801
802This program is free software; you can redistribute it and/or modify it under
803the same terms as Perl itself.
804
805=cut