This is a live mirror of the Perl 5 development currently hosted at https://github.com/perl/perl5
perldelta for 9407f9c1
[perl5.git] / pod / perlre.pod
CommitLineData
a0d0e21e 1=head1 NAME
d74e8afc 2X<regular expression> X<regex> X<regexp>
a0d0e21e
LW
3
4perlre - Perl regular expressions
5
6=head1 DESCRIPTION
7
5d458dd8 8This page describes the syntax of regular expressions in Perl.
91e0c79e 9
cc46d5f2 10If you haven't used regular expressions before, a quick-start
91e0c79e
MJD
11introduction is available in L<perlrequick>, and a longer tutorial
12introduction is available in L<perlretut>.
13
14For reference on how regular expressions are used in matching
15operations, plus various examples of the same, see discussions of
16C<m//>, C<s///>, C<qr//> and C<??> in L<perlop/"Regexp Quote-Like
17Operators">.
cb1a09d0 18
0d017f4d
WL
19
20=head2 Modifiers
21
19799a22 22Matching operations can have various modifiers. Modifiers
5a964f20 23that relate to the interpretation of the regular expression inside
19799a22 24are listed below. Modifiers that alter the way a regular expression
5d458dd8 25is used by Perl are detailed in L<perlop/"Regexp Quote-Like Operators"> and
1e66bd83 26L<perlop/"Gory details of parsing quoted constructs">.
a0d0e21e 27
55497cff
PP
28=over 4
29
54310121 30=item m
d74e8afc 31X</m> X<regex, multiline> X<regexp, multiline> X<regular expression, multiline>
55497cff
PP
32
33Treat string as multiple lines. That is, change "^" and "$" from matching
14218588 34the start or end of the string to matching the start or end of any
7f761169 35line anywhere within the string.
55497cff 36
54310121 37=item s
d74e8afc
ITB
38X</s> X<regex, single-line> X<regexp, single-line>
39X<regular expression, single-line>
55497cff
PP
40
41Treat string as single line. That is, change "." to match any character
19799a22 42whatsoever, even a newline, which normally it would not match.
55497cff 43
34d67d80 44Used together, as C</ms>, they let the "." match any character whatsoever,
fb55449c 45while still allowing "^" and "$" to match, respectively, just after
19799a22 46and just before newlines within the string.
7b8d334a 47
87e95b7f
YO
48=item i
49X</i> X<regex, case-insensitive> X<regexp, case-insensitive>
50X<regular expression, case-insensitive>
51
52Do case-insensitive pattern matching.
53
54If C<use locale> is in effect, the case map is taken from the current
17580e7a
KW
55locale for code points less than 255, and from Unicode rules for larger
56code points. See L<perllocale>.
87e95b7f 57
54310121 58=item x
d74e8afc 59X</x>
55497cff
PP
60
61Extend your pattern's legibility by permitting whitespace and comments.
62
87e95b7f
YO
63=item p
64X</p> X<regex, preserve> X<regexp, preserve>
65
632a1772 66Preserve the string matched such that ${^PREMATCH}, ${^MATCH}, and
87e95b7f
YO
67${^POSTMATCH} are available for use after matching.
68
e2e6bec7
DN
69=item g and c
70X</g> X</c>
71
72Global matching, and keep the Current position after failed matching.
73Unlike i, m, s and x, these two flags affect the way the regex is used
74rather than the regex itself. See
75L<perlretut/"Using regular expressions in Perl"> for further explanation
76of the g and c modifiers.
77
b6fa137b
FC
78=item a, d, l and u
79X</a> X</d> X</l> X</u>
80
81These modifiers, new in 5.14, affect which character-set semantics
82(Unicode, ASCII, etc.) are used, as described below.
83
55497cff 84=back
a0d0e21e
LW
85
86These are usually written as "the C</x> modifier", even though the delimiter
b6fa137b 87in question might not really be a slash. The modifiers C</imsxadlup>
ab7bb42d 88may also be embedded within the regular expression itself using
b6fa137b
FC
89the C<(?...)> construct.
90
91The C</x>, C</l>, C</u>, C</a> and C</d> modifiers need a little more
92explanation.
a0d0e21e 93
b6fa137b 94C</x> tells
7b059540 95the regular expression parser to ignore most whitespace that is neither
55497cff 96backslashed nor within a character class. You can use this to break up
4633a7c4 97your regular expression into (slightly) more readable parts. The C<#>
54310121 98character is also treated as a metacharacter introducing a comment,
55497cff 99just as in ordinary Perl code. This also means that if you want real
14218588 100whitespace or C<#> characters in the pattern (outside a character
f9a3ff1a 101class, where they are unaffected by C</x>), then you'll either have to
7b059540
KW
102escape them (using backslashes or C<\Q...\E>) or encode them using octal,
103hex, or C<\N{}> escapes. Taken together, these features go a long way towards
8933a740
RGS
104making Perl's regular expressions more readable. Note that you have to
105be careful not to include the pattern delimiter in the comment--perl has
106no way of knowing you did not intend to close the pattern early. See
107the C-comment deletion code in L<perlop>. Also note that anything inside
7651b971
KW
108a C<\Q...\E> stays unaffected by C</x>. And note that C</x> doesn't affect
109whether space interpretation within a single multi-character construct. For
110example in C<\x{...}>, regardless of the C</x> modifier, there can be no
9bb1f947 111spaces. Same for a L<quantifier|/Quantifiers> such as C<{3}> or
f9e949fd
KW
112C<{5,}>. Similarly, C<(?:...)> can't have a space between the C<?> and C<:>,
113but can between the C<(> and C<?>. Within any delimiters for such a
114construct, allowed spaces are not affected by C</x>, and depend on the
115construct. For example, C<\x{...}> can't have spaces because hexadecimal
116numbers don't have spaces in them. But, Unicode properties can have spaces, so
117in C<\p{...}> there can be spaces that follow the Unicode rules, for which see
9bb1f947 118L<perluniprops/Properties accessible through \p{} and \P{}>.
d74e8afc 119X</x>
a0d0e21e 120
b6fa137b
FC
121C</l> means to use a locale (see L<perllocale>) when pattern matching.
122The locale used will be the one in effect at the time of execution of
123the pattern match. This may not be the same as the compilation-time
124locale, and can differ from one match to another if there is an
125intervening call of the
126L<setlocale() function|perllocale/The setlocale function>.
127This modifier is automatically set if the regular expression is compiled
128within the scope of a C<"use locale"> pragma.
129Perl only allows single-byte locales. This means that code points above
130255 are treated as Unicode no matter what locale is in effect.
131Under Unicode rules, there are a few case-insensitive matches that cross the
132boundary 255/256 boundary. These are disallowed. For example,
1330xFF does not caselessly match the character at 0x178, LATIN CAPITAL
134LETTER Y WITH DIAERESIS, because 0xFF may not be LATIN SMALL LETTER Y
135in the current locale, and Perl has no way of knowing if that character
136even exists in the locale, much less what code point it is.
137X</l>
138
139C</u> means to use Unicode semantics when pattern matching. It is
140automatically set if the regular expression is encoded in utf8, or is
141compiled within the scope of a
142L<C<"use feature 'unicode_strings">|feature> pragma (and isn't also in
143the scope of L<C<"use locale">|locale> nor L<C<"use bytes">|bytes>
144pragmas). On ASCII platforms, the code points between 128 and 255 take on their
145Latin-1 (ISO-8859-1) meanings (which are the same as Unicode's), whereas
146in strict ASCII their meanings are undefined. Thus the platform
147effectively becomes a Unicode platform. The ASCII characters remain as
148ASCII characters (since ASCII is a subset of Latin-1 and Unicode). For
149example, when this option is not on, on a non-utf8 string, C<"\w">
150matches precisely C<[A-Za-z0-9_]>. When the option is on, it matches
151not just those, but all the Latin-1 word characters (such as an "n" with
152a tilde). On EBCDIC platforms, which already are equivalent to Latin-1,
153this modifier changes behavior only when the C<"/i"> modifier is also
154specified, and affects only two characters, giving them full Unicode
155semantics: the C<MICRO SIGN> will match the Greek capital and
156small letters C<MU>; otherwise not; and the C<LATIN CAPITAL LETTER SHARP
157S> will match any of C<SS>, C<Ss>, C<sS>, and C<ss>, otherwise not.
158(This last case is buggy, however.)
159X</u>
160
161C</a> is the same as C</u>, except that C<\d>, C<\s>, C<\w>, and the
162Posix character classes are restricted to matching in the ASCII range
163only. That is, with this modifier, C<\d> always means precisely the
164digits C<"0"> to C<"9">; C<\s> means the five characters C<[ \f\n\r\t]>;
165C<\w> means the 63 characters C<[A-Za-z0-9_]>; and likewise, all the
166Posix classes such as C<[[:print:]]> match only the appropriate
167ASCII-range characters. As you would expect, this modifier causes, for
168example, C<\D> to mean the same thing as C<[^0-9]>; in fact, all
169non-ASCII characters match C<\D>, C<\S>, and C<\W>. C<\b> still means
170to match at the boundary between C<\w> and C<\W>, using the C<"a">
171definitions of them (similarly for C<\B>). Otherwise, C<"a"> behaves
172like the C<"u"> modifier, in that case-insensitive matching uses Unicode
173semantics; for example, "k" will match the Unicode C<\N{KELVIN SIGN}>
174under C</i> matching, and code points in the Latin1 range, above ASCII
175will have Unicode semantics when it comes to case-insensitive matching.
176But writing two in "a"'s in a row will increase its effect, causing the
177Kelvin sign and all other non-ASCII characters to not match any ASCII
178character under C</i> matching.
179X</a>
180
181C</d> means to use the traditional Perl pattern-matching behavior.
182This is dualistic (hence the name C</d>, which also could stand for
183"depends"). When this is in effect, Perl matches according to the
184platform's native character set rules unless there is something that
185indicates to use Unicode rules. If either the target string or the
186pattern itself is encoded in UTF-8, Unicode rules are used. Also, if
187the pattern contains Unicode-only features, such as code points above
188255, C<\p()> Unicode properties or C<\N{}> Unicode names, Unicode rules
189will be used. It is automatically selected by default if the regular
190expression is compiled neither within the scope of a C<"use locale">
191pragma nor a <C<"use feature 'unicode_strings"> pragma.
192This behavior causes a number of glitches, see
193L<perlunicode/The "Unicode Bug">.
194X</d>
195
a0d0e21e
LW
196=head2 Regular Expressions
197
04838cea
RGS
198=head3 Metacharacters
199
384f06ae 200The patterns used in Perl pattern matching evolved from those supplied in
14218588 201the Version 8 regex routines. (The routines are derived
19799a22
GS
202(distantly) from Henry Spencer's freely redistributable reimplementation
203of the V8 routines.) See L<Version 8 Regular Expressions> for
204details.
a0d0e21e
LW
205
206In particular the following metacharacters have their standard I<egrep>-ish
207meanings:
d74e8afc
ITB
208X<metacharacter>
209X<\> X<^> X<.> X<$> X<|> X<(> X<()> X<[> X<[]>
210
a0d0e21e 211
f793d64a
KW
212 \ Quote the next metacharacter
213 ^ Match the beginning of the line
214 . Match any character (except newline)
215 $ Match the end of the line (or before newline at the end)
216 | Alternation
217 () Grouping
218 [] Bracketed Character class
a0d0e21e 219
14218588
GS
220By default, the "^" character is guaranteed to match only the
221beginning of the string, the "$" character only the end (or before the
222newline at the end), and Perl does certain optimizations with the
a0d0e21e
LW
223assumption that the string contains only one line. Embedded newlines
224will not be matched by "^" or "$". You may, however, wish to treat a
4a6725af 225string as a multi-line buffer, such that the "^" will match after any
0d520e8e
YO
226newline within the string (except if the newline is the last character in
227the string), and "$" will match before any newline. At the
a0d0e21e
LW
228cost of a little more overhead, you can do this by using the /m modifier
229on the pattern match operator. (Older programs did this by setting C<$*>,
f02c194e 230but this practice has been removed in perl 5.9.)
d74e8afc 231X<^> X<$> X</m>
a0d0e21e 232
14218588 233To simplify multi-line substitutions, the "." character never matches a
55497cff 234newline unless you use the C</s> modifier, which in effect tells Perl to pretend
f02c194e 235the string is a single line--even if it isn't.
d74e8afc 236X<.> X</s>
a0d0e21e 237
04838cea
RGS
238=head3 Quantifiers
239
a0d0e21e 240The following standard quantifiers are recognized:
d74e8afc 241X<metacharacter> X<quantifier> X<*> X<+> X<?> X<{n}> X<{n,}> X<{n,m}>
a0d0e21e 242
f793d64a
KW
243 * Match 0 or more times
244 + Match 1 or more times
245 ? Match 1 or 0 times
246 {n} Match exactly n times
247 {n,} Match at least n times
248 {n,m} Match at least n but not more than m times
a0d0e21e
LW
249
250(If a curly bracket occurs in any other context, it is treated
b975c076 251as a regular character. In particular, the lower bound
527e91da
BB
252is not optional.) The "*" quantifier is equivalent to C<{0,}>, the "+"
253quantifier to C<{1,}>, and the "?" quantifier to C<{0,1}>. n and m are limited
d0b16107 254to non-negative integral values less than a preset limit defined when perl is built.
9c79236d
GS
255This is usually 32766 on the most common platforms. The actual limit can
256be seen in the error message generated by code such as this:
257
820475bd 258 $_ **= $_ , / {$_} / for 2 .. 42;
a0d0e21e 259
54310121
PP
260By default, a quantified subpattern is "greedy", that is, it will match as
261many times as possible (given a particular starting location) while still
262allowing the rest of the pattern to match. If you want it to match the
263minimum number of times possible, follow the quantifier with a "?". Note
264that the meanings don't change, just the "greediness":
0d017f4d 265X<metacharacter> X<greedy> X<greediness>
d74e8afc 266X<?> X<*?> X<+?> X<??> X<{n}?> X<{n,}?> X<{n,m}?>
a0d0e21e 267
f793d64a
KW
268 *? Match 0 or more times, not greedily
269 +? Match 1 or more times, not greedily
270 ?? Match 0 or 1 time, not greedily
271 {n}? Match exactly n times, not greedily
272 {n,}? Match at least n times, not greedily
273 {n,m}? Match at least n but not more than m times, not greedily
a0d0e21e 274
b9b4dddf
YO
275By default, when a quantified subpattern does not allow the rest of the
276overall pattern to match, Perl will backtrack. However, this behaviour is
0d017f4d 277sometimes undesirable. Thus Perl provides the "possessive" quantifier form
b9b4dddf
YO
278as well.
279
f793d64a
KW
280 *+ Match 0 or more times and give nothing back
281 ++ Match 1 or more times and give nothing back
282 ?+ Match 0 or 1 time and give nothing back
283 {n}+ Match exactly n times and give nothing back (redundant)
284 {n,}+ Match at least n times and give nothing back
285 {n,m}+ Match at least n but not more than m times and give nothing back
b9b4dddf
YO
286
287For instance,
288
289 'aaaa' =~ /a++a/
290
291will never match, as the C<a++> will gobble up all the C<a>'s in the
292string and won't leave any for the remaining part of the pattern. This
293feature can be extremely useful to give perl hints about where it
294shouldn't backtrack. For instance, the typical "match a double-quoted
295string" problem can be most efficiently performed when written as:
296
297 /"(?:[^"\\]++|\\.)*+"/
298
0d017f4d 299as we know that if the final quote does not match, backtracking will not
b9b4dddf
YO
300help. See the independent subexpression C<< (?>...) >> for more details;
301possessive quantifiers are just syntactic sugar for that construct. For
302instance the above example could also be written as follows:
303
304 /"(?>(?:(?>[^"\\]+)|\\.)*)"/
305
04838cea
RGS
306=head3 Escape sequences
307
5f05dabc 308Because patterns are processed as double quoted strings, the following
a0d0e21e
LW
309also work:
310
f793d64a
KW
311 \t tab (HT, TAB)
312 \n newline (LF, NL)
313 \r return (CR)
314 \f form feed (FF)
315 \a alarm (bell) (BEL)
316 \e escape (think troff) (ESC)
f793d64a 317 \cK control char (example: VT)
dc0d9c48 318 \x{}, \x00 character whose ordinal is the given hexadecimal number
fb121860 319 \N{name} named Unicode character or character sequence
f793d64a 320 \N{U+263D} Unicode character (example: FIRST QUARTER MOON)
f0a2b745 321 \o{}, \000 character whose ordinal is the given octal number
f793d64a
KW
322 \l lowercase next char (think vi)
323 \u uppercase next char (think vi)
324 \L lowercase till \E (think vi)
325 \U uppercase till \E (think vi)
326 \Q quote (disable) pattern metacharacters till \E
327 \E end either case modification or quoted section, think vi
a0d0e21e 328
9bb1f947 329Details are in L<perlop/Quote and Quote-like Operators>.
1d2dff63 330
e1d1eefb 331=head3 Character Classes and other Special Escapes
04838cea 332
a0d0e21e 333In addition, Perl defines the following:
d0b16107 334X<\g> X<\k> X<\K> X<backreference>
a0d0e21e 335
f793d64a
KW
336 Sequence Note Description
337 [...] [1] Match a character according to the rules of the
338 bracketed character class defined by the "...".
339 Example: [a-z] matches "a" or "b" or "c" ... or "z"
340 [[:...:]] [2] Match a character according to the rules of the POSIX
341 character class "..." within the outer bracketed
342 character class. Example: [[:upper:]] matches any
343 uppercase character.
d35dd6c6
KW
344 \w [3] Match a "word" character (alphanumeric plus "_", plus
345 other connector punctuation chars plus Unicode
346 marks
f793d64a
KW
347 \W [3] Match a non-"word" character
348 \s [3] Match a whitespace character
349 \S [3] Match a non-whitespace character
350 \d [3] Match a decimal digit character
351 \D [3] Match a non-digit character
352 \pP [3] Match P, named property. Use \p{Prop} for longer names
353 \PP [3] Match non-P
354 \X [4] Match Unicode "eXtended grapheme cluster"
355 \C Match a single C-language char (octet) even if that is
356 part of a larger UTF-8 character. Thus it breaks up
357 characters into their UTF-8 bytes, so you may end up
358 with malformed pieces of UTF-8. Unsupported in
359 lookbehind.
c27a5cfe 360 \1 [5] Backreference to a specific capture group or buffer.
f793d64a
KW
361 '1' may actually be any positive integer.
362 \g1 [5] Backreference to a specific or previous group,
363 \g{-1} [5] The number may be negative indicating a relative
c27a5cfe 364 previous group and may optionally be wrapped in
f793d64a
KW
365 curly brackets for safer parsing.
366 \g{name} [5] Named backreference
367 \k<name> [5] Named backreference
368 \K [6] Keep the stuff left of the \K, don't include it in $&
369 \N [7] Any character but \n (experimental). Not affected by
370 /s modifier
371 \v [3] Vertical whitespace
372 \V [3] Not vertical whitespace
373 \h [3] Horizontal whitespace
374 \H [3] Not horizontal whitespace
375 \R [4] Linebreak
e1d1eefb 376
9bb1f947
KW
377=over 4
378
379=item [1]
380
381See L<perlrecharclass/Bracketed Character Classes> for details.
df225385 382
9bb1f947 383=item [2]
b8c5462f 384
9bb1f947 385See L<perlrecharclass/POSIX Character Classes> for details.
b8c5462f 386
9bb1f947 387=item [3]
5496314a 388
9bb1f947 389See L<perlrecharclass/Backslash sequences> for details.
5496314a 390
9bb1f947 391=item [4]
5496314a 392
9bb1f947 393See L<perlrebackslash/Misc> for details.
d0b16107 394
9bb1f947 395=item [5]
b8c5462f 396
c27a5cfe 397See L</Capture groups> below for details.
93733859 398
9bb1f947 399=item [6]
b8c5462f 400
9bb1f947
KW
401See L</Extended Patterns> below for details.
402
403=item [7]
404
405Note that C<\N> has two meanings. When of the form C<\N{NAME}>, it matches the
fb121860
KW
406character or character sequence whose name is C<NAME>; and similarly
407when of the form C<\N{U+I<hex>}>, it matches the character whose Unicode
408code point is I<hex>. Otherwise it matches any character but C<\n>.
9bb1f947
KW
409
410=back
d0b16107 411
04838cea
RGS
412=head3 Assertions
413
a0d0e21e 414Perl defines the following zero-width assertions:
d74e8afc
ITB
415X<zero-width assertion> X<assertion> X<regex, zero-width assertion>
416X<regexp, zero-width assertion>
417X<regular expression, zero-width assertion>
418X<\b> X<\B> X<\A> X<\Z> X<\z> X<\G>
a0d0e21e 419
9bb1f947
KW
420 \b Match a word boundary
421 \B Match except at a word boundary
422 \A Match only at beginning of string
423 \Z Match only at end of string, or before newline at the end
424 \z Match only at end of string
425 \G Match only at pos() (e.g. at the end-of-match position
9da458fc 426 of prior m//g)
a0d0e21e 427
14218588 428A word boundary (C<\b>) is a spot between two characters
19799a22
GS
429that has a C<\w> on one side of it and a C<\W> on the other side
430of it (in either order), counting the imaginary characters off the
431beginning and end of the string as matching a C<\W>. (Within
432character classes C<\b> represents backspace rather than a word
433boundary, just as it normally does in any double-quoted string.)
434The C<\A> and C<\Z> are just like "^" and "$", except that they
435won't match multiple times when the C</m> modifier is used, while
436"^" and "$" will match at every internal line boundary. To match
437the actual end of the string and not ignore an optional trailing
438newline, use C<\z>.
d74e8afc 439X<\b> X<\A> X<\Z> X<\z> X</m>
19799a22
GS
440
441The C<\G> assertion can be used to chain global matches (using
442C<m//g>), as described in L<perlop/"Regexp Quote-Like Operators">.
443It is also useful when writing C<lex>-like scanners, when you have
444several patterns that you want to match against consequent substrings
445of your string, see the previous reference. The actual location
446where C<\G> will match can also be influenced by using C<pos()> as
58e23c8d
YO
447an lvalue: see L<perlfunc/pos>. Note that the rule for zero-length
448matches is modified somewhat, in that contents to the left of C<\G> is
449not counted when determining the length of the match. Thus the following
450will not match forever:
d74e8afc 451X<\G>
c47ff5f1 452
e761bb84
CO
453 my $string = 'ABC';
454 pos($string) = 1;
455 while ($string =~ /(.\G)/g) {
456 print $1;
457 }
58e23c8d
YO
458
459It will print 'A' and then terminate, as it considers the match to
460be zero-width, and thus will not match at the same position twice in a
461row.
462
463It is worth noting that C<\G> improperly used can result in an infinite
464loop. Take care when using patterns that include C<\G> in an alternation.
465
c27a5cfe 466=head3 Capture groups
04838cea 467
c27a5cfe
KW
468The bracketing construct C<( ... )> creates capture groups (also referred to as
469capture buffers). To refer to the current contents of a group later on, within
d8b950dc
KW
470the same pattern, use C<\g1> (or C<\g{1}>) for the first, C<\g2> (or C<\g{2}>)
471for the second, and so on.
472This is called a I<backreference>.
d74e8afc 473X<regex, capture buffer> X<regexp, capture buffer>
c27a5cfe 474X<regex, capture group> X<regexp, capture group>
d74e8afc 475X<regular expression, capture buffer> X<backreference>
c27a5cfe 476X<regular expression, capture group> X<backreference>
1f1031fe 477X<\g{1}> X<\g{-1}> X<\g{name}> X<relative backreference> X<named backreference>
d8b950dc
KW
478X<named capture buffer> X<regular expression, named capture buffer>
479X<named capture group> X<regular expression, named capture group>
480X<%+> X<$+{name}> X<< \k<name> >>
481There is no limit to the number of captured substrings that you may use.
482Groups are numbered with the leftmost open parenthesis being number 1, etc. If
483a group did not match, the associated backreference won't match either. (This
484can happen if the group is optional, or in a different branch of an
485alternation.)
486You can omit the C<"g">, and write C<"\1">, etc, but there are some issues with
487this form, described below.
488
489You can also refer to capture groups relatively, by using a negative number, so
490that C<\g-1> and C<\g{-1}> both refer to the immediately preceding capture
491group, and C<\g-2> and C<\g{-2}> both refer to the group before it. For
492example:
5624f11d
YO
493
494 /
c27a5cfe
KW
495 (Y) # group 1
496 ( # group 2
497 (X) # group 3
498 \g{-1} # backref to group 3
499 \g{-3} # backref to group 1
5624f11d
YO
500 )
501 /x
502
d8b950dc
KW
503would match the same as C</(Y) ( (X) \g3 \g1 )/x>. This allows you to
504interpolate regexes into larger regexes and not have to worry about the
505capture groups being renumbered.
506
507You can dispense with numbers altogether and create named capture groups.
508The notation is C<(?E<lt>I<name>E<gt>...)> to declare and C<\g{I<name>}> to
509reference. (To be compatible with .Net regular expressions, C<\g{I<name>}> may
510also be written as C<\k{I<name>}>, C<\kE<lt>I<name>E<gt>> or C<\k'I<name>'>.)
511I<name> must not begin with a number, nor contain hyphens.
512When different groups within the same pattern have the same name, any reference
513to that name assumes the leftmost defined group. Named groups count in
514absolute and relative numbering, and so can also be referred to by those
515numbers.
516(It's possible to do things with named capture groups that would otherwise
517require C<(??{})>.)
518
519Capture group contents are dynamically scoped and available to you outside the
520pattern until the end of the enclosing block or until the next successful
521match, whichever comes first. (See L<perlsyn/"Compound Statements">.)
522You can refer to them by absolute number (using C<"$1"> instead of C<"\g1">,
523etc); or by name via the C<%+> hash, using C<"$+{I<name>}">.
524
525Braces are required in referring to named capture groups, but are optional for
526absolute or relative numbered ones. Braces are safer when creating a regex by
527concatenating smaller strings. For example if you have C<qr/$a$b/>, and C<$a>
528contained C<"\g1">, and C<$b> contained C<"37">, you would get C</\g137/> which
529is probably not what you intended.
530
531The C<\g> and C<\k> notations were introduced in Perl 5.10.0. Prior to that
532there were no named nor relative numbered capture groups. Absolute numbered
533groups were referred to using C<\1>, C<\2>, etc, and this notation is still
534accepted (and likely always will be). But it leads to some ambiguities if
535there are more than 9 capture groups, as C<\10> could mean either the tenth
536capture group, or the character whose ordinal in octal is 010 (a backspace in
537ASCII). Perl resolves this ambiguity by interpreting C<\10> as a backreference
538only if at least 10 left parentheses have opened before it. Likewise C<\11> is
539a backreference only if at least 11 left parentheses have opened before it.
e1f120a9
KW
540And so on. C<\1> through C<\9> are always interpreted as backreferences.
541There are several examples below that illustrate these perils. You can avoid
542the ambiguity by always using C<\g{}> or C<\g> if you mean capturing groups;
543and for octal constants always using C<\o{}>, or for C<\077> and below, using 3
544digits padded with leading zeros, since a leading zero implies an octal
545constant.
d8b950dc
KW
546
547The C<\I<digit>> notation also works in certain circumstances outside
548the pattern. See L</Warning on \1 Instead of $1> below for details.)
81714fb9 549
14218588 550Examples:
a0d0e21e
LW
551
552 s/^([^ ]*) *([^ ]*)/$2 $1/; # swap first two words
553
d8b950dc 554 /(.)\g1/ # find first doubled char
81714fb9
YO
555 and print "'$1' is the first doubled character\n";
556
557 /(?<char>.)\k<char>/ # ... a different way
558 and print "'$+{char}' is the first doubled character\n";
559
d8b950dc 560 /(?'char'.)\g1/ # ... mix and match
81714fb9 561 and print "'$1' is the first doubled character\n";
c47ff5f1 562
14218588 563 if (/Time: (..):(..):(..)/) { # parse out values
f793d64a
KW
564 $hours = $1;
565 $minutes = $2;
566 $seconds = $3;
a0d0e21e 567 }
c47ff5f1 568
9d860678
KW
569 /(.)(.)(.)(.)(.)(.)(.)(.)(.)\g10/ # \g10 is a backreference
570 /(.)(.)(.)(.)(.)(.)(.)(.)(.)\10/ # \10 is octal
571 /((.)(.)(.)(.)(.)(.)(.)(.)(.))\10/ # \10 is a backreference
572 /((.)(.)(.)(.)(.)(.)(.)(.)(.))\010/ # \010 is octal
573
574 $a = '(.)\1'; # Creates problems when concatenated.
575 $b = '(.)\g{1}'; # Avoids the problems.
576 "aa" =~ /${a}/; # True
577 "aa" =~ /${b}/; # True
578 "aa0" =~ /${a}0/; # False!
579 "aa0" =~ /${b}0/; # True
dc0d9c48
KW
580 "aa\x08" =~ /${a}0/; # True!
581 "aa\x08" =~ /${b}0/; # False
9d860678 582
14218588
GS
583Several special variables also refer back to portions of the previous
584match. C<$+> returns whatever the last bracket match matched.
585C<$&> returns the entire matched string. (At one point C<$0> did
586also, but now it returns the name of the program.) C<$`> returns
77ea4f6d
JV
587everything before the matched string. C<$'> returns everything
588after the matched string. And C<$^N> contains whatever was matched by
589the most-recently closed group (submatch). C<$^N> can be used in
590extended patterns (see below), for example to assign a submatch to a
81714fb9 591variable.
d74e8afc 592X<$+> X<$^N> X<$&> X<$`> X<$'>
14218588 593
d8b950dc
KW
594These special variables, like the C<%+> hash and the numbered match variables
595(C<$1>, C<$2>, C<$3>, etc.) are dynamically scoped
14218588
GS
596until the end of the enclosing block or until the next successful
597match, whichever comes first. (See L<perlsyn/"Compound Statements">.)
d74e8afc
ITB
598X<$+> X<$^N> X<$&> X<$`> X<$'>
599X<$1> X<$2> X<$3> X<$4> X<$5> X<$6> X<$7> X<$8> X<$9>
600
0d017f4d 601B<NOTE>: Failed matches in Perl do not reset the match variables,
5146ce24 602which makes it easier to write code that tests for a series of more
665e98b9
JH
603specific cases and remembers the best match.
604
14218588
GS
605B<WARNING>: Once Perl sees that you need one of C<$&>, C<$`>, or
606C<$'> anywhere in the program, it has to provide them for every
607pattern match. This may substantially slow your program. Perl
d8b950dc 608uses the same mechanism to produce C<$1>, C<$2>, etc, so you also pay a
14218588
GS
609price for each pattern that contains capturing parentheses. (To
610avoid this cost while retaining the grouping behaviour, use the
611extended regular expression C<(?: ... )> instead.) But if you never
612use C<$&>, C<$`> or C<$'>, then patterns I<without> capturing
613parentheses will not be penalized. So avoid C<$&>, C<$'>, and C<$`>
614if you can, but if you can't (and some algorithms really appreciate
615them), once you've used them once, use them at will, because you've
616already paid the price. As of 5.005, C<$&> is not so costly as the
617other two.
d74e8afc 618X<$&> X<$`> X<$'>
68dc0745 619
99d59c4d 620As a workaround for this problem, Perl 5.10.0 introduces C<${^PREMATCH}>,
cde0cee5
YO
621C<${^MATCH}> and C<${^POSTMATCH}>, which are equivalent to C<$`>, C<$&>
622and C<$'>, B<except> that they are only guaranteed to be defined after a
87e95b7f 623successful match that was executed with the C</p> (preserve) modifier.
cde0cee5
YO
624The use of these variables incurs no global performance penalty, unlike
625their punctuation char equivalents, however at the trade-off that you
626have to tell perl when you want to use them.
87e95b7f 627X</p> X<p modifier>
cde0cee5 628
9d727203
KW
629=head2 Quoting metacharacters
630
19799a22
GS
631Backslashed metacharacters in Perl are alphanumeric, such as C<\b>,
632C<\w>, C<\n>. Unlike some other regular expression languages, there
633are no backslashed symbols that aren't alphanumeric. So anything
c47ff5f1 634that looks like \\, \(, \), \<, \>, \{, or \} is always
19799a22
GS
635interpreted as a literal character, not a metacharacter. This was
636once used in a common idiom to disable or quote the special meanings
637of regular expression metacharacters in a string that you want to
36bbe248 638use for a pattern. Simply quote all non-"word" characters:
a0d0e21e
LW
639
640 $pattern =~ s/(\W)/\\$1/g;
641
f1cbbd6e 642(If C<use locale> is set, then this depends on the current locale.)
14218588
GS
643Today it is more common to use the quotemeta() function or the C<\Q>
644metaquoting escape sequence to disable all metacharacters' special
645meanings like this:
a0d0e21e
LW
646
647 /$unquoted\Q$quoted\E$unquoted/
648
9da458fc
IZ
649Beware that if you put literal backslashes (those not inside
650interpolated variables) between C<\Q> and C<\E>, double-quotish
651backslash interpolation may lead to confusing results. If you
652I<need> to use literal backslashes within C<\Q...\E>,
653consult L<perlop/"Gory details of parsing quoted constructs">.
654
19799a22
GS
655=head2 Extended Patterns
656
14218588
GS
657Perl also defines a consistent extension syntax for features not
658found in standard tools like B<awk> and B<lex>. The syntax is a
659pair of parentheses with a question mark as the first thing within
660the parentheses. The character after the question mark indicates
661the extension.
19799a22 662
14218588
GS
663The stability of these extensions varies widely. Some have been
664part of the core language for many years. Others are experimental
665and may change without warning or be completely removed. Check
666the documentation on an individual feature to verify its current
667status.
19799a22 668
14218588
GS
669A question mark was chosen for this and for the minimal-matching
670construct because 1) question marks are rare in older regular
671expressions, and 2) whenever you see one, you should stop and
672"question" exactly what is going on. That's psychology...
a0d0e21e
LW
673
674=over 10
675
cc6b7395 676=item C<(?#text)>
d74e8afc 677X<(?#)>
a0d0e21e 678
14218588 679A comment. The text is ignored. If the C</x> modifier enables
19799a22 680whitespace formatting, a simple C<#> will suffice. Note that Perl closes
259138e3
GS
681the comment as soon as it sees a C<)>, so there is no way to put a literal
682C<)> in the comment.
a0d0e21e 683
cfaf538b 684=item C<(?adlupimsx-imsx)>
fb85c044 685
cfaf538b 686=item C<(?^alupimsx)>
fb85c044 687X<(?)> X<(?^)>
19799a22 688
0b6d1084
JH
689One or more embedded pattern-match modifiers, to be turned on (or
690turned off, if preceded by C<->) for the remainder of the pattern or
fb85c044
KW
691the remainder of the enclosing pattern group (if any).
692
fb85c044 693This is particularly useful for dynamic patterns, such as those read in from a
0d017f4d
WL
694configuration file, taken from an argument, or specified in a table
695somewhere. Consider the case where some patterns want to be case
696sensitive and some do not: The case insensitive ones merely need to
697include C<(?i)> at the front of the pattern. For example:
19799a22
GS
698
699 $pattern = "foobar";
5d458dd8 700 if ( /$pattern/i ) { }
19799a22
GS
701
702 # more flexible:
703
704 $pattern = "(?i)foobar";
5d458dd8 705 if ( /$pattern/ ) { }
19799a22 706
0b6d1084 707These modifiers are restored at the end of the enclosing group. For example,
19799a22 708
d8b950dc 709 ( (?i) blah ) \s+ \g1
19799a22 710
0d017f4d
WL
711will match C<blah> in any case, some spaces, and an exact (I<including the case>!)
712repetition of the previous word, assuming the C</x> modifier, and no C</i>
713modifier outside this group.
19799a22 714
8eb5594e
DR
715These modifiers do not carry over into named subpatterns called in the
716enclosing group. In other words, a pattern such as C<((?i)(&NAME))> does not
717change the case-sensitivity of the "NAME" pattern.
718
dc925305
KW
719Any of these modifiers can be set to apply globally to all regular
720expressions compiled within the scope of a C<use re>. See
a0bbd6ff 721L<re/"'/flags' mode">.
dc925305 722
9de15fec
KW
723Starting in Perl 5.14, a C<"^"> (caret or circumflex accent) immediately
724after the C<"?"> is a shorthand equivalent to C<d-imsx>. Flags (except
725C<"d">) may follow the caret to override it.
726But a minus sign is not legal with it.
727
dc925305 728Note that the C<a>, C<d>, C<l>, C<p>, and C<u> modifiers are special in
e1d8d8ac 729that they can only be enabled, not disabled, and the C<a>, C<d>, C<l>, and
dc925305 730C<u> modifiers are mutually exclusive: specifying one de-specifies the
bdc22dd5 731others, and a maximum of one may appear in the construct. Thus, for
b6fa137b
FC
732example, C<(?-p)>, will warn when compiled under C<use warnings>;
733C<(?-d:...)> and C<(?dl:...)> are fatal errors.
9de15fec
KW
734
735Note also that the C<p> modifier is special in that its presence
736anywhere in a pattern has a global effect.
cde0cee5 737
5a964f20 738=item C<(?:pattern)>
d74e8afc 739X<(?:)>
a0d0e21e 740
cfaf538b 741=item C<(?adluimsx-imsx:pattern)>
ca9dfc88 742
cfaf538b 743=item C<(?^aluimsx:pattern)>
fb85c044
KW
744X<(?^:)>
745
5a964f20
TC
746This is for clustering, not capturing; it groups subexpressions like
747"()", but doesn't make backreferences as "()" does. So
a0d0e21e 748
5a964f20 749 @fields = split(/\b(?:a|b|c)\b/)
a0d0e21e
LW
750
751is like
752
5a964f20 753 @fields = split(/\b(a|b|c)\b/)
a0d0e21e 754
19799a22
GS
755but doesn't spit out extra fields. It's also cheaper not to capture
756characters if you don't need to.
a0d0e21e 757
19799a22 758Any letters between C<?> and C<:> act as flags modifiers as with
cfaf538b 759C<(?adluimsx-imsx)>. For example,
ca9dfc88
IZ
760
761 /(?s-i:more.*than).*million/i
762
14218588 763is equivalent to the more verbose
ca9dfc88
IZ
764
765 /(?:(?s-i)more.*than).*million/i
766
fb85c044 767Starting in Perl 5.14, a C<"^"> (caret or circumflex accent) immediately
9de15fec
KW
768after the C<"?"> is a shorthand equivalent to C<d-imsx>. Any positive
769flags (except C<"d">) may follow the caret, so
fb85c044
KW
770
771 (?^x:foo)
772
773is equivalent to
774
775 (?x-ims:foo)
776
777The caret tells Perl that this cluster doesn't inherit the flags of any
9de15fec 778surrounding pattern, but to go back to the system defaults (C<d-imsx>),
fb85c044
KW
779modified by any flags specified.
780
781The caret allows for simpler stringification of compiled regular
782expressions. These look like
783
784 (?^:pattern)
785
786with any non-default flags appearing between the caret and the colon.
787A test that looks at such stringification thus doesn't need to have the
788system default flags hard-coded in it, just the caret. If new flags are
789added to Perl, the meaning of the caret's expansion will change to include
790the default for those flags, so the test will still work, unchanged.
791
792Specifying a negative flag after the caret is an error, as the flag is
793redundant.
794
795Mnemonic for C<(?^...)>: A fresh beginning since the usual use of a caret is
796to match at the beginning.
797
594d7033
YO
798=item C<(?|pattern)>
799X<(?|)> X<Branch reset>
800
801This is the "branch reset" pattern, which has the special property
c27a5cfe 802that the capture groups are numbered from the same starting point
99d59c4d 803in each alternation branch. It is available starting from perl 5.10.0.
4deaaa80 804
c27a5cfe 805Capture groups are numbered from left to right, but inside this
693596a8 806construct the numbering is restarted for each branch.
4deaaa80 807
c27a5cfe 808The numbering within each branch will be as normal, and any groups
4deaaa80
PJ
809following this construct will be numbered as though the construct
810contained only one branch, that being the one with the most capture
c27a5cfe 811groups in it.
4deaaa80
PJ
812
813This construct will be useful when you want to capture one of a
814number of alternative matches.
815
816Consider the following pattern. The numbers underneath show in
c27a5cfe 817which group the captured content will be stored.
594d7033
YO
818
819
820 # before ---------------branch-reset----------- after
821 / ( a ) (?| x ( y ) z | (p (q) r) | (t) u (v) ) ( z ) /x
822 # 1 2 2 3 2 3 4
823
ab106183
A
824Be careful when using the branch reset pattern in combination with
825named captures. Named captures are implemented as being aliases to
c27a5cfe 826numbered groups holding the captures, and that interferes with the
ab106183
A
827implementation of the branch reset pattern. If you are using named
828captures in a branch reset pattern, it's best to use the same names,
829in the same order, in each of the alternations:
830
831 /(?| (?<a> x ) (?<b> y )
832 | (?<a> z ) (?<b> w )) /x
833
834Not doing so may lead to surprises:
835
836 "12" =~ /(?| (?<a> \d+ ) | (?<b> \D+))/x;
837 say $+ {a}; # Prints '12'
838 say $+ {b}; # *Also* prints '12'.
839
c27a5cfe
KW
840The problem here is that both the group named C<< a >> and the group
841named C<< b >> are aliases for the group belonging to C<< $1 >>.
90a18110 842
ee9b8eae
YO
843=item Look-Around Assertions
844X<look-around assertion> X<lookaround assertion> X<look-around> X<lookaround>
845
846Look-around assertions are zero width patterns which match a specific
847pattern without including it in C<$&>. Positive assertions match when
848their subpattern matches, negative assertions match when their subpattern
849fails. Look-behind matches text up to the current match position,
850look-ahead matches text following the current match position.
851
852=over 4
853
5a964f20 854=item C<(?=pattern)>
d74e8afc 855X<(?=)> X<look-ahead, positive> X<lookahead, positive>
a0d0e21e 856
19799a22 857A zero-width positive look-ahead assertion. For example, C</\w+(?=\t)/>
a0d0e21e
LW
858matches a word followed by a tab, without including the tab in C<$&>.
859
5a964f20 860=item C<(?!pattern)>
d74e8afc 861X<(?!)> X<look-ahead, negative> X<lookahead, negative>
a0d0e21e 862
19799a22 863A zero-width negative look-ahead assertion. For example C</foo(?!bar)/>
a0d0e21e 864matches any occurrence of "foo" that isn't followed by "bar". Note
19799a22
GS
865however that look-ahead and look-behind are NOT the same thing. You cannot
866use this for look-behind.
7b8d334a 867
5a964f20 868If you are looking for a "bar" that isn't preceded by a "foo", C</(?!foo)bar/>
7b8d334a
GS
869will not do what you want. That's because the C<(?!foo)> is just saying that
870the next thing cannot be "foo"--and it's not, it's a "bar", so "foobar" will
871match. You would have to do something like C</(?!foo)...bar/> for that. We
872say "like" because there's the case of your "bar" not having three characters
873before it. You could cover that this way: C</(?:(?!foo)...|^.{0,2})bar/>.
874Sometimes it's still easier just to say:
a0d0e21e 875
a3cb178b 876 if (/bar/ && $` !~ /foo$/)
a0d0e21e 877
19799a22 878For look-behind see below.
c277df42 879
ee9b8eae
YO
880=item C<(?<=pattern)> C<\K>
881X<(?<=)> X<look-behind, positive> X<lookbehind, positive> X<\K>
c277df42 882
c47ff5f1 883A zero-width positive look-behind assertion. For example, C</(?<=\t)\w+/>
19799a22
GS
884matches a word that follows a tab, without including the tab in C<$&>.
885Works only for fixed-width look-behind.
c277df42 886
ee9b8eae
YO
887There is a special form of this construct, called C<\K>, which causes the
888regex engine to "keep" everything it had matched prior to the C<\K> and
889not include it in C<$&>. This effectively provides variable length
890look-behind. The use of C<\K> inside of another look-around assertion
891is allowed, but the behaviour is currently not well defined.
892
c62285ac 893For various reasons C<\K> may be significantly more efficient than the
ee9b8eae
YO
894equivalent C<< (?<=...) >> construct, and it is especially useful in
895situations where you want to efficiently remove something following
896something else in a string. For instance
897
898 s/(foo)bar/$1/g;
899
900can be rewritten as the much more efficient
901
902 s/foo\Kbar//g;
903
5a964f20 904=item C<(?<!pattern)>
d74e8afc 905X<(?<!)> X<look-behind, negative> X<lookbehind, negative>
c277df42 906
19799a22
GS
907A zero-width negative look-behind assertion. For example C</(?<!bar)foo/>
908matches any occurrence of "foo" that does not follow "bar". Works
909only for fixed-width look-behind.
c277df42 910
ee9b8eae
YO
911=back
912
81714fb9
YO
913=item C<(?'NAME'pattern)>
914
915=item C<< (?<NAME>pattern) >>
916X<< (?<NAME>) >> X<(?'NAME')> X<named capture> X<capture>
917
c27a5cfe 918A named capture group. Identical in every respect to normal capturing
90a18110 919parentheses C<()> but for the additional fact that C<%+> or C<%-> may be
1b2010d5 920used after a successful match to refer to a named group. See L<perlvar>
90a18110 921for more details on the C<%+> and C<%-> hashes.
81714fb9 922
c27a5cfe
KW
923If multiple distinct capture groups have the same name then the
924$+{NAME} will refer to the leftmost defined group in the match.
81714fb9 925
0d017f4d 926The forms C<(?'NAME'pattern)> and C<< (?<NAME>pattern) >> are equivalent.
81714fb9
YO
927
928B<NOTE:> While the notation of this construct is the same as the similar
c27a5cfe 929function in .NET regexes, the behavior is not. In Perl the groups are
81714fb9
YO
930numbered sequentially regardless of being named or not. Thus in the
931pattern
932
933 /(x)(?<foo>y)(z)/
934
935$+{foo} will be the same as $2, and $3 will contain 'z' instead of
936the opposite which is what a .NET regex hacker might expect.
937
1f1031fe
YO
938Currently NAME is restricted to simple identifiers only.
939In other words, it must match C</^[_A-Za-z][_A-Za-z0-9]*\z/> or
940its Unicode extension (see L<utf8>),
941though it isn't extended by the locale (see L<perllocale>).
81714fb9 942
1f1031fe 943B<NOTE:> In order to make things easier for programmers with experience
ae5648b3 944with the Python or PCRE regex engines, the pattern C<< (?PE<lt>NAMEE<gt>pattern) >>
0d017f4d 945may be used instead of C<< (?<NAME>pattern) >>; however this form does not
64c5a566 946support the use of single quotes as a delimiter for the name.
81714fb9 947
1f1031fe
YO
948=item C<< \k<NAME> >>
949
950=item C<< \k'NAME' >>
81714fb9
YO
951
952Named backreference. Similar to numeric backreferences, except that
953the group is designated by name and not number. If multiple groups
954have the same name then it refers to the leftmost defined group in
955the current match.
956
0d017f4d 957It is an error to refer to a name not defined by a C<< (?<NAME>) >>
81714fb9
YO
958earlier in the pattern.
959
960Both forms are equivalent.
961
1f1031fe 962B<NOTE:> In order to make things easier for programmers with experience
0d017f4d 963with the Python or PCRE regex engines, the pattern C<< (?P=NAME) >>
64c5a566 964may be used instead of C<< \k<NAME> >>.
1f1031fe 965
cc6b7395 966=item C<(?{ code })>
d74e8afc 967X<(?{})> X<regex, code in> X<regexp, code in> X<regular expression, code in>
c277df42 968
19799a22 969B<WARNING>: This extended regular expression feature is considered
b9b4dddf
YO
970experimental, and may be changed without notice. Code executed that
971has side effects may not perform identically from version to version
972due to the effect of future optimisations in the regex engine.
c277df42 973
cc46d5f2 974This zero-width assertion evaluates any embedded Perl code. It
19799a22
GS
975always succeeds, and its C<code> is not interpolated. Currently,
976the rules to determine where the C<code> ends are somewhat convoluted.
977
77ea4f6d
JV
978This feature can be used together with the special variable C<$^N> to
979capture the results of submatches in variables without having to keep
980track of the number of nested parentheses. For example:
981
982 $_ = "The brown fox jumps over the lazy dog";
983 /the (\S+)(?{ $color = $^N }) (\S+)(?{ $animal = $^N })/i;
984 print "color = $color, animal = $animal\n";
985
754091cb
RGS
986Inside the C<(?{...})> block, C<$_> refers to the string the regular
987expression is matching against. You can also use C<pos()> to know what is
fa11829f 988the current position of matching within this string.
754091cb 989
19799a22
GS
990The C<code> is properly scoped in the following sense: If the assertion
991is backtracked (compare L<"Backtracking">), all changes introduced after
992C<local>ization are undone, so that
b9ac3b5b
GS
993
994 $_ = 'a' x 8;
5d458dd8 995 m<
f793d64a 996 (?{ $cnt = 0 }) # Initialize $cnt.
b9ac3b5b 997 (
5d458dd8 998 a
b9ac3b5b 999 (?{
f793d64a 1000 local $cnt = $cnt + 1; # Update $cnt, backtracking-safe.
b9ac3b5b 1001 })
5d458dd8 1002 )*
b9ac3b5b 1003 aaaa
f793d64a
KW
1004 (?{ $res = $cnt }) # On success copy to
1005 # non-localized location.
b9ac3b5b
GS
1006 >x;
1007
0d017f4d 1008will set C<$res = 4>. Note that after the match, C<$cnt> returns to the globally
14218588 1009introduced value, because the scopes that restrict C<local> operators
b9ac3b5b
GS
1010are unwound.
1011
19799a22
GS
1012This assertion may be used as a C<(?(condition)yes-pattern|no-pattern)>
1013switch. If I<not> used in this way, the result of evaluation of
1014C<code> is put into the special variable C<$^R>. This happens
1015immediately, so C<$^R> can be used from other C<(?{ code })> assertions
1016inside the same regular expression.
b9ac3b5b 1017
19799a22
GS
1018The assignment to C<$^R> above is properly localized, so the old
1019value of C<$^R> is restored if the assertion is backtracked; compare
1020L<"Backtracking">.
b9ac3b5b 1021
19799a22
GS
1022For reasons of security, this construct is forbidden if the regular
1023expression involves run-time interpolation of variables, unless the
1024perilous C<use re 'eval'> pragma has been used (see L<re>), or the
1025variables contain results of C<qr//> operator (see
b6fa137b 1026L<perlop/"qr/STRINGE<sol>msixpodual">).
871b0233 1027
0d017f4d 1028This restriction is due to the wide-spread and remarkably convenient
19799a22 1029custom of using run-time determined strings as patterns. For example:
871b0233
IZ
1030
1031 $re = <>;
1032 chomp $re;
1033 $string =~ /$re/;
1034
14218588
GS
1035Before Perl knew how to execute interpolated code within a pattern,
1036this operation was completely safe from a security point of view,
1037although it could raise an exception from an illegal pattern. If
1038you turn on the C<use re 'eval'>, though, it is no longer secure,
1039so you should only do so if you are also using taint checking.
1040Better yet, use the carefully constrained evaluation within a Safe
cc46d5f2 1041compartment. See L<perlsec> for details about both these mechanisms.
871b0233 1042
e95d7314
GG
1043B<WARNING>: Use of lexical (C<my>) variables in these blocks is
1044broken. The result is unpredictable and will make perl unstable. The
1045workaround is to use global (C<our>) variables.
1046
8525cfae
FC
1047B<WARNING>: In perl 5.12.x and earlier, the regex engine
1048was not re-entrant, so interpolated code could not
1049safely invoke the regex engine either directly with
e95d7314 1050C<m//> or C<s///>), or indirectly with functions such as
8525cfae 1051C<split>. Invoking the regex engine in these blocks would make perl
e95d7314 1052unstable.
8988a1bb 1053
14455d6c 1054=item C<(??{ code })>
d74e8afc
ITB
1055X<(??{})>
1056X<regex, postponed> X<regexp, postponed> X<regular expression, postponed>
0f5d15d6 1057
19799a22 1058B<WARNING>: This extended regular expression feature is considered
b9b4dddf
YO
1059experimental, and may be changed without notice. Code executed that
1060has side effects may not perform identically from version to version
1061due to the effect of future optimisations in the regex engine.
0f5d15d6 1062
19799a22
GS
1063This is a "postponed" regular subexpression. The C<code> is evaluated
1064at run time, at the moment this subexpression may match. The result
1065of evaluation is considered as a regular expression and matched as
61528107 1066if it were inserted instead of this construct. Note that this means
c27a5cfe 1067that the contents of capture groups defined inside an eval'ed pattern
6bda09f9 1068are not available outside of the pattern, and vice versa, there is no
c27a5cfe 1069way for the inner pattern to refer to a capture group defined outside.
6bda09f9
YO
1070Thus,
1071
1072 ('a' x 100)=~/(??{'(.)' x 100})/
1073
81714fb9 1074B<will> match, it will B<not> set $1.
0f5d15d6 1075
428594d9 1076The C<code> is not interpolated. As before, the rules to determine
19799a22
GS
1077where the C<code> ends are currently somewhat convoluted.
1078
1079The following pattern matches a parenthesized group:
0f5d15d6
IZ
1080
1081 $re = qr{
f793d64a
KW
1082 \(
1083 (?:
1084 (?> [^()]+ ) # Non-parens without backtracking
1085 |
1086 (??{ $re }) # Group with matching parens
1087 )*
1088 \)
1089 }x;
0f5d15d6 1090
6bda09f9
YO
1091See also C<(?PARNO)> for a different, more efficient way to accomplish
1092the same task.
1093
0b370c0a
A
1094For reasons of security, this construct is forbidden if the regular
1095expression involves run-time interpolation of variables, unless the
1096perilous C<use re 'eval'> pragma has been used (see L<re>), or the
1097variables contain results of C<qr//> operator (see
b6fa137b 1098L<perlop/"qrE<sol>STRINGE<sol>msixpodual">).
0b370c0a 1099
8525cfae
FC
1100In perl 5.12.x and earlier, because the regex engine was not re-entrant,
1101delayed code could not safely invoke the regex engine either directly with
1102C<m//> or C<s///>), or indirectly with functions such as C<split>.
8988a1bb 1103
5d458dd8
YO
1104Recursing deeper than 50 times without consuming any input string will
1105result in a fatal error. The maximum depth is compiled into perl, so
6bda09f9
YO
1106changing it requires a custom build.
1107
542fa716
YO
1108=item C<(?PARNO)> C<(?-PARNO)> C<(?+PARNO)> C<(?R)> C<(?0)>
1109X<(?PARNO)> X<(?1)> X<(?R)> X<(?0)> X<(?-1)> X<(?+1)> X<(?-PARNO)> X<(?+PARNO)>
6bda09f9 1110X<regex, recursive> X<regexp, recursive> X<regular expression, recursive>
542fa716 1111X<regex, relative recursion>
6bda09f9 1112
81714fb9 1113Similar to C<(??{ code })> except it does not involve compiling any code,
c27a5cfe
KW
1114instead it treats the contents of a capture group as an independent
1115pattern that must match at the current position. Capture groups
81714fb9 1116contained by the pattern will have the value as determined by the
6bda09f9
YO
1117outermost recursion.
1118
894be9b7 1119PARNO is a sequence of digits (not starting with 0) whose value reflects
c27a5cfe 1120the paren-number of the capture group to recurse to. C<(?R)> recurses to
894be9b7 1121the beginning of the whole pattern. C<(?0)> is an alternate syntax for
542fa716 1122C<(?R)>. If PARNO is preceded by a plus or minus sign then it is assumed
c27a5cfe 1123to be relative, with negative numbers indicating preceding capture groups
542fa716 1124and positive ones following. Thus C<(?-1)> refers to the most recently
c27a5cfe 1125declared group, and C<(?+1)> indicates the next group to be declared.
c74340f9 1126Note that the counting for relative recursion differs from that of
c27a5cfe 1127relative backreferences, in that with recursion unclosed groups B<are>
c74340f9 1128included.
6bda09f9 1129
81714fb9 1130The following pattern matches a function foo() which may contain
f145b7e9 1131balanced parentheses as the argument.
6bda09f9
YO
1132
1133 $re = qr{ ( # paren group 1 (full function)
81714fb9 1134 foo
6bda09f9
YO
1135 ( # paren group 2 (parens)
1136 \(
1137 ( # paren group 3 (contents of parens)
1138 (?:
1139 (?> [^()]+ ) # Non-parens without backtracking
1140 |
1141 (?2) # Recurse to start of paren group 2
1142 )*
1143 )
1144 \)
1145 )
1146 )
1147 }x;
1148
1149If the pattern was used as follows
1150
1151 'foo(bar(baz)+baz(bop))'=~/$re/
1152 and print "\$1 = $1\n",
1153 "\$2 = $2\n",
1154 "\$3 = $3\n";
1155
1156the output produced should be the following:
1157
1158 $1 = foo(bar(baz)+baz(bop))
1159 $2 = (bar(baz)+baz(bop))
81714fb9 1160 $3 = bar(baz)+baz(bop)
6bda09f9 1161
c27a5cfe 1162If there is no corresponding capture group defined, then it is a
61528107 1163fatal error. Recursing deeper than 50 times without consuming any input
81714fb9 1164string will also result in a fatal error. The maximum depth is compiled
6bda09f9
YO
1165into perl, so changing it requires a custom build.
1166
542fa716
YO
1167The following shows how using negative indexing can make it
1168easier to embed recursive patterns inside of a C<qr//> construct
1169for later use:
1170
1171 my $parens = qr/(\((?:[^()]++|(?-1))*+\))/;
1172 if (/foo $parens \s+ + \s+ bar $parens/x) {
1173 # do something here...
1174 }
1175
81714fb9 1176B<Note> that this pattern does not behave the same way as the equivalent
0d017f4d 1177PCRE or Python construct of the same form. In Perl you can backtrack into
6bda09f9 1178a recursed group, in PCRE and Python the recursed into group is treated
542fa716
YO
1179as atomic. Also, modifiers are resolved at compile time, so constructs
1180like (?i:(?1)) or (?:(?i)(?1)) do not affect how the sub-pattern will
1181be processed.
6bda09f9 1182
894be9b7
YO
1183=item C<(?&NAME)>
1184X<(?&NAME)>
1185
0d017f4d
WL
1186Recurse to a named subpattern. Identical to C<(?PARNO)> except that the
1187parenthesis to recurse to is determined by name. If multiple parentheses have
894be9b7
YO
1188the same name, then it recurses to the leftmost.
1189
1190It is an error to refer to a name that is not declared somewhere in the
1191pattern.
1192
1f1031fe
YO
1193B<NOTE:> In order to make things easier for programmers with experience
1194with the Python or PCRE regex engines the pattern C<< (?P>NAME) >>
64c5a566 1195may be used instead of C<< (?&NAME) >>.
1f1031fe 1196
e2e6a0f1
YO
1197=item C<(?(condition)yes-pattern|no-pattern)>
1198X<(?()>
286f584a 1199
e2e6a0f1 1200=item C<(?(condition)yes-pattern)>
286f584a 1201
e2e6a0f1
YO
1202Conditional expression. C<(condition)> should be either an integer in
1203parentheses (which is valid if the corresponding pair of parentheses
1204matched), a look-ahead/look-behind/evaluate zero-width assertion, a
c27a5cfe 1205name in angle brackets or single quotes (which is valid if a group
e2e6a0f1
YO
1206with the given name matched), or the special symbol (R) (true when
1207evaluated inside of recursion or eval). Additionally the R may be
1208followed by a number, (which will be true when evaluated when recursing
1209inside of the appropriate group), or by C<&NAME>, in which case it will
1210be true only when evaluated during recursion in the named group.
1211
1212Here's a summary of the possible predicates:
1213
1214=over 4
1215
1216=item (1) (2) ...
1217
c27a5cfe 1218Checks if the numbered capturing group has matched something.
e2e6a0f1
YO
1219
1220=item (<NAME>) ('NAME')
1221
c27a5cfe 1222Checks if a group with the given name has matched something.
e2e6a0f1 1223
f01cd190
FC
1224=item (?=...) (?!...) (?<=...) (?<!...)
1225
1226Checks whether the pattern matches (or does not match, for the '!'
1227variants).
1228
e2e6a0f1
YO
1229=item (?{ CODE })
1230
f01cd190 1231Treats the return value of the code block as the condition.
e2e6a0f1
YO
1232
1233=item (R)
1234
1235Checks if the expression has been evaluated inside of recursion.
1236
1237=item (R1) (R2) ...
1238
1239Checks if the expression has been evaluated while executing directly
1240inside of the n-th capture group. This check is the regex equivalent of
1241
1242 if ((caller(0))[3] eq 'subname') { ... }
1243
1244In other words, it does not check the full recursion stack.
1245
1246=item (R&NAME)
1247
1248Similar to C<(R1)>, this predicate checks to see if we're executing
1249directly inside of the leftmost group with a given name (this is the same
1250logic used by C<(?&NAME)> to disambiguate). It does not check the full
1251stack, but only the name of the innermost active recursion.
1252
1253=item (DEFINE)
1254
1255In this case, the yes-pattern is never directly executed, and no
1256no-pattern is allowed. Similar in spirit to C<(?{0})> but more efficient.
1257See below for details.
1258
1259=back
1260
1261For example:
1262
1263 m{ ( \( )?
1264 [^()]+
1265 (?(1) \) )
1266 }x
1267
1268matches a chunk of non-parentheses, possibly included in parentheses
1269themselves.
1270
1271A special form is the C<(DEFINE)> predicate, which never executes directly
1272its yes-pattern, and does not allow a no-pattern. This allows to define
1273subpatterns which will be executed only by using the recursion mechanism.
1274This way, you can define a set of regular expression rules that can be
1275bundled into any pattern you choose.
1276
1277It is recommended that for this usage you put the DEFINE block at the
1278end of the pattern, and that you name any subpatterns defined within it.
1279
1280Also, it's worth noting that patterns defined this way probably will
1281not be as efficient, as the optimiser is not very clever about
1282handling them.
1283
1284An example of how this might be used is as follows:
1285
2bf803e2 1286 /(?<NAME>(?&NAME_PAT))(?<ADDR>(?&ADDRESS_PAT))
e2e6a0f1 1287 (?(DEFINE)
2bf803e2
YO
1288 (?<NAME_PAT>....)
1289 (?<ADRESS_PAT>....)
e2e6a0f1
YO
1290 )/x
1291
c27a5cfe
KW
1292Note that capture groups matched inside of recursion are not accessible
1293after the recursion returns, so the extra layer of capturing groups is
e2e6a0f1
YO
1294necessary. Thus C<$+{NAME_PAT}> would not be defined even though
1295C<$+{NAME}> would be.
286f584a 1296
c47ff5f1 1297=item C<< (?>pattern) >>
6bda09f9 1298X<backtrack> X<backtracking> X<atomic> X<possessive>
5a964f20 1299
19799a22
GS
1300An "independent" subexpression, one which matches the substring
1301that a I<standalone> C<pattern> would match if anchored at the given
9da458fc 1302position, and it matches I<nothing other than this substring>. This
19799a22
GS
1303construct is useful for optimizations of what would otherwise be
1304"eternal" matches, because it will not backtrack (see L<"Backtracking">).
9da458fc
IZ
1305It may also be useful in places where the "grab all you can, and do not
1306give anything back" semantic is desirable.
19799a22 1307
c47ff5f1 1308For example: C<< ^(?>a*)ab >> will never match, since C<< (?>a*) >>
19799a22
GS
1309(anchored at the beginning of string, as above) will match I<all>
1310characters C<a> at the beginning of string, leaving no C<a> for
1311C<ab> to match. In contrast, C<a*ab> will match the same as C<a+b>,
1312since the match of the subgroup C<a*> is influenced by the following
1313group C<ab> (see L<"Backtracking">). In particular, C<a*> inside
1314C<a*ab> will match fewer characters than a standalone C<a*>, since
1315this makes the tail match.
1316
c47ff5f1 1317An effect similar to C<< (?>pattern) >> may be achieved by writing
d8b950dc
KW
1318C<(?=(pattern))\g1>. This matches the same substring as a standalone
1319C<a+>, and the following C<\g1> eats the matched string; it therefore
c47ff5f1 1320makes a zero-length assertion into an analogue of C<< (?>...) >>.
19799a22
GS
1321(The difference between these two constructs is that the second one
1322uses a capturing group, thus shifting ordinals of backreferences
1323in the rest of a regular expression.)
1324
1325Consider this pattern:
c277df42 1326
871b0233 1327 m{ \(
e2e6a0f1 1328 (
f793d64a 1329 [^()]+ # x+
e2e6a0f1 1330 |
871b0233
IZ
1331 \( [^()]* \)
1332 )+
e2e6a0f1 1333 \)
871b0233 1334 }x
5a964f20 1335
19799a22
GS
1336That will efficiently match a nonempty group with matching parentheses
1337two levels deep or less. However, if there is no such group, it
1338will take virtually forever on a long string. That's because there
1339are so many different ways to split a long string into several
1340substrings. This is what C<(.+)+> is doing, and C<(.+)+> is similar
1341to a subpattern of the above pattern. Consider how the pattern
1342above detects no-match on C<((()aaaaaaaaaaaaaaaaaa> in several
1343seconds, but that each extra letter doubles this time. This
1344exponential performance will make it appear that your program has
14218588 1345hung. However, a tiny change to this pattern
5a964f20 1346
e2e6a0f1
YO
1347 m{ \(
1348 (
f793d64a 1349 (?> [^()]+ ) # change x+ above to (?> x+ )
e2e6a0f1 1350 |
871b0233
IZ
1351 \( [^()]* \)
1352 )+
e2e6a0f1 1353 \)
871b0233 1354 }x
c277df42 1355
c47ff5f1 1356which uses C<< (?>...) >> matches exactly when the one above does (verifying
5a964f20
TC
1357this yourself would be a productive exercise), but finishes in a fourth
1358the time when used on a similar string with 1000000 C<a>s. Be aware,
1359however, that this pattern currently triggers a warning message under
9f1b1f2d 1360the C<use warnings> pragma or B<-w> switch saying it
6bab786b 1361C<"matches null string many times in regex">.
c277df42 1362
c47ff5f1 1363On simple groups, such as the pattern C<< (?> [^()]+ ) >>, a comparable
19799a22 1364effect may be achieved by negative look-ahead, as in C<[^()]+ (?! [^()] )>.
c277df42
IZ
1365This was only 4 times slower on a string with 1000000 C<a>s.
1366
9da458fc
IZ
1367The "grab all you can, and do not give anything back" semantic is desirable
1368in many situations where on the first sight a simple C<()*> looks like
1369the correct solution. Suppose we parse text with comments being delimited
1370by C<#> followed by some optional (horizontal) whitespace. Contrary to
4375e838 1371its appearance, C<#[ \t]*> I<is not> the correct subexpression to match
9da458fc
IZ
1372the comment delimiter, because it may "give up" some whitespace if
1373the remainder of the pattern can be made to match that way. The correct
1374answer is either one of these:
1375
1376 (?>#[ \t]*)
1377 #[ \t]*(?![ \t])
1378
1379For example, to grab non-empty comments into $1, one should use either
1380one of these:
1381
1382 / (?> \# [ \t]* ) ( .+ ) /x;
1383 / \# [ \t]* ( [^ \t] .* ) /x;
1384
1385Which one you pick depends on which of these expressions better reflects
1386the above specification of comments.
1387
6bda09f9
YO
1388In some literature this construct is called "atomic matching" or
1389"possessive matching".
1390
b9b4dddf
YO
1391Possessive quantifiers are equivalent to putting the item they are applied
1392to inside of one of these constructs. The following equivalences apply:
1393
1394 Quantifier Form Bracketing Form
1395 --------------- ---------------
1396 PAT*+ (?>PAT*)
1397 PAT++ (?>PAT+)
1398 PAT?+ (?>PAT?)
1399 PAT{min,max}+ (?>PAT{min,max})
1400
e2e6a0f1
YO
1401=back
1402
1403=head2 Special Backtracking Control Verbs
1404
1405B<WARNING:> These patterns are experimental and subject to change or
0d017f4d 1406removal in a future version of Perl. Their usage in production code should
e2e6a0f1
YO
1407be noted to avoid problems during upgrades.
1408
1409These special patterns are generally of the form C<(*VERB:ARG)>. Unless
1410otherwise stated the ARG argument is optional; in some cases, it is
1411forbidden.
1412
1413Any pattern containing a special backtracking verb that allows an argument
e1020413 1414has the special behaviour that when executed it sets the current package's
5d458dd8
YO
1415C<$REGERROR> and C<$REGMARK> variables. When doing so the following
1416rules apply:
e2e6a0f1 1417
5d458dd8
YO
1418On failure, the C<$REGERROR> variable will be set to the ARG value of the
1419verb pattern, if the verb was involved in the failure of the match. If the
1420ARG part of the pattern was omitted, then C<$REGERROR> will be set to the
1421name of the last C<(*MARK:NAME)> pattern executed, or to TRUE if there was
1422none. Also, the C<$REGMARK> variable will be set to FALSE.
e2e6a0f1 1423
5d458dd8
YO
1424On a successful match, the C<$REGERROR> variable will be set to FALSE, and
1425the C<$REGMARK> variable will be set to the name of the last
1426C<(*MARK:NAME)> pattern executed. See the explanation for the
1427C<(*MARK:NAME)> verb below for more details.
e2e6a0f1 1428
5d458dd8
YO
1429B<NOTE:> C<$REGERROR> and C<$REGMARK> are not magic variables like C<$1>
1430and most other regex related variables. They are not local to a scope, nor
1431readonly, but instead are volatile package variables similar to C<$AUTOLOAD>.
1432Use C<local> to localize changes to them to a specific scope if necessary.
e2e6a0f1
YO
1433
1434If a pattern does not contain a special backtracking verb that allows an
5d458dd8 1435argument, then C<$REGERROR> and C<$REGMARK> are not touched at all.
e2e6a0f1
YO
1436
1437=over 4
1438
1439=item Verbs that take an argument
1440
1441=over 4
1442
5d458dd8 1443=item C<(*PRUNE)> C<(*PRUNE:NAME)>
f7819f85 1444X<(*PRUNE)> X<(*PRUNE:NAME)>
54612592 1445
5d458dd8
YO
1446This zero-width pattern prunes the backtracking tree at the current point
1447when backtracked into on failure. Consider the pattern C<A (*PRUNE) B>,
1448where A and B are complex patterns. Until the C<(*PRUNE)> verb is reached,
1449A may backtrack as necessary to match. Once it is reached, matching
1450continues in B, which may also backtrack as necessary; however, should B
1451not match, then no further backtracking will take place, and the pattern
1452will fail outright at the current starting position.
54612592
YO
1453
1454The following example counts all the possible matching strings in a
1455pattern (without actually matching any of them).
1456
e2e6a0f1 1457 'aaab' =~ /a+b?(?{print "$&\n"; $count++})(*FAIL)/;
54612592
YO
1458 print "Count=$count\n";
1459
1460which produces:
1461
1462 aaab
1463 aaa
1464 aa
1465 a
1466 aab
1467 aa
1468 a
1469 ab
1470 a
1471 Count=9
1472
5d458dd8 1473If we add a C<(*PRUNE)> before the count like the following
54612592 1474
5d458dd8 1475 'aaab' =~ /a+b?(*PRUNE)(?{print "$&\n"; $count++})(*FAIL)/;
54612592
YO
1476 print "Count=$count\n";
1477
1478we prevent backtracking and find the count of the longest matching
353c6505 1479at each matching starting point like so:
54612592
YO
1480
1481 aaab
1482 aab
1483 ab
1484 Count=3
1485
5d458dd8 1486Any number of C<(*PRUNE)> assertions may be used in a pattern.
54612592 1487
5d458dd8
YO
1488See also C<< (?>pattern) >> and possessive quantifiers for other ways to
1489control backtracking. In some cases, the use of C<(*PRUNE)> can be
1490replaced with a C<< (?>pattern) >> with no functional difference; however,
1491C<(*PRUNE)> can be used to handle cases that cannot be expressed using a
1492C<< (?>pattern) >> alone.
54612592 1493
e2e6a0f1 1494
5d458dd8
YO
1495=item C<(*SKIP)> C<(*SKIP:NAME)>
1496X<(*SKIP)>
e2e6a0f1 1497
5d458dd8 1498This zero-width pattern is similar to C<(*PRUNE)>, except that on
e2e6a0f1 1499failure it also signifies that whatever text that was matched leading up
5d458dd8
YO
1500to the C<(*SKIP)> pattern being executed cannot be part of I<any> match
1501of this pattern. This effectively means that the regex engine "skips" forward
1502to this position on failure and tries to match again, (assuming that
1503there is sufficient room to match).
1504
1505The name of the C<(*SKIP:NAME)> pattern has special significance. If a
1506C<(*MARK:NAME)> was encountered while matching, then it is that position
1507which is used as the "skip point". If no C<(*MARK)> of that name was
1508encountered, then the C<(*SKIP)> operator has no effect. When used
1509without a name the "skip point" is where the match point was when
1510executing the (*SKIP) pattern.
1511
1512Compare the following to the examples in C<(*PRUNE)>, note the string
24b23f37
YO
1513is twice as long:
1514
5d458dd8 1515 'aaabaaab' =~ /a+b?(*SKIP)(?{print "$&\n"; $count++})(*FAIL)/;
24b23f37
YO
1516 print "Count=$count\n";
1517
1518outputs
1519
1520 aaab
1521 aaab
1522 Count=2
1523
5d458dd8 1524Once the 'aaab' at the start of the string has matched, and the C<(*SKIP)>
353c6505 1525executed, the next starting point will be where the cursor was when the
5d458dd8
YO
1526C<(*SKIP)> was executed.
1527
5d458dd8
YO
1528=item C<(*MARK:NAME)> C<(*:NAME)>
1529X<(*MARK)> C<(*MARK:NAME)> C<(*:NAME)>
1530
1531This zero-width pattern can be used to mark the point reached in a string
1532when a certain part of the pattern has been successfully matched. This
1533mark may be given a name. A later C<(*SKIP)> pattern will then skip
1534forward to that point if backtracked into on failure. Any number of
b4222fa9 1535C<(*MARK)> patterns are allowed, and the NAME portion may be duplicated.
5d458dd8
YO
1536
1537In addition to interacting with the C<(*SKIP)> pattern, C<(*MARK:NAME)>
1538can be used to "label" a pattern branch, so that after matching, the
1539program can determine which branches of the pattern were involved in the
1540match.
1541
1542When a match is successful, the C<$REGMARK> variable will be set to the
1543name of the most recently executed C<(*MARK:NAME)> that was involved
1544in the match.
1545
1546This can be used to determine which branch of a pattern was matched
c27a5cfe 1547without using a separate capture group for each branch, which in turn
5d458dd8
YO
1548can result in a performance improvement, as perl cannot optimize
1549C</(?:(x)|(y)|(z))/> as efficiently as something like
1550C</(?:x(*MARK:x)|y(*MARK:y)|z(*MARK:z))/>.
1551
1552When a match has failed, and unless another verb has been involved in
1553failing the match and has provided its own name to use, the C<$REGERROR>
1554variable will be set to the name of the most recently executed
1555C<(*MARK:NAME)>.
1556
1557See C<(*SKIP)> for more details.
1558
b62d2d15
YO
1559As a shortcut C<(*MARK:NAME)> can be written C<(*:NAME)>.
1560
5d458dd8
YO
1561=item C<(*THEN)> C<(*THEN:NAME)>
1562
241e7389 1563This is similar to the "cut group" operator C<::> from Perl 6. Like
5d458dd8
YO
1564C<(*PRUNE)>, this verb always matches, and when backtracked into on
1565failure, it causes the regex engine to try the next alternation in the
1566innermost enclosing group (capturing or otherwise).
1567
1568Its name comes from the observation that this operation combined with the
1569alternation operator (C<|>) can be used to create what is essentially a
1570pattern-based if/then/else block:
1571
1572 ( COND (*THEN) FOO | COND2 (*THEN) BAR | COND3 (*THEN) BAZ )
1573
1574Note that if this operator is used and NOT inside of an alternation then
1575it acts exactly like the C<(*PRUNE)> operator.
1576
1577 / A (*PRUNE) B /
1578
1579is the same as
1580
1581 / A (*THEN) B /
1582
1583but
1584
1585 / ( A (*THEN) B | C (*THEN) D ) /
1586
1587is not the same as
1588
1589 / ( A (*PRUNE) B | C (*PRUNE) D ) /
1590
1591as after matching the A but failing on the B the C<(*THEN)> verb will
1592backtrack and try C; but the C<(*PRUNE)> verb will simply fail.
24b23f37 1593
e2e6a0f1
YO
1594=item C<(*COMMIT)>
1595X<(*COMMIT)>
24b23f37 1596
241e7389 1597This is the Perl 6 "commit pattern" C<< <commit> >> or C<:::>. It's a
5d458dd8
YO
1598zero-width pattern similar to C<(*SKIP)>, except that when backtracked
1599into on failure it causes the match to fail outright. No further attempts
1600to find a valid match by advancing the start pointer will occur again.
1601For example,
24b23f37 1602
e2e6a0f1 1603 'aaabaaab' =~ /a+b?(*COMMIT)(?{print "$&\n"; $count++})(*FAIL)/;
24b23f37
YO
1604 print "Count=$count\n";
1605
1606outputs
1607
1608 aaab
1609 Count=1
1610
e2e6a0f1
YO
1611In other words, once the C<(*COMMIT)> has been entered, and if the pattern
1612does not match, the regex engine will not try any further matching on the
1613rest of the string.
c277df42 1614
e2e6a0f1 1615=back
9af228c6 1616
e2e6a0f1 1617=item Verbs without an argument
9af228c6
YO
1618
1619=over 4
1620
e2e6a0f1
YO
1621=item C<(*FAIL)> C<(*F)>
1622X<(*FAIL)> X<(*F)>
9af228c6 1623
e2e6a0f1
YO
1624This pattern matches nothing and always fails. It can be used to force the
1625engine to backtrack. It is equivalent to C<(?!)>, but easier to read. In
1626fact, C<(?!)> gets optimised into C<(*FAIL)> internally.
9af228c6 1627
e2e6a0f1 1628It is probably useful only when combined with C<(?{})> or C<(??{})>.
9af228c6 1629
e2e6a0f1
YO
1630=item C<(*ACCEPT)>
1631X<(*ACCEPT)>
9af228c6 1632
e2e6a0f1
YO
1633B<WARNING:> This feature is highly experimental. It is not recommended
1634for production code.
9af228c6 1635
e2e6a0f1
YO
1636This pattern matches nothing and causes the end of successful matching at
1637the point at which the C<(*ACCEPT)> pattern was encountered, regardless of
1638whether there is actually more to match in the string. When inside of a
0d017f4d 1639nested pattern, such as recursion, or in a subpattern dynamically generated
e2e6a0f1 1640via C<(??{})>, only the innermost pattern is ended immediately.
9af228c6 1641
c27a5cfe 1642If the C<(*ACCEPT)> is inside of capturing groups then the groups are
e2e6a0f1
YO
1643marked as ended at the point at which the C<(*ACCEPT)> was encountered.
1644For instance:
9af228c6 1645
e2e6a0f1 1646 'AB' =~ /(A (A|B(*ACCEPT)|C) D)(E)/x;
9af228c6 1647
e2e6a0f1 1648will match, and C<$1> will be C<AB> and C<$2> will be C<B>, C<$3> will not
0d017f4d 1649be set. If another branch in the inner parentheses were matched, such as in the
e2e6a0f1 1650string 'ACDE', then the C<D> and C<E> would have to be matched as well.
9af228c6
YO
1651
1652=back
c277df42 1653
a0d0e21e
LW
1654=back
1655
c07a80fd 1656=head2 Backtracking
d74e8afc 1657X<backtrack> X<backtracking>
c07a80fd 1658
35a734be
IZ
1659NOTE: This section presents an abstract approximation of regular
1660expression behavior. For a more rigorous (and complicated) view of
1661the rules involved in selecting a match among possible alternatives,
0d017f4d 1662see L<Combining RE Pieces>.
35a734be 1663
c277df42 1664A fundamental feature of regular expression matching involves the
5a964f20 1665notion called I<backtracking>, which is currently used (when needed)
0d017f4d 1666by all regular non-possessive expression quantifiers, namely C<*>, C<*?>, C<+>,
9da458fc
IZ
1667C<+?>, C<{n,m}>, and C<{n,m}?>. Backtracking is often optimized
1668internally, but the general principle outlined here is valid.
c07a80fd
PP
1669
1670For a regular expression to match, the I<entire> regular expression must
1671match, not just part of it. So if the beginning of a pattern containing a
1672quantifier succeeds in a way that causes later parts in the pattern to
1673fail, the matching engine backs up and recalculates the beginning
1674part--that's why it's called backtracking.
1675
1676Here is an example of backtracking: Let's say you want to find the
1677word following "foo" in the string "Food is on the foo table.":
1678
1679 $_ = "Food is on the foo table.";
1680 if ( /\b(foo)\s+(\w+)/i ) {
f793d64a 1681 print "$2 follows $1.\n";
c07a80fd
PP
1682 }
1683
1684When the match runs, the first part of the regular expression (C<\b(foo)>)
1685finds a possible match right at the beginning of the string, and loads up
1686$1 with "Foo". However, as soon as the matching engine sees that there's
1687no whitespace following the "Foo" that it had saved in $1, it realizes its
68dc0745 1688mistake and starts over again one character after where it had the
c07a80fd
PP
1689tentative match. This time it goes all the way until the next occurrence
1690of "foo". The complete regular expression matches this time, and you get
1691the expected output of "table follows foo."
1692
1693Sometimes minimal matching can help a lot. Imagine you'd like to match
1694everything between "foo" and "bar". Initially, you write something
1695like this:
1696
1697 $_ = "The food is under the bar in the barn.";
1698 if ( /foo(.*)bar/ ) {
f793d64a 1699 print "got <$1>\n";
c07a80fd
PP
1700 }
1701
1702Which perhaps unexpectedly yields:
1703
1704 got <d is under the bar in the >
1705
1706That's because C<.*> was greedy, so you get everything between the
14218588 1707I<first> "foo" and the I<last> "bar". Here it's more effective
c07a80fd
PP
1708to use minimal matching to make sure you get the text between a "foo"
1709and the first "bar" thereafter.
1710
1711 if ( /foo(.*?)bar/ ) { print "got <$1>\n" }
1712 got <d is under the >
1713
0d017f4d 1714Here's another example. Let's say you'd like to match a number at the end
b6e13d97 1715of a string, and you also want to keep the preceding part of the match.
c07a80fd
PP
1716So you write this:
1717
1718 $_ = "I have 2 numbers: 53147";
f793d64a
KW
1719 if ( /(.*)(\d*)/ ) { # Wrong!
1720 print "Beginning is <$1>, number is <$2>.\n";
c07a80fd
PP
1721 }
1722
1723That won't work at all, because C<.*> was greedy and gobbled up the
1724whole string. As C<\d*> can match on an empty string the complete
1725regular expression matched successfully.
1726
8e1088bc 1727 Beginning is <I have 2 numbers: 53147>, number is <>.
c07a80fd
PP
1728
1729Here are some variants, most of which don't work:
1730
1731 $_ = "I have 2 numbers: 53147";
1732 @pats = qw{
f793d64a
KW
1733 (.*)(\d*)
1734 (.*)(\d+)
1735 (.*?)(\d*)
1736 (.*?)(\d+)
1737 (.*)(\d+)$
1738 (.*?)(\d+)$
1739 (.*)\b(\d+)$
1740 (.*\D)(\d+)$
c07a80fd
PP
1741 };
1742
1743 for $pat (@pats) {
f793d64a
KW
1744 printf "%-12s ", $pat;
1745 if ( /$pat/ ) {
1746 print "<$1> <$2>\n";
1747 } else {
1748 print "FAIL\n";
1749 }
c07a80fd
PP
1750 }
1751
1752That will print out:
1753
1754 (.*)(\d*) <I have 2 numbers: 53147> <>
1755 (.*)(\d+) <I have 2 numbers: 5314> <7>
1756 (.*?)(\d*) <> <>
1757 (.*?)(\d+) <I have > <2>
1758 (.*)(\d+)$ <I have 2 numbers: 5314> <7>
1759 (.*?)(\d+)$ <I have 2 numbers: > <53147>
1760 (.*)\b(\d+)$ <I have 2 numbers: > <53147>
1761 (.*\D)(\d+)$ <I have 2 numbers: > <53147>
1762
1763As you see, this can be a bit tricky. It's important to realize that a
1764regular expression is merely a set of assertions that gives a definition
1765of success. There may be 0, 1, or several different ways that the
1766definition might succeed against a particular string. And if there are
5a964f20
TC
1767multiple ways it might succeed, you need to understand backtracking to
1768know which variety of success you will achieve.
c07a80fd 1769
19799a22 1770When using look-ahead assertions and negations, this can all get even
8b19b778 1771trickier. Imagine you'd like to find a sequence of non-digits not
c07a80fd
PP
1772followed by "123". You might try to write that as
1773
871b0233 1774 $_ = "ABC123";
f793d64a
KW
1775 if ( /^\D*(?!123)/ ) { # Wrong!
1776 print "Yup, no 123 in $_\n";
871b0233 1777 }
c07a80fd
PP
1778
1779But that isn't going to match; at least, not the way you're hoping. It
1780claims that there is no 123 in the string. Here's a clearer picture of
9b9391b2 1781why that pattern matches, contrary to popular expectations:
c07a80fd 1782
4358a253
SS
1783 $x = 'ABC123';
1784 $y = 'ABC445';
c07a80fd 1785
4358a253
SS
1786 print "1: got $1\n" if $x =~ /^(ABC)(?!123)/;
1787 print "2: got $1\n" if $y =~ /^(ABC)(?!123)/;
c07a80fd 1788
4358a253
SS
1789 print "3: got $1\n" if $x =~ /^(\D*)(?!123)/;
1790 print "4: got $1\n" if $y =~ /^(\D*)(?!123)/;
c07a80fd
PP
1791
1792This prints
1793
1794 2: got ABC
1795 3: got AB
1796 4: got ABC
1797
5f05dabc 1798You might have expected test 3 to fail because it seems to a more
c07a80fd
PP
1799general purpose version of test 1. The important difference between
1800them is that test 3 contains a quantifier (C<\D*>) and so can use
1801backtracking, whereas test 1 will not. What's happening is
1802that you've asked "Is it true that at the start of $x, following 0 or more
5f05dabc 1803non-digits, you have something that's not 123?" If the pattern matcher had
c07a80fd 1804let C<\D*> expand to "ABC", this would have caused the whole pattern to
54310121 1805fail.
14218588 1806
c07a80fd 1807The search engine will initially match C<\D*> with "ABC". Then it will
14218588 1808try to match C<(?!123> with "123", which fails. But because
c07a80fd
PP
1809a quantifier (C<\D*>) has been used in the regular expression, the
1810search engine can backtrack and retry the match differently
54310121 1811in the hope of matching the complete regular expression.
c07a80fd 1812
5a964f20
TC
1813The pattern really, I<really> wants to succeed, so it uses the
1814standard pattern back-off-and-retry and lets C<\D*> expand to just "AB" this
c07a80fd 1815time. Now there's indeed something following "AB" that is not
14218588 1816"123". It's "C123", which suffices.
c07a80fd 1817
14218588
GS
1818We can deal with this by using both an assertion and a negation.
1819We'll say that the first part in $1 must be followed both by a digit
1820and by something that's not "123". Remember that the look-aheads
1821are zero-width expressions--they only look, but don't consume any
1822of the string in their match. So rewriting this way produces what
c07a80fd
PP
1823you'd expect; that is, case 5 will fail, but case 6 succeeds:
1824
4358a253
SS
1825 print "5: got $1\n" if $x =~ /^(\D*)(?=\d)(?!123)/;
1826 print "6: got $1\n" if $y =~ /^(\D*)(?=\d)(?!123)/;
c07a80fd
PP
1827
1828 6: got ABC
1829
5a964f20 1830In other words, the two zero-width assertions next to each other work as though
19799a22 1831they're ANDed together, just as you'd use any built-in assertions: C</^$/>
c07a80fd
PP
1832matches only if you're at the beginning of the line AND the end of the
1833line simultaneously. The deeper underlying truth is that juxtaposition in
1834regular expressions always means AND, except when you write an explicit OR
1835using the vertical bar. C</ab/> means match "a" AND (then) match "b",
1836although the attempted matches are made at different positions because "a"
1837is not a zero-width assertion, but a one-width assertion.
1838
0d017f4d 1839B<WARNING>: Particularly complicated regular expressions can take
14218588 1840exponential time to solve because of the immense number of possible
0d017f4d 1841ways they can use backtracking to try for a match. For example, without
9da458fc
IZ
1842internal optimizations done by the regular expression engine, this will
1843take a painfully long time to run:
c07a80fd 1844
e1901655
IZ
1845 'aaaaaaaaaaaa' =~ /((a{0,5}){0,5})*[c]/
1846
1847And if you used C<*>'s in the internal groups instead of limiting them
1848to 0 through 5 matches, then it would take forever--or until you ran
1849out of stack space. Moreover, these internal optimizations are not
1850always applicable. For example, if you put C<{0,5}> instead of C<*>
1851on the external group, no current optimization is applicable, and the
1852match takes a long time to finish.
c07a80fd 1853
9da458fc
IZ
1854A powerful tool for optimizing such beasts is what is known as an
1855"independent group",
96090e4f 1856which does not backtrack (see L</C<< (?>pattern) >>>). Note also that
9da458fc 1857zero-length look-ahead/look-behind assertions will not backtrack to make
5d458dd8 1858the tail match, since they are in "logical" context: only
14218588 1859whether they match is considered relevant. For an example
9da458fc 1860where side-effects of look-ahead I<might> have influenced the
96090e4f 1861following match, see L</C<< (?>pattern) >>>.
c277df42 1862
a0d0e21e 1863=head2 Version 8 Regular Expressions
d74e8afc 1864X<regular expression, version 8> X<regex, version 8> X<regexp, version 8>
a0d0e21e 1865
5a964f20 1866In case you're not familiar with the "regular" Version 8 regex
a0d0e21e
LW
1867routines, here are the pattern-matching rules not described above.
1868
54310121 1869Any single character matches itself, unless it is a I<metacharacter>
a0d0e21e 1870with a special meaning described here or above. You can cause
5a964f20 1871characters that normally function as metacharacters to be interpreted
5f05dabc 1872literally by prefixing them with a "\" (e.g., "\." matches a ".", not any
0d017f4d
WL
1873character; "\\" matches a "\"). This escape mechanism is also required
1874for the character used as the pattern delimiter.
1875
1876A series of characters matches that series of characters in the target
1877string, so the pattern C<blurfl> would match "blurfl" in the target
1878string.
a0d0e21e
LW
1879
1880You can specify a character class, by enclosing a list of characters
5d458dd8 1881in C<[]>, which will match any character from the list. If the
a0d0e21e 1882first character after the "[" is "^", the class matches any character not
14218588 1883in the list. Within a list, the "-" character specifies a
5a964f20 1884range, so that C<a-z> represents all characters between "a" and "z",
8a4f6ac2
GS
1885inclusive. If you want either "-" or "]" itself to be a member of a
1886class, put it at the start of the list (possibly after a "^"), or
1887escape it with a backslash. "-" is also taken literally when it is
1888at the end of the list, just before the closing "]". (The
84850974
DD
1889following all specify the same class of three characters: C<[-az]>,
1890C<[az-]>, and C<[a\-z]>. All are different from C<[a-z]>, which
5d458dd8
YO
1891specifies a class containing twenty-six characters, even on EBCDIC-based
1892character sets.) Also, if you try to use the character
1893classes C<\w>, C<\W>, C<\s>, C<\S>, C<\d>, or C<\D> as endpoints of
1894a range, the "-" is understood literally.
a0d0e21e 1895
8ada0baa
JH
1896Note also that the whole range idea is rather unportable between
1897character sets--and even within character sets they may cause results
1898you probably didn't expect. A sound principle is to use only ranges
0d017f4d 1899that begin from and end at either alphabetics of equal case ([a-e],
8ada0baa
JH
1900[A-E]), or digits ([0-9]). Anything else is unsafe. If in doubt,
1901spell out the character sets in full.
1902
54310121 1903Characters may be specified using a metacharacter syntax much like that
a0d0e21e
LW
1904used in C: "\n" matches a newline, "\t" a tab, "\r" a carriage return,
1905"\f" a form feed, etc. More generally, \I<nnn>, where I<nnn> is a string
dc0d9c48 1906of three octal digits, matches the character whose coded character set value
5d458dd8 1907is I<nnn>. Similarly, \xI<nn>, where I<nn> are hexadecimal digits,
dc0d9c48 1908matches the character whose ordinal is I<nn>. The expression \cI<x>
5d458dd8 1909matches the character control-I<x>. Finally, the "." metacharacter
fb55449c 1910matches any character except "\n" (unless you use C</s>).
a0d0e21e
LW
1911
1912You can specify a series of alternatives for a pattern using "|" to
1913separate them, so that C<fee|fie|foe> will match any of "fee", "fie",
5a964f20 1914or "foe" in the target string (as would C<f(e|i|o)e>). The
a0d0e21e
LW
1915first alternative includes everything from the last pattern delimiter
1916("(", "[", or the beginning of the pattern) up to the first "|", and
1917the last alternative contains everything from the last "|" to the next
14218588
GS
1918pattern delimiter. That's why it's common practice to include
1919alternatives in parentheses: to minimize confusion about where they
a3cb178b
GS
1920start and end.
1921
5a964f20 1922Alternatives are tried from left to right, so the first
a3cb178b
GS
1923alternative found for which the entire expression matches, is the one that
1924is chosen. This means that alternatives are not necessarily greedy. For
628afcb5 1925example: when matching C<foo|foot> against "barefoot", only the "foo"
a3cb178b
GS
1926part will match, as that is the first alternative tried, and it successfully
1927matches the target string. (This might not seem important, but it is
1928important when you are capturing matched text using parentheses.)
1929
5a964f20 1930Also remember that "|" is interpreted as a literal within square brackets,
a3cb178b 1931so if you write C<[fee|fie|foe]> you're really only matching C<[feio|]>.
a0d0e21e 1932
14218588
GS
1933Within a pattern, you may designate subpatterns for later reference
1934by enclosing them in parentheses, and you may refer back to the
1935I<n>th subpattern later in the pattern using the metacharacter
1936\I<n>. Subpatterns are numbered based on the left to right order
1937of their opening parenthesis. A backreference matches whatever
1938actually matched the subpattern in the string being examined, not
d8b950dc 1939the rules for that subpattern. Therefore, C<(0|0x)\d*\s\g1\d*> will
14218588
GS
1940match "0x1234 0x4321", but not "0x1234 01234", because subpattern
19411 matched "0x", even though the rule C<0|0x> could potentially match
1942the leading 0 in the second number.
cb1a09d0 1943
0d017f4d 1944=head2 Warning on \1 Instead of $1
cb1a09d0 1945
5a964f20 1946Some people get too used to writing things like:
cb1a09d0
AD
1947
1948 $pattern =~ s/(\W)/\\\1/g;
1949
3ff1c45a
KW
1950This is grandfathered (for \1 to \9) for the RHS of a substitute to avoid
1951shocking the
cb1a09d0 1952B<sed> addicts, but it's a dirty habit to get into. That's because in
d1be9408 1953PerlThink, the righthand side of an C<s///> is a double-quoted string. C<\1> in
cb1a09d0
AD
1954the usual double-quoted string means a control-A. The customary Unix
1955meaning of C<\1> is kludged in for C<s///>. However, if you get into the habit
1956of doing that, you get yourself into trouble if you then add an C</e>
1957modifier.
1958
f793d64a 1959 s/(\d+)/ \1 + 1 /eg; # causes warning under -w
cb1a09d0
AD
1960
1961Or if you try to do
1962
1963 s/(\d+)/\1000/;
1964
1965You can't disambiguate that by saying C<\{1}000>, whereas you can fix it with
14218588 1966C<${1}000>. The operation of interpolation should not be confused
cb1a09d0
AD
1967with the operation of matching a backreference. Certainly they mean two
1968different things on the I<left> side of the C<s///>.
9fa51da4 1969
0d017f4d 1970=head2 Repeated Patterns Matching a Zero-length Substring
c84d73f1 1971
19799a22 1972B<WARNING>: Difficult material (and prose) ahead. This section needs a rewrite.
c84d73f1
IZ
1973
1974Regular expressions provide a terse and powerful programming language. As
1975with most other power tools, power comes together with the ability
1976to wreak havoc.
1977
1978A common abuse of this power stems from the ability to make infinite
628afcb5 1979loops using regular expressions, with something as innocuous as:
c84d73f1
IZ
1980
1981 'foo' =~ m{ ( o? )* }x;
1982
0d017f4d 1983The C<o?> matches at the beginning of C<'foo'>, and since the position
c84d73f1 1984in the string is not moved by the match, C<o?> would match again and again
527e91da 1985because of the C<*> quantifier. Another common way to create a similar cycle
c84d73f1
IZ
1986is with the looping modifier C<//g>:
1987
1988 @matches = ( 'foo' =~ m{ o? }xg );
1989
1990or
1991
1992 print "match: <$&>\n" while 'foo' =~ m{ o? }xg;
1993
1994or the loop implied by split().
1995
1996However, long experience has shown that many programming tasks may
14218588
GS
1997be significantly simplified by using repeated subexpressions that
1998may match zero-length substrings. Here's a simple example being:
c84d73f1 1999
f793d64a 2000 @chars = split //, $string; # // is not magic in split
c84d73f1
IZ
2001 ($whitewashed = $string) =~ s/()/ /g; # parens avoid magic s// /
2002
9da458fc 2003Thus Perl allows such constructs, by I<forcefully breaking
c84d73f1 2004the infinite loop>. The rules for this are different for lower-level
527e91da 2005loops given by the greedy quantifiers C<*+{}>, and for higher-level
c84d73f1
IZ
2006ones like the C</g> modifier or split() operator.
2007
19799a22
GS
2008The lower-level loops are I<interrupted> (that is, the loop is
2009broken) when Perl detects that a repeated expression matched a
2010zero-length substring. Thus
c84d73f1
IZ
2011
2012 m{ (?: NON_ZERO_LENGTH | ZERO_LENGTH )* }x;
2013
5d458dd8 2014is made equivalent to
c84d73f1 2015
5d458dd8
YO
2016 m{ (?: NON_ZERO_LENGTH )*
2017 |
2018 (?: ZERO_LENGTH )?
c84d73f1
IZ
2019 }x;
2020
2021The higher level-loops preserve an additional state between iterations:
5d458dd8 2022whether the last match was zero-length. To break the loop, the following
c84d73f1 2023match after a zero-length match is prohibited to have a length of zero.
5d458dd8 2024This prohibition interacts with backtracking (see L<"Backtracking">),
c84d73f1
IZ
2025and so the I<second best> match is chosen if the I<best> match is of
2026zero length.
2027
19799a22 2028For example:
c84d73f1
IZ
2029
2030 $_ = 'bar';
2031 s/\w??/<$&>/g;
2032
20fb949f 2033results in C<< <><b><><a><><r><> >>. At each position of the string the best
5d458dd8 2034match given by non-greedy C<??> is the zero-length match, and the I<second
c84d73f1
IZ
2035best> match is what is matched by C<\w>. Thus zero-length matches
2036alternate with one-character-long matches.
2037
5d458dd8 2038Similarly, for repeated C<m/()/g> the second-best match is the match at the
c84d73f1
IZ
2039position one notch further in the string.
2040
19799a22 2041The additional state of being I<matched with zero-length> is associated with
c84d73f1 2042the matched string, and is reset by each assignment to pos().
9da458fc
IZ
2043Zero-length matches at the end of the previous match are ignored
2044during C<split>.
c84d73f1 2045
0d017f4d 2046=head2 Combining RE Pieces
35a734be
IZ
2047
2048Each of the elementary pieces of regular expressions which were described
2049before (such as C<ab> or C<\Z>) could match at most one substring
2050at the given position of the input string. However, in a typical regular
2051expression these elementary pieces are combined into more complicated
2052patterns using combining operators C<ST>, C<S|T>, C<S*> etc
2053(in these examples C<S> and C<T> are regular subexpressions).
2054
2055Such combinations can include alternatives, leading to a problem of choice:
2056if we match a regular expression C<a|ab> against C<"abc">, will it match
2057substring C<"a"> or C<"ab">? One way to describe which substring is
2058actually matched is the concept of backtracking (see L<"Backtracking">).
2059However, this description is too low-level and makes you think
2060in terms of a particular implementation.
2061
2062Another description starts with notions of "better"/"worse". All the
2063substrings which may be matched by the given regular expression can be
2064sorted from the "best" match to the "worst" match, and it is the "best"
2065match which is chosen. This substitutes the question of "what is chosen?"
2066by the question of "which matches are better, and which are worse?".
2067
2068Again, for elementary pieces there is no such question, since at most
2069one match at a given position is possible. This section describes the
2070notion of better/worse for combining operators. In the description
2071below C<S> and C<T> are regular subexpressions.
2072
13a2d996 2073=over 4
35a734be
IZ
2074
2075=item C<ST>
2076
2077Consider two possible matches, C<AB> and C<A'B'>, C<A> and C<A'> are
2078substrings which can be matched by C<S>, C<B> and C<B'> are substrings
5d458dd8 2079which can be matched by C<T>.
35a734be
IZ
2080
2081If C<A> is better match for C<S> than C<A'>, C<AB> is a better
2082match than C<A'B'>.
2083
2084If C<A> and C<A'> coincide: C<AB> is a better match than C<AB'> if
2085C<B> is better match for C<T> than C<B'>.
2086
2087=item C<S|T>
2088
2089When C<S> can match, it is a better match than when only C<T> can match.
2090
2091Ordering of two matches for C<S> is the same as for C<S>. Similar for
2092two matches for C<T>.
2093
2094=item C<S{REPEAT_COUNT}>
2095
2096Matches as C<SSS...S> (repeated as many times as necessary).
2097
2098=item C<S{min,max}>
2099
2100Matches as C<S{max}|S{max-1}|...|S{min+1}|S{min}>.
2101
2102=item C<S{min,max}?>
2103
2104Matches as C<S{min}|S{min+1}|...|S{max-1}|S{max}>.
2105
2106=item C<S?>, C<S*>, C<S+>
2107
2108Same as C<S{0,1}>, C<S{0,BIG_NUMBER}>, C<S{1,BIG_NUMBER}> respectively.
2109
2110=item C<S??>, C<S*?>, C<S+?>
2111
2112Same as C<S{0,1}?>, C<S{0,BIG_NUMBER}?>, C<S{1,BIG_NUMBER}?> respectively.
2113
c47ff5f1 2114=item C<< (?>S) >>
35a734be
IZ
2115
2116Matches the best match for C<S> and only that.
2117
2118=item C<(?=S)>, C<(?<=S)>
2119
2120Only the best match for C<S> is considered. (This is important only if
2121C<S> has capturing parentheses, and backreferences are used somewhere
2122else in the whole regular expression.)
2123
2124=item C<(?!S)>, C<(?<!S)>
2125
2126For this grouping operator there is no need to describe the ordering, since
2127only whether or not C<S> can match is important.
2128
6bda09f9 2129=item C<(??{ EXPR })>, C<(?PARNO)>
35a734be
IZ
2130
2131The ordering is the same as for the regular expression which is
c27a5cfe 2132the result of EXPR, or the pattern contained by capture group PARNO.
35a734be
IZ
2133
2134=item C<(?(condition)yes-pattern|no-pattern)>
2135
2136Recall that which of C<yes-pattern> or C<no-pattern> actually matches is
2137already determined. The ordering of the matches is the same as for the
2138chosen subexpression.
2139
2140=back
2141
2142The above recipes describe the ordering of matches I<at a given position>.
2143One more rule is needed to understand how a match is determined for the
2144whole regular expression: a match at an earlier position is always better
2145than a match at a later position.
2146
0d017f4d 2147=head2 Creating Custom RE Engines
c84d73f1
IZ
2148
2149Overloaded constants (see L<overload>) provide a simple way to extend
2150the functionality of the RE engine.
2151
2152Suppose that we want to enable a new RE escape-sequence C<\Y|> which
0d017f4d 2153matches at a boundary between whitespace characters and non-whitespace
c84d73f1
IZ
2154characters. Note that C<(?=\S)(?<!\S)|(?!\S)(?<=\S)> matches exactly
2155at these positions, so we want to have each C<\Y|> in the place of the
2156more complicated version. We can create a module C<customre> to do
2157this:
2158
2159 package customre;
2160 use overload;
2161
2162 sub import {
2163 shift;
2164 die "No argument to customre::import allowed" if @_;
2165 overload::constant 'qr' => \&convert;
2166 }
2167
2168 sub invalid { die "/$_[0]/: invalid escape '\\$_[1]'"}
2169
580a9fe1
RGS
2170 # We must also take care of not escaping the legitimate \\Y|
2171 # sequence, hence the presence of '\\' in the conversion rules.
5d458dd8 2172 my %rules = ( '\\' => '\\\\',
f793d64a 2173 'Y|' => qr/(?=\S)(?<!\S)|(?!\S)(?<=\S)/ );
c84d73f1
IZ
2174 sub convert {
2175 my $re = shift;
5d458dd8 2176 $re =~ s{
c84d73f1
IZ
2177 \\ ( \\ | Y . )
2178 }
5d458dd8 2179 { $rules{$1} or invalid($re,$1) }sgex;
c84d73f1
IZ
2180 return $re;
2181 }
2182
2183Now C<use customre> enables the new escape in constant regular
2184expressions, i.e., those without any runtime variable interpolations.
2185As documented in L<overload>, this conversion will work only over
2186literal parts of regular expressions. For C<\Y|$re\Y|> the variable
2187part of this regular expression needs to be converted explicitly
2188(but only if the special meaning of C<\Y|> should be enabled inside $re):
2189
2190 use customre;
2191 $re = <>;
2192 chomp $re;
2193 $re = customre::convert $re;
2194 /\Y|$re\Y|/;
2195
1f1031fe
YO
2196=head1 PCRE/Python Support
2197
99d59c4d 2198As of Perl 5.10.0, Perl supports several Python/PCRE specific extensions
1f1031fe 2199to the regex syntax. While Perl programmers are encouraged to use the
99d59c4d 2200Perl specific syntax, the following are also accepted:
1f1031fe
YO
2201
2202=over 4
2203
ae5648b3 2204=item C<< (?PE<lt>NAMEE<gt>pattern) >>
1f1031fe 2205
c27a5cfe 2206Define a named capture group. Equivalent to C<< (?<NAME>pattern) >>.
1f1031fe
YO
2207
2208=item C<< (?P=NAME) >>
2209
c27a5cfe 2210Backreference to a named capture group. Equivalent to C<< \g{NAME} >>.
1f1031fe
YO
2211
2212=item C<< (?P>NAME) >>
2213
c27a5cfe 2214Subroutine call to a named capture group. Equivalent to C<< (?&NAME) >>.
1f1031fe 2215
ee9b8eae 2216=back
1f1031fe 2217
19799a22
GS
2218=head1 BUGS
2219
78288b8e
KW
2220There are numerous problems with case insensitive matching of characters
2221outside the ASCII range, especially with those whose folds are multiple
2222characters, such as ligatures like C<LATIN SMALL LIGATURE FF>.
2223
f253210b
KW
2224In a bracketed character class with case insensitive matching, ranges only work
2225for ASCII characters. For example,
2226C<m/[\N{CYRILLIC CAPITAL LETTER A}-\N{CYRILLIC CAPITAL LETTER YA}]/i>
2227doesn't match all the Russian upper and lower case letters.
2228
88c9975e
KW
2229Many regular expression constructs don't work on EBCDIC platforms.
2230
9da458fc
IZ
2231This document varies from difficult to understand to completely
2232and utterly opaque. The wandering prose riddled with jargon is
2233hard to fathom in several places.
2234
2235This document needs a rewrite that separates the tutorial content
2236from the reference content.
19799a22
GS
2237
2238=head1 SEE ALSO
9fa51da4 2239
91e0c79e
MJD
2240L<perlrequick>.
2241
2242L<perlretut>.
2243
9b599b2a
GS
2244L<perlop/"Regexp Quote-Like Operators">.
2245
1e66bd83
PP
2246L<perlop/"Gory details of parsing quoted constructs">.
2247
14218588
GS
2248L<perlfaq6>.
2249
9b599b2a
GS
2250L<perlfunc/pos>.
2251
2252L<perllocale>.
2253
fb55449c
JH
2254L<perlebcdic>.
2255
14218588
GS
2256I<Mastering Regular Expressions> by Jeffrey Friedl, published
2257by O'Reilly and Associates.