This is a live mirror of the Perl 5 development currently hosted at https://github.com/perl/perl5
Fix a2p translation of '{print "a" "b" "c"}'
[perl5.git] / pod / perlre.pod
CommitLineData
a0d0e21e
LW
1=head1 NAME
2
3perlre - Perl regular expressions
4
5=head1 DESCRIPTION
6
cb1a09d0
AD
7This page describes the syntax of regular expressions in Perl. For a
8description of how to actually I<use> regular expressions in matching
9operations, plus various examples of the same, see C<m//> and C<s///> in
10L<perlop>.
11
12The matching operations can
a0d0e21e
LW
13have various modifiers, some of which relate to the interpretation of
14the regular expression inside. These are:
15
16 i Do case-insensitive pattern matching.
17 m Treat string as multiple lines.
18 s Treat string as single line.
c07a80fd 19 x Extend your pattern's legibility with whitespace and comments.
a0d0e21e
LW
20
21These are usually written as "the C</x> modifier", even though the delimiter
22in question might not actually be a slash. In fact, any of these
23modifiers may also be embedded within the regular expression itself using
24the new C<(?...)> construct. See below.
25
4633a7c4
LW
26The C</x> modifier itself needs a little more explanation. It tells
27the regular expression parser to ignore whitespace that is not
28backslashed or within a character class. You can use this to break up
29your regular expression into (slightly) more readable parts. The C<#>
30character is also treated as a metacharacter introducing a comment,
31just as in ordinary Perl code. Taken together, these features go a
32long way towards making Perl 5 a readable language. See the C comment
a0d0e21e
LW
33deletion code in L<perlop>.
34
35=head2 Regular Expressions
36
37The patterns used in pattern matching are regular expressions such as
38those supplied in the Version 8 regexp routines. (In fact, the
39routines are derived (distantly) from Henry Spencer's freely
40redistributable reimplementation of the V8 routines.)
41See L<Version 8 Regular Expressions> for details.
42
43In particular the following metacharacters have their standard I<egrep>-ish
44meanings:
45
46 \ Quote the next metacharacter
47 ^ Match the beginning of the line
48 . Match any character (except newline)
c07a80fd 49 $ Match the end of the line (or before newline at the end)
a0d0e21e
LW
50 | Alternation
51 () Grouping
52 [] Character class
53
54By default, the "^" character is guaranteed to match only at the
55beginning of the string, the "$" character only at the end (or before the
56newline at the end) and Perl does certain optimizations with the
57assumption that the string contains only one line. Embedded newlines
58will not be matched by "^" or "$". You may, however, wish to treat a
59string as a multi-line buffer, such that the "^" will match after any
60newline within the string, and "$" will match before any newline. At the
61cost of a little more overhead, you can do this by using the /m modifier
62on the pattern match operator. (Older programs did this by setting C<$*>,
63but this practice is deprecated in Perl 5.)
64
65To facilitate multi-line substitutions, the "." character never matches a
66newline unless you use the C</s> modifier, which tells Perl to pretend
67the string is a single line--even if it isn't. The C</s> modifier also
68overrides the setting of C<$*>, in case you have some (badly behaved) older
69code that sets it in another module.
70
71The following standard quantifiers are recognized:
72
73 * Match 0 or more times
74 + Match 1 or more times
75 ? Match 1 or 0 times
76 {n} Match exactly n times
77 {n,} Match at least n times
78 {n,m} Match at least n but not more than m times
79
80(If a curly bracket occurs in any other context, it is treated
81as a regular character.) The "*" modifier is equivalent to C<{0,}>, the "+"
25f94b33 82modifier to C<{1,}>, and the "?" modifier to C<{0,1}>. n and m are limited
c07a80fd 83to integral values less than 65536.
a0d0e21e
LW
84
85By default, a quantified subpattern is "greedy", that is, it will match as
32fd1c90
PP
86many times as possible without causing the rest of the pattern not to match.
87The standard quantifiers are all "greedy", in that they match as many
a0d0e21e
LW
88occurrences as possible (given a particular starting location) without
89causing the pattern to fail. If you want it to match the minimum number
90of times possible, follow the quantifier with a "?" after any of them.
91Note that the meanings don't change, just the "gravity":
92
93 *? Match 0 or more times
94 +? Match 1 or more times
95 ?? Match 0 or 1 time
96 {n}? Match exactly n times
97 {n,}? Match at least n times
98 {n,m}? Match at least n but not more than m times
99
100Since patterns are processed as double quoted strings, the following
101also work:
102
0f36ee90
PP
103 \t tab (HT, TAB)
104 \n newline (LF, NL)
105 \r return (CR)
106 \f form feed (FF)
107 \a alarm (bell) (BEL)
108 \e escape (think troff) (ESC)
cb1a09d0
AD
109 \033 octal char (think of a PDP-11)
110 \x1B hex char
a0d0e21e 111 \c[ control char
cb1a09d0
AD
112 \l lowercase next char (think vi)
113 \u uppercase next char (think vi)
114 \L lowercase till \E (think vi)
115 \U uppercase till \E (think vi)
116 \E end case modification (think vi)
a0d0e21e
LW
117 \Q quote regexp metacharacters till \E
118
119In addition, Perl defines the following:
120
121 \w Match a "word" character (alphanumeric plus "_")
122 \W Match a non-word character
123 \s Match a whitespace character
124 \S Match a non-whitespace character
125 \d Match a digit character
126 \D Match a non-digit character
127
128Note that C<\w> matches a single alphanumeric character, not a whole
cb1a09d0
AD
129word. To match a word you'd need to say C<\w+>. You may use C<\w>,
130C<\W>, C<\s>, C<\S>, C<\d> and C<\D> within character classes (though not
131as either end of a range).
a0d0e21e
LW
132
133Perl defines the following zero-width assertions:
134
135 \b Match a word boundary
136 \B Match a non-(word boundary)
137 \A Match only at beginning of string
c07a80fd 138 \Z Match only at end of string (or before newline at the end)
a0d0e21e
LW
139 \G Match only where previous m//g left off
140
141A word boundary (C<\b>) is defined as a spot between two characters that
142has a C<\w> on one side of it and and a C<\W> on the other side of it (in
143either order), counting the imaginary characters off the beginning and
144end of the string as matching a C<\W>. (Within character classes C<\b>
145represents backspace rather than a word boundary.) The C<\A> and C<\Z> are
146just like "^" and "$" except that they won't match multiple times when the
147C</m> modifier is used, while "^" and "$" will match at every internal line
c07a80fd
PP
148boundary. To match the actual end of the string, not ignoring newline,
149you can use C<\Z(?!\n)>.
a0d0e21e 150
0f36ee90 151When the bracketing construct C<( ... )> is used, \E<lt>digitE<gt> matches the
cb1a09d0 152digit'th substring. Outside of the pattern, always use "$" instead of "\"
0f36ee90 153in front of the digit. (While the \E<lt>digitE<gt> notation can on rare occasion work
cb1a09d0 154outside the current pattern, this should not be relied upon. See the
0f36ee90 155WARNING below.) The scope of $E<lt>digitE<gt> (and C<$`>, C<$&>, and C<$'>)
cb1a09d0
AD
156extends to the end of the enclosing BLOCK or eval string, or to the next
157successful pattern match, whichever comes first. If you want to use
32fd1c90 158parentheses to delimit a subpattern (e.g. a set of alternatives) without
84dc3c4d 159saving it as a subpattern, follow the ( with a ?:.
cb1a09d0
AD
160
161You may have as many parentheses as you wish. If you have more
a0d0e21e
LW
162than 9 substrings, the variables $10, $11, ... refer to the
163corresponding substring. Within the pattern, \10, \11, etc. refer back
164to substrings if there have been at least that many left parens before
c07a80fd 165the backreference. Otherwise (for backward compatibility) \10 is the
a0d0e21e
LW
166same as \010, a backspace, and \11 the same as \011, a tab. And so
167on. (\1 through \9 are always backreferences.)
168
169C<$+> returns whatever the last bracket match matched. C<$&> returns the
0f36ee90 170entire matched string. (C<$0> used to return the same thing, but not any
a0d0e21e
LW
171more.) C<$`> returns everything before the matched string. C<$'> returns
172everything after the matched string. Examples:
173
174 s/^([^ ]*) *([^ ]*)/$2 $1/; # swap first two words
175
176 if (/Time: (..):(..):(..)/) {
177 $hours = $1;
178 $minutes = $2;
179 $seconds = $3;
180 }
181
182You will note that all backslashed metacharacters in Perl are
183alphanumeric, such as C<\b>, C<\w>, C<\n>. Unlike some other regular expression
184languages, there are no backslashed symbols that aren't alphanumeric.
0f36ee90 185So anything that looks like \\, \(, \), \E<lt>, \E<gt>, \{, or \} is always
a0d0e21e
LW
186interpreted as a literal character, not a metacharacter. This makes it
187simple to quote a string that you want to use for a pattern but that
188you are afraid might contain metacharacters. Simply quote all the
189non-alphanumeric characters:
190
191 $pattern =~ s/(\W)/\\$1/g;
192
193You can also use the built-in quotemeta() function to do this.
194An even easier way to quote metacharacters right in the match operator
c07a80fd 195is to say
a0d0e21e
LW
196
197 /$unquoted\Q$quoted\E$unquoted/
198
199Perl 5 defines a consistent extension syntax for regular expressions.
200The syntax is a pair of parens with a question mark as the first thing
201within the parens (this was a syntax error in Perl 4). The character
202after the question mark gives the function of the extension. Several
203extensions are already supported:
204
205=over 10
206
207=item (?#text)
208
cb1a09d0
AD
209A comment. The text is ignored. If the C</x> switch is used to enable
210whitespace formatting, a simple C<#> will suffice.
a0d0e21e
LW
211
212=item (?:regexp)
213
0f36ee90 214This groups things like "()" but doesn't make backreferences like "()" does. So
a0d0e21e
LW
215
216 split(/\b(?:a|b|c)\b/)
217
218is like
219
220 split(/\b(a|b|c)\b/)
221
222but doesn't spit out extra fields.
223
224=item (?=regexp)
225
226A zero-width positive lookahead assertion. For example, C</\w+(?=\t)/>
227matches a word followed by a tab, without including the tab in C<$&>.
228
229=item (?!regexp)
230
231A zero-width negative lookahead assertion. For example C</foo(?!bar)/>
232matches any occurrence of "foo" that isn't followed by "bar". Note
233however that lookahead and lookbehind are NOT the same thing. You cannot
234use this for lookbehind: C</(?!foo)bar/> will not find an occurrence of
235"bar" that is preceded by something which is not "foo". That's because
236the C<(?!foo)> is just saying that the next thing cannot be "foo"--and
237it's not, it's a "bar", so "foobar" will match. You would have to do
0f36ee90 238something like C</(?!foo)...bar/> for that. We say "like" because there's
a0d0e21e 239the case of your "bar" not having three characters before it. You could
c07a80fd 240cover that this way: C</(?:(?!foo)...|^..?)bar/>. Sometimes it's still
a0d0e21e
LW
241easier just to say:
242
c07a80fd 243 if (/foo/ && $` =~ /bar$/)
a0d0e21e
LW
244
245
246=item (?imsx)
247
248One or more embedded pattern-match modifiers. This is particularly
249useful for patterns that are specified in a table somewhere, some of
250which want to be case sensitive, and some of which don't. The case
251insensitive ones merely need to include C<(?i)> at the front of the
252pattern. For example:
253
254 $pattern = "foobar";
c07a80fd 255 if ( /$pattern/i )
a0d0e21e
LW
256
257 # more flexible:
258
259 $pattern = "(?i)foobar";
c07a80fd 260 if ( /$pattern/ )
a0d0e21e
LW
261
262=back
263
264The specific choice of question mark for this and the new minimal
265matching construct was because 1) question mark is pretty rare in older
266regular expressions, and 2) whenever you see one, you should stop
267and "question" exactly what is going on. That's psychology...
268
c07a80fd
PP
269=head2 Backtracking
270
271A fundamental feature of regular expression matching involves the notion
272called I<backtracking>. which is used (when needed) by all regular
273expression quantifiers, namely C<*>, C<*?>, C<+>, C<+?>, C<{n,m}>, and
274C<{n,m}?>.
275
276For a regular expression to match, the I<entire> regular expression must
277match, not just part of it. So if the beginning of a pattern containing a
278quantifier succeeds in a way that causes later parts in the pattern to
279fail, the matching engine backs up and recalculates the beginning
280part--that's why it's called backtracking.
281
282Here is an example of backtracking: Let's say you want to find the
283word following "foo" in the string "Food is on the foo table.":
284
285 $_ = "Food is on the foo table.";
286 if ( /\b(foo)\s+(\w+)/i ) {
287 print "$2 follows $1.\n";
288 }
289
290When the match runs, the first part of the regular expression (C<\b(foo)>)
291finds a possible match right at the beginning of the string, and loads up
292$1 with "Foo". However, as soon as the matching engine sees that there's
293no whitespace following the "Foo" that it had saved in $1, it realizes its
294mistake and starts over again one character after where it had had the
295tentative match. This time it goes all the way until the next occurrence
296of "foo". The complete regular expression matches this time, and you get
297the expected output of "table follows foo."
298
299Sometimes minimal matching can help a lot. Imagine you'd like to match
300everything between "foo" and "bar". Initially, you write something
301like this:
302
303 $_ = "The food is under the bar in the barn.";
304 if ( /foo(.*)bar/ ) {
305 print "got <$1>\n";
306 }
307
308Which perhaps unexpectedly yields:
309
310 got <d is under the bar in the >
311
312That's because C<.*> was greedy, so you get everything between the
313I<first> "foo" and the I<last> "bar". In this case, it's more effective
314to use minimal matching to make sure you get the text between a "foo"
315and the first "bar" thereafter.
316
317 if ( /foo(.*?)bar/ ) { print "got <$1>\n" }
318 got <d is under the >
319
320Here's another example: let's say you'd like to match a number at the end
321of a string, and you also want to keep the preceding part the match.
322So you write this:
323
324 $_ = "I have 2 numbers: 53147";
325 if ( /(.*)(\d*)/ ) { # Wrong!
326 print "Beginning is <$1>, number is <$2>.\n";
327 }
328
329That won't work at all, because C<.*> was greedy and gobbled up the
330whole string. As C<\d*> can match on an empty string the complete
331regular expression matched successfully.
332
8e1088bc 333 Beginning is <I have 2 numbers: 53147>, number is <>.
c07a80fd
PP
334
335Here are some variants, most of which don't work:
336
337 $_ = "I have 2 numbers: 53147";
338 @pats = qw{
339 (.*)(\d*)
340 (.*)(\d+)
341 (.*?)(\d*)
342 (.*?)(\d+)
343 (.*)(\d+)$
344 (.*?)(\d+)$
345 (.*)\b(\d+)$
346 (.*\D)(\d+)$
347 };
348
349 for $pat (@pats) {
350 printf "%-12s ", $pat;
351 if ( /$pat/ ) {
352 print "<$1> <$2>\n";
353 } else {
354 print "FAIL\n";
355 }
356 }
357
358That will print out:
359
360 (.*)(\d*) <I have 2 numbers: 53147> <>
361 (.*)(\d+) <I have 2 numbers: 5314> <7>
362 (.*?)(\d*) <> <>
363 (.*?)(\d+) <I have > <2>
364 (.*)(\d+)$ <I have 2 numbers: 5314> <7>
365 (.*?)(\d+)$ <I have 2 numbers: > <53147>
366 (.*)\b(\d+)$ <I have 2 numbers: > <53147>
367 (.*\D)(\d+)$ <I have 2 numbers: > <53147>
368
369As you see, this can be a bit tricky. It's important to realize that a
370regular expression is merely a set of assertions that gives a definition
371of success. There may be 0, 1, or several different ways that the
372definition might succeed against a particular string. And if there are
373multiple ways it might succeed, you need to understand backtracking in
374order to know which variety of success you will achieve.
375
376When using lookahead assertions and negations, this can all get even
377tricker. Imagine you'd like to find a sequence of nondigits not
378followed by "123". You might try to write that as
379
380 $_ = "ABC123";
381 if ( /^\D*(?!123)/ ) { # Wrong!
382 print "Yup, no 123 in $_\n";
383 }
384
385But that isn't going to match; at least, not the way you're hoping. It
386claims that there is no 123 in the string. Here's a clearer picture of
387why it that pattern matches, contrary to popular expectations:
388
389 $x = 'ABC123' ;
390 $y = 'ABC445' ;
391
392 print "1: got $1\n" if $x =~ /^(ABC)(?!123)/ ;
393 print "2: got $1\n" if $y =~ /^(ABC)(?!123)/ ;
394
395 print "3: got $1\n" if $x =~ /^(\D*)(?!123)/ ;
396 print "4: got $1\n" if $y =~ /^(\D*)(?!123)/ ;
397
398This prints
399
400 2: got ABC
401 3: got AB
402 4: got ABC
403
404You might have expected test 3 to fail because it just seems to a more
405general purpose version of test 1. The important difference between
406them is that test 3 contains a quantifier (C<\D*>) and so can use
407backtracking, whereas test 1 will not. What's happening is
408that you've asked "Is it true that at the start of $x, following 0 or more
409nondigits, you have something that's not 123?" If the pattern matcher had
410let C<\D*> expand to "ABC", this would have caused the whole pattern to
411fail.
412The search engine will initially match C<\D*> with "ABC". Then it will
413try to match C<(?!123> with "123" which, of course, fails. But because
414a quantifier (C<\D*>) has been used in the regular expression, the
415search engine can backtrack and retry the match differently
416in the hope of matching the complete regular expression.
417
418Well now,
419the pattern really, I<really> wants to succeed, so it uses the
420standard regexp backoff-and-retry and lets C<\D*> expand to just "AB" this
421time. Now there's indeed something following "AB" that is not
422"123". It's in fact "C123", which suffices.
423
424We can deal with this by using both an assertion and a negation. We'll
425say that the first part in $1 must be followed by a digit, and in fact, it
426must also be followed by something that's not "123". Remember that the
427lookaheads are zero-width expressions--they only look, but don't consume
428any of the string in their match. So rewriting this way produces what
429you'd expect; that is, case 5 will fail, but case 6 succeeds:
430
431 print "5: got $1\n" if $x =~ /^(\D*)(?=\d)(?!123)/ ;
432 print "6: got $1\n" if $y =~ /^(\D*)(?=\d)(?!123)/ ;
433
434 6: got ABC
435
436In other words, the two zero-width assertions next to each other work like
437they're ANDed together, just as you'd use any builtin assertions: C</^$/>
438matches only if you're at the beginning of the line AND the end of the
439line simultaneously. The deeper underlying truth is that juxtaposition in
440regular expressions always means AND, except when you write an explicit OR
441using the vertical bar. C</ab/> means match "a" AND (then) match "b",
442although the attempted matches are made at different positions because "a"
443is not a zero-width assertion, but a one-width assertion.
444
445One warning: particularly complicated regular expressions can take
446exponential time to solve due to the immense number of possible ways they
447can use backtracking to try match. For example this will take a very long
448time to run
449
450 /((a{0,5}){0,5}){0,5}/
451
452And if you used C<*>'s instead of limiting it to 0 through 5 matches, then
453it would take literally forever--or until you ran out of stack space.
454
a0d0e21e
LW
455=head2 Version 8 Regular Expressions
456
457In case you're not familiar with the "regular" Version 8 regexp
458routines, here are the pattern-matching rules not described above.
459
460Any single character matches itself, unless it is a I<metacharacter>
461with a special meaning described here or above. You can cause
462characters which normally function as metacharacters to be interpreted
463literally by prefixing them with a "\" (e.g. "\." matches a ".", not any
464character; "\\" matches a "\"). A series of characters matches that
465series of characters in the target string, so the pattern C<blurfl>
466would match "blurfl" in the target string.
467
468You can specify a character class, by enclosing a list of characters
469in C<[]>, which will match any one of the characters in the list. If the
470first character after the "[" is "^", the class matches any character not
471in the list. Within a list, the "-" character is used to specify a
472range, so that C<a-z> represents all the characters between "a" and "z",
473inclusive.
474
475Characters may be specified using a metacharacter syntax much like that
476used in C: "\n" matches a newline, "\t" a tab, "\r" a carriage return,
477"\f" a form feed, etc. More generally, \I<nnn>, where I<nnn> is a string
478of octal digits, matches the character whose ASCII value is I<nnn>.
0f36ee90 479Similarly, \xI<nn>, where I<nn> are hexadecimal digits, matches the
a0d0e21e
LW
480character whose ASCII value is I<nn>. The expression \cI<x> matches the
481ASCII character control-I<x>. Finally, the "." metacharacter matches any
482character except "\n" (unless you use C</s>).
483
484You can specify a series of alternatives for a pattern using "|" to
485separate them, so that C<fee|fie|foe> will match any of "fee", "fie",
486or "foe" in the target string (as would C<f(e|i|o)e>). Note that the
487first alternative includes everything from the last pattern delimiter
488("(", "[", or the beginning of the pattern) up to the first "|", and
489the last alternative contains everything from the last "|" to the next
490pattern delimiter. For this reason, it's common practice to include
491alternatives in parentheses, to minimize confusion about where they
748a9306
LW
492start and end. Note however that "|" is interpreted as a literal with
493square brackets, so if you write C<[fee|fie|foe]> you're really only
494matching C<[feio|]>.
a0d0e21e
LW
495
496Within a pattern, you may designate subpatterns for later reference by
497enclosing them in parentheses, and you may refer back to the I<n>th
c07a80fd 498subpattern later in the pattern using the metacharacter \I<n>.
a0d0e21e
LW
499Subpatterns are numbered based on the left to right order of their
500opening parenthesis. Note that a backreference matches whatever
501actually matched the subpattern in the string being examined, not the
748a9306 502rules for that subpattern. Therefore, C<(0|0x)\d*\s\1\d*> will
a0d0e21e 503match "0x1234 0x4321",but not "0x1234 01234", since subpattern 1
748a9306 504actually matched "0x", even though the rule C<0|0x> could
a0d0e21e 505potentially match the leading 0 in the second number.
cb1a09d0
AD
506
507=head2 WARNING on \1 vs $1
508
509Some people get too used to writing things like
510
511 $pattern =~ s/(\W)/\\\1/g;
512
513This is grandfathered for the RHS of a substitute to avoid shocking the
514B<sed> addicts, but it's a dirty habit to get into. That's because in
515PerlThink, the right-hand side of a C<s///> is a double-quoted string. C<\1> in
516the usual double-quoted string means a control-A. The customary Unix
517meaning of C<\1> is kludged in for C<s///>. However, if you get into the habit
518of doing that, you get yourself into trouble if you then add an C</e>
519modifier.
520
521 s/(\d+)/ \1 + 1 /eg;
522
523Or if you try to do
524
525 s/(\d+)/\1000/;
526
527You can't disambiguate that by saying C<\{1}000>, whereas you can fix it with
528C<${1}000>. Basically, the operation of interpolation should not be confused
529with the operation of matching a backreference. Certainly they mean two
530different things on the I<left> side of the C<s///>.