Commit | Line | Data |
---|---|---|
a0d0e21e LW |
1 | =head1 NAME |
2 | ||
3 | perlre - Perl regular expressions | |
4 | ||
5 | =head1 DESCRIPTION | |
6 | ||
cb1a09d0 | 7 | This page describes the syntax of regular expressions in Perl. For a |
5f05dabc | 8 | description of how to I<use> regular expressions in matching |
cb1a09d0 AD |
9 | operations, plus various examples of the same, see C<m//> and C<s///> in |
10 | L<perlop>. | |
11 | ||
68dc0745 | 12 | The matching operations can have various modifiers. The modifiers |
13 | which relate to the interpretation of the regular expression inside | |
14 | are listed below. For the modifiers that alter the behaviour of the | |
15 | operation, see L<perlop/"m//"> and L<perlop/"s//">. | |
a0d0e21e | 16 | |
55497cff | 17 | =over 4 |
18 | ||
19 | =item i | |
20 | ||
21 | Do case-insensitive pattern matching. | |
22 | ||
a034a98d DD |
23 | If C<use locale> is in effect, the case map is taken from the current |
24 | locale. See L<perllocale>. | |
25 | ||
54310121 | 26 | =item m |
55497cff | 27 | |
28 | Treat string as multiple lines. That is, change "^" and "$" from matching | |
5f05dabc | 29 | at only the very start or end of the string to the start or end of any |
55497cff | 30 | line anywhere within the string, |
31 | ||
54310121 | 32 | =item s |
55497cff | 33 | |
34 | Treat string as single line. That is, change "." to match any character | |
35 | whatsoever, even a newline, which it normally would not match. | |
36 | ||
54310121 | 37 | =item x |
55497cff | 38 | |
39 | Extend your pattern's legibility by permitting whitespace and comments. | |
40 | ||
41 | =back | |
a0d0e21e LW |
42 | |
43 | These are usually written as "the C</x> modifier", even though the delimiter | |
44 | in question might not actually be a slash. In fact, any of these | |
45 | modifiers may also be embedded within the regular expression itself using | |
46 | the new C<(?...)> construct. See below. | |
47 | ||
4633a7c4 | 48 | The C</x> modifier itself needs a little more explanation. It tells |
55497cff | 49 | the regular expression parser to ignore whitespace that is neither |
50 | backslashed nor within a character class. You can use this to break up | |
4633a7c4 | 51 | your regular expression into (slightly) more readable parts. The C<#> |
54310121 | 52 | character is also treated as a metacharacter introducing a comment, |
55497cff | 53 | just as in ordinary Perl code. This also means that if you want real |
54 | whitespace or C<#> characters in the pattern that you'll have to either | |
55 | escape them or encode them using octal or hex escapes. Taken together, | |
56 | these features go a long way towards making Perl's regular expressions | |
57 | more readable. See the C comment deletion code in L<perlop>. | |
a0d0e21e LW |
58 | |
59 | =head2 Regular Expressions | |
60 | ||
61 | The patterns used in pattern matching are regular expressions such as | |
62 | those supplied in the Version 8 regexp routines. (In fact, the | |
63 | routines are derived (distantly) from Henry Spencer's freely | |
64 | redistributable reimplementation of the V8 routines.) | |
65 | See L<Version 8 Regular Expressions> for details. | |
66 | ||
67 | In particular the following metacharacters have their standard I<egrep>-ish | |
68 | meanings: | |
69 | ||
54310121 | 70 | \ Quote the next metacharacter |
a0d0e21e LW |
71 | ^ Match the beginning of the line |
72 | . Match any character (except newline) | |
c07a80fd | 73 | $ Match the end of the line (or before newline at the end) |
a0d0e21e LW |
74 | | Alternation |
75 | () Grouping | |
76 | [] Character class | |
77 | ||
5f05dabc | 78 | By default, the "^" character is guaranteed to match at only the |
79 | beginning of the string, the "$" character at only the end (or before the | |
a0d0e21e LW |
80 | newline at the end) and Perl does certain optimizations with the |
81 | assumption that the string contains only one line. Embedded newlines | |
82 | will not be matched by "^" or "$". You may, however, wish to treat a | |
4a6725af | 83 | string as a multi-line buffer, such that the "^" will match after any |
a0d0e21e LW |
84 | newline within the string, and "$" will match before any newline. At the |
85 | cost of a little more overhead, you can do this by using the /m modifier | |
86 | on the pattern match operator. (Older programs did this by setting C<$*>, | |
5f05dabc | 87 | but this practice is now deprecated.) |
a0d0e21e | 88 | |
4a6725af | 89 | To facilitate multi-line substitutions, the "." character never matches a |
55497cff | 90 | newline unless you use the C</s> modifier, which in effect tells Perl to pretend |
a0d0e21e LW |
91 | the string is a single line--even if it isn't. The C</s> modifier also |
92 | overrides the setting of C<$*>, in case you have some (badly behaved) older | |
93 | code that sets it in another module. | |
94 | ||
95 | The following standard quantifiers are recognized: | |
96 | ||
97 | * Match 0 or more times | |
98 | + Match 1 or more times | |
99 | ? Match 1 or 0 times | |
100 | {n} Match exactly n times | |
101 | {n,} Match at least n times | |
102 | {n,m} Match at least n but not more than m times | |
103 | ||
104 | (If a curly bracket occurs in any other context, it is treated | |
105 | as a regular character.) The "*" modifier is equivalent to C<{0,}>, the "+" | |
25f94b33 | 106 | modifier to C<{1,}>, and the "?" modifier to C<{0,1}>. n and m are limited |
c07a80fd | 107 | to integral values less than 65536. |
a0d0e21e | 108 | |
54310121 | 109 | By default, a quantified subpattern is "greedy", that is, it will match as |
110 | many times as possible (given a particular starting location) while still | |
111 | allowing the rest of the pattern to match. If you want it to match the | |
112 | minimum number of times possible, follow the quantifier with a "?". Note | |
113 | that the meanings don't change, just the "greediness": | |
a0d0e21e LW |
114 | |
115 | *? Match 0 or more times | |
116 | +? Match 1 or more times | |
117 | ?? Match 0 or 1 time | |
118 | {n}? Match exactly n times | |
119 | {n,}? Match at least n times | |
120 | {n,m}? Match at least n but not more than m times | |
121 | ||
5f05dabc | 122 | Because patterns are processed as double quoted strings, the following |
a0d0e21e LW |
123 | also work: |
124 | ||
0f36ee90 | 125 | \t tab (HT, TAB) |
126 | \n newline (LF, NL) | |
127 | \r return (CR) | |
128 | \f form feed (FF) | |
129 | \a alarm (bell) (BEL) | |
130 | \e escape (think troff) (ESC) | |
cb1a09d0 AD |
131 | \033 octal char (think of a PDP-11) |
132 | \x1B hex char | |
a0d0e21e | 133 | \c[ control char |
cb1a09d0 AD |
134 | \l lowercase next char (think vi) |
135 | \u uppercase next char (think vi) | |
136 | \L lowercase till \E (think vi) | |
137 | \U uppercase till \E (think vi) | |
138 | \E end case modification (think vi) | |
201ecf35 | 139 | \Q quote (disable) regexp metacharacters till \E |
a0d0e21e | 140 | |
a034a98d DD |
141 | If C<use locale> is in effect, the case map used by C<\l>, C<\L>, C<\u> |
142 | and <\U> is taken from the current locale. See L<perllocale>. | |
143 | ||
a0d0e21e LW |
144 | In addition, Perl defines the following: |
145 | ||
146 | \w Match a "word" character (alphanumeric plus "_") | |
147 | \W Match a non-word character | |
148 | \s Match a whitespace character | |
149 | \S Match a non-whitespace character | |
150 | \d Match a digit character | |
151 | \D Match a non-digit character | |
152 | ||
153 | Note that C<\w> matches a single alphanumeric character, not a whole | |
a034a98d DD |
154 | word. To match a word you'd need to say C<\w+>. If C<use locale> is in |
155 | effect, the list of alphabetic characters generated by C<\w> is taken | |
156 | from the current locale. See L<perllocale>. You may use C<\w>, C<\W>, | |
157 | C<\s>, C<\S>, C<\d>, and C<\D> within character classes (though not as | |
158 | either end of a range). | |
a0d0e21e LW |
159 | |
160 | Perl defines the following zero-width assertions: | |
161 | ||
162 | \b Match a word boundary | |
163 | \B Match a non-(word boundary) | |
5f05dabc | 164 | \A Match at only beginning of string |
165 | \Z Match at only end of string (or before newline at the end) | |
a99df21c | 166 | \G Match only where previous m//g left off (works only with /g) |
a0d0e21e LW |
167 | |
168 | A word boundary (C<\b>) is defined as a spot between two characters that | |
68dc0745 | 169 | has a C<\w> on one side of it and a C<\W> on the other side of it (in |
a0d0e21e LW |
170 | either order), counting the imaginary characters off the beginning and |
171 | end of the string as matching a C<\W>. (Within character classes C<\b> | |
172 | represents backspace rather than a word boundary.) The C<\A> and C<\Z> are | |
173 | just like "^" and "$" except that they won't match multiple times when the | |
174 | C</m> modifier is used, while "^" and "$" will match at every internal line | |
c07a80fd | 175 | boundary. To match the actual end of the string, not ignoring newline, |
a99df21c GS |
176 | you can use C<\Z(?!\n)>. The C<\G> assertion can be used to chain global |
177 | matches (using C<m//g>), as described in | |
e7ea3e70 | 178 | L<perlop/"Regexp Quote-Like Operators">. |
a99df21c | 179 | |
e7ea3e70 IZ |
180 | It is also useful when writing C<lex>-like scanners, when you have several |
181 | regexps which you want to match against consequent substrings of your | |
182 | string, see the previous reference. | |
44a8e56a | 183 | The actual location where C<\G> will match can also be influenced |
184 | by using C<pos()> as an lvalue. See L<perlfunc/pos>. | |
a0d0e21e | 185 | |
0f36ee90 | 186 | When the bracketing construct C<( ... )> is used, \E<lt>digitE<gt> matches the |
cb1a09d0 | 187 | digit'th substring. Outside of the pattern, always use "$" instead of "\" |
0f36ee90 | 188 | in front of the digit. (While the \E<lt>digitE<gt> notation can on rare occasion work |
cb1a09d0 | 189 | outside the current pattern, this should not be relied upon. See the |
0f36ee90 | 190 | WARNING below.) The scope of $E<lt>digitE<gt> (and C<$`>, C<$&>, and C<$'>) |
cb1a09d0 AD |
191 | extends to the end of the enclosing BLOCK or eval string, or to the next |
192 | successful pattern match, whichever comes first. If you want to use | |
5f05dabc | 193 | parentheses to delimit a subpattern (e.g., a set of alternatives) without |
84dc3c4d | 194 | saving it as a subpattern, follow the ( with a ?:. |
cb1a09d0 AD |
195 | |
196 | You may have as many parentheses as you wish. If you have more | |
a0d0e21e LW |
197 | than 9 substrings, the variables $10, $11, ... refer to the |
198 | corresponding substring. Within the pattern, \10, \11, etc. refer back | |
5f05dabc | 199 | to substrings if there have been at least that many left parentheses before |
c07a80fd | 200 | the backreference. Otherwise (for backward compatibility) \10 is the |
a0d0e21e LW |
201 | same as \010, a backspace, and \11 the same as \011, a tab. And so |
202 | on. (\1 through \9 are always backreferences.) | |
203 | ||
204 | C<$+> returns whatever the last bracket match matched. C<$&> returns the | |
0f36ee90 | 205 | entire matched string. (C<$0> used to return the same thing, but not any |
a0d0e21e LW |
206 | more.) C<$`> returns everything before the matched string. C<$'> returns |
207 | everything after the matched string. Examples: | |
208 | ||
209 | s/^([^ ]*) *([^ ]*)/$2 $1/; # swap first two words | |
210 | ||
211 | if (/Time: (..):(..):(..)/) { | |
212 | $hours = $1; | |
213 | $minutes = $2; | |
214 | $seconds = $3; | |
215 | } | |
216 | ||
68dc0745 | 217 | Once perl sees that you need one of C<$&>, C<$`> or C<$'> anywhere in |
218 | the program, it has to provide them on each and every pattern match. | |
219 | This can slow your program down. The same mechanism that handles | |
220 | these provides for the use of $1, $2, etc., so you pay the same price | |
221 | for each regexp that contains capturing parentheses. But if you never | |
222 | use $&, etc., in your script, then regexps I<without> capturing | |
223 | parentheses won't be penalized. So avoid $&, $', and $` if you can, | |
224 | but if you can't (and some algorithms really appreciate them), once | |
225 | you've used them once, use them at will, because you've already paid | |
226 | the price. | |
227 | ||
a0d0e21e | 228 | You will note that all backslashed metacharacters in Perl are |
201ecf35 AL |
229 | alphanumeric, such as C<\b>, C<\w>, C<\n>. Unlike some other regular |
230 | expression languages, there are no backslashed symbols that aren't | |
231 | alphanumeric. So anything that looks like \\, \(, \), \E<lt>, \E<gt>, | |
232 | \{, or \} is always interpreted as a literal character, not a | |
233 | metacharacter. This was once used in a common idiom to disable or | |
234 | quote the special meanings of regular expression metacharacters in a | |
235 | string that you want to use for a pattern. Simply quote all the | |
a0d0e21e LW |
236 | non-alphanumeric characters: |
237 | ||
238 | $pattern =~ s/(\W)/\\$1/g; | |
239 | ||
201ecf35 AL |
240 | Now it is much more common to see either the quotemeta() function or |
241 | the \Q escape sequence used to disable the metacharacters special | |
242 | meanings like this: | |
a0d0e21e LW |
243 | |
244 | /$unquoted\Q$quoted\E$unquoted/ | |
245 | ||
5f05dabc | 246 | Perl defines a consistent extension syntax for regular expressions. |
247 | The syntax is a pair of parentheses with a question mark as the first | |
248 | thing within the parentheses (this was a syntax error in older | |
249 | versions of Perl). The character after the question mark gives the | |
250 | function of the extension. Several extensions are already supported: | |
a0d0e21e LW |
251 | |
252 | =over 10 | |
253 | ||
cc6b7395 | 254 | =item C<(?#text)> |
a0d0e21e | 255 | |
cb1a09d0 AD |
256 | A comment. The text is ignored. If the C</x> switch is used to enable |
257 | whitespace formatting, a simple C<#> will suffice. | |
a0d0e21e | 258 | |
cc6b7395 | 259 | =item C<(?:regexp)> |
a0d0e21e | 260 | |
0f36ee90 | 261 | This groups things like "()" but doesn't make backreferences like "()" does. So |
a0d0e21e LW |
262 | |
263 | split(/\b(?:a|b|c)\b/) | |
264 | ||
265 | is like | |
266 | ||
267 | split(/\b(a|b|c)\b/) | |
268 | ||
269 | but doesn't spit out extra fields. | |
270 | ||
cc6b7395 | 271 | =item C<(?=regexp)> |
a0d0e21e LW |
272 | |
273 | A zero-width positive lookahead assertion. For example, C</\w+(?=\t)/> | |
274 | matches a word followed by a tab, without including the tab in C<$&>. | |
275 | ||
cc6b7395 | 276 | =item C<(?!regexp)> |
a0d0e21e LW |
277 | |
278 | A zero-width negative lookahead assertion. For example C</foo(?!bar)/> | |
279 | matches any occurrence of "foo" that isn't followed by "bar". Note | |
280 | however that lookahead and lookbehind are NOT the same thing. You cannot | |
a3cb178b GS |
281 | use this for lookbehind. If you are looking for a "bar" which isn't preceeded |
282 | "foo", C</(?!foo)bar/> will not do what you want. That's because | |
a0d0e21e LW |
283 | the C<(?!foo)> is just saying that the next thing cannot be "foo"--and |
284 | it's not, it's a "bar", so "foobar" will match. You would have to do | |
0f36ee90 | 285 | something like C</(?!foo)...bar/> for that. We say "like" because there's |
a0d0e21e | 286 | the case of your "bar" not having three characters before it. You could |
a3cb178b | 287 | cover that this way: C</(?:(?!foo)...|^.{0,2})bar/>. Sometimes it's still |
a0d0e21e LW |
288 | easier just to say: |
289 | ||
a3cb178b | 290 | if (/bar/ && $` !~ /foo$/) |
a0d0e21e | 291 | |
c277df42 IZ |
292 | For lookbehind see below. |
293 | ||
cc6b7395 | 294 | =item C<(?<=regexp)> |
c277df42 IZ |
295 | |
296 | A zero-width positive lookbehind assertion. For example, C</(?=\t)\w+/> | |
297 | matches a word following a tab, without including the tab in C<$&>. | |
298 | Works only for fixed-width lookbehind. | |
299 | ||
cc6b7395 | 300 | =item C<(?<!regexp)> |
c277df42 IZ |
301 | |
302 | A zero-width negative lookbehind assertion. For example C</(?<!bar)foo/> | |
303 | matches any occurrence of "foo" that isn't following "bar". | |
304 | Works only for fixed-width lookbehind. | |
305 | ||
cc6b7395 | 306 | =item C<(?{ code })> |
c277df42 IZ |
307 | |
308 | Experimental "evaluate any Perl code" zero-width assertion. Always | |
cc6b7395 IZ |
309 | succeeds. C<code> is not interpolated. Currently the rules to |
310 | determine where the C<code> ends are somewhat convoluted. | |
c277df42 IZ |
311 | |
312 | =item C<(?E<gt>regexp)> | |
313 | ||
314 | An "independend" subexpression. Matches the substring which a | |
315 | I<standalone> C<regexp> would match if anchored at the given position, | |
316 | B<and only this substring>. | |
317 | ||
318 | Say, C<^(?E<gt>a*)ab> will never match, since C<(?E<gt>a*)> (anchored | |
319 | at the beginning of string, as above) will match I<all> the characters | |
320 | C<a> at the beginning of string, leaving no C<a> for C<ab> to match. | |
321 | In contrast, C<a*ab> will match the same as C<a+b>, since the match of | |
322 | the subgroup C<a*> is influenced by the following group C<ab> (see | |
323 | L<"Backtracking">). In particular, C<a*> inside C<a*ab> will match | |
324 | less characters that a standalone C<a*>, since this makes the tail match. | |
325 | ||
326 | Note that a similar effect to C<(?E<gt>regexp)> may be achieved by | |
327 | ||
328 | (?=(regexp))\1 | |
329 | ||
330 | since the lookahead is in I<"logical"> context, thus matches the same | |
331 | substring as a standalone C<a+>. The following C<\1> eats the matched | |
332 | string, thus making a zero-length assertion into an analogue of | |
333 | C<(?>...)>. (The difference of these two constructions is that the | |
334 | second one uses a catching group, thus shifts ordinals of | |
335 | backreferences in the rest of a regular expression.) | |
336 | ||
337 | This construction is very useful for optimizations of "eternal" | |
338 | matches, since it will not backtrack (see L<"Backtracking">). Say, | |
339 | ||
340 | / \( ( | |
341 | [^()]+ | |
342 | | | |
343 | \( [^()]* \) | |
344 | )+ | |
345 | \) /x | |
346 | ||
347 | will match a nonempty group with matching two-or-less-level-deep | |
348 | parentheses. It is very efficient in finding such groups. However, | |
349 | if there is no such group, it is going to take forever (on reasonably | |
350 | long string), since there are so many different ways to split a long | |
351 | string into several substrings (this is essentially what C<(.+)+> is | |
352 | doing, and this is a subpattern of the above pattern). Say, on | |
353 | C<((()aaaaaaaaaaaaaaaaaa> the above pattern detects no-match in 5sec | |
354 | (on kitchentop'96 processor), and each extra letter doubles this time. | |
355 | ||
356 | However, a tiny modification of this | |
357 | ||
358 | / \( ( | |
359 | (?> [^()]+ ) | |
360 | | | |
361 | \( [^()]* \) | |
362 | )+ | |
363 | \) /x | |
364 | ||
365 | which uses (?>...) matches exactly when the above one does (it is a | |
366 | good excercise to check this), but finishes in a fourth of the above | |
367 | time on a similar string with 1000000 C<a>s. | |
368 | ||
369 | Note that on simple groups like the above C<(?> [^()]+ )> a similar | |
370 | effect may be achieved by negative lookahead, as in C<[^()]+ (?! [^()] )>. | |
371 | This was only 4 times slower on a string with 1000000 C<a>s. | |
372 | ||
cc6b7395 | 373 | =item C<(?(condition)yes-regexp|no-regexp)> |
c277df42 | 374 | |
cc6b7395 | 375 | =item C<(?(condition)yes-regexp)> |
c277df42 IZ |
376 | |
377 | Conditional expression. C<(condition)> should be either an integer in | |
378 | parentheses (which is valid if the corresponding pair of parentheses | |
379 | matched), or lookahead/lookbehind/evaluate zero-width assertion. | |
380 | ||
381 | Say, | |
382 | ||
383 | / ( \( )? | |
384 | [^()]+ | |
385 | (?(1) \) )/x | |
386 | ||
387 | matches a chunk of non-parentheses, possibly included in parentheses | |
388 | themselves. | |
a0d0e21e | 389 | |
cc6b7395 | 390 | =item C<(?imsx)> |
a0d0e21e LW |
391 | |
392 | One or more embedded pattern-match modifiers. This is particularly | |
393 | useful for patterns that are specified in a table somewhere, some of | |
394 | which want to be case sensitive, and some of which don't. The case | |
5f05dabc | 395 | insensitive ones need to include merely C<(?i)> at the front of the |
a0d0e21e LW |
396 | pattern. For example: |
397 | ||
398 | $pattern = "foobar"; | |
c07a80fd | 399 | if ( /$pattern/i ) |
a0d0e21e LW |
400 | |
401 | # more flexible: | |
402 | ||
403 | $pattern = "(?i)foobar"; | |
c07a80fd | 404 | if ( /$pattern/ ) |
a0d0e21e | 405 | |
c277df42 IZ |
406 | Note that these modifiers are localized inside an enclosing group (if |
407 | any). Say, | |
408 | ||
409 | ( (?i) blah ) \s+ \1 | |
410 | ||
411 | (assuming C<x> modifier, and no C<i> modifier outside of this group) | |
412 | will match a repeated (I<including the case>!) word C<blah> in any | |
413 | case. | |
414 | ||
a0d0e21e LW |
415 | =back |
416 | ||
417 | The specific choice of question mark for this and the new minimal | |
418 | matching construct was because 1) question mark is pretty rare in older | |
419 | regular expressions, and 2) whenever you see one, you should stop | |
420 | and "question" exactly what is going on. That's psychology... | |
421 | ||
c07a80fd | 422 | =head2 Backtracking |
423 | ||
c277df42 IZ |
424 | A fundamental feature of regular expression matching involves the |
425 | notion called I<backtracking>. which is currently used (when needed) | |
426 | by all regular expression quantifiers, namely C<*>, C<*?>, C<+>, | |
427 | C<+?>, C<{n,m}>, and C<{n,m}?>. | |
c07a80fd | 428 | |
429 | For a regular expression to match, the I<entire> regular expression must | |
430 | match, not just part of it. So if the beginning of a pattern containing a | |
431 | quantifier succeeds in a way that causes later parts in the pattern to | |
432 | fail, the matching engine backs up and recalculates the beginning | |
433 | part--that's why it's called backtracking. | |
434 | ||
435 | Here is an example of backtracking: Let's say you want to find the | |
436 | word following "foo" in the string "Food is on the foo table.": | |
437 | ||
438 | $_ = "Food is on the foo table."; | |
439 | if ( /\b(foo)\s+(\w+)/i ) { | |
440 | print "$2 follows $1.\n"; | |
441 | } | |
442 | ||
443 | When the match runs, the first part of the regular expression (C<\b(foo)>) | |
444 | finds a possible match right at the beginning of the string, and loads up | |
445 | $1 with "Foo". However, as soon as the matching engine sees that there's | |
446 | no whitespace following the "Foo" that it had saved in $1, it realizes its | |
68dc0745 | 447 | mistake and starts over again one character after where it had the |
c07a80fd | 448 | tentative match. This time it goes all the way until the next occurrence |
449 | of "foo". The complete regular expression matches this time, and you get | |
450 | the expected output of "table follows foo." | |
451 | ||
452 | Sometimes minimal matching can help a lot. Imagine you'd like to match | |
453 | everything between "foo" and "bar". Initially, you write something | |
454 | like this: | |
455 | ||
456 | $_ = "The food is under the bar in the barn."; | |
457 | if ( /foo(.*)bar/ ) { | |
458 | print "got <$1>\n"; | |
459 | } | |
460 | ||
461 | Which perhaps unexpectedly yields: | |
462 | ||
463 | got <d is under the bar in the > | |
464 | ||
465 | That's because C<.*> was greedy, so you get everything between the | |
466 | I<first> "foo" and the I<last> "bar". In this case, it's more effective | |
467 | to use minimal matching to make sure you get the text between a "foo" | |
468 | and the first "bar" thereafter. | |
469 | ||
470 | if ( /foo(.*?)bar/ ) { print "got <$1>\n" } | |
471 | got <d is under the > | |
472 | ||
473 | Here's another example: let's say you'd like to match a number at the end | |
474 | of a string, and you also want to keep the preceding part the match. | |
475 | So you write this: | |
476 | ||
477 | $_ = "I have 2 numbers: 53147"; | |
478 | if ( /(.*)(\d*)/ ) { # Wrong! | |
479 | print "Beginning is <$1>, number is <$2>.\n"; | |
480 | } | |
481 | ||
482 | That won't work at all, because C<.*> was greedy and gobbled up the | |
483 | whole string. As C<\d*> can match on an empty string the complete | |
484 | regular expression matched successfully. | |
485 | ||
8e1088bc | 486 | Beginning is <I have 2 numbers: 53147>, number is <>. |
c07a80fd | 487 | |
488 | Here are some variants, most of which don't work: | |
489 | ||
490 | $_ = "I have 2 numbers: 53147"; | |
491 | @pats = qw{ | |
492 | (.*)(\d*) | |
493 | (.*)(\d+) | |
494 | (.*?)(\d*) | |
495 | (.*?)(\d+) | |
496 | (.*)(\d+)$ | |
497 | (.*?)(\d+)$ | |
498 | (.*)\b(\d+)$ | |
499 | (.*\D)(\d+)$ | |
500 | }; | |
501 | ||
502 | for $pat (@pats) { | |
503 | printf "%-12s ", $pat; | |
504 | if ( /$pat/ ) { | |
505 | print "<$1> <$2>\n"; | |
506 | } else { | |
507 | print "FAIL\n"; | |
508 | } | |
509 | } | |
510 | ||
511 | That will print out: | |
512 | ||
513 | (.*)(\d*) <I have 2 numbers: 53147> <> | |
514 | (.*)(\d+) <I have 2 numbers: 5314> <7> | |
515 | (.*?)(\d*) <> <> | |
516 | (.*?)(\d+) <I have > <2> | |
517 | (.*)(\d+)$ <I have 2 numbers: 5314> <7> | |
518 | (.*?)(\d+)$ <I have 2 numbers: > <53147> | |
519 | (.*)\b(\d+)$ <I have 2 numbers: > <53147> | |
520 | (.*\D)(\d+)$ <I have 2 numbers: > <53147> | |
521 | ||
522 | As you see, this can be a bit tricky. It's important to realize that a | |
523 | regular expression is merely a set of assertions that gives a definition | |
524 | of success. There may be 0, 1, or several different ways that the | |
525 | definition might succeed against a particular string. And if there are | |
5f05dabc | 526 | multiple ways it might succeed, you need to understand backtracking to know which variety of success you will achieve. |
c07a80fd | 527 | |
528 | When using lookahead assertions and negations, this can all get even | |
54310121 | 529 | tricker. Imagine you'd like to find a sequence of non-digits not |
c07a80fd | 530 | followed by "123". You might try to write that as |
531 | ||
532 | $_ = "ABC123"; | |
533 | if ( /^\D*(?!123)/ ) { # Wrong! | |
534 | print "Yup, no 123 in $_\n"; | |
535 | } | |
536 | ||
537 | But that isn't going to match; at least, not the way you're hoping. It | |
538 | claims that there is no 123 in the string. Here's a clearer picture of | |
539 | why it that pattern matches, contrary to popular expectations: | |
540 | ||
541 | $x = 'ABC123' ; | |
542 | $y = 'ABC445' ; | |
543 | ||
544 | print "1: got $1\n" if $x =~ /^(ABC)(?!123)/ ; | |
545 | print "2: got $1\n" if $y =~ /^(ABC)(?!123)/ ; | |
546 | ||
547 | print "3: got $1\n" if $x =~ /^(\D*)(?!123)/ ; | |
548 | print "4: got $1\n" if $y =~ /^(\D*)(?!123)/ ; | |
549 | ||
550 | This prints | |
551 | ||
552 | 2: got ABC | |
553 | 3: got AB | |
554 | 4: got ABC | |
555 | ||
5f05dabc | 556 | You might have expected test 3 to fail because it seems to a more |
c07a80fd | 557 | general purpose version of test 1. The important difference between |
558 | them is that test 3 contains a quantifier (C<\D*>) and so can use | |
559 | backtracking, whereas test 1 will not. What's happening is | |
560 | that you've asked "Is it true that at the start of $x, following 0 or more | |
5f05dabc | 561 | non-digits, you have something that's not 123?" If the pattern matcher had |
c07a80fd | 562 | let C<\D*> expand to "ABC", this would have caused the whole pattern to |
54310121 | 563 | fail. |
c07a80fd | 564 | The search engine will initially match C<\D*> with "ABC". Then it will |
565 | try to match C<(?!123> with "123" which, of course, fails. But because | |
566 | a quantifier (C<\D*>) has been used in the regular expression, the | |
567 | search engine can backtrack and retry the match differently | |
54310121 | 568 | in the hope of matching the complete regular expression. |
c07a80fd | 569 | |
54310121 | 570 | Well now, |
c07a80fd | 571 | the pattern really, I<really> wants to succeed, so it uses the |
5f05dabc | 572 | standard regexp back-off-and-retry and lets C<\D*> expand to just "AB" this |
c07a80fd | 573 | time. Now there's indeed something following "AB" that is not |
574 | "123". It's in fact "C123", which suffices. | |
575 | ||
576 | We can deal with this by using both an assertion and a negation. We'll | |
577 | say that the first part in $1 must be followed by a digit, and in fact, it | |
578 | must also be followed by something that's not "123". Remember that the | |
579 | lookaheads are zero-width expressions--they only look, but don't consume | |
580 | any of the string in their match. So rewriting this way produces what | |
581 | you'd expect; that is, case 5 will fail, but case 6 succeeds: | |
582 | ||
583 | print "5: got $1\n" if $x =~ /^(\D*)(?=\d)(?!123)/ ; | |
584 | print "6: got $1\n" if $y =~ /^(\D*)(?=\d)(?!123)/ ; | |
585 | ||
586 | 6: got ABC | |
587 | ||
588 | In other words, the two zero-width assertions next to each other work like | |
589 | they're ANDed together, just as you'd use any builtin assertions: C</^$/> | |
590 | matches only if you're at the beginning of the line AND the end of the | |
591 | line simultaneously. The deeper underlying truth is that juxtaposition in | |
592 | regular expressions always means AND, except when you write an explicit OR | |
593 | using the vertical bar. C</ab/> means match "a" AND (then) match "b", | |
594 | although the attempted matches are made at different positions because "a" | |
595 | is not a zero-width assertion, but a one-width assertion. | |
596 | ||
597 | One warning: particularly complicated regular expressions can take | |
598 | exponential time to solve due to the immense number of possible ways they | |
599 | can use backtracking to try match. For example this will take a very long | |
600 | time to run | |
601 | ||
602 | /((a{0,5}){0,5}){0,5}/ | |
603 | ||
604 | And if you used C<*>'s instead of limiting it to 0 through 5 matches, then | |
605 | it would take literally forever--or until you ran out of stack space. | |
606 | ||
c277df42 IZ |
607 | A powerful tool for optimizing such beasts is "independent" groups, |
608 | which do not backtrace (see L<C<(?E<gt>regexp)>>). Note also that | |
609 | zero-length lookahead/lookbehind assertions will not backtrace to make | |
610 | the tail match, since they are in "logical" context: only the fact | |
611 | whether they match or not is considered relevant. For an example | |
612 | where side-effects of a lookahead I<might> have influenced the | |
613 | following match, see L<C<(?E<gt>regexp)>>. | |
614 | ||
a0d0e21e LW |
615 | =head2 Version 8 Regular Expressions |
616 | ||
617 | In case you're not familiar with the "regular" Version 8 regexp | |
618 | routines, here are the pattern-matching rules not described above. | |
619 | ||
54310121 | 620 | Any single character matches itself, unless it is a I<metacharacter> |
a0d0e21e LW |
621 | with a special meaning described here or above. You can cause |
622 | characters which normally function as metacharacters to be interpreted | |
5f05dabc | 623 | literally by prefixing them with a "\" (e.g., "\." matches a ".", not any |
a0d0e21e LW |
624 | character; "\\" matches a "\"). A series of characters matches that |
625 | series of characters in the target string, so the pattern C<blurfl> | |
626 | would match "blurfl" in the target string. | |
627 | ||
628 | You can specify a character class, by enclosing a list of characters | |
629 | in C<[]>, which will match any one of the characters in the list. If the | |
630 | first character after the "[" is "^", the class matches any character not | |
631 | in the list. Within a list, the "-" character is used to specify a | |
632 | range, so that C<a-z> represents all the characters between "a" and "z", | |
84850974 DD |
633 | inclusive. If you want "-" itself to be a member of a class, put it |
634 | at the start or end of the list, or escape it with a backslash. (The | |
635 | following all specify the same class of three characters: C<[-az]>, | |
636 | C<[az-]>, and C<[a\-z]>. All are different from C<[a-z]>, which | |
637 | specifies a class containing twenty-six characters.) | |
a0d0e21e | 638 | |
54310121 | 639 | Characters may be specified using a metacharacter syntax much like that |
a0d0e21e LW |
640 | used in C: "\n" matches a newline, "\t" a tab, "\r" a carriage return, |
641 | "\f" a form feed, etc. More generally, \I<nnn>, where I<nnn> is a string | |
642 | of octal digits, matches the character whose ASCII value is I<nnn>. | |
0f36ee90 | 643 | Similarly, \xI<nn>, where I<nn> are hexadecimal digits, matches the |
a0d0e21e | 644 | character whose ASCII value is I<nn>. The expression \cI<x> matches the |
54310121 | 645 | ASCII character control-I<x>. Finally, the "." metacharacter matches any |
a0d0e21e LW |
646 | character except "\n" (unless you use C</s>). |
647 | ||
648 | You can specify a series of alternatives for a pattern using "|" to | |
649 | separate them, so that C<fee|fie|foe> will match any of "fee", "fie", | |
650 | or "foe" in the target string (as would C<f(e|i|o)e>). Note that the | |
651 | first alternative includes everything from the last pattern delimiter | |
652 | ("(", "[", or the beginning of the pattern) up to the first "|", and | |
653 | the last alternative contains everything from the last "|" to the next | |
654 | pattern delimiter. For this reason, it's common practice to include | |
655 | alternatives in parentheses, to minimize confusion about where they | |
a3cb178b GS |
656 | start and end. |
657 | ||
658 | Note that alternatives are tried from left to right, so the first | |
659 | alternative found for which the entire expression matches, is the one that | |
660 | is chosen. This means that alternatives are not necessarily greedy. For | |
661 | example: when mathing C<foo|foot> against "barefoot", only the "foo" | |
662 | part will match, as that is the first alternative tried, and it successfully | |
663 | matches the target string. (This might not seem important, but it is | |
664 | important when you are capturing matched text using parentheses.) | |
665 | ||
666 | Also note that "|" is interpreted as a literal within square brackets, | |
667 | so if you write C<[fee|fie|foe]> you're really only matching C<[feio|]>. | |
a0d0e21e | 668 | |
54310121 | 669 | Within a pattern, you may designate subpatterns for later reference by |
a0d0e21e | 670 | enclosing them in parentheses, and you may refer back to the I<n>th |
54310121 | 671 | subpattern later in the pattern using the metacharacter \I<n>. |
672 | Subpatterns are numbered based on the left to right order of their | |
a0d0e21e | 673 | opening parenthesis. Note that a backreference matches whatever |
54310121 | 674 | actually matched the subpattern in the string being examined, not the |
675 | rules for that subpattern. Therefore, C<(0|0x)\d*\s\1\d*> will | |
676 | match "0x1234 0x4321",but not "0x1234 01234", because subpattern 1 | |
748a9306 | 677 | actually matched "0x", even though the rule C<0|0x> could |
a0d0e21e | 678 | potentially match the leading 0 in the second number. |
cb1a09d0 AD |
679 | |
680 | =head2 WARNING on \1 vs $1 | |
681 | ||
682 | Some people get too used to writing things like | |
683 | ||
684 | $pattern =~ s/(\W)/\\\1/g; | |
685 | ||
686 | This is grandfathered for the RHS of a substitute to avoid shocking the | |
687 | B<sed> addicts, but it's a dirty habit to get into. That's because in | |
5f05dabc | 688 | PerlThink, the righthand side of a C<s///> is a double-quoted string. C<\1> in |
cb1a09d0 AD |
689 | the usual double-quoted string means a control-A. The customary Unix |
690 | meaning of C<\1> is kludged in for C<s///>. However, if you get into the habit | |
691 | of doing that, you get yourself into trouble if you then add an C</e> | |
692 | modifier. | |
693 | ||
694 | s/(\d+)/ \1 + 1 /eg; | |
695 | ||
696 | Or if you try to do | |
697 | ||
698 | s/(\d+)/\1000/; | |
699 | ||
700 | You can't disambiguate that by saying C<\{1}000>, whereas you can fix it with | |
701 | C<${1}000>. Basically, the operation of interpolation should not be confused | |
702 | with the operation of matching a backreference. Certainly they mean two | |
703 | different things on the I<left> side of the C<s///>. | |
9fa51da4 CS |
704 | |
705 | =head2 SEE ALSO | |
706 | ||
9b599b2a GS |
707 | L<perlop/"Regexp Quote-Like Operators">. |
708 | ||
709 | L<perlfunc/pos>. | |
710 | ||
711 | L<perllocale>. | |
712 | ||
9fa51da4 | 713 | "Mastering Regular Expressions" (see L<perlbook>) by Jeffrey Friedl. |