The dot (or period), C<.> is probably the most used, and certainly
the most well-known character class. By default, a dot matches any
-character, except for the newline. The default can be changed to
+character, except for the newline. That default can be changed to
add matching the newline by using the I<single line> modifier: either
for the entire regular expression with the C</s> modifier, or
locally with C<(?s)>. (The experimental C<\N> backslash sequence, described
\pP, \p{Prop} Match a character that has the given Unicode property.
\PP, \P{Prop} Match a character that doesn't have the Unicode property
+=head3 \N
+
+C<\N> is new in 5.12, and is experimental. It, like the dot, matches any
+character that is not a newline. The difference is that C<\N> is not influenced
+by the I<single line> regular expression modifier (see L</The dot> above). Note
+that the form C<\N{...}> may mean something completely different. When the
+C<{...}> is a L<quantifier|perlre/Quantifiers>, it means to match a non-newline
+character that many times. For example, C<\N{3}> means to match 3
+non-newlines; C<\N{5,}> means to match 5 or more non-newlines. But if C<{...}>
+is not a legal quantifier, it is presumed to be a named character. See
+L<charnames> for those. For example, none of C<\N{COLON}>, C<\N{4F}>, and
+C<\N{F4}> contain legal quantifiers, so Perl will try to find characters whose
+names are respectively C<COLON>, C<4F>, and C<F4>.
+
=head3 Digits
C<\d> matches a single character considered to be a decimal I<digit>.
-If the C</a> modifier in effect, it matches [0-9]. Otherwise, it
+If the C</a> regular expression modifier is in effect, it matches [0-9].
+Otherwise, it
matches anything that is matched by C<\p{Digit}>, which includes [0-9].
(An unlikely possible exception is that under locale matching rules, the
current locale might not have [0-9] matched by C<\d>, and/or might match
is expecting only the ASCII digits might be misled, or if the match is
C<\d+>, the matched string might contain a mixture of digits from
different writing systems that look like they signify a number different
-than they actually do. L<Unicode::UCD/num()> can be used to safely
+than they actually do. L<Unicode::UCD/num()> can
+be used to safely
calculate the value, returning C<undef> if the input string contains
such a mixture.
to characters that match the other type of "digit",
C<\p{Numeric_Type=Digit}>, and so C<\d> doesn't match them.
-In Unicode 5.2, the Tamil digits (U+0BE6 - U+0BEF) can also legally be
+The Tamil digits (U+0BE6 - U+0BEF) can also legally be
used in old-style Tamil numbers in which they would appear no more than
one in a row, separated by characters that mean "times 10", "times 100",
etc. (See L<http://www.unicode.org/notes/tn21>.)
=head3 Word characters
A C<\w> matches a single alphanumeric character (an alphabetic character, or a
-decimal digit) or a connecting punctuation character, such as an
-underscore ("_"). It does not match a whole word. To match a whole
-word, use C<\w+>. This isn't the same thing as matching an English word, but
-in the ASCII range it is the same as a string of Perl-identifier
-characters.
+decimal digit); or a connecting punctuation character, such as an
+underscore ("_"); or a "mark" character (like some sort of accent) that
+attaches to one of those. It does not match a whole word. To match a
+whole word, use C<\w+>. This isn't the same thing as matching an
+English word, but in the ASCII range it is the same as a string of
+Perl-identifier characters.
=over
Also, for a somewhat finer-grained set of characters that are in programming
language identifiers beyond the ASCII range, you may wish to instead use the
-more customized Unicode properties, "ID_Start", ID_Continue", "XID_Start", and
-"XID_Continue". See L<http://unicode.org/reports/tr31>.
+more customized L</Unicode Properties>, C<\p{ID_Start}>,
+C<\p{ID_Continue}>, C<\p{XID_Start}>, and C<\p{XID_Continue}>. See
+L<http://unicode.org/reports/tr31>.
Any character not matched by C<\w> is matched by C<\W>.
Any character not matched by C<\s> is matched by C<\S>.
C<\h> matches any character considered horizontal whitespace;
-this includes the space and tab characters and several others
+this includes the platform's space and tab characters and several others
listed in the table below. C<\H> matches any character
-not considered horizontal whitespace.
+not considered horizontal whitespace. They use the platform's native
+character set, and do not consider any locale that may otherwise be in
+use.
C<\v> matches any character considered vertical whitespace;
-this includes the carriage return and line feed characters (newline)
+this includes the platform's carriage return and line feed characters (newline)
plus several other characters, all listed in the table below.
C<\V> matches any character not considered vertical whitespace.
+They use the platform's native character set, and do not consider any
+locale that may otherwise be in use.
C<\R> matches anything that can be considered a newline under Unicode
rules. It's not a character class, as it can match a multi-character
sequence. Therefore, it cannot be used inside a bracketed character
-class; use C<\v> instead (vertical whitespace).
+class; use C<\v> instead (vertical whitespace). It uses the platform's
+native character set, and does not consider any locale that may
+otherwise be in use.
Details are discussed in L<perlrebackslash>.
Note that unlike C<\s> (and C<\d> and C<\w>), C<\h> and C<\v> always match
-the same characters, without regard to other factors, such as whether the
-source string is in UTF-8 format.
+the same characters, without regard to other factors, such as the active
+locale or whether the source string is in UTF-8 format.
One might think that C<\s> is equivalent to C<[\h\v]>. This is not true.
-For example, the vertical tab (C<"\x0b">) is not matched by C<\s>, it is
-however considered vertical whitespace.
+The difference is that the vertical tab (C<"\x0b">) is not matched by
+C<\s>; it is however considered vertical whitespace.
The following table is a complete listing of characters matched by
C<\s>, C<\h> and C<\v> as of Unicode 6.0.
-The first column gives the code point of the character (in hex format),
+The first column gives the Unicode code point of the character (in hex format),
the second column gives the (Unicode) name. The third column indicates
by which class(es) the character is matched (assuming no locale or EBCDIC code
page is in effect that changes the C<\s> matching).
- 0x00009 CHARACTER TABULATION h s
- 0x0000a LINE FEED (LF) vs
- 0x0000b LINE TABULATION v
- 0x0000c FORM FEED (FF) vs
- 0x0000d CARRIAGE RETURN (CR) vs
- 0x00020 SPACE h s
- 0x00085 NEXT LINE (NEL) vs [1]
- 0x000a0 NO-BREAK SPACE h s [1]
- 0x01680 OGHAM SPACE MARK h s
- 0x0180e MONGOLIAN VOWEL SEPARATOR h s
- 0x02000 EN QUAD h s
- 0x02001 EM QUAD h s
- 0x02002 EN SPACE h s
- 0x02003 EM SPACE h s
- 0x02004 THREE-PER-EM SPACE h s
- 0x02005 FOUR-PER-EM SPACE h s
- 0x02006 SIX-PER-EM SPACE h s
- 0x02007 FIGURE SPACE h s
- 0x02008 PUNCTUATION SPACE h s
- 0x02009 THIN SPACE h s
- 0x0200a HAIR SPACE h s
- 0x02028 LINE SEPARATOR vs
- 0x02029 PARAGRAPH SEPARATOR vs
- 0x0202f NARROW NO-BREAK SPACE h s
- 0x0205f MEDIUM MATHEMATICAL SPACE h s
- 0x03000 IDEOGRAPHIC SPACE h s
+ 0x0009 CHARACTER TABULATION h s
+ 0x000a LINE FEED (LF) vs
+ 0x000b LINE TABULATION v
+ 0x000c FORM FEED (FF) vs
+ 0x000d CARRIAGE RETURN (CR) vs
+ 0x0020 SPACE h s
+ 0x0085 NEXT LINE (NEL) vs [1]
+ 0x00a0 NO-BREAK SPACE h s [1]
+ 0x1680 OGHAM SPACE MARK h s
+ 0x180e MONGOLIAN VOWEL SEPARATOR h s
+ 0x2000 EN QUAD h s
+ 0x2001 EM QUAD h s
+ 0x2002 EN SPACE h s
+ 0x2003 EM SPACE h s
+ 0x2004 THREE-PER-EM SPACE h s
+ 0x2005 FOUR-PER-EM SPACE h s
+ 0x2006 SIX-PER-EM SPACE h s
+ 0x2007 FIGURE SPACE h s
+ 0x2008 PUNCTUATION SPACE h s
+ 0x2009 THIN SPACE h s
+ 0x200a HAIR SPACE h s
+ 0x2028 LINE SEPARATOR vs
+ 0x2029 PARAGRAPH SEPARATOR vs
+ 0x202f NARROW NO-BREAK SPACE h s
+ 0x205f MEDIUM MATHEMATICAL SPACE h s
+ 0x3000 IDEOGRAPHIC SPACE h s
=over 4
=back
-=head3 \N
-
-C<\N> is new in 5.12, and is experimental. It, like the dot, matches any
-character that is not a newline. The difference is that C<\N> is not influenced
-by the I<single line> regular expression modifier (see L</The dot> above). Note
-that the form C<\N{...}> may mean something completely different. When the
-C<{...}> is a L<quantifier|perlre/Quantifiers>, it means to match a non-newline
-character that many times. For example, C<\N{3}> means to match 3
-non-newlines; C<\N{5,}> means to match 5 or more non-newlines. But if C<{...}>
-is not a legal quantifier, it is presumed to be a named character. See
-L<charnames> for those. For example, none of C<\N{COLON}>, C<\N{4F}>, and
-C<\N{F4}> contain legal quantifiers, so Perl will try to find characters whose
-names are respectively C<COLON>, C<4F>, and C<F4>.
-
=head3 Unicode Properties
C<\pP> and C<\p{Prop}> are character classes to match characters that fit given
For instance, a match for a number can be written as C</\pN/> or as
C</\p{Number}/>, or as C</\p{Number=True}/>.
Lowercase letters are matched by the property I<Lowercase_Letter> which
-has as short form I<Ll>. They need the braces, so are written as C</\p{Ll}/> or
+has the short form I<Ll>. They need the braces, so are written as C</\p{Ll}/> or
C</\p{Lowercase_Letter}/>, or C</\p{General_Category=Lowercase_Letter}/>
(the underscores are optional).
C</\pLl/> is valid, but means something different.
It matches a two character string: a letter (Unicode property C<\pL>),
followed by a lowercase C<l>.
-If neither the C</a> modifier nor locale rules are in effect, the use of
+If locale rules are not in effect, the use of
a Unicode property will force the regular expression into using Unicode
-rules.
+rules, if it isn't already.
Note that almost all properties are immune to case-insensitive matching.
That is, adding a C</i> regular expression modifier does not change what
and C<Titlecase>,
all of which match C<Cased> under C</i> matching.
(The difference between these sets is that some things, such as Roman
-Numerals, come in both upper and lower case so they are C<Cased>, but
+numerals, come in both upper and lower case, so they are C<Cased>, but
aren't considered to be letters, so they aren't C<Cased_Letter>s. They're
actually C<Letter_Number>s.)
This set also includes its subsets C<PosixUpper> and C<PosixLower>, both
-of which under C</i> match
ing match C<PosixAlpha>.
+of which under C</i> match C<PosixAlpha>.
For more details on Unicode properties, see L<perlunicode/Unicode
Character Properties>; for a
It is also possible to define your own properties. This is discussed in
L<perlunicode/User-Defined Character Properties>.
+Unicode properties are defined (surprise!) only on Unicode code points.
+A warning is raised and all matches fail on non-Unicode code points
+(those above the legal Unicode maximum of 0x10FFFF). This can be
+somewhat surprising,
+
+ chr(0x110000) =~ \p{ASCII_Hex_Digit=True} # Fails.
+ chr(0x110000) =~ \p{ASCII_Hex_Digit=False} # Also fails!
+
+Even though these two matches might be thought of as complements, they
+are so only on Unicode code points.
+
=head4 Examples
"a" =~ /\w/ # Match, "a" is a 'word' character.
is the bracketed character class. In its simplest form, it lists the characters
that may be matched, surrounded by square brackets, like this: C<[aeiou]>.
This matches one of C<a>, C<e>, C<i>, C<o> or C<u>. Like the other
-character classes, exactly one character is matched. To match
+character classes, exactly one character is matched.* To match
a longer string consisting of characters mentioned in the character
class, follow the character class with a L<quantifier|perlre/Quantifiers>. For
instance, C<[aeiou]+> matches one or more lowercase English vowels.
# a single character.
"ae" =~ /^[aeiou]+$/ # Match, due to the quantifier.
+ -------
+
+* There is an exception to a bracketed character class matching a
+single character only. When the class is to match caselessly under C</i>
+matching rules, and a character that is explicitly mentioned inside the
+class matches a
+multiple-character sequence caselessly under Unicode rules, the class
+(when not L<inverted|/Negation>) will also match that sequence. For
+example, Unicode says that the letter C<LATIN SMALL LETTER SHARP S>
+should match the sequence C<ss> under C</i> rules. Thus,
+
+ 'ss' =~ /\A\N{LATIN SMALL LETTER SHARP S}\z/i # Matches
+ 'ss' =~ /\A[aeioust\N{LATIN SMALL LETTER SHARP S}]\z/i # Matches
+
+For this to happen, the character must be explicitly specified, and not
+be part of a multi-character range (not even as one of its endpoints).
+(L</Character Ranges> will be explained shortly.) Therefore,
+
+ 'ss' =~ /\A[\0-\x{ff}]\z/i # Doesn't match
+ 'ss' =~ /\A[\0-\N{LATIN SMALL LETTER SHARP S}]\z/i # No match
+ 'ss' =~ /\A[\xDF-\xDF]\z/i # Matches on ASCII platforms, since \XDF
+ # is LATIN SMALL LETTER SHARP S, and the
+ # range is just a single element
+
+Note that it isn't a good idea to specify these types of ranges anyway.
+
=head3 Special Characters Inside a Bracketed Character Class
Most characters that are meta characters in regular expressions (that
If inside a bracketed character class you have two characters separated
by a hyphen, it's treated as if all characters between the two were in
the class. For instance, C<[0-9]> matches any ASCII digit, and C<[a-m]>
-matches any lowercase letter from the first half of the old ASCII alphabet.
+matches any lowercase letter from the first half of the ASCII alphabet.
Note that the two characters on either side of the hyphen are not
necessarily both letters or both digits. Any character is possible,
It is also possible to instead list the characters you do not want to
match. You can do so by using a caret (C<^>) as the first character in the
character class. For instance, C<[^a-z]> matches any character that is not a
-lowercase ASCII letter, which therefore includes almost a hundred thousand
-Unicode letters.
+lowercase ASCII letter, which therefore includes more than a million
+Unicode code points. The class is said to be "negated" or "inverted".
This syntax make the caret a special character inside a bracketed character
class, but only if it is the first character of the class. So if you want
the caret as one of the characters to match, either escape the caret or
-else not list it first.
+else don't list it first.
+
+In inverted bracketed character classes, Perl ignores the Unicode rules
+that normally say that certain characters should match a sequence of
+multiple characters under caseless C</i> matching. Following those
+rules could lead to highly confusing situations:
+
+ "ss" =~ /^[^\xDF]+$/ui; # Matches!
+
+This should match any sequences of characters that aren't C<\xDF> nor
+what C<\xDF> matches under C</i>. C<"s"> isn't C<\xDF>, but Unicode
+says that C<"ss"> is what C<\xDF> matches under C</i>. So which one
+"wins"? Do you fail the match because the string has C<ss> or accept it
+because it has an C<s> followed by another C<s>? Perl has chosen the
+latter.
Examples:
appropriate characters in the full Unicode character set. For example,
C<\p{Alpha}> matches not just the ASCII alphabetic characters, but any
character in the entire Unicode character set considered alphabetic.
-The column labelled "backslash sequence" is a (short) synonym for
-the Full-range Unicode form.
-
-(Each of the counterparts has various synonyms as well.
-L<perluniprops/Properties accessible through \p{} and \P{}> lists all
-synonyms, plus all characters matched by each ASCII-range property.
-For example, C<\p{AHex}> is a synonym for C<\p{ASCII_Hex_Digit}>,
-and any C<\p> property name can be prefixed with "Is" such as C<\p{IsAlpha}>.)
-
-Both the C<\p> counterparts always assume Unicode rules are in effect.
-On ASCII platforms, this means they assume that the code points from 128
-to 255 are Latin-1, and that means that using them under locale rules is
-unwise unless the locale is guaranteed to be Latin-1 or UTF-8. In contrast, the
-POSIX character classes are useful under locale rules. They are
-affected by the actual rules in effect, as follows:
-
-=over
-
-=item If the C</a> modifier, is in effect ...
-
-Each of the POSIX classes matches exactly the same as their ASCII-range
-counterparts.
-
-=item otherwise ...
-
-=over
-
-=item For code points above 255 ...
-
-The POSIX class matches the same as its Full-range counterpart.
-
-=item For code points below 256 ...
-
-=over
-
-=item if locale rules are in effect ...
-
-The POSIX class matches according to the locale.
-
-=item if Unicode rules are in effect or if on an EBCDIC platform ...
-
-The POSIX class matches the same as the Full-range counterpart.
-
-=item otherwise ...
-
-The POSIX class matches the same as the ASCII range counterpart.
-
-=back
-
-=back
-
-=back
-
-Which rules apply are determined as described in
-L<perlre/Which character set modifier is in effect?>.
-
-It is proposed to change this behavior in a future release of Perl so that
-whether or not Unicode rules are in effect would not change the
-behavior: Outside of locale or an EBCDIC code page, the POSIX classes
-would behave like their ASCII-range counterparts. If you wish to
-comment on this proposal, send email to C<perl5-porters@perl.org>.
+An entry in the column labelled "backslash sequence" is a (short)
+equivalent.
[[:...:]] ASCII-range Full-range backslash Note
Unicode Unicode sequence
On EBCDIC platforms, it is likely that the code page will define C<[[:cntrl:]]>
to be the EBCDIC equivalents of the ASCII controls, plus the controls
-that in Unicode have code pointss from 128 through 159.
+that in Unicode have code points from 128 through 159.
=item [3]
The similarly named property, C<\p{Punct}>, matches a somewhat different
set in the ASCII range, namely
-C<[-!"#%&'()*,./:;?@[\\\]_{}]>. That is, it is missing C<[$+E<lt>=E<gt>^`|~]>.
+C<[-!"#%&'()*,./:;?@[\\\]_{}]>. That is, it is missing the nine
+characters C<[$+E<lt>=E<gt>^`|~]>.
This is because Unicode splits what POSIX considers to be punctuation into two
categories, Punctuation and Symbols.
-C<\p{XPosixPunct}> and (in Unicode mode) C<[[:punct:]]>, match what
+C<\p{XPosixPunct}> and (under Unicode rules) C<[[:punct:]]>, match what
C<\p{PosixPunct}> matches in the ASCII range, plus what C<\p{Punct}>
matches. This is different than strictly matching according to
C<\p{Punct}>. Another way to say it is that
=back
-There are various other synonyms that can be used for these besides
-C<\p{HorizSpace}> and \C<\p{XPosixBlank}>. For example,
-C<\p{PosixAlpha}> can be written as C<\p{Alpha}>. All are listed
-in L<perluniprops/Properties accessible through \p{} and \P{}>.
+There are various other synonyms that can be used besides the names
+listed in the table. For example, C<\p{PosixAlpha}> can be written as
+C<\p{Alpha}>. All are listed in
+L<perluniprops/Properties accessible through \p{} and \P{}>,
+plus all characters matched by each ASCII-range property.
+
+Both the C<\p> counterparts always assume Unicode rules are in effect.
+On ASCII platforms, this means they assume that the code points from 128
+to 255 are Latin-1, and that means that using them under locale rules is
+unwise unless the locale is guaranteed to be Latin-1 or UTF-8. In contrast, the
+POSIX character classes are useful under locale rules. They are
+affected by the actual rules in effect, as follows:
+
+=over
+
+=item If the C</a> modifier, is in effect ...
+
+Each of the POSIX classes matches exactly the same as their ASCII-range
+counterparts.
+
+=item otherwise ...
+
+=over
+
+=item For code points above 255 ...
+
+The POSIX class matches the same as its Full-range counterpart.
+
+=item For code points below 256 ...
+
+=over
+
+=item if locale rules are in effect ...
+
+The POSIX class matches according to the locale, except that
+C<word> uses the platform's native underscore character, no matter what
+the locale is.
+
+=item if Unicode rules are in effect or if on an EBCDIC platform ...
+
+The POSIX class matches the same as the Full-range counterpart.
+
+=item otherwise ...
+
+The POSIX class matches the same as the ASCII range counterpart.
+
+=back
+
+=back
+
+=back
+
+Which rules apply are determined as described in
+L<perlre/Which character set modifier is in effect?>.
+
+It is proposed to change this behavior in a future release of Perl so that
+whether or not Unicode rules are in effect would not change the
+behavior: Outside of locale or an EBCDIC code page, the POSIX classes
+would behave like their ASCII-range counterparts. If you wish to
+comment on this proposal, send email to C<perl5-porters@perl.org>.
-=head4 Negation
+=head4 Negation of POSIX character classes
X<character class, negation>
A Perl extension to the POSIX character class is the ability to
/[01[:lower:]]/ # Matches a character that is either a
# lowercase letter, or '0' or '1'.
/[[:digit:][:^xdigit:]]/ # Matches a character that can be anything
- # except the letters 'a' to 'f'. This is
- # because the main character class is composed
- # of two POSIX character classes that are ORed
- # together, one that matches any digit, and
- # the other that matches anything that isn't a
- # hex digit. The result matches all
- # characters except the letters 'a' to 'f' and
- # 'A' to 'F'.
+ # except the letters 'a' to 'f' and 'A' to
+ # 'F'. This is because the main character
+ # class is composed of two POSIX character
+ # classes that are ORed together, one that
+ # matches any digit, and the other that
+ # matches anything that isn't a hex digit.
+ # The OR adds the digits, leaving only the
+ # letters 'a' to 'f' and 'A' to 'F' excluded.
https://perl5.git.perl.org / perl5.git / blobdiff