5 use unicore::Name; # mktables-generated algorithmically-defined names
6 use _charnames (); # The submodule for this where most of the work gets done
8 use bytes (); # for $bytes::hint_bits
9 use re "/aa"; # Everything in here should be ASCII
11 # Translate between Unicode character names and their code points.
12 # This is a wrapper around the submodule C<_charnames>. This design allows
13 # C<_charnames> to be autoloaded to enable use of \N{...}, but requires this
14 # module to be explicitly requested for the functions API.
16 $Carp::Internal{ (__PACKAGE__) } = 1;
20 shift; ## ignore class name
21 _charnames->import(@_);
24 # Cache of already looked-up values. This is set to only contain
25 # official values, and user aliases can't override them, so scoping is
30 return _charnames::viacode(@_);
36 _charnames::carp "charnames::vianame() expects one name argument";
40 # Looks up the character name and returns its ordinal if
41 # found, undef otherwise.
45 if ($arg =~ /^U\+([0-9a-fA-F]+)$/) {
47 # khw claims that this is poor interface design. The function should
48 # return either a an ord or a chr for all inputs; not be bipolar. But
49 # can't change it because of backward compatibility. New code can use
50 # string_vianame() instead.
51 my $ord = CORE::hex $1;
52 return chr $ord if $ord <= 255 || ! ((caller 0)[8] & $bytes::hint_bits);
53 _charnames::carp _charnames::not_legal_use_bytes_msg($arg, chr $ord);
57 # The first 1 arg means wants an ord returned; the second that we are in
58 # runtime, and this is the first level routine called from the user
59 return _charnames::lookup_name($arg, 1, 1);
64 # Looks up the character name and returns its string representation if
65 # found, undef otherwise.
68 _charnames::carp "charnames::string_vianame() expects one name argument";
74 if ($arg =~ /^U\+([0-9a-fA-F]+)$/) {
76 my $ord = CORE::hex $1;
77 return chr $ord if $ord <= 255 || ! ((caller 0)[8] & $bytes::hint_bits);
79 _charnames::carp _charnames::not_legal_use_bytes_msg($arg, chr $ord);
83 # The 0 arg means wants a string returned; the 1 arg means that we are in
84 # runtime, and this is the first level routine called from the user
85 return _charnames::lookup_name($arg, 0, 1);
93 charnames - access to Unicode character names and named character sequences; also define character names
97 use charnames ':full';
98 print "\N{GREEK SMALL LETTER SIGMA} is called sigma.\n";
99 print "\N{LATIN CAPITAL LETTER E WITH VERTICAL LINE BELOW}",
100 " is an officially named sequence of two Unicode characters\n";
102 use charnames ':loose';
103 print "\N{Greek small-letter sigma}",
104 "can be used to ignore case, underscores, most blanks,"
105 "and when you aren't sure if the official name has hyphens\n";
107 use charnames ':short';
108 print "\N{greek:Sigma} is an upper-case sigma.\n";
110 use charnames qw(cyrillic greek);
111 print "\N{sigma} is Greek sigma, and \N{be} is Cyrillic b.\n";
113 use charnames ":full", ":alias" => {
114 e_ACUTE => "LATIN SMALL LETTER E WITH ACUTE",
115 mychar => 0xE8000, # Private use area
117 print "\N{e_ACUTE} is a small letter e with an acute.\n";
118 print "\N{mychar} allows me to name private use characters.\n";
121 print charnames::viacode(0x1234); # prints "ETHIOPIC SYLLABLE SEE"
122 printf "%04X", charnames::vianame("GOTHIC LETTER AHSA"); # prints
124 print charnames::vianame("LATIN CAPITAL LETTER A"); # prints 65 on
127 print charnames::string_vianame("LATIN CAPITAL LETTER A"); # prints "A"
131 Pragma C<use charnames> is used to gain access to the names of the
132 Unicode characters and named character sequences, and to allow you to define
133 your own character and character sequence names.
135 All forms of the pragma enable use of the following 3 functions:
141 L</charnames::string_vianame(I<name>)> for run-time lookup of a
142 either a character name or a named character sequence, returning its string
147 L</charnames::vianame(I<name>)> for run-time lookup of a
148 character name (but not a named character sequence) to get its ordinal value
153 L</charnames::viacode(I<code>)> for run-time lookup of a code point to get its
158 Starting in Perl v5.16, any occurrence of C<\N{I<CHARNAME>}> sequences
159 in a double-quotish string automatically loads this module with arguments
160 C<:full> and C<:short> (described below) if it hasn't already been loaded with
161 different arguments, in order to compile the named Unicode character into
162 position in the string. Prior to v5.16, an explicit S<C<use charnames>> was
163 required to enable this usage. (However, prior to v5.16, the form C<S<"use
164 charnames ();">> did not enable C<\N{I<CHARNAME>}>.)
166 Note that C<\N{U+I<...>}>, where the I<...> is a hexadecimal number,
167 also inserts a character into a string.
168 The character it inserts is the one whose code point
169 (ordinal value) is equal to the number. For example, C<"\N{U+263a}"> is
170 the Unicode (white background, black foreground) smiley face
171 equivalent to C<"\N{WHITE SMILING FACE}">.
172 Also note, C<\N{I<...>}> can mean a regex quantifier instead of a character
173 name, when the I<...> is a number (or comma separated pair of numbers
174 (see L<perlreref/QUANTIFIERS>), and is not related to this pragma.
176 The C<charnames> pragma supports arguments C<:full>, C<:loose>, C<:short>,
177 script names and L<customized aliases|/CUSTOM ALIASES>.
179 If C<:full> is present, for expansion of
180 C<\N{I<CHARNAME>}>, the string I<CHARNAME> is first looked up in the list of
181 standard Unicode character names.
183 C<:loose> is a variant of C<:full> which allows I<CHARNAME> to be less
184 precisely specified. Details are in L</LOOSE MATCHES>.
186 If C<:short> is present, and
187 I<CHARNAME> has the form C<I<SCRIPT>:I<CNAME>>, then I<CNAME> is looked up
188 as a letter in script I<SCRIPT>, as described in the next paragraph.
189 Or, if C<use charnames> is used
190 with script name arguments, then for C<\N{I<CHARNAME>}> the name
191 I<CHARNAME> is looked up as a letter in the given scripts (in the
192 specified order). Customized aliases can override these, and are explained in
195 For lookup of I<CHARNAME> inside a given script I<SCRIPTNAME>,
196 this pragma looks in the table of standard Unicode names for the names
198 SCRIPTNAME CAPITAL LETTER CHARNAME
199 SCRIPTNAME SMALL LETTER CHARNAME
200 SCRIPTNAME LETTER CHARNAME
202 If I<CHARNAME> is all lowercase,
203 then the C<CAPITAL> variant is ignored, otherwise the C<SMALL> variant
204 is ignored, and both I<CHARNAME> and I<SCRIPTNAME> are converted to all
205 uppercase for look-up. Other than that, both of them follow L<loose|/LOOSE
206 MATCHES> rules if C<:loose> is also specified; strict otherwise.
208 Note that C<\N{...}> is compile-time; it's a special form of string
209 constant used inside double-quotish strings; this means that you cannot
210 use variables inside the C<\N{...}>. If you want similar run-time
212 L<charnames::string_vianame()|/charnames::string_vianame(I<name>)>.
214 Since Unicode 6.0, it is deprecated to use C<BELL>. Instead use C<ALERT> (but
215 C<BEL> will continue to work).
217 If the input name is unknown, C<\N{NAME}> raises a warning and
218 substitutes the Unicode REPLACEMENT CHARACTER (U+FFFD).
220 For C<\N{NAME}>, it is a fatal error if C<use bytes> is in effect and the
221 input name is that of a character that won't fit into a byte (i.e., whose
222 ordinal is above 255).
224 Otherwise, any string that includes a C<\N{I<charname>}> or
225 C<S<\N{U+I<code point>}>> will automatically have Unicode semantics (see
226 L<perlunicode/Byte and Character Semantics>).
230 By specifying C<:loose>, Unicode's L<loose character name
231 matching|http://www.unicode.org/reports/tr44#Matching_Rules> rules are
232 selected instead of the strict exact match used otherwise.
233 That means that I<CHARNAME> doesn't have to be so precisely specified.
234 Upper/lower case doesn't matter (except with scripts as mentioned above), nor
235 do any underscores, and the only hyphens that matter are those at the
236 beginning or end of a word in the name (with one exception: the hyphen in
237 U+1180 C<HANGUL JUNGSEONG O-E> does matter).
238 Also, blanks not adjacent to hyphens don't matter.
239 The official Unicode names are quite variable as to where they use hyphens
240 versus spaces to separate word-like units, and this option allows you to not
241 have to care as much.
242 The reason non-medial hyphens matter is because of cases like
243 U+0F60 C<TIBETAN LETTER -A> versus U+0F68 C<TIBETAN LETTER A>.
244 The hyphen here is significant, as is the space before it, and so both must be
247 C<:loose> slows down look-ups by a factor of 2 to 3 versus
248 C<:full>, but the trade-off may be worth it to you. Each individual look-up
249 takes very little time, and the results are cached, so the speed difference
250 would become a factor only in programs that do look-ups of many different
251 spellings, and probably only when those look-ups are through vianame() and
252 string_vianame(), since C<\N{...}> look-ups are done at compile time.
256 Starting in Unicode 6.1 and Perl v5.16, Unicode defines many abbreviations and
257 names that were formerly Perl extensions, and some additional ones that Perl
258 did not previously accept. The list is getting too long to reproduce here,
259 but you can get the complete list from the Unicode web site:
260 L<http://www.unicode.org/Public/UNIDATA/NameAliases.txt>.
262 Earlier versions of Perl accepted almost all the 6.1 names. These were most
263 extensively documented in the v5.14 version of this pod:
264 L<http://perldoc.perl.org/5.14.0/charnames.html#ALIASES>.
266 =head1 CUSTOM ALIASES
268 You can add customized aliases to standard (C<:full>) Unicode naming
269 conventions. The aliases override any standard definitions, so, if
270 you're twisted enough, you can change C<"\N{LATIN CAPITAL LETTER A}"> to
273 Note that an alias should not be something that is a legal curly
274 brace-enclosed quantifier (see L<perlreref/QUANTIFIERS>). For example
275 C<\N{123}> means to match 123 non-newline characters, and is not treated as a
276 charnames alias. Aliases are discouraged from beginning with anything
277 other than an alphabetic character and from containing anything other
278 than alphanumerics, spaces, dashes, parentheses, and underscores.
279 Currently they must be ASCII.
281 An alias can map to either an official Unicode character name (not a loose
282 matched name) or to a
283 numeric code point (ordinal). The latter is useful for assigning names
284 to code points in Unicode private use areas such as U+E800 through
286 A numeric code point must be a non-negative integer or a string beginning
287 with C<"U+"> or C<"0x"> with the remainder considered to be a
288 hexadecimal integer. A literal numeric constant must be unsigned; it
289 will be interpreted as hex if it has a leading zero or contains
290 non-decimal hex digits; otherwise it will be interpreted as decimal.
292 Aliases are added either by the use of anonymous hashes:
294 use charnames ":alias" => {
295 e_ACUTE => "LATIN SMALL LETTER E WITH ACUTE",
298 my $str = "\N{e_ACUTE}";
300 or by using a file containing aliases:
302 use charnames ":alias" => "pro";
304 This will try to read C<"unicore/pro_alias.pl"> from the C<@INC> path. This
305 file should return a list in plain perl:
308 A_GRAVE => "LATIN CAPITAL LETTER A WITH GRAVE",
309 A_CIRCUM => "LATIN CAPITAL LETTER A WITH CIRCUMFLEX",
310 A_DIAERES => "LATIN CAPITAL LETTER A WITH DIAERESIS",
311 A_TILDE => "LATIN CAPITAL LETTER A WITH TILDE",
312 A_BREVE => "LATIN CAPITAL LETTER A WITH BREVE",
313 A_RING => "LATIN CAPITAL LETTER A WITH RING ABOVE",
314 A_MACRON => "LATIN CAPITAL LETTER A WITH MACRON",
315 mychar2 => "U+E8001",
318 Both these methods insert C<":full"> automatically as the first argument (if no
319 other argument is given), and you can give the C<":full"> explicitly as
322 use charnames ":full", ":alias" => "pro";
324 C<":loose"> has no effect with these. Input names must match exactly, using
327 Also, both these methods currently allow only single characters to be named.
328 To name a sequence of characters, use a
329 L<custom translator|/CUSTOM TRANSLATORS> (described below).
331 =head1 charnames::string_vianame(I<name>)
333 This is a runtime equivalent to C<\N{...}>. I<name> can be any expression
334 that evaluates to a name accepted by C<\N{...}> under the L<C<:full>
335 option|/DESCRIPTION> to C<charnames>. In addition, any other options for the
336 controlling C<"use charnames"> in the same scope apply, like C<:loose> or any
337 L<script list, C<:short> option|/DESCRIPTION>, or L<custom aliases|/CUSTOM
338 ALIASES> you may have defined.
340 The only difference is that if the input name is unknown, C<string_vianame>
341 returns C<undef> instead of the REPLACEMENT CHARACTER and does not raise a
344 =head1 charnames::vianame(I<name>)
346 This is similar to C<string_vianame>. The main difference is that under most
347 circumstances, vianame returns an ordinal code
348 point, whereas C<string_vianame> returns a string. For example,
350 printf "U+%04X", charnames::vianame("FOUR TEARDROP-SPOKED ASTERISK");
354 This leads to the other two differences. Since a single code point is
355 returned, the function can't handle named character sequences, as these are
356 composed of multiple characters (it returns C<undef> for these. And, the code
357 point can be that of any
358 character, even ones that aren't legal under the C<S<use bytes>> pragma,
360 See L</BUGS> for the circumstances in which the behavior differs
361 from that described above.
363 =head1 charnames::viacode(I<code>)
365 Returns the full name of the character indicated by the numeric code.
368 print charnames::viacode(0x2722);
370 prints "FOUR TEARDROP-SPOKED ASTERISK".
372 The name returned is the "best" (defined below) official name or alias
373 for the code point, if
374 available; otherwise your custom alias for it, if defined; otherwise C<undef>.
375 This means that your alias will only be returned for code points that don't
376 have an official Unicode name (nor alias) such as private use code points.
378 If you define more than one name for the code point, it is indeterminate
379 which one will be returned.
381 As mentioned, the function returns C<undef> if no name is known for the code
382 point. In Unicode the proper name of these is the empty string, which
383 C<undef> stringifies to. (If you ask for a code point past the legal
384 Unicode maximum of U+10FFFF that you haven't assigned an alias to, you
385 get C<undef> plus a warning.)
387 The input number must be a non-negative integer, or a string beginning
388 with C<"U+"> or C<"0x"> with the remainder considered to be a
389 hexadecimal integer. A literal numeric constant must be unsigned; it
390 will be interpreted as hex if it has a leading zero or contains
391 non-decimal hex digits; otherwise it will be interpreted as decimal.
393 As mentioned above under L</ALIASES>, Unicode 6.1 defines extra names
394 (synonyms or aliases) for some code points, most of which were already
395 available as Perl extensions. All these are accepted by C<\N{...}> and the
396 other functions in this module, but C<viacode> has to choose which one
397 name to return for a given input code point, so it returns the "best" name.
398 To understand how this works, it is helpful to know more about the Unicode
399 name properties. All code points actually have only a single name, which
400 (starting in Unicode 2.0) can never change once a character has been assigned
401 to the code point. But mistakes have been made in assigning names, for
402 example sometimes a clerical error was made during the publishing of the
403 Standard which caused words to be misspelled, and there was no way to correct
404 those. The Name_Alias property was eventually created to handle these
405 situations. If a name was wrong, a corrected synonym would be published for
406 it, using Name_Alias. C<viacode> will return that corrected synonym as the
407 "best" name for a code point. (It is even possible, though it hasn't happened
408 yet, that the correction itself will need to be corrected, and so another
409 Name_Alias can be created for that code point; C<viacode> will return the
410 most recent correction.)
412 The Unicode name for each of the control characters (such as LINE FEED) is the
413 empty string. However almost all had names assigned by other standards, such
414 as the ASCII Standard, or were in common use. C<viacode> returns these names
415 as the "best" ones available. Unicode 6.1 has created Name_Aliases for each
416 of them, including alternate names, like NEW LINE. C<viacode> uses the
417 original name, "LINE FEED" in preference to the alternate. Similarly the
418 name returned for U+FEFF is "ZERO WIDTH NO-BREAK SPACE", not "BYTE ORDER
421 Until Unicode 6.1, the 4 control characters U+0080, U+0081, U+0084, and U+0099
422 did not have names nor aliases.
423 To preserve backwards compatibility, any alias you define for these code
424 points will be returned by this function, in preference to the official name.
426 Some code points also have abbreviated names, such as "LF" or "NL".
427 C<viacode> never returns these.
429 Because a name correction may be added in future Unicode releases, the name
430 that C<viacode> returns may change as a result. This is a rare event, but it
433 =head1 CUSTOM TRANSLATORS
435 The mechanism of translation of C<\N{...}> escapes is general and not
436 hardwired into F<charnames.pm>. A module can install custom
437 translations (inside the scope which C<use>s the module) with the
438 following magic incantation:
442 $^H{charnames} = \&translator;
445 Here translator() is a subroutine which takes I<CHARNAME> as an
446 argument, and returns text to insert into the string instead of the
447 C<\N{I<CHARNAME>}> escape.
449 This is the only way you can create a custom named sequence of code points.
451 Since the text to insert should be different
452 in C<bytes> mode and out of it, the function should check the current
453 state of C<bytes>-flag as in:
455 use bytes (); # for $bytes::hint_bits
457 if ($^H & $bytes::hint_bits) {
458 return bytes_translator(@_);
461 return utf8_translator(@_);
465 See L</CUSTOM ALIASES> above for restrictions on I<CHARNAME>.
467 Of course, C<vianame>, C<viacode>, and C<string_vianame> would need to be
472 vianame() normally returns an ordinal code point, but when the input name is of
473 the form C<U+...>, it returns a chr instead. In this case, if C<use bytes> is
474 in effect and the character won't fit into a byte, it returns C<undef> and
477 Names must be ASCII characters only, which means that you are out of luck if
478 you want to create aliases in a language where some or all the characters of
479 the desired aliases are non-ASCII.
481 Since evaluation of the translation function (see L</CUSTOM
482 TRANSLATORS>) happens in the middle of compilation (of a string
483 literal), the translation function should not do any C<eval>s or
484 C<require>s. This restriction should be lifted (but is low priority) in
485 a future version of Perl.
489 # ex: set ts=8 sts=2 sw=2 et: