Commit | Line | Data |
---|---|---|
5f05dabc | 1 | =head1 NAME |
2 | ||
b0c42ed9 | 3 | perllocale - Perl locale handling (internationalization and localization) |
5f05dabc | 4 | |
5 | =head1 DESCRIPTION | |
6 | ||
e199995e KW |
7 | Locales these days have been mostly been supplanted by Unicode, but Perl |
8 | continues to support them. See L</Unicode and UTF-8> below. | |
9 | ||
5a964f20 TC |
10 | Perl supports language-specific notions of data such as "is this |
11 | a letter", "what is the uppercase equivalent of this letter", and | |
12 | "which of these letters comes first". These are important issues, | |
13 | especially for languages other than English--but also for English: it | |
14 | would be naE<iuml>ve to imagine that C<A-Za-z> defines all the "letters" | |
b4ffc3db TC |
15 | needed to write correct English. Perl is also aware that some character other |
16 | than "." may be preferred as a decimal point, and that output date | |
5a964f20 TC |
17 | representations may be language-specific. The process of making an |
18 | application take account of its users' preferences in such matters is | |
19 | called B<internationalization> (often abbreviated as B<i18n>); telling | |
20 | such an application about a particular set of preferences is known as | |
21 | B<localization> (B<l10n>). | |
14280422 DD |
22 | |
23 | Perl can understand language-specific data via the standardized (ISO C, | |
24 | XPG4, POSIX 1.c) method called "the locale system". The locale system is | |
b0c42ed9 | 25 | controlled per application using one pragma, one function call, and |
14280422 DD |
26 | several environment variables. |
27 | ||
28 | B<NOTE>: This feature is new in Perl 5.004, and does not apply unless an | |
5a964f20 | 29 | application specifically requests it--see L<Backward compatibility>. |
e38874e2 DD |
30 | The one exception is that write() now B<always> uses the current locale |
31 | - see L<"NOTES">. | |
5f05dabc | 32 | |
33 | =head1 PREPARING TO USE LOCALES | |
34 | ||
5a964f20 | 35 | If Perl applications are to understand and present your data |
14280422 | 36 | correctly according a locale of your choice, B<all> of the following |
5f05dabc | 37 | must be true: |
38 | ||
39 | =over 4 | |
40 | ||
41 | =item * | |
42 | ||
43 | B<Your operating system must support the locale system>. If it does, | |
14280422 | 44 | you should find that the setlocale() function is a documented part of |
5f05dabc | 45 | its C library. |
46 | ||
47 | =item * | |
48 | ||
5a964f20 | 49 | B<Definitions for locales that you use must be installed>. You, or |
14280422 DD |
50 | your system administrator, must make sure that this is the case. The |
51 | available locales, the location in which they are kept, and the manner | |
5a964f20 TC |
52 | in which they are installed all vary from system to system. Some systems |
53 | provide only a few, hard-wired locales and do not allow more to be | |
54 | added. Others allow you to add "canned" locales provided by the system | |
55 | supplier. Still others allow you or the system administrator to define | |
14280422 | 56 | and add arbitrary locales. (You may have to ask your supplier to |
5a964f20 | 57 | provide canned locales that are not delivered with your operating |
14280422 | 58 | system.) Read your system documentation for further illumination. |
5f05dabc | 59 | |
60 | =item * | |
61 | ||
62 | B<Perl must believe that the locale system is supported>. If it does, | |
63 | C<perl -V:d_setlocale> will say that the value for C<d_setlocale> is | |
64 | C<define>. | |
65 | ||
66 | =back | |
67 | ||
68 | If you want a Perl application to process and present your data | |
69 | according to a particular locale, the application code should include | |
2ae324a7 | 70 | the S<C<use locale>> pragma (see L<The use locale pragma>) where |
5f05dabc | 71 | appropriate, and B<at least one> of the following must be true: |
72 | ||
73 | =over 4 | |
74 | ||
75 | =item * | |
76 | ||
14280422 | 77 | B<The locale-determining environment variables (see L<"ENVIRONMENT">) |
5a964f20 TC |
78 | must be correctly set up> at the time the application is started, either |
79 | by yourself or by whoever set up your system account. | |
5f05dabc | 80 | |
81 | =item * | |
82 | ||
14280422 DD |
83 | B<The application must set its own locale> using the method described in |
84 | L<The setlocale function>. | |
5f05dabc | 85 | |
86 | =back | |
87 | ||
88 | =head1 USING LOCALES | |
89 | ||
90 | =head2 The use locale pragma | |
91 | ||
14280422 DD |
92 | By default, Perl ignores the current locale. The S<C<use locale>> |
93 | pragma tells Perl to use the current locale for some operations: | |
5f05dabc | 94 | |
95 | =over 4 | |
96 | ||
97 | =item * | |
98 | ||
14280422 DD |
99 | B<The comparison operators> (C<lt>, C<le>, C<cmp>, C<ge>, and C<gt>) and |
100 | the POSIX string collation functions strcoll() and strxfrm() use | |
5a964f20 TC |
101 | C<LC_COLLATE>. sort() is also affected if used without an |
102 | explicit comparison function, because it uses C<cmp> by default. | |
14280422 | 103 | |
5a964f20 | 104 | B<Note:> C<eq> and C<ne> are unaffected by locale: they always |
de108802 | 105 | perform a char-by-char comparison of their scalar operands. What's |
14280422 DD |
106 | more, if C<cmp> finds that its operands are equal according to the |
107 | collation sequence specified by the current locale, it goes on to | |
de108802 RGS |
108 | perform a char-by-char comparison, and only returns I<0> (equal) if the |
109 | operands are char-for-char identical. If you really want to know whether | |
5a964f20 | 110 | two strings--which C<eq> and C<cmp> may consider different--are equal |
14280422 DD |
111 | as far as collation in the locale is concerned, see the discussion in |
112 | L<Category LC_COLLATE: Collation>. | |
5f05dabc | 113 | |
114 | =item * | |
115 | ||
14280422 DD |
116 | B<Regular expressions and case-modification functions> (uc(), lc(), |
117 | ucfirst(), and lcfirst()) use C<LC_CTYPE> | |
5f05dabc | 118 | |
119 | =item * | |
120 | ||
903eb63f | 121 | B<Format declarations> (format()) use C<LC_NUMERIC> |
5f05dabc | 122 | |
123 | =item * | |
124 | ||
14280422 | 125 | B<The POSIX date formatting function> (strftime()) uses C<LC_TIME>. |
5f05dabc | 126 | |
127 | =back | |
128 | ||
13a2d996 SP |
129 | C<LC_COLLATE>, C<LC_CTYPE>, and so on, are discussed further in |
130 | L<LOCALE CATEGORIES>. | |
5f05dabc | 131 | |
5a964f20 TC |
132 | The default behavior is restored with the S<C<no locale>> pragma, or |
133 | upon reaching the end of block enclosing C<use locale>. | |
5f05dabc | 134 | |
5a964f20 | 135 | The string result of any operation that uses locale |
14280422 DD |
136 | information is tainted, as it is possible for a locale to be |
137 | untrustworthy. See L<"SECURITY">. | |
5f05dabc | 138 | |
139 | =head2 The setlocale function | |
140 | ||
14280422 DD |
141 | You can switch locales as often as you wish at run time with the |
142 | POSIX::setlocale() function: | |
5f05dabc | 143 | |
144 | # This functionality not usable prior to Perl 5.004 | |
145 | require 5.004; | |
146 | ||
147 | # Import locale-handling tool set from POSIX module. | |
148 | # This example uses: setlocale -- the function call | |
149 | # LC_CTYPE -- explained below | |
150 | use POSIX qw(locale_h); | |
151 | ||
14280422 | 152 | # query and save the old locale |
5f05dabc | 153 | $old_locale = setlocale(LC_CTYPE); |
154 | ||
155 | setlocale(LC_CTYPE, "fr_CA.ISO8859-1"); | |
156 | # LC_CTYPE now in locale "French, Canada, codeset ISO 8859-1" | |
157 | ||
158 | setlocale(LC_CTYPE, ""); | |
159 | # LC_CTYPE now reset to default defined by LC_ALL/LC_CTYPE/LANG | |
160 | # environment variables. See below for documentation. | |
161 | ||
162 | # restore the old locale | |
163 | setlocale(LC_CTYPE, $old_locale); | |
164 | ||
14280422 DD |
165 | The first argument of setlocale() gives the B<category>, the second the |
166 | B<locale>. The category tells in what aspect of data processing you | |
167 | want to apply locale-specific rules. Category names are discussed in | |
168 | L<LOCALE CATEGORIES> and L<"ENVIRONMENT">. The locale is the name of a | |
169 | collection of customization information corresponding to a particular | |
170 | combination of language, country or territory, and codeset. Read on for | |
171 | hints on the naming of locales: not all systems name locales as in the | |
172 | example. | |
173 | ||
502a173a JH |
174 | If no second argument is provided and the category is something else |
175 | than LC_ALL, the function returns a string naming the current locale | |
176 | for the category. You can use this value as the second argument in a | |
177 | subsequent call to setlocale(). | |
178 | ||
179 | If no second argument is provided and the category is LC_ALL, the | |
180 | result is implementation-dependent. It may be a string of | |
181 | concatenated locales names (separator also implementation-dependent) | |
f979aebc | 182 | or a single locale name. Please consult your setlocale(3) man page for |
502a173a JH |
183 | details. |
184 | ||
185 | If a second argument is given and it corresponds to a valid locale, | |
186 | the locale for the category is set to that value, and the function | |
187 | returns the now-current locale value. You can then use this in yet | |
188 | another call to setlocale(). (In some implementations, the return | |
189 | value may sometimes differ from the value you gave as the second | |
190 | argument--think of it as an alias for the value you gave.) | |
5f05dabc | 191 | |
192 | As the example shows, if the second argument is an empty string, the | |
193 | category's locale is returned to the default specified by the | |
194 | corresponding environment variables. Generally, this results in a | |
5a964f20 | 195 | return to the default that was in force when Perl started up: changes |
54310121 | 196 | to the environment made by the application after startup may or may not |
5a964f20 | 197 | be noticed, depending on your system's C library. |
5f05dabc | 198 | |
14280422 DD |
199 | If the second argument does not correspond to a valid locale, the locale |
200 | for the category is not changed, and the function returns I<undef>. | |
5f05dabc | 201 | |
f979aebc | 202 | For further information about the categories, consult setlocale(3). |
3e6e419a JH |
203 | |
204 | =head2 Finding locales | |
205 | ||
f979aebc | 206 | For locales available in your system, consult also setlocale(3) to |
5a964f20 TC |
207 | see whether it leads to the list of available locales (search for the |
208 | I<SEE ALSO> section). If that fails, try the following command lines: | |
5f05dabc | 209 | |
210 | locale -a | |
211 | ||
212 | nlsinfo | |
213 | ||
214 | ls /usr/lib/nls/loc | |
215 | ||
216 | ls /usr/lib/locale | |
217 | ||
218 | ls /usr/lib/nls | |
219 | ||
b478f28d JH |
220 | ls /usr/share/locale |
221 | ||
5f05dabc | 222 | and see whether they list something resembling these |
223 | ||
2bdf8add | 224 | en_US.ISO8859-1 de_DE.ISO8859-1 ru_RU.ISO8859-5 |
502a173a | 225 | en_US.iso88591 de_DE.iso88591 ru_RU.iso88595 |
2bdf8add | 226 | en_US de_DE ru_RU |
14280422 | 227 | en de ru |
2bdf8add JH |
228 | english german russian |
229 | english.iso88591 german.iso88591 russian.iso88595 | |
502a173a | 230 | english.roman8 russian.koi8r |
5f05dabc | 231 | |
528d65ad JH |
232 | Sadly, even though the calling interface for setlocale() has been |
233 | standardized, names of locales and the directories where the | |
5a964f20 | 234 | configuration resides have not been. The basic form of the name is |
528d65ad JH |
235 | I<language_territory>B<.>I<codeset>, but the latter parts after |
236 | I<language> are not always present. The I<language> and I<country> | |
237 | are usually from the standards B<ISO 3166> and B<ISO 639>, the | |
238 | two-letter abbreviations for the countries and the languages of the | |
239 | world, respectively. The I<codeset> part often mentions some B<ISO | |
240 | 8859> character set, the Latin codesets. For example, C<ISO 8859-1> | |
241 | is the so-called "Western European codeset" that can be used to encode | |
242 | most Western European languages adequately. Again, there are several | |
243 | ways to write even the name of that one standard. Lamentably. | |
5f05dabc | 244 | |
14280422 DD |
245 | Two special locales are worth particular mention: "C" and "POSIX". |
246 | Currently these are effectively the same locale: the difference is | |
5a964f20 TC |
247 | mainly that the first one is defined by the C standard, the second by |
248 | the POSIX standard. They define the B<default locale> in which | |
14280422 | 249 | every program starts in the absence of locale information in its |
5a964f20 | 250 | environment. (The I<default> default locale, if you will.) Its language |
14280422 | 251 | is (American) English and its character codeset ASCII. |
5f05dabc | 252 | |
14280422 DD |
253 | B<NOTE>: Not all systems have the "POSIX" locale (not all systems are |
254 | POSIX-conformant), so use "C" when you need explicitly to specify this | |
255 | default locale. | |
5f05dabc | 256 | |
3e6e419a JH |
257 | =head2 LOCALE PROBLEMS |
258 | ||
5a964f20 | 259 | You may encounter the following warning message at Perl startup: |
3e6e419a JH |
260 | |
261 | perl: warning: Setting locale failed. | |
262 | perl: warning: Please check that your locale settings: | |
263 | LC_ALL = "En_US", | |
264 | LANG = (unset) | |
265 | are supported and installed on your system. | |
266 | perl: warning: Falling back to the standard locale ("C"). | |
267 | ||
5a964f20 TC |
268 | This means that your locale settings had LC_ALL set to "En_US" and |
269 | LANG exists but has no value. Perl tried to believe you but could not. | |
270 | Instead, Perl gave up and fell back to the "C" locale, the default locale | |
271 | that is supposed to work no matter what. This usually means your locale | |
272 | settings were wrong, they mention locales your system has never heard | |
273 | of, or the locale installation in your system has problems (for example, | |
274 | some system files are broken or missing). There are quick and temporary | |
275 | fixes to these problems, as well as more thorough and lasting fixes. | |
3e6e419a JH |
276 | |
277 | =head2 Temporarily fixing locale problems | |
278 | ||
5a964f20 | 279 | The two quickest fixes are either to render Perl silent about any |
3e6e419a JH |
280 | locale inconsistencies or to run Perl under the default locale "C". |
281 | ||
282 | Perl's moaning about locale problems can be silenced by setting the | |
900bd440 JH |
283 | environment variable PERL_BADLANG to a zero value, for example "0". |
284 | This method really just sweeps the problem under the carpet: you tell | |
285 | Perl to shut up even when Perl sees that something is wrong. Do not | |
286 | be surprised if later something locale-dependent misbehaves. | |
3e6e419a JH |
287 | |
288 | Perl can be run under the "C" locale by setting the environment | |
5a964f20 TC |
289 | variable LC_ALL to "C". This method is perhaps a bit more civilized |
290 | than the PERL_BADLANG approach, but setting LC_ALL (or | |
291 | other locale variables) may affect other programs as well, not just | |
292 | Perl. In particular, external programs run from within Perl will see | |
3e6e419a | 293 | these changes. If you make the new settings permanent (read on), all |
f979aebc | 294 | programs you run see the changes. See L<"ENVIRONMENT"> for |
5a964f20 TC |
295 | the full list of relevant environment variables and L<USING LOCALES> |
296 | for their effects in Perl. Effects in other programs are | |
297 | easily deducible. For example, the variable LC_COLLATE may well affect | |
b432a672 | 298 | your B<sort> program (or whatever the program that arranges "records" |
3e6e419a JH |
299 | alphabetically in your system is called). |
300 | ||
5a964f20 TC |
301 | You can test out changing these variables temporarily, and if the |
302 | new settings seem to help, put those settings into your shell startup | |
303 | files. Consult your local documentation for the exact details. For in | |
304 | Bourne-like shells (B<sh>, B<ksh>, B<bash>, B<zsh>): | |
3e6e419a JH |
305 | |
306 | LC_ALL=en_US.ISO8859-1 | |
307 | export LC_ALL | |
308 | ||
5a964f20 TC |
309 | This assumes that we saw the locale "en_US.ISO8859-1" using the commands |
310 | discussed above. We decided to try that instead of the above faulty | |
311 | locale "En_US"--and in Cshish shells (B<csh>, B<tcsh>) | |
3e6e419a JH |
312 | |
313 | setenv LC_ALL en_US.ISO8859-1 | |
c47ff5f1 | 314 | |
c406981e JH |
315 | or if you have the "env" application you can do in any shell |
316 | ||
317 | env LC_ALL=en_US.ISO8859-1 perl ... | |
318 | ||
5a964f20 | 319 | If you do not know what shell you have, consult your local |
3e6e419a JH |
320 | helpdesk or the equivalent. |
321 | ||
322 | =head2 Permanently fixing locale problems | |
323 | ||
5a964f20 TC |
324 | The slower but superior fixes are when you may be able to yourself |
325 | fix the misconfiguration of your own environment variables. The | |
3e6e419a JH |
326 | mis(sing)configuration of the whole system's locales usually requires |
327 | the help of your friendly system administrator. | |
328 | ||
5a964f20 TC |
329 | First, see earlier in this document about L<Finding locales>. That tells |
330 | how to find which locales are really supported--and more importantly, | |
331 | installed--on your system. In our example error message, environment | |
332 | variables affecting the locale are listed in the order of decreasing | |
333 | importance (and unset variables do not matter). Therefore, having | |
334 | LC_ALL set to "En_US" must have been the bad choice, as shown by the | |
335 | error message. First try fixing locale settings listed first. | |
3e6e419a | 336 | |
5a964f20 TC |
337 | Second, if using the listed commands you see something B<exactly> |
338 | (prefix matches do not count and case usually counts) like "En_US" | |
339 | without the quotes, then you should be okay because you are using a | |
340 | locale name that should be installed and available in your system. | |
4a4eefd0 | 341 | In this case, see L<Permanently fixing your system's locale configuration>. |
3e6e419a | 342 | |
4a4eefd0 | 343 | =head2 Permanently fixing your system's locale configuration |
3e6e419a | 344 | |
5a964f20 | 345 | This is when you see something like: |
3e6e419a JH |
346 | |
347 | perl: warning: Please check that your locale settings: | |
348 | LC_ALL = "En_US", | |
349 | LANG = (unset) | |
350 | are supported and installed on your system. | |
351 | ||
352 | but then cannot see that "En_US" listed by the above-mentioned | |
5a964f20 TC |
353 | commands. You may see things like "en_US.ISO8859-1", but that isn't |
354 | the same. In this case, try running under a locale | |
355 | that you can list and which somehow matches what you tried. The | |
3e6e419a | 356 | rules for matching locale names are a bit vague because |
13a2d996 SP |
357 | standardization is weak in this area. See again the |
358 | L<Finding locales> about general rules. | |
3e6e419a | 359 | |
b687b08b | 360 | =head2 Fixing system locale configuration |
3e6e419a | 361 | |
5a964f20 TC |
362 | Contact a system administrator (preferably your own) and report the exact |
363 | error message you get, and ask them to read this same documentation you | |
364 | are now reading. They should be able to check whether there is something | |
365 | wrong with the locale configuration of the system. The L<Finding locales> | |
366 | section is unfortunately a bit vague about the exact commands and places | |
367 | because these things are not that standardized. | |
3e6e419a | 368 | |
5f05dabc | 369 | =head2 The localeconv function |
370 | ||
14280422 DD |
371 | The POSIX::localeconv() function allows you to get particulars of the |
372 | locale-dependent numeric formatting information specified by the current | |
373 | C<LC_NUMERIC> and C<LC_MONETARY> locales. (If you just want the name of | |
374 | the current locale for a particular category, use POSIX::setlocale() | |
5a964f20 | 375 | with a single parameter--see L<The setlocale function>.) |
5f05dabc | 376 | |
377 | use POSIX qw(locale_h); | |
5f05dabc | 378 | |
379 | # Get a reference to a hash of locale-dependent info | |
380 | $locale_values = localeconv(); | |
381 | ||
382 | # Output sorted list of the values | |
383 | for (sort keys %$locale_values) { | |
14280422 | 384 | printf "%-20s = %s\n", $_, $locale_values->{$_} |
5f05dabc | 385 | } |
386 | ||
14280422 | 387 | localeconv() takes no arguments, and returns B<a reference to> a hash. |
5a964f20 | 388 | The keys of this hash are variable names for formatting, such as |
502a173a | 389 | C<decimal_point> and C<thousands_sep>. The values are the |
cea6626f | 390 | corresponding, er, values. See L<POSIX/localeconv> for a longer |
502a173a JH |
391 | example listing the categories an implementation might be expected to |
392 | provide; some provide more and others fewer. You don't need an | |
393 | explicit C<use locale>, because localeconv() always observes the | |
394 | current locale. | |
5f05dabc | 395 | |
5a964f20 TC |
396 | Here's a simple-minded example program that rewrites its command-line |
397 | parameters as integers correctly formatted in the current locale: | |
5f05dabc | 398 | |
399 | # See comments in previous example | |
400 | require 5.004; | |
401 | use POSIX qw(locale_h); | |
5f05dabc | 402 | |
403 | # Get some of locale's numeric formatting parameters | |
404 | my ($thousands_sep, $grouping) = | |
14280422 | 405 | @{localeconv()}{'thousands_sep', 'grouping'}; |
5f05dabc | 406 | |
407 | # Apply defaults if values are missing | |
408 | $thousands_sep = ',' unless $thousands_sep; | |
502a173a JH |
409 | |
410 | # grouping and mon_grouping are packed lists | |
411 | # of small integers (characters) telling the | |
412 | # grouping (thousand_seps and mon_thousand_seps | |
413 | # being the group dividers) of numbers and | |
414 | # monetary quantities. The integers' meanings: | |
415 | # 255 means no more grouping, 0 means repeat | |
416 | # the previous grouping, 1-254 means use that | |
417 | # as the current grouping. Grouping goes from | |
418 | # right to left (low to high digits). In the | |
419 | # below we cheat slightly by never using anything | |
420 | # else than the first grouping (whatever that is). | |
421 | if ($grouping) { | |
422 | @grouping = unpack("C*", $grouping); | |
423 | } else { | |
424 | @grouping = (3); | |
425 | } | |
5f05dabc | 426 | |
427 | # Format command line params for current locale | |
14280422 DD |
428 | for (@ARGV) { |
429 | $_ = int; # Chop non-integer part | |
5f05dabc | 430 | 1 while |
502a173a | 431 | s/(\d)(\d{$grouping[0]}($|$thousands_sep))/$1$thousands_sep$2/; |
14280422 | 432 | print "$_"; |
5f05dabc | 433 | } |
434 | print "\n"; | |
435 | ||
74c76037 | 436 | =head2 I18N::Langinfo |
4bbcc6e8 JH |
437 | |
438 | Another interface for querying locale-dependent information is the | |
e1020413 | 439 | I18N::Langinfo::langinfo() function, available at least in Unix-like |
4bbcc6e8 JH |
440 | systems and VMS. |
441 | ||
74c76037 JH |
442 | The following example will import the langinfo() function itself and |
443 | three constants to be used as arguments to langinfo(): a constant for | |
444 | the abbreviated first day of the week (the numbering starts from | |
445 | Sunday = 1) and two more constants for the affirmative and negative | |
446 | answers for a yes/no question in the current locale. | |
4bbcc6e8 | 447 | |
74c76037 | 448 | use I18N::Langinfo qw(langinfo ABDAY_1 YESSTR NOSTR); |
4bbcc6e8 | 449 | |
74c76037 | 450 | my ($abday_1, $yesstr, $nostr) = map { langinfo } qw(ABDAY_1 YESSTR NOSTR); |
4bbcc6e8 | 451 | |
74c76037 | 452 | print "$abday_1? [$yesstr/$nostr] "; |
4bbcc6e8 | 453 | |
74c76037 JH |
454 | In other words, in the "C" (or English) locale the above will probably |
455 | print something like: | |
456 | ||
457 | Sun? [yes/no] | |
4bbcc6e8 JH |
458 | |
459 | See L<I18N::Langinfo> for more information. | |
460 | ||
5f05dabc | 461 | =head1 LOCALE CATEGORIES |
462 | ||
5a964f20 TC |
463 | The following subsections describe basic locale categories. Beyond these, |
464 | some combination categories allow manipulation of more than one | |
465 | basic category at a time. See L<"ENVIRONMENT"> for a discussion of these. | |
5f05dabc | 466 | |
467 | =head2 Category LC_COLLATE: Collation | |
468 | ||
5a964f20 TC |
469 | In the scope of S<C<use locale>>, Perl looks to the C<LC_COLLATE> |
470 | environment variable to determine the application's notions on collation | |
b4ffc3db TC |
471 | (ordering) of characters. For example, "b" follows "a" in Latin |
472 | alphabets, but where do "E<aacute>" and "E<aring>" belong? And while | |
473 | "color" follows "chocolate" in English, what about in Spanish? | |
5f05dabc | 474 | |
60f0fa02 JH |
475 | The following collations all make sense and you may meet any of them |
476 | if you "use locale". | |
477 | ||
478 | A B C D E a b c d e | |
35316ca3 | 479 | A a B b C c D d E e |
60f0fa02 JH |
480 | a A b B c C d D e E |
481 | a b c d e A B C D E | |
482 | ||
f1cbbd6e | 483 | Here is a code snippet to tell what "word" |
5a964f20 | 484 | characters are in the current locale, in that locale's order: |
5f05dabc | 485 | |
486 | use locale; | |
35316ca3 | 487 | print +(sort grep /\w/, map { chr } 0..255), "\n"; |
5f05dabc | 488 | |
14280422 DD |
489 | Compare this with the characters that you see and their order if you |
490 | state explicitly that the locale should be ignored: | |
5f05dabc | 491 | |
492 | no locale; | |
35316ca3 | 493 | print +(sort grep /\w/, map { chr } 0..255), "\n"; |
5f05dabc | 494 | |
495 | This machine-native collation (which is what you get unless S<C<use | |
496 | locale>> has appeared earlier in the same block) must be used for | |
497 | sorting raw binary data, whereas the locale-dependent collation of the | |
b0c42ed9 | 498 | first example is useful for natural text. |
5f05dabc | 499 | |
14280422 DD |
500 | As noted in L<USING LOCALES>, C<cmp> compares according to the current |
501 | collation locale when C<use locale> is in effect, but falls back to a | |
de108802 | 502 | char-by-char comparison for strings that the locale says are equal. You |
14280422 DD |
503 | can use POSIX::strcoll() if you don't want this fall-back: |
504 | ||
505 | use POSIX qw(strcoll); | |
506 | $equal_in_locale = | |
507 | !strcoll("space and case ignored", "SpaceAndCaseIgnored"); | |
508 | ||
509 | $equal_in_locale will be true if the collation locale specifies a | |
5a964f20 | 510 | dictionary-like ordering that ignores space characters completely and |
9e3a2af8 | 511 | which folds case. |
14280422 | 512 | |
5a964f20 | 513 | If you have a single string that you want to check for "equality in |
14280422 DD |
514 | locale" against several others, you might think you could gain a little |
515 | efficiency by using POSIX::strxfrm() in conjunction with C<eq>: | |
516 | ||
517 | use POSIX qw(strxfrm); | |
518 | $xfrm_string = strxfrm("Mixed-case string"); | |
519 | print "locale collation ignores spaces\n" | |
520 | if $xfrm_string eq strxfrm("Mixed-casestring"); | |
521 | print "locale collation ignores hyphens\n" | |
522 | if $xfrm_string eq strxfrm("Mixedcase string"); | |
523 | print "locale collation ignores case\n" | |
524 | if $xfrm_string eq strxfrm("mixed-case string"); | |
525 | ||
526 | strxfrm() takes a string and maps it into a transformed string for use | |
de108802 | 527 | in char-by-char comparisons against other transformed strings during |
14280422 | 528 | collation. "Under the hood", locale-affected Perl comparison operators |
de108802 | 529 | call strxfrm() for both operands, then do a char-by-char |
5a964f20 | 530 | comparison of the transformed strings. By calling strxfrm() explicitly |
14280422 | 531 | and using a non locale-affected comparison, the example attempts to save |
5a964f20 | 532 | a couple of transformations. But in fact, it doesn't save anything: Perl |
2ae324a7 | 533 | magic (see L<perlguts/Magic Variables>) creates the transformed version of a |
5a964f20 | 534 | string the first time it's needed in a comparison, then keeps this version around |
14280422 | 535 | in case it's needed again. An example rewritten the easy way with |
e38874e2 | 536 | C<cmp> runs just about as fast. It also copes with null characters |
14280422 | 537 | embedded in strings; if you call strxfrm() directly, it treats the first |
5a964f20 TC |
538 | null it finds as a terminator. don't expect the transformed strings |
539 | it produces to be portable across systems--or even from one revision | |
e38874e2 DD |
540 | of your operating system to the next. In short, don't call strxfrm() |
541 | directly: let Perl do it for you. | |
14280422 | 542 | |
5a964f20 | 543 | Note: C<use locale> isn't shown in some of these examples because it isn't |
14280422 DD |
544 | needed: strcoll() and strxfrm() exist only to generate locale-dependent |
545 | results, and so always obey the current C<LC_COLLATE> locale. | |
5f05dabc | 546 | |
547 | =head2 Category LC_CTYPE: Character Types | |
548 | ||
5a964f20 | 549 | In the scope of S<C<use locale>>, Perl obeys the C<LC_CTYPE> locale |
14280422 DD |
550 | setting. This controls the application's notion of which characters are |
551 | alphabetic. This affects Perl's C<\w> regular expression metanotation, | |
f1cbbd6e GS |
552 | which stands for alphanumeric characters--that is, alphabetic, |
553 | numeric, and including other special characters such as the underscore or | |
554 | hyphen. (Consult L<perlre> for more information about | |
14280422 | 555 | regular expressions.) Thanks to C<LC_CTYPE>, depending on your locale |
b4ffc3db TC |
556 | setting, characters like "E<aelig>", "E<eth>", "E<szlig>", and |
557 | "E<oslash>" may be understood as C<\w> characters. | |
5f05dabc | 558 | |
2c268ad5 | 559 | The C<LC_CTYPE> locale also provides the map used in transliterating |
68dc0745 | 560 | characters between lower and uppercase. This affects the case-mapping |
5a964f20 TC |
561 | functions--lc(), lcfirst, uc(), and ucfirst(); case-mapping |
562 | interpolation with C<\l>, C<\L>, C<\u>, or C<\U> in double-quoted strings | |
563 | and C<s///> substitutions; and case-independent regular expression | |
e38874e2 DD |
564 | pattern matching using the C<i> modifier. |
565 | ||
5a964f20 TC |
566 | Finally, C<LC_CTYPE> affects the POSIX character-class test |
567 | functions--isalpha(), islower(), and so on. For example, if you move | |
568 | from the "C" locale to a 7-bit Scandinavian one, you may find--possibly | |
569 | to your surprise--that "|" moves from the ispunct() class to isalpha(). | |
5f05dabc | 570 | |
14280422 DD |
571 | B<Note:> A broken or malicious C<LC_CTYPE> locale definition may result |
572 | in clearly ineligible characters being considered to be alphanumeric by | |
e199995e | 573 | your application. For strict matching of (mundane) ASCII letters and |
5a964f20 | 574 | digits--for example, in command strings--locale-aware applications |
e199995e | 575 | should use C<\w> with the C</a> regular expression modifier. See L<"SECURITY">. |
5f05dabc | 576 | |
577 | =head2 Category LC_NUMERIC: Numeric Formatting | |
578 | ||
2095dafa RGS |
579 | After a proper POSIX::setlocale() call, Perl obeys the C<LC_NUMERIC> |
580 | locale information, which controls an application's idea of how numbers | |
581 | should be formatted for human readability by the printf(), sprintf(), and | |
582 | write() functions. String-to-numeric conversion by the POSIX::strtod() | |
5a964f20 | 583 | function is also affected. In most implementations the only effect is to |
b4ffc3db | 584 | change the character used for the decimal point--perhaps from "." to ",". |
5a964f20 | 585 | These functions aren't aware of such niceties as thousands separation and |
2095dafa | 586 | so on. (See L<The localeconv function> if you care about these things.) |
5a964f20 | 587 | |
3cf03d68 | 588 | Output produced by print() is also affected by the current locale: it |
3cf03d68 JH |
589 | corresponds to what you'd get from printf() in the "C" locale. The |
590 | same is true for Perl's internal conversions between numeric and | |
591 | string formats: | |
5f05dabc | 592 | |
2095dafa RGS |
593 | use POSIX qw(strtod setlocale LC_NUMERIC); |
594 | ||
595 | setlocale LC_NUMERIC, ""; | |
14280422 | 596 | |
5f05dabc | 597 | $n = 5/2; # Assign numeric 2.5 to $n |
598 | ||
35316ca3 | 599 | $a = " $n"; # Locale-dependent conversion to string |
5f05dabc | 600 | |
35316ca3 | 601 | print "half five is $n\n"; # Locale-dependent output |
5f05dabc | 602 | |
603 | printf "half five is %g\n", $n; # Locale-dependent output | |
604 | ||
14280422 DD |
605 | print "DECIMAL POINT IS COMMA\n" |
606 | if $n == (strtod("2,5"))[0]; # Locale-dependent conversion | |
5f05dabc | 607 | |
4bbcc6e8 JH |
608 | See also L<I18N::Langinfo> and C<RADIXCHAR>. |
609 | ||
5f05dabc | 610 | =head2 Category LC_MONETARY: Formatting of monetary amounts |
611 | ||
e199995e | 612 | The C standard defines the C<LC_MONETARY> category, but not a function |
5a964f20 | 613 | that is affected by its contents. (Those with experience of standards |
b0c42ed9 | 614 | committees will recognize that the working group decided to punt on the |
14280422 | 615 | issue.) Consequently, Perl takes no notice of it. If you really want |
13a2d996 SP |
616 | to use C<LC_MONETARY>, you can query its contents--see |
617 | L<The localeconv function>--and use the information that it returns in your | |
618 | application's own formatting of currency amounts. However, you may well | |
619 | find that the information, voluminous and complex though it may be, still | |
620 | does not quite meet your requirements: currency formatting is a hard nut | |
621 | to crack. | |
5f05dabc | 622 | |
4bbcc6e8 JH |
623 | See also L<I18N::Langinfo> and C<CRNCYSTR>. |
624 | ||
5f05dabc | 625 | =head2 LC_TIME |
626 | ||
5a964f20 | 627 | Output produced by POSIX::strftime(), which builds a formatted |
5f05dabc | 628 | human-readable date/time string, is affected by the current C<LC_TIME> |
629 | locale. Thus, in a French locale, the output produced by the C<%B> | |
630 | format element (full month name) for the first month of the year would | |
5a964f20 | 631 | be "janvier". Here's how to get a list of long month names in the |
5f05dabc | 632 | current locale: |
633 | ||
634 | use POSIX qw(strftime); | |
14280422 DD |
635 | for (0..11) { |
636 | $long_month_name[$_] = | |
637 | strftime("%B", 0, 0, 0, 1, $_, 96); | |
5f05dabc | 638 | } |
639 | ||
5a964f20 | 640 | Note: C<use locale> isn't needed in this example: as a function that |
14280422 DD |
641 | exists only to generate locale-dependent results, strftime() always |
642 | obeys the current C<LC_TIME> locale. | |
5f05dabc | 643 | |
4bbcc6e8 | 644 | See also L<I18N::Langinfo> and C<ABDAY_1>..C<ABDAY_7>, C<DAY_1>..C<DAY_7>, |
2a2bf5f4 | 645 | C<ABMON_1>..C<ABMON_12>, and C<ABMON_1>..C<ABMON_12>. |
4bbcc6e8 | 646 | |
5f05dabc | 647 | =head2 Other categories |
648 | ||
5a964f20 TC |
649 | The remaining locale category, C<LC_MESSAGES> (possibly supplemented |
650 | by others in particular implementations) is not currently used by | |
98a6f11e | 651 | Perl--except possibly to affect the behavior of library functions |
652 | called by extensions outside the standard Perl distribution and by the | |
653 | operating system and its utilities. Note especially that the string | |
654 | value of C<$!> and the error messages given by external utilities may | |
655 | be changed by C<LC_MESSAGES>. If you want to have portable error | |
265f5c4a | 656 | codes, use C<%!>. See L<Errno>. |
14280422 DD |
657 | |
658 | =head1 SECURITY | |
659 | ||
5a964f20 | 660 | Although the main discussion of Perl security issues can be found in |
14280422 DD |
661 | L<perlsec>, a discussion of Perl's locale handling would be incomplete |
662 | if it did not draw your attention to locale-dependent security issues. | |
5a964f20 TC |
663 | Locales--particularly on systems that allow unprivileged users to |
664 | build their own locales--are untrustworthy. A malicious (or just plain | |
14280422 DD |
665 | broken) locale can make a locale-aware application give unexpected |
666 | results. Here are a few possibilities: | |
667 | ||
668 | =over 4 | |
669 | ||
670 | =item * | |
671 | ||
672 | Regular expression checks for safe file names or mail addresses using | |
5a964f20 | 673 | C<\w> may be spoofed by an C<LC_CTYPE> locale that claims that |
14280422 DD |
674 | characters such as "E<gt>" and "|" are alphanumeric. |
675 | ||
676 | =item * | |
677 | ||
e38874e2 DD |
678 | String interpolation with case-mapping, as in, say, C<$dest = |
679 | "C:\U$name.$ext">, may produce dangerous results if a bogus LC_CTYPE | |
680 | case-mapping table is in effect. | |
681 | ||
682 | =item * | |
683 | ||
14280422 DD |
684 | A sneaky C<LC_COLLATE> locale could result in the names of students with |
685 | "D" grades appearing ahead of those with "A"s. | |
686 | ||
687 | =item * | |
688 | ||
5a964f20 | 689 | An application that takes the trouble to use information in |
14280422 | 690 | C<LC_MONETARY> may format debits as if they were credits and vice versa |
5a964f20 | 691 | if that locale has been subverted. Or it might make payments in US |
14280422 DD |
692 | dollars instead of Hong Kong dollars. |
693 | ||
694 | =item * | |
695 | ||
696 | The date and day names in dates formatted by strftime() could be | |
697 | manipulated to advantage by a malicious user able to subvert the | |
5a964f20 | 698 | C<LC_DATE> locale. ("Look--it says I wasn't in the building on |
14280422 DD |
699 | Sunday.") |
700 | ||
701 | =back | |
702 | ||
703 | Such dangers are not peculiar to the locale system: any aspect of an | |
5a964f20 | 704 | application's environment which may be modified maliciously presents |
14280422 | 705 | similar challenges. Similarly, they are not specific to Perl: any |
5a964f20 | 706 | programming language that allows you to write programs that take |
14280422 DD |
707 | account of their environment exposes you to these issues. |
708 | ||
5a964f20 TC |
709 | Perl cannot protect you from all possibilities shown in the |
710 | examples--there is no substitute for your own vigilance--but, when | |
14280422 | 711 | C<use locale> is in effect, Perl uses the tainting mechanism (see |
5a964f20 | 712 | L<perlsec>) to mark string results that become locale-dependent, and |
14280422 | 713 | which may be untrustworthy in consequence. Here is a summary of the |
5a964f20 | 714 | tainting behavior of operators and functions that may be affected by |
14280422 DD |
715 | the locale: |
716 | ||
717 | =over 4 | |
718 | ||
551e1d92 RB |
719 | =item * |
720 | ||
721 | B<Comparison operators> (C<lt>, C<le>, C<ge>, C<gt> and C<cmp>): | |
14280422 DD |
722 | |
723 | Scalar true/false (or less/equal/greater) result is never tainted. | |
724 | ||
551e1d92 RB |
725 | =item * |
726 | ||
727 | B<Case-mapping interpolation> (with C<\l>, C<\L>, C<\u> or C<\U>) | |
e38874e2 DD |
728 | |
729 | Result string containing interpolated material is tainted if | |
730 | C<use locale> is in effect. | |
731 | ||
551e1d92 RB |
732 | =item * |
733 | ||
734 | B<Matching operator> (C<m//>): | |
14280422 DD |
735 | |
736 | Scalar true/false result never tainted. | |
737 | ||
5a964f20 | 738 | Subpatterns, either delivered as a list-context result or as $1 etc. |
14280422 | 739 | are tainted if C<use locale> is in effect, and the subpattern regular |
e38874e2 | 740 | expression contains C<\w> (to match an alphanumeric character), C<\W> |
6b0ac556 OK |
741 | (non-alphanumeric character), C<\s> (whitespace character), or C<\S> |
742 | (non whitespace character). The matched-pattern variable, $&, $` | |
e38874e2 DD |
743 | (pre-match), $' (post-match), and $+ (last match) are also tainted if |
744 | C<use locale> is in effect and the regular expression contains C<\w>, | |
745 | C<\W>, C<\s>, or C<\S>. | |
14280422 | 746 | |
551e1d92 RB |
747 | =item * |
748 | ||
749 | B<Substitution operator> (C<s///>): | |
14280422 | 750 | |
e38874e2 | 751 | Has the same behavior as the match operator. Also, the left |
5a964f20 TC |
752 | operand of C<=~> becomes tainted when C<use locale> in effect |
753 | if modified as a result of a substitution based on a regular | |
e38874e2 | 754 | expression match involving C<\w>, C<\W>, C<\s>, or C<\S>; or of |
7b8d334a | 755 | case-mapping with C<\l>, C<\L>,C<\u> or C<\U>. |
14280422 | 756 | |
551e1d92 RB |
757 | =item * |
758 | ||
759 | B<Output formatting functions> (printf() and write()): | |
14280422 | 760 | |
3cf03d68 JH |
761 | Results are never tainted because otherwise even output from print, |
762 | for example C<print(1/7)>, should be tainted if C<use locale> is in | |
763 | effect. | |
14280422 | 764 | |
551e1d92 RB |
765 | =item * |
766 | ||
767 | B<Case-mapping functions> (lc(), lcfirst(), uc(), ucfirst()): | |
14280422 DD |
768 | |
769 | Results are tainted if C<use locale> is in effect. | |
770 | ||
551e1d92 RB |
771 | =item * |
772 | ||
773 | B<POSIX locale-dependent functions> (localeconv(), strcoll(), | |
14280422 DD |
774 | strftime(), strxfrm()): |
775 | ||
776 | Results are never tainted. | |
777 | ||
551e1d92 RB |
778 | =item * |
779 | ||
780 | B<POSIX character class tests> (isalnum(), isalpha(), isdigit(), | |
14280422 DD |
781 | isgraph(), islower(), isprint(), ispunct(), isspace(), isupper(), |
782 | isxdigit()): | |
783 | ||
784 | True/false results are never tainted. | |
785 | ||
786 | =back | |
787 | ||
788 | Three examples illustrate locale-dependent tainting. | |
789 | The first program, which ignores its locale, won't run: a value taken | |
54310121 | 790 | directly from the command line may not be used to name an output file |
14280422 DD |
791 | when taint checks are enabled. |
792 | ||
793 | #/usr/local/bin/perl -T | |
794 | # Run with taint checking | |
795 | ||
54310121 | 796 | # Command line sanity check omitted... |
14280422 DD |
797 | $tainted_output_file = shift; |
798 | ||
799 | open(F, ">$tainted_output_file") | |
800 | or warn "Open of $untainted_output_file failed: $!\n"; | |
801 | ||
802 | The program can be made to run by "laundering" the tainted value through | |
5a964f20 TC |
803 | a regular expression: the second example--which still ignores locale |
804 | information--runs, creating the file named on its command line | |
14280422 DD |
805 | if it can. |
806 | ||
807 | #/usr/local/bin/perl -T | |
808 | ||
809 | $tainted_output_file = shift; | |
810 | $tainted_output_file =~ m%[\w/]+%; | |
811 | $untainted_output_file = $&; | |
812 | ||
813 | open(F, ">$untainted_output_file") | |
814 | or warn "Open of $untainted_output_file failed: $!\n"; | |
815 | ||
5a964f20 | 816 | Compare this with a similar but locale-aware program: |
14280422 DD |
817 | |
818 | #/usr/local/bin/perl -T | |
819 | ||
820 | $tainted_output_file = shift; | |
821 | use locale; | |
822 | $tainted_output_file =~ m%[\w/]+%; | |
823 | $localized_output_file = $&; | |
824 | ||
825 | open(F, ">$localized_output_file") | |
826 | or warn "Open of $localized_output_file failed: $!\n"; | |
827 | ||
828 | This third program fails to run because $& is tainted: it is the result | |
5a964f20 | 829 | of a match involving C<\w> while C<use locale> is in effect. |
5f05dabc | 830 | |
831 | =head1 ENVIRONMENT | |
832 | ||
833 | =over 12 | |
834 | ||
835 | =item PERL_BADLANG | |
836 | ||
14280422 | 837 | A string that can suppress Perl's warning about failed locale settings |
54310121 | 838 | at startup. Failure can occur if the locale support in the operating |
5a964f20 | 839 | system is lacking (broken) in some way--or if you mistyped the name of |
900bd440 JH |
840 | a locale when you set up your environment. If this environment |
841 | variable is absent, or has a value that does not evaluate to integer | |
842 | zero--that is, "0" or ""-- Perl will complain about locale setting | |
843 | failures. | |
5f05dabc | 844 | |
14280422 DD |
845 | B<NOTE>: PERL_BADLANG only gives you a way to hide the warning message. |
846 | The message tells about some problem in your system's locale support, | |
847 | and you should investigate what the problem is. | |
5f05dabc | 848 | |
849 | =back | |
850 | ||
851 | The following environment variables are not specific to Perl: They are | |
14280422 DD |
852 | part of the standardized (ISO C, XPG4, POSIX 1.c) setlocale() method |
853 | for controlling an application's opinion on data. | |
5f05dabc | 854 | |
855 | =over 12 | |
856 | ||
857 | =item LC_ALL | |
858 | ||
5a964f20 | 859 | C<LC_ALL> is the "override-all" locale environment variable. If |
5f05dabc | 860 | set, it overrides all the rest of the locale environment variables. |
861 | ||
528d65ad JH |
862 | =item LANGUAGE |
863 | ||
864 | B<NOTE>: C<LANGUAGE> is a GNU extension, it affects you only if you | |
865 | are using the GNU libc. This is the case if you are using e.g. Linux. | |
e1020413 | 866 | If you are using "commercial" Unixes you are most probably I<not> |
22b6f60d JH |
867 | using GNU libc and you can ignore C<LANGUAGE>. |
868 | ||
869 | However, in the case you are using C<LANGUAGE>: it affects the | |
870 | language of informational, warning, and error messages output by | |
871 | commands (in other words, it's like C<LC_MESSAGES>) but it has higher | |
96090e4f | 872 | priority than C<LC_ALL>. Moreover, it's not a single value but |
22b6f60d JH |
873 | instead a "path" (":"-separated list) of I<languages> (not locales). |
874 | See the GNU C<gettext> library documentation for more information. | |
528d65ad | 875 | |
5f05dabc | 876 | =item LC_CTYPE |
877 | ||
878 | In the absence of C<LC_ALL>, C<LC_CTYPE> chooses the character type | |
879 | locale. In the absence of both C<LC_ALL> and C<LC_CTYPE>, C<LANG> | |
880 | chooses the character type locale. | |
881 | ||
882 | =item LC_COLLATE | |
883 | ||
14280422 DD |
884 | In the absence of C<LC_ALL>, C<LC_COLLATE> chooses the collation |
885 | (sorting) locale. In the absence of both C<LC_ALL> and C<LC_COLLATE>, | |
886 | C<LANG> chooses the collation locale. | |
5f05dabc | 887 | |
888 | =item LC_MONETARY | |
889 | ||
14280422 DD |
890 | In the absence of C<LC_ALL>, C<LC_MONETARY> chooses the monetary |
891 | formatting locale. In the absence of both C<LC_ALL> and C<LC_MONETARY>, | |
892 | C<LANG> chooses the monetary formatting locale. | |
5f05dabc | 893 | |
894 | =item LC_NUMERIC | |
895 | ||
896 | In the absence of C<LC_ALL>, C<LC_NUMERIC> chooses the numeric format | |
897 | locale. In the absence of both C<LC_ALL> and C<LC_NUMERIC>, C<LANG> | |
898 | chooses the numeric format. | |
899 | ||
900 | =item LC_TIME | |
901 | ||
14280422 DD |
902 | In the absence of C<LC_ALL>, C<LC_TIME> chooses the date and time |
903 | formatting locale. In the absence of both C<LC_ALL> and C<LC_TIME>, | |
904 | C<LANG> chooses the date and time formatting locale. | |
5f05dabc | 905 | |
906 | =item LANG | |
907 | ||
14280422 DD |
908 | C<LANG> is the "catch-all" locale environment variable. If it is set, it |
909 | is used as the last resort after the overall C<LC_ALL> and the | |
5f05dabc | 910 | category-specific C<LC_...>. |
911 | ||
912 | =back | |
913 | ||
7e4353e9 RGS |
914 | =head2 Examples |
915 | ||
916 | The LC_NUMERIC controls the numeric output: | |
917 | ||
918 | use locale; | |
919 | use POSIX qw(locale_h); # Imports setlocale() and the LC_ constants. | |
920 | setlocale(LC_NUMERIC, "fr_FR") or die "Pardon"; | |
921 | printf "%g\n", 1.23; # If the "fr_FR" succeeded, probably shows 1,23. | |
922 | ||
923 | and also how strings are parsed by POSIX::strtod() as numbers: | |
924 | ||
925 | use locale; | |
926 | use POSIX qw(locale_h strtod); | |
2095dafa | 927 | setlocale(LC_NUMERIC, "de_DE") or die "Entschuldigung"; |
7e4353e9 RGS |
928 | my $x = strtod("2,34") + 5; |
929 | print $x, "\n"; # Probably shows 7,34. | |
930 | ||
5f05dabc | 931 | =head1 NOTES |
932 | ||
933 | =head2 Backward compatibility | |
934 | ||
b0c42ed9 | 935 | Versions of Perl prior to 5.004 B<mostly> ignored locale information, |
5a964f20 TC |
936 | generally behaving as if something similar to the C<"C"> locale were |
937 | always in force, even if the program environment suggested otherwise | |
938 | (see L<The setlocale function>). By default, Perl still behaves this | |
939 | way for backward compatibility. If you want a Perl application to pay | |
940 | attention to locale information, you B<must> use the S<C<use locale>> | |
b687b08b | 941 | pragma (see L<The use locale pragma>) to instruct it to do so. |
b0c42ed9 JH |
942 | |
943 | Versions of Perl from 5.002 to 5.003 did use the C<LC_CTYPE> | |
5a964f20 TC |
944 | information if available; that is, C<\w> did understand what |
945 | were the letters according to the locale environment variables. | |
b0c42ed9 JH |
946 | The problem was that the user had no control over the feature: |
947 | if the C library supported locales, Perl used them. | |
948 | ||
949 | =head2 I18N:Collate obsolete | |
950 | ||
5a964f20 | 951 | In versions of Perl prior to 5.004, per-locale collation was possible |
b0c42ed9 JH |
952 | using the C<I18N::Collate> library module. This module is now mildly |
953 | obsolete and should be avoided in new applications. The C<LC_COLLATE> | |
954 | functionality is now integrated into the Perl core language: One can | |
955 | use locale-specific scalar data completely normally with C<use locale>, | |
956 | so there is no longer any need to juggle with the scalar references of | |
957 | C<I18N::Collate>. | |
5f05dabc | 958 | |
14280422 | 959 | =head2 Sort speed and memory use impacts |
5f05dabc | 960 | |
961 | Comparing and sorting by locale is usually slower than the default | |
14280422 DD |
962 | sorting; slow-downs of two to four times have been observed. It will |
963 | also consume more memory: once a Perl scalar variable has participated | |
964 | in any string comparison or sorting operation obeying the locale | |
965 | collation rules, it will take 3-15 times more memory than before. (The | |
966 | exact multiplier depends on the string's contents, the operating system | |
967 | and the locale.) These downsides are dictated more by the operating | |
968 | system's implementation of the locale system than by Perl. | |
5f05dabc | 969 | |
e38874e2 DD |
970 | =head2 write() and LC_NUMERIC |
971 | ||
903eb63f NT |
972 | If a program's environment specifies an LC_NUMERIC locale and C<use |
973 | locale> is in effect when the format is declared, the locale is used | |
974 | to specify the decimal point character in formatted output. Formatted | |
975 | output cannot be controlled by C<use locale> at the time when write() | |
976 | is called. | |
e38874e2 | 977 | |
5f05dabc | 978 | =head2 Freely available locale definitions |
979 | ||
08d7a6b2 LB |
980 | There is a large collection of locale definitions at: |
981 | ||
982 | http://std.dkuug.dk/i18n/WG15-collection/locales/ | |
983 | ||
984 | You should be aware that it is | |
14280422 | 985 | unsupported, and is not claimed to be fit for any purpose. If your |
5a964f20 | 986 | system allows installation of arbitrary locales, you may find the |
14280422 DD |
987 | definitions useful as they are, or as a basis for the development of |
988 | your own locales. | |
5f05dabc | 989 | |
14280422 | 990 | =head2 I18n and l10n |
5f05dabc | 991 | |
b0c42ed9 JH |
992 | "Internationalization" is often abbreviated as B<i18n> because its first |
993 | and last letters are separated by eighteen others. (You may guess why | |
994 | the internalin ... internaliti ... i18n tends to get abbreviated.) In | |
995 | the same way, "localization" is often abbreviated to B<l10n>. | |
14280422 DD |
996 | |
997 | =head2 An imperfect standard | |
998 | ||
999 | Internationalization, as defined in the C and POSIX standards, can be | |
1000 | criticized as incomplete, ungainly, and having too large a granularity. | |
1001 | (Locales apply to a whole process, when it would arguably be more useful | |
1002 | to have them apply to a single thread, window group, or whatever.) They | |
1003 | also have a tendency, like standards groups, to divide the world into | |
1004 | nations, when we all know that the world can equally well be divided | |
e199995e | 1005 | into bankers, bikers, gamers, and so on. |
5f05dabc | 1006 | |
b310b053 JH |
1007 | =head1 Unicode and UTF-8 |
1008 | ||
e199995e | 1009 | The support of Unicode is new starting from Perl version 5.6, and more fully |
b4ffc3db TC |
1010 | implemented in version 5.8 and later. See L<perluniintro>. Perl tries to |
1011 | work with both Unicode and locales--but of course, there are problems. | |
e199995e KW |
1012 | |
1013 | Perl does not handle multi-byte locales, such as have been used for various | |
b4ffc3db TC |
1014 | Asian languages, such as Big5 or Shift JIS. However, the increasingly common |
1015 | multi-byte UTF-8 locales, if properly implemented, tend to work | |
1016 | reasonably well in Perl, simply because both they and Perl store | |
e199995e KW |
1017 | characters that take up multiple bytes the same way. |
1018 | ||
1019 | Perl generally takes the tack to use locale rules on code points that can fit | |
1020 | in a single byte, and Unicode rules for those that can't (though this wasn't | |
1021 | uniformly applied prior to Perl 5.14). This prevents many problems in locales | |
1022 | that aren't UTF-8. Suppose the locale is ISO8859-7, Greek. The character at | |
1023 | 0xD7 there is a capital Chi. But in the ISO8859-1 locale, Latin1, it is a | |
1024 | multiplication sign. The POSIX regular expression character class | |
b4ffc3db TC |
1025 | C<[[:alpha:]]> will magically match 0xD7 in the Greek locale but not in the |
1026 | Latin one, even if the string is encoded in UTF-8, which would normally imply | |
1027 | Unicode semantics. (The "U" in UTF-8 stands for Unicode.) | |
e199995e KW |
1028 | |
1029 | However, there are places where this breaks down. Certain constructs are | |
b4ffc3db TC |
1030 | for Unicode only, such as C<\p{Alpha}>. They assume that 0xD7 always has its |
1031 | Unicode meaning (or the equivalent on EBCDIC platforms). Since Latin1 is a | |
1032 | subset of Unicode and 0xD7 is the multiplication sign in both Latin1 and | |
1033 | Unicode, C<\p{Alpha}> will never match it, regardless of locale. A similar | |
1034 | issue occurs with C<\N{...}>. It is therefore a bad idea to use C<\p{}> or | |
1035 | C<\N{}> under C<use locale>--I<unless> you can guarantee that the locale will | |
1036 | be a ISO8859-1 or UTF-8 one. Use POSIX character classes instead. | |
1037 | ||
e199995e KW |
1038 | |
1039 | The same problem ensues if you enable automatic UTF-8-ification of your | |
1040 | standard file handles, default C<open()> layer, and C<@ARGV> on non-ISO8859-1, | |
b4ffc3db TC |
1041 | non-UTF-8 locales (by using either the B<-C> command line switch or the |
1042 | C<PERL_UNICODE> environment variable; see L<perlrun>). | |
1043 | Things are read in as UTF-8, which would normally imply a Unicode | |
1044 | interpretation, but the presence of a locale causes them to be interpreted | |
1045 | in that locale instead. For example, a 0xD7 code point in the Unicode | |
1046 | input, which should mean the multiplication sign, won't be interpreted by | |
1047 | Perl that way under the Greek locale. Again, this is not a problem | |
1048 | I<provided> you make certain that all locales will always and only be either | |
1049 | an ISO8859-1 or a UTF-8 locale. | |
1050 | ||
1051 | Vendor locales are notoriously buggy, and it is difficult for Perl to test | |
1052 | its locale-handling code because this interacts with code that Perl has no | |
1053 | control over; therefore the locale-handling code in Perl may be buggy as | |
1054 | well. But if you I<do> have locales that work, using them may be | |
1055 | worthwhile for certain specific purposes, as long as you keep in mind the | |
1056 | gotchas already mentioned. For example, collation runs faster under | |
1057 | locales than under L<Unicode::Collate> (albeit with less flexibility), and | |
1058 | you gain access to such things as the local currency symbol and the names | |
1059 | of the months and days of the week. | |
b310b053 | 1060 | |
5f05dabc | 1061 | =head1 BUGS |
1062 | ||
1063 | =head2 Broken systems | |
1064 | ||
5a964f20 | 1065 | In certain systems, the operating system's locale support |
2bdf8add | 1066 | is broken and cannot be fixed or used by Perl. Such deficiencies can |
b4ffc3db | 1067 | and will result in mysterious hangs and/or Perl core dumps when |
2bdf8add | 1068 | C<use locale> is in effect. When confronted with such a system, |
7f2de2d2 | 1069 | please report in excruciating detail to <F<perlbug@perl.org>>, and |
b4ffc3db | 1070 | also contact your vendor: bug fixes may exist for these problems |
2bdf8add JH |
1071 | in your operating system. Sometimes such bug fixes are called an |
1072 | operating system upgrade. | |
5f05dabc | 1073 | |
1074 | =head1 SEE ALSO | |
1075 | ||
b310b053 JH |
1076 | L<I18N::Langinfo>, L<perluniintro>, L<perlunicode>, L<open>, |
1077 | L<POSIX/isalnum>, L<POSIX/isalpha>, | |
4bbcc6e8 JH |
1078 | L<POSIX/isdigit>, L<POSIX/isgraph>, L<POSIX/islower>, |
1079 | L<POSIX/isprint>, L<POSIX/ispunct>, L<POSIX/isspace>, | |
1080 | L<POSIX/isupper>, L<POSIX/isxdigit>, L<POSIX/localeconv>, | |
1081 | L<POSIX/setlocale>, L<POSIX/strcoll>, L<POSIX/strftime>, | |
1082 | L<POSIX/strtod>, L<POSIX/strxfrm>. | |
5f05dabc | 1083 | |
1084 | =head1 HISTORY | |
1085 | ||
b0c42ed9 | 1086 | Jarkko Hietaniemi's original F<perli18n.pod> heavily hacked by Dominic |
5a964f20 TC |
1087 | Dunlop, assisted by the perl5-porters. Prose worked over a bit by |
1088 | Tom Christiansen. |