2 # Time-stamp: "2004-01-11 18:35:34 AST"
6 Locale::Maketext - framework for localization
13 # ...which inherits from Locale::Maketext
14 my $lh = MyProgram::L10N->get_handle() || die "What language?";
16 # And then any messages your program emits, like:
17 warn $lh->maketext( "Can't open file [_1]: [_2]\n", $f, $! );
22 It is a common feature of applications (whether run directly,
23 or via the Web) for them to be "localized" -- i.e., for them
24 to a present an English interface to an English-speaker, a German
25 interface to a German-speaker, and so on for all languages it's
26 programmed with. Locale::Maketext
27 is a framework for software localization; it provides you with the
28 tools for organizing and accessing the bits of text and text-processing
29 code that you need for producing localized applications.
31 In order to make sense of Maketext and how all its
32 components fit together, you should probably
33 go read L<Locale::Maketext::TPJ13|Locale::Maketext::TPJ13>, and
34 I<then> read the following documentation.
36 You may also want to read over the source for C<File::Findgrep>
37 and its constituent modules -- they are a complete (if small)
38 example application that uses Maketext.
42 The basic design of Locale::Maketext is object-oriented, and
43 Locale::Maketext is an abstract base class, from which you
44 derive a "project class".
45 The project class (with a name like "TkBocciBall::Localize",
46 which you then use in your module) is in turn the base class
47 for all the "language classes" for your project
48 (with names "TkBocciBall::Localize::it",
49 "TkBocciBall::Localize::en",
50 "TkBocciBall::Localize::fr", etc.).
53 a class containing a lexicon of phrases as class data,
54 and possibly also some methods that are of use in interpreting
55 phrases in the lexicon, or otherwise dealing with text in that
58 An object belonging to a language class is called a "language
59 handle"; it's typically a flyweight object.
61 The normal course of action is to call:
63 use TkBocciBall::Localize; # the localization project class
64 $lh = TkBocciBall::Localize->get_handle();
65 # Depending on the user's locale, etc., this will
66 # make a language handle from among the classes available,
67 # and any defaults that you declare.
68 die "Couldn't make a language handle??" unless $lh;
70 From then on, you use the C<maketext> function to access
71 entries in whatever lexicon(s) belong to the language handle
74 print $lh->maketext("You won!"), "\n";
76 ...emits the right text for this language. If the object
77 in C<$lh> belongs to class "TkBocciBall::Localize::fr" and
78 %TkBocciBall::Localize::fr::Lexicon contains C<("You won!"
79 =E<gt> "Tu as gagnE<eacute>!")>, then the above
80 code happily tells the user "Tu as gagnE<eacute>!".
84 Locale::Maketext offers a variety of methods, which fall
85 into three categories:
91 Methods to do with constructing language handles.
95 C<maketext> and other methods to do with accessing %Lexicon data
96 for a given language handle.
100 Methods that you may find it handy to use, from routines of
101 yours that you put in %Lexicon entries.
105 These are covered in the following section.
107 =head2 Construction Methods
109 These are to do with constructing a language handle:
115 $lh = YourProjClass->get_handle( ...langtags... ) || die "lg-handle?";
117 This tries loading classes based on the language-tags you give (like
118 C<("en-US", "sk", "kon", "es-MX", "ja", "i-klingon")>, and for the first class
119 that succeeds, returns YourProjClass::I<language>->new().
121 If it runs thru the entire given list of language-tags, and finds no classes
122 for those exact terms, it then tries "superordinate" language classes.
123 So if no "en-US" class (i.e., YourProjClass::en_us)
124 was found, nor classes for anything else in that list, we then try
125 its superordinate, "en" (i.e., YourProjClass::en), and so on thru
126 the other language-tags in the given list: "es".
127 (The other language-tags in our example list:
128 happen to have no superordinates.)
130 If none of those language-tags leads to loadable classes, we then
131 try classes derived from YourProjClass->fallback_languages() and
132 then if nothing comes of that, we use classes named by
133 YourProjClass->fallback_language_classes(). Then in the (probably
134 quite unlikely) event that that fails, we just return undef.
138 $lh = YourProjClass->get_handleB<()> || die "lg-handle?";
140 When C<get_handle> is called with an empty parameter list, magic happens:
142 If C<get_handle> senses that it's running in program that was
143 invoked as a CGI, then it tries to get language-tags out of the
144 environment variable "HTTP_ACCEPT_LANGUAGE", and it pretends that
145 those were the languages passed as parameters to C<get_handle>.
147 Otherwise (i.e., if not a CGI), this tries various OS-specific ways
148 to get the language-tags for the current locale/language, and then
149 pretends that those were the value(s) passed to C<get_handle>.
151 Currently this OS-specific stuff consists of looking in the environment
152 variables "LANG" and "LANGUAGE"; and on MSWin machines (where those
153 variables are typically unused), this also tries using
154 the module Win32::Locale to get a language-tag for whatever language/locale
155 is currently selected in the "Regional Settings" (or "International"?)
156 Control Panel. I welcome further
157 suggestions for making this do the Right Thing under other operating
158 systems that support localization.
160 If you're using localization in an application that keeps a configuration
161 file, you might consider something like this in your project class:
163 sub get_handle_via_config {
165 my $chosen_language = $Config_settings{'language'};
167 if($chosen_language) {
168 $lh = $class->get_handle($chosen_language)
169 || die "No language handle for \"$chosen_language\""
172 # Config file missing, maybe?
173 $lh = $class->get_handle()
174 || die "Can't get a language handle";
181 $lh = YourProjClass::langname->new();
183 This constructs a language handle. You usually B<don't> call this
184 directly, but instead let C<get_handle> find a language class to C<use>
185 and to then call ->new on.
191 This is called by ->new to initialize newly-constructed language handles.
192 If you define an init method in your class, remember that it's usually
193 considered a good idea to call $lh->SUPER::init in it (presumably at the
194 beginning), so that all classes get a chance to initialize a new object
195 however they see fit.
199 YourProjClass->fallback_languages()
201 C<get_handle> appends the return value of this to the end of
202 whatever list of languages you pass C<get_handle>. Unless
203 you override this method, your project class
204 will inherit Locale::Maketext's C<fallback_languages>, which
205 currently returns C<('i-default', 'en', 'en-US')>.
206 ("i-default" is defined in RFC 2277).
208 This method (by having it return the name
209 of a language-tag that has an existing language class)
210 can be used for making sure that
211 C<get_handle> will always manage to construct a language
212 handle (assuming your language classes are in an appropriate
213 @INC directory). Or you can use the next method:
217 YourProjClass->fallback_language_classes()
219 C<get_handle> appends the return value of this to the end
220 of the list of classes it will try using. Unless
221 you override this method, your project class
222 will inherit Locale::Maketext's C<fallback_language_classes>,
223 which currently returns an empty list, C<()>.
224 By setting this to some value (namely, the name of a loadable
225 language class), you can be sure that
226 C<get_handle> will always manage to construct a language
231 =head2 The "maketext" Method
233 This is the most important method in Locale::Maketext:
235 $text = $lh->maketext(I<key>, ...parameters for this phrase...);
237 This looks in the %Lexicon of the language handle
238 $lh and all its superclasses, looking
239 for an entry whose key is the string I<key>. Assuming such
240 an entry is found, various things then happen, depending on the
243 If the value is a scalarref, the scalar is dereferenced and returned
244 (and any parameters are ignored).
246 If the value is a coderef, we return &$value($lh, ...parameters...).
248 If the value is a string that I<doesn't> look like it's in Bracket Notation,
249 we return it (after replacing it with a scalarref, in its %Lexicon).
251 If the value I<does> look like it's in Bracket Notation, then we compile
252 it into a sub, replace the string in the %Lexicon with the new coderef,
253 and then we return &$new_sub($lh, ...parameters...).
255 Bracket Notation is discussed in a later section. Note
256 that trying to compile a string into Bracket Notation can throw
257 an exception if the string is not syntactically valid (say, by not
258 balancing brackets right.)
260 Also, calling &$coderef($lh, ...parameters...) can throw any sort of
261 exception (if, say, code in that sub tries to divide by zero). But
262 a very common exception occurs when you have Bracket
263 Notation text that says to call a method "foo", but there is no such
264 method. (E.g., "You have [quaB<tn>,_1,ball]." will throw an exception
265 on trying to call $lh->quaB<tn>($_[1],'ball') -- you presumably meant
266 "quant".) C<maketext> catches these exceptions, but only to make the
267 error message more readable, at which point it rethrows the exception.
269 An exception I<may> be thrown if I<key> is not found in any
270 of $lh's %Lexicon hashes. What happens if a key is not found,
271 is discussed in a later section, "Controlling Lookup Failure".
273 Note that you might find it useful in some cases to override
274 the C<maketext> method with an "after method", if you want to
275 translate encodings, or even scripts:
277 package YrProj::zh_cn; # Chinese with PRC-style glyphs
278 use base ('YrProj::zh_tw'); # Taiwan-style
280 my $self = shift(@_);
281 my $value = $self->maketext(@_);
282 return Chineeze::taiwan2mainland($value);
285 Or you may want to override it with something that traps
286 any exceptions, if that's critical to your program:
289 my($lh, @stuff) = @_;
291 eval { $out = $lh->SUPER::maketext(@stuff) };
292 return $out unless $@;
293 ...otherwise deal with the exception...
296 Other than those two situations, I don't imagine that
297 it's useful to override the C<maketext> method. (If
298 you run into a situation where it is useful, I'd be
299 interested in hearing about it.)
303 =item $lh->fail_with I<or> $lh->fail_with(I<PARAM>)
305 =item $lh->failure_handler_auto
307 These two methods are discussed in the section "Controlling
312 =head2 Utility Methods
314 These are methods that you may find it handy to use, generally
315 from %Lexicon routines of yours (whether expressed as
316 Bracket Notation or not).
320 =item $language->quant($number, $singular)
322 =item $language->quant($number, $singular, $plural)
324 =item $language->quant($number, $singular, $plural, $negative)
326 This is generally meant to be called from inside Bracket Notation
327 (which is discussed later), as in
329 "Your search matched [quant,_1,document]!"
331 It's for I<quantifying> a noun (i.e., saying how much of it there is,
332 while giving the correct form of it). The behavior of this method is
333 handy for English and a few other Western European languages, and you
334 should override it for languages where it's not suitable. You can feel
335 free to read the source, but the current implementation is basically
336 as this pseudocode describes:
338 if $number is 0 and there's a $negative,
341 return "1 $singular";
342 elsif there's a $plural,
343 return "$number $plural";
345 return "$number " . $singular . "s";
347 # ...except that we actually call numf to
348 # stringify $number before returning it.
350 So for English (with Bracket Notation)
351 C<"...[quant,_1,file]..."> is fine (for 0 it returns "0 files",
352 for 1 it returns "1 file", and for more it returns "2 files", etc.)
354 But for "directory", you'd want C<"[quant,_1,directory,directories]">
355 so that our elementary C<quant> method doesn't think that the
356 plural of "directory" is "directorys". And you might find that the
357 output may sound better if you specify a negative form, as in:
359 "[quant,_1,file,files,No files] matched your query.\n"
361 Remember to keep in mind verb agreement (or adjectives too, in
362 other languages), as in:
364 "[quant,_1,document] were matched.\n"
366 Because if _1 is one, you get "1 document B<were> matched".
367 An acceptable hack here is to do something like this:
369 "[quant,_1,document was, documents were] matched.\n"
371 =item $language->numf($number)
373 This returns the given number formatted nicely according to
374 this language's conventions. Maketext's default method is
375 mostly to just take the normal string form of the number
376 (applying sprintf "%G" for only very large numbers), and then
377 to add commas as necessary. (Except that
378 we apply C<tr/,./.,/> if $language->{'numf_comma'} is true;
379 that's a bit of a hack that's useful for languages that express
380 two million as "2.000.000" and not as "2,000,000").
382 If you want anything fancier, consider overriding this with something
383 that uses L<Number::Format|Number::Format>, or does something else
386 Note that numf is called by quant for stringifying all quantifying
389 =item $language->sprintf($format, @items)
391 This is just a wrapper around Perl's normal C<sprintf> function.
392 It's provided so that you can use "sprintf" in Bracket Notation:
394 "Couldn't access datanode [sprintf,%10x=~[%s~],_1,_2]!\n"
398 Couldn't access datanode Stuff=[thangamabob]!
400 =item $language->language_tag()
402 Currently this just takes the last bit of C<ref($language)>, turns
403 underscores to dashes, and returns it. So if $language is
404 an object of class Hee::HOO::Haw::en_us, $language->language_tag()
405 returns "en-us". (Yes, the usual representation for that language
406 tag is "en-US", but case is I<never> considered meaningful in
407 language-tag comparison.)
409 You may override this as you like; Maketext doesn't use it for
412 =item $language->encoding()
414 Currently this isn't used for anything, but it's provided
415 (with default value of
416 C<(ref($language) && $language-E<gt>{'encoding'})) or "iso-8859-1">
417 ) as a sort of suggestion that it may be useful/necessary to
418 associate encodings with your language handles (whether on a
419 per-class or even per-handle basis.)
423 =head2 Language Handle Attributes and Internals
425 A language handle is a flyweight object -- i.e., it doesn't (necessarily)
426 carry any data of interest, other than just being a member of
427 whatever class it belongs to.
429 A language handle is implemented as a blessed hash. Subclasses of yours
430 can store whatever data you want in the hash. Currently the only hash
431 entry used by any crucial Maketext method is "fail", so feel free to
432 use anything else as you like.
434 B<Remember: Don't be afraid to read the Maketext source if there's
435 any point on which this documentation is unclear.> This documentation
436 is vastly longer than the module source itself.
442 =head1 LANGUAGE CLASS HIERARCHIES
444 These are Locale::Maketext's assumptions about the class
445 hierarchy formed by all your language classes:
451 You must have a project base class, which you load, and
452 which you then use as the first argument in
453 the call to YourProjClass->get_handle(...). It should derive
454 (whether directly or indirectly) from Locale::Maketext.
455 It B<doesn't matter> how you name this class, although assuming this
456 is the localization component of your Super Mega Program,
457 good names for your project class might be
458 SuperMegaProgram::Localization, SuperMegaProgram::L10N,
459 SuperMegaProgram::I18N, SuperMegaProgram::International,
460 or even SuperMegaProgram::Languages or SuperMegaProgram::Messages.
464 Language classes are what YourProjClass->get_handle will try to load.
465 It will look for them by taking each language-tag (B<skipping> it
466 if it doesn't look like a language-tag or locale-tag!), turning it to
467 all lowercase, turning dashes to underscores, and appending it
468 to YourProjClass . "::". So this:
470 $lh = YourProjClass->get_handle(
471 'en-US', 'fr', 'kon', 'i-klingon', 'i-klingon-romanized'
474 will try loading the classes
475 YourProjClass::en_us (note lowercase!), YourProjClass::fr,
477 YourProjClass::i_klingon
478 and YourProjClass::i_klingon_romanized. (And it'll stop at the
479 first one that actually loads.)
483 I assume that each language class derives (directly or indirectly)
484 from your project class, and also defines its @ISA, its %Lexicon,
485 or both. But I anticipate no dire consequences if these assumptions
490 Language classes may derive from other language classes (although they
491 should have "use I<Thatclassname>" or "use base qw(I<...classes...>)").
492 They may derive from the project
493 class. They may derive from some other class altogether. Or via
494 multiple inheritance, it may derive from any mixture of these.
498 I foresee no problems with having multiple inheritance in
499 your hierarchy of language classes. (As usual, however, Perl will
500 complain bitterly if you have a cycle in the hierarchy: i.e., if
501 any class is its own ancestor.)
505 =head1 ENTRIES IN EACH LEXICON
507 A typical %Lexicon entry is meant to signify a phrase,
508 taking some number (0 or more) of parameters. An entry
509 is meant to be accessed by via
510 a string I<key> in $lh->maketext(I<key>, ...parameters...),
511 which should return a string that is generally meant for
512 be used for "output" to the user -- regardless of whether
513 this actually means printing to STDOUT, writing to a file,
514 or putting into a GUI widget.
516 While the key must be a string value (since that's a basic
517 restriction that Perl places on hash keys), the value in
518 the lexicon can currently be of several types:
519 a defined scalar, scalarref, or coderef. The use of these is
520 explained above, in the section 'The "maketext" Method', and
521 Bracket Notation for strings is discussed in the next section.
523 While you can use arbitrary unique IDs for lexicon keys
524 (like "_min_larger_max_error"), it is often
525 useful for if an entry's key is itself a valid value, like
526 this example error message:
528 "Minimum ([_1]) is larger than maximum ([_2])!\n",
530 Compare this code that uses an arbitrary ID...
532 die $lh->maketext( "_min_larger_max_error", $min, $max )
535 ...to this code that uses a key-as-value:
538 "Minimum ([_1]) is larger than maximum ([_2])!\n",
542 The second is, in short, more readable. In particular, it's obvious
543 that the number of parameters you're feeding to that phrase (two) is
544 the number of parameters that it I<wants> to be fed. (Since you see
545 _1 and a _2 being used in the key there.)
547 Also, once a project is otherwise
548 complete and you start to localize it, you can scrape together
549 all the various keys you use, and pass it to a translator; and then
550 the translator's work will go faster if what he's presented is this:
552 "Minimum ([_1]) is larger than maximum ([_2])!\n",
553 => "", # fill in something here, Jacques!
555 rather than this more cryptic mess:
557 "_min_larger_max_error"
558 => "", # fill in something here, Jacques
560 I think that keys as lexicon values makes the completed lexicon
561 entries more readable:
563 "Minimum ([_1]) is larger than maximum ([_2])!\n",
564 => "Le minimum ([_1]) est plus grand que le maximum ([_2])!\n",
566 Also, having valid values as keys becomes very useful if you set
567 up an _AUTO lexicon. _AUTO lexicons are discussed in a later
570 I almost always use keys that are themselves
571 valid lexicon values. One notable exception is when the value is
572 quite long. For example, to get the screenful of data that
573 a command-line program might return when given an unknown switch,
574 I often just use a brief, self-explanatory key such as "_USAGE_MESSAGE". At that point I then go
575 and immediately to define that lexicon entry in the
576 ProjectClass::L10N::en lexicon (since English is always my "project
579 '_USAGE_MESSAGE' => <<'EOSTUFF',
580 ...long long message...
583 and then I can use it as:
585 getopt('oDI', \%opts) or die $lh->maketext('_USAGE_MESSAGE');
588 note that each class's C<%Lexicon> inherits-and-extends
589 the lexicons in its superclasses. This is not because these are
590 special hashes I<per se>, but because you access them via the
591 C<maketext> method, which looks for entries across all the
592 C<%Lexicon> hashes in a language class I<and> all its ancestor classes.
593 (This is because the idea of "class data" isn't directly implemented
594 in Perl, but is instead left to individual class-systems to implement
597 Note that you may have things stored in a lexicon
598 besides just phrases for output: for example, if your program
599 takes input from the keyboard, asking a "(Y/N)" question,
600 you probably need to know what the equivalent of "Y[es]/N[o]" is
601 in whatever language. You probably also need to know what
602 the equivalents of the answers "y" and "n" are. You can
603 store that information in the lexicon (say, under the keys
604 "~answer_y" and "~answer_n", and the long forms as
605 "~answer_yes" and "~answer_no", where "~" is just an ad-hoc
606 character meant to indicate to programmers/translators that
607 these are not phrases for output).
609 Or instead of storing this in the language class's lexicon,
610 you can (and, in some cases, really should) represent the same bit
611 of knowledge as code in a method in the language class. (That
612 leaves a tidy distinction between the lexicon as the things we
613 know how to I<say>, and the rest of the things in the lexicon class
614 as things that we know how to I<do>.) Consider
615 this example of a processor for responses to French "oui/non"
619 return undef unless defined $_[1] and length $_[1];
620 my $answer = lc $_[1]; # smash case
621 return 1 if $answer eq 'o' or $answer eq 'oui';
622 return 0 if $answer eq 'n' or $answer eq 'non';
626 ...which you'd then call in a construct like this:
629 until(defined $response) {
630 print $lh->maketext("Open the pod bay door (y/n)? ");
631 $response = $lh->y_or_n( get_input_from_keyboard_somehow() );
633 if($response) { $pod_bay_door->open() }
634 else { $pod_bay_door->leave_closed() }
636 Other data worth storing in a lexicon might be things like
637 filenames for language-targetted resources:
641 => "/styles/en_us/main_splash.png",
642 "_main_splash_imagemap"
643 => "/styles/en_us/main_splash.incl",
644 "_general_graphics_path"
647 => "/styles/en_us/hey_there.wav",
651 => "right_arrow.png",
652 # In some other languages, left equals
653 # BACKwards, and right is FOREwards.
656 You might want to do the same thing for expressing key bindings
657 or the like (since hardwiring "q" as the binding for the function
658 that quits a screen/menu/program is useful only if your language
659 happens to associate "q" with "quit"!)
661 =head1 BRACKET NOTATION
663 Bracket Notation is a crucial feature of Locale::Maketext. I mean
664 Bracket Notation to provide a replacement for the use of sprintf formatting.
665 Everything you do with Bracket Notation could be done with a sub block,
666 but bracket notation is meant to be much more concise.
668 Bracket Notation is a like a miniature "template" system (in the sense
669 of L<Text::Template|Text::Template>, not in the sense of C++ templates),
670 where normal text is passed thru basically as is, but text in special
671 regions is specially interpreted. In Bracket Notation, you use square brackets ("[...]"),
672 not curly braces ("{...}") to note sections that are specially interpreted.
674 For example, here all the areas that are taken literally are underlined with
675 a "^", and all the in-bracket special regions are underlined with an X:
677 "Minimum ([_1]) is larger than maximum ([_2])!\n",
678 ^^^^^^^^^ XX ^^^^^^^^^^^^^^^^^^^^^^^^^^ XX ^^^^
680 When that string is compiled from bracket notation into a real Perl sub,
681 it's basically turned into:
689 ") is larger than maximum (",
693 # to be called by $lh->maketext(KEY, params...)
695 In other words, text outside bracket groups is turned into string
696 literals. Text in brackets is rather more complex, and currently follows
703 Bracket groups that are empty, or which consist only of whitespace,
704 are ignored. (Examples: "[]", "[ ]", or a [ and a ] with returns
705 and/or tabs and/or spaces between them.
707 Otherwise, each group is taken to be a comma-separated group of items,
708 and each item is interpreted as follows:
712 An item that is "_I<digits>" or "_-I<digits>" is interpreted as
713 $_[I<value>]. I.e., "_1" becomes with $_[1], and "_-3" is interpreted
714 as $_[-3] (in which case @_ should have at least three elements in it).
715 Note that $_[0] is the language handle, and is typically not named
720 An item "_*" is interpreted to mean "all of @_ except $_[0]".
721 I.e., C<@_[1..$#_]>. Note that this is an empty list in the case
722 of calls like $lh->maketext(I<key>) where there are no
723 parameters (except $_[0], the language handle).
727 Otherwise, each item is interpreted as a string literal.
731 The group as a whole is interpreted as follows:
737 If the first item in a bracket group looks like a method name,
738 then that group is interpreted like this:
740 $lh->that_method_name(
741 ...rest of items in this group...
746 If the first item in a bracket group is "*", it's taken as shorthand
747 for the so commonly called "quant" method. Similarly, if the first
748 item in a bracket group is "#", it's taken to be shorthand for
753 If the first item in a bracket group is the empty-string, or "_*"
754 or "_I<digits>" or "_-I<digits>", then that group is interpreted
755 as just the interpolation of all its items:
758 ...rest of items in this group...
761 Examples: "[_1]" and "[,_1]", which are synonymous; and
762 "C<[,ID-(,_4,-,_2,)]>", which compiles as
763 C<join "", "ID-(", $_[4], "-", $_[2], ")">.
767 Otherwise this bracket group is invalid. For example, in the group
768 "[!@#,whatever]", the first item C<"!@#"> is neither the empty-string,
769 "_I<number>", "_-I<number>", "_*", nor a valid method name; and so
770 Locale::Maketext will throw an exception of you try compiling an
771 expression containing this bracket group.
775 Note, incidentally, that items in each group are comma-separated,
776 not C</\s*,\s*/>-separated. That is, you might expect that this
779 "Hoohah [foo, _1 , bar ,baz]!"
781 would compile to this:
787 $lh->foo( $_[1], "bar", "baz"),
791 But it actually compiles as this:
797 $lh->foo(" _1 ", " bar ", "baz"), # note the <space> in " bar "
801 In the notation discussed so far, the characters "[" and "]" are given
802 special meaning, for opening and closing bracket groups, and "," has
803 a special meaning inside bracket groups, where it separates items in the
804 group. This begs the question of how you'd express a literal "[" or
805 "]" in a Bracket Notation string, and how you'd express a literal
806 comma inside a bracket group. For this purpose I've adopted "~" (tilde)
807 as an escape character: "~[" means a literal '[' character anywhere
808 in Bracket Notation (i.e., regardless of whether you're in a bracket
809 group or not), and ditto for "~]" meaning a literal ']', and "~," meaning
810 a literal comma. (Altho "," means a literal comma outside of
811 bracket groups -- it's only inside bracket groups that commas are special.)
813 And on the off chance you need a literal tilde in a bracket expression,
814 you get it with "~~".
816 Currently, an unescaped "~" before a character
817 other than a bracket or a comma is taken to mean just a "~" and that
818 character. I.e., "~X" means the same as "~~X" -- i.e., one literal tilde,
819 and then one literal "X". However, by using "~X", you are assuming that
820 no future version of Maketext will use "~X" as a magic escape sequence.
821 In practice this is not a great problem, since first off you can just
822 write "~~X" and not worry about it; second off, I doubt I'll add lots
823 of new magic characters to bracket notation; and third off, you
824 aren't likely to want literal "~" characters in your messages anyway,
825 since it's not a character with wide use in natural language text.
827 Brackets must be balanced -- every openbracket must have
828 one matching closebracket, and vice versa. So these are all B<invalid>:
830 "I ate [quant,_1,rhubarb pie."
831 "I ate [quant,_1,rhubarb pie[."
832 "I ate quant,_1,rhubarb pie]."
833 "I ate quant,_1,rhubarb pie[."
835 Currently, bracket groups do not nest. That is, you B<cannot> say:
837 "Foo [bar,baz,[quux,quuux]]\n";
839 If you need a notation that's that powerful, use normal Perl:
847 $lh->bar('baz', $lh->quux('quuux')),
853 Or write the "bar" method so you don't need to pass it the
854 output from calling quux.
856 I do not anticipate that you will need (or particularly want)
857 to nest bracket groups, but you are welcome to email me with
858 convincing (real-life) arguments to the contrary.
862 If maketext goes to look in an individual %Lexicon for an entry
863 for I<key> (where I<key> does not start with an underscore), and
864 sees none, B<but does see> an entry of "_AUTO" => I<some_true_value>,
865 then we actually define $Lexicon{I<key>} = I<key> right then and there,
866 and then use that value as if it had been there all
867 along. This happens before we even look in any superclass %Lexicons!
869 (This is meant to be somewhat like the AUTOLOAD mechanism in
870 Perl's function call system -- or, looked at another way,
871 like the L<AutoLoader|AutoLoader> module.)
873 I can picture all sorts of circumstances where you just
874 do not want lookup to be able to fail (since failing
875 normally means that maketext throws a C<die>, although
876 see the next section for greater control over that). But
877 here's one circumstance where _AUTO lexicons are meant to
878 be I<especially> useful:
880 As you're writing an application, you decide as you go what messages
881 you need to emit. Normally you'd go to write this:
884 go_process_file($filename)
886 print qq{Couldn't find file "$filename"!\n};
889 but since you anticipate localizing this, you write:
891 use ThisProject::I18N;
892 my $lh = ThisProject::I18N->get_handle();
893 # For the moment, assume that things are set up so
894 # that we load class ThisProject::I18N::en
895 # and that that's the class that $lh belongs to.
898 go_process_file($filename)
901 qq{Couldn't find file "[_1]"!\n}, $filename
905 Now, right after you've just written the above lines, you'd
906 normally have to go open the file
907 ThisProject/I18N/en.pm, and immediately add an entry:
909 "Couldn't find file \"[_1]\"!\n"
910 => "Couldn't find file \"[_1]\"!\n",
912 But I consider that somewhat of a distraction from the work
913 of getting the main code working -- to say nothing of the fact
914 that I often have to play with the program a few times before
915 I can decide exactly what wording I want in the messages (which
916 in this case would require me to go changing three lines of code:
917 the call to maketext with that key, and then the two lines in
918 ThisProject/I18N/en.pm).
920 However, if you set "_AUTO => 1" in the %Lexicon in,
921 ThisProject/I18N/en.pm (assuming that English (en) is
922 the language that all your programmers will be using for this
923 project's internal message keys), then you don't ever have to
924 go adding lines like this
926 "Couldn't find file \"[_1]\"!\n"
927 => "Couldn't find file \"[_1]\"!\n",
929 to ThisProject/I18N/en.pm, because if _AUTO is true there,
930 then just looking for an entry with the key "Couldn't find
931 file \"[_1]\"!\n" in that lexicon will cause it to be added,
934 Note that the reason that keys that start with "_"
935 are immune to _AUTO isn't anything generally magical about
936 the underscore character -- I just wanted a way to have most
937 lexicon keys be autoable, except for possibly a few, and I
938 arbitrarily decided to use a leading underscore as a signal
939 to distinguish those few.
941 =head1 READONLY LEXICONS
943 If your lexicon is a tied hash the simple act of caching the compiled value can be fatal.
945 For example a L<GDBM_File> GDBM_READER tied hash will die with something like:
947 gdbm store returned -1, errno 2, key "..." at ...
949 All you need to do is turn on caching outside of the lexicon hash itself like so:
954 $lh->{'use_external_lex_cache'} = 1;
958 And then instead of storing the compiled value in the lexicon hash it will store it in $lh->{'_external_lex_cache'}
960 =head1 CONTROLLING LOOKUP FAILURE
962 If you call $lh->maketext(I<key>, ...parameters...),
963 and there's no entry I<key> in $lh's class's %Lexicon, nor
964 in the superclass %Lexicon hash, I<and> if we can't auto-make
965 I<key> (because either it starts with a "_", or because none
966 of its lexicons have C<_AUTO =E<gt> 1,>), then we have
967 failed to find a normal way to maketext I<key>. What then
968 happens in these failure conditions, depends on the $lh object's
971 If the language handle has no "fail" attribute, maketext
972 will simply throw an exception (i.e., it calls C<die>, mentioning
973 the I<key> whose lookup failed, and naming the line number where
974 the calling $lh->maketext(I<key>,...) was.
976 If the language handle has a "fail" attribute whose value is a
977 coderef, then $lh->maketext(I<key>,...params...) gives up and calls:
979 return $that_subref->($lh, $key, @params);
981 Otherwise, the "fail" attribute's value should be a string denoting
982 a method name, so that $lh->maketext(I<key>,...params...) can
985 return $lh->$that_method_name($phrase, @params);
987 The "fail" attribute can be accessed with the C<fail_with> method:
990 $lh->fail_with( \&failure_handler );
992 # Set to a method name:
993 $lh->fail_with( 'failure_method' );
995 # Set to nothing (i.e., so failure throws a plain exception)
996 $lh->fail_with( undef );
998 # Get the current value
999 $handler = $lh->fail_with();
1001 Now, as to what you may want to do with these handlers: Maybe you'd
1002 want to log what key failed for what class, and then die. Maybe
1003 you don't like C<die> and instead you want to send the error message
1004 to STDOUT (or wherever) and then merely C<exit()>.
1006 Or maybe you don't want to C<die> at all! Maybe you could use a
1009 # Make all lookups fall back onto an English value,
1010 # but only after we log it for later fingerpointing.
1011 my $lh_backup = ThisProject->get_handle('en');
1012 open(LEX_FAIL_LOG, ">>wherever/lex.log") || die "GNAARGH $!";
1014 my($failing_lh, $key, $params) = @_;
1015 print LEX_FAIL_LOG scalar(localtime), "\t",
1016 ref($failing_lh), "\t", $key, "\n";
1017 return $lh_backup->maketext($key,@params);
1020 Some users have expressed that they think this whole mechanism of
1021 having a "fail" attribute at all, seems a rather pointless complication.
1022 But I want Locale::Maketext to be usable for software projects of I<any>
1023 scale and type; and different software projects have different ideas
1024 of what the right thing is to do in failure conditions. I could simply
1025 say that failure always throws an exception, and that if you want to be
1026 careful, you'll just have to wrap every call to $lh->maketext in an
1027 S<eval { }>. However, I want programmers to reserve the right (via
1028 the "fail" attribute) to treat lookup failure as something other than
1029 an exception of the same level of severity as a config file being
1030 unreadable, or some essential resource being inaccessible.
1032 One possibly useful value for the "fail" attribute is the method name
1033 "failure_handler_auto". This is a method defined in the class
1034 Locale::Maketext itself. You set it with:
1036 $lh->fail_with('failure_handler_auto');
1038 Then when you call $lh->maketext(I<key>, ...parameters...) and
1039 there's no I<key> in any of those lexicons, maketext gives up with
1041 return $lh->failure_handler_auto($key, @params);
1043 But failure_handler_auto, instead of dying or anything, compiles
1046 $lh->{'failure_lex'}{$key} = $complied
1048 and then calls the compiled value, and returns that. (I.e., if
1049 $key looks like bracket notation, $compiled is a sub, and we return
1050 &{$compiled}(@params); but if $key is just a plain string, we just
1053 The effect of using "failure_auto_handler"
1054 is like an AUTO lexicon, except that it 1) compiles $key even if
1055 it starts with "_", and 2) you have a record in the new hashref
1056 $lh->{'failure_lex'} of all the keys that have failed for
1057 this object. This should avoid your program dying -- as long
1058 as your keys aren't actually invalid as bracket code, and as
1059 long as they don't try calling methods that don't exist.
1061 "failure_auto_handler" may not be exactly what you want, but I
1062 hope it at least shows you that maketext failure can be mitigated
1063 in any number of very flexible ways. If you can formalize exactly
1064 what you want, you should be able to express that as a failure
1065 handler. You can even make it default for every object of a given
1066 class, by setting it in that class's init:
1069 my $lh = $_[0]; # a newborn handle
1071 $lh->fail_with('my_clever_failure_handler');
1074 sub my_clever_failure_handler {
1075 ...you clever things here...
1078 =head1 HOW TO USE MAKETEXT
1080 Here is a brief checklist on how to use Maketext to localize
1087 Decide what system you'll use for lexicon keys. If you insist,
1088 you can use opaque IDs (if you're nostalgic for C<catgets>),
1089 but I have better suggestions in the
1090 section "Entries in Each Lexicon", above. Assuming you opt for
1091 meaningful keys that double as values (like "Minimum ([_1]) is
1092 larger than maximum ([_2])!\n"), you'll have to settle on what
1093 language those should be in. For the sake of argument, I'll
1094 call this English, specifically American English, "en-US".
1098 Create a class for your localization project. This is
1099 the name of the class that you'll use in the idiom:
1102 my $lh = Projname::L10N->get_handle(...) || die "Language?";
1104 Assuming you call your class Projname::L10N, create a class
1105 consisting minimally of:
1107 package Projname::L10N;
1108 use base qw(Locale::Maketext);
1109 ...any methods you might want all your languages to share...
1111 # And, assuming you want the base class to be an _AUTO lexicon,
1112 # as is discussed a few sections up:
1118 Create a class for the language your internal keys are in. Name
1119 the class after the language-tag for that language, in lowercase,
1120 with dashes changed to underscores. Assuming your project's first
1121 language is US English, you should call this Projname::L10N::en_us.
1122 It should consist minimally of:
1124 package Projname::L10N::en_us;
1125 use base qw(Projname::L10N);
1131 (For the rest of this section, I'll assume that this "first
1132 language class" of Projname::L10N::en_us has
1137 Go and write your program. Everywhere in your program where
1140 print "Foobar $thing stuff\n";
1142 instead do it thru maketext, using no variable interpolation in
1145 print $lh->maketext("Foobar [_1] stuff\n", $thing);
1147 If you get tired of constantly saying C<print $lh-E<gt>maketext>,
1148 consider making a functional wrapper for it, like so:
1152 $lh = Projname::L10N->get_handle(...) || die "Language?";
1153 sub pmt (@) { print( $lh->maketext(@_)) }
1154 # "pmt" is short for "Print MakeText"
1156 # so if maketext fails, we see made the call to pmt
1158 Besides whole phrases meant for output, anything language-dependent
1159 should be put into the class Projname::L10N::en_us,
1160 whether as methods, or as lexicon entries -- this is discussed
1161 in the section "Entries in Each Lexicon", above.
1165 Once the program is otherwise done, and once its localization for
1166 the first language works right (via the data and methods in
1167 Projname::L10N::en_us), you can get together the data for translation.
1168 If your first language lexicon isn't an _AUTO lexicon, then you already
1169 have all the messages explicitly in the lexicon (or else you'd be
1170 getting exceptions thrown when you call $lh->maketext to get
1171 messages that aren't in there). But if you were (advisedly) lazy and are
1172 using an _AUTO lexicon, then you've got to make a list of all the phrases
1173 that you've so far been letting _AUTO generate for you. There are very
1174 many ways to assemble such a list. The most straightforward is to simply
1175 grep the source for every occurrence of "maketext" (or calls
1176 to wrappers around it, like the above C<pmt> function), and to log the
1181 You may at this point want to consider whether your base class
1182 (Projname::L10N), from which all lexicons inherit from (Projname::L10N::en,
1183 Projname::L10N::es, etc.), should be an _AUTO lexicon. It may be true
1184 that in theory, all needed messages will be in each language class;
1185 but in the presumably unlikely or "impossible" case of lookup failure,
1186 you should consider whether your program should throw an exception,
1187 emit text in English (or whatever your project's first language is),
1188 or some more complex solution as described in the section
1189 "Controlling Lookup Failure", above.
1193 Submit all messages/phrases/etc. to translators.
1195 (You may, in fact, want to start with localizing to I<one> other language
1196 at first, if you're not sure that you've properly abstracted the
1197 language-dependent parts of your code.)
1199 Translators may request clarification of the situation in which a
1200 particular phrase is found. For example, in English we are entirely happy
1201 saying "I<n> files found", regardless of whether we mean "I looked for files,
1202 and found I<n> of them" or the rather distinct situation of "I looked for
1203 something else (like lines in files), and along the way I saw I<n>
1204 files." This may involve rethinking things that you thought quite clear:
1205 should "Edit" on a toolbar be a noun ("editing") or a verb ("to edit")? Is
1206 there already a conventionalized way to express that menu option, separate
1207 from the target language's normal word for "to edit"?
1209 In all cases where the very common phenomenon of quantification
1210 (saying "I<N> files", for B<any> value of N)
1211 is involved, each translator should make clear what dependencies the
1212 number causes in the sentence. In many cases, dependency is
1213 limited to words adjacent to the number, in places where you might
1214 expect them ("I found the-?PLURAL I<N>
1215 empty-?PLURAL directory-?PLURAL"), but in some cases there are
1216 unexpected dependencies ("I found-?PLURAL ..."!) as well as long-distance
1217 dependencies "The I<N> directory-?PLURAL could not be deleted-?PLURAL"!).
1219 Remind the translators to consider the case where N is 0:
1220 "0 files found" isn't exactly natural-sounding in any language, but it
1221 may be unacceptable in many -- or it may condition special
1222 kinds of agreement (similar to English "I didN'T find ANY files").
1224 Remember to ask your translators about numeral formatting in their
1225 language, so that you can override the C<numf> method as
1226 appropriate. Typical variables in number formatting are: what to
1227 use as a decimal point (comma? period?); what to use as a thousands
1228 separator (space? nonbreaking space? comma? period? small
1229 middot? prime? apostrophe?); and even whether the so-called "thousands
1230 separator" is actually for every third digit -- I've heard reports of
1231 two hundred thousand being expressible as "2,00,000" for some Indian
1232 (Subcontinental) languages, besides the less surprising "S<200 000>",
1233 "200.000", "200,000", and "200'000". Also, using a set of numeral
1234 glyphs other than the usual ASCII "0"-"9" might be appreciated, as via
1235 C<tr/0-9/\x{0966}-\x{096F}/> for getting digits in Devanagari script
1236 (for Hindi, Konkani, others).
1238 The basic C<quant> method that Locale::Maketext provides should be
1239 good for many languages. For some languages, it might be useful
1240 to modify it (or its constituent C<numerate> method)
1241 to take a plural form in the two-argument call to C<quant>
1242 (as in "[quant,_1,files]") if
1243 it's all-around easier to infer the singular form from the plural, than
1244 to infer the plural form from the singular.
1246 But for other languages (as is discussed at length
1247 in L<Locale::Maketext::TPJ13|Locale::Maketext::TPJ13>), simple
1248 C<quant>/C<numerify> is not enough. For the particularly problematic
1249 Slavic languages, what you may need is a method which you provide
1250 with the number, the citation form of the noun to quantify, and
1251 the case and gender that the sentence's syntax projects onto that
1252 noun slot. The method would then be responsible for determining
1253 what grammatical number that numeral projects onto its noun phrase,
1254 and what case and gender it may override the normal case and gender
1255 with; and then it would look up the noun in a lexicon providing
1256 all needed inflected forms.
1260 You may also wish to discuss with the translators the question of
1261 how to relate different subforms of the same language tag,
1262 considering how this reacts with C<get_handle>'s treatment of
1263 these. For example, if a user accepts interfaces in "en, fr", and
1264 you have interfaces available in "en-US" and "fr", what should
1265 they get? You may wish to resolve this by establishing that "en"
1266 and "en-US" are effectively synonymous, by having one class
1267 zero-derive from the other.
1269 For some languages this issue may never come up (Danish is rarely
1270 expressed as "da-DK", but instead is just "da"). And for other
1271 languages, the whole concept of a "generic" form may verge on
1272 being uselessly vague, particularly for interfaces involving voice
1273 media in forms of Arabic or Chinese.
1277 Once you've localized your program/site/etc. for all desired
1278 languages, be sure to show the result (whether live, or via
1279 screenshots) to the translators. Once they approve, make every
1280 effort to have it then checked by at least one other speaker of
1281 that language. This holds true even when (or especially when) the
1282 translation is done by one of your own programmers. Some
1283 kinds of systems may be harder to find testers for than others,
1284 depending on the amount of domain-specific jargon and concepts
1285 involved -- it's easier to find people who can tell you whether
1286 they approve of your translation for "delete this message" in an
1287 email-via-Web interface, than to find people who can give you
1288 an informed opinion on your translation for "attribute value"
1289 in an XML query tool's interface.
1295 I recommend reading all of these:
1297 L<Locale::Maketext::TPJ13|Locale::Maketext::TPJ13> -- my I<The Perl
1298 Journal> article about Maketext. It explains many important concepts
1299 underlying Locale::Maketext's design, and some insight into why
1300 Maketext is better than the plain old approach of having
1301 message catalogs that are just databases of sprintf formats.
1303 L<File::Findgrep|File::Findgrep> is a sample application/module
1304 that uses Locale::Maketext to localize its messages. For a larger
1305 internationalized system, see also L<Apache::MP3>.
1307 L<I18N::LangTags|I18N::LangTags>.
1309 L<Win32::Locale|Win32::Locale>.
1311 RFC 3066, I<Tags for the Identification of Languages>,
1312 as at http://sunsite.dk/RFC/rfc/rfc3066.html
1314 RFC 2277, I<IETF Policy on Character Sets and Languages>
1315 is at http://sunsite.dk/RFC/rfc/rfc2277.html -- much of it is
1316 just things of interest to protocol designers, but it explains
1317 some basic concepts, like the distinction between locales and
1320 The manual for GNU C<gettext>. The gettext dist is available in
1321 C<ftp://prep.ai.mit.edu/pub/gnu/> -- get
1322 a recent gettext tarball and look in its "doc/" directory, there's
1323 an easily browsable HTML version in there. The
1324 gettext documentation asks lots of questions worth thinking
1325 about, even if some of their answers are sometimes wonky,
1326 particularly where they start talking about pluralization.
1328 The Locale/Maketext.pm source. Obverse that the module is much
1329 shorter than its documentation!
1331 =head1 COPYRIGHT AND DISCLAIMER
1333 Copyright (c) 1999-2004 Sean M. Burke. All rights reserved.
1335 This library is free software; you can redistribute it and/or modify
1336 it under the same terms as Perl itself.
1338 This program is distributed in the hope that it will be useful, but
1339 without any warranty; without even the implied warranty of
1340 merchantability or fitness for a particular purpose.
1344 Sean M. Burke C<sburke@cpan.org>