This is a live mirror of the Perl 5 development currently hosted at https://github.com/perl/perl5
Integrate Locale::Maketext 1.01 from Sean Burke.
[perl5.git] / lib / Locale / Maketext.pod
CommitLineData
9378c581
JH
1
2# Time-stamp: "2001-05-25 07:50:08 MDT"
3
4=head1 NAME
5
6Locale::Maketext -- framework for localization
7
8=head1 SYNOPSIS
9
10 package MyProgram;
11 use strict;
12 use MyProgram::L10N;
13 # ...which inherits from Locale::Maketext
14 my $lh = MyProgram::L10N->get_handle() || die "What language?";
15 ...
16 # And then any messages your program emits, like:
17 warn $lh->maketext( "Can't open file [_1]: [_2]\n", $f, $! );
18 ...
19
20=head1 DESCRIPTION
21
22It is a common feature of applications (whether run directly,
23or via the Web) for them to be "localized" -- i.e., for them
24to a present an English interface to an English-speaker, a German
25interface to a German-speaker, and so on for all languages it's
26programmed with. Locale::Maketext
27is a framework for software localization; it provides you with the
28tools for organizing and accessing the bits of text and text-processing
29code that you need for producing localized applications.
30
31In order to make sense of Maketext and how all its
32components fit together, you should probably
33go read L<Locale::Maketext::TPJ13|Locale::Maketext::TPJ13>, and
34I<then> read the following documentation.
35
36You may also want to read over the source for C<File::Findgrep>
37and its constituent modules -- they are a complete (if small)
38example application that uses Maketext.
39
40=head1 QUICK OVERVIEW
41
42The basic design of Locale::Maketext is object-oriented, and
43Locale::Maketext is an abstract base class, from which you
44derive a "project class".
45The project class (with a name like "TkBocciBall::Localize",
46which you then use in your module) is in turn the base class
47for all the "language classes" for your project
48(with names "TkBocciBall::Localize::it",
49"TkBocciBall::Localize::en",
50"TkBocciBall::Localize::fr", etc.).
51
52A language class is
53a class containing a lexicon of phrases as class data,
54and possibly also some methods that are of use in interpreting
55phrases in the lexicon, or otherwise dealing with text in that
56language.
57
58An object belonging to a language class is called a "language
59handle"; it's typically a flyweight object.
60
61The normal course of action is to call:
62
63 use TkBocciBall::Localize; # the localization project class
64 $lh = TkBocciBall::Localize->get_handle();
65 # Depending on the user's locale, etc., this will
66 # make a language handle from among the classes available,
67 # and any defaults that you declare.
68 die "Couldn't make a language handle??" unless $lh;
69
70From then on, you use the C<maketext> function to access
71entries in whatever lexicon(s) belong to the language handle
72you got. So, this:
73
74 print $lh->maketext("You won!"), "\n";
75
76...emits the right text for this language. If the object
77in C<$lh> belongs to class "TkBocciBall::Localize::fr" and
78%TkBocciBall::Localize::fr::Lexicon contains C<("You won!"
79=E<gt> "Tu as gagnE<eacute>!")>, then the above
80code happily tells the user "Tu as gagnE<eacute>!".
81
82=head1 METHODS
83
84Locale::Maketext offers a variety of methods, which fall
85into three categories:
86
87=over
88
89=item *
90
91Methods to do with constructing language handles.
92
93=item *
94
95C<maketext> and other methods to do with accessing %Lexicon data
96for a given language handle.
97
98=item *
99
100Methods that you may find it handy to use, from routines of
101yours that you put in %Lexicon entries.
102
103=back
104
105These are covered in the following section.
106
107=head2 Construction Methods
108
109These are to do with constructing a language handle:
110
111=over
112
113=item $lh = YourProjClass->get_handle( ...langtags... ) || die "lg-handle?";
114
115This tries loading classes based on the language-tags you give (like
116C<("en-US", "sk", "kon", "es-MX", "ja", "i-klingon")>, and for the first class
117that succeeds, returns YourProjClass::I<language>->new().
118
119It runs thru the entire given list of language-tags, and finds no classes
120for those exact terms, it then tries "superordinate" language classes.
121So if no "en-US" class (i.e., YourProjClass::en_us)
122was found, nor classes for anything else in that list, we then try
123its superordinate, "en" (i.e., YourProjClass::en), and so on thru
124the other language-tags in the given list: "es".
125(The other language-tags in our example list:
126happen to have no superordinates.)
127
128If none of those language-tags leads to loadable classes, we then
129try classes derived from YourProjClass->fallback_languages() and
130then if nothing comes of that, we use classes named by
131YourProjClass->fallback_language_classes(). Then in the (probably
132quite unlikely) event that that fails, we just return undef.
133
134=item $lh = YourProjClass->get_handleB<()> || die "lg-handle?";
135
136When C<get_handle> is called with an empty parameter list, magic happens:
137
138If C<get_handle> senses that it's running in program that was
139invoked as a CGI, then it tries to get language-tags out of the
140environment variable "HTTP_ACCEPT_LANGUAGE", and it pretends that
141those were the languages passed as parameters to C<get_handle>.
142
143Otherwise (i.e., if not a CGI), this tries various OS-specific ways
144to get the language-tags for the current locale/language, and then
145pretends that those were the value(s) passed to C<cet_handle>.
146
147Currently this OS-specific stuff consists of looking in the environment
148variables "LANG" and "LANGUAGE"; and on MSWin machines (where those
149variables are typically unused), this also tries using
150the module Win32::Locale to get a language-tag for whatever language/locale
151is currently selected in the "Regional Settings" (or "International"?)
152Control Panel. I welcome further
153suggestions for making this do the Right Thing under other operating
154systems that support localization.
155
156If you're using localization in an application that keeps a configuration
157file, you might consider something like this in your project class:
158
159 sub get_handle_via_config {
160 my $class = $_[0];
161 my $preferred_language = $Config_settings{'language'};
162 my $lh;
163 if($preferred_language) {
164 $lh = $class->get_handle($chosen_language)
165 || die "No language handle for \"$chosen_language\" or the like";
166 } else {
167 # Config file missing, maybe?
168 $lh = $class->get_handle()
169 || die "Can't get a language handle";
170 }
171 return $lh;
172 }
173
174=item $lh = YourProjClass::langname->new();
175
176This constructs a language handle. You usually B<don't> call this
177directly, but instead let C<get_handle> find a language class to C<use>
178and to then call ->new on.
179
180=item $lh->init();
181
182This is called by ->new to initialize newly-constructed language handles.
183If you define an init method in your class, remember that it's usually
184considered a good idea to call $lh->SUPER::init in it (presumably at the
185beginning), so that all classes get a chance to initialize a new object
186however they see fit.
187
188=item YourProjClass->fallback_languages()
189
190C<get_handle> appends the return value of this to the end of
191whatever list of languages you pass C<get_handle>. Unless
192you override this method, your project class
193will inherit Locale::Maketext's C<fallback_languages>, which
194currently returns C<('i-default', 'en', 'en-US')>.
195("i-default" is defined in RFC 2277).
196
197This method (by having it return the name
198of a language-tag that has an existing language class)
199can be used for making sure that
200C<get_handle> will always manage to construct a language
201handle (assuming your language classes are in an appropriate
202@INC directory). Or you can use the next method:
203
204=item YourProjClass->fallback_language_classes()
205
206C<get_handle> appends the return value of this to the end
207of the list of classes it will try using. Unless
208you override this method, your project class
209will inherit Locale::Maketext's C<fallback_language_classes>,
210which currently returns an empty list, C<()>.
211By setting this to some value (namely, the name of a loadable
212language class), you can be sure that
213C<get_handle> will always manage to construct a language
214handle.
215
216=back
217
218=head2 The "maketext" Method
219
220This is the most important method in Locale::Maketext:
221
222$text = $lh->maketext(I<key>, ...parameters for this phrase...);
223
224This looks in the %Lexicon of the language handle
225$lh and all its superclasses, looking
226for an entry whose key is the string I<key>. Assuming such
227an entry is found, various things then happen, depending on the
228value found:
229
230If the value is a scalarref, the scalar is dereferenced and returned
231(and any parameters are ignored).
232If the value is a coderef, we return &$value($lh, ...parameters...).
233If the value is a string that I<doesn't> look like it's in Bracket Notation,
234we return it (after replacing it with a scalarref, in its %Lexicon).
235If the value I<does> look like it's in Bracket Notation, then we compile
236it into a sub, replace the string in the %Lexicon with the new coderef,
237and then we return &$new_sub($lh, ...parameters...).
238
239Bracket Notation is discussed in a later section. Note
240that trying to compile a string into Bracket Notation can throw
241an exception if the string is not syntactically valid (say, by not
242balancing brackets right.)
243
244Also, calling &$coderef($lh, ...parameters...) can throw any sort of
245exception (if, say, code in that sub tries to divide by zero). But
246a very common exception occurs when you have Bracket
247Notation text that says to call a method "foo", but there is no such
248method. (E.g., "You have [quaB<tn>,_1,ball]." will throw an exception
249on trying to call $lh->quaB<tn>($_[1],'ball') -- you presumably meant
250"quant".) C<maketext> catches these exceptions, but only to make the
251error message more readable, at which point it rethrows the exception.
252
253An exception I<may> be thrown if I<key> is not found in any
254of $lh's %Lexicon hashes. What happens if a key is not found,
255is discussed in a later section, "Controlling Lookup Failure".
256
257Note that you might find it useful in some cases to override
258the C<maketext> method with an "after method", if you want to
259translate encodings, or even scripts:
260
261 package YrProj::zh_cn; # Chinese with PRC-style glyphs
262 use base ('YrProj::zh_tw'); # Taiwan-style
263 sub maketext {
264 my $self = shift(@_);
265 my $value = $self->maketext(@_);
266 return Chineeze::taiwan2mainland($value);
267 }
268
269Or you may want to override it with something that traps
270any exceptions, if that's critical to your program:
271
272 sub maketext {
273 my($lh, @stuff) = @_;
274 my $out;
275 eval { $out = $lh->SUPER::maketext(@stuff) };
276 return $out unless $@;
277 ...otherwise deal with the exception...
278 }
279
280Other than those two situations, I don't imagine that
281it's useful to override the C<maketext> method. (If
282you run into a situation where it is useful, I'd be
283interested in hearing about it.)
284
285=over
286
287=item $lh->fail_with I<or> $lh->fail_with(I<PARAM>)
288
289=item $lh->failure_handler_auto
290
291These two methods are discussed in the section "Controlling
292Lookup Failure".
293
294=back
295
296=head2 Utility Methods
297
298These are methods that you may find it handy to use, generally
299from %Lexicon routines of yours (whether expressed as
300Bracket Notation or not).
301
302=over
303
304=item $language->quant($number, $singular)
305
306=item $language->quant($number, $singular, $plural)
307
308=item $language->quant($number, $singular, $plural, $negative)
309
310This is generally meant to be called from inside Bracket Notation
311(which is discussed later), as in
312
313 "Your search matched [quant,_1,document]!"
314
315It's for I<quantifying> a noun (i.e., saying how much of it there is,
316while giving the currect form of it). The behavior of this method is
317handy for English and a few other Western European languages, and you
318should override it for languages where it's not suitable. You can feel
319free to read the source, but the current implementation is basically
320as this pseudocode describes:
321
322 if $number is 0 and there's a $negative,
323 return $negative;
324 elsif $number is 1,
325 return "1 $singular";
326 elsif there's a $plural,
327 return "$number $plural";
328 else
329 return "$number " . $singular . "s";
330 #
331 # ...except that we actually call numf to
332 # stringify $number before returning it.
333
334So for English (with Bracket Notation)
335C<"...[quant,_1,file]..."> is fine (for 0 it returns "0 files",
336for 1 it returns "1 file", and for more it returns "2 files", etc.)
337
338But for "directory", you'd want C<"[quant,_1,direcory,directories]">
339so that our elementary C<quant> method doesn't think that the
340plural of "directory" is "directorys". And you might find that the
341output may sound better if you specify a negative form, as in:
342
343 "[quant,_1,file,files,No files] matched your query.\n"
344
345Remember to keep in mind verb agreement (or adjectives too, in
346other languages), as in:
347
348 "[quant,_1,document] were matched.\n"
349
350Because if _1 is one, you get "1 document B<were> matched".
351An acceptable hack here is to do something like this:
352
353 "[quant,_1,document was, documents were] matched.\n"
354
355=item $language->numf($number)
356
357This returns the given number formatted nicely according to
358this language's conventions. Maketext's default method is
359mostly to just take the normal string form of the number
360(applying sprintf "%G" for only very large numbers), and then
361to add commas as necessary. (Except that
362we apply C<tr/,./.,/> if $language->{'numf_comma'} is true;
363that's a bit of a hack that's useful for languages that express
364two million as "2.000.000" and not as "2,000,000").
365
366If you want anything fancier, consider overriding this with something
367that uses L<Number::Format|Number::Format>, or does something else
368entirely.
369
370Note that numf is called by quant for stringifying all quantifying
371numbers.
372
373=item $language->sprintf($format, @items)
374
375This is just a wrapper around Perl's normal C<sprintf> function.
376It's provided so that you can use "sprintf" in Bracket Notation:
377
378 "Couldn't access datanode [sprintf,%10x=~[%s~],_1,_2]!\n"
379
380returning...
381
382 Couldn't access datanode Stuff=[thangamabob]!
383
384=item $language->language_tag()
385
386Currently this just takes the last bit of C<ref($language)>, turns
387underscores to dashes, and returns it. So if $language is
388an object of class Hee::HOO::Haw::en_us, $language->language_tag()
389returns "en-us". (Yes, the usual representation for that language
390tag is "en-US", but case is I<never> considered meaningful in
391language-tag comparison.)
392
393You may override this as you like; Maketext doesn't use it for
394anything.
395
396=item $language->encoding()
397
398Currently this isn't used for anything, but it's provided
399(with default value of
400C<(ref($language) && $language-E<gt>{'encoding'})) or "iso-8859-1">
401) as a sort of suggestion that it may be useful/necessary to
402associate encodings with your language handles (whether on a
403per-class or even per-handle basis.)
404
405=back
406
407=head2 Language Handle Attributes and Internals
408
409A language handle is a flyweight object -- i.e., it doesn't (necessarily)
410carry any data of interest, other than just being a member of
411whatever class it belongs to.
412
413A language handle is implemented as a blessed hash. Subclasses of yours
414can store whatever data you want in the hash. Currently the only hash
415entry used by any crucial Maketext method is "fail", so feel free to
416use anything else as you like.
417
418B<Remember: Don't be afraid to read the Maketext source if there's
419any point on which this documentation is unclear.> This documentation
420is vastly longer than the module source itself.
421
422=over
423
424=back
425
426=head1 LANGUAGE CLASS HIERARCHIES
427
428These are Locale::Maketext's assumptions about the class
429hierarchy formed by all your language classes:
430
431=over
432
433=item *
434
435You must have a project base class, which you load, and
436which you then use as the first argument in
437the call to YourProjClass->get_handle(...). It should derive
438(whether directly or indirectly) from Locale::Maketext.
439It B<doesn't matter> how you name this class, altho assuming this
440is the localization component of your Super Mega Program,
441good names for your project class might be
442SuperMegaProgram::Localization, SuperMegaProgram::L10N,
443SuperMegaProgram::I18N, SuperMegaProgram::International,
444or even SuperMegaProgram::Languages or SuperMegaProgram::Messages.
445
446=item *
447
448Language classes are what YourProjClass->get_handle will try to load.
449It will look for them by taking each language-tag (B<skipping> it
450if it doesn't look like a language-tag or locale-tag!), turning it to
451all lowercase, turning and dashes to underscores, and appending it
452to YourProjClass . "::". So this:
453
454 $lh = YourProjClass->get_handle(
455 'en-US', 'fr', 'kon', 'i-klingon', 'i-klingon-romanized'
456 );
457
458will try loading the classes
459YourProjClass::en_us (note lowercase!), YourProjClass::fr,
460YourProjClass::kon,
461YourProjClass::i_klingon
462and YourProjClass::i_klingon_romanized. (And it'll stop at the
463first one that actually loads.)
464
465=item *
466
467I assume that each language class derives (directly or indirectly)
468from your project class, and also defines its @ISA, its %Lexicon,
469or both. But I anticipate no dire consequences if these assumptions
470do not hold.
471
472=item *
473
474Language classes may derive from other language classes (altho they
475should have "use I<Thatclassname>" or "use base qw(I<...classes...>)").
476They may derive from the project
477class. They may derive from some other class altogether. Or via
478multiple inheritance, it may derive from any mixture of these.
479
480=item *
481
482I foresee no problems with having multiple inheritance in
483your hierarchy of language classes. (As usual, however, Perl will
484complain bitterly if you have a cycle in the hierarchy: i.e., if
485any class is its own ancestor.)
486
487=back
488
489=head1 ENTRIES IN EACH LEXICON
490
491A typical %Lexicon entry is meant to signify a phrase,
492taking some number (0 or more) of parameters. An entry
493is meant to be accessed by via
494a string I<key> in $lh->maketext(I<key>, ...parameters...),
495which should return a string that is generally meant for
496be used for "output" to the user -- regardless of whether
497this actually means printing to STDOUT, writing to a file,
498or putting into a GUI widget.
499
500While the key must be a string value (since that's a basic
501restriction that Perl places on hash keys), the value in
502the lexicon can currenly be of several types:
503a defined scalar, scalarref, or coderef. The use of these is
504explained above, in the section 'The "maketext" Method', and
505Bracket Notation for strings is discussed in the next section.
506
507While you can use arbitrary unique IDs for lexicon keys
508(like "_min_larger_max_error"), it is often
509useful for if an entry's key is itself a valid value, like
510this example error message:
511
512 "Minimum ([_1]) is larger than maximum ([_2])!\n",
513
514Compare this code that uses an arbitrary ID...
515
516 die $lh->maketext( "_min_larger_max_error", $min, $max )
517 if $min > $max;
518
519...to this code that uses a key-as-value:
520
521 die $lh->maketext(
522 "Minimum ([_1]) is larger than maximum ([_2])!\n",
523 $min, $max
524 ) if $min > $max;
525
526The second is, in short, more readable. In particular, it's obvious
527that the number of parameters you're feeding to that phrase (two) is
528the number of parameters that it I<wants> to be fed. (Since you see
529_1 and a _2 being used in the key there.)
530
531Also, once a project is otherwise
532complete and you start to localize it, you can scrape together
533all the various keys you use, and pass it to a translator; and then
534the translator's work will go faster if what he's presented is this:
535
536 "Minimum ([_1]) is larger than maximum ([_2])!\n",
537 => "", # fill in something here, Jacques!
538
539rather than this more cryptic mess:
540
541 "_min_larger_max_error"
542 => "", # fill in something here, Jacques
543
544I think that keys as lexicon values makes the completed lexicon
545entries more readable:
546
547 "Minimum ([_1]) is larger than maximum ([_2])!\n",
548 => "Le minimum ([_1]) est plus grand que le maximum ([_2])!\n",
549
550Also, having valid values as keys becomes very useful if you set
551up an _AUTO lexicon. _AUTO lexicons are discussed in a later
552section.
553
554I almost always use keys that are themselves
555valid lexicon values. One notable exception is when the value is
556quite long. For example, to get the screenful of data that
557a command-line program might returns when given an unknown switch,
558I often just use a key "_USAGE_MESSAGE". At that point I then go
559and immediately to define that lexicon entry in the
560ProjectClass::L10N::en lexicon (since English is always my "project
561lanuage"):
562
563 '_USAGE_MESSAGE' => <<'EOSTUFF',
564 ...long long message...
565 EOSTUFF
566
567and then I can use it as:
568
569 getopt('oDI', \%opts) or die $lh->maketext('_USAGE_MESSAGE');
570
571Incidentally,
572note that each class's C<%Lexicon> inherits-and-extends
573the lexicons in its superclasses. This is not because these are
574special hashes I<per se>, but because you access them via the
575C<maketext> method, which looks for entries across all the
576C<%Lexicon>'s in a language class I<and> all its ancestor classes.
577(This is because the idea of "class data" isn't directly implemented
578in Perl, but is instead left to individual class-systems to implement
579as they see fit..)
580
581Note that you may have things stored in a lexicon
582besides just phrases for output: for example, if your program
583takes input from the keyboard, asking a "(Y/N)" question,
584you probably need to know what equivalent of "Y[es]/N[o]" is
585in whatever language. You probably also need to know what
586the equivalents of the answers "y" and "n" are. You can
587store that information in the lexicon (say, under the keys
588"~answer_y" and "~answer_n", and the long forms as
589"~answer_yes" and "~answer_no", where "~" is just an ad-hoc
590character meant to indicate to programmers/translators that
591these are not phrases for output).
592
593Or instead of storing this in the language class's lexicon,
594you can (and, in some cases, really should) represent the same bit
595of knowledge as code is a method in the language class. (That
596leaves a tidy distinction between the lexicon as the things we
597know how to I<say>, and the rest of the things in the lexicon class
598as things that we know how to I<do>.) Consider
599this example of a processor for responses to French "oui/non"
600questions:
601
602 sub y_or_n {
603 return undef unless defined $_[1] and length $_[1];
604 my $answer = lc $_[1]; # smash case
605 return 1 if $answer eq 'o' or $answer eq 'oui';
606 return 0 if $answer eq 'n' or $answer eq 'non';
607 return undef;
608 }
609
610...which you'd then call in a construct like this:
611
612 my $response;
613 until(defined $response) {
614 print $lh->maketext("Open the pod bay door (y/n)? ");
615 $response = $lh->y_or_n( get_input_from_keyboard_somehow() );
616 }
617 if($response) { $pod_bay_door->open() }
618 else { $pod_bay_door->leave_closed() }
619
620Other data worth storing in a lexicon might be things like
621filenames for language-targetted resources:
622
623 ...
624 "_main_splash_png"
625 => "/styles/en_us/main_splash.png",
626 "_main_splash_imagemap"
627 => "/styles/en_us/main_splash.incl",
628 "_general_graphics_path"
629 => "/styles/en_us/",
630 "_alert_sound"
631 => "/styles/en_us/hey_there.wav",
632 "_forward_icon"
633 => "left_arrow.png",
634 "_backward_icon"
635 => "right_arrow.png",
636 # In some other languages, left equals
637 # BACKwards, and right is FOREwards.
638 ...
639
640You might want to do the same thing for expressing key bindings
641or the like (since hardwiring "q" as the binding for the function
642that quits a screen/menu/program is useful only if your language
643happens to associate "q" with "quit"!)
644
645=head1 BRACKET NOTATION
646
647Bracket Notation is a crucial feature of Locale::Maketext. I mean
648Bracket Notation to provide a replacement for sprintf formatting.
649Everything you do with Bracket Notation could be done with a sub block,
650but bracket notation is meant to be much more concise.
651
652Bracket Notation is a like a miniature "template" system (in the sense
653of L<Text::Template|Text::Template>, not in the sense of C++ templates),
654where normal text is passed thru basically as is, but text is special
655regions is specially interpreted. In Bracket Notation, you use brackets
656("[...]" -- not "{...}"!) to note sections that are specially interpreted.
657
658For example, here all the areas that are taken literally are underlined with
659a "^", and all the in-bracket special regions are underlined with an X:
660
661 "Minimum ([_1]) is larger than maximum ([_2])!\n",
662 ^^^^^^^^^ XX ^^^^^^^^^^^^^^^^^^^^^^^^^^ XX ^^^^
663
664When that string is compiled from bracket notation into a real Perl sub,
665it's basically turned into:
666
667 sub {
668 my $lh = $_[0];
669 my @params = @_;
670 return join '',
671 "Minimum (",
672 ...some code here...
673 ") is larger than maximum (",
674 ...some code here...
675 ")!\n",
676 }
677 # to be called by $lh->maketext(KEY, params...)
678
679In other words, text outside bracket groups is turned into string
680literals. Text in brackets is rather more complex, and currently follows
681these rules:
682
683=over
684
685=item *
686
687Bracket groups that are empty, or which consist only of whitespace,
688are ignored. (Examples: "[]", "[ ]", or a [ and a ] with returns
689and/or tabs and/or spaces between them.
690
691Otherwise, each group is taken to be a comma-separated group of items,
692and each item is interpreted as follows:
693
694=item *
695
696An item that is "_I<digits>" or "_-I<digits>" is interpreted as
697$_[I<value>]. I.e., "_1" is becomes with $_[1], and "_-3" is interpreted
698as $_[-3] (in which case @_ should have at least three elements in it).
699Note that $_[0] is the language handle, and is typically not named
700directly.
701
702=item *
703
704An item "_*" is interpreted to mean "all of @_ except $_[0]".
705I.e., C<@_[1..$#_]>. Note that this is an empty list in the case
706of calls like $lh->maketext(I<key>) where there are no
707parameters (except $_[0], the language handle).
708
709=item *
710
711Otherwise, each item is interpreted as a string literal.
712
713=back
714
715The group as a whole is interpreted as follows:
716
717=over
718
719=item *
720
721If the first item in a bracket group looks like a method name,
722then that group is interpreted like this:
723
724 $lh->that_method_name(
725 ...rest of items in this group...
726 ),
727
728=item *
729
730If the first item in a bracket group is empty-string, or "_*"
731or "_I<digits>" or "_-I<digits>", then that group is interpreted
732as just the interpolation of all its items:
733
734 join('',
735 ...rest of items in this group...
736 ),
737
738Examples: "[_1]" and "[,_1]", which are synonymous; and
739"[,ID-(,_4,-,_2,)]", which compiles as
740C<join "", "ID-(", $_[4], "-", $_[2], ")">.
741
742=item *
743
744Otherwise this bracket group is invalid. For example, in the group
745"[!@#,whatever]", the first item C<"!@#"> is neither empty-string,
746"_I<number>", "_-I<number>", "_*", nor a valid method name; and so
747Locale::Maketext will throw an exception of you try compiling an
748expression containing this bracket group.
749
750=back
751
752Note, incidentally, that items in each group are comma-separated,
753not C</\s*,\s*/>-separated. That is, you might expect that this
754bracket group:
755
756 "Hoohah [foo, _1 , bar ,baz]!"
757
758would compile to this:
759
760 sub {
761 my $lh = $_[0];
762 return join '',
763 "Hoohah ",
764 $lh->foo( $_[1], "bar", "baz"),
765 "!",
766 }
767
768But it actually compiles as this:
769
770 sub {
771 my $lh = $_[0];
772 return join '',
773 "Hoohah ",
774 $lh->foo(" _1 ", " bar ", "baz"), #!!!
775 "!",
776 }
777
778In the notation discussed so far, the characters "[" and "]" are given
779special meaning, for opening and closing bracket groups, and "," has
780a special meaning inside bracket groups, where it separates items in the
781group. This begs the question of how you'd express a literal "[" or
782"]" in a Bracket Notation string, and how you'd express a literal
783comma inside a bracket group. For this purpose I've adopted "~" (tilde)
784as an escape character: "~[" means a literal '[' character anywhere
785in Bracket Notation (i.e., regardless of whether you're in a bracket
786group or not), and ditto for "~]" meaning a literal ']', and "~," meaning
787a literal comma. (Altho "," means a literal comma outside of
788bracket groups -- it's only inside bracket groups that commas are special.)
789
790And on the off chance you need a literal tilde in a bracket expression,
791you get it with "~~".
792
793Currently, an unescaped "~" before a character
794other than a bracket or a comma is taken to mean just a "~" and that
795charecter. I.e., "~X" means the same as "~~X" -- i.e., one literal tilde,
796and then one literal "X". However, by using "~X", you are assuming that
797no future version of Maketext will use "~X" as a magic escape sequence.
798In practice this is not a great problem, since first off you can just
799write "~~X" and not worry about it; second off, I doubt I'll add lots
800of new magic characters to bracket notation; and third off, you
801aren't likely to want literal "~" characters in your messages anyway,
802since it's not a character with wide use in natural language text.
803
804Brackets must be balanced -- every openbracket must have
805one matching closebracket, and vice versa. So these are all B<invalid>:
806
807 "I ate [quant,_1,rhubarb pie."
808 "I ate [quant,_1,rhubarb pie[."
809 "I ate quant,_1,rhubarb pie]."
810 "I ate quant,_1,rhubarb pie[."
811
812Currently, bracket groups do not nest. That is, you B<cannot> say:
813
814 "Foo [bar,baz,[quux,quuux]]\n";
815
816If you need a notation that's that powerful, use normal Perl:
817
818 %Lexicon = (
819 ...
820 "some_key" => sub {
821 my $lh = $_[0];
822 join '',
823 "Foo ",
824 $lh->bar('baz', $lh->quux('quuux')),
825 "\n",
826 },
827 ...
828 );
829
830Or write the "bar" method so you don't need to pass it the
831output from calling quux.
832
833I do not anticipate that you will need (or particularly want)
834to nest bracket groups, but you are welcome to email me with
835convincing (real-life) arguments to the contrary.
836
837=head1 AUTO LEXICONS
838
839If maketext goes to look in an individual %Lexicon for an entry
840for I<key> (where I<key> does not start with an underscore), and
841sees none, B<but does see> an entry of "_AUTO" => I<some_true_value>,
842then we actually define $Lexicon{I<key>} = I<key> right then and there,
843and then use that value as if it had been there all
844along. This happens before we even look in any superclass %Lexicons!
845
846(This is meant to be somewhat like the AUTOLOAD mechanism in
847Perl's function call system -- or, looked at another way,
848like the L<AutoLoader|AutoLoader> module.)
849
850I can picture all sorts of circumstances where you just
851do not want lookup to be able to fail (since failing
852normally means that maketext throws a C<die>, altho
853see the next section for greater control over that). But
854here's one circumstance where _AUTO lexicons are meant to
855be I<especially> useful:
856
857As you're writing an application, you decide as you go what messages
858you need to emit. Normally you'd go to write this:
859
860 if(-e $filename) {
861 go_process_file($filename)
862 } else {
863 print "Couldn't find file \"$filename\"!\n";
864 }
865
866but since you anticipate localizing this, you write:
867
868 use ThisProject::I18N;
869 my $lh = ThisProject::I18N->get_handle();
870 # For the moment, assume that things are set up so
871 # that we load class ThisProject::I18N::en
872 # and that that's the class that $lh belongs to.
873 ...
874 if(-e $filename) {
875 go_process_file($filename)
876 } else {
877 print $lh->maketext(
878 "Couldn't find file \"[_1]\"!\n", $filename
879 );
880 }
881
882Now, right after you've just written the above lines, you'd
883normally have to go open the file
884ThisProject/I18N/en.pm, and immediately add an entry:
885
886 "Couldn't find file \"[_1]\"!\n"
887 => "Couldn't find file \"[_1]\"!\n",
888
889But I consider that somewhat of a distraction from the work
890of getting the main code working -- to say nothing of the fact
891that I often have to play with the program a few times before
892I can decide exactly what wording I want in the messages (which
893in this case would require me to go changing three lines of code:
894the call to maketext with that key, and then the two lines in
895ThisProject/I18N/en.pm).
896
897However, if you set "_AUTO => 1" in the %Lexicon in,
898ThisProject/I18N/en.pm (assuming that English (en) is
899the language that all your programmers will be using for this
900project's internal message keys), then you don't ever have to
901go adding lines like this
902
903 "Couldn't find file \"[_1]\"!\n"
904 => "Couldn't find file \"[_1]\"!\n",
905
906to ThisProject/I18N/en.pm, because if _AUTO is true there,
907then just looking for an entry with the key "Couldn't find
908file \"[_1]\"!\n" in that lexicon will cause it to be added,
909with that value!
910
911Note that the reason that keys that start with "_"
912are immune to _AUTO isn't anything generally magical about
913the underscore character -- I just wanted a way to have most
914lexicon keys be autoable, except for possibly a few, and I
915arbitrarily decided to use a leading underscore as a signal
916to distinguish those few.
917
918=head1 CONTROLLING LOOKUP FAILURE
919
920If you call $lh->maketext(I<key>, ...parameters...),
921and there's no entry I<key> in $lh's class's %Lexicon, nor
922in the superclass %Lexicon hash, I<and> if we can't auto-make
923I<key> (because either it starts with a "_", or because none
924of its lexicons have C<_AUTO =E<gt> 1,>), then we have
925failed to find a normal way to maketext I<key>. What then
926happens in these failure conditions, depends on the $lh object
927"fail" attribute.
928
929If the language handle has no "fail" attribute, maketext
930will simply throw an exception (i.e., it calls C<die>, mentioning
931the I<key> whose lookup failed, and naming the line number where
932the calling $lh->maketext(I<key>,...) was.
933
934If the language handle has a "fail" attribute whose value is a
935coderef, then $lh->maketext(I<key>,...params...) gives up and calls:
936
937 return &{$that_subref}($lh, $key, @params);
938
939Otherwise, the "fail" attribute's value should be a string denoting
940a method name, so that $lh->maketext(I<key>,...params...) can
941give up with:
942
943 return $lh->$that_method_name($phrase, @params);
944
945The "fail" attribute can be accessed with the C<fail_with> method:
946
947 # Set to a coderef:
948 $lh->fail_with( \&failure_handler );
949
950 # Set to a method name:
951 $lh->fail_with( 'failure_method' );
952
953 # Set to nothing (i.e., so failure throws a plain exception)
954 $lh->fail_with( undef );
955
956 # Simply read:
957 $handler = $lh->fail_with();
958
959Now, as to what you may want to do with these handlers: Maybe you'd
960want to log what key failed for what class, and then die. Maybe
961you don't like C<die> and instead you want to send the error message
962to STDOUT (or wherever) and then merely C<exit()>.
963
964Or maybe you don't want to C<die> at all! Maybe you could use a
965handler like this:
966
967 # Make all lookups fall back onto an English value,
968 # but after we log it for later fingerpointing.
969 my $lh_backup = ThisProject->get_handle('en');
970 open(LEX_FAIL_LOG, ">>wherever/lex.log") || die "GNAARGH $!";
971 sub lex_fail {
972 my($failing_lh, $key, $params) = @_;
973 print LEX_FAIL_LOG scalar(localtime), "\t",
974 ref($failing_lh), "\t", $key, "\n";
975 return $lh_backup->maketext($key,@params);
976 }
977
978Some users have expressed that they think this whole mechanism of
979having a "fail" attribute at all, seems a rather pointless complication.
980But I want Locale::Maketext to be usable for software projects of I<any>
981scale and type; and different software projects have different ideas
982of what the right thing is to do in failure conditions. I could simply
983say that failure always throws an exception, and that if you want to be
984careful, you'll just have to wrap every call to $lh->maketext in an
985S<eval { }>. However, I want programmers to reserve the right (via
986the "fail" attribute) to treat lookup failure as something other than
987an exception of the same level of severity as a config file being
988unreadable, or some essential resource being inaccessable.
989
990One possibly useful value for the "fail" attribute is the method name
991"failure_handler_auto". This is a method defined in class
992Locale::Maketext itself. You set it with:
993
994 $lh->fail_with('failure_handler_auto');
995
996Then when you call $lh->maketext(I<key>, ...parameters...) and
997there's no I<key> in any of those lexicons, maketext gives up with
998
999 return $lh->failure_handler_auto($key, @params);
1000
1001But failure_handler_auto, instead of dying or anything, compiles
1002$key, caching it in $lh->{'failure_lex'}{$key} = $complied,
1003and then calls the compiled value, and returns that. (I.e., if
1004$key looks like bracket notation, $compiled is a sub, and we return
1005&{$compiled}(@params); but if $key is just a plain string, we just
1006return that.)
1007
1008The effect of using "failure_auto_handler"
1009is like an AUTO lexicon, except that it 1) compiles $key even if
1010it starts with "_", and 2) you have a record in the new hashref
1011$lh->{'failure_lex'} of all the keys that have failed for
1012this object. This should avoid your program dying -- as long
1013as your keys aren't actually invalid as bracket code, and as
1014long as they don't try calling methods that don't exist.
1015
1016"failure_auto_handler" may not be exactly what you want, but I
1017hope it at least shows you that maketext failure can be mitigated
1018in any number of very flexible ways. If you can formalize exactly
1019what you want, you should be able to express that as a failure
1020handler. You can even make it default for every object of a given
1021class, by setting it in that class's init:
1022
1023 sub init {
1024 my $lh = $_[0]; # a newborn handle
1025 $lh->SUPER::init();
1026 $lh->fail_with('my_clever_failure_handler');
1027 return;
1028 }
1029 sub my_clever_failure_handler {
1030 ...you clever things here...
1031 }
1032
1033=head1 HOW TO USE MAKETEXT
1034
1035Here is a brief checklist on how to use Maketext to localize
1036applications:
1037
1038=over
1039
1040=item *
1041
1042Decide what system you'll use for lexicon keys. If you insist,
1043you can use opaque IDs (if you're nostalgic for C<catgets>),
1044but I have better suggestions in the
1045section "Entries in Each Lexicon", above. Assuming you opt for
1046meaningful keys that double as values (like "Minimum ([_1]) is
1047larger than maximum ([_2])!\n"), you'll have to settle on what
1048language those should be in. For the sake of argument, I'll
1049call this English, specifically American English, "en-US".
1050
1051=item *
1052
1053Create a class for your localization project. This is
1054the name of the class that you'll use in the idiom:
1055
1056 use Projname::L10N;
1057 my $lh = Projname::L10N->get_handle(...) || die "Language?";
1058
1059Assuming your call your class Projname::L10N, create a class
1060consisting minimally of:
1061
1062 package Projname::L10N;
1063 use base qw(Locale::Maketext);
1064 ...any methods you might want all your languages to share...
1065
1066 # And, assuming you want the base class to be an _AUTO lexicon,
1067 # as is discussed a few sections up:
1068
1069 1;
1070
1071=item *
1072
1073Create a class for the language your internal keys are in. Name
1074the class after the language-tag for that language, in lowercase,
1075with dashes changed to underscores. Assuming your project's first
1076language is US English, you should call this Projname::L10N::en_us.
1077It should consist minimally of:
1078
1079 package Projname::L10N::en_us;
1080 use base qw(Projname::L10N);
1081 %Lexicon = (
1082 '_AUTO' => 1,
1083 );
1084 1;
1085
1086(For the rest of this section, I'll assume that this "first
1087language class" of Projname::L10N::en_us has
1088_AUTO lexicon.)
1089
1090=item *
1091
1092Go and write your program. Everywhere in your program where
1093you would say:
1094
1095 print "Foobar $thing stuff\n";
1096
1097instead do it thru maketext, using no variable interpolation in
1098the key:
1099
1100 print $lh->maketext("Foobar [_1] stuff\n", $thing);
1101
1102If you get tired of constantly saying C<print $lh-E<gt>maketext>,
1103consider making a functional wrapper for it, like so:
1104
1105 use Projname::L10N;
1106 use vars qw($lh);
1107 $lh = Projname::L10N->get_handle(...) || die "Language?";
1108 sub pmt (@) { print( $lh->maketext(@_)) }
1109 # "pmt" is short for "Print MakeText"
1110 $Carp::Verbose = 1;
1111 # so if maketext fails, we see made the call to pmt
1112
1113Besides whole phrases meant for output, anything language-dependent
1114should be put into the class Projname::L10N::en_us,
1115whether as methods, or as lexicon entries -- this is discussed
1116in the section "Entries in Each Lexicon", above.
1117
1118=item *
1119
1120Once the program is otherwise done, and once its localization for
1121the first language works right (via the data and methods in
1122Projname::L10N::en_us), you can get together the data for translation.
1123If your first language lexicon isn't an _AUTO lexicon, then you already
1124have all the messages explicitly in the lexicon (or else you'd be
1125getting exceptions thrown when you call $lh->maketext to get
1126messages that aren't in there). But if you were (advisedly) lazy and are
1127using an _AUTO lexicon, then you've got to make a list of all the phrases
1128that you've so far been letting _AUTO generate for you. There are very
1129many ways to assemble such a list. The most straightforward is to simply
1130grep the source for every occurrence of "maketext" (or calls
1131to wrappers around it, like the above C<pmt> function), and to log the
1132following phrase.
1133
1134=item *
1135
1136You may at this point want to consider whether the your base class
1137(Projname::L10N) that all lexicons inherit from (Projname::L10N::en,
1138Projname::L10N::es, etc.) should be an _AUTO lexicon. It may be true
1139that in theory, all needed messages will be in each language class;
1140but in the presumably unlikely or "impossible" case of lookup failure,
1141you should consider whether your program should throw an exception,
1142emit text in English (or whatever your project's first language is),
1143or some more complex solution as described in the section
1144"Controlling Lookup Failure", above.
1145
1146=item *
1147
1148Submit all messages/phrases/etc. to translators.
1149
1150(You may, in fact, want to start with localizing to I<one> other language
1151at first, if you're not sure that you've property abstracted the
1152language-dependent parts of your code.)
1153
1154Translators may request clarification of the situation in which a
1155particular phrase is found. For example, in English we are entirely happy
1156saying "I<n> files found", regardless of whether we mean "I looked for files,
1157and found I<n> of them" or the rather distinct situation of "I looked for
1158something else (like lines in files), and along the way I saw I<n>
1159files." This may involve rethinking things that you thought quite clear:
1160should "Edit" on a toolbar be a noun ("editing") or a verb ("to edit")? Is
1161there already a conventionalized way to express that menu option, separate
1162from the target language's normal word for "to edit"?
1163
1164In all cases where the very common phenomenon of quantification
1165(saying "I<N> files", for B<any> value of N)
1166is involved, each translator should make clear what dependencies the
1167number causes in the sentence. In many cases, dependency is
1168limited to words adjacent to the number, in places where you might
1169expect them ("I found the-?PLURAL I<N>
1170empty-?PLURAL directory-?PLURAL"), but in some cases there are
1171unexpected dependencies ("I found-?PLURAL ..."!) as well as long-distance
1172dependencies "The I<N> directory-?PLURAL could not be deleted-?PLURAL"!).
1173
1174Remind the translators to consider the case where N is 0:
1175"0 files found" isn't exactly natural-sounding in any language, but it
1176may be unacceptable in many -- or it may condition special
1177kinds of agreement (similar to English "I didN'T find ANY files").
1178
1179Remember to ask your translators about numeral formatting in their
1180language, so that you can override the C<numf> method as
1181appropriate. Typical variables in number formatting are: what to
1182use as a decimal point (comma? period?); what to use as a thousands
1183separator (space? nonbreakinng space? comma? period? small
1184middot? prime? apostrophe?); and even whether the so-called "thousands
1185separator" is actually for every third digit -- I've heard reports of
1186two hundred thousand being expressable as "2,00,000" for some Indian
1187(Subcontinental) languages, besides the less surprising "S<200 000>",
1188"200.000", "200,000", and "200'000". Also, using a set of numeral
1189glyphs other than the usual ASCII "0"-"9" might be appreciated, as via
1190C<tr/0-9/\x{0966}-\x{096F}/> for getting digits in Devanagari script
1191(for Hindi, Konkani, others).
1192
1193The basic C<quant> method that Locale::Maketext provides should be
1194good for many languages. For some languages, it might be useful
1195to modify it (or its constituent C<numerate> method)
1196to take a plural form in the two-argument call to C<quant>
1197(as in "[quant,_1,files]") if
1198it's all-around easier to infer the singular form from the plural, than
1199to infer the plural form from the singular.
1200
1201But for other languages (as is discussed at length
1202in L<Locale::Maketext::TPJ13|Locale::Maketext::TPJ13>), simple
1203C<quant>/C<numerify> is not enough. For the particularly problematic
1204Slavic languages, what you may need is a method which you provide
1205with the number, the citation form of the noun to quantify, and
1206the case and gender that the sentence's syntax projects onto that
1207noun slot. The method would then be responsible for determining
1208what grammatical number that numeral projects onto its noun phrase,
1209and what case and gender it may override the normal case and gender
1210with; and then it would look up the noun in a lexicon providing
1211all needed inflected forms.
1212
1213=item *
1214
1215You may also wish to discuss with the translators the question of
1216how to relate different subforms of the same language tag,
1217considering how this reacts with C<get_handle>'s treatment of
1218these. For example, if a user accepts interfaces in "en, fr", and
1219you have interfaces available in "en-US" and "fr", what should
1220they get? You may wish to resolve this by establishing that "en"
1221and "en-US" are effectively synonymous, by having one class
1222zero-derive from the other.
1223
1224For some languages this issue may never come up (Danish is rarely
1225expressed as "da-DK", but instead is just "da"). And for other
1226languages, the whole concept of a "generic" form may verge on
1227being uselessly vague, particularly for interfaces involving voice
1228media in forms of Arabic or Chinese.
1229
1230=item *
1231
1232Once you've localized your program/site/etc. for all desired
1233languages, be sure to show the result (whether live, or via
1234screenshots) to the translators. Once they approve, make every
1235effort to have it then checked by at least one other speaker of
1236that language. This holds true even when (or especially when) the
1237translation is done by one of your own programmers. Some
1238kinds of systems may be harder to find testers for than others,
1239depending on the amount of domain-specific jargon and concepts
1240involved -- it's easier to find people who can tell you whether
1241they approve of your translation for "delete this message" in an
1242email-via-Web interface, than to find people who can give you
1243an informed opinion on your translation for "attribute value"
1244in an XML query tool's interface.
1245
1246=back
1247
1248=head1 SEE ALSO
1249
1250I recommend reading all of these:
1251
1252L<Locale::Maketext::TPJ13|Locale::Maketext::TPJ13> -- my I<The Perl
1253Journal> article about Maketext. It explains many important concepts
1254underlying Locale::Maketext's design, and some insight into why
1255Maketext is better than the plain old approach of just having
1256message catalogs that are just databases of sprintf formats.
1257
1258L<File::Findgrep|File::Findgrep> is a sample application/module
1259that uses Locale::Maketext to localize its messages.
1260
1261L<I18N::LangTags|I18N::LangTags>.
1262
1263L<Win32::Locale|Win32::Locale>.
1264
1265RFC 3066, I<Tags for the Identification of Languages>,
1266as at http://sunsite.dk/RFC/rfc/rfc3066.html
1267
1268RFC 2277, I<IETF Policy on Character Sets and Languages>
1269is at http://sunsite.dk/RFC/rfc/rfc2277.html -- much of it is
1270just things of interest to protocol designers, but it explains
1271some basic concepts, like the distinction between locales and
1272language-tags.
1273
1274The manual for GNU C<gettext>. The gettext dist is available in
1275C<ftp://prep.ai.mit.edu/pub/gnu/> -- get
1276a recent gettext tarball and look in its "doc/" directory, there's
1277an easily browsable HTML version in there. The
1278gettext documentation asks lots of questions worth thinking
1279about, even if some of their answers are sometimes wonky,
1280particularly where they start talking about pluralization.
1281
1282The Locale/Maketext.pm source. Obverse that the module is much
1283shorter than its documentation!
1284
1285=head1 COPYRIGHT AND DISCLAIMER
1286
1287Copyright (c) 1999-2001 Sean M. Burke. All rights reserved.
1288
1289This library is free software; you can redistribute it and/or modify
1290it under the same terms as Perl itself.
1291
1292This program is distributed in the hope that it will be useful, but
1293without any warranty; without even the implied warranty of
1294merchantability or fitness for a particular purpose.
1295
1296=head1 AUTHOR
1297
1298Sean M. Burke C<sburke@cpan.org>
1299
1300=cut
1301
1302# Zing!