package Encode::Encoding; # Base class for classes which implement encodings use strict; our $VERSION = do { my @r = (q$Revision: 1.25 $ =~ /\d+/g); sprintf "%d."."%02d" x $#r, @r }; sub Define { my $obj = shift; my $canonical = shift; $obj = bless { Name => $canonical },$obj unless ref $obj; # warn "$canonical => $obj\n"; Encode::define_encoding($obj, $canonical, @_); } sub name { shift->{'Name'} } # Temporary legacy methods sub toUnicode { shift->decode(@_) } sub fromUnicode { shift->encode(@_) } sub new_sequence { return $_[0] } sub needs_lines { 0 } sub DESTROY {} 1; __END__ =head1 NAME Encode::Encoding - Encode Implementation Base Class =head1 SYNOPSIS package Encode::MyEncoding; use base qw(Encode::Encoding); __PACKAGE__->Define(qw(myCanonical myAlias)); =head1 DESCRIPTION As mentioned in L, encodings are (in the current implementation at least) defined by objects. The mapping of encoding name to object is via the C<%encodings> hash. The values of the hash can currently be either strings or objects. The string form may go away in the future. The string form occurs when C has scanned C<@INC> for loadable encodings but has not actually loaded the encoding in question. This is because the current "loading" process is all Perl and a bit slow. Once an encoding is loaded then value of the hash is object which implements the encoding. The object should provide the following interface: =over 4 =item -Ename Should return the string representing the canonical name of the encoding. =item -Enew_sequence This is a placeholder for encodings with state. It should return an object which implements this interface, all current implementations return the original object. =item -Eencode($string,$check) Should return the octet sequence representing I<$string>. If I<$check> is true it should modify I<$string> in place to remove the converted part (i.e. the whole string unless there is an error). If an error occurs it should return the octet sequence for the fragment of string that has been converted, and modify $string in-place to remove the converted part leaving it starting with the problem fragment. If check is is false then C should make a "best effort" to convert the string - for example by using a replacement character. =item -Edecode($octets,$check) Should return the string that I<$octets> represents. If I<$check> is true it should modify I<$octets> in place to remove the converted part (i.e. the whole sequence unless there is an error). If an error occurs it should return the fragment of string that has been converted, and modify $octets in-place to remove the converted part leaving it starting with the problem fragment. If check is is false then C should make a "best effort" to convert the string - for example by using Unicode's "\x{FFFD}" as a replacement character. =back It should be noted that the check behaviour is different from the outer public API. The logic is that the "unchecked" case is useful when encoding is part of a stream which may be reporting errors (e.g. STDERR). In such cases it is desirable to get everything through somehow without causing additional errors which obscure the original one. Also the encoding is best placed to know what the correct replacement character is, so if that is the desired behaviour then letting low level code do it is the most efficient. In contrast if check is true, the scheme above allows the encoding to do as much as it can and tell layer above how much that was. What is lacking at present is a mechanism to report what went wrong. The most likely interface will be an additional method call to the object, or perhaps (to avoid forcing per-stream objects on otherwise stateless encodings) and additional parameter. It is also highly desirable that encoding classes inherit from C as a base class. This allows that class to define additional behaviour for all encoding objects. For example built in Unicode, UCS-2 and UTF-8 classes use : package Encode::MyEncoding; use base qw(Encode::Encoding); __PACKAGE__->Define(qw(myCanonical myAlias)); To create an object with bless {Name => ...},$class, and call define_encoding. They inherit their C method from C. =head2 Compiled Encodings For the sake of speed and efficiency, Most of the encodings are now supported via I that are XS modules generated from UCM files. Encode provides enc2xs tool to achieve that. Please see L for more details. =head1 SEE ALSO L, L =for future =over 4 =item Scheme 1 Passed remaining fragment of string being processed. Modifies it in place to remove bytes/characters it can understand and returns a string used to represent them. e.g. sub fixup { my $ch = substr($_[0],0,1,''); return sprintf("\x{%02X}",ord($ch); } This scheme is close to how underlying C code for Encode works, but gives the fixup routine very little context. =item Scheme 2 Passed original string, and an index into it of the problem area, and output string so far. Appends what it will to output string and returns new index into original string. For example: sub fixup { # my ($s,$i,$d) = @_; my $ch = substr($_[0],$_[1],1); $_[2] .= sprintf("\x{%02X}",ord($ch); return $_[1]+1; } This scheme gives maximal control to the fixup routine but is more complicated to code, and may need internals of Encode to be tweaked to keep original string intact. =item Other Schemes Hybrids of above. Multiple return values rather than in-place modifications. Index into the string could be C allowing C. =back =cut