file begins with the two literal byte values 0xFE 0xFF, this is
the BOM for big-endian UTF-16. If the file begins with the two
literal byte value 0xFF 0xFE, this is the BOM for little-endian
-UTF-16. If the file begins with the three literal byte values
+UTF-16. On an ASCII platform, if the file begins with the three literal
+byte values
0xEF 0xBB 0xBF, this is the BOM for UTF-8.
+(A mechanism portable to EBCDIC platforms is to:
+
+ my $utf8_bom = "\x{FEFF}";
+ utf8::encode($utf8_bom);
=for comment
use bytes; print map sprintf(" 0x%02X", ord $_), split '', "\x{feff}";
=item *
-A naive but often sufficient heuristic for testing the first highbit
+A naive, but often sufficient heuristic on ASCII platforms, for testing
+the first highbit
byte-sequence in a BOM-less file (whether in code or in Pod!), to see
whether that sequence is valid as UTF-8 (RFC 2279) is to check whether
that the first byte in the sequence is in the range 0xC2 - 0xFD
0x80 - 0xBF. If so, the parser may conclude that this file is in
UTF-8, and all highbit sequences in the file should be assumed to
be UTF-8. Otherwise the parser should treat the file as being
-in CP-1252. (A better check is to pass a copy of the sequence to
+in CP-1252. (A better check, and which works on EBCDIC platforms as
+well, is to pass a copy of the sequence to
L<utf8::decode()|utf8> which performs a full validity check on the
sequence and returns TRUE if it is valid UTF-8, FALSE otherwise. This
function is always pre-loaded, is fast because it is written in C, and
=item *
-This document's requirements and suggestions about encodings
-do not apply to Pod processors running on non-ASCII platforms,
-notably EBCDIC platforms.
-
-=item *
-
Pod processors must treat a "=for [label] [content...]" paragraph as
meaning the same thing as a "=begin [label]" paragraph, content, and
an "=end [label]" paragraph. (The parser may conflate these two
When referring to characters by using a EE<lt>n> numeric code, numbers
in the range 32-126 refer to those well known US-ASCII characters (also
defined there by Unicode, with the same meaning), which all Pod
-formatters must render faithfully. Numbers in the ranges 0-31 and
-127-159 should not be used (neither as literals, nor as EE<lt>number>
-codes), except for the literal byte-sequences for newline (13, 13 10, or
-10), and tab (9).
+formatters must render faithfully. Characters whose EE<lt>E<gt> numbers
+are in the ranges 0-31 and 127-159 should not be used (neither as
+literals,
+nor as EE<lt>number> codes), except for the literal byte-sequences for
+newline (ASCII 13, ASCII 13 10, or ASCII 10), and tab (ASCII 9).
Numbers in the range 160-255 refer to Latin-1 characters (also
defined there by Unicode, with the same meaning). Numbers above