perldelta for f31006c

[perl5.git] / pod / perldelta.pod

=encoding utf8

=head1 NAME

[ this is a template for a new perldelta file.  Any text flagged as XXX needs
to be processed before release. ]

perldelta - what is new for perl v5.21.1

=head1 DESCRIPTION

This document describes differences between the 5.21.0 release and the 5.21.1
release.

If you are upgrading from an earlier release such as 5.20.0, first read
L<perl5210delta>, which describes differences between 5.20.0 and 5.21.0.

=head1 Notice

XXX Any important notices here

=head1 Core Enhancements

XXX New core language features go here.  Summarize user-visible core language
enhancements.  Particularly prominent performance optimisations could go
here, but most should go in the L</Performance Enhancements> section.

[ List each enhancement as a =head2 entry ]

=head2 Unicode 7.0 is now supported

For details on what is in this release, see
L<http://www.unicode.org/versions/Unicode7.0.0/>.

=head2 Experimental C Backtrace API

Starting from Perl 5.21.1, on some platforms Perl supports retrieving
the C level backtrace (similar to what symbolic debuggers like gdb do).

The backtrace returns the stack trace of the C call frames,
with the symbol names (function names), the object names (like "perl"),
and if it can, also the source code locations (file:line).

The supported platforms are Linux and OS X (some *BSD might work at
least partly, but they have not yet been tested).

The feature needs to be enabled with C<Configure -Dusecbacktrace>.

Also included is a C API to retrieve backtraces.

See L<perlhacktips/"C backtrace"> for more information.

=head2 C<qr/foo/x> now ignores any Unicode pattern white space

The C</x> regular expression modifier allows the pattern to contain
white space and comments, both of which are ignored, for improved
readability.  Until now, not all the white space characters that Unicode
designates for this purpose were handled.  The additional ones now
recognized are
U+0085 NEXT LINE,
U+200E LEFT-TO-RIGHT MARK,
U+200F RIGHT-TO-LEFT MARK,
U+2028 LINE SEPARATOR,
and
U+2029 PARAGRAPH SEPARATOR.

=head2 S<C<use locale>> can restrict which locale categories are affected

It is now possible to pass a parameter to S<C<use locale>> to specify
a subset of locale categories to be locale-aware, with the remaining
ones unaffected.  See L<perllocale/The "use locale" pragma> for details.

=head1 Security

XXX Any security-related notices go here.  In particular, any security
vulnerabilities closed should be noted here rather than in the
L</Selected Bug Fixes> section.

[ List each security issue as a =head2 entry ]

=head1 Incompatible Changes

XXX For a release on a stable branch, this section aspires to be:

    There are no changes intentionally incompatible with 5.XXX.XXX
    If any exist, they are bugs, and we request that you submit a
    report.  See L</Reporting Bugs> below.

=head2 S<C<"use encoding">> now is a fatal error

The C<encoding> pragma has been deprecated since v5.18, and its use now
is a fatal error.

=head2 C<\N{}> with a sequence of multiple spaces is now a fatal error.

This has been deprecated since v5.18.

=head2 In double-quotish C<\cI<X>>, I<X> must now be a printable ASCII character

In prior releases, failure to do this raised a deprecation warning.

=head2 Splitting the tokens C<(?> and C<(*> in regular expressions is
now a fatal compilation error.

These had been deprecated since v5.18.

=head2 5 additional characters are treated as white space under C</x> in
regex patterns (unless escaped)

The use of these characters with C</x> outside bracketed character
classes and when not preceeded by a backslash has raised a deprecation
warning since v5.18.  Now they will be ignored.  See L</qrE<sol>fooE<sol>x>
for the list of the five characters.

=head2 Comment lines within S<C<(?[ ])>> now are ended only by a C<\n>

S<C<(?[ ])>>  is an experimental feature, introduced in v5.18.  It operates
as if C</x> is always enabled.  But there was a difference, comment
lines (following a C<#> character) were terminated by anything matching
C<\R> which includes all vertical whitespace, such as form feeds.  For
consistency, this is now changed to match what terminates comment lines
outside S<C<(?[ ])>>, namely a C<\n> (even if escaped), which is the
same as what terminates a heredoc string and formats.

=head2 Omitting % and @ on hash and array names is no longer permitted

Really old Perl let you omit the @ on array names and the % on hash
names in some spots.  This has issued a deprecation warning since Perl
5.0, and is no longer permitted.

=head2 C<"$!"> text is now in English outside C<"use locale"> scope

Previously, the text, unlike almost everything else, always came out
based on the current underlying locale of the program.  (Also affected
on some systems is C<"$^E>".)  For programs that are unprepared to
handle locale, this can cause garbage text to be displayed.  It's better
to display text that is translatable via some tool than garbage text
which is much harder to figure out.

=head2 C<"$!"> text will be returned in UTF-8 when appropriate

The stringification of C<$!> and C<$^E> will have the UTF-8 flag set
when the text is actually non-ASCII UTF-8.  This will enable programs
that are set up to be locale-aware to properly output messages in the
user's native language.  Code that needs to continue the 5.20 and
earlier behavior can do the stringification within the scopes of both
'use bytes' and 'use locale ":messages".  No other Perl operations will
be affected by locale; only C<$!> and C<$^E> stringification.  The
'bytes' pragma causes the UTF-8 flag to not be set, just as in previous
Perl releases.  This resolves [perl #112208].

=head2 MAD build option has been removed

MAD = Misc Attribute Decoration; unmaintained attempt at preserving
the Perl parse tree more faithfully so that automatic conversion of
Perl 5 to Perl 6 would have been easier.

This build-time configuration option had been unmaintained for years,
and had probably seriously diverged on both Perl 5 and Perl 6 sides.

=head2 Support for C<?PATTERN?> without explicit operator has been removed

Starting regular expressions matching only once directly with the
question mark delimiter is now a syntax error, so that the question mark
can be available for use in new operators.  Write C<m?PATTERN?> instead,
explicitly using the C<m> operator: the question mark delimiter still
invokes match-once behaviour.

=head1 Deprecations

XXX Any deprecated features, syntax, modules etc. should be listed here.

=head2 Using a NO-BREAK space in a character alias for C<\N{...}> is now
deprecated

This non-graphic character is essentially indistinguishable from a
regular space, and so should not be allowed.  See
L<charnames/CUSTOM ALIASES>.

=head2 A literal C<"{"> should now be escaped in a pattern

If you want a literal left curly bracket (also called a left brace) in a
regular expression pattern, you should now escape it by either
preceding it with a backslash (C<"\{">) or enclosing it within square
brackets C<"[{]">, or by using C<\Q>; otherwise a deprecation warning
will be raised.  This was first announced as forthcoming in the v5.16
release; it will allow future extensions to the language to happen.

=head2 Module removals

XXX Remove this section if inapplicable.

The following modules will be removed from the core distribution in a
future release, and will at that time need to be installed from CPAN.
Distributions on CPAN which require these modules will need to list them as
prerequisites.

The core versions of these modules will now issue C<"deprecated">-category
warnings to alert you to this fact.  To silence these deprecation warnings,
install the modules in question from CPAN.

Note that these are (with rare exceptions) fine modules that you are encouraged
to continue to use.  Their disinclusion from core primarily hinges on their
necessity to bootstrapping a fully functional, CPAN-capable Perl installation,
not usually on concerns over their design.

=over

=item XXX

XXX Note that deprecated modules should be listed here even if they are listed
as an updated module in the L</Modules and Pragmata> section.

=back

[ List each other deprecation as a =head2 entry ]

=head1 Performance Enhancements

XXX Changes which enhance performance without changing behaviour go here.
There may well be none in a stable release.

[ List each enhancement as a =item entry ]

=over 4

=item *

Many internal functions have been refactored to improve performance and reduce
their memory footprints.

L<[perl #121436]|https://rt.perl.org/Ticket/Display.html?id=121436>
L<[perl #121906]|https://rt.perl.org/Ticket/Display.html?id=121906>
L<[perl #121969]|https://rt.perl.org/Ticket/Display.html?id=121969>

=item *

C<-T> and C<-B> filetests will return sooner when an empty file is detected.

L<perl #121489|https://rt.perl.org/Ticket/Display.html?id=121489>

=back

=head1 Modules and Pragmata

XXX All changes to installed files in F<cpan/>, F<dist/>, F<ext/> and F<lib/>
go here.  If Module::CoreList is updated, generate an initial draft of the
following sections using F<Porting/corelist-perldelta.pl>.  A paragraph summary
for important changes should then be added by hand.  In an ideal world,
dual-life modules would have a F<Changes> file that could be cribbed.

[ Within each section, list entries as a =item entry ]

=head2 New Modules and Pragmata

=over 4

=item *

XXX

=back

=head2 Updated Modules and Pragmata

=over 4

=item *

L<Carp> has been upgraded from version 1.3301 to 1.34.

Carp::Heavy now ignores version mismatches with Carp if Carp is newer
than 1.12, since Carp::Heavy's guts were merged into Carp at that
point.
L<[perl #121574]|https://rt.perl.org/Ticket/Display.html?id=121574>

=item *

L<Data::Dumper> has been upgraded from version 2.151 to 2.152.

Changes to resolve Coverity issues.

XS dumps incorrectly stored the name of code references stored in a
GLOB.
L<[perl #122070]|https://rt.perl.org/Ticket/Display.html?id=122070>

=item *

L<Encode> has been upgraded from version 2.60_01 to 2.62.

B<piconv> now has better error handling when the encoding name is nonexistent,
and a build breakage when upgrading L<Encode> in perl-5.8.2 and earlier has
been fixed.

=item *

L<Hash::Util> has been upgraded from version 0.16 to 0.17.

Minor bug fixes and documentation fixes to Hash::Util::hash_stats()

=item *

The libnet collection of modules has been upgraded from version 1.25 to 1.27.

There are only whitespace changes to the installed files.

=item *

The Locale-Codes collection of modules has been upgraded from vesion 3.30 to 3.31.

Fixed a bug in the scripts used to extract data from spreadsheets that
prevented the SHP currency code from being found.
L<[cpan #94229]|https://rt.cpan.org/Ticket/Display.html?id=94229>

=item *

L<Math::BigInt> has been upgraded from version 1.9993 to 1.9994.

Synchronize POD changes from the CPAN release.

C<< Math::BigFloat->blog(x) >> would sometimes return blog(2*x) when
the accuracy was greater than 70 digits.

The result of C<< Math::BigFloat->bdiv() >> in list context now
satisfies C<< x = quotient * divisor + remainder >>.

=item *

L<Math::BigRat> has been upgraded from version 0.2606 to 0.2607.

Synchronize POD changes from the CPAN release.

=item *

L<Module::Metadata> has been upgraded from version 1.000022 to 1.000024.

Support installations on older perls with an L<ExtUtils::MakeMaker> earlier
than 6.63_03

=item *

L<OS2::Process> has been upgraded from version 1.09 to 1.10.

=item *

L<perl5db.pl> has been upgraded from version 1.44 to 1.45.

fork() in the debugger under C<tmux> will now create a new window for
the forked process. L<[perl
#121333]|https://rt.perl.org/Ticket/Display.html?id=121333>

The debugger now saves the current working directory on startup and
restores it when you restart your program with C<R> or <rerun>. L<[perl
#121509]|https://rt.perl.org/Ticket/Display.html?id=121509>

=item *

L<PerlIO::encoding> has been upgraded from version 0.18 to 0.19.

No changes in behaviour.

=item *

L<PerlIO::mmap> has been upgraded from version 0.012 to 0.013.

No changes in behaviour.

=item *

L<PerlIO::scalar> has been upgraded from version 0.18 to 0.19.

No changes in behaviour.

=item *

L<Unicode::Collate> has been upgraded from version 1.04 to 1.07.

Version 0.67's improved discontiguous contractions is invalidated by default
and is supported as a parameter 'long_contraction'.

=item *

L<Unicode::Normalize> has been upgraded from version 1.17 to 1.18.

The XSUB implementation has been removed in favour of pure Perl.

=item *

A mismatch between the documentation and the code in utf8::downgrade()
was fixed in favour of the documentation. The optional second argument
is now correctly treated as a perl boolean (true/false semantics) and
not as an integer.

=back

=head2 Removed Modules and Pragmata

=over 4

=item *

XXX

=back

=head1 Documentation

XXX Changes to files in F<pod/> go here.  Consider grouping entries by
file and be sure to link to the appropriate page, e.g. L<perlfunc>.

=head2 New Documentation

XXX Changes which create B<new> files in F<pod/> go here.

=head3 L<XXX>

XXX Description of the purpose of the new file here

=head2 Changes to Existing Documentation

XXX Changes which significantly change existing files in F<pod/> go here.
However, any changes to F<pod/perldiag.pod> should go in the L</Diagnostics>
section.

=head3 L<perlfunc>

=over 4

=item *

C<-l> now notes that it will return false if symlinks aren't supported by the 
file system.

L<[perl #121523]|https://rt.perl.org/Ticket/Display.html?id=121523>

=item *

Note that C<exec LIST> and C<system LIST> may fall back to the shell on
Win32. Only C<exec PROGRAM LIST> and C<system PROGRAM LIST> indirect object
syntax will reliably avoid using the shell.

This has also been noted in L<perlport>.

L<[perl #122046]|https://rt.perl.org/Ticket/Display.html?id=122046>

=back

=head3 L<perlapi>

=over 4

=item *

Note that C<SvSetSV> doesn't do set magic.

=item *

C<sv_usepvn_flags> - Fix documentation to mention the use of C<NewX> instead of 
C<malloc>.

L<[perl #121869]|https://rt.perl.org/Ticket/Display.html?id=121869>

=item *

Clarify where C<NUL> may be embedded or is required to terminate a string.

=item *

Previously missing documentation due to formatting errors are now included.

=item *

Entries are now organized into groups rather than by file where they are found.

=item *

Alphabetical sorting of entries is now handled by the POD generator to make 
entries easier to find when scanning.

=back

=head3 L<perlhacktips>

=over 4

=item *

Updated documentation for the C<test.valgrind> C<make> target.

L<[perl #121431]|https://rt.perl.org/Ticket/Display.html?id=121431>

=back

=head3 L<perlre>

=over 4

=item *

The C</x> modifier has been clarified to note that comments cannot be continued 
onto the next line by escaping them.

=back

=head3 L<Unicode::UCD>

=over 4

=item *

The documentation includes many clarifications and fixes.

=back

=head1 Diagnostics

The following additions or changes have been made to diagnostic output,
including warnings and fatal error messages.  For the complete list of
diagnostic messages, see L<perldiag>.

XXX New or changed warnings emitted by the core's C<C> code go here.  Also
include any changes in L<perldiag> that reconcile it to the C<C> code.

=head2 New Diagnostics

XXX Newly added diagnostic messages go under here, separated into New Errors
and New Warnings

=head3 New Errors

=over 4

=item *

XXX L<message|perldiag/"message">

=back

=head3 New Warnings

=over 4

=item *

L<PerlIO layer ':win32' is experimental|perldiag/"PerlIO layer ':win32' is experimental">:

(S experimental::win32_perlio) The C<:win32> PerlIO layer is
experimental.  If you want to take the risk of using this layer,
simply disable this warning:

    no warnings "experimental::win32_perlio";

=item  *

L<Negative repeat count does nothing|perldiag/Negative repeat count does nothing>

(W numeric)  This warns when the repeat count of the
L<C<x>|perlop/Multiplicative Operators> repetition operator is
negative.

This warning may be changed or removed if it turn out that it was
unwise to have added it.

=back

=head2 Changes to Existing Diagnostics

XXX Changes (i.e. rewording) of diagnostic messages go here

=over 4

=item *

<> should be quotes

This warning has been changed to
L<< <> at require-statement should be quotes|perldiag/"<> at require-statement should be quotes" >>
to make the issue more identifiable.

=item *

L<Unsuccessful %s on filename containing newline|perldiag/"Unsuccessful %s on filename containing newline">

This warning is now only produced when the newline is at the end of
the filename.

=back

=head1 Utility Changes

XXX Changes to installed programs such as F<perlbug> and F<xsubpp> go here.
Most of these are built within the directory F<utils>.

[ List utility changes as a =head2 entry for each utility and =item
entries for each change
Use L<XXX> with program names to get proper documentation linking. ]

=head2 F<x2p/>

=over 4

=item *

The F<x2p/> directory has been removed from the Perl core. 

This removes find2perl, s2p and a2p. They have all been released to CPAN as 
separate distributions (App::find2perl, App::s2p, App::a2p).

=back

=head1 Configuration and Compilation

XXX Changes to F<Configure>, F<installperl>, F<installman>, and analogous tools
go here.  Any other changes to the Perl build process should be listed here.
However, any platform-specific changes should be listed in the
L</Platform Support> section, instead.

[ List changes as a =item entry ].

=over 4

=item *

C<make test.valgrind> now supports parallel testing.

For example:

    TEST_JOBS=9 make test.valgrind

See L<perlhacktips/valgrind> for more information.

L<[perl #121431]|https://rt.perl.org/Ticket/Display.html?id=121431>

=back

=head1 Testing

XXX Any significant changes to the testing of a freshly built perl should be
listed here.  Changes which create B<new> files in F<t/> go here as do any
large changes to the testing harness (e.g. when parallel testing was added).
Changes to existing files in F<t/> aren't worth summarizing, although the bugs
that they represent may be covered elsewhere.

[ List each test improvement as a =item entry ]

=over 4

=item *

XXX

=back

=head1 Platform Support

XXX Any changes to platform support should be listed in the sections below.

[ Within the sections, list each platform as a =item entry with specific
changes as paragraphs below it. ]

=head2 New Platforms

XXX List any platforms that this version of perl compiles on, that previous
versions did not.  These will either be enabled by new files in the F<hints/>
directories, or new subdirectories and F<README> files at the top level of the
source tree.

=over 4

=item XXX-some-platform

XXX

=back

=head2 Discontinued Platforms

XXX List any platforms that this version of perl no longer compiles on.

=over 4

=item NeXTSTEP/OPENSTEP

NeXTSTEP was proprietary OS bundled with NeXT's workstations in the early
to mid 90's; OPENSTEP was an API specification that provided a NeXTSTEP-like 
environment on a non-NeXTSTEP system.  Both are now long dead, so support
for building Perl on them has been removed.

=back

=head2 Platform-Specific Notes

XXX List any changes for specific platforms.  This could include configuration
and compilation changes or changes in portability/compatibility.  However,
changes within modules for platforms should generally be listed in the
L</Modules and Pragmata> section.

=over 4

=item XXX-some-platform

XXX

=item OpenBSD

On OpenBSD, Perl will now default to using the system C<malloc> due to the
security features it provides. Perl's own malloc wrapper has been in use
since v5.14 due to performance reasons, but the OpenBSD project believes
the tradeoff is worth it and would prefer that users who need the speed
specifically ask for it.

L<[perl #122000]|https://rt.perl.org/Ticket/Display.html?id=122000>.

=back

=head1 Internal Changes

XXX Changes which affect the interface available to C<XS> code go here.  Other
significant internal changes for future core maintainers should be noted as
well.

=over 4

=item *

The deprecated variable C<PL_sv_objcount> has been removed.

=item *

Perl now tries to keep the locale category C<LC_NUMERIC> set to "C"
except around operations that need it to be set to the program's
underlying locale.  This protects the many XS modules that cannot cope
with the decimal radix character not being a dot.  Prior to this
release, Perl initialized this category to "C", but a call to
C<POSIX::setlocale()> would change it.  Now such a call will change the
underlying locale of the C<LC_NUMERIC> category for the program, but the
locale exposed to XS code will remain "C".  There is an API under
development for those relatively few modules that need to use the
underlying locale.  This API will be nailed down during the course of
developing v5.21.  Send email to L<mailto:perl5-porters@perl.org> for
guidance.

=item *

A new macro L<C<isUTF8_CHAR>|perlapi/isUTF8_CHAR> has been written which
efficiently determines if the string given by its parameters begins
with a well-formed UTF-8 encoded character.

=back

=head1 Selected Bug Fixes

XXX Important bug fixes in the core language are summarized here.  Bug fixes in
files in F<ext/> and F<lib/> are best summarized in L</Modules and Pragmata>.

[ List each fix as a =item entry ]

=over 4

=item *

index() and rindex() no longer crash when used on strings over 2GB in
size.
L<[perl #121562]|https://rt.perl.org/Ticket/Display.html?id=121562>.

=item *

A small previously intentional memory leak in PERL_SYS_INIT/PERL_SYS_INIT3 on
Win32 builds was fixed. This might affect embedders who repeatedly create and
destroy perl engines within the same process.

=item *

C<POSIX::localeconv()> now returns the data for the program's underlying
locale even when called from outside the scope of S<C<use locale>>.

=item *

C<POSIX::localeconv()> now works properly on platforms which don't have
C<LC_NUMERIC> and/or C<LC_MONETARY>, or for which Perl has been compiled
to disregard either or both of these locale categories.  In such
circumstances, there are now no entries for the corresponding values in
the hash returned by C<localeconv()>.

=item *

C<POSIX::localeconv()> now marks appropriately the values it returns as
UTF-8 or not.  Previously they were always returned as a bytes, even if
they were supposed to be encoded as UTF-8.

=item *

On Microsoft Windows, within the scope of C<S<use locale>>, the following
POSIX character classes gave results for many locales that did not
conform to the POSIX standard:
C<[[:alnum:]]>,
C<[[:alpha:]]>,
C<[[:blank:]]>,
C<[[:digit:]]>,
C<[[:graph:]]>,
C<[[:lower:]]>,
C<[[:print:]]>,
C<[[:punct:]]>,
C<[[:upper:]]>,
C<[[:word:]]>,
and
C<[[:xdigit:]]>.
These are because the underlying Microsoft implementation does not
follow the standard.  Perl now takes special precautions to correct for
this.

=item *

Many issues have been detected by L<Coverity|http://www.coverity.com/> and 
fixed.

=item *

system() and friends should now work properly on more Android builds.

Due to an oversight, the value specified through -Dtargetsh to Configure
would end up being ignored by some of the build process.  This caused perls
cross-compiled for Android to end up with defective versions of system(),
exec() and backticks: the commands would end up looking for C</bin/sh>
instead of C</system/bin/sh>, and so would fail for the vast majority
of devices, leaving C<$!> as C<ENOENT>.

=item *

C<qr(...\(...\)...)>,
C<qr[...\[...\]...]>,
and
C<qr{...\{...\}...}>
now work.  Previously it was impossible to escape these three
left-characters with a backslash within a regular expression pattern
where otherwise they would be considered metacharacters, and the pattern
opening delimiter was the character, and the closing delimiter was its
mirror character.

=back

=head1 Known Problems

XXX Descriptions of platform agnostic bugs we know we can't fix go here.  Any
tests that had to be C<TODO>ed for the release would be noted here.  Unfixed
platform specific bugs also go here.

[ List each fix as a =item entry ]

=over 4

=item *

XXX

=back

=head1 Errata From Previous Releases

=over 4

=item *

XXX Add anything here that we forgot to add, or were mistaken about, in
the perldelta of a previous release.

=back

=head1 Obituary

XXX If any significant core contributor has died, we've added a short obituary
here.

=head1 Acknowledgements

XXX Generate this with:

  perl Porting/acknowledgements.pl v5.21.1..HEAD

=head1 Reporting Bugs

If you find what you think is a bug, you might check the articles recently
posted to the comp.lang.perl.misc newsgroup and the perl bug database at
https://rt.perl.org/ .  There may also be information at
http://www.perl.org/ , the Perl Home Page.

If you believe you have an unreported bug, please run the L<perlbug> program
included with your release.  Be sure to trim your bug down to a tiny but
sufficient test case.  Your bug report, along with the output of C<perl -V>,
will be sent off to perlbug@perl.org to be analysed by the Perl porting team.

If the bug you are reporting has security implications, which make it
inappropriate to send to a publicly archived mailing list, then please send it
to perl5-security-report@perl.org.  This points to a closed subscription
unarchived mailing list, which includes all the core committers, who will be
able to help assess the impact of issues, figure out a resolution, and help
co-ordinate the release of patches to mitigate or fix the problem across all
platforms on which Perl is supported.  Please only use this address for
security issues in the Perl core, not for modules independently distributed on
CPAN.

=head1 SEE ALSO

The F<Changes> file for an explanation of how to view exhaustive details on
what changed.

The F<INSTALL> file for how to build Perl.

The F<README> file for general stuff.

The F<Artistic> and F<Copying> files for copyright information.

=cut