begin filling the 5.16.0 delta from 5.15.0

[perl5.git] / Porting / perl5160delta.pod

=encoding utf8

=head1 NAME

perl5160delta - what is new for perl v5.16.0

=head1 DESCRIPTION

This document describes differences between the 5.14.0 release and
the 5.16.0 release.

If you are upgrading from an earlier release such as 5.12.0, first read
L<perl5140delta>, which describes differences between 5.12.0 and
5.14.0.

=head1 Notice

XXX Any important notices here

=head1 Core Enhancements

=head2 C<CORE::> works on all keywords

The C<CORE::> prefix can now be used on keywords enabled by
L<feature.pm|feature>, even outside the scope of C<use feature>.  Relevant
documentation files L<CORE>, L<feature>, L<perlfunc>, L<perlsub>, and
L<perlsyn> have been updated.

=head2 C<continue> no longer requires the "switch" feature

The C<continue> keyword has two meanings.  It can introduce a C<continue>
block after a loop, or it can exit the current C<when> block.  Up till now,
the latter meaning was only valid with the "switch" feature enabled, and
was a syntax error otherwise.  Since the main purpose of feature.pm is to
avoid conflicts with user-defined subroutines, there is no reason for
C<continue> to depend on it.

=head2 C<$$> can be assigned to

C<$$> was made read-only in Perl 5.8.0.  But only sometimes: C<local $$>
would make it writable again.  Some CPAN modules were using C<local $$> or
XS code to bypass the read-only check, so there is no reason to keep C<$$>
read-only.  (This change also allowed a bug to be fixed while maintaining
backward compatibility.)

=head2 Features inside the debugger

The current Perl's feature bundle is now enabled for commands entered in
the interactive debugger.

=head2 C<\N{...}> can now have Unicode loose name matching

This is described in the C<charnames> item in
L</Updated Modules and Pragmata> below.

=head2 Breakpoints with file names

The debugger's "b" command for setting breakpoints now allows a line number
to be prefixed with a file name.  See
L<perldebug/"b [file]:[line] [condition]">.


=head1 Security

XXX Any security-related notices go here.  In particular, any security
vulnerabilities closed should be noted here rather than in the
L</Selected Bug Fixes> section.

[ List each security issue as a =head2 entry ]

=head1 Incompatible Changes

=head2 Tying scalars that hold typeglobs

Attempting to tie a scalar after a typeglob was assigned to it would
instead tie the handle in the typeglob's IO slot.  This meant that it was
impossible to tie the scalar itself.  Similar problems affected C<tied> and
C<untie>: C<tied $scalar> would return false on a tied scalar if the last
thing returned was a typeglob, and C<untie $scalar> on such a tied scalar
would do nothing.

We fixed this problem before Perl 5.14.0, but it caused problems with some
CPAN modules, so we put in a deprecation cycle instead.

Now the deprecation has been removed and this bug has been fixed.  So
C<tie $scalar> will always tie the scalar, not the handle it holds.  To tie
the handle, use C<tie *$scalar> (with an explicit asterisk).  The same
applies to C<tied *$scalar> and C<untie *$scalar>.

=head2 IPC::Open3 no longer provides C<xfork()>, C<xclose_on_exec()>
and C<xpipe_anon()>

All three functions were private, undocumented and unexported.  They do
not appear to be used by any code on CPAN.  Two have been inlined and one
deleted entirely.

=head2 C<$$> no longer caches PID

Previously, if one embeds Perl or uses XS and calls fork(3) from C, Perls
notion of C<$$> could go out of sync with what getpid() returns. By always
fetching the value of C<$$> via getpid(), this potential bug is eliminated.
Code that depends on the caching behavior will break.  As described in
L</Core Enhancements>, C<$$> is now writable, but it will be reset during a
fork.

=head1 Deprecations

XXX Any deprecated features, syntax, modules etc. should be listed here.
In particular, deprecated modules should be listed here even if they are
listed as an updated module in the L</Modules and Pragmata> section.

[ List each deprecation as a =head2 entry ]

=head1 Future Deprecations

This section serves as a notice of feature that are I<likely> to be
L<deprecated|perlpolicy/deprecated> in the next release of perl (5.18.0).  If
your code depends on these features, you should contact the Perl 5 Porters via
the L<mailing list|http://lists.perl.org/list/perl5-porters.html> or L<perlbug>
to explain your use case and inform the deprecation process.

=head1 Performance Enhancements

XXX Changes which enhance performance without changing behaviour go here. There
may well be none in a stable release.

[ List each enhancement as a =item entry ]

=over 4

=item *

XXX

=back

=head1 Modules and Pragmata

XXX All changes to installed files in F<cpan/>, F<dist/>, F<ext/> and F<lib/>
go here.  If Module::CoreList is updated, generate an initial draft of the
following sections using F<Porting/corelist-perldelta.pl>, which prints stub
entries to STDOUT.  Results can be pasted in place of the '=head2' entries
below.  A paragraph summary for important changes should then be added by hand.
In an ideal world, dual-life modules would have a F<Changes> file that could be
cribbed.

[ Within each section, list entries as a =item entry ]

=head2 New Modules and Pragmata

=over 4

=item *

XXX

=back

=head2 Updated Modules and Pragmata

=over 4

=item *

L<XXX> has been upgraded from version 0.69 to version 0.70.

=back

=head2 Removed Modules and Pragmata

As promised in Perl 5.14.0's release notes, the following modules have
been removed from the core distribution, and if needed should be installed
from CPAN instead.

=over

=item *

C<Devel::DProf> has been removed from the Perl core.  Prior version was 20110228.00.

=item *

C<Shell> has been removed from the Perl core.  Prior version was 0.72_01.

=back

=head1 Documentation

XXX Changes to files in F<pod/> go here.  Consider grouping entries by
file and be sure to link to the appropriate page, e.g. L<perlfunc>.

=head2 New Documentation

XXX Changes which create B<new> files in F<pod/> go here.

=head3 L<XXX>

XXX Description of the purpose of the new file here

=head2 Changes to Existing Documentation

XXX Changes which significantly change existing files in F<pod/> go here.
However, any changes to F<pod/perldiag.pod> should go in the L</Diagnostics>
section.

=head3 L<XXX>

=over 4

=item *

XXX Description of the change here

=back

=head1 Diagnostics

The following additions or changes have been made to diagnostic output,
including warnings and fatal error messages.  For the complete list of
diagnostic messages, see L<perldiag>.

XXX New or changed warnings emitted by the core's C<C> code go here. Also
include any changes in L<perldiag> that reconcile it to the C<C> code.

[ Within each section, list entries as a =item entry that links to perldiag,
  e.g.

  =item *

  L<Invalid version object|perldiag/"Invalid version object">
]

=head2 New Diagnostics

XXX Newly added diagnostic messages go here

=head3 New Errors

=over 4

=item *

XXX L<message|perldiag/"message">

=back

=head3 New Warnings

=over 4

=item L<Useless assignment to a temporary|perldiag/"Useless assignment to a temporary">

Assigning to a temporary returned from an XS lvalue subroutine now produces a
warning [perl #31946].

=back

=back

=head2 Changes to Existing Diagnostics

XXX Changes (i.e. rewording) of diagnostic messages go here

=over 4

=item *

XXX Describe change here

=back

=head1 Utility Changes

XXX Changes to installed programs such as F<perlbug> and F<xsubpp> go
here. Most of these are built within the directories F<utils> and F<x2p>.

[ List utility changes as a =head3 entry for each utility and =item
entries for each change
Use L<XXX> with program names to get proper documentation linking. ]

=head3 L<XXX>

=over 4

=item *

XXX

=back

=head1 Configuration and Compilation

=over 4

=item *

F<regexp.h> has been modified for compatibility with GCC's B<-Werror>
option, as used by some projects that include perl's header files (5.14.1).

=item *

C<USE_LOCALE{,_COLLATE,_CTYPE,_NUMERIC}> have been added the output of perl -V
as they have affect the behaviour of the interpreter binary (albeit only
in a small area).

=item *

The code and tests for L<IPC::Open2> have been moved from F<ext/IPC-Open2>
into F<ext/IPC-Open3>, as C<IPC::Open2::open2()> is implemented as a thin
wrapper around C<IPC::Open3::_open3()>, and hence is very tightly coupled to
it.

=item *

The magic types and magic vtables are now generated from data in a new script
F<regen/mg_vtable.pl>, instead of being maintained by hand. As different EBCDIC
variants can't agree on the code point for '~', the character to code point
conversion is done at build time by F<generate_uudmap> to a new generated header
F<mg_data.h>. C<PL_vtbl_bm> and C<PL_vtbl_fm> are now defined by the
pre-processor as C<PL_vtbl_regexp>, instead of being distinct C variables.
C<PL_vtbl_sig> has been removed.

=item *

Building with C<-DPERL_GLOBAL_STRUCT> works again. This configuration is not
generally used.

=item *

Perl configured with I<MAD> now correctly frees C<MADPROP> structures when
OPs are freed. C<MADPROP>s are now allocated with C<PerlMemShared_malloc()>

=back

=back

=head1 Testing

XXX Any significant changes to the testing of a freshly built perl should be
listed here.  Changes which create B<new> files in F<t/> go here as do any
large changes to the testing harness (e.g. when parallel testing was added).
Changes to existing files in F<t/> aren't worth summarising, although the bugs
that they represent may be covered elsewhere.

[ List each test improvement as a =item entry ]

=over 4

=item *

XXX

=back

=head1 Platform Support

XXX Any changes to platform support should be listed in the sections below.

[ Within the sections, list each platform as a =item entry with specific
changes as paragraphs below it. ]

=head2 New Platforms

XXX List any platforms that this version of perl compiles on, that previous
versions did not. These will either be enabled by new files in the F<hints/>
directories, or new subdirectories and F<README> files at the top level of the
source tree.

=over 4

=item XXX-some-platform

XXX

=back

=head2 Discontinued Platforms

XXX List any platforms that this version of perl no longer compiles on.

=over 4

=item XXX-some-platform

XXX

=back

=head2 Platform-Specific Notes

XXX List any changes for specific platforms. This could include configuration
and compilation changes or changes in portability/compatibility.  However,
changes within modules for platforms should generally be listed in the
L</Modules and Pragmata> section.

=over 4

=item XXX-some-platform

XXX

=back

=head1 Internal Changes

XXX Changes which affect the interface available to C<XS> code go here.
Other significant internal changes for future core maintainers should
be noted as well.

[ List each change as a =item entry ]

=over 4

=item *

XXX

=back

=head1 Selected Bug Fixes

=head2 Regular expressions and character classes

=over 4

=item *

The new (in 5.14.0) regular expression modifier C</a> when repeated like
C</aa> forbids the characters outside the ASCII range that match
characters inside that range from matching under C</i>.  This did not
work under some circumstances, all involving alternation, such as:

 "\N{KELVIN SIGN}" =~ /k|foo/iaa;

succeeded inappropriately.  This is now fixed.

=item *

5.14.0 introduced some memory leaks in regular expression character
classes such as C<[\w\s]>, which have now been fixed (5.14.1)

=item *

An edge case in regular expression matching could potentially loop.
This happened only under C</i> in bracketed character classes that have
characters with multi-character folds, and the target string to match
against includes the first portion of the fold, followed by another
character that has a multi-character fold that begins with the remaining
portion of the fold, plus some more.

 "s\N{U+DF}" =~ /[\x{DF}foo]/i

is one such case.  C<\xDF> folds to C<"ss">. (5.14.1)

=item *

A few characters in regular expression pattern matches did not
match correctly in some circumstances, all involving C</i>.  The
affected characters are:
COMBINING GREEK YPOGEGRAMMENI,
GREEK CAPITAL LETTER IOTA,
GREEK CAPITAL LETTER UPSILON,
GREEK PROSGEGRAMMENI,
GREEK SMALL LETTER IOTA WITH DIALYTIKA AND OXIA,
GREEK SMALL LETTER IOTA WITH DIALYTIKA AND TONOS,
GREEK SMALL LETTER UPSILON WITH DIALYTIKA AND OXIA,
GREEK SMALL LETTER UPSILON WITH DIALYTIKA AND TONOS,
LATIN SMALL LETTER LONG S,
LATIN SMALL LIGATURE LONG S T,
and
LATIN SMALL LIGATURE ST.

=item *

Fixed memory leak regression in regular expression compilation
under threading

=back

=head2 Formats

=over

=item *

A number of edge cases have been fixed with formats and C<formline>;
in particular, where the format itself is potentially variable (such as
with ties and overloading), and where the format and data differ in their
encoding. In both these cases, it used to possible for the output to be
corrupted [perl #91032].

=item *

C<formline> no longer converts its argument into a string in-place.  So
passing a reference to C<formline> no longer destroys the reference
[perl #79532].

=back

=head2 Copy-on-write scalars

Copy-on-write scalars were introduced in 5.8.0, but most Perl code
did not encounter them (they were used mostly internally).  Perl
5.10.0 extended them, such that assigning C<__PACKAGE__> or a
hash key to a scalar would make it copy-on-write.  Several parts
of Perl were not updated to account for them, but have now been fixed.

=over

=item *

C<utf8::decode> had a nasty bug that would modify copy-on-write scalars'
string buffers in place (i.e., skipping the copy).  This could result in
hashes having two elements with the same key [perl #91834].

=item *

Lvalue subroutines were not allowing COW scalars to be returned.  This was
fixed for lvalue scalar context in Perl 5.12.3 and 5.14.0, but list context
was not fixed until this release.

=item *

Elements of restricted hashes (see the L<fields> pragma) containing
copy-on-write values couldn't be deleted, nor could such hashes be cleared
(C<%hash = ()>).

=item *

Localising a tied variable used to make it read-only if it contained a
copy-on-write string.

=item *

L<Storable>, L<Devel::Peek> and L<PerlIO::scalar> had similar problems.
See L</Updated Modules and Pragmata>, above.

=back

=head2 lvalue subroutines

There have been various fixes to lvalue subroutines.

=over

=item *

Explicit return now returns the actual argument passed to return, instead
of copying it [perl #72724] [perl #72706].

B<Note:> There are still some discrepancies between explicit and implicit
return, which will hopefully be resolved soon.  So the exact behaviour is
not set in stone yet.

=item *

Lvalue subroutines used to enforce lvalue syntax (i.e., whatever can go on
the left-hand side of C<=>) for the last statement and the arguments to
return.  Since lvalue subroutines are not always called in lvalue context,
this restriction has been lifted.

=item *

Lvalue subroutines are less restrictive as to what values can be returned.
It used to croak on values returned by C<shift> and C<delete> and from
other subroutines, but no longer does so [perl #71172].

=item *

Empty lvalue subroutines (C<sub :lvalue {}>) used to return C<@_> in list
context.  In fact, all subroutines used to, but regular subs were fixed in
Perl 5.8.2.  Now lvalue subroutines have been likewise fixed.

=item *

Lvalue subroutines used to copy their return values in rvalue context.  Not
only was this a waste of CPU cycles, but it also caused bugs.  A C<($)>
prototype would cause an lvalue sub to copy its return value [perl #51408],
and C<while(lvalue_sub() =~ m/.../g) { ... }> would loop endlessly
[perl #78680].

=item *

Autovivification now works on values returned from lvalue subroutines
[perl #7946].

=item *

When called in pass-by-reference context (e.g., subroutine arguments or a list
passed to C<for>), an lvalue subroutine returning arrays or hashes used to bind
the arrays (or hashes) to scalar variables--something that is not supposed to
happen.  This could result in "Bizarre copy of ARRAY" errors or C<print>
ignoring its arguments.  It also made nonsensical code like C<@{\$_}> "work".
This was fixed in 5.14.0 if an array were the first thing returned from the
subroutine (but not for C<$scalar, @array> or hashes being returned).  Now a
more general fix has been applied [perl #23790].

=item *

When called in pass-by-reference context, lvalue subroutines used to copy
any read-only value that was returned.  E.g., C< sub :lvalue { $] } >
would not return C<$]>, but a copy of it.

=item *

Assignment to C<keys> returned from an lvalue sub used not to work, but now
it does.

=item *

Applying the C<:lvalue> attribute to an XSUB or to an aliased subroutine
stub with C<< sub foo :lvalue; >> syntax stopped working in Perl 5.12.
This has been fixed.

=back

=head2 Fixes related to hashes

=over

=item *

A bug has been fixed that would cause a "Use of freed value in iteration"
error if the next two hash elements that would be iterated over are
deleted [perl #85026]. (5.14.1)

=item *

Freeing deeply nested hashes no longer crashes [perl #44225].

=item *

Deleting the current hash iterator (the hash element that would be returend
by the next call to C<each>) in void context used not to free it.  The hash
would continue to reference it until the next iteration.  This has been
fixed [perl #85026].

=back

=head2 Other notable fixes

=over

=item *

Passing the same constant subroutine to both C<index> and C<formline> no
longer causes one or the other to fail [perl #89218]. (5.14.1)

=item *

List assignment to lexical variables declared with attributes in the same
statement (C<my ($x,@y) : blimp = (72,94)>) stopped working in Perl 5.8.0.
It has now been fixed.

=item *

Perl 5.10.0 introduced some faulty logic that made "U*" in the middle of
a pack template equivalent to "U0" if the input string was empty.  This has
been fixed [perl #90160].

=item *

Destructors on objects were not called during global destruction on objects
that were not referenced by any scalars.  This could happen if an array
element were blessed (e.g., C<bless \$a[0]>) or if a closure referenced a
blessed variable (C<bless \my @a; sub foo { @a }>).

Now there is an extra pass during global destruction to fire destructors on
any objects that might be left after the usual passes that check for
objects referenced by scalars [perl #36347].

This bug fix was added in Perl 5.13.9, but caused problems with some CPAN
modules that were relying on the bug.  Since it was so close to Perl
5.14.0, the fix was reverted in 5.13.10, to allow more time for the modules
to adapt.  Hopefully they will be fixed soon (see L</Known Problems>,
below).

=item *

C<given> was not calling set-magic on the implicit lexical C<$_> that it
uses.  This meant, for example, that C<pos> would be remembered from one
execution of the same C<given> block to the next, even if the input were a
different variable [perl #84526].

=item *

The "R" command for restarting a debugger session has been fixed to work on
Windows, or any other system lacking a C<POSIX::_SC_OPEN_MAX> constant
[perl #87740].

=item *

Fixed a case where it was possible that a freed buffer may have been read
from when parsing a here document [perl #90128]. (5.14.1)

=item *

The C<study> function could become confused if fed a string longer than
2**31 characters.  Now it simply skips such strings.

=item *

C<each(I<ARRAY>)> is now wrapped in C<defined(...)>, like C<each(I<HASH>)>,
inside a C<while> condition [perl #90888].

=item *

In @INC filters (subroutines returned by subroutines in @INC), $_ used to
misbehave: If returned from a subroutine, it would not be copied, but the
variable itself would be returned; and freeing $_ (e.g., with C<undef *_>)
would cause perl to crash.  This has been fixed [perl #91880].

=item *

An ASCII single quote (') in a symbol name is meant to be equivalent to a
double colon (::) except at the end of the name.  It was not equivalent if
followed by a null character, but now it is [perl #88138].

=item *

The abbreviations for four C1 control characters
C<MW>
C<PM>,
C<RI>,
and
C<ST>
were previously unrecognized by C<\N{}>,
vianame(), and string_vianame().

=item *

Some cases of threads crashing due to memory allocation during cloning have
been fixed [perl #90006].

=item *

Attempting to C<goto> out of a tied handle method used to cause memory
corruption or crashes.  Now it produces an error message instead
[perl #8611].

=item *

Perl skips copying values returned from a subroutine if it thinks the value
is not in use elsewhere.  Due to faulty logic, this would happen with the
result of C<delete>, C<shift> or C<splice>, even if the result was
referenced elsewhere.  So C<< \sub { delete $_[0] }->($x) >> would return a
reference to C<$x>.  This has been fixed [perl #91844].

=back

=head1 Known Problems

XXX Descriptions of platform agnostic bugs we know we can't fix go here. Any
tests that had to be C<TODO>ed for the release would be noted here, unless
they were specific to a particular platform (see below).

This is a list of some significant unfixed bugs, which are regressions
from either 5.XXX.XXX or 5.XXX.XXX.

[ List each fix as a =item entry ]

=over 4

=item *

XXX

=back

=head1 Obituary

XXX If any significant core contributor has died, we've added a short obituary
here.

=head1 Acknowledgements

XXX Generate this with:

  perl Porting/acknowledgements.pl v5.15.7..HEAD

=head1 Reporting Bugs

If you find what you think is a bug, you might check the articles
recently posted to the comp.lang.perl.misc newsgroup and the perl
bug database at http://rt.perl.org/perlbug/ .  There may also be
information at http://www.perl.org/ , the Perl Home Page.

If you believe you have an unreported bug, please run the L<perlbug>
program included with your release.  Be sure to trim your bug down
to a tiny but sufficient test case.  Your bug report, along with the
output of C<perl -V>, will be sent off to perlbug@perl.org to be
analysed by the Perl porting team.

If the bug you are reporting has security implications, which make it
inappropriate to send to a publicly archived mailing list, then please send
it to perl5-security-report@perl.org. This points to a closed subscription
unarchived mailing list, which includes
all the core committers, who will be able
to help assess the impact of issues, figure out a resolution, and help
co-ordinate the release of patches to mitigate or fix the problem across all
platforms on which Perl is supported. Please only use this address for
security issues in the Perl core, not for modules independently
distributed on CPAN.

=head1 SEE ALSO

The F<Changes> file for an explanation of how to view exhaustive details
on what changed.

The F<INSTALL> file for how to build Perl.

The F<README> file for general stuff.

The F<Artistic> and F<Copying> files for copyright information.

=cut