import perl5171delta content to perl5180delta

[perl5.git] / Porting / perl5180delta.pod

=encoding utf8

=head1 NAME

perl5180delta - what is new for perl v5.18.0

=head1 DESCRIPTION

This document describes differences between the 5.16.0 release and the 5.18.0
release.

If you are upgrading from an earlier release such as 5.14.0, first read
L<perl5140delta>, which describes differences between 5.12.0 and 5.14.0.

=head1 Notice

XXX Any important notices here

=head1 Core Enhancements

=head2 More CORE:: subs

Several more built-in functions have been added as subroutines to the
CORE:: namespace, namely, those non-overridable keywords that can be
implemented without custom parsers: C<defined>, C<delete>, C<exists>,
C<glob>, C<pos>, C<protoytpe>, C<scalar>, C<split>, C<study>, and C<undef>.

As some of these have prototypes, C<prototype('CORE::...')> has been
changed to not make a distinction between overridable and non-overridable
keywords.  This is to make C<prototype('CORE::pos')> consistent with
C<prototype(&CORE::pos)>.


=head1 Security

XXX Any security-related notices go here.  In particular, any security
vulnerabilities closed should be noted here rather than in the
L</Selected Bug Fixes> section.

[ List each security issue as a =head2 entry ]

=head1 Incompatible Changes

=head2 qw(...) can no longer be used as parentheses

C<qw> lists used to fool the parser into thinking they were always
surrounded by parentheses.  This permitted some surprising constructions
such as C<foreach $x qw(a b c) {...}>, which should really be written
C<foreach $x (qw(a b c)) {...}>.  These would sometimes get the lexer into
the wrong state, so they didn't fully work, and the similar C<foreach qw(a
b c) {...}> that one might expect to be permitted never worked at all.

This side effect of C<qw> has now been abolished.  It has been deprecated
since Perl 5.13.11.  It is now necessary to use real parentheses
everywhere that the grammar calls for them.

=head2 C<\s> in regular expressions now matches a Vertical Tab

[ XXX ]

=head2 C</(?{})/> and C</(??{})/> have been heavily reworked

The implementation of this feature has been almost completely rewritten.
Although its main intent is to fix bugs, some behaviors, especially
related to the scope of lexical variables, will have changed.  This is
described more fully in the L</Selected Bug Fixes> section.

=head2 C<\N{BELL}> now refers to U+1F514 instead of U+0007

Unicode 6.0 reused the name "BELL" for a different code point than it
traditionally had meant.  Since Perl v5.14, use of this name still
referred to U+0007, but would raise a deprecation warning.  Now, "BELL"
refers to U+1F514, and the name for U+0007 is "ALERT".  All the
functions in L<charnames> have been correspondingly updated.

=head2 Alphanumeric operators must now be separated from the closing
delimiter of regular expressions

You may no longer write something like:

 m/a/and 1

Instead you must write

 m/a/ and 1

with whitespace separating the operator from the closing delimiter of
the regular expression.  Not having whitespace has resulted in a
deprecation warning since Perl v5.14.0.

=head2 C<require> dies for unreadable files

When C<require> encounters an unreadable file, it now dies.  It used to
ignore the file and continue searching the directories in @INC
[perl #113422].

=head2 Upgrade to the Unicode 6.2 beta

Unicode 6.2 is proposing some changes that may very well break some CPAN
modules.  The timing of this nicely coincides with Perl's being early in the
release cycle.  This commit takes the current beta 6.2, adds the proposed
changes that aren't yet in it, and subtracts the changes that would affect \X
processing, as those turn out to have errors, and may have to be rethought.
Unicode has been notified of these problems.

This will allow us to gather data as to whether or not the proposed changes
cause us problems.  These will be presented to Unicode to aid in their final
decision as to whether or not to go forward with the changes.

These changes will be replaced by the final version of Unicode 6.2 before
5.18.0 is released.

=head1 Deprecations

XXX Any deprecated features, syntax, modules etc. should be listed here.  In
particular, deprecated modules should be listed here even if they are listed as
an updated module in the L</Modules and Pragmata> section.

[ List each deprecation as a =head2 entry ]

=head1 Performance Enhancements

XXX Changes which enhance performance without changing behaviour go here.
There may well be none in a stable release.

[ List each enhancement as a =item entry ]

=over 4

=item *

Filetest ops manage the stack in a fractionally more efficient manner.

=item *

Globs used in a numeric context are now numerified directly in most cases,
rather than being numerified via stringification.

=item *

The C<x> repetition operator is now folded to a single constant at compile
time if called in scalar context with constant operands and no parentheses
around the left operand.

=back

=head1 Modules and Pragmata

XXX All changes to installed files in F<cpan/>, F<dist/>, F<ext/> and F<lib/>
go here.  If Module::CoreList is updated, generate an initial draft of the
following sections using F<Porting/corelist-perldelta.pl>, which prints stub
entries to STDOUT.  Results can be pasted in place of the '=head2' entries
below.  A paragraph summary for important changes should then be added by hand.
In an ideal world, dual-life modules would have a F<Changes> file that could be
cribbed.

[ Within each section, list entries as a =item entry ]

=head2 New Modules and Pragmata

=over 4

=item *

XXX

=back

=head2 Updated Modules and Pragmata

=over 4

=item *

L<XXX> has been upgraded from version A.xx to B.yy.

=back

=head2 Removed Modules and Pragmata

=over 

=item *

L<Version::Requirements> has been removed from the core distribution.  It is
available under a different name: L<CPAN::Meta::Requirements>.

=back

=head1 Documentation

XXX Changes to files in F<pod/> go here.  Consider grouping entries by
file and be sure to link to the appropriate page, e.g. L<perlfunc>.

=head2 New Documentation

XXX Changes which create B<new> files in F<pod/> go here.

=head3 L<XXX>

XXX Description of the purpose of the new file here

=head2 Changes to Existing Documentation

=head3 L<perldata>

=over 4

=item *

Now explicitly documents the behaviour of hash initializer lists that
contain duplicate keys.

=back

=head3 L<perldiag>

=over 4

=item *

The explanation of symbolic references being prevented by "strict refs"
now doesn't assume that the reader knows what symbolic references are.

=back

=head3 L<perlfunc>

=over 4

=item *

The return value of C<pipe> is now documented.

=back

=head3 L<perlfaq>

=over 4

=item *

L<perlfaq> has been synchronized with version 5.0150040 from CPAN.

=back

=head3 L<perlcheat>

=over 4

=item *

L<perlcheat> has been reorganized, and a few new sections were added.

=back

=head3 Diagnostics

The following additions or changes have been made to diagnostic output,
including warnings and fatal error messages.  For the complete list of
diagnostic messages, see L<perldiag>.

XXX New or changed warnings emitted by the core's C<C> code go here.  Also
include any changes in L<perldiag> that reconcile it to the C<C> code.

=head2 New Diagnostics

XXX Newly added diagnostic messages go under here, separated into New Errors
and New Warnings

=head3 New Errors

=over 4

=item *

XXX L<message|perldiag/"message">

=back

=head3 New Warnings

=over 4

=item *

XXX L<message|perldiag/"message">

=back

=head2 Changes to Existing Diagnostics

XXX Changes (i.e. rewording) of diagnostic messages go here

=over 4

=item *

The "Runaway prototype" warning that occurs in bizarre cases has been
removed as being unhelpful and inconsistent.

=item *

The "Not a format reference" error has been removed, as the only case in
which it could be triggered was a bug.

=item *

The "Unable to create sub named %s" error has been removed for the same
reason.

=back

=head1 Utility Changes

XXX Changes to installed programs such as F<perlbug> and F<xsubpp> go here.
Most of these are built within the directories F<utils> and F<x2p>.

[ List utility changes as a =head3 entry for each utility and =item
entries for each change
Use L<XXX> with program names to get proper documentation linking. ]

=head3 L<XXX>

=over 4

=item *

XXX

=back

=head1 Configuration and Compilation

XXX Changes to F<Configure>, F<installperl>, F<installman>, and analogous tools
go here.  Any other changes to the Perl build process should be listed here.
However, any platform-specific changes should be listed in the
L</Platform Support> section, instead.

[ List changes as a =item entry ].

=over 4

=item *

XXX

=back

=head1 Testing

XXX Any significant changes to the testing of a freshly built perl should be
listed here.  Changes which create B<new> files in F<t/> go here as do any
large changes to the testing harness (e.g. when parallel testing was added).
Changes to existing files in F<t/> aren't worth summarizing, although the bugs
that they represent may be covered elsewhere.

[ List each test improvement as a =item entry ]

=over 4

=item *

The test suite now has a section for tests that require very large amounts
of memory.  These tests won't run by default; they can be enabled by
setting the C<PERL_TEST_MEMORY> environment variable to the number of
gibibytes of memory that may be safely used.

=back

=head1 Platform Support

XXX Any changes to platform support should be listed in the sections below.

[ Within the sections, list each platform as a =item entry with specific
changes as paragraphs below it. ]

=head2 New Platforms

XXX List any platforms that this version of perl compiles on, that previous
versions did not.  These will either be enabled by new files in the F<hints/>
directories, or new subdirectories and F<README> files at the top level of the
source tree.

=over 4

=item XXX-some-platform

XXX

=back

=head2 Discontinued Platforms

XXX List any platforms that this version of perl no longer compiles on.

=over 4

=item XXX-some-platform

XXX

=back

=head2 Platform-Specific Notes

XXX List any changes for specific platforms.  This could include configuration
and compilation changes or changes in portability/compatibility.  However,
changes within modules for platforms should generally be listed in the
L</Modules and Pragmata> section.

=over 4

=item clang++

There is now a workaround for a compiler bug that prevented compiling
with clang++ since Perl 5.15.7 [perl #112786].

=item C++

When compiling the Perl core as C++ (which is only semi-supported), the
mathom functions are now compiled as C<extern "C">, to ensure proper
binary compatibility.  (However, binary compatibility isn't generally
guaranteed anyway in the situations where this would matter.)

=item VMS

It should now be possible to compile Perl as C++ on VMS.

=item Win32

C<link> on Win32 now attempts to set C<$!> to more appropriate values
based on the Win32 API error code. [perl #112272]

Perl no longer mangles the environment block, e.g. when launching a new
sub-process, when the environment contains non-ASCII characters. Known
problems still remain, however, when the environment contains characters
outside of the current ANSI codepage (e.g. see the item about Unicode in
C<%ENV> in L<http://perl5.git.perl.org/perl.git/blob/HEAD:/Porting/todo.pod>).
[perl #113536]

=item VMS

All C header files from the top-level directory of the distribution are now
installed on VMS, providing consistency with a long-standing practice on other
platforms. Previously only a subset were installed, which broke non-core
extension builds for extensions that depended on the missing include files.

=back

=head1 Internal Changes

=over

=item *

The C<CV *> typemap entry now supports C<&{}> overloading and typeglobs,
just like C<&{...}> [perl #96872].

=item *

The C<SVf_AMAGIC> flag to indicate overloading is now on the stash, not the
object.  It is now set automatically whenever a method or @ISA changes, so
its meaning has changed, too.  It now means "potentially overloaded".  When
the overload table is calculated, the flag is automatically turned off if
there is no overloading, so there should be no noticeable slowdown.

The staleness of the overload tables is now checked when overload methods
are invoked, rather than during C<bless>.

"A" magic is gone.  The changes to the handling of the C<SVf_AMAGIC> flag
eliminate the need for it.

C<PL_amagic_generation> has been removed as no longer necessary.  For XS
modules, it is now a macro alias to C<PL_na>.

The fallback overload setting is now stored in a stash entry separate from
overloadedness itself.

=item *

The character-processing code has been cleaned up in places.  The changes
should be operationally invisible.

=item *

The C<study> function was made a no-op in 5.16.  It was simply disabled via
a C<return> statement; the code was left in place.  Now the code supporting
what C<study> used to do has been removed.

=item *

Under threaded perls, there is no longer a separate PV allocated for every
COP to store its package name (C<< cop->stashpv >>).  Instead, there is an
offset (C<< cop->stashoff >>) into the new C<PL_stashpad> array, which
holds stash pointers.

=item *

In the pluggable regex API, the C<regexp_engine> struct has acquired a new
field C<op_comp>, which is currently just for perl's internal use, and
should be initialised to NULL by other regex plugin modules.

=item *

A new function C<alloccoptash> has been added to the API, but is considered
experimental.  See L<perlapi>.

=back

=head1 Selected Bug Fixes

XXX Important bug fixes in the core language are summarized here.  Bug fixes in
files in F<ext/> and F<lib/> are best summarized in L</Modules and Pragmata>.

[ List each fix as a =item entry ]

=over 4

=item *

C<goto ''> now looks for an empty label, producing the "goto must have
label" error message, instead of exiting the program [perl #111794].

=item *

C<goto "\0"> now dies with "Can't find label" instead of "goto must have
label".

=item *

The C function C<hv_store> used to result in crashes when used on C<%^H>
[perl #111000].

=item *

A call checker attached to a closure prototype via C<cv_set_call_checker>
is now copied to closures cloned from it.  So C<cv_set_call_checker> now
works inside an attribute handler for a closure.

=item *

Writing to C<$^N> used to have no effect.  Now it croaks with "Modification
of a read-only value" by default, but that can be overridden by a custom
regular expression engine, as with C<$1> [perl #112184].

=item *

C<undef> on a control character glob (C<undef *^H>) no longer emits an
erroneous warning about ambiguity [perl #112456].

=item *

For efficiency's sake, many operators and built-in functions return the
same scalar each time.  Lvalue subroutines and subroutines in the CORE::
namespace were allowing this implementation detail to leak through.
C<print &CORE::uc("a"), &CORE::uc("b")> used to print "BB".  The same thing
would happen with an lvalue subroutine returning the return value of C<uc>.
Now the value is copied in such cases.

=item *

C<method {}> syntax with an empty block or a block returning an empty list
used to crash or use some random value left on the stack as its invocant.
Now it produces an error.

=item *

C<vec> now works with extremely large offsets (E<gt>2 GB) [perl #111730].

=item *

Changes to overload settings now take effect immediately, as do changes to
inheritance that affect overloading.  They used to take effect only after
C<bless>.

Objects that were created before a class had any overloading used to remain
non-overloaded even if the class gained overloading through C<use overload>
or @ISA changes, and even after C<bless>.  This has been fixed
[perl #112708].

=item *

Classes with overloading can now inherit fallback values.

=item *

Overloading was not respecting a fallback value of 0 if there were
overloaded objects on both sides of an assignment operator like C<+=>
[perl #111856].

=item *

C<pos> now croaks with hash and array arguments, instead of producing
erroneous warnings.

=item *

C<while(each %h)> now implies C<while(defined($_ = each %h))>, like
C<readline> and C<readdir>.

=item *

Subs in the CORE:: namespace no longer crash after C<undef *_> when called
with no argument list (C<&CORE::time> with no parentheses).

=item *

Unicode 6.1 published an incorrect alias for one of the
Canonical_Combining_Class property's values (which range between 0 and
254).  The alias C<CCC133> should have been C<CCC132>.  Perl now
overrides the data file furnished by Unicode to give the correct value.

=item *

C<unpack> no longer produces the "'/' must follow a numeric type in unpack"
error when it is the data that are at fault [perl #60204].

=item *

C<join> and C<"@array"> now call FETCH only once on a tied C<$">
[perl #8931].

=item *

Some subroutine calls generated by compiling core ops affected by a
C<CORE::GLOBAL> override had op checking performed twice.  The checking
is always idempotent for pure Perl code, but the double checking can
matter when custom call checkers are involved.

=item *

A race condition used to exist around fork that could cause a signal sent to
the parent to be handled by both parent and child. Signals are now blocked
briefly around fork to prevent this from happening [perl #82580].

=item *

The implementation of code blocks in regular expressions, such as C<(?{})>
and C<(??{})>, has been heavily reworked to eliminate a whole slew of bugs.
The main user-visible changes are:

=over 4

=item *

Code blocks within patterns are now parsed in the same pass as the
surrounding code; in particular it is no longer necessary to have balanced
braces: this now works:

    /(?{  $x='{'  })/

This means that this error message is longer generated:

    Sequence (?{...}) not terminated or not {}-balanced in regex

but a new error may be seen:

    Sequence (?{...}) not terminated with ')'

In addition, literal code blocks within run-time patterns are only
compiled once, at perl compile-time:

    for my $p (...) {
        # this 'FOO' block of code is compiled once,
	# at the same time as the surrounding 'for' loop
        /$p{(?{FOO;})/;
    }

=item *

Lexical variables are now sane as regards scope, recursion and closure
behavior. In particular, C</A(?{B})C/> behaves (from a closure viewpoint)
exactly like C</A/ && do { B } && /C/>, while  C<qr/A(?{B})C/> is like
C<sub {/A/ && do { B } && /C/}>. So this code now works how you might
expect, creating three regexes that match 0, 1, and 2:

    for my $i (0..2) {
        push @r, qr/^(??{$i})$/;
    }
    "1" =~ $r[1]; # matches

=item *

The C<use re 'eval'> pragma is now only required for code blocks defined
at runtime; in particular in the following, the text of the C<$r> pattern is
still interpolated into the new pattern and recompiled, but the individual
compiled code-blocks within C<$r> are reused rather than being recompiled,
and C<use re 'eval'> isn't needed any more:

    my $r = qr/abc(?{....})def/;
    /xyz$r/;

=item *

Flow control operators no longer crash. Each code block runs in a new
dynamic scope, so C<next> etc. will not see
any enclosing loops. C<return> returns a value
from the code block, not from any enclosing subroutine.

=item *

Perl normally caches the compilation of run-time patterns, and doesn't
recompile if the pattern hasn't changed, but this is now disabled if
required for the correct behavior of closures. For example:

    my $code = '(??{$x})';
    for my $x (1..3) {
	# recompile to see fresh value of $x each time
        $x =~ /$code/;
    }


=item *

The C</msix> and C<(?msix)> etc. flags are now propagated into the return
value from C<(??{})>; this now works:

    "AB" =~ /a(??{'b'})/i;

=item *

Warnings and errors will appear to come from the surrounding code (or for
run-time code blocks, from an eval) rather than from an C<re_eval>:

    use re 'eval'; $c = '(?{ warn "foo" })'; /$c/;
    /(?{ warn "foo" })/;

formerly gave:

    foo at (re_eval 1) line 1.
    foo at (re_eval 2) line 1.

and now gives:

    foo at (eval 1) line 1.
    foo at /some/prog line 2.

=back

=item *

Perl now works as well as can be expected on all releases of Unicode so
far.  In v5.16, it worked on Unicodes 6.0 and 6.1, but there were
various bugs for earlier releases; the older the release the more
problems.

=item *

C<vec> no longer produces "uninitialized" warnings in lvalue context
[perl #9423].

=item *

An optimization involving fixed strings in regular expressions could cause
a severe performance penalty in edge cases.  This has been fixed
[perl #76546].

=item *

In certain cases, including empty subpatterns within a regular expression (such
as C<(?:)> or C<(?:|)>) could disable some optimizations. This has been fixed.

=item *

The "Can't find an opnumber" message that C<prototype> produces when passed
a string like "CORE::nonexistent_keyword" now passes UTF-8 and embedded
NULs through unchanged [perl #97478].

=item *

C<prototype> now treats magical variables like C<$1> the same way as
non-magical variables when checking for the CORE:: prefix, instead of
treating them as subroutine names.

=item *

Under threaded perls, a runtime code block in a regular expression could
corrupt the package name stored in the op tree, resulting in bad reads
in C<caller>, and possibly crashes [perl #113060].

=item *

Referencing a closure prototype (C<\&{$_[1]}> in an attribute handler for a
closure) no longer results in a copy of the subroutine (or assertion
failures on debugging builds).

=item *

C<eval '__PACKAGE__'> now returns the right answer on threaded builds if
the current package has been assigned over (as in
C<*ThisPackage:: = *ThatPackage::>) [perl #78742].

=item *

If a package is deleted by code that it calls, it is possible for C<caller>
to see a stack frame belonging to that deleted package.  C<caller> could
crash if the stash's memory address was reused for a scalar and a
substitution was performed on the same scalar [perl #113486].

=item *

C<UNIVERSAL::can> no longer treats its first argument differently
depending on whether it is a string or number internally.

=item *

C<open> with C<< <& >> for the mode checks to see whether the third argument is
a number, in determining whether to treat it as a file descriptor or a handle
name.  Magical variables like C<$1> were always failing the numeric check and
being treated as handle names.

=item *

C<warn>'s handling of magical variables (C<$1>, ties) has undergone several
fixes.  C<FETCH> is only called once now on a tied argument or a tied C<$@>
[perl #97480].  Tied variables returning objects that stringify as "" are
no longer ignored.  A tied C<$@> that happened to return a reference the
I<previous> time it was used is no longer ignored.

=item *

C<warn ""> now treats C<$@> with a number in it the same way, regardless of
whether it happened via C<$@=3> or C<$@="3">.  It used to ignore the
former.  Now it appends "\t...caught", as it has always done with
C<$@="3">.

=item *

Numeric operators on magical variables (e.g., S<C<$1 + 1>>) used to use
floating point operations even where integer operations were more appropriate,
resulting in loss of accuracy on 64-bit platforms [perl #109542].

=item *

Unary negation no longer treats a string as a number if the string happened
to be used as a number at some point.  So, if C<$x> contains the string "dogs",
C<-$x> returns "-dogs" even if C<$y=0+$x> has happened at some point.

=item *

In Perl 5.14, C<-'-10'> was fixed to return "10", not "+10".  But magical
variables (C<$1>, ties) were not fixed till now [perl #57706].

=item *

Unary negation now treats strings consistently, regardless of the internal
C<UTF8> flag.

=item *

A regression introduced in Perl v5.16.0 involving
C<tr/I<SEARCHLIST>/I<REPLACEMENTLIST>/> has been fixed.  Only the first
instance is supposed to be meaningful if a character appears more than
once in C<I<SEARCHLIST>>.  Under some circumstances, the final instance
was overriding all earlier ones.  [perl #113584]

=item *

Regular expressions like C<qr/\87/> previously silently inserted a NUL
character, thus matching as if it had been written C<qr/\00087/>.  Now it
matches as if it had been written as C<qr/87/>, with a message that the
sequence C<"\8"> is unrecognized.

=item *

C<__SUB__> now works in special blocks (C<BEGIN>, C<END>, etc.).

=item *

Thread creation on Windows could theoretically result in a crash if done
inside a C<BEGIN> block.  It still does not work properly, but it no longer
crashes [perl #111610].

=item *

C<\&{''}> (with the empty string) now autovivifies a stub like any other
sub name, and no longer produces the "Unable to create sub" error
[perl #94476].

=back

=head1 Known Problems

XXX Descriptions of platform agnostic bugs we know we can't fix go here.  Any
tests that had to be C<TODO>ed for the release would be noted here.  Unfixed
platform specific bugs also go here.

[ List each fix as a =item entry ]

=over 4

=item *

XXX

=back

=head1 Acknowledgements

XXX Generate this with:

  perl Porting/acknowledgements.pl v5.17.12..HEAD

=head1 Reporting Bugs

If you find what you think is a bug, you might check the articles recently
posted to the comp.lang.perl.misc newsgroup and the perl bug database at
http://rt.perl.org/perlbug/ .  There may also be information at
http://www.perl.org/ , the Perl Home Page.

If you believe you have an unreported bug, please run the L<perlbug> program
included with your release.  Be sure to trim your bug down to a tiny but
sufficient test case.  Your bug report, along with the output of C<perl -V>,
will be sent off to perlbug@perl.org to be analysed by the Perl porting team.

If the bug you are reporting has security implications, which make it
inappropriate to send to a publicly archived mailing list, then please send it
to perl5-security-report@perl.org.  This points to a closed subscription
unarchived mailing list, which includes all the core committers, who will be
able to help assess the impact of issues, figure out a resolution, and help
co-ordinate the release of patches to mitigate or fix the problem across all
platforms on which Perl is supported.  Please only use this address for
security issues in the Perl core, not for modules independently distributed on
CPAN.

=head1 SEE ALSO

The F<Changes> file for an explanation of how to view exhaustive details on
what changed.

The F<INSTALL> file for how to build Perl.

The F<README> file for general stuff.

The F<Artistic> and F<Copying> files for copyright information.

=cut