perldelta: Blanket entry for memory leaks

[perl5.git] / pod / perl5171delta.pod

=encoding utf8

=head1 NAME

perl5171delta - what is new for perl v5.17.1

=head1 DESCRIPTION

This document describes differences between the 5.17.0 release and
the 5.17.1 release.

If you are upgrading from an earlier release such as 5.16.0, first read
L<perl5170delta>, which describes differences between 5.16.0 and
5.17.0.

=head1 Core Enhancements

=head2 More CORE:: subs

Several more built-in functions have been added as subroutines to the
CORE:: namespace, namely, those non-overridable keywords that can be
implemented without custom parsers: C<defined>, C<delete>, C<exists>,
C<glob>, C<pos>, C<protoytpe>, C<scalar>, C<split>, C<study>, and C<undef>.

As some of these have prototypes, C<prototype('CORE::...')> has been
changed to not make a distinction between overridable and non-overridable
keywords.  This is to make C<prototype('CORE::pos')> consistent with
C<prototype(&CORE::pos)>.

=head1 Incompatible Changes

=head2 C</(?{})/> and C</(??{})/> have been heavily reworked

The implementation of this feature has been almost completely rewritten.
Although its main intent is to fix bugs, some behaviors, especially
related to the scope of lexical variables, will have changed.  This is
described more fully in the L</Selected Bug Fixes> section.

=head2 C<\N{BELL}> now refers to U+1F514 instead of U+0007

Unicode 6.0 reused the name "BELL" for a different code point than it
traditionally had meant.  Since Perl v5.14, use of this name still
referred to U+0007, but would raise a deprecation warning.  Now, "BELL"
refers to U+1F514, and the name for U+0007 is "ALERT".  All the
functions in L<charnames> have been correspondingly updated.

=head2 Alphanumeric operators must now be separated from the closing
delimiter of regular expressions

You may no longer write something like:

 m/a/and 1

Instead you must write

 m/a/ and 1

with whitespace separating the operator from the closing delimiter of
the regular expression.  Not having whitespace has resulted in a
deprecation warning since Perl v5.14.0.

=head2 C<require> dies for unreadable files

When C<require> encounters an unreadable file, it now dies.  It used to
ignore the file and continue searching the directories in @INC
[perl #113422].

=head2 Upgrade to the Unicode 6.2 beta

Unicode 6.2 is proposing some changes that may very well break some CPAN
modules.  The timing of this nicely coincides with Perl's being early in the
release cycle.  This commit takes the current beta 6.2, adds the proposed
changes that aren't yet in it, and subtracts the changes that would affect \X
processing, as those turn out to have errors, and may have to be rethought.
Unicode has been notified of these problems.

This will allow us to gather data as to whether or not the proposed changes
cause us problems.  These will be presented to Unicode to aid in their final
decision as to whether or not to go forward with the changes.

These changes will be replaced by the final version of Unicode 6.2 before
5.18.0 is released.

=head1 Performance Enhancements

=over 4

=item *

The C<x> repetition operator is now folded to a single constant at compile
time if called in scalar context with constant operands and no parentheses
around the left operand.

=back

=head1 Modules and Pragmata

=head2 Updated Modules and Pragmata

=over 4

=item *

L<Archive::Extract> has been upgraded from version 0.58 to 0.60.

Work around an edge case on Linux with Busybox's unzip.

=item *

L<Archive::Tar> has been upgraded from version 1.82 to 1.88.

ptar now supports the -T option as well as dashless options
[rt.cpan.org #75473], [rt.cpan.org #75475].

Auto-encode filenames marked as UTF-8 [rt.cpan.org #75474].

Don't use C<tell> on L<IO::Zlib> handles [rt.cpan.org #64339].

Don't try to C<chown> on symlinks.

=item *

L<attributes> has been upgraded from version 0.19 to 0.20.

=item *

L<autodie> has been upgraded from version 2.10 to 2.11.

=item *

L<B> has been upgraded from version 1.35 to 1.36.

C<B::COP::stashlen> has been replaced with C<B::COP::stashoff>.

C<B::COP::stashpv> now supports UTF-8 package names and embedded NULs.

=item *

L<B::Deparse> has been upgraded from version 1.14 to 1.15.

Avoid warning when run under C<perl -w>.

=item *

L<Class::Struct> has been upgraded from version 0.63 to 0.64.

The constructor now respects overridden accessor methods [perl #29230].

=item *

L<Compress::Raw::Bzip2> has been upgraded from version 2.048 to 2.052.

=item *

L<Compress::Raw::Zlib> has been upgraded from version 2.048 to 2.054.

Upgrade bundled zlib to version 1.2.7.

Fix build failures on Irix, Solaris, and Win32, and also when building as C++
[rt.cpan.org #69985], [rt.cpan.org #77030], [rt.cpan.org #75222].

=item *

L<CPAN::Meta> has been upgraded from version 2.120630 to 2.120921.

=item *

L<CPAN::Meta::Requirements> has been upgraded from version 2.120630 to 2.122.

Treat undef requirements to C<from_string_hash> as 0 (with a warning).

Added C<requirements_for_module> method.

=item *

L<CPAN::Meta::YAML> has been upgraded from version 0.007 to 0.008.

=item *

L<CPANPLUS> has been upgraded from version 0.9121 to 0.9130.

Allow adding F<blib/script> to PATH.

Save the history between invocations of the shell.

Handle multiple C<makemakerargs> and C<makeflags> arguments better.

Use C<File::HomeDir> when available, and provide C<PERL5_CPANPLUS_HOME> to
override the autodetection.

Always re-fetch F<CHECKSUMS> if C<fetchdir> is set.

=item *

L<Digest::MD5> has been upgraded from version 2.51 to 2.52.

Fix C<Digest::Perl::MD5> OO fallback [rt.cpan.org #66634].

=item *

L<DynaLoader> has been upgraded from version 1.14 to 1.15.

This is due to a minor code change in the XS for the VMS implementation.

=item *

L<ExtUtils::CBuilder> has been upgraded from version 0.280206 to 0.280208.

Manifest files are now correctly embedded for those versions of VC++ which
make use of them. [perl #111782, #111798].

=item *

L<File::DosGlob> has been upgraded from version 1.07 to 1.08.

=item *

L<File::Fetch> has been upgraded from version 0.32 to 0.34.

=item *

L<File::Spec::Unix> has been upgraded from version 3.39_02 to 3.39_03.

C<abs2rel> could produce incorrect results when given two relative paths or
the root directory twice [perl #111510].

=item *

L<Filter::Util::Call> has been upgraded from version 1.40 to 1.45.

=item *

L<HTTP::Tiny> has been upgraded from version 0.017 to 0.022.

Add SSL verification features [github #6], [github #9].

Include the final URL in the response hashref.

Add C<local_address> option.

=item *

L<IO> has been upgraded from version 1.25_06 to 1.25_07.

C<sync()> can now be called on read-only file handles [perl #64772].

=item *

L<IPC::Cmd> has been upgraded from version 0.76 to 0.78.

Use C<POSIX::_exit> instead of C<exit> in C<run_forked> [rt.cpan.org #76901].

=item *

L<Memoize> has been upgraded from version 1.02 to 1.03.

Fix the C<MERGE> cache option.

=item *

L<Module::Build> has been upgraded from version 0.39_01 to 0.40.

Fixed bug where modules without C<$VERSION> might have a version of '0' listed
in 'provides' metadata, which will be rejected by PAUSE.

Fixed bug in PodParser to allow numerals in module names.

Fixed bug where giving arguments twice led to them becoming arrays, resulting
in install paths like F<ARRAY(0xdeadbeef)/lib/Foo.pm>.

=item *

L<Module::CoreList> has been upgraded from version 2.67 to 2.68.

=item *

L<Module::Load::Conditional> has been upgraded from version 0.46 to 0.50.

Fix use of C<requires> on perls installed to a path with spaces.

=item *

L<Object::Accessor> has been upgraded from version 0.42 to 0.44.

=item *

L<Params::Check> has been upgraded from version 0.32 to 0.36.

=item *

L<Parse::CPAN::Meta> has been upgraded from version 1.4402 to 1.4404.

=item *

L<PerlIO::mmap> has been upgraded from version 0.010 to 0.011.

=item *

L<PerlIO::via::QuotedPrint> has been upgraded from version 0.06 to 0.07.

=item *

L<Pod::Html> has been upgraded from version 1.15_02 to 1.16.

The option C<--libpods> has been reinstated. It is deprecated, and its use
does nothing other than issue a warning that it is no longer supported.

Since the HTML files generated by pod2html claim to have a UTF-8 charset,
actually write the files out using UTF-8 [perl #111446].

=item *

L<POSIX> has been upgraded from version 1.30 to 1.31.

=item *

L<re> has been upgraded from version 0.19 to 0.20.

See note about C<op_comp> in the L</Internal Changes> section below.

=item *

L<Safe> has been upgraded from version 2.31_01 to 2.33_01.

Fix interactions with C<Devel::Cover>.

Don't eval code under C<no strict>.

=item *

L<Scalar::Util> has been upgraded from version 1.23 to version 1.25.

Fix an overloading issue with C<sum>.

C<first> and C<reduce> now check the callback first (so C<&first(1)> is
disallowed).

Fix C<tainted> on magical values [rt.cpan.org #55763].

Fix C<sum> on previously magical values [rt.cpan.org #61118].

Fix reading past the end of a fixed buffer [rt.cpan.org #72700].

=item *

L<Search::Dict> has been upgraded from version 1.04 to 1.07.

No longer require C<stat> on filehandles.

Use C<fc> for casefolding.

=item *

L<Storable> has been upgraded from version 2.35 to 2.36.

=item *

L<Term::ANSIColor> has been upgraded from version 3.01 to 3.02.

Add support for italics.

Improve error handling.

=item *

L<Test::Harness> has been upgraded from version 3.23 to 3.25.

Fix glob semantics on Win32 [rt.cpan.org #49732].

Don't use C<Win32::GetShortPathName> when calling perl [rt.cpan.org #47890].

Ignore -T when reading shebang [rt.cpan.org #64404].

Handle the case where we don't know the wait status of the test more
gracefully.

Make the test summary 'ok' line overridable so that it can be changed to a
plugin to make the output of prove idempotent.

Don't run world-writable files.

=item *

L<Unicode::UCD> has been upgraded from version 0.43 to 0.44.

This adds a function L<all_casefolds()|Unicode::UCD/all_casefolds()>
that returns all the casefolds.

=back

=head1 Documentation

=head2 Changes to Existing Documentation

=head3 L<perlfaq>

=over 4

=item *

L<perlfaq> has been synchronized with version 5.0150040 from CPAN.

=back

=head3 L<perlcheat>

=over 4

=item *

L<perlcheat> has been reorganized, and a few new sections were added.

=back

=head1 Diagnostics

=head2 Removals of Diagnostics

=over 4

=item *

The "Runaway prototype" warning that occurs in bizarre cases has been
removed as being unhelpful and inconsistent.

=item *

The "Not a format reference" error has been removed, as the only case in
which it could be triggered was a bug.

=item *

The "Unable to create sub named %s" error has been removed for the same
reason.

=back

=head1 Platform Support

=head2 Platform-Specific Notes

=over 4

=item Win32

C<link> on Win32 now attempts to set C<$!> to more appropriate values
based on the Win32 API error code. [perl #112272]

Perl no longer mangles the environment block, e.g. when launching a new
sub-process, when the environment contains non-ASCII characters. Known
problems still remain, however, when the environment contains characters
outside of the current ANSI codepage (e.g. see the item about Unicode in
C<%ENV> in L<http://perl5.git.perl.org/perl.git/blob/HEAD:/Porting/todo.pod>).
[perl #113536]

=item VMS

All C header files from the top-level directory of the distribution are now
installed on VMS, providing consistency with a long-standing practice on other
platforms. Previously only a subset were installed, which broke non-core
extension builds for extensions that depended on the missing include files.

=back

=head1 Internal Changes

=over 4

=item *

The C<study> function was made a no-op in 5.16.  It was simply disabled via
a C<return> statement; the code was left in place.  Now the code supporting
what C<study> used to do has been removed.

=item *

Under threaded perls, there is no longer a separate PV allocated for every
COP to store its package name (C<< cop->stashpv >>).  Instead, there is an
offset (C<< cop->stashoff >>) into the new C<PL_stashpad> array, which
holds stash pointers.

=item *

In the pluggable regex API, the C<regexp_engine> struct has acquired a new
field C<op_comp>, which is currently just for perl's internal use, and
should be initialised to NULL by other regex plugin modules.

=item *

A new function C<alloccoptash> has been added to the API, but is considered
experimental.  See L<perlapi>.

=back

=head1 Selected Bug Fixes

=over 4

=item *

The implementation of code blocks in regular expressions, such as C<(?{})>
and C<(??{})>, has been heavily reworked to eliminate a whole slew of bugs.
The main user-visible changes are:

=over 4

=item *

Code blocks within patterns are now parsed in the same pass as the
surrounding code; in particular it is no longer necessary to have balanced
braces: this now works:

    /(?{  $x='{'  })/

This means that this error message is longer generated:

    Sequence (?{...}) not terminated or not {}-balanced in regex

but a new error may be seen:

    Sequence (?{...}) not terminated with ')'

In addition, literal code blocks within run-time patterns are only
compiled once, at perl compile-time:

    for my $p (...) {
        # this 'FOO' block of code is compiled once,
	# at the same time as the surrounding 'for' loop
        /$p{(?{FOO;})/;
    }

=item *

Lexical variables are now sane as regards scope, recursion and closure
behavior. In particular, C</A(?{B})C/> behaves (from a closure viewpoint)
exactly like C</A/ && do { B } && /C/>, while  C<qr/A(?{B})C/> is like
C<sub {/A/ && do { B } && /C/}>. So this code now works how you might
expect, creating three regexes that match 0, 1, and 2:

    for my $i (0..2) {
        push @r, qr/^(??{$i})$/;
    }
    "1" =~ $r[1]; # matches

=item *

The C<use re 'eval'> pragma is now only required for code blocks defined
at runtime; in particular in the following, the text of the C<$r> pattern is
still interpolated into the new pattern and recompiled, but the individual
compiled code-blocks within C<$r> are reused rather than being recompiled,
and C<use re 'eval'> isn't needed any more:

    my $r = qr/abc(?{....})def/;
    /xyz$r/;

=item *

Flow control operators no longer crash. Each code block runs in a new
dynamic scope, so C<next> etc. will not see
any enclosing loops. C<return> returns a value
from the code block, not from any enclosing subroutine.

=item *

Perl normally caches the compilation of run-time patterns, and doesn't
recompile if the pattern hasn't changed, but this is now disabled if
required for the correct behavior of closures. For example:

    my $code = '(??{$x})';
    for my $x (1..3) {
	# recompile to see fresh value of $x each time
        $x =~ /$code/;
    }


=item *

The C</msix> and C<(?msix)> etc. flags are now propagated into the return
value from C<(??{})>; this now works:

    "AB" =~ /a(??{'b'})/i;

=item *

Warnings and errors will appear to come from the surrounding code (or for
run-time code blocks, from an eval) rather than from an C<re_eval>:

    use re 'eval'; $c = '(?{ warn "foo" })'; /$c/;
    /(?{ warn "foo" })/;

formerly gave:

    foo at (re_eval 1) line 1.
    foo at (re_eval 2) line 1.

and now gives:

    foo at (eval 1) line 1.
    foo at /some/prog line 2.

=back

=item *

Perl now works as well as can be expected on all releases of Unicode so
far.  In v5.16, it worked on Unicodes 6.0 and 6.1, but there were
various bugs for earlier releases; the older the release the more
problems.

=item *

C<vec> no longer produces "uninitialized" warnings in lvalue context
[perl #9423].

=item *

An optimization involving fixed strings in regular expressions could cause
a severe performance penalty in edge cases.  This has been fixed
[perl #76546].

=item *

In certain cases, including empty subpatterns within a regular expression (such
as C<(?:)> or C<(?:|)>) could disable some optimizations. This has been fixed.

=item *

The "Can't find an opnumber" message that C<prototype> produces when passed
a string like "CORE::nonexistent_keyword" now passes UTF-8 and embedded
NULs through unchanged [perl #97478].

=item *

C<prototype> now treats magical variables like C<$1> the same way as
non-magical variables when checking for the CORE:: prefix, instead of
treating them as subroutine names.

=item *

Under threaded perls, a runtime code block in a regular expression could
corrupt the package name stored in the op tree, resulting in bad reads
in C<caller>, and possibly crashes [perl #113060].

=item *

Referencing a closure prototype (C<\&{$_[1]}> in an attribute handler for a
closure) no longer results in a copy of the subroutine (or assertion
failures on debugging builds).

=item *

C<eval '__PACKAGE__'> now returns the right answer on threaded builds if
the current package has been assigned over (as in
C<*ThisPackage:: = *ThatPackage::>) [perl #78742].

=item *

If a package is deleted by code that it calls, it is possible for C<caller>
to see a stack frame belonging to that deleted package.  C<caller> could
crash if the stash's memory address was reused for a scalar and a
substitution was performed on the same scalar [perl #113486].

=item *

C<UNIVERSAL::can> no longer treats its first argument differently
depending on whether it is a string or number internally.

=item *

C<open> with C<< <& >> for the mode checks to see whether the third argument is
a number, in determining whether to treat it as a file descriptor or a handle
name.  Magical variables like C<$1> were always failing the numeric check and
being treated as handle names.

=item *

C<warn>'s handling of magical variables (C<$1>, ties) has undergone several
fixes.  C<FETCH> is only called once now on a tied argument or a tied C<$@>
[perl #97480].  Tied variables returning objects that stringify as "" are
no longer ignored.  A tied C<$@> that happened to return a reference the
I<previous> time it was used is no longer ignored.

=item *

C<warn ""> now treats C<$@> with a number in it the same way, regardless of
whether it happened via C<$@=3> or C<$@="3">.  It used to ignore the
former.  Now it appends "\t...caught", as it has always done with
C<$@="3">.

=item *

Numeric operators on magical variables (e.g., S<C<$1 + 1>>) used to use
floating point operations even where integer operations were more appropriate,
resulting in loss of accuracy on 64-bit platforms [perl #109542].

=item *

Unary negation no longer treats a string as a number if the string happened
to be used as a number at some point.  So, if C<$x> contains the string "dogs",
C<-$x> returns "-dogs" even if C<$y=0+$x> has happened at some point.

=item *

In Perl 5.14, C<-'-10'> was fixed to return "10", not "+10".  But magical
variables (C<$1>, ties) were not fixed till now [perl #57706].

=item *

Unary negation now treats strings consistently, regardless of the internal
C<UTF8> flag.

=item *

A regression introduced in Perl v5.16.0 involving
C<tr/I<SEARCHLIST>/I<REPLACEMENTLIST>/> has been fixed.  Only the first
instance is supposed to be meaningful if a character appears more than
once in C<I<SEARCHLIST>>.  Under some circumstances, the final instance
was overriding all earlier ones.  [perl #113584]

=item *

Regular expressions like C<qr/\87/> previously silently inserted a NUL
character, thus matching as if it had been written C<qr/\00087/>.  Now it
matches as if it had been written as C<qr/87/>, with a message that the
sequence C<"\8"> is unrecognized.

=item *

C<__SUB__> now works in special blocks (C<BEGIN>, C<END>, etc.).

=item *

Thread creation on Windows could theoretically result in a crash if done
inside a C<BEGIN> block.  It still does not work properly, but it no longer
crashes [perl #111610].

=item *

C<\&{''}> (with the empty string) now autovivifies a stub like any other
sub name, and no longer produces the "Unable to create sub" error
[perl #94476].

=back

=head1 Known Problems

=over 4

=item *

On VMS, L<Module::Build> still fails its test suite.

=item *

On Win32, several tests fail intermittently, and may hang unless STDERR is
redirected.

=back

=head1 Acknowledgements

Perl 5.17.1 represents approximately 4 weeks of development since Perl 5.17.0
and contains approximately 37,000 lines of changes across 710 files from 35
authors.

Perl continues to flourish into its third decade thanks to a vibrant community
of users and developers. The following people are known to have contributed the
improvements that became Perl 5.17.1:

?, Andy Dougherty, Aristotle Pagaltzis, Breno G. de Oliveira, Brian Fraser,
Chris 'BinGOs' Williams, Craig A. Berry, David Mitchell, Dominic Hargreaves,
Evan Miller, Father Chrysostomos, Florian Ragwitz, H.Merijn Brand, Herbert
Breunung, Hugo van der Sanden, Jesse Luehrs, Karl Williamson, Karthik
Rajagopalan, Lukas Mai, Martin Hasch, Michael Schroeder, Nicholas Clark, Paul
Johnson, Reini Urban, Renee Baecker, Rhesa Rozendaal, Ricardo Signes, Shlomi
Fish, Steve Hay, Steve Peters, Sullivan Beck, Tony Cook, Volker Schatz, Yves
Orton, Zefram.

The list above is almost certainly incomplete as it is automatically generated
from version control history. In particular, it does not include the names of
the (very much appreciated) contributors who reported issues to the Perl bug
tracker.

Many of the changes included in this version originated in the CPAN modules
included in Perl's core. We're grateful to the entire CPAN community for
helping Perl to flourish.

For a more complete list of all of Perl's historical contributors, please see
the F<AUTHORS> file in the Perl source distribution.

=head1 Reporting Bugs

If you find what you think is a bug, you might check the articles
recently posted to the comp.lang.perl.misc newsgroup and the perl
bug database at http://rt.perl.org/perlbug/ .  There may also be
information at http://www.perl.org/ , the Perl Home Page.

If you believe you have an unreported bug, please run the L<perlbug>
program included with your release.  Be sure to trim your bug down
to a tiny but sufficient test case.  Your bug report, along with the
output of C<perl -V>, will be sent off to perlbug@perl.org to be
analysed by the Perl porting team.

If the bug you are reporting has security implications, which make it
inappropriate to send to a publicly archived mailing list, then please send
it to perl5-security-report@perl.org. This points to a closed subscription
unarchived mailing list, which includes
all the core committers, who will be able
to help assess the impact of issues, figure out a resolution, and help
co-ordinate the release of patches to mitigate or fix the problem across all
platforms on which Perl is supported. Please only use this address for
security issues in the Perl core, not for modules independently
distributed on CPAN.

=head1 SEE ALSO

The F<Changes> file for an explanation of how to view exhaustive details
on what changed.

The F<INSTALL> file for how to build Perl.

The F<README> file for general stuff.

The F<Artistic> and F<Copying> files for copyright information.

=cut