Test out-of-bounds warning with lv substr

[perl5.git] / pod / perldelta.pod

=encoding utf8

=for comment
This has been completed up to a74fb2cdc8f, except for
e032854 khw [perl #32080] is_utf8_string() reads too far

=head1 NAME

[ this is a template for a new perldelta file. Any text flagged as
XXX needs to be processed before release. ]

perldelta - what is new for perl v5.15.6

=head1 DESCRIPTION

This document describes differences between the 5.15.5 release and
the 5.15.6 release.

If you are upgrading from an earlier release such as 5.15.4, first read
L<perl5155delta>, which describes differences between 5.15.4 and
5.15.5.

=head1 Notice

XXX Any important notices here

=head1 Core Enhancements

XXX New core language features go here. Summarise user-visible core language
enhancements. Particularly prominent performance optimisations could go
here, but most should go in the L</Performance Enhancements> section.

[ List each enhancement as a =head2 entry ]

=head2 C<__SUB__>

The new C<__SUB__> token, available under the "current_sub" feature (see
L<feature>) or C<use v5.15>, returns a reference to the current subroutine,
making it easier to write recursive closures.

=head2 New option for the debugger's B<t> command

The B<t> command in the debugger, which toggles tracing mode, now accepts a
numerical argument that determines how many levels of subroutine calls to
trace.

=head1 Security

XXX Any security-related notices go here.  In particular, any security
vulnerabilities closed should be noted here rather than in the
L</Selected Bug Fixes> section.

[ List each security issue as a =head2 entry ]

=head1 Incompatible Changes

XXX For a release on a stable branch, this section aspires to be:

    There are no changes intentionally incompatible with 5.XXX.XXX
    If any exist, they are bugs and reports are welcome.

[ List each incompatible change as a =head2 entry ]

=head2 C<substr> lvalue revamp

When C<substr> is called in lvalue or potential lvalue context with two or
three arguments, a special lvalue scalar is returned that modifies the
original string (the first argument) when assigned to.

Previously, the offsets (the second and third arguments) passed to
C<substr> would be converted immediately to match the string, negative
offsets being translated to positive and offsets beyond the end of the
string being truncated.

Now, the offsets are recorded without modification in the special lvalue
scalar that is returned, and the original string is not even looked at by
C<substr> itself, but only when the returned lvalue is read or modified.

These changes result in several incompatible changes and bug fixes:

=over

=item *

If the original string changes length after the call to C<substr> but
before assignment to its return value, negative offsets will remember
their position from the end of the string, affecting code like this:

    my $string = "string";
    my $lvalue = \substr $string, -4, 2;
    print $lvalue, "\n"; # prints "ri"
    $string = "bailing twine";
    print $lvalue, "\n"; # prints "wi"; used to print "il"

The same thing happens with an omitted third argument.  The returned lvalue
will always extend to the end of the string, even if the string becomes
longer.

=item *

Tied (and otherwise magical) variables are no longer exempt from the
"Attempt ot use reference as lvalue in substr" warning.

=item *

That warning now occurs when the returned lvalue is assigned to, not when
C<substr> itself is called.  This only makes a difference if the return
value of C<substr> is referenced and assigned to later.

=item *

The order in which "uninitialized" warnings occur for arguments to
C<substr> has changed.

=item *

Passing a substring of a read-only value or a typeglob to a function (potential lvalue context) no longer causes an immediate "Can't coerce" or "Modification of a read-only value" error.  That error only occurs if and
when the value passed is assigned to.

The same thing happens with the "substr outside of string" error.

=item *

C<substr> assignments no longer call FETCH twice if the first argument is a
tied variable, but just once.

=back

It was impossible to fix all the bugs without an incompatible change, and
the behaviour of negative offsets was never specified, so the change was
deemed acceptable.

=head2 XS API tweak

The C<newCONSTSUB_flags> C-level function, added in 5.15.4, now has a
C<len> parameter.

=head1 Deprecations

XXX Any deprecated features, syntax, modules etc. should be listed here.
In particular, deprecated modules should be listed here even if they are
listed as an updated module in the L</Modules and Pragmata> section.

[ List each deprecation as a =head2 entry ]

=head1 Performance Enhancements

XXX Changes which enhance performance without changing behaviour go here. There
may well be none in a stable release.

[ List each enhancement as a =item entry ]

=over 4

=item *

Perl 5.12.0 sped up the destruction of objects whose classes define empty
C<DESTROY> methods (to prevent autoloading), simply by not calling such
empty methods.  This release takes this optimisation a step further, by not
calling any C<DESTROY> method that begins with an C<return> statement.
This can be useful for destructors that are only used for debugging:

    use constant DEBUG => 1;
    sub DESTROY { return unless DEBUG; ... }

Constant-folding will reduce the first statement to C<return;> if DEBUG is
set to 0, triggering this optimisation.

=item *

Assign to a variable that holds a typeglob or copy-on-write scalar is now
much faster.  Previously the typeglob would be stringified or the
copy-on-write scalar would be copied before being clobbered.

=back

=head1 Modules and Pragmata

XXX All changes to installed files in F<cpan/>, F<dist/>, F<ext/> and F<lib/>
go here.  If Module::CoreList is updated, generate an initial draft of the
following sections using F<Porting/corelist-perldelta.pl>, which prints stub
entries to STDOUT.  Results can be pasted in place of the '=head2' entries
below.  A paragraph summary for important changes should then be added by hand.
In an ideal world, dual-life modules would have a F<Changes> file that could be
cribbed.

[ Within each section, list entries as a =item entry ]

=head2 New Modules and Pragmata

=over 4

=item *

XXX

=back

=head2 Updated Modules and Pragmata

=over 4

=item *

L<Archive::Tar> has been upgraded from version 1.80 to version 1.82.

Adjustments to handle files >8gb (>0777777777777 octal) and a feature to
return the MD5SUM of files in the archive.

=item *

L<AutoLoader> has been upgraded from version 5.71 to version 5.72.

=item *

L<B::Debug> has been upgraded from version 1.16 to version 1.17.

=item *

L<B::Deparse> has been upgraded from version 1.09 to 1.10.

C<sort(foo(bar))> is now deparsed correctly. (C<sort foo(bar)>, how it used
to deparse, makes foo the sort routine, rather than a regular function
call.)

=item *

L<Compress::Raw::Zlib> has been upgraded from version 2.042 to version 2.045.

=item *

L<Compress::Raw::Bzip2> has been upgraded from version 2.042 to version 2.045.

=item *

L<Data::Dumper> has been upgraded from version 2.134 to 2.135.

The XS implementation has been updated to account for the Unicode symbol
changes in Perl 5.15.4.  It also knows how to output typeglobs with nulls
in their names.

=item *

L<ExtUtils::ParseXS> has been upgraded from version 3.05 to version 3.07.

=item *

L<IO::Compress::Base> has been upgraded from version 2.042 to version 2.045.

Added zipdetails utility.

=item *

L<Locale::Codes> has been upgraded from version 3.18 to version 3.20.

The code2XXX, XXX2code, all_XXX_codes, and all_XXX_names functions now support retired codes.
All codesets may be specified by a constant or by their name now. Previously,
they were specified only by a constant.
The alias_code function exists for backward compatibility. It has been replaced by rename_country_code.
The alias_code function will be removed sometime after September, 2013.
All work is now done in the central module (Locale::Codes). Previously, some was still done in the
wrapper modules (Locale::Codes::*) but that is gone now.
Added Language Family codes (langfam) as defined in ISO 639-5.

=item *

L<Module::Loaded> has been uprgaded from version 0.06 to version 0.08.

=item *

L<Pod::LaTeX> has been upgraded from version 0.59 to version 0.60.

Added another LaTeX escape: --- => -{}-{}-

Pod::LaTeX doesn't handle -- in PODs specially, passing it directly to
LaTeX, which then proceeds to replace it with a single -. This patch
replaces ----- with -{}-{}-{}-{}-

=item *

L<Unicode::Collate> has been upgraded from version 0.85 to version 0.87.

Tailored compatibility ideographs as well as unified ideographs for
the locales: ja, ko, zh__big5han, zh__gb2312han, zh__pinyin, zh__stroke.

Now Locale/*.pl files are searched in @INC.

=back

=head2 Removed Modules and Pragmata

=over 4

=item *

XXX

=back

=head1 Documentation

XXX Changes to files in F<pod/> go here.  Consider grouping entries by
file and be sure to link to the appropriate page, e.g. L<perlfunc>.

=head2 New Documentation

XXX Changes which create B<new> files in F<pod/> go here.

=head3 L<XXX>

XXX Description of the purpose of the new file here

=head2 Changes to Existing Documentation

XXX Changes which significantly change existing files in F<pod/> go here.
However, any changes to F<pod/perldiag.pod> should go in the L</Diagnostics>
section.

=head3 L<XXX>

=over 4

=item *

XXX Description of the change here

=back

=head1 Diagnostics

The following additions or changes have been made to diagnostic output,
including warnings and fatal error messages.  For the complete list of
diagnostic messages, see L<perldiag>.

XXX New or changed warnings emitted by the core's C<C> code go here. Also
include any changes in L<perldiag> that reconcile it to the C<C> code.

[ Within each section, list entries as a =item entry that links to perldiag,
  e.g.

  =item *

  L<Invalid version object|perldiag/"Invalid version object">
]

=head2 New Diagnostics

XXX Newly added diagnostic messages go here

=head3 New Errors

=over 4

=item *

XXX L<message|perldiag/"message">

=back

=head3 New Warnings

=over 4

=item *

XXX L<message|perldiag/"message">

=back

=head2 Changes to Existing Diagnostics

XXX Changes (i.e. rewording) of diagnostic messages go here

=over 4

=item *

Redefinition warnings for constant subroutines used to be mandatory, even
occurring under C<no warnings>.  Now they respect the L<warnings> pragma.

=item *

The "Attempt to free non-existent shared string" has had the spelling of
"non-existent" corrected to "nonexistent".  It was already listed with the
correct spelling in L<perldiag>.

=item *

The 'Use of "foo" without parentheses is ambiguous' warning has been
extended to apply also to user-defined subroutines with a (;$) prototype,
and not just to built-in functions.

=back

=head1 Utility Changes

XXX Changes to installed programs such as F<perlbug> and F<xsubpp> go
here. Most of these are built within the directories F<utils> and F<x2p>.

[ List utility changes as a =head3 entry for each utility and =item
entries for each change
Use L<XXX> with program names to get proper documentation linking. ]

=head3 L<zipdetails>

=over 4

=item *

L<zipdetails> displays information about the internal record structure of the zip file.
It is not concerned with displaying any details of the compressed data stored in the zip file.

=back

=head1 Configuration and Compilation

XXX Changes to F<Configure>, F<installperl>, F<installman>, and analogous tools
go here.  Any other changes to the Perl build process should be listed here.
However, any platform-specific changes should be listed in the
L</Platform Support> section, instead.

[ List changes as a =item entry ].

=over 4

=item *

F<pod/roffitall> is now build by F<pod/buildtoc>, instead of being shipped
with the distribution. Its list of manpages is now generated (and therefore
current). See also RT #103202 for an unresolved related issue.

=item *

Perl 5.15.5 had a bug in its installation script, which did not install
F<unicore/Name.pm>.  This has been corrected [perl #104226].

XXX Is that Perl version correct?  Is the file path correct?

=back

=head1 Testing

XXX Any significant changes to the testing of a freshly built perl should be
listed here.  Changes which create B<new> files in F<t/> go here as do any
large changes to the testing harness (e.g. when parallel testing was added).
Changes to existing files in F<t/> aren't worth summarising, although the bugs
that they represent may be covered elsewhere.

[ List each test improvement as a =item entry ]

=over 4

=item *

The F<substr.t> and F<substr_thr.t> scripts for testing C<substr> have been
moved under F<t/op/>, where they were originally.  They had been moved
under F<t/re/> along with the substitution tests when that directory was
created.

=back

=head1 Platform Support

XXX Any changes to platform support should be listed in the sections below.

[ Within the sections, list each platform as a =item entry with specific
changes as paragraphs below it. ]

=head2 New Platforms

XXX List any platforms that this version of perl compiles on, that previous
versions did not. These will either be enabled by new files in the F<hints/>
directories, or new subdirectories and F<README> files at the top level of the
source tree.

=over 4

=item XXX-some-platform

XXX

=back

=head2 Discontinued Platforms

XXX List any platforms that this version of perl no longer compiles on.

=over 4

=item XXX-some-platform

XXX

=back

=head2 Platform-Specific Notes

XXX List any changes for specific platforms. This could include configuration
and compilation changes or changes in portability/compatibility.  However,
changes within modules for platforms should generally be listed in the
L</Modules and Pragmata> section.

=over 4

=item XXX-some-platform

XXX

=back

=head1 Internal Changes

XXX Changes which affect the interface available to C<XS> code go here.
Other significant internal changes for future core maintainers should
be noted as well.

[ List each change as a =item entry ]

=over 4

=item *

XXX

=back

=head1 Selected Bug Fixes

XXX Important bug fixes in the core language are summarised here.
Bug fixes in files in F<ext/> and F<lib/> are best summarised in
L</Modules and Pragmata>.

[ List each fix as a =item entry ]

=over 4

=item *

A constant subroutine assigned to a glob whose name contains a null will no
longer cause extra globs to pop into existence when the constant is
referenced under its new name.

=item *

C<sort> was not treating C<sub {}> and C<sub {()}> as equivalent when such
a sub was provided as the comparison routine.  It used to croak on
C<sub {()}>.

=item *

Subroutines from the C<autouse> namespace are once more exempt from
redefinition warnings.  This used to work in 5.005, but was broken in 5.6
for most subroutines.  For subs created via XS that redefine subroutines
from the C<autouse> package, this stopped working in 5.10.

=item *

New XSUBs now produce redefinition warnings if they overwrite existing
subs, as they did in 5.8.x.  (The C<autouse> logic was reversed in 5.10-14.
Only subroutines from the C<autouse> namespace would warn when clobbered.)

=item *

Redefinition warnings triggered by the creation of XSUBs now respect
Unicode glob names, instead of using the internal representation.  This was
missed in 5.15.4, partly because this warning was so hard to trigger.  (See
the previous item.)

=item *

C<newCONSTSUB> used to use compile-time warning hints, instead of run-time
hints.  The following code should never produce a redefinition warning, but
it used to, if C<newCONSTSUB> redefine and existing subroutine:

    use warnings;
    BEGIN {
	no warnings;
	some_XS_function_that_calls_new_CONSTSUB();
    }

=item *

Redefinition warnings for constant subroutines are on by default (what are
known as severe warnings in L<perldiag>).  This was only the case when it
was a glob assignment or declaration of a Perl subroutine that caused the
warning.  If the creation of XSUBs triggered the warning, it was not a
default warning.  This has been corrected.

=item *

The internal check to see whether a redefinition warning should occur used
to emit "uninitialized" warnings in cases like this:

    use warnings "uninitialized";
    use constant {u=>undef,v=>undef};
    sub foo(){u} sub foo(){v}

=item *

A bug fix in Perl 5.14 introduced a new bug, causing "uninitialized"
warnings to report the wrong variable if the operator in question has
two operands and one is C<%{...}> or C<@{...}>.  This has been fixed
[perl #103766].

=item *

C<< version->new("version") >> and C<printf "%vd", "version"> no longer
crash [perl #102586].

=item *

C<$tied =~ y/a/b/>, C<chop $tied> and C<chomp $tied> now call FETCH just
once when $tied holds a reference.

=item *

Four-argument C<select> now always calls FETCH on tied arguments.  It used
to skip the call if the tied argument happened to hold C<undef> or a
typeglob.

=item *

Four-argument C<select> no longer produces its "Non-string passed as
bitmask" warning on tied or tainted variables that are strings.

=item *

C<sysread> now always calls FETCH on the buffer passed to it if it is tied.
It used to skip the call if the tied variable happened to hold a typeglob.

=item *

C<< $tied .= <> >> now calls FETCH once on C<$tied>.  It used to call it
multiple times if the last value assigned to or returned from the tied
variable was anything other than a string or typeglob.

=item *

The C<evalbytes> keyword added in 5.15.5 was respecting C<use utf8>
declarations from the outer scope, when it should have been ignoring them.

=back

=head1 Known Problems

XXX Descriptions of platform agnostic bugs we know we can't fix go here. Any
tests that had to be C<TODO>ed for the release would be noted here, unless
they were specific to a particular platform (see below).

This is a list of some significant unfixed bugs, which are regressions
from either 5.XXX.XXX or 5.XXX.XXX.

[ List each fix as a =item entry ]

=over 4

=item *

XXX

=back

=head1 Obituary

XXX If any significant core contributor has died, we've added a short obituary
here.

=head1 Acknowledgements

XXX Generate this with:

  perl Porting/acknowledgements.pl v5.15.5..HEAD

=head1 Reporting Bugs

If you find what you think is a bug, you might check the articles
recently posted to the comp.lang.perl.misc newsgroup and the perl
bug database at http://rt.perl.org/perlbug/ .  There may also be
information at http://www.perl.org/ , the Perl Home Page.

If you believe you have an unreported bug, please run the L<perlbug>
program included with your release.  Be sure to trim your bug down
to a tiny but sufficient test case.  Your bug report, along with the
output of C<perl -V>, will be sent off to perlbug@perl.org to be
analysed by the Perl porting team.

If the bug you are reporting has security implications, which make it
inappropriate to send to a publicly archived mailing list, then please send
it to perl5-security-report@perl.org. This points to a closed subscription
unarchived mailing list, which includes
all the core committers, who will be able
to help assess the impact of issues, figure out a resolution, and help
co-ordinate the release of patches to mitigate or fix the problem across all
platforms on which Perl is supported. Please only use this address for
security issues in the Perl core, not for modules independently
distributed on CPAN.

=head1 SEE ALSO

The F<Changes> file for an explanation of how to view exhaustive details
on what changed.

The F<INSTALL> file for how to build Perl.

The F<README> file for general stuff.

The F<Artistic> and F<Copying> files for copyright information.

=cut