This is a live mirror of the Perl 5 development currently hosted at https://github.com/perl/perl5
Yves Orton [Fri, 9 Dec 2016 07:42:21 +0000 (08:42 +0100)]
use IVSIZE to see if we should use 64bit hashing
Craig A. Berry [Thu, 8 Dec 2016 19:14:19 +0000 (13:14 -0600)]
Escape extended characters in configure.com's manifest check.
Math::BigInt now has a file with '#' in the name. This broke the
build on VMS because the manifest checker couldn't find that file.
While the current file system can have files with a hash character
or other top-row-of-the-keyboard characters, most of them need
escaping with a caret when used in a filename in native syntax.
So add a more-or-less general purpose escaping routine and run it
on each filename in the manifest before converting from Unix to
VMS syntax.
Karl Williamson [Thu, 8 Dec 2016 02:04:52 +0000 (19:04 -0700)]
Revert "APItest/t/handy.t: Slightly simplify"
This reverts commit
96970a3c8cbe5ad489ce831380f0e208315ea651.
I misread the code and this commit should never have been done. The
statements I thought were identical did have some differences.
Steve Hay [Wed, 7 Dec 2016 08:32:05 +0000 (08:32 +0000)]
Upgrade bignum from version 0.44 to 0.45
Steve Hay [Wed, 7 Dec 2016 08:30:29 +0000 (08:30 +0000)]
Upgrade Math::BigRat from version 0.260805 to 0.2609
Steve Hay [Wed, 7 Dec 2016 08:28:53 +0000 (08:28 +0000)]
Upgrade Math::BigInt::FastCalc from version 0.5002 to 0.5003
Steve Hay [Wed, 7 Dec 2016 08:25:50 +0000 (08:25 +0000)]
Upgrade Math::BigInt from version 1.999802 to 1.999803
Sullivan Beck [Tue, 29 Nov 2016 20:56:59 +0000 (15:56 -0500)]
Bump Locale-Codes from 3.40 to 3.42
Yves Orton [Wed, 7 Dec 2016 22:40:13 +0000 (23:40 +0100)]
fixup new hash benchmarks to be lighter
Incorperate some changes suggested by Dave.
Yves Orton [Wed, 7 Dec 2016 22:36:38 +0000 (23:36 +0100)]
bench - fixup some interface issues from allowing two step benchmarks
This restores the old behavior of not showing a report if
the --write option is used, but now the --show option overrides
that behavior. It is an error to use --show with only one perl.
This also fixes the error if there are no perls provided, and
some other diagnostics.
David Mitchell [Wed, 7 Dec 2016 15:15:26 +0000 (15:15 +0000)]
perly.c: silence a -Wempty-body warning
perly.c:350:21: warning: suggest braces around empty body in an ‘if’ statement [
-Wempty-body]
David Mitchell [Wed, 7 Dec 2016 14:48:50 +0000 (14:48 +0000)]
t/perf/benchmarks: tweak cachegrind explanation
I pushed the previous commit before remembering to proofread what I
wrote.
David Mitchell [Wed, 7 Dec 2016 14:40:47 +0000 (14:40 +0000)]
t/perf/benchmarks: explain how benchmarks are run
I'd forgotten, and it confused other people, so explain how the
'setup' and 'code' fields are used to create a benchmark script.
Karl Williamson [Wed, 7 Dec 2016 05:28:59 +0000 (22:28 -0700)]
lib/locale.t: Up the permissible failure % for os390
Recent changes to the os390 locales have caused the number of failing
ones to exceed the cutoff of acceptable bad locales before the .t shows
failure.
os390 has more problematic locales than typical, because it has locales
for various IBM code pages, and some of these locales are incompatible
with perl. For example there is a CP 037 locale available which is
supposed to allow os390 (running CP 1047) to emulate machines where CP
037 is native. But these two code pages have different positions for
various critical characters that perl assumes have a particular ordinal
value, e.g. '['. That being placed differently in a locale means that
patterns with bracketed character classes won't work in perl on that
locale, and the locales fail locale.t
This commit changes the acceptable failing rate to 10% (up from 5% most
everywhere else) on os390, reflecting the higher percentage of base
incompatibilities.
Karl Williamson [Wed, 7 Dec 2016 04:34:32 +0000 (21:34 -0700)]
t/test.pl: Clarify syntax for tests in t/lib/*
Karl Williamson [Tue, 6 Dec 2016 19:12:22 +0000 (12:12 -0700)]
utf8.c: Remove unused variable
The called function can cope with a NULL parameter here, so no need to
use a dummy one.
Karl Williamson [Tue, 6 Dec 2016 19:10:34 +0000 (12:10 -0700)]
APItest/t/utf8.t: Fix EBCDIC test
One can tell if this code point is above Unicode from just the first
byte, unlike what it was specifying.
Karl Williamson [Sat, 3 Dec 2016 19:01:15 +0000 (12:01 -0700)]
toke.c: Swap 'if' and 'else' clauses
It is easier to read if the trivial case comes before the much longer
case, so complement the sense of this 'if', and swap the current 'else'
and 'if' clauses
Karl Williamson [Tue, 29 Nov 2016 01:01:42 +0000 (18:01 -0700)]
APItest/t/handy.t: Slightly simplify
This combines two adjacent 'if' blocks into a single one, as the 'if'
clause is identical in both.
Tony Cook [Tue, 25 Oct 2016 05:17:18 +0000 (16:17 +1100)]
(perl #129788) IO::Poll: fix memory leak
Whenever a magical/tied scalar which dies upon read was passed to _poll()
temporary buffer for events was not freed.
Adapted from a patch by Sergey Aleynikov <sergey.aleynikov@gmail.com>
Yves Orton [Tue, 6 Dec 2016 23:56:33 +0000 (00:56 +0100)]
update perldelta with hash function changes
Yves Orton [Tue, 6 Dec 2016 23:44:49 +0000 (00:44 +0100)]
cleanup hv_func.h - remove non-recommended hash variants
We dont need to offer so many options in terms of hash
function. Many, if not all, of the removed functions are
deficient in one way or another.
Yves Orton [Tue, 6 Dec 2016 23:20:05 +0000 (00:20 +0100)]
fixup pod in Porting/bench.pl
vim and podcheck.t didnt agree on how long the line was, so I just gave in and submitted
to its tyranny.
Yves Orton [Tue, 6 Dec 2016 22:48:52 +0000 (23:48 +0100)]
use a hybrid hash function, OAATH for short keys, Siphash 1-3 for longer ones
Switch to an optimized/unrolled variant of OAATH for keys
16 bytes and less, and use Siphash 1-3 for longer keys. (On
64 bit builds, 32 bit is untouched.)
I've done a bunch of benchmarking with Porting/bench.pl to
see at what point the 8 byte reads of Siphash 1-3 start to
win over the low overhead costs of OAATH. It seems to be around
16 bytes. At the same time, we unroll OAATH so that we can save
some instructions per character. The net result is all key lengths
get faster as measured by Porting/bench.pl, and hashing longer keys
becomes *much* faster.
Interestingly the actual crossover point of OAATH being slower
than Siphash 1-3 is much higher than might be guessed from bulk
testing either in a raw benchmark. For instance, basic benchmarks
of the hash function alone predicts that Siphash 1-3 should "win"
at around 4 bytes. However, in practice it is four times longer.
I believe this is because setting up Siphash blows away a fair
amount of CPU state compared to OAATH, which is more important
when the hash is going to be used to manage a data structure.
So it requires a correspondingly longer string before the larger
sized read starts to win.
Summarized stats (higher is better):
AVERAGE
blead yves
------ ------
Ir 100.00 104.47
Dr 100.00 103.97
Dw 100.00 107.63
COND 100.00 112.16
IND 100.00 75.07
COND_m 100.00 125.75
IND_m 100.00 97.42
Ir_m1 100.00 137.11
Dr_m1 100.00 100.10
Dw_m1 100.00 99.62
Ir_mm 100.00 100.00
Dr_mm 100.00 100.00
Dw_mm 100.00 100.00
Yves Orton [Tue, 6 Dec 2016 22:27:38 +0000 (23:27 +0100)]
Add some tests for hash keys of different sizes.
We test every key length from 1..24, and then various
larger sizes. We expect most of our keys to be short
so we check each length to see how performance is affected
as key length increases.
Yves Orton [Tue, 6 Dec 2016 22:16:44 +0000 (23:16 +0100)]
allow two-step benchmarks with Porting/bench.pl
with this patch you can time blead, and save the results:
bench.pl --tests=/whatever/ --write=blead.time -- ./perl=blead
then iterate over hacking the code and running
bench.pl --read=blead.time --write=last.time -- ./perl=hacked
and seeing the results of the --tests defined in the first run,
and the results from the save file 'blead.time' compared against
the modified version of perl in the most recent run. At the same
time the merged data is written as last.time, allowing the usual
post-test analysis of the results.
Yves Orton [Tue, 6 Dec 2016 21:29:03 +0000 (22:29 +0100)]
tweak perlmod.pod
James E Keenan [Tue, 6 Dec 2016 14:15:09 +0000 (09:15 -0500)]
Improve discussion of packages and their scopes.
For: RT #129345
Steve Hay [Tue, 6 Dec 2016 08:41:46 +0000 (08:41 +0000)]
Upgrade Encode from version 2.86 to 2.88
(Unicode.pm is customized for a version-bump only, to silence
t/porting/cmp_version.t since Unicode.xs has changed.)
David Mitchell [Mon, 5 Dec 2016 14:54:44 +0000 (14:54 +0000)]
assertion failure in ... or ((0) x 0))
[perl #130247] Perl_rpeep(OP *): Assertion `oldop' failed
the 'x 0' optimising code in rpeep didn't expect the repeat expression
to occur on the op_other side of an op_next chain.
David Mitchell [Mon, 5 Dec 2016 12:47:42 +0000 (12:47 +0000)]
add some /* NOTREACHED */
The previous commit added some
NOT_REACHED;
but these days the correct voodoo is, apparently,
NOT_REACHED; /* NOTREACHED */
to please as many compilers and static analysers as possible
Andy Lester [Sat, 3 Dec 2016 03:07:26 +0000 (22:07 -0500)]
Clean up warnings uncovered by 'clang -Weverything'.
For: RT #130195
David Mitchell [Mon, 5 Dec 2016 09:37:36 +0000 (09:37 +0000)]
optimise Perl_sv_gets(): use memchr() for loop
The inner loop which searches for the next separator character and copies
buffer bytes: replace with memchr() and Copy(). These functions are
likely to be optimised to use whatever hardware facilities are available.
(But first check that the first char in the buffer isn't the separator:
blank lines are quite common, and we can skip the overhead of calling
memchr() in that case).
David Mitchell [Sun, 4 Dec 2016 08:10:27 +0000 (08:10 +0000)]
yyparse: only calculate yytoken on yychar change
yytoken is a translated (via lookup table) version of parser->yychar.
So we only need to recalculate it when yychar changes (usually by
assigning the result of yylex() to it). This means when multiple
reductions are done without shifting another token, we skip the extra
overhead each time.
David Mitchell [Sat, 3 Dec 2016 20:58:37 +0000 (20:58 +0000)]
yyparse(): only check stack size in outer loop
Rather than checking before each individual shift whether the parse stack
needs extending, only check once per rule, making sure there's enough
space to shift all the items for the longest possible rule
David Mitchell [Sat, 3 Dec 2016 16:27:24 +0000 (16:27 +0000)]
yyparse: reindent
whitespace-only.
The previous commit wrapped the main body of code in a
while (1) { ...}
David Mitchell [Sat, 3 Dec 2016 16:23:55 +0000 (16:23 +0000)]
yyparse: replace yynewstate label with while(1) {}
makes the code easier to understand
David Mitchell [Sat, 3 Dec 2016 16:00:58 +0000 (16:00 +0000)]
yyparse: eliminate yyreduce label
Makes things slightly simpler.
David Mitchell [Sat, 3 Dec 2016 15:12:49 +0000 (15:12 +0000)]
yyparse: reindent
whitespace-only; previous commit wrapped a block of code in while (1){}
David Mitchell [Sat, 3 Dec 2016 15:11:11 +0000 (15:11 +0000)]
yyparse: replace some gotos with a while(1) loop
Just as efficient, and more readable.
Welcome to 1970's structured programming!
David Mitchell [Sat, 3 Dec 2016 14:08:56 +0000 (14:08 +0000)]
optimising yyparse: avoid a < 0 check
casting to unsigned allows (0 <= yyn <= YYLAST) to be done in a single
conditional.
David Mitchell [Sat, 3 Dec 2016 14:01:19 +0000 (14:01 +0000)]
optimising yyparse: replace stack_size with a ptr
Makes testing whether the parser stack needs extending cheaper
Yves Orton [Sat, 26 Nov 2016 19:12:41 +0000 (20:12 +0100)]
make sure that new heredoc parsing doesn't COW during prefix strip
Yves Orton [Fri, 18 Nov 2016 10:18:00 +0000 (11:18 +0100)]
fix sort order to be by key alone, not key and value
Sorting with the value part of the statement means that
"perl5=foo" sorts before "perl=foo", where sorting by the
keys alone would put "perl" before "perl5".
This makes the order in the data correspond to what you
would see with sort keys %Config.
Chris 'BinGOs' Williams [Sat, 3 Dec 2016 07:34:55 +0000 (07:34 +0000)]
Update bignum to CPAN version 0.44
[DELTA]
2016-11-15 pjacklam 0.44
* Update bundled Module::Install from version 1.16 to version 1.17.
* Replace function calls with method calls.
* Include patch for CPAN RT #116506.
Chris 'BinGOs' Williams [Sat, 3 Dec 2016 07:25:54 +0000 (07:25 +0000)]
Update Math-BigRat to CPAN version 0.260805
[DELTA]
2016-11-15 v0.260805 pjacklam
* Standardize on using $LIB for the variable holding the class of the backend
library.
* Improve bmul() by doing _gcd() twice on smaller numbers rather than once on
larger numbers.
Chris 'BinGOs' Williams [Sat, 3 Dec 2016 07:23:23 +0000 (07:23 +0000)]
Update Math-BigInt-FastCalc to CPAN version 0.5002
[DELTA]
2016-11-23 v0.5002 pjacklam
* Sync test files with Math-BigInt-1.999801. Add t/biglog.t and t/bigroot.t
since these are included in the test suite for the other backend libraries
(GMP and Pari).
2016-11-17 v0.5001 pjacklam
* Avoid 'use parent' to retain compatibility with old Perls.
* Fix incorrect version numbers in the CHANGES file.
* Trim whitespace.
2016-11-15 v0.5000 pjacklam
* Sync test files with Math-BigInt-1.999800.
* Math::BigInt::FastCalc is now a subclass of Math::BigInt::Calc, so remove
aliases like *Math::BigInt::FastCalc::_xxx = \&Math::BigInt::Calc::_xxx.
* Use OO-calls rather than function calls.
* Fix documentation error in Math::BigInt::FastCalc
Chris 'BinGOs' Williams [Sat, 3 Dec 2016 07:14:13 +0000 (07:14 +0000)]
Update Math-BigInt to CPAN version 1.999802
[DELTA]
2016-11-15 v1.999800 pjacklam
* Upgrade bundled Module::Install from version 1.16 to version 1.17.
* Add Math::BigInt::Lib (lib/Math/BigInt/Lib.pm), a parent class for
Math::BigInt backend libraries.
* Use objects in Math::BigInt::Calc, not just array refs. Also use OO-style,
i.e., use $class->_add($x, $y) rather than _add($class, $x, $y).
* Not all library methods modify the invocand, so call library methods as,
e.g, $x = $LIB->method($x, $y) rather than just $LIB->method($x, $y).
* Math::BigInt::Calc is now a subclass of Math::BigInt::Lib.
* Add Math::BigInt::Lib::Minimal (t/Math/BigInt/Lib/Minimal.pm) for testing
inheritance from Math::BigInt::Lib.
* Minor simplification in Math::BigInt::Calc->_str().
* Speed up Math::BigInt::Calc->_root().
* Remove test files that were included in the previous release by accident.
* Add more tests and use more verbose output in some tests.
* Fix typo in lib/Math/BigFloat.pm
* Fix documentation error in lib/Math/Calc.pm
* Use Config::Tiny and an .ini file to handle the library specific
configuration for the author-lib*.t test files.
2016-11-23 v1.999801 pjacklam
* Fix, hopefully once and for all, the longstanding problem of handling undef
as an operand to mathematical methods. The only method that accepts undef as
an operand is blog(), where the second operand might be undef, as in
$x->blog() or $x->blog($b), where $b is undef. The undef signifies that
Euler's number should be used as the base. With this fix, we should be able
to get Math::BigInt::Lite working again.
* Add least common multiple method _lcm() to Math::BigInt::Lib, and add
corresponding test file t/author-lib-arithmetic-binary-_lcm.t and test data
file t/author-lib-arithmetic-binary-_lcm.dat.
* Remove internal function __lcm() which has become redundant now that _lcm()
is in the library.
* Make it possible to use bgcd() and blcm() as class methods, since other
methods can be used as class methods. This applies to both Math::BigInt and
Math::BigFloat.
* Fix blcm() with negative input. The LCM should always be non-negative. This
applies to both Math::BigInt and Math::BigFloat.
* Add tests for bgcd() and blcm() in t/bigintpm.t and t/bigfltpm.t.
* Fix tests for blcm() assuming that LCM(0,0) should be a NaN. LCM(0,0) is 0
by convention.
* Prefer Class->config('option') over Class->config()->{option}. However, this
does not seem to be working for all options. It seems that this won't work
properly until we move the global variables into the OO interface.
* Explicitly specify the library in all test files that are shared between
Math-BigInt and the library distributions (FaatCalc, GMP, Pari, ...) with,
e.g., "use Math::BigInt only => 'Calc';". This way, it will fail immediately
if the specified library can't be loaded rather than using the fallback
library.
2016-11-28 v1.999802 pjacklam
* When bzero(), bone(), binf(), and bnan() are used as constructors, don't
check whether the class allows the object to be modified. This applies to
both Math::BigInt and Math::BigFloat.
* Improve bgcd() and blcm(). This applies to both Math::BigInt and
Math::BigFloat.
Chris 'BinGOs' Williams [Sat, 3 Dec 2016 06:47:12 +0000 (06:47 +0000)]
Update Test-Simple to CPAN version 1.302067
[DELTA]
1.302067 2016-11-23 07:37:56-08:00 America/Los_Angeles
- Fix context test for recent blead.
1.302066 2016-11-08 07:58:39-08:00 America/Los_Angeles (TRIAL RELEASE)
- Handle cases where SysV IPC can be available but not enabled
- Import 'context' into Test2::IPC, it is used by 'cull'
- Propogate warnings settings to use_ok (#736)
1.302065 2016-10-30 11:54:37-07:00 America/Los_Angeles (TRIAL RELEASE)
- Set the TEST_ACTIVE env var to true
- Set the TEST2_ACTIVE env var to true
- Fix the oldest bug still in the bug list (#6)
This fixes cmp_ok output is some confusing cases
- Update travis config
- Add missing author deps
- Fix handling of negative pid's on windows
- Add can() to Test::Tester::Delegate (despite deprecation)
- Fix some minor test issues
1.302064 2016-10-24 21:03:24-07:00 America/Los_Angeles (TRIAL RELEASE)
- Repo management improvements
- Better handling of info vs diag in ->send_event
- Fix test that used 'parent'
- Better handling of non-bumping failures (#728)
1.302063 2016-10-23 21:31:20-07:00 America/Los_Angeles (TRIAL RELEASE)
- Fix double release when 'throw' is used in context_do()
Chris 'BinGOs' Williams [Sat, 3 Dec 2016 06:45:17 +0000 (06:45 +0000)]
Reconcile Time-HiRes with CPAN release
Chris 'BinGOs' Williams [Sat, 3 Dec 2016 06:42:34 +0000 (06:42 +0000)]
Update Time-Local to CPAN version 1.25
[DELTA]
1.25 2016-11-17
- Reduce memory usage by only loading Config if needed and not importing from
Carp. Based on PR #2 from J. Nick Coston.
Chris 'BinGOs' Williams [Sat, 3 Dec 2016 06:38:51 +0000 (06:38 +0000)]
Update Unicode-Collate to CPAN version 1.19
[DELTA]
1.19 Sat Dec 3 09:32:31 2016
- U::C::Locale newly supports locales: he, vo.
- locales updated to CLDR 24: az, haw.
- locale updated to CLDR 26: et.
Andy Lester [Sat, 3 Dec 2016 02:51:14 +0000 (21:51 -0500)]
Move miscategorized files.
The pod/.gitignore file has sections of files that should be ignored.
Three of the files are in the wrong section, in the
architecture-specific section instead of the generated-files section.
This changes no functionality, but may help future maintainers from
confusion.
For: RT #130194
Petr Písař [Mon, 28 Nov 2016 12:06:24 +0000 (13:06 +0100)]
Silent const correctnes warnings in utf8_hop functions
GCC -Wcast-qual option reports a const violation in utf8_hop
functions. They take a pointer to constant data but returns pointer to
non-constant data.
It's impossible to fix this without changing their prototype.
Therefore this patch asks a compiler to ignore the violations.
Signed-off-by: Petr Písař <ppisar@redhat.com>
Petr Písař [Thu, 24 Nov 2016 16:12:41 +0000 (17:12 +0100)]
Fix const correctness in utf8.h
The original code was generated and then hand-tunes. Therefore
I edited the code in place instead of fixing the regen/regcharclass.pl
generator.
Signed-off-by: Petr Písař <ppisar@redhat.com>
Petr Písař [Thu, 24 Nov 2016 15:34:09 +0000 (16:34 +0100)]
Fix const correctness in hv_func.h
Building an XS code with -Wcast-qual yielded warnings about discarding
const qualifiers from pointer targets like:
$ printf '#include "EXTERN.h"\n#include "perl.h"\n' | gcc -Wcast-qual -I/usr/lib64/perl5/CORE -c -x c -
In file included from /usr/lib64/perl5/CORE/hv.h:629:0,
from /usr/lib64/perl5/CORE/perl.h:3740,
from <stdin>:2:
/usr/lib64/perl5/CORE/hv_func.h: In function ‘S_perl_hash_siphash_2_4’:
/usr/lib64/perl5/CORE/hv_func.h:213:17: warning: cast discards ‘const’ qualifier from pointer target type [-Wcast-qual]
U64TYPE k0 = ((U64TYPE*)seed)[0];
^
Signed-off-by: Petr Písař <ppisar@redhat.com>
David Mitchell [Wed, 30 Nov 2016 08:59:01 +0000 (08:59 +0000)]
split was leaving PL_sv_undef in unused ary slots
This:
@a = split(/-/,"-");
$a[1] = undef;
$a[0] = 0;
was giving
Modification of a read-only value attempted at foo line 3.
This is because:
1) unused slots in AvARRAY between AvFILL and AvMAX should always be
null; av_clear(), av_extend() etc do this; while av_store(), if storing
to a slot N somewhere between AvFILL and AvMAX, doesn't bother to clear
between (AvFILL+1)..(N-1) on the assumption that everyone else plays
nicely.
2) pp_split() when splitting directly to an array, sometimes over-splits
and has to null out the excess elements;
3) Since perl 5.19.4, unused AV slots are now marked with NULL rather than
&PL_sv_undef;
4) pp_split was still using &PL_sv_undef;
The fault was with (4), and is easily fixed.
Andy Lester [Wed, 30 Nov 2016 05:39:08 +0000 (16:39 +1100)]
Remove support for Splint static source code analyzer
Splint has not been updated since 2007 and doesn’t even build for me.
As far as I know, I'm the only person who's ever worked with Splint on
Perl 5.
Here's what changes:
* Makefile target "splint"
* Macros in XSUB.h and perl.h
* Support in regen/embed.pl
Karl Williamson [Tue, 29 Nov 2016 18:28:52 +0000 (11:28 -0700)]
lib/locale.t: Improve test when NUL isn't a control
Commit
535a3fb3ec9051c531a7797f1de40cbfc39e3f7f changed lib/locale.t to
not fail on locales that don't have NUL be a control. It did this by
skipping the failing test. However, I realized later that there is a
way to handle the situation without skipping a test, and that is to add
NUL to the list of controls if it isn't already there.
Andy Lester [Tue, 29 Nov 2016 00:33:23 +0000 (11:33 +1100)]
A few small tweaks in time64.c
* The struct TM * pointers to S_check_tm and S_timegm64 functions is
not modified, so make it be a const and let the compiler know it won't
get modified.
* Localize variables to their innermost scopes
* Consting various local variables
Karl Williamson [Mon, 28 Nov 2016 22:22:08 +0000 (15:22 -0700)]
Change name of PL_ variable
This variable really means the character that replaces any embedded NULs
when doing collation. Change the name accordingly. (Embedded NULs must
be replaced because the libc function strxfrm is used, and it operates
on C strings which have no embedded NULs.)
Karl Williamson [Mon, 28 Nov 2016 16:09:23 +0000 (09:09 -0700)]
PATCH: [perl #129953] lib/locale.t failures on FREEBSD
I thought this bug was in FREEBSD, but when I went to gather the info
needed to report it to the vendor, it turned out to be a mistake I had
made.
The problem is basically doubly encoding into UTF-8. In order to save
CPU time, in a UTF-8 locale, I had stored a string as UTF-8 encoded.
This string is to be inserted into a larger string. What I neglected to
consider in this situation is that not all strings in such locales need
be in UTF-8. The UTF-8 encoded insert could get added to a non-UTF-8
string, and the result later was switched to UTF-8, so the inserted
string's bytes were individually converted to UTF-8, effectively a
second time. This is a problem only if the inserted string is different
when encoded in UTF-8 than not, and for this particular usage, on most
platforms it was UTF-8 invariant, so did not show up, except on those
platforms where it was variant.
The solution is to store the replacement as a code point, and encode it
as UTF-8 only if necessary, once. This actually simplifies the code.
Karl Williamson [Sun, 27 Nov 2016 17:25:49 +0000 (10:25 -0700)]
locale.c: Add some comments
Karl Williamson [Mon, 28 Nov 2016 21:16:18 +0000 (14:16 -0700)]
lib/locale.t: Don't assume NUL is a control
A test that assumed NUL would be considered a control fails in locales
where it isn't considered a control.
This was found on FREEBSD, where the locale "hi_IN.ISCII-DEV" is
defective (based on the Wikipedia article on ISCII
https://en.wikipedia.org/wiki/Indian_Script_Code_for_Information_Interchange).
Only the code points 0x09-0x0D are considered controls in this
implementation, whereas the article says ISCII is a superset of ASCII,
so should have 33 controls, not just 5. (Unrelated to this ticket, but
another apparent defect I saw is that this implementation defines 0x91,
but the article says that code point is unassigned.)
Karl Williamson [Sat, 29 Oct 2016 18:24:21 +0000 (12:24 -0600)]
lib/locale.t: Improve skipping of incompatible locales
Perl works properly with only a subset of the possible locales out
there. This improves the detection of those and the skipping.
Tony Cook [Mon, 28 Nov 2016 23:26:55 +0000 (10:26 +1100)]
J. Nick Koston is now a perl AUTHOR
Tony Cook [Mon, 28 Nov 2016 23:24:15 +0000 (10:24 +1100)]
bump $Data::Dumper::VERSION and update Changes entries
J. Nick Koston [Mon, 14 Nov 2016 09:57:14 +0000 (03:57 -0600)]
dist/Data-Dumper/Dumper.pm: Reduce memory usage by removing overload and not importing Carp
David Mitchell [Mon, 28 Nov 2016 08:03:49 +0000 (08:03 +0000)]
crash on explicit return from s///e
RT #130188
In
sub f {
my $x = 'a';
$x =~ s/./return;/e;
}
the 'return' triggers popping any contexts above the subroutine context:
in this case, a CXt_SUBST context. In this case, Perl_dounwind() calls
cx_popblock() for the bottom-most popped context, to restore any saved
vars. However, CXt_SUBST is the one context type which *doesn't* use
'struct block' as part of its context struct union, so you can't
cx_popblock() a CXt_SUBST context.
This commit makes it skip the cx_popblock() in this case.
Bug was introduced by me with v5.23.7-235-gfc6e609.
Aristotle Pagaltzis [Fri, 25 Nov 2016 01:04:52 +0000 (02:04 +0100)]
perlfunc/open: clarify role of mode for undef filename
Karl Williamson [Thu, 24 Nov 2016 17:54:23 +0000 (10:54 -0700)]
utf8.c: Fix EBCDIC detection of above-31 bit code points
This was failing on EBCDIC machines, and I couldn't figure out why. I
didn't see any flaws in the logic. It required special debugging code
to answer. It turns out it was a bad declaration, and not logic at all.
The declaration should have been
const U8 prefix[] = ...
instead it was
const U8 * const prefix = ...
sizeof() the latter is 4, as it is looking at pointer size. What was
intended was the size of the array this was initialized to, which was
longer. What this led to was later in the routine, a comparison was
stopping too early.
Karl Williamson [Wed, 23 Nov 2016 03:30:56 +0000 (20:30 -0700)]
APItest/t/utf8.t: White space only
This indents the new block formed by the previous commit. However,
since the indentation is getting too much, it also changes the indents
for all the nested for loops to 2 spaces to allow room on the line.
Karl Williamson [Wed, 23 Nov 2016 00:47:35 +0000 (17:47 -0700)]
Split diagnostics for two UTF-8 malformations
Some UTF-8 sequences may have multiple malformations. Commit
2b5e7bc2e60b4c4b5d87aa66e066363d9dce7930 tried to make sure that all
possible ones are raised, instead of abandoning searching after one is
found. Since, I realized that there was yet another case of two
malformations that it returned only one or the other of.
An input buffer may be too short to fully express the code point it
purports to. This can be determined by the first byte of the UTF-8
sequence indicating a longer sequence is requred than the space
available. But also, that shortened sequence can have a premature
beginning of another character earlier than the shortness. This commit
causes these to be both raised, instead of the previous behavior of
noting just one.
Karl Williamson [Wed, 23 Nov 2016 01:14:45 +0000 (18:14 -0700)]
APItest/t/utf8.t: Partially refactor to use table data
This removes kludgy code that was trying, given a partial
character, to determine if there enough bytes present to guarantee that
the whole character must belong to a class of characters or not. Now
the necessary length to make that determination has instead manually
been placed in a table, so it can be looked up. In doing so, I
corrected one length that was failing on EBCDIC.
Karl Williamson [Wed, 23 Nov 2016 01:07:31 +0000 (18:07 -0700)]
APItest/t/utf8.t: Fix test
It turns out that this test has two malformations, and should only have
one; a future commit will remove the masking of the 2nd one.
Karl Williamson [Wed, 23 Nov 2016 01:01:21 +0000 (18:01 -0700)]
APItest/t/utf8.t: Comments only
Karl Williamson [Wed, 23 Nov 2016 00:55:10 +0000 (17:55 -0700)]
APItest/t/utf8.t: Add some indentation to diagnositcs
This is so they don't interrupt reading the output when there are
errors.
Karl Williamson [Tue, 22 Nov 2016 20:15:18 +0000 (13:15 -0700)]
utf8.c: Clarify warning message.
This warning was changed recently in the 5.25 series, and has not been
in a stable release.
Karl Williamson [Mon, 21 Nov 2016 21:59:47 +0000 (14:59 -0700)]
APItest/t/utf8.t: Simplify expression slightly
Karl Williamson [Sun, 20 Nov 2016 14:56:40 +0000 (07:56 -0700)]
APItest/t/handy.t: Output details if test fails
There should be no warnings generated, but if there are, we want to see
what they were.
David Mitchell [Thu, 24 Nov 2016 14:42:27 +0000 (14:42 +0000)]
add comments explaining the point of SVf_PROTECT
The flag was added by v5.21.4-17-ga623f89, but outside of the commit
message itself, there didn't seem to be any explanation for this flag.
David Mitchell [Thu, 24 Nov 2016 13:52:30 +0000 (13:52 +0000)]
POSIX.pm: shut up g++ compiler warning
g++ is too stupid to know that strtoul(..,&unparsed,...) will modify
unparsed. So initialise it to NULL.
POSIX.xs: In function ‘void XS_POSIX_strtoul(PerlInterpreter*, CV*)’:
POSIX.xs:3468:17: warning: ‘unparsed’ may be used uninitialized in this function [-Wmaybe-uninitialized]
if (unparsed)
David Mitchell [Thu, 24 Nov 2016 09:40:44 +0000 (09:40 +0000)]
add sv_set_undef() API function
This function is equivalent to sv_setsv(sv, &PL_sv_undef), but more
efficient.
Also change the obvious places in the core to use the new idiom.
David Mitchell [Tue, 22 Nov 2016 16:41:54 +0000 (16:41 +0000)]
avoid premature free of referent in list assign
RT #130132
My recent commit v5.25.6-266-ga083329 made it so that perl could
sometimes avoid mortalising the referent when assigning to a reference
(e.g. for $ref1 = $ref2, where $$ref1 has a ref count of 1).
Unfortunately it turns out that list assign relied on this behaviour
to avoid premature freeing, e.g.
($ref1, $x) = ($y, $$ref1);
where $$ref1 needs to continue to live for at least the rest of the
assign.
This commit fixes it by mortalising the referent in pp_assign when
required.
Chris 'BinGOs' Williams [Thu, 24 Nov 2016 11:49:29 +0000 (11:49 +0000)]
This one goes up to 11
Karl Williamson [Tue, 22 Nov 2016 18:30:28 +0000 (11:30 -0700)]
embed.fnc: Remove inappropriate pure attributes
In looking at the diffs from
f6e4c377437f9b2ded79fb55f66983a6d4f761f9,
just committed, I realized that some of the functions marked pure,
really have side effects, if only initialization if not called
previously. Still, the compiler could optimize things so it is wrong.
I then did a manual audit of embed.fnc, and removed the pure attribute
of the functions that I found that have some potential side effects.
Andy Lester [Tue, 22 Nov 2016 13:17:45 +0000 (06:17 -0700)]
Make (P)ure funcs in embed.fnc imply (R)eturn must be checked
embed.fnc defines that a pure function is one that has no effects other
than its return value. Therefore, calling such a function without using
the return value must be an error.
This patch makes a "P" "pure function" flag also imply the "R" "return
value must be checked" flag.
Niko Tyni [Sat, 19 Nov 2016 07:45:45 +0000 (09:45 +0200)]
Configure: also zero out high bytes of 80-bit ldnan
These are currently zero anyway, but things are probably not guaranteed
to stay so.
Niko Tyni [Fri, 18 Nov 2016 16:36:34 +0000 (18:36 +0200)]
Configure: fix garbage filtering with 80-bit long doubles
The test had several problems that resulted in the excess
bytes not getting zeroed out. This caused random contents in
$Config{longdblinfbytes}, observed on Debian with GCC 6.2.0 (but not
5.4.1).
Bug-Debian: https://bugs.debian.org/844752
Matthew Horsfall [Mon, 21 Nov 2016 14:07:21 +0000 (09:07 -0500)]
Recommend the quicker "make test_prep" for t/TEST usage.
Chad Granum [Sun, 20 Nov 2016 22:05:26 +0000 (14:05 -0800)]
Bump Module::CoreList version
Chad Granum [Sun, 20 Nov 2016 22:02:47 +0000 (14:02 -0800)]
Bump the perl version in various places for 5.25.8
Chad Granum [Sun, 20 Nov 2016 21:30:09 +0000 (13:30 -0800)]
new perldelta
Chad Granum [Sun, 20 Nov 2016 21:29:08 +0000 (13:29 -0800)]
Update epigraphs and release schedule
Chad Granum [Sun, 20 Nov 2016 20:10:44 +0000 (12:10 -0800)]
update t/porting/known_pod_issues.dat
Chad Granum [Sun, 20 Nov 2016 19:21:53 +0000 (11:21 -0800)]
Update perlhist.pod for 5.25.7
Chad Granum [Sun, 20 Nov 2016 19:17:09 +0000 (11:17 -0800)]
Finalize perldelta for 5.25.7
Following release manager doc
Chad Granum [Sun, 20 Nov 2016 18:51:39 +0000 (10:51 -0800)]
Update Module::CoreLiast for 5.25.7
Chad Granum [Sun, 20 Nov 2016 18:49:18 +0000 (10:49 -0800)]
Fix Module-CoreList version info for 5.25.7
Chad Granum [Sun, 20 Nov 2016 15:52:19 +0000 (07:52 -0800)]
Final manual updates to perldelta
This will still likeyl get minor changes in the rest of the release
process, but the pre-release editing is done.