This is a live mirror of the Perl 5 development currently hosted at https://github.com/perl/perl5
Tony Cook [Mon, 8 Aug 2016 05:11:55 +0000 (15:11 +1000)]
[perl #123981] Travis-CI support
This is largely based on work done by "c9s <yoanlin93@gmail.com>" and
Dennis Kaarsemaker (Seveas)
Tony Cook [Wed, 14 Jun 2017 05:02:33 +0000 (15:02 +1000)]
perldelta for
e80af1fd276d
Tony Cook [Tue, 11 Oct 2016 23:42:47 +0000 (10:42 +1100)]
(perl 129183) don't treat \ as an escape in PATH for -S
Tony Cook [Tue, 13 Jun 2017 23:42:31 +0000 (09:42 +1000)]
(perl #131526) don't go beyond the end of the NUL in my_atof2
Perl_my_atof2() calls GROK_NUMERIC_RADIX() to detect and skip past
a decimal point and then can increment the parse pointer (s) before
checking what it points at, so skipping the terminating NUL if the
decimal point is immediately before the NUL.
David Mitchell [Mon, 12 Jun 2017 15:26:02 +0000 (16:26 +0100)]
keep PERL_HASH_SEED_BYTES as an int
adding a sizeof() expression was converting it into an unsigned long,
which was triggering this warning in Hash-Util:
Util.xs:99:45: warning: format specifies type 'int' but the argument has type 'unsigned long' [-Wformat]
David Mitchell [Mon, 12 Jun 2017 12:31:22 +0000 (13:31 +0100)]
gv.c, mg.c: fix 32-bit compiler warnings
mg.c:641:21: warning: cast from pointer to integer of different size
[-Wpointer-to-int-cast]
UV uv = (UV)mg->mg_obj;
It's storing a char-ranged integer value as an SV*. By using SSize_t
rather than UV to store the char, it avoids mismatched size warnings.
David Mitchell [Mon, 12 Jun 2017 12:05:23 +0000 (13:05 +0100)]
Storable: fix a couple of 32-bit warnings
Storable.xs: In function 'retrieve_blessed':
Storable.xs:1015:39: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
else if (PerlIO_read(cxt->fio, x, y) != y) { \
^
Storable.xs:4078:2: note: in expansion of macro 'SAFEPVREAD'
SAFEPVREAD(classname, len, malloced_classname);
David Mitchell [Mon, 12 Jun 2017 11:57:27 +0000 (12:57 +0100)]
PerlIO-scalar: fix 32-bit compiler warning
David Mitchell [Mon, 12 Jun 2017 10:42:22 +0000 (11:42 +0100)]
fix utf char > IV_MAX on 32-bit platforms
Commit v5.27.0-132-g13f4dd3 forbade out of range Unicode code points,
and fixed/removed tests that no longer apply.
However, this was probably only tested on a 64-bit platform. Several
tests now fail on 32-bit platforms, due to things like "\x{
80000000}".
This commit fixes up those tests too.
Andy Dougherty [Mon, 12 Jun 2017 12:02:10 +0000 (08:02 -0400)]
Simpler hints fix for [perl #131337].
The Configure scan fails to find dlopen() with g++. Explicitly making
it availble allows Configure to default to using dynamic loading, but
still allows the user to override and use static loading.
Chris 'BinGOs' Williams [Mon, 12 Jun 2017 12:36:19 +0000 (13:36 +0100)]
Update ExtUtils-MakeMaker to CPAN version 7.30
[DELTA]
7.30 Mon Jun 12 13:17:29 BST 2017
No changes since v7.29_02
7.29_02 Sun Jun 11 12:00:33 BST 2017
Doc fixes:
- Fixed POD errors in FAQ
7.29_01 Wed May 31 08:34:10 BST 2017
Bug fixes:
- Prune .a without extralibs.ld from search paths
Karl Williamson [Sun, 11 Jun 2017 14:31:06 +0000 (08:31 -0600)]
perldelta: Use new boiler plate
This uses for the current perldelta, the boiler plate from the perldelta
template added in the previous commit.
It also documents a significant test change in the current development
release.
Karl Williamson [Sun, 11 Jun 2017 14:20:21 +0000 (08:20 -0600)]
Porting/perldelta_template.pod: Add boiler plate
This adds text about documentation and testing changes, as agreed to in
the thread starting with
http://nntp.perl.org/group/perl.perl5.porters/244582
Karl Williamson [Fri, 12 May 2017 00:30:34 +0000 (18:30 -0600)]
utf8.c: Add missing 'const' to formal parameter
This makes it match the embed.fnc entry. For some reason no warning was
given except by the Win32 compiler
Chris 'BinGOs' Williams [Sat, 10 Jun 2017 11:29:23 +0000 (12:29 +0100)]
Update Test-Harness to CPAN version 3.39
[DELTA]
3.39 06-04-2017
- Make tests pass when PERL_USE_UNSAFE_INC=0
James E Keenan [Fri, 9 Jun 2017 23:06:57 +0000 (19:06 -0400)]
commit
c93f220f failed to update DirHandle's $VERSION.
Aristotle Pagaltzis [Wed, 7 Sep 2016 20:20:52 +0000 (22:20 +0200)]
DirHandle: document as obsolete
Aristotle Pagaltzis [Sat, 23 Apr 2016 18:43:00 +0000 (20:43 +0200)]
Regularise @_ unpacking in Module::CoreList
Mainly this means regularising the awful heuristic for detecting when
a function is being called as a method.
Chris 'BinGOs' Williams [Fri, 9 Jun 2017 16:41:20 +0000 (17:41 +0100)]
Update ExtUtils-MakeMaker to CPAN version 7.28
[DELTA]
7.28 Tue May 30 22:01:08 BST 2017
No changes since v7.27_02
7.27_02 Tue May 30 09:27:56 BST 2017
Bug fixes:
- Prune auto/share from search paths RT#121918
7.27_01 Sun May 28 11:35:46 BST 2017
Bug fixes:
- Fix regression with metadata RT#121913
7.26 Sat May 27 21:01:47 BST 2017
No changes since 7.25_06
7.25_06 Tue May 23 20:18:01 BST 2017
Bug fixes:
- Fix regression with XS tests on MSWin32 with MS toolchain
7.25_05 Mon May 15 10:18:01 BST 2017
Bug fixes:
- Make MakeMaker pass compilation tests on AIX again
- Test, fix test dep on SKIPped linktype
7.25_04 Fri May 12 12:24:09 BST 2017
Enhancements:
- Add os_unsupported() function
7.25_03 Thu May 11 17:51:23 BST 2017
Bug fixes:
- processPL now depends on 'pure_all' instead of 'all'
7.25_02 Thu May 11 11:54:42 BST 2017
Bug fixes:
- Only add staticlibs that are installed under auto/
- Correct the order of tests of chmod()
Doc fixes:
- Fixed typo in MakeMaker.pm
7.25_01 Fri Feb 3 13:36:25 GMT 2017
Bug fixes:
- Make perllocal.pod files reproducible
- META_ADD/MERGE default meta version based on each other
- Eliminate an ancient, unneeded, dangerous call to Carp::longmess
Karl Williamson [Sat, 3 Jun 2017 15:08:50 +0000 (09:08 -0600)]
Make LOCK_LC_NUMERIC_STANDARD recursive
Same for UNLOCK_LC_NUMERIC_STANDARD.
This partially fixes [perl #128207]
Karl Williamson [Tue, 6 Jun 2017 07:54:46 +0000 (01:54 -0600)]
utf8.c: Clarify pod for three functions
utf8_to_bytes(), bytes_from_utf8(), bytes_to_utf8()
Karl Williamson [Tue, 6 Jun 2017 07:45:32 +0000 (01:45 -0600)]
utf8.c: Change formal parameter name
The parameter "len" really is a pointer in utf8_to_bytes(),
bytes_from_utf8(), and bytes_to_utf8(). Call it lenp. The
documentation was sloppy about it; clean that up.
Karl Williamson [Tue, 6 Jun 2017 01:26:37 +0000 (19:26 -0600)]
utf8.c: White-space, comment only
This adjusts the indentation to reflect changes in the previous commit.
Karl Williamson [Tue, 6 Jun 2017 01:21:41 +0000 (19:21 -0600)]
utf8_to_bytes(): Avoid work if possible
This converts to use the new function is_utf8_invariant_string_loc() to
find the first variant in the input. If none are found, the function is
a no-op. If the intial part of the input is all invariants, they are
now skipped during conversion, resulting in less work for such input.
The new function could also be optimized to speed up searching.
Karl Williamson [Tue, 6 Jun 2017 08:01:10 +0000 (02:01 -0600)]
utf8.c: Change UTF8 to UVCHR
There is no practical difference between UTF8_IS_INVARIANT and
UVCHR_IS_INVARIANT, except that the latter is supposed to be used on
characters. Fix to conform.
Karl Williamson [Thu, 8 Jun 2017 03:51:03 +0000 (21:51 -0600)]
sv.c: Refactor slightly to avoid a goto
The introduction of the inline function in the previous commit makes it
clear that the code can be refactored to be more structured.
Karl Williamson [Tue, 6 Jun 2017 00:51:28 +0000 (18:51 -0600)]
sv.c: Convert to use is_utf8_invariant_string_loc
This inline function was added in the previous commit.
And the function has the potential to be sped up by using word-at-a-time
operations.
Karl Williamson [Thu, 8 Jun 2017 03:41:12 +0000 (21:41 -0600)]
sv.c: Clarify some comments
Karl Williamson [Tue, 6 Jun 2017 00:33:05 +0000 (18:33 -0600)]
Add XS-callable function is_utf8_invariant_string_loc()
This is like is_utf8_invariant_string(), but takes an additional
parameter, a pointer into which it stores the location of the first
variant if any are found.
Karl Williamson [Mon, 5 Jun 2017 18:56:28 +0000 (12:56 -0600)]
bytes_to_utf8(): Remove obsolete comment
It said the logic was duplicated elsewhere, but now the essence of the
logic is in an inlined function used in both places.
Karl Williamson [Thu, 8 Jun 2017 03:17:56 +0000 (21:17 -0600)]
utf8.c: White_space only
The previous commit created a block around the lines changed by this
current commit.
Karl Williamson [Mon, 5 Jun 2017 18:26:06 +0000 (12:26 -0600)]
bytes_from_utf8(): Use memcpy if all invariant
This function does two passes over the input. In the first it decides
if the string can be downgraded, and computes the size needed for the
downgraded string. In the 2nd pass, it does the conversion.
Adding a single 'if' to the function can bypass the 2nd pass completely
if only invariants are found. The 2nd pass is replaced by a memcpy().
Karl Williamson [Mon, 5 Jun 2017 18:24:39 +0000 (12:24 -0600)]
utf8.c: A byte count should be Size_t, not I32
(or STRLEN. I thought we were converting to Size_t, so that's what I
chose here.)
Craig A. Berry [Thu, 8 Jun 2017 13:53:48 +0000 (08:53 -0500)]
Update test skips for no fchdir.
Follow-up to
489c16bfa14d46. There are fewer tests now, so skip
fewer when skipping is necessary.
Tony Cook [Thu, 8 Jun 2017 01:06:39 +0000 (11:06 +1000)]
[perl #131221] sv_dup/sv_dup_inc are only available under threads
Steve Hay [Wed, 7 Jun 2017 21:55:28 +0000 (22:55 +0100)]
Add Jacques Germishuys to AUTHORS
Jacques Germishuys [Sat, 3 Jun 2017 09:16:06 +0000 (11:16 +0200)]
correct PREMSVC80 typo
Abigail [Wed, 7 Jun 2017 21:56:09 +0000 (23:56 +0200)]
Move forward in time.
Now that all the deprecations which were marked to be fatal in 5.28
are actually fatal, it's time to update the wording of the Perl 5.28
section of pod/perldeprecation. Things will not disappear, they are
gone now.
Karl Williamson [Wed, 7 Jun 2017 21:40:33 +0000 (15:40 -0600)]
doop.c: White-space only
Outdent, since the previous commit removed an enclosing block
Karl Williamson [Wed, 7 Jun 2017 20:52:52 +0000 (14:52 -0600)]
Use simple-minded approach to bitwise UTF-8 operations
Commit
5d09ee1cb7b68f5e6fd15233bfe5048612e8f949 fatalized bitwise
operations of operands with wide characters in them. It retained the
regular UTF-8 handling, but throws an error when a wide character is
encountered.
But this code is complicated because of its original intended
generality. It can essentially be ripped out, replaced by code that
just downgrades the operand to non-UTF-8. Then we use the regular code
to do the operation. In the complement case, that's all that need be
done to mimic earlier behavior, as the result has not been in UTF-8.
For the other operations, the result is simply upgraded to UTF-8.
This removes quite a few lines of code, and now the UTF-8 handling uses
the same tight loops as the non-UTF-8. Downgrading and upgrading had to
be done specially before, but now they are done in tight loops, before
the operation, and after the operation
Karl Williamson [Wed, 7 Jun 2017 20:20:08 +0000 (14:20 -0600)]
t/op/bop.t: Re-add in some tests
Commit
5d09ee1cb7b68f5e6fd15233bfe5048612e8f949 fatalized above 0xFF
code points in the bitwise operators. It removed a bunch of tests in
t/op/bop.t. I had independently been working on this task. An issue
with removing such tests is are they there to be testing wide
characters, or are they there to test UTF-8, and the use of above-FF was
merely incidental? I had undertaken the tedious work of looking through
the commits for all the tests that now fail to determine which category
they were in. I concur mostly with the previous commit, but there were
a few tests that I found that should have been retained, and modified to
pass under the new scheme. This commit does that.
Deprecated warnings no longer have to be turned off, as they were
because of the previous tests that have now been removed.
Karl Williamson [Wed, 7 Jun 2017 19:31:10 +0000 (13:31 -0600)]
Update pods about bitwise UTF-8 above 0xFF being fatal
Karl Williamson [Sat, 3 Jun 2017 14:34:56 +0000 (08:34 -0600)]
t/op/bop.t: Verify complement downgrades UTF-8.
This adds a test for the existing result, so it doesn't get changed in
the future by mistake. I make no claims that it should work this way.
Karl Williamson [Wed, 7 Jun 2017 19:57:41 +0000 (13:57 -0600)]
t/op/bop.t: Verify bitwise & ^ | retain UTF-8
If one of the inputs is encoded in UTF-8, the behavior has been to have
the output be also in UTF-8. This has not been really documented, but I
think that is the way it should be. In any case, this commit adds tests
to make sure it doesn't inadvertently get changed.
Steve Hay [Wed, 7 Jun 2017 16:58:11 +0000 (17:58 +0100)]
Fix Windows build following commit
9d5e3f1aaa
Dagfinn Ilmari Mannsåker [Thu, 1 Jun 2017 17:16:56 +0000 (18:16 +0100)]
Make setting ${^ENCODING} to a defined value fatal
This has been deprecated since 5.22 and a no-op since 5.26.
Remove the now-obsolete t/uni/heavy.t test, which only tested that
utf8_heavy.pl didn't fail to load when ${^ENCODING} was set.
David Mitchell [Wed, 8 Mar 2017 14:57:23 +0000 (14:57 +0000)]
perly.y: add $$ = 0 for midrule code blocks
In places where a rule contains multiple code blocks, ensure that
$$ is assigned a valid value at the end of midrule blocks, so that
valgrind ./perl -Dpv ...
doesn't display zillions of
Conditional jump or move depends on uninitialised value
errors, when perl tries to display the parse stack.
I've only done the various newish top-level grammar entries - these all
seemed to have the same defect, while from a quick glance elsewhere in the
file, it seemed like older rules already do this.
David Mitchell [Wed, 7 Jun 2017 10:09:38 +0000 (11:09 +0100)]
[MERGE] rework Perl_sv_vcatpvfn_flags
Bug fixes:
Size modifiers with integer Inf/Nan, e.g. sprintf("%hi", Inf)
used to print spurious warnings.
The very-special-cased "%.NNNg" code has been removed, and is now
handled by the special-cased "...%.NNNg..." code instead, which makes
it about 4% slower, but handles unicode radix points better.
"%n" with a missing arg now croaks with a specific error message
(previously it croaked about modifying a read-only variable).
sprintf("%p", 0+Inf) now prints the address of an SV, not the literal
string "Inf" (or "NaN").
It now warns on a missing explicit width or vector arg, e.g. "%*2$d"
or "%3$vd".
Widths and precisions specified via an argument used to wrap; for
example, this used to output " abc"; it now croaks with "Integer
overflow in format string":
$w = 0x100000005; printf "%*s", $w, "abc";
Illegal formats didn't work well with utf8. It now resets to the
character following the '%' and continues from there.
The locale-specific radix point code now uses
SvCUR(PL_numeric_radix_sv) rather than SvLEN(PL_numeric_radix_sv) for
buffer size allocation. Also, several places weren't taking into
account the length of the radix point string when calculating buffer
sizes; they were only being saved by the general 'add 40' fudge
factor.
Buffer sizing and length calculation overflow/wraps:
The code has been thoroughly audited to look for any issues where the
format and/or args are malicious, and some possible (although not
exploitable) wraps have been fixed.
int's and I32's etc have been replaced with STRLEN where appropriate.
Tests for overflow/wrap have been added to the test suite.
The code has been cleaned up a lot and better commented, so it will be
easier to avoid wrapping in future.
Optimisations:
The special-cased "...%.0f..." is now detected and handled earlier in
the loop, so is about 35% faster.
It now sets and restores locale only once per function call, not
multiple times per format element.
Generally the code has been heavily tweaked.
On average, the newly-added benchmarks are about 10-15% faster; the
mixed-utf8 benchmark is about about 80% faster.
Other changes:
The HAS_LDBL_SPRINTF_BUG code that worked around a 2002 Irix 6
bug has been removed.
The 300-line block of code which handles %a hex floating-point formats
has been moved into a separate static function.
The 'svmax' arg to the sv_vcatpvfn() family has been changed from I32
to STRLEN and renamed to 'sv_count'.
A "Redundant argument in printf" warning is no longer emitted after an
invalid % format element has been encountered. This makes the code
simpler, and since you get an "Invalid conversion in printf" warning
anyway, does no harm (and in fact may be less confusing).
Several locals vars have been eliminated and the scope of many others
has been reduced.
David Mitchell [Fri, 2 Jun 2017 14:57:29 +0000 (15:57 +0100)]
add some sprintf benchmarks
David Mitchell [Fri, 2 Jun 2017 14:12:46 +0000 (15:12 +0100)]
Perl_sv_vcatpvfn_flags: rename a label
s/donevalidconversion/done_valid_conversion/
so its a bit easier to read.
David Mitchell [Fri, 2 Jun 2017 14:07:10 +0000 (15:07 +0100)]
sv_vcatpvfn_flags and wrappers: s/svmax/sv_count/
Rename the 'svmax' parameter of
Perl_sv_vcatpvfn_flags(),
Perl_sv_vcatpvfn(),
Perl_sv_vsetpvfn(),
to 'sv_count'.
'max' often implies N-1 (e.g. svarsg[0]..svargs[svmax]), whereas it's
actually the number of SV args passed to the functions.
David Mitchell [Fri, 2 Jun 2017 13:47:11 +0000 (14:47 +0100)]
Perl_sv_vcatpvfn_flags: handle mixed utf8 better
Once the output string gets upgraded to utf8 (e.g. due to a utf8 %s
argument), any remaining appending of plain (non-%) parts of the
format string becomes very inefficient. It basically creates an
SV out of the next format chunk, upgrades that SV to utf8, then
appends the upgraded buffer.
This commits makes it just append the format chunk byte by byte, upgrading
in the fly if that byte is !NATIVE_BYTE_IS_INVARIANT
David Mitchell [Fri, 2 Jun 2017 12:51:45 +0000 (13:51 +0100)]
add S_sv_catpvn_simple() for use by sprintf
Currently Perl_sv_vcatpvfn_flags() uses an unrolled sv_catpvn_nomg()
to append floating point formats, a call to sv_catpvn_nomg() to append
non-% parts of the format, and a few other non-performance-critical
calls to sv_catpvn_nomg().
Move the unrolled code block into an inline static function, and make
the non-% appending use it too.
David Mitchell [Fri, 2 Jun 2017 12:08:12 +0000 (13:08 +0100)]
Perl_sv_vcatpvfn_flags: re-indent a code block
whitespace only
David Mitchell [Fri, 2 Jun 2017 12:00:52 +0000 (13:00 +0100)]
Perl_sv_vcatpvfn_flags: eliminate p var
It has 1500-line scope, and is equal to fmtstart-1 for most of the
time.
This also allows us to 'const'ify some variables better.
David Mitchell [Fri, 2 Jun 2017 11:23:32 +0000 (12:23 +0100)]
Perl_sv_vcatpvfn_flags: clarify GCC bug comments
In particular it wasn't clear what bug was being worked around, nor that
'#13488' referred to a GNU ticket rather than a perl ticket.
This bug was fixed back in 2004, but the workaround is fairly harmless, so
I've left it as-is.
David Mitchell [Fri, 2 Jun 2017 10:57:11 +0000 (11:57 +0100)]
Perl_sv_vcatpvfn_flags: simplify alt handling
only do calculations for alt (#) formatting in the branches which use it
David Mitchell [Fri, 2 Jun 2017 10:41:41 +0000 (11:41 +0100)]
Perl_sv_vcatpvfn_flags: rename 'p' var 's'
In the 'append # block of code at the end of the loop, don't re-use the
widely-scoped 'p' pointer; instead use a tightly scope var instead
(named 's' do it doesn't clash with p which is still valid in an outer
scope.)
David Mitchell [Fri, 2 Jun 2017 08:51:40 +0000 (09:51 +0100)]
Perl_sv_vcatpvfn_flags: simplify format appending
The bit at the end of the main loop has a whole bunch of conditionals
along the lines of
if (gap && !left)
apppend gap
if (esignlen && !fill)
append esignbuf
if (zeros)
append zeroes
if (elen)
append ebuf
if (gap && left)
append gap
This involves many tests along the main code path to cope with all the
possibilities (e.g. if left, gap is output before ebuf, otherwise after)
Instead split it into a couple of major branches with duplication between
the branches, but requiring few tests along any one code path.
For example, sprintf("%5d", -1) formerly required 9 branches, 1 for loop,
and 1 memset(). It now requires 2 branches and 3 for loops,
I've removed memset()s and replaced them with for loops. For the short
padding typically used (e.g. "%9d" rather than "%8192d") a loop is faster.
David Mitchell [Thu, 1 Jun 2017 15:05:59 +0000 (16:05 +0100)]
Perl_sv_vcatpvfn_flags: eliminate a wrap check
This is one case where it can never wrap, so don't check.
David Mitchell [Thu, 1 Jun 2017 11:46:23 +0000 (12:46 +0100)]
Perl_sv_vcatpvfn_flags: simpler special formats
At the top of Perl_sv_vcatpvfn_flags(), certain fixed formats are
special-cased: "", "%s", "%-p", "%.0f".
Simplify the code which handles these. In particular, don't try to issue
"missing" or "redundant" arg warnings there. Instead, check for the
correct number of args as part of the test for whether this can be
special-cased, and if not, fall through to the general code in the main
body of the function to handle that format and issue any warnings.
This makes the code a lot simpler. It also now detects the redundant arg
in printf("%.0f",1,2).
The code is now also more efficient - it tries to check for things like
pat[0] == '%' only once, rather than re-checking for every special-case
variant its trying.
David Mitchell [Thu, 1 Jun 2017 10:55:47 +0000 (11:55 +0100)]
Perl_sv_vcatpvfn_flags: simpler redundant arg test
5.24.0 added a new warning:
Redundant argument in printf at ....
That warning is issued if there are more args than format elements.
However, it may also warn for invalid format - e.g. for something like
printf("%Z%d", 1,2) you get both
Invalid conversion in printf: "%Z" at ...
Redundant argument in printf at ...
Personally I think once once part of the format has been determined to be
invalid, its hard for perl to second-guess in what way the format was
invalid, and thus to be able to conclude that there is in fact a redundant
arg.
So this commit commit suppresses any "redundant" warning once an "invalid"
warning has been issued.
Doing this makes it possible to simplify the code and remove the
used_explicit_ix variable.
Apart from warnings, used_explicit_ix was only used in %p to check for
'simple' special forms - but that code checks for a trailing '$' character
anyway, so that test was redundant.
David Mitchell [Thu, 1 Jun 2017 10:29:35 +0000 (11:29 +0100)]
Perl_sv_vcatpvfn_flags: fix comment typo
David Mitchell [Thu, 1 Jun 2017 10:27:20 +0000 (11:27 +0100)]
Perl_sv_vcatpvfn_flags: add comment about wrap
David Mitchell [Thu, 1 Jun 2017 10:08:27 +0000 (11:08 +0100)]
Perl_sv_vcatpvfn_flags: only do utf8 in radix code
For floating point formats, the output can only be utf8 if the radix point
is utf8. Currently the radix point code sets the is_utf8 variable, then
later, in the main floating-point code path, it tests is_utf8 and
upgrades the output string to utf8.
Instead, just do the upgrade directly in the radix code block.
David Mitchell [Thu, 1 Jun 2017 10:00:26 +0000 (11:00 +0100)]
Perl_sv_vcatpvfn_flags: simplify radix len adding
Assume the length of the radix point is a constant 1 (i.e. length('.'))
and only increment float_need further if we're in a locale.
David Mitchell [Thu, 1 Jun 2017 09:52:12 +0000 (10:52 +0100)]
sprintf %a/%A more sanity checks
For the code which generates hexadecimal floating-point formats,
add extra sanity checks against buffer overruns.
David Mitchell [Thu, 1 Jun 2017 09:32:36 +0000 (10:32 +0100)]
S_hextract(): fix #if indentation
a complex set of nested #if/#else/#endif's had incorrect and confusing
indentation.
whitespace-only change
David Mitchell [Wed, 31 May 2017 11:35:34 +0000 (12:35 +0100)]
Perl_sv_vcatpvfn_flags: simplify some wrap checks
Skip doing some overflow checks when we know it can't overflow.
David Mitchell [Wed, 31 May 2017 10:59:48 +0000 (11:59 +0100)]
Perl_sv_vcatpvfn_flags: simplify float_need calc
Include another constant addition in the initial assignment, to eliminate
a later wrap check.
David Mitchell [Wed, 31 May 2017 10:15:15 +0000 (11:15 +0100)]
S_format_hexfp(): s/int/STRLEN/
In the helper function that sprintf's %a/%A hex floating point values,
the calculation of the number of zeros to pad with should be in terms of
STRLEN rather than int.
A bit academic unless someone ever tries to print a hex f/p value with a
precision > 2Gb digits.
David Mitchell [Wed, 31 May 2017 08:47:27 +0000 (09:47 +0100)]
op/infnam.t: skip unportable tests
sprintf size modifiers L and q aren't available on all platform sizes,
so skip them.
David Mitchell [Tue, 30 May 2017 15:11:37 +0000 (16:11 +0100)]
Perl_sv_vcatpvfn_flags: add inits to silence gcc
Add a couple of unnecessary variable initialisers, to keep gcc's "this
variable might be used uninitialised - then again it might not - in fact I
don't really know what I'm talking about, but I've decided to annoy you
with it anyway" warning at bay.
David Mitchell [Tue, 30 May 2017 14:55:29 +0000 (15:55 +0100)]
Perl_sv_vcatpvfn_flags: avoid wrap on precision
Where the precision is specified literally in the format string,
the integer precision value could wrap. Instead, make it croak with
Integer overflow in format string
As in other recent commits, the upper limit is set at 1/4 of STRLEN.
David Mitchell [Tue, 30 May 2017 14:27:00 +0000 (15:27 +0100)]
Perl_sv_vcatpvfn_flags: s/int/STRLEN/g
There wee a few residual places that used int loop counters, e.g. to
prepend N '0's to a number. Since the N's are of type STRLEN, make the
loop counters STRLEN too.
Its a bit academic since you're unlikely to have a number needing >2Gb
worth of zero padding, but it makes things consistent and easier to audit.
At this point I believe that any remaining usage of int / I32 / U32 in
Perl_sv_vcatpvfn_flags() is legitimate.
David Mitchell [Tue, 30 May 2017 14:11:24 +0000 (15:11 +0100)]
Perl_sv_vcatpvfn_flags: %n: avoid wrap
Its a bit academic, but in principle if a string was longer than 2Gb
chars, the length as set by %n could wrap. So use the correct type(s).
David Mitchell [Tue, 30 May 2017 12:45:35 +0000 (13:45 +0100)]
Perl_sv_vcatpvfn_flags: width/precis arg wrap
When the width or precision is specified via an argument rather than
literally, check whether the value wraps.
Formerly, something like
$w = 0x100000005;
printf "%*s", $w, "abc";
might print " abc" or similar, depending on platform.
Now it croaks with "Integer overflow in format string".
I did wonder whether it should just warn instead, but:
1) over-large literal widths/precisions already croak.
2) Code that has wild field specifiers like that is already likely
to crash with an out-of-memory error.
3) At least this croak is trappable via eval - OOM isn't.
I also set the maximum allowed value to be 1/4 of the size of a pointer,
to give a safety margin for possible wrapping later
David Mitchell [Mon, 29 May 2017 16:06:06 +0000 (17:06 +0100)]
Perl_sv_vcatpvfn_flags: move vector initialisation
Move the generation of vecstr/veclen/vec_utf8 into the
vector-initialisation block, rather than being part of the general
'get next arg' block.
Also, stop vecsv being in scope for the whole of the loop block, and make
it two separate tightly-scope vars (with different purposes).
David Mitchell [Mon, 29 May 2017 15:53:06 +0000 (16:53 +0100)]
Perl_sv_vcatpvfn_flags: warn on missing %v arg
The explicit arg variant, e.g. %3$vd, didn't give 'missing arg' warning.
David Mitchell [Mon, 29 May 2017 15:20:17 +0000 (16:20 +0100)]
Perl_sv_vcatpvfn_flags: warn on missing width arg
It didn't used to warn when the width value was obtained from the next or
specified arg, and there wasn't such an arg.
David Mitchell [Mon, 29 May 2017 15:11:01 +0000 (16:11 +0100)]
Eliminate FETCH_VCATPVFN_ARGUMENT macro
This can be simplified so much now that it might as well just be expanded
in situ for its 3 uses.
David Mitchell [Mon, 29 May 2017 15:01:26 +0000 (16:01 +0100)]
Perl_sv_vcatpvfn_flags: re-indent block
whitespace-only
David Mitchell [Mon, 29 May 2017 14:27:18 +0000 (15:27 +0100)]
Perl_sv_vcatpvfn_flags: unify %v vers obj handling
Cureently sv_vcatpvfn_flags() has special handling of the arg under %v
when the arg is a version object, but only via the perlish interface
(argsv and svmax). This commit extends that handling to the C-sih
interface (args).
There seems no good reason not to, and it simplifies the code.
David Mitchell [Mon, 29 May 2017 12:49:42 +0000 (13:49 +0100)]
Perl_sv_vcatpvfn_flags: unify args handling
Several places do something along the lines of:
if (explicit arg index)
FETCH_VCATPVFN_ARGUMENT(...., svargs[ix-1])
else
FETCH_VCATPVFN_ARGUMENT(...., svargs[svix++])
For each of these, reduce the duplicate code by changing the above to
(approximately)
ix = ix ? ix - 1 : svix++;
FETCH_VCATPVFN_ARGUMENT(...., svargs[ix])
David Mitchell [Mon, 29 May 2017 10:16:49 +0000 (11:16 +0100)]
sv_vcatpvfn() family: make svmax arg Size_t
It was formerly I32. It should be unsigned since you can't have a negative
number of args. And although you're unlikely to call sprintf with more
than 0x7fffffff args, it makes it more consistent with other APIs which
we've been gradually expanding to 64-bit/ptrsize. It also makes the
code internal to Perl_sv_vcatpvfn_flags more consistent, when
dealing with explict arg index formats like "%10$s". This function still
has a mix of STRLEN (for string lengths) and Size_t (for arg indexes)
but they are aliases for each other.
I made Perl_do_sprintf()'s len arg SSize_t rather than Size_t, since
it typically gets called with ptr diff arithmetic. Not sure if this is
being overly cautious.
David Mitchell [Mon, 29 May 2017 08:59:16 +0000 (09:59 +0100)]
S_expect_number(): return STRLEN not I32
This static function is used by Perl_sv_vcatpvfn_flags() to read in
a width or explicit argument number. It currently returns an I32 result
(and croaks if the number exceeds the maximum possible I32 value).
Change it to return STRLEN, and to croak on the value being greater than
max(STRLEN) / 4.
This doesn't make a lot of difference in practice, since no code is ever
going to be able to successfully create a formatted string that large
without running out of memory anyway. But by making it unsigned and of the
same type used elsewhere in sv_vcatpvfn_flags(), it simplifies auditing
the code for possible wrapping/truncating etc.
The change in the limit where it croaks with "Integer overflow in format
string" has changed as follows:
previously now
32-bit system 0x7fffffff 0x3fffffff
32/64bit system 0x7fffffff 0x3fffffff
64bit system 0x7fffffff 0x3fffffffffffffff
Setting the limit as 1/4 max rather than 1/2 max is just a safety
net to help avoid wraps/overflows elsewhere.
David Mitchell [Sun, 28 May 2017 17:07:14 +0000 (18:07 +0100)]
Perl_sv_vcatpvfn_flags: simplify 'c' var
Make it so that its now *always* the format type ('s', 'd' etc).
Don';t bother initialising it, and *don't* use as as a temporary
buffer (eptr = &c), so it can be stored in a register.
David Mitchell [Sun, 28 May 2017 16:59:47 +0000 (17:59 +0100)]
Perl_sv_vcatpvfn_flags: reduce scope of 'iv' var
David Mitchell [Sun, 28 May 2017 16:52:25 +0000 (17:52 +0100)]
Perl_sv_vcatpvfn_flags: eliminate 'epix' var
Or rather, reduce its scope to a small block and rename to 'ix'.
David Mitchell [Sun, 28 May 2017 16:49:10 +0000 (17:49 +0100)]
S_expect_number() re-indent code
.. following previous commit. Whitespace only.
David Mitchell [Sun, 28 May 2017 16:43:36 +0000 (17:43 +0100)]
sprintf: move 1..9 test out of S_expect_number()
Currently Perl_sv_vcatpvfn_flags() does several checks for "is the next
part of the format a number starting with a '1'..'9'?" It does this by
calling S_expect_number(), which returns 0 if not, or the value of the
number otherwise. For a simple format specifier, this results in multiple
fruitless calls to S_expect_number.
This commits makes it that the caller of S_expect_number is responsible
for checking for the presence of 1..9.
David Mitchell [Fri, 26 May 2017 23:57:47 +0000 (00:57 +0100)]
Perl_sv_vcatpvfn_flags: more %v optimisation
Only do the code for appending the vector separator in the vector branch.
In particular, don't size the SvGROW for dotstrlen outside of %v.
This makes the %v code a bit slower but everything else a bit faster.
David Mitchell [Fri, 26 May 2017 23:17:35 +0000 (00:17 +0100)]
Perl_sv_vcatpvfn_flags: test for valid %vX once
Rather than testing for !vectorize in every conversion case which doesn't
support %v, test once for supported types in the if (vectorize) branch.
That way code which doesn't use %v never has to test for it.
David Mitchell [Fri, 26 May 2017 23:07:48 +0000 (00:07 +0100)]
Perl_sv_vcatpvfn_flags: join two if blocks
convert if (x); if (!x); into an single if/else
David Mitchell [Fri, 26 May 2017 23:00:10 +0000 (00:00 +0100)]
Perl_sv_vcatpvfn_flags: delay vector arg get
Move the block of code which retrieves the SV which the %v will iterate
over, from just before the /* SIZE */ block to just after. Since that
block doesn't do anything with args or svargs, this should make no
functional difference - but it will allow the next commit to coalesce
if (x); if (!x); into an single if/else.
Apart from cutting and pasting the code block, no other changes have been
made to it.
David Mitchell [Fri, 26 May 2017 22:49:58 +0000 (23:49 +0100)]
Perl_sv_vcatpvfn_flags: eliminate VECTORIZE_ARGS
This macro is only used once. Just expand it.
David Mitchell [Fri, 26 May 2017 22:42:07 +0000 (23:42 +0100)]
Perl_sv_vcatpvfn_flags: eliminate ewix local var
It's now only used within one code block.
David Mitchell [Fri, 26 May 2017 22:34:25 +0000 (23:34 +0100)]
Perl_sv_vcatpvfn_flags: remove 'asterisk' var
There was only one remaining use of this local var: in %p, to distinguish
between explicit and implicit width specifier, e.g. %*p or %1$p, vs %2p.
This can be done by just checking whether the char before the p was a '*'
or '$'.
David Mitchell [Fri, 26 May 2017 22:20:22 +0000 (23:20 +0100)]
Perl_sv_vcatpvfn_flags: further simplify %v logic
For the common case with no * or v, there now are only 2 test-and-branch
(! '*', ! 'v') rather than 3 (! '*', ! 'v', !asterisk)
This works by putting the *v handling code in the * branch
David Mitchell [Fri, 26 May 2017 21:45:02 +0000 (22:45 +0100)]
Perl_sv_vcatpvfn_flags: eliminate evix local var