This is a live mirror of the Perl 5 development currently hosted at https://github.com/perl/perl5
perl5.git
3 years agohandy.h: Change references to swashes
Karl Williamson [Tue, 5 Nov 2019 05:17:08 +0000 (22:17 -0700)]
handy.h: Change references to swashes

As these are no longer used.

3 years agoPorting/todo.pod: Rmv reference to fixing swashes
Karl Williamson [Tue, 5 Nov 2019 05:14:30 +0000 (22:14 -0700)]
Porting/todo.pod: Rmv reference to fixing swashes

3 years agoUnTODO some tests fixed by the previous commit
Karl Williamson [Tue, 5 Nov 2019 05:10:56 +0000 (22:10 -0700)]
UnTODO some tests fixed by the previous commit

3 years agoReimplement tr/// without swashes
Karl Williamson [Tue, 5 Nov 2019 04:30:48 +0000 (21:30 -0700)]
Reimplement tr/// without swashes

This large commit removes the last use of swashes from core.

It replaces swashes by inversion maps.  This data structure is already
in use for some Unicode properties, such as case changing.

The inversion map data structure leads to straight forward
implementation code, so I collapsed the two doop.c routines
do_trans_complex_utf8() and do_trans_simple_utf8() into one.  A few
conditionals could be avoided in the loop if this function were split so
that one version didn't have to test for, e.g., squashing, but I suspect
these are in the noise in the loop, which has to deal with UTF-8
conversions.  This should be faster than the previous implementation
anyway.  I measured the differences some releases back, and inversion
maps were faster than the equivalent swash for up to 512 or 1024
different ranges.  These numbers are unlikely to be exceeded in tr///
except possibly in machine-generated ones.

Inversion maps are capable of handling both UTF-8 and non-UTF-8 cases,
but I left in the existing non-UTF-8 implementation, which uses tables,
because I suspect it is faster.  This means that there is extra code,
purely for runtime performance.

An inversion map is always created from the input, and then if the table
implementation is to be used, the table is easily derived from the map.
Prior to this commit, the table implementation was used in certain edge
cases involving code points above 255.  Those cases are now handled by
the inversion map implementation, because it would have taken extra code
to detect them, and I didn't think it was worth it.  That could be
changed if I am wrong.

Creating an inversion map for all inputs essentially normalizes them,
and then the same logic is usable for all.  This fixes some false
negatives in the previous implementation.  It also allows for detecting
if the actual transliteration can be done in place.  Previously, the
code mostly punted on that detection for the UTF-8 case.

This also allows for accurate counting of the lengths of the two sides,
fixing some longstanding TODO warning tests.

A new flag is created, OPpTRANS_CAN_FORCE_UTF8, when the tr/// has a
below 256 character resolving to one that requires UTF-8.  If this isn't
set, the code knows that a non-UTF-8 input won't become UTF-8 in the
process, and so can take short cuts.  The bit representing this flag is
the same as OPpTRANS_FROM_UTF, which is no longer used.  That name is
left in so that the dozen-ish modules in cpan that refer to it can still
compile.  AFAICT none of them actually use the flag, as well they
shouldn't since it is private to the core.

Inversion maps are ideally suited for tr/// implementations.  An issue
with them in general is that for some pathological data, they can become
fragmented requiring more space than you would expect, to represent the
underlying data.  However, the typical tr/// would not have this issue,
requiring only very short inversion maps to represent; in some cases
shorter than the table implementation.

Inversion maps are also easier to deparse than swashes.  A deparse TODO
was also fixed by this commit, and the code to deparse UTF-8 inputs is
simplified.

One could implement specialized data structures for specific types of
inputs.  For example, a common tr/// form is a single range, like
tr/A-Z/a-z/.  That could be implemented without a table and be quite
fast.  An intermediate step would be to use the inversion map
implementation always when the transliteration is a single range, and
then special case length=1 maps at execution time.

Thanks to Nicholas Rochemagne for his help on B

3 years agointrpvar.h: Add variable for use in tr///
Karl Williamson [Thu, 3 Oct 2019 04:34:37 +0000 (22:34 -0600)]
intrpvar.h: Add variable for use in tr///

This is part of this branch of changes.

3 years agoop.c: Add debugging dump function
Karl Williamson [Tue, 19 Feb 2019 04:14:47 +0000 (21:14 -0700)]
op.c: Add debugging dump function

This function dumps out an inversion map

3 years agoop.h: Add synonyms for some tr/// values
Karl Williamson [Mon, 4 Nov 2019 21:59:02 +0000 (14:59 -0700)]
op.h: Add synonyms for some tr/// values

3 years agoChange names of some OPpTRANS flags
Karl Williamson [Mon, 4 Nov 2019 21:55:16 +0000 (14:55 -0700)]
Change names of some OPpTRANS flags

These two flags will shortly become obsolete, replaced by ones with
different meanings.  This flag makes the new ones the normal ones, and
makes the old names synonyms so that code that refers to them can
compile.

3 years agodoop.c: Refactor do_trans_complex()
Karl Williamson [Mon, 4 Nov 2019 21:38:58 +0000 (14:38 -0700)]
doop.c: Refactor do_trans_complex()

I had trouble understanding how this uncommented routine worked.  And it
turned out to be broken, squeezing the pre-transliterated characters
instead of the post-transliterated ones.  This fixes the TODO test added
in the previous commit.

3 years agot/op/tr.t: Add tests, incl. a TODO
Karl Williamson [Tue, 5 Nov 2019 05:13:43 +0000 (22:13 -0700)]
t/op/tr.t: Add tests, incl. a TODO

This adds a TODO test which demonstrates that the current tr/// is
broken, to be fixed by the next commit.

It adds other tests designed to stress the forthcoming revisions in the
implementation of tr///.

3 years agodoop.c: Change name of variable
Karl Williamson [Mon, 4 Nov 2019 21:17:09 +0000 (14:17 -0700)]
doop.c: Change name of variable

This helped me understand what was going on in this function

3 years agodoop.c: Change out-of-bounds value
Karl Williamson [Mon, 4 Nov 2019 20:12:21 +0000 (13:12 -0700)]
doop.c: Change out-of-bounds value

This currently uses 0xfeedface as a marker for something that isn't a
legal value.  But that could in fact become legal at same point.  This
defines a value TR_OOB that can be guaranteed not to become legal.

3 years agodoop.c: Add, revise comments
Karl Williamson [Sat, 2 Nov 2019 16:15:22 +0000 (10:15 -0600)]
doop.c: Add, revise comments

3 years agoop.c: Simplify expression.
Karl Williamson [Sun, 27 Oct 2019 15:55:10 +0000 (09:55 -0600)]
op.c: Simplify expression.

This also makes sure 'struct_size' has the correct value in it for any
future uses.

3 years agoregen/ebcdic.pl: Add tables that partition by UTF-8 length
Karl Williamson [Wed, 2 Oct 2019 21:36:19 +0000 (15:36 -0600)]
regen/ebcdic.pl: Add tables that partition by UTF-8 length

These will be used in a future commit.  This creates equivalence classes
of ranges of code points whose UTF-8 representations are the same length

3 years agoop.c, doop.c Use mnemonics instead of numeric values
Karl Williamson [Wed, 2 Oct 2019 20:47:24 +0000 (14:47 -0600)]
op.c, doop.c Use mnemonics instead of numeric values

For legibility and maintainability

3 years agodoop.c: Add a parameter to a few fcns
Karl Williamson [Wed, 2 Oct 2019 18:33:01 +0000 (12:33 -0600)]
doop.c: Add a parameter to a few fcns

instead of deriving it each time from inside the function.  This is in
preparation for future commits.

3 years agoChange macro name in tr/// code
Karl Williamson [Wed, 2 Oct 2019 15:38:53 +0000 (09:38 -0600)]
Change macro name in tr/// code

This makes it more mnemonic.  Also add an explanation in toke.c

3 years agoop.c: Comments only
Karl Williamson [Wed, 27 Feb 2019 21:05:18 +0000 (14:05 -0700)]
op.c: Comments only

Indent for clarity, and add a comment

3 years agodoop.c, op.c: White-space only
Karl Williamson [Wed, 27 Feb 2019 20:56:00 +0000 (13:56 -0700)]
doop.c, op.c: White-space only

Remove trailing blanks and outdent a doubly indented block

3 years agoop.c: Indent some code
Karl Williamson [Wed, 27 Feb 2019 20:26:53 +0000 (13:26 -0700)]
op.c: Indent some code

This is in preparation for a future commit which will surround this with
an 'if'.

3 years agotoke.c: comment, White-space only
Karl Williamson [Sat, 2 Nov 2019 20:45:23 +0000 (14:45 -0600)]
toke.c: comment, White-space only

Wrap a too-long line

3 years agoAllow core to work with code points above IV_MAX
Karl Williamson [Wed, 2 Oct 2019 21:29:05 +0000 (15:29 -0600)]
Allow core to work with code points above IV_MAX

Higher has been reserved for core use, and a future commit will want to
finally do this.

3 years agoMove some static fcns from regcomp.c to invlist_inline.h
Karl Williamson [Tue, 19 Feb 2019 02:37:53 +0000 (19:37 -0700)]
Move some static fcns from regcomp.c to invlist_inline.h

They are still only accessible from regcomp.c, but this is in
preparation for them to be usable from other core files as well.

3 years agoregcomp.c: Change name of static function.
Karl Williamson [Wed, 2 Oct 2019 03:57:59 +0000 (21:57 -0600)]
regcomp.c: Change name of static function.

This removes an unnecessary leading underscore

3 years agoinvlist_inline.h: White space only
Karl Williamson [Tue, 19 Feb 2019 02:31:25 +0000 (19:31 -0700)]
invlist_inline.h: White space only

Fold a too-long line

3 years agoinvlist_inline.h: Restrict files symbols are in
Karl Williamson [Tue, 19 Feb 2019 02:30:31 +0000 (19:30 -0700)]
invlist_inline.h: Restrict files symbols are in

These are only needed in regcomp.c, so restrict them to that file

3 years agoUpdate IO-Compress to CPAN version 2.089
Chris 'BinGOs' Williams [Wed, 6 Nov 2019 22:06:58 +0000 (22:06 +0000)]
Update IO-Compress to CPAN version 2.089

  [DELTA]

  2.089 3 November 2019

      * bin/streamzip
        Add zipstream to EXE_FILES
        https://github.com/pmqs/IO-Compress/issues/5
        7f5ed78e5652125c9ba78aab720e18d384d59c00
        fb8cd6480af6303f58fd2e12d4999cd1830f0c5f

  2.088 31 October 2019

      * t/105oneshot-zip-only.t
        Fix reset of CompSize
        6034da95f1dc5a594edc0d26e6add8d86990ad56

      * Add Support Details
        ad66b6ec4cf175a70e53f74572eed5f403648f11

      * Update site for Bzip2 to sourceware
        77497aeb2a234889a2b2c4ac7ea2f620895b16a9

      * Fix number of tests
        bc4e234449a82fb00f66af752dfc4c96097b2a4d

      * Add streamzip script to bin
        76d2795d0587bafb0cc398e97142740acba82a42

      * zipdetails

        * Update zipdetails to version 1.11
         8958cb3aa90745a4b3369479846846fdca6b4f76

        * Zip64 extra field typo
          f186380d701fe5257f9fc92d69160dc6382cfc24

      * t/105oneshot-zip-only.t
        test with deflated directory
        16bfffcf5089af67cb7f68685cc61d06409cba73

      * t/105oneshot-zip-only.t
        Add test for encrypted Zip files
        5ad813115aed000f88d7df28261b43c00ae56525
        2c64e255feb5a1ee33d033f7eccb6feca12ebe97

      * Documentation Updates
        https://github.com/pmqs/IO-Compress/issues/2
        e1fd0d4eda0a8496981cbd83ad06906f4ae586a5

      * Mention xz, lzma etc
        https://github.com/pmqs/IO-Compress/issues/4
        126f7b9da97b572d0fb89a9bdcc190c5405c72b8

3 years agoUpdate Compress-Raw-Bzip2 to CPAN version 2.089
Chris 'BinGOs' Williams [Wed, 6 Nov 2019 22:00:47 +0000 (22:00 +0000)]
Update Compress-Raw-Bzip2 to CPAN version 2.089

  [Delta]

  2.089 3 November 2019

      * No Changes

  2.088 31 October 2019

      * Add Support Details
        37e04fb3dc50287a3512b004282425c6eedb8af6

      * upgrade to Bzip2 1.0.8
        027a30848ee57731b435d2ea7af09e532bf6fbfd

3 years agoUpdate Compress-Raw-Zlib to CPAN version 2.089
Chris 'BinGOs' Williams [Wed, 6 Nov 2019 21:59:15 +0000 (21:59 +0000)]
Update Compress-Raw-Zlib to CPAN version 2.089

  [DELTA]

  2.089 3 November 2019

      * No Changes

  2.088 31 October 2019

      * Add SUPPORT section
        d348ad76c2073a2973d094891fbd0c2e24bf397d

      * 000prereq.t: dump Perl version
        e1afe502818cb1ccf5bad917b14b029b408f47f1

3 years agoprevent a race between name-based stat and an open modifying atime
Tony Cook [Sun, 3 Nov 2019 22:52:22 +0000 (09:52 +1100)]
prevent a race between name-based stat and an open modifying atime

Most linux systems rarely update atime, so it's very unlikely
for this issue to trigger there, but on a system with classic atime
behaviour this was a race between open modifying atime and time()
ticking over.

gh #17234

3 years agoadd defensive parens
Yves Orton [Tue, 5 Nov 2019 23:32:06 +0000 (00:32 +0100)]
add defensive parens

3 years agorework U8TOxx_LE macros to force unsigned access
Yves Orton [Tue, 5 Nov 2019 23:05:17 +0000 (00:05 +0100)]
rework U8TOxx_LE macros to force unsigned access

This introduces a _shifted_octet() utility macro to make things
more clear, it also adds support for USE_UNALIGNED_PTR_DEREF for
little-endian platforms that allow unaligned access. This must
be manually defined and ONLY affects little endian builds currently,
and is there primarily for -g builds on x86 (eg for perl developers
themselves).

3 years agoFix the UTO*_LE macros
Matt Turner [Tue, 5 Nov 2019 18:22:58 +0000 (10:22 -0800)]
Fix the UTO*_LE macros

Embarrassingly I got confused and swapped them (BYTEORDER == 0x1234 etc
is not great...). Somehow the perl test suite still passes with this,
but fortunately the test suite for modules like
Algorithm-MinPerfHashTwoLevel caught the problem.

Fixes: https://github.com/Perl/perl5/issues/17244
Fixes: https://github.com/Perl/perl5/issues/17247
Fixes: https://rt.cpan.org/Ticket/Display.html?id=130890

3 years agoRemove compiler in .travis.yml
Steve Peters [Tue, 5 Nov 2019 17:08:13 +0000 (11:08 -0600)]
Remove compiler  in .travis.yml

SInce `clang` was removed as a compiler option in the previous change to `.travis.yml`, no need to exclude Linux from using it.  Also, no need for the excludes for OS X since was removed.

3 years agoperlop.pod: Slight clarification
Karl Williamson [Sat, 2 Nov 2019 20:41:49 +0000 (14:41 -0600)]
perlop.pod: Slight clarification

3 years agotoke.c: comment changes
Karl Williamson [Tue, 5 Nov 2019 04:55:53 +0000 (21:55 -0700)]
toke.c: comment changes

These should have been included in
0c311b7c345769239f38d0139ea7738feec5ca4d

3 years agogv.c: SVf needs to be surrounded by spaces
Karl Williamson [Mon, 4 Nov 2019 20:54:41 +0000 (13:54 -0700)]
gv.c: SVf needs to be surrounded by spaces

perl can be compiled with C++, and this is illegal there.

3 years agoRemove unused `key` and `orig_keyword` parameters from `yyl_key_core`
Dagfinn Ilmari Mannsåker [Tue, 5 Nov 2019 12:31:28 +0000 (12:31 +0000)]
Remove unused `key` and `orig_keyword` parameters from `yyl_key_core`

They were only ever passed as zeros, so just make them local to the
function.

3 years agoRename `tmp` local to `key` in `yyl_keylookup`
Dagfinn Ilmari Mannsåker [Tue, 5 Nov 2019 12:29:39 +0000 (12:29 +0000)]
Rename `tmp` local to `key` in `yyl_keylookup`

Also only initialise it just before it's actually used.

3 years agoRemove unused `key` parameter from `yyl_just_a_word`
Dagfinn Ilmari Mannsåker [Tue, 5 Nov 2019 12:25:14 +0000 (12:25 +0000)]
Remove unused `key` parameter from `yyl_just_a_word`

3 years agot/re/speed.t: increase timeout
David Mitchell [Tue, 5 Nov 2019 11:43:57 +0000 (11:43 +0000)]
t/re/speed.t: increase timeout

Test 58 is supposed to finish in milliseconds, or take 10s of seconds
if it hits the bug. It currently tests for <= 1s and is failing
occasionally in smokes. Increase the timeout to 2s and see if the issue
goes away. Also add a diag() to display the elapsed time on failure.

3 years agoenforce strict for barewords in multiconcat
Tony Cook [Mon, 4 Nov 2019 21:00:25 +0000 (08:00 +1100)]
enforce strict for barewords in multiconcat

gh #17254

3 years agoDoc: fix sv_setsv_cow reference
Sergey Aleynikov [Mon, 4 Nov 2019 17:58:32 +0000 (20:58 +0300)]
Doc: fix sv_setsv_cow reference

It's no more present in pp_hot.c

3 years agoS_gv_fetchmeth_internal fix STRLEN warning
Nicolas R [Mon, 4 Nov 2019 19:03:54 +0000 (13:03 -0600)]
S_gv_fetchmeth_internal fix STRLEN warning

Fixes #17250

cast STRLEN to int

Fix warnings from recent change GH #17222
We could also consider casting it using '
x & PERL_INT_MAX'

3 years ago[MERGE] Even smaller toke
Aaron Crane [Mon, 4 Nov 2019 11:00:14 +0000 (11:00 +0000)]
[MERGE] Even smaller toke

This continues the work merged in 5015bd0bb5ee7e0fa1ede1669bdfcd7bb5f10ebd,
factoring code out of Perl_yylex() and its callees, in the hope of making
the lexer easier to understand locally.

The work in the previous branch used unbounded tail recursion in some
places. Some compilers may have been able to compile the tail-recursive
calls as jumps under some optimisation modes, but we cannot rely on that. So
this further work replaces the unbounded recursion with gotos or other
control structures. It's a little annoying not to have been able to
eliminate those gotos, but these remaining gotos all occur within a function
that's considerably smaller and easier to understand, and they merely jump
all the way back to the top of that function.

After these changes, the largest remaining piece of Perl_yylex() is just
over 900 lines (down from originally >4100), and consists of a single switch
statement, all of whose case groups are independent.

This branch also contains a note in perldelta that this major refactoring
has taken place.

Closes #17220

3 years agoperldelta for recent toke.c refactoring 17241/head
Aaron Crane [Fri, 25 Oct 2019 10:21:51 +0000 (11:21 +0100)]
perldelta for recent toke.c refactoring

3 years agotoke.c: const-ify formbrack parameters
Aaron Crane [Fri, 1 Nov 2019 15:29:03 +0000 (15:29 +0000)]
toke.c: const-ify formbrack parameters

3 years agotoke.c: replace recursive calls to yyl_try() with goto
Aaron Crane [Fri, 1 Nov 2019 15:10:14 +0000 (15:10 +0000)]
toke.c: replace recursive calls to yyl_try() with goto

The downside of writing these calls recursively is that not all compilers
will compile the tail-position calls as jumps; that's especially true in
earlier versions of this refactoring process (where yyl_try() took a large
number of arguments), but it's not in general something we can expect to
happen — especially in the presence of `-O0` or similar compiler options.
This can lead to call-stack overflow in some circumstances.

Most recursive calls to yyl_try() occur within yyl_try() itself, so we can
easily replace them with an explicit `goto` (which is what most compilers
would use for the recursive calls anyway, now that yyl_try() takes ≤3
parameters).

There are only two other recursive-call cases. One is yyl_fake_eof(), which
as far as I can tell is never called repeatedly within a single file; this
seems safe.

The other is yyl_eol(). It has exactly two distinct return paths, so this
commit moves the retry logic into its yyl_try() caller.

With this change, we no longer seem to trigger call-stack overflow.

Closes #17220

3 years agotoke.c: delete unused bof parameters
Aaron Crane [Fri, 25 Oct 2019 14:43:00 +0000 (15:43 +0100)]
toke.c: delete unused bof parameters

3 years agotoke.c: don't pass around a copy of PL_parser->saw_infix_sigil
Aaron Crane [Fri, 25 Oct 2019 10:18:39 +0000 (11:18 +0100)]
toke.c: don't pass around a copy of PL_parser->saw_infix_sigil

There's exactly one place where we need to consult it (and that only for
producing good error messages in a specific group of term-after-term
situations).

The reason for passing it around was so that it could be reset to false
early on in the process of lexing a token, while then allowing the three
separate cases that might need to set it true to do so independently.

Instead, centralise the logic of determining when it needs to be true.

3 years agotoke.c: remove some spurious orig_keyword uses
Aaron Crane [Wed, 23 Oct 2019 21:00:47 +0000 (22:00 +0100)]
toke.c: remove some spurious orig_keyword uses

3 years agotoke.c: remove formbrack argument from yyl_try()
Aaron Crane [Wed, 23 Oct 2019 20:50:28 +0000 (21:50 +0100)]
toke.c: remove formbrack argument from yyl_try()

With this commit, yyl_try() has few enough arguments that the RETRY()
macro no longer serves any useful purpose; delete it too.

3 years agotoke.c: delete weird initial_state arg to yyl_try()
Aaron Crane [Wed, 23 Oct 2019 20:32:51 +0000 (21:32 +0100)]
toke.c: delete weird initial_state arg to yyl_try()

I thought I was going to end up using this for more stuff, but I've
found better approaches.

This commit also removes two more goto targets.

3 years agotoke.c: factor out static yyl_keylookup()
Aaron Crane [Wed, 23 Oct 2019 16:14:16 +0000 (17:14 +0100)]
toke.c: factor out static yyl_keylookup()

3 years agotoke.c: factor out static yyl_key_core() and yyl_word_or_keyword()
Aaron Crane [Wed, 23 Oct 2019 15:51:58 +0000 (16:51 +0100)]
toke.c: factor out static yyl_key_core() and yyl_word_or_keyword()

3 years agotoke.c: bundle some yyl_just_a_word() params into a struct
Aaron Crane [Wed, 23 Oct 2019 12:11:02 +0000 (13:11 +0100)]
toke.c: bundle some yyl_just_a_word() params into a struct

This makes calls to it much easier to understand.

3 years agotoke.c: factor out static yyl_just_a_word()
Aaron Crane [Wed, 23 Oct 2019 11:21:53 +0000 (12:21 +0100)]
toke.c: factor out static yyl_just_a_word()

3 years agotoke.c: stop passing around several needless local variables
Aaron Crane [Tue, 22 Oct 2019 21:49:00 +0000 (22:49 +0100)]
toke.c: stop passing around several needless local variables

I introduced these parameters as part of mechanically refactoring goto-heavy
logic into subroutines. However, they aren't actually needed through most of
the code. Even in the recursive case (in which yyl_try() or one of its
callees will call itself), we can reset the variables to zero.

3 years agotoke.c: factor out static yyl_strictwarn_bareword()
Aaron Crane [Mon, 21 Oct 2019 15:51:54 +0000 (17:51 +0200)]
toke.c: factor out static yyl_strictwarn_bareword()

3 years agotoke.c: remove the really_sub goto label
Aaron Crane [Mon, 21 Oct 2019 12:12:16 +0000 (14:12 +0200)]
toke.c: remove the really_sub goto label

This permits some additional pleasing simplifications.

3 years agotoke.c: factor out static yyl_constant_op()
Aaron Crane [Mon, 21 Oct 2019 11:55:22 +0000 (13:55 +0200)]
toke.c: factor out static yyl_constant_op()

With the removal of another goto label!

3 years agotoke.c: factor out static yyl_safe_bareword()
Aaron Crane [Mon, 21 Oct 2019 11:39:10 +0000 (13:39 +0200)]
toke.c: factor out static yyl_safe_bareword()

3 years agotoke.c: fold some initialisations into the corresponding declarations
Aaron Crane [Mon, 21 Oct 2019 11:34:33 +0000 (13:34 +0200)]
toke.c: fold some initialisations into the corresponding declarations

3 years agotoke.c: factor out static yyl_fatcomma()
Aaron Crane [Mon, 21 Oct 2019 10:43:17 +0000 (12:43 +0200)]
toke.c: factor out static yyl_fatcomma()

This removes a goto label.

3 years agotoke.c: factor out static yyl_fake_eof()
Aaron Crane [Mon, 21 Oct 2019 10:17:40 +0000 (12:17 +0200)]
toke.c: factor out static yyl_fake_eof()

3 years agoFix copy-and-paste mistake in U8TO64_LE
Matt Turner [Mon, 4 Nov 2019 00:13:48 +0000 (16:13 -0800)]
Fix copy-and-paste mistake in U8TO64_LE

Fixes: https://github.com/Perl/perl5/issues/17244
Fixes: https://github.com/Perl/perl5/issues/17247
Fixes: https://rt.cpan.org/Ticket/Display.html?id=130890

3 years agoFactor out common code from sv_derived_from_* subs family
Sergey Aleynikov [Tue, 29 Oct 2019 20:40:03 +0000 (23:40 +0300)]
Factor out common code from sv_derived_from_* subs family
into one that takes both SV*/char*+len arguments, like hv_common,
to be able to use speedups from SV* stash lookup API.

3 years agoCorrections: one grammatical; one POD formatting
James E Keenan [Sun, 3 Nov 2019 20:24:36 +0000 (15:24 -0500)]
Corrections: one grammatical; one POD formatting

3 years agoop.h: Remove obsolete #define
Karl Williamson [Sat, 2 Nov 2019 21:02:19 +0000 (15:02 -0600)]
op.h: Remove obsolete #define

This is no longer used.

3 years agotoke.c: Fix bug tr/// upgrading to UTF-8 in middle
Karl Williamson [Sat, 2 Nov 2019 19:59:38 +0000 (13:59 -0600)]
toke.c: Fix bug tr/// upgrading to UTF-8 in middle

Consider tr/\x{ff}-\x{100}/AB/.

While parsing, the code keeps an offset from the beginning of the output
to the beginning of the second number in the range.  This is purely for
speed so that it wouldn't have to re-find the beginning of that value,
when it already knew it.

But the example above shows the folly of this shortcut.  The second
number in the range causes the output to be upgraded to UTF-8, which
makes that offset invalid in general.  Change to re-find the beginning.

3 years agomathoms.c,utf8.c: Update to use UTF8_CHK_SKIP
Karl Williamson [Sun, 3 Nov 2019 17:23:05 +0000 (10:23 -0700)]
mathoms.c,utf8.c: Update to use UTF8_CHK_SKIP

This new macro expands to what these did.  Update to use the common
code.

3 years agoPATCH: Character class code broke MSWin32 compilation
Karl Williamson [Sun, 3 Nov 2019 16:50:56 +0000 (09:50 -0700)]
PATCH: Character class code broke MSWin32 compilation

I'm not sure why this didn't show up elsewhere, but we have embed.fnc
entries for non-existent functions that should have been removed in
dd1a3ba7882ca70c1e85b0fd6c03d07856672075.

In addition, I see several more functions that should have been removed,
and this commit removes them.

3 years agoFix typo
Max Maischein [Sun, 3 Nov 2019 19:44:44 +0000 (20:44 +0100)]
Fix typo

As noted by @bulk88 in #12898

3 years agoAnd silence some silly examples.
Max Maischein [Fri, 25 Oct 2019 18:20:56 +0000 (20:20 +0200)]
And silence some silly examples.

From RT88754

Adapted to the current Perl by Max Maischein

Created from https://github.com/Perl/perl5/issues/11259

3 years agoAdd explicit list of supported Perl versions and URL where to find it
Max Maischein [Sun, 3 Nov 2019 18:15:29 +0000 (19:15 +0100)]
Add explicit list of supported Perl versions and URL where to find it

This addresses the comments of @bulk88 made in #12898

3 years agoa template for vulnerability announcements
Tony Cook [Tue, 10 Dec 2013 04:42:30 +0000 (15:42 +1100)]
a template for vulnerability announcements

not that we ever make mistakes

3 years agoInversion lists are SvPOK
Karl Williamson [Sat, 2 Nov 2019 16:06:33 +0000 (10:06 -0600)]
Inversion lists are SvPOK

They always have a string, and making them this allows B to access it.

3 years agoOn OP_READLINE, OPf_SPECIAL is set for <<>>, clear for <>.
Nicholas Clark [Sat, 2 Nov 2019 21:08:48 +0000 (22:08 +0100)]
On OP_READLINE, OPf_SPECIAL is set for <<>>, clear for <>.

3 years agoUpdate perlfaq to CPAN version 5.20191102
Karen Etheridge [Sat, 2 Nov 2019 05:39:50 +0000 (22:39 -0700)]
Update perlfaq to CPAN version 5.20191102

  [DELTA]

5.20191102  2019-11-02 05:34:43Z
  * fix bad pod markup in perlfaq8 (PR #78; thanks, Joaquín Ferrero!)
  * remove stale section about lib.pm (PR #82, Dan Book)
  * update perlfaq9 to reference Email::Stuffer (PR #79, Dan Book)
  * update perlfaq9 to reference URL::Search (PR #80, Dan Book)
  * update perlfaq9 to use HTTP::Tiny (PR #81, Dan Book)
  * fix some broken links (issue #71, dctabuyz)

3 years agoignore $^H{feature/bits'} when deparsing
Tony Cook [Sat, 2 Nov 2019 03:37:29 +0000 (14:37 +1100)]
ignore $^H{feature/bits'} when deparsing

This hints hash member is used to pass the scope's feature bits to
eval, and was confusing deparse.

Since the bits are also represented as the feature_ prefixed members
of %^H, there's no need for Deparse to do anything else with them.

3 years agoRmv more deprecated characlassify/case change macros
Karl Williamson [Fri, 1 Nov 2019 01:23:09 +0000 (19:23 -0600)]
Rmv more deprecated characlassify/case change macros

These were missed by 059703b088f44d5665f67fba0b9d80cad89085fd.

3 years agoperl.c: Remove obsolete comment
Karl Williamson [Fri, 1 Nov 2019 01:42:23 +0000 (19:42 -0600)]
perl.c: Remove obsolete comment

3 years agoConsolidate uses of PERL_SMALL_MACRO_BUFFER
Karl Williamson [Fri, 1 Nov 2019 00:59:17 +0000 (18:59 -0600)]
Consolidate uses of PERL_SMALL_MACRO_BUFFER

At the moment the _ASSERT_() is the one which has been showing large
expansions.  Change so it doesn't do anything if PERL_SMALL_MACRO_BUFFER
is defined.  That means various other calls that use
PERL_SMALL_MACRO_BUFFER can be simplified to not use it.

3 years agoperldelta for removing experimental status
Karl Williamson [Fri, 1 Nov 2019 00:46:05 +0000 (18:46 -0600)]
perldelta for removing experimental status

For both script runs and alpha assertions.

3 years agoperlreref: alpha assertions are valid in (COND)
Karl Williamson [Sun, 27 Oct 2019 15:01:55 +0000 (09:01 -0600)]
perlreref: alpha assertions are valid in (COND)

3 years agoAccept experimental script_run feature
Karl Williamson [Sun, 27 Oct 2019 14:45:32 +0000 (08:45 -0600)]
Accept experimental script_run feature

3 years agoAccept experimental alpha_assertions feature
Karl Williamson [Sun, 27 Oct 2019 14:23:01 +0000 (08:23 -0600)]
Accept experimental alpha_assertions feature

3 years agoperlexperiment: Was missing vlb
Karl Williamson [Sun, 27 Oct 2019 14:17:19 +0000 (08:17 -0600)]
perlexperiment: Was missing vlb

This experimental feature didn't make it into this pod.

3 years agoperlexperiment: Remove Unicode in EBCDIC
Karl Williamson [Sun, 27 Oct 2019 14:14:10 +0000 (08:14 -0600)]
perlexperiment: Remove Unicode in EBCDIC

This looks like a stray line.  We have supported Unicode in EBCDIC for
many releases with no experimental warnings raised.

3 years agofix panic caused by printf with %n with utf8 length magic
Tony Cook [Wed, 30 Oct 2019 02:06:34 +0000 (13:06 +1100)]
fix panic caused by printf with %n with utf8 length magic

fix the reported issue with gh #17221

3 years agoadd ${^FEATURE_BITS} to perlvar.pod
Tony Cook [Mon, 28 Oct 2019 23:45:11 +0000 (10:45 +1100)]
add ${^FEATURE_BITS} to perlvar.pod

3 years agoremove now unneeded FEATURE_IS_ENABLED()
Tony Cook [Mon, 28 Oct 2019 19:57:29 +0000 (06:57 +1100)]
remove now unneeded FEATURE_IS_ENABLED()

3 years agoFaster feature checks
Tony Cook [Thu, 24 Oct 2019 18:26:53 +0000 (05:26 +1100)]
Faster feature checks

Perform only a bit check instead of a much more expensive hash
lookup to test features.

For now I've just added a U32 to the cop structure to store the bits,
if we need more we could either add more bits directly, or make it a
pointer.

We don't have the immediate need for a pointer that warning do since
we don't dynamically add new features during compilation/runtime.

The changes to %^H are retained so that caller() can be used from perl
code to check the features enabled at a given caller's scope.

3 years agoMake me as doing the business for 20th November
Chris 'BinGOs' Williams [Wed, 30 Oct 2019 17:38:02 +0000 (17:38 +0000)]
Make me as doing the business for 20th November

3 years agoMention Termux in README.android
Richard Leach [Mon, 20 May 2019 01:37:04 +0000 (01:37 +0000)]
Mention Termux in README.android

3 years agoremove CONSERVATIVE and LIBERAL
Tomasz Konojacki [Tue, 29 Oct 2019 16:33:00 +0000 (17:33 +0100)]
remove CONSERVATIVE and LIBERAL

These constants were undocumented and don't do anything useful.
Saving a few kilobytes of memory doesn't justify the complexity
caused by adding a new build flag.

All platforms except 64-bit Windows were using LIBERAL. It's not
clear why win64 was using -DCONSERVATIVE, but removing it doesn't
break anything.

[gh #17232]

3 years agouse PTR2nat() instead of casting pointers to unsigned long
Tomasz Konojacki [Tue, 29 Oct 2019 15:31:19 +0000 (16:31 +0100)]
use PTR2nat() instead of casting pointers to unsigned long

Casting a pointer to unsigned long will result in truncation when
sizeof(void*) > sizeof(unsigned long)

[gh #17232]

3 years agoChange the names of some regnodes
Karl Williamson [Mon, 30 Sep 2019 02:55:45 +0000 (20:55 -0600)]
Change the names of some regnodes

The new name is shorter and I believe, clearer.

3 years agoregcomp.c: Comments, white-space only
Karl Williamson [Mon, 30 Sep 2019 02:47:46 +0000 (20:47 -0600)]
regcomp.c: Comments, white-space only