Add file existence test for Porting/makerel This came up in the release of 5.39.7. Previously autodie.pm had been upgraded, which removed the file touch_me from the Perl distribution. The name touch_me was still mentioned in Porting/makerel , but this only became apparent when makerel was used to actually build a tarball. This test crudely parses the list of filenames from Porting/makerel and checks that all the files referenced there exist.
Handle Linux extra locale categories Linux has six locale categories that aren't in the POSIX standard. This commit enables their use. For example there is meta data about a locale, and information about how telephone numbers are formatted for it. On non-Linux boxes, you can now request them as well, but you will get stub values, as if the locale were C.
OP_AELEMFASTLEX_STORE: only in void context For the optimisation which converts $lex[N] = expr into an OP_AELEMFASTLEX_STORE op, only optimise if the scalar assign is in VOID context. (See the previous commit for a very similar change to OP_PADSV_STORE.) This allows us to stop pp_aelemfastlex_store() from uselessly pushing the result onto the stack, only to be immediately popped again by the following nextstate or unstack op. This becomes more important on PERL_RC_STACK builds, as each push or pop involves manipulating the SV's reference count. I'm working on the assumption that scalar/list/unknown cxt lexical array element assigns are rare enough that not optimising them is less of a loss than optimising the void case. So: $lex[0] = ....; # void: now even faster $other = $lex[0] = ...; # scalar: now slow again foo($lex[0] = ..., ....); # list: now slow again sub { .... $lex[0] = ...; # unknown: now slow again }
OP_PADSV_STORE: only in void context For the optimisation which converts $lex = expr into an OP_PADSV_STORE op, only optimise if the scalar assign is in VOID context. This allows us to stop pp_padsv_store() from uselessly pushing the result onto the stack, only to be immediately popped again by the following nextstate or unstack op. This becomes more important on PERL_RC_STACK builds, as each push or pop involves manipulating the SV's reference count. I'm working on the assumption that scalar/list/unknown cxt lexical assigns are rare enough that not optimising them is less of a loss than optimising the void case. So: $lex = ....; # void: now even faster $other = $lex = ...; # scalar: now slow again foo($lex = ..., ....); # list: now slow again sub { .... $lex = ...; # unknown: now slow again }
allow some basic infrastructure to load with -Dusedefaultstrict The changes to t/test.pl appear to be real bugs. This allows `make test_harness` to run, but many tests will still fail under -Dusedefaultstrict This addresses #21732 but does not fix it. I'm unsure how supported that build option is.
locale.c: Be sure to toggle into dot radix locale This fixes GH #21746 Perl keeps the LC_NUMERIC category in a locale where the radix character is a dot, regardless of what the user has requested. This is because much XS code has been written with the dot assumption. When the user's actual radix character is desired, the locale is briefly toggled to that one for the duration of the operation. When the user changes the LC_NUMERIC locale, the new one is noted, but the attempted change is otherwise ignored unless its radix is a dot. The new one will be briefly toggled into when appropriate. The blamed commit contains a logic error commit 818cdb7aa9f85227c1c7313257c6204c872beb94 Author: Karl Williamson <khw@cpan.org> AuthorDate: Sun Apr 11 05:57:07 2021 -0600 Commit: Karl Williamson <khw@cpan.org> CommitDate: Thu Sep 1 09:02:04 2022 -0600 locale.c: Skip code if will be a no-op It decided it was a no-op if the new locale that the user is changing to is the same as the previous locale. But it didn't consider that what actually happens is that the new locale does actually get changed, and this code is supposed to make sure that, before returning control to the user, that a dot radix locale is in effect. If the new locale is a dot radix locale, then no harm is done by skipping the code, but otherwise things can go wrong. I am chagrined that I made this logic error without noticing before it got pushed, and am surprised that it took this long for the error to surrface. There must be something else intervening to make this not a problem in most circumstances, but I haven't analyzed what it might be. The details as to why it happened in this test case are pretty obscure. The locale in effect is looking for a comma radix, but what is being checked for is a Perl version number, like 5.0936. When converting that to a floating point number, the dot is not recognized, and only the initial '5' is found. The failing code in a module has different actions depending on the current perl version it is being called from, and the conditional got the answer wrong because 5 is less than 5.0936, whereas the actual version is above that. So it did the wrong thing and caused an error.
Fix tr/\N{latin1}...\N{above latin1}/ When a string is being parsed, it isn't made UTF-8 until necessary; that is, when it first finds a character that requires UTF-8 to represent. If all the characters prior to that one are ASCII, all that is needed is to convert that one to UTF-8 and to turn on the UTF-8 flag, so that all future characters encountered in the parse will be represented in UTF-8. This is because all ASCII characters have the same representation in UTF-8 as not; they are "UTF-8 invariant". But if a UTF-8 *variant* character was in the string prior to the UTF-8-required one, it must be converted to its UTF-8 representation, when the string is converted. All that is needed is to increment a count of variant characters as the parse proceeds. If nothing in the string requires UTF-8 by the end of the parse, the count is ignored and the string remains non-UTF-8. And if the count is zero when a UTF-8-required character is found, as mentioned above, that character is converted to UTF-8, and the flag is set to use UTF-8 going forward. But a non-zero count at the first UTF-8-required character indicates that before proceeding, the already-parsed string must be reparsed to convert the variant characters already in it to UTF-8. The count was not being incremented when the input notation used \N{}; this commit fixes that. It was being incremented when the input notation used \x{}, which is much more common in the field, so this bug was unnoticed for a long time. Fixes #21748 (Just for the record, on EBCDIC platforms more characters are UTF-8 invariant than ASCII platforms; the macros called here hide that from the code.)
make generic "Out of memory!" error more specific The problem: When perl runs out of memory, it outputs a generic "Out of memory!" error and exits. This makes it hard to track down what's happening in a complex system, especially since the message does not even mention perl. This patch contains two main changes: 1. vec() in lvalue context can throw fake "Out of memory!" errors when it discovers that the index being assigned to is too big. Unlike real allocation errors, these are trappable with try {}/eval {}. This message has been changed to "Out of memory during vec in lvalue context" (and since it comes from a Perl_croak() call, it will generally have a script name and line number attached). 2. All other places in the source code that can emit "Out of memory!" errors have been changed to attach a location identifier to the message. For example: "Out of memory in perl:util:safesysmalloc" This way the error message at least mentions "perl". Fixes #21672.
allow "used only once" warnings to be fatal "used only once" warnings are special, instead of being emitted at the code where the name in question is used, they are emitted during a scan of the symbol table done after parsing has finished. This meant that any FATAL flags set in the COP for the parse point of the name is no longer in scope, so the warnings we emit can't be treated as fatal. To make them behave as FATAL set a new flag on the name if fatal WARN_ONCE warnings are enabled and use that to dispatch the warnings as normal or fatally when we do the symbol table scan. I originally approached the dispatch as fatal or non-fatal by messing around with cop_warnings, but that was dumb, and I went for a much simpler change. Fixes #13814
skip porting/libperl.t on maint releases maint-* branches, whether supported or not, may not be updated to handle changes to nm's output due to updates to the operating system or toolchain. The same applies doubly to release tarballs. So skip this test for maint/version numbers. Keep the test for devel releases, since they have limited support windows anyway, and we want this test to be done where possible. Fixes #21677
fix Perl_rpp_is_lone() for immortals. Perl_rpp_is_lone() checks whether an SV is kept alive only by the args and/or temps stack, and thus can in various circumstances be used directly rather than copying - such as a return value from a function, or assigning a value to an array or hash. However, this assumption fails on immortal SVs like PL_sv_undef. Their refcount can reach 1 but they should still be copied rather than stolen. This showed itself as an extremely intermittent failure in cpan/Test-Harness/t/compat/test-harness-compat.t It only showed up when both these conditions were met: - on DEBUGGING builds, because that sets the initial ref count of the immortals to 1000 rather than I32_MAX, making such problems easier to reproduce; *and* - under PERL_RC_STACK, because only on that build do you have SVs on the args stack (but not temps stack) with RC==1 which are stealable. Also under the RC build, as an optimisation, &PL_sv_undef is often pushed on the stack without incrementing its RC. Thus it is more liklely to reach RC==1. I don't know why the failure on that test was intermittent, even after setting PERL_HASH_SEED=0. Perhaps it was timing related. Certainly anything which made it slower, such as strace, valgrind or ASAN, made the problem go away or fail only after days rather than minutes. This commit also decreases the value of SvREFCNT_IMMORTAL on debugging builds from 1000 to 10 under PERL_RC_STACK, to make such issues/bugs more likely to be triggered.
pp_aassign: remove panic check under PERL_RC_STACK There is some code in S_aassign_copy_common() (helper function for pp_aassign()) which checks for SvIS_FREED() of the current arg. The comments imply that the check is there specifically to help a TODO test in t/op/sort.t not randomly crash. That test does stuff which triggers a premature free of args on the stack. This test was later disabled entirely, as whatever panics or crashes were being triggered weren't being caught by the eval {} it was wrapped in. This commit, only on PERL_RC_STACK builds, skips the SvIS_FREED check, and re-enables the test in sort.t (as a real test, not a TODO).
op/array.t: skip \$#{[]} test under PERL_RC_STACK PERL_RC_STACK builds fixes a bug whereby the [] was being prematurely freed. So the test, which was testing that perl didn't crash when trying to set the length of a freed array, was now trying to set the size of the array to some large number, and was failing under ASAN.