This is a live mirror of the Perl 5 development currently hosted at https://github.com/perl/perl5
James E Keenan [Sun, 21 Aug 2022 17:58:13 +0000 (17:58 +0000)]
Correct .mailmap entry for
a08a101a45
Bram [Fri, 12 Aug 2022 21:26:39 +0000 (23:26 +0200)]
t/io/eintr.t: only show diag message on failure
Also improve the error message by capturing the exception/error
returned by the `fnctl` call.
Bram [Fri, 12 Aug 2022 13:40:19 +0000 (15:40 +0200)]
t/io/eintr.t: Add `diag` when F_GETPIPE_SZ failed
When the fcntl call to get the size of the pipe buffer failed then
it would silently fallback to 0xfffff.
-> Make this fallback no longer silent and add a diagnostic message
when it happens.
Bram [Fri, 12 Aug 2022 13:27:13 +0000 (15:27 +0200)]
t/io/eintr.t: Fix sporadic test failure
The test would sporadically fail; troubleshooting this revealed that
the size of the pipe buffer could vary between pipes which this code
didn't take into account.
What the code did:
1. Create two pipes ($in, $out)
2. Get buffer size of '$out' pipe
3. Do something that should block (using the buffer size)
4. Create two new pipes ($in, $out)
5. Do something that should block (using the buffer size)
The problem: the pipe created in 4. does not always have the same
buffer size as the pipe created in 1.
Adding some debug info in the test:
# pipe buf size: 8193
ok 4 - read/die: read status
ok 5 - read/die: close status
# pipe size: 65537
not ok 6 - print/close: sig handler close status
During test 6 it reused the smaller buffer size and wrote data
to the new pipe. It expected this to block but the buffer of the pipe
used for test 6 is 65536 bytes. End result: the pipe buffer is not
full (only 8194 bytes were written) so the test does not block which
causes the alarm not to trigger which causes the test failure.
Fix all this by reinitialising `$pipe_buf_size` when creating new pipes.
(Observed by @khwilliamson, diagnosed with help of @wolfsage)
Bram [Fri, 12 Aug 2022 13:25:39 +0000 (15:25 +0200)]
t/io/eintr.t: Whitespace fixes
Most of the code was intended with 4 spaces but some of it was
intended with tab. Fix the inconsistenty and replace tabs with spaces.
Bram [Fri, 12 Aug 2022 13:09:36 +0000 (15:09 +0200)]
t/io/eintr.t: Rename variable to `$pipe_buf_size`
Rename the `$surely_this_arbitrary_number_is_fine` to `$pipe_buf_size`
since it's no longer an arbitrary number.
Yves Orton [Sun, 21 Aug 2022 10:29:30 +0000 (12:29 +0200)]
updateAUTHORS.pm - detect missing preambles in files
If the preamble is missing it is highly likely the file is corrupt
in some way, so refuse to process further. In the authors case it
means the file could be quite significantly misparsed.
This was suggested by @bram-perl.
mohawk2 [Tue, 19 Jul 2022 15:57:05 +0000 (16:57 +0100)]
remove_outputmap POD was titled remove_inputmap
Yves Orton [Sat, 20 Aug 2022 17:20:30 +0000 (19:20 +0200)]
updateAUTHORS.p[lm] - add testing for dupe names and emails in AUTHORS
And remove the duplicate for "vividsnow" that this found.
Yves Orton [Sat, 20 Aug 2022 16:27:32 +0000 (18:27 +0200)]
AUTHORS - remove GitHub (and exclude it explicitly)
Yves Orton [Thu, 11 Aug 2022 16:22:32 +0000 (18:22 +0200)]
AUTHORS, .mailmap - resolve issue #20010
Contributor has requested they be excluded from our contributor
databases.
Yves Orton [Thu, 11 Aug 2022 20:24:48 +0000 (22:24 +0200)]
checkAUTHORS.pl - delete and replace with updateAUTHORS.pl
updateAUTHORS.pl now can replace all the capabilities of
checkAUTHORS.pl, and checkAUTHORS.pl has a slightly different idea of
what needs to be validated, and is not aware of the
Porting/exclude_contrib.txt. Instead of teaching it to be aware and
maintaining two tools this drops checkAUTHORS.pl
In the process we move the logic for t/porting/pending-author.t into
t/porting/authors.t, by letting updateAUTHORS.pl do it as well. This is
helpful because updateAUTHORS.pl contains similar core git logic so that
it can implement some of its options.
Yves Orton [Tue, 9 Aug 2022 15:38:18 +0000 (17:38 +0200)]
updateAUTHORS.p[lm] - Add a way to rename authors properly
With the removal of the redundant data from .mailmap we lacked a way to
rename an author properly. Previously someone could manually update a
name or email in AUTHORS files and things would "just work" as the
.mailmap file would tell us the old mapping. Now we have removed those
mappings we need a way to easily rename someones canonical data. This
also means manually changing the AUTHORS file is now a problem.
Yves Orton [Tue, 9 Aug 2022 15:21:35 +0000 (17:21 +0200)]
updateAUTHORS.pm - remove redundant data from .mailmap
A number of devs have noted and raised concerns that having .mailmap
entries where the LHS and RHS of the entry is redundant. A few have also
expressed the view that this exposes unnecessarily exposes their email
address in an easily harvestible form.
On the technical level as far as git specifically is concerned it is
true this data is redundant, as git uses .mailmap to transform user data
that match the RHS of an entry into the value on LHS, and when they are
the same obviously it is a no-op. However on the technical level for our
infra these entries are not redundant. We can use them to identify and
correctly respond to many cases of manual update of the AUTHORS file,
for instance changing ones preferred name. With the .mailmap entries we
have the data to identify the old preferred name, and join it together
with the unchanged email for the user and then automatically update
their .mailmap entries. This is why these entries were created
originally.
However, I believe that this functionality is not useful enough to
require us to have discussions with contributors on this subject on a
regular basis. We can add command line options that allow people to
change the AUTHORS file and the .mailmap file properly, so we can drop
the "redundant" data and avoid the need to talk about why it is there.
The required functionality for changing names will come in a follow up
patch in this PR.
Yves Orton [Thu, 11 Aug 2022 16:17:11 +0000 (18:17 +0200)]
updateAUTHORS.pl - improved pod explanations
Improved POD verbiage. Explain how to exclude people in a more friendly
way. Various other pod cleanups and improvements.
Yves Orton [Fri, 12 Aug 2022 06:53:26 +0000 (08:53 +0200)]
updateAUTHORS.pl - add support for --tap
This emulates the test feature in checkAUTHORS.pl and in
t/porting/pending-author.t which overlaps with the tests that
checkAUTHORS.pl does on behalf of t/porting/authors.t
We test that the .mailmap and AUTHORS files are up to date. If the
working directory is not clean (eg git status --porcelain returns a non
empty list of files) then we also validate the git credentials in use
to make sure that if the changes were applied we would also still pass
the test.
The script implements the same logic that `git commit` uses to decide
what identity a commit would use with the exception of the last resort
measures git undertakes to use the hostname and similar files.
Yves Orton [Thu, 11 Aug 2022 16:23:06 +0000 (18:23 +0200)]
AUTHORS - update the blurb
The blurb mentions checkAUTHORS.pl, and we should point people at
updateAUTHORS.pl instead.
Yves Orton [Thu, 11 Aug 2022 16:18:42 +0000 (18:18 +0200)]
updateAUTHORS.pl - Add debugging option
This is helpful when adding a new option to updateAUTHORS.pl. It not
really meant for users, more for someone working on the tool itself, and
thus is not documented.
Yves Orton [Thu, 11 Aug 2022 15:51:48 +0000 (17:51 +0200)]
updateAUTHORS.pm - use fc() for sorting, and improved subs to do so
Use consistent hash key and list sort orders.
Yves Orton [Mon, 8 Aug 2022 20:31:41 +0000 (22:31 +0200)]
updateAUTHORS.pm - Fix windows support for git operations
Binmode the pipe, use tabs not nulls, and on windows use double quotes
not single quotes for arguments to subprocess scripts.
Since we weren't using Porting/updateAUTHORS.pm as part of our test
pipeline we didn't know it didnt work on Windows builds.
Yves Orton [Thu, 11 Aug 2022 14:02:57 +0000 (16:02 +0200)]
updateAUTHORS.p[lm] - add support for reports like checkAUTHORS.pl has
Adds the --stats, --files, --who, and related options similar to
what checkAUTHORS.pl offers. See perldoc Porting/updateAUTHORS.pl for
the list of options it supports.
Yves Orton [Fri, 12 Aug 2022 17:32:16 +0000 (19:32 +0200)]
updateAUTHORS.p[lm] - add exclusion support
This add a new file Porting/exclude_contrib.txt to hold a list
of base64 SHA-256 digests of the user name and email who should be
excluded.
This adds the options
--exclude-me
--exclude-contrib=NAME_AND_EMAIL
--exclude-missing
to add exclusions in different ways. --exclude-me uses the git
credentials for the current user, --exclude-contrib expects a name
and email, and --exclude-missing excludes someone who is missing from
the change log.
When excluding someone their name will be removed from AUTHORS if it is
already listed, and the relevant entries to .mailmap will be removed,
and if necessary additional exclusion entries will be added for any of
their old identities which were mentioned in .mailmap.
This feature will be used in a later patch to resolve GH issue #20010.
Yves Orton [Thu, 18 Aug 2022 14:36:53 +0000 (16:36 +0200)]
updateAUTHORS.pm - use _file suffix for methods that read/update files
Exactly what "update_authors" does is a little less clear than what
"update_authors_file" does.
Yves Orton [Thu, 18 Aug 2022 09:22:09 +0000 (11:22 +0200)]
updateAUTHORS.pl - add support for a commit range
checkAUTHORS.pl supports specifying a commit range, and we actually
need it even more as we do not support piping from git log (as our
git log format is ungainly) so this allows us to be feature compatible.
Yves Orton [Thu, 11 Aug 2022 13:51:53 +0000 (15:51 +0200)]
updateAUTHORS.pm - don't update files if they don't change
If we want to just check if the files are up to date we dont want
to have to write to the disk. We will use this in a future patch.
Yves Orton [Thu, 11 Aug 2022 16:08:27 +0000 (18:08 +0200)]
updateAUTHORS.pl - add support for verbose
Makes it easy to add diagnostics while developing.
Yves Orton [Thu, 18 Aug 2022 07:39:49 +0000 (09:39 +0200)]
updateAUTHORS.pl - use Getopt::Long in "hash ref" mode
This makes it easier and simpler to add new options and
extend the script in the future.
Yves Orton [Thu, 11 Aug 2022 13:17:47 +0000 (15:17 +0200)]
updateAUTHORS.pm - use object to store state and avoid passing args
It is much cleaner when the object stores its state internally. That
way we don't have to adjust the transitive caller graph when we need
something new.
Also normalize some var names, we were using "author_info" and
"authors_info" in a way that was confusing, now it is consistently
"author_info".
Yves Orton [Thu, 11 Aug 2022 12:59:27 +0000 (14:59 +0200)]
updateAUTHORS.pl - split into module Porting/updateAUTHORS.pm
As we extend the module logic mixing the docs for the module and the
command line tool becomes problematic, and it is pain passing around the
state via distinct variables. This is first step to moving to a simpler
model. We don't make much use of the OO in this patch at all, that will
come in a follow up patch.
Yves Orton [Sat, 20 Aug 2022 15:37:40 +0000 (17:37 +0200)]
t/porting/update_authors.t - initial tests for Porting/updateAUTHORS.pl
We can at least make sure it compiles... Later we can build on this, for
now this sanity checks that we haven't *completely* broken the script
while we refactor it.
Yves Orton [Thu, 18 Aug 2022 19:55:57 +0000 (21:55 +0200)]
testsuite.yml - fix and rename "authors" CI workflow to "authors_involved_debug"
The existing name is confusing to me, especially when the workflow gets
skipped so change the name to something more intuitive.
We also change it to respect .mailmap by using %aN and %aE instead of
%an and %ae.
We also change it to use ".." and "..." in the git log statement, and
use `sort | uniq -c` instead of `sort -u`, it is helpful to see counts.
We also show the resolved begin and end commits.
Yves Orton [Sat, 20 Aug 2022 15:50:14 +0000 (17:50 +0200)]
t/porting/podcheck.t - ignore .bak files
they are not pod, but rather editor droppings.
Yves Orton [Sun, 7 Aug 2022 08:11:31 +0000 (10:11 +0200)]
.mailmap/AUTHORS - remove duplicate listing
A. Sinan Unur was listed twice in AUTHORS, under Sinan Unur
as well. This removes the dupe.
This is in prep for various changes to the updateAUTHORS.pl process.
Karl Williamson [Tue, 6 Jun 2017 08:06:30 +0000 (02:06 -0600)]
Per-word utf8_to_bytes()
This changes utf8_to_bytes() to do a per-word initial scan to see if the
source is actually downgradable, before starting the conversion. This
is significantly faster than the current per-character scan. However,
the speed advantage evaporates in doing the actual conversion to being a
wash with the previous scheme.
Thus it finds out quicker if the source is downgradable.
cache grind yields this, based on a 100K character string; the
non-downgradable one has the next character after that be the only one
that's too large.:
Key:
Ir Instruction read
Dr Data read
Dw Data write
COND conditional branches
IND indirect branches
_m branch predict miss
_m1 level 1 cache miss
_mm last cache (e.g. L3) miss
- indeterminate percentage (e.g. 1/0)
The numbers represent relative counts per loop iteration, compared to
blead at 100.0%.
Higher is better: for example, using half as many instructions gives 200%,
while using twice as many gives 50%.
unicode::bytes_to_utf8_legal_API_test
Downgrading 100K valid characters
blead proposed
------ ------
Ir 100.00 99.99
Dr 100.00 100.03
Dw 100.00 100.04
COND 100.00 100.05
IND 100.00 100.00
COND_m 100.00 87.25
IND_m 100.00 100.00
Ir_m1 100.00 123.25
Dr_m1 100.00 100.18
Dw_m1 100.00 99.94
Ir_mm 100.00 100.00
Dr_mm 100.00 100.00
Dw_mm 100.00 100.00
unicode::bytes_to_utf8_illegal
Finding too high a character after 100K valid ones
blead fast
------ ------
Ir 100.00 188.91
Dr 100.00 179.77
Dw 100.00 66.75
COND 100.00 278.47
IND 100.00 100.00
COND_m 100.00 88.71
IND_m 100.00 100.00
Ir_m1 100.00 121.86
Dr_m1 100.00 100.01
Dw_m1 100.00 100.03
Ir_mm 100.00 100.00
Dr_mm 100.00 100.00
Dw_mm 100.00 100.00
Neil Bowers [Sat, 20 Aug 2022 21:45:23 +0000 (22:45 +0100)]
Updated Module::CoreList ready for 5.37.4
Neil Bowers [Sat, 20 Aug 2022 20:26:54 +0000 (21:26 +0100)]
Bumping perl version in various places for 5.37.4
Neil Bowers [Sat, 20 Aug 2022 20:06:36 +0000 (21:06 +0100)]
new perldelta for 5.37.4
Neil Bowers [Sat, 20 Aug 2022 20:02:22 +0000 (21:02 +0100)]
marked 5.37.3 as done in the release schedule
Neil Bowers [Sat, 20 Aug 2022 20:01:24 +0000 (21:01 +0100)]
Updated epigraphs.pod with the epigraph for 5.37.3
Neil Bowers [Sat, 20 Aug 2022 17:04:38 +0000 (18:04 +0100)]
Late addition to perldelta for a change related to FreeBSD
Neil Bowers [Sat, 20 Aug 2022 16:15:07 +0000 (17:15 +0100)]
Added 5.37.3 to perlhist
Neil Bowers [Sat, 20 Aug 2022 15:58:27 +0000 (16:58 +0100)]
updated perldelta for CPAN module changes
Neil Bowers [Sat, 20 Aug 2022 15:34:59 +0000 (16:34 +0100)]
Update Module::CoreList for 5.37.3
Neil Bowers [Sat, 20 Aug 2022 13:34:51 +0000 (14:34 +0100)]
perldelta WIP
Neil Bowers [Sat, 20 Aug 2022 13:34:25 +0000 (14:34 +0100)]
Updates from Porting/updateAUTHORS.pl
Neil Bowers [Sat, 20 Aug 2022 10:38:43 +0000 (11:38 +0100)]
version of Module-CoreList wasn't matching what's on CPAN
BinGOs told me that this updated was all that was required,
and I chose to trust him.
Neil Bowers [Sat, 20 Aug 2022 09:21:17 +0000 (10:21 +0100)]
Importing Socket 2.036 from CPAN
Neil Bowers [Sat, 20 Aug 2022 09:18:26 +0000 (10:18 +0100)]
Import Scalar-Util 1.63 from CPAN
Neil Bowers [Sat, 20 Aug 2022 09:16:38 +0000 (10:16 +0100)]
Importing Filter 1.64 from CPAN
Neil Bowers [Sat, 20 Aug 2022 09:15:35 +0000 (10:15 +0100)]
Import ExtUtils-PL2Bat 0.005 from CPAN
Karl Williamson [Sat, 20 Aug 2022 14:00:38 +0000 (08:00 -0600)]
Make freebsd not use POSIX 2008 locales
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=265950
We do not yet know the extent of the problem.
Karl Williamson [Thu, 18 Aug 2022 19:37:45 +0000 (13:37 -0600)]
makedef: Export certain symbols
This fixes #20090
These symbols were changed by fcbef0e to be defined in more
Configurations, but as @bram-perl pointed out, makedef.pl didn't also
change to export them in those expanded situations.
Nicholas Clark [Fri, 5 Aug 2022 13:05:54 +0000 (15:05 +0200)]
dl_dyld.xs should not create a `mode` variable only to ignore it
This unused variable dates back to when the file was first added. The code
was structured as a static C wrapper emulating the dlopen() API with the
native APIs, and then a copy of the existing XS code that called dlopen().
However, there is (and was) no need to keep it exactly this way, as the
wrapper is only visible inside this file, meaning that it does not need to
conform to any externally defined prototype.
小鸡 [Tue, 16 Aug 2022 07:00:07 +0000 (15:00 +0800)]
Pathtools: update version info in files meta.json and meta.xml to adapt the real module version
Graham Knop [Thu, 18 Aug 2022 10:27:01 +0000 (12:27 +0200)]
fix t/porting/authors.t detection of merge commits
git-rev-parse HEAD^2` was used to detect if HEAD was a merge commit.
When HEAD is not a merge commit then HEAD^2 produces an error;
When HEAD is a merge commit then HEAD^2 points to the second parent.
The problem was that `git rev-parse "HEAD^2"` also produced output on STDOUT
which caused `$revision_range` to be set to something bogus.
Example:
$ git rev-parse HEAD^2 2>/dev/null ; echo $?
HEAD^2
128
$ git log HEAD^1..HEAD^2
fatal: ambiguous argument 'HEAD^1..HEAD^2': unknown revision or path not in the working tree.
Use '--' to separate paths from revisions, like this:
'git <command> [<revision>...] -- [<file>...]'
The fix is to use --verify which doesn't give output on STDOUT for something invalid.
It does print an error on STDERR (for invalid things) which can be silenced using -q
$ git rev-parse --verify HEAD^2
fatal: Needed a single revision
$ git rev-parse -q --verify HEAD^2
$
For an actual merge commit:
$ git rev-parse -q --verify e91a97a^2
bee6601bbf238b5366ce11cd41f6b095bfeebfd8
(i.e. working as expected)
Karl Williamson [Mon, 15 Aug 2022 19:11:37 +0000 (13:11 -0600)]
diag.t: Rmv obsolete don't cares
These messages are obsolete. They exist to tell perl not to complain
when it finds a matching diagnostic that isn't listed in perldiag. But
the messages don't get generated any more, so running the test without
them still passes.
Many of these were spotted by @bram-perl
Karl Williamson [Mon, 15 Aug 2022 19:28:45 +0000 (13:28 -0600)]
diag.t: Collapse "Function not implemented" entries
These are clear enough without explanation in perldiag. There's a bunch
of them that can be represented by just one.
Karl Williamson [Mon, 15 Aug 2022 19:24:33 +0000 (13:24 -0600)]
diag.t: Collapse all Usage: entries
These are clear enough without explanation in perldiag. There's a bunch
of them that can really be represented by just one.
Karl Williamson [Mon, 15 Aug 2022 19:21:41 +0000 (13:21 -0600)]
plan9/plan9.c: Rmv double space
This diagnostic is the outlier of all the other ones.
Bram [Tue, 16 Aug 2022 13:49:38 +0000 (15:49 +0200)]
Fix make_patchnum.pl when run from subdir
When running 'make_patchnum.pl' from a subdir then 'make_patchnum.pl'
failed to write a proper lib/Config_git.pl. That is: all values in it
were set to the empty string instead to the actual values.
Example:
$ cd perl5
$ mkdir foo && cd $foo
$ ../miniperl -I../lib ../make_patchnum.pl
Updating 'git_version.h' and 'lib/Config_git.pl'
$ cat ../lib/Config_git.pl
######################################################################
# WARNING: 'lib/Config_git.pl' is generated by make_patchnum.pl
# DO NOT EDIT DIRECTLY - edit make_patchnum.pl instead
######################################################################
$Config::Git_Data=<<'ENDOFGIT';
git_commit_id=''
git_describe=''
git_branch=''
git_uncommitted_changes=''
git_commit_id_title=''
This got broken in commit
3f1788e11f2685299067ac0f8d3e4fd141a5b5cd which
added support for -Dmksymlink.
That commit basically assumed that the stating point was always '.' and
that the './.git' directory exist. Neither are true when the command is
run from inside a subdir.
The proper thing to do is to first find the `$root` directory and then
check if it's in -Dmksymlinks target dir. (Luckily all that code already
exists, we just need to move the -Dmksymlinks logic a bit down.)
With this change applied:
$ ../miniperl -I../lib ../make_patchnum.pl
Updating 'git_version.h' and 'lib/Config_git.pl'
$ cat lib/Config_git.pl
######################################################################
# WARNING: 'lib/Config_git.pl' is generated by make_patchnum.pl
# DO NOT EDIT DIRECTLY - edit make_patchnum.pl instead
######################################################################
$Config::Git_Data=<<'ENDOFGIT';
git_commit_id='
c589bf60082d119a14e11ef05a77fed6d36aba4b'
git_describe='v5.37.2-170-gc589bf6'
git_branch='bram/fix-make_patchnum-in-subdir'
git_uncommitted_changes='0'
git_commit_id_title='Commit id:'
git_commit_date='2022-08-16 15:49:38 +0200'
ENDOFGIT
Note:
- to build on Windows one must do a `cd win32` before building. This is how
I noticed the file was incorrect.
- I've tested running make_patchnum from:
* the root
* a subdir
* a '-Dmksymlinks' target dir
* a subdir inside a '-Dmksymlinks' target
(PS: I also fixed the indentation from tabs to 4 spaces)
Karl Williamson [Thu, 18 Aug 2022 13:20:32 +0000 (07:20 -0600)]
locale.c: Make sure LC_ALL gets changed when necessary
This fixes #20085.
In the new POSIX API for thread-safe locales, introduced in the 2008 version
of the standard, no method was defined to find out what the current locale
is. (That has finally been changed in the 2017 version, but I am
unaware of any boxes (accessible to me) implementing it.)
Darwin added their own extension to accomplish this, and Windows instead
used an extension to the old API to accomplish thread-safety. And glibc
has their own, hidden extension, which it's possible to Configure Perl
to use, but, since, it's not documented, it is not enabled by default.
Perl maps the old API to the new one when compiled for threads, and
where thread-safety is available. On Configurations where there isn't
the ability to query the current locale, it itself keeps a record of
that information, maintained from start-up of the process.
Commit 19e550d changed things so that when calling the function that
changes a category's locale, if the value to change to is the same as it
already was, it is a no-op.
But LC_ALL is precisely the sum of all the other locale categories. If
any one of them changes, LC_ALL must be updated. There are occasions
where all subcategories are updated in a tight loop, and, to save
unnecessary work, only the final iteration updates LC_ALL (as each
iteration would throw away the results of the previous iteration).
The blamed commit inadvertently caused the code to skip updating LC_ALL
when the final iteration wouldn't result in a change in the locale of
its category, whereas LC_ALL should have been updated on that iteration
if any previous iteration affected a change.
This new commit causes LC_ALL to always be updated on the final
iteration, regardless of what happened in the previous ones. To only
update it when absolutely necessary, we would have to have an extra
internal state (meaning a new PL_ interpreter variable) indicating that
we are in a loop and whether changes have occurred while in that loop.
That's certainly feasible, but I don't think the extra complexity is
worth it.
Karl Williamson [Thu, 18 Aug 2022 13:13:48 +0000 (07:13 -0600)]
locale.c: Change to use an enum parameter
These two functions take as input a parameter that is three-valued.
Basically, the meanings are 'true', 'false', and 'special'. This
changes them to use an enum instead, so that the compiler will detect if
they are being called with an unanticipated value.
The parameter names now become more self-documenting.
Comments are added to better explain what 'special' means, and why it
exists.
Karl Williamson [Sun, 14 Aug 2022 12:38:22 +0000 (06:38 -0600)]
locale.c: Remove redundant loop iteration
This loop recalculates LC_ALL on the final non-LC_ALL iteration, and
then it did an extra iteration, just to recalculate LC_ALL. This is
unnecessary and could cause memory leaks.
Yves Orton [Wed, 10 Aug 2022 10:51:46 +0000 (12:51 +0200)]
perl.h - remove visibility attribute on cygwin
There is no attribute visibility on cygwin apparently, and the warnings
seem to overwhelm the cygwin build process.
Karl Williamson [Thu, 18 Aug 2022 14:57:41 +0000 (08:57 -0600)]
Merge branch 'more loc_tools beyond
2fbb1a06df' into blead
These commits were inadvertently left out of
2fbb1a06df7219e842b3a2a0c9dcbab10de433fc
Karl Williamson [Mon, 12 Apr 2021 11:34:25 +0000 (05:34 -0600)]
t/loc_tools.pl: Turn off warnings in a timely manner
It doesn't matter much, but some warnings might be output by doing the
'use locale' before turning off warnings.
Karl Williamson [Mon, 12 Apr 2021 11:27:43 +0000 (05:27 -0600)]
t/loc_tools.pl: White-space only
Karl Williamson [Mon, 12 Apr 2021 11:13:25 +0000 (05:13 -0600)]
t/loc_tools.pl: Check for unsupported locales
This commit causes us not to view unsupported locales as legitimate for
testing. Core dumps occurred on some platforms without this.
It looks for a diagnostic that the next commit in this series will
generate.
Karl Williamson [Wed, 27 Jul 2022 18:40:53 +0000 (12:40 -0600)]
Langinfo.t: Handle systems without LC_ALL
Bram [Wed, 17 Aug 2022 21:08:39 +0000 (23:08 +0200)]
win32: check if CCHOME is correctly set
When building on Windows one must provide a value for 'CCHOME'.
The value of 'CCHOME' is used to set the value of 'CCLIBDIR' and 'CCINCDIR'.
(These eventually become `$Config{libpth}` and `$Config{incpath}`.)
When 'CCHOME' is incorrectly set the build mostly works; what fails
is building cpan/Win32 and this then fails with non-obvious errors such as:
Warning (mostly harmless): No library found for -luserenv
Warning (mostly harmless): No library found for -lwinhttp
...
g++ Win32.def -o ....
C:/.../ld.exe: ... undefined reference to `WinHttpCrackUrl'
...
collect2.exe: error: ld returned 1 exit status
gmake[1]: *** [Makefile:472: ..\..\lib\auto\Win32\Win32.dll] Error 1
gmake[1]: Leaving directory 'C:/Perl/perl5/cpan/Win32'
Unsuccessful make(cpan/Win32): code=512 at ..\make_ext.pl line 584.
Make this a bit more obvious by checking if the path(s) specified in
'CCLIBDIR' and 'CCINCDIR' actually exist.
If these do not exist then stop the build process at an early stage with
a clear(er) error.
Example error:
..\miniperl.exe -I..\lib config_sh.PL ...
'CCLIBDIR' contains the following non-existing paths:
C:\MinGW\lib
C:\MinGW\x86_64-w64-mingw32\lib
C:\MinGW\lib\gcc\x86_64-w64-mingw32\8.3.0
Did you provide a correct value for the 'CCHOME' option?
(This check can be skipped by using the SKIP_CCHOME_CHECK=1 option)
gmake: *** [GNUmakefile:1176: ..\config.sh] Error 1
Some notes:
- This check can be skipped/overridden by adding 'SKIP_CCHOME_CHECK=1'
on the make cmd line. (I added this because I don't know if there are
valid use cases for having a non-existing path in 'libpth'/'incpath')
- My first idea was to do this in the Makefile/GNUmakefile at a very early
stage but that is more complex to do and the resulting code is messy/ugly.
So instead I opted to do it in 'config_sh.PL' since this is the
first(/only) place that uses these vars.
- I (only) tested on Windows 10 using GNU Make and gcc
Karl Williamson [Thu, 18 Aug 2022 13:58:45 +0000 (07:58 -0600)]
Merge branch 'loc_tools' into blead
t/loc_tools.pl is a set of utility functions for testing scripts to use
to query the current platform to see if various capabilies exist on the
platform.
Locale handling is not very uniform across platforms, and these
utilities are to allow test scripts to easily skip tests, or to find
valid locales to test on.
Some platforms that are bare-bones, or which emphasize security, turn off
all or most locale handling altogether. Others accept just about any
garbage as parameters to their functions, while doing nothing. Others
act reliably.
Locale-related bugs reported to vendors don't tend to get fixed. We
reported an incorrect symbolic link about Darwin long ago, trivial to
fix, but last I checked no action had been taken. glibc has a warnocked
bug reported by us, as does freebsd. (There may be others.) I happen
to have the link to the freebsd one handy, in case you're interested,
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=255646
This series of commits adds some defenses against locale issues.
A locale name is legally allowed to be anything, opaque to the caller as
to its meaning. The only requirement is that what setlocale returns as
a value for a locale should be able to be passed into a future setlocale
as valid input. Windows has at least one locale that violates this.
However, the X/Open Portability Guide (XPG) does specify a locale name
syntax, and it appears that most modern Unixy systems tend to follow
this.
The problematic Windows locale has a comma in its name; commas are
illegal in the XPG specification. One of the commits in this series
hence won't return a locale whose name contains a comma. Hence a
user of these utilities doesn't have to cope with a one-off locale name
of this form. (There are previously existing checks in loc_tools.pl
that keep other glitchy locales from being returned).
Undocumented, AFAICT, is that, on some platforms (but I don't think all)
a locale category that has associated strings has to have LC_CTYPE set
to the same locale as that category for things to work. An example is
LC_TIME where the names of the days of the week, etc, are strings. Core
dumps have occurred when LC_CTYPE and LC_TIME aren't the same value.
One commit here makes sure that in testing a category, LC_CTYPE is
brought along, so as to avoid a potential core dump
Karl Williamson [Mon, 12 Apr 2021 11:10:37 +0000 (05:10 -0600)]
t/loc_tools.pl: Don't allow commas in locale names
This is a problem in some buggy Windows versions. A Macedonian
sub-locale has a comma in its returned name, and Windows can't parse its
own name.
This addition keeps locales with this bug from being tested, as the XPG
standard doesn't allow them.
Karl Williamson [Mon, 12 Apr 2021 11:08:10 +0000 (05:08 -0600)]
t/loc_tools.pl: Add checks
This verifies that we can restore a locale that we were previously in,
and makes sure that we don't stay in a locale that doesn't work well.
Doing so has led to crashes.
Karl Williamson [Mon, 12 Apr 2021 11:00:35 +0000 (05:00 -0600)]
t/loc_tools.pl: Fail earlier
Move the code that returns failure into the loop, so won't keep
iterating if failure is going to happen anyway.
Karl Williamson [Mon, 12 Apr 2021 10:36:44 +0000 (04:36 -0600)]
t/loc_tools.pl: Refactor _trylocale()
This function is used to see if a locale actually works on the current
platform. I was not fully aware of the glitches if a category is in one
locale, and LC_CTYPE is in another. This makes sure they are both the
same; and this results in some simplification.
Karl Williamson [Mon, 12 Apr 2021 02:49:59 +0000 (20:49 -0600)]
t/loc_tools.pl: Print better failing test numbers
This test module is 'required' from other perl test files. If it
encounters an error, it doesn't know how to report it. What it does is
call fail() if available, and a home-grown one otherwise. Prior to this
commit the home-grown version just made all tests number 0. This commit
changes that to make them sequentially numbered from a very high
starting one, so as not to interfere with the outer calls.
Karl Williamson [Wed, 10 Aug 2022 15:06:25 +0000 (09:06 -0600)]
loc_tools.pl: Move code in file
This is for future commits to call after it is defined
Karl Williamson [Wed, 3 Aug 2022 14:12:22 +0000 (08:12 -0600)]
loc_tools: Create function, refactor
category_excluded() is a better name for what the current name
'is_category_valid" does. And it turns out that we need to use this
function on (those few) systems that don't have LC_CTYPE to avoid
executing code that depends on LC_CTYPE.
And we do need an improved is_category_valid() for other checks.
Karl Williamson [Thu, 11 Aug 2022 00:05:44 +0000 (18:05 -0600)]
perl.c: Rm redundant initialization of PL_C_locale_obj
This is also initialized in locale.c
Spotted by Tony Cook.
Karl Williamson [Thu, 18 Aug 2022 03:11:53 +0000 (21:11 -0600)]
hints/freebsd.sh: Don't use querylocale() due to bugs
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=255646
Karl Williamson [Thu, 18 Aug 2022 02:58:05 +0000 (20:58 -0600)]
locale.c: Fix wrongly worded panic error message
This was a copy/paste error. The message wasn't changed to match the
new situation it was being called from.
Karl Williamson [Wed, 17 Aug 2022 22:26:49 +0000 (16:26 -0600)]
run/locale.t: Skip collation test if no LC_COLLATION
Bram [Tue, 16 Aug 2022 14:14:11 +0000 (16:14 +0200)]
win32/GNUmakefile: correct $(CONFIGPM) deps
The `$(CONFIGPM}` target depends on the 'lib/Config_git.pl' file which
is created by the '..\git_version.h' target (this runs `make_patchnum.pl`
which creates both files).
Build log when dependency is missing:
..\miniperl.exe -I..\lib ..\configpm --chdir=..
written lib/Config.pod
updated lib/Config.pm
updated lib/Config_heavy.pl
Warning: failed to load Config_git.pl, something strange about this perl...
..\miniperl.exe -I..\lib config_h.PL "ARCHPREFIX="
Running config_h.PL
Writing config.h
Warning: failed to load Config_git.pl, something strange about this perl...
config.h has changed
(....)
..\miniperl.exe -I..\lib ..\make_patchnum.pl
Updating 'git_version.h' and 'lib/Config_git.pl'
-> It first ran `configpm` and only later it ran `make_patchnum.pl`
Build log with the dependency:
..\miniperl.exe -I..\lib ..\make_patchnum.pl
Updating 'git_version.h' and 'lib/Config_git.pl'
(...)
..\miniperl.exe -I..\lib ..\configpm --chdir=..
written lib/Config.pod
updated lib/Config.pm
updated lib/Config_heavy.pl
..\miniperl.exe -I..\lib config_h.PL "ARCHPREFIX="
Running config_h.PL
Writing config.h
config.h has changed
Notes:
- in ./Makefile.SH the same dependency was added in
commit
744ac0eac539aead596d9fa09cc15a20b09a5884 (and later updated in
commit
0f13ebd5d71f81771c1044e2c89aff29b408bfec)
- in ./win32/Makefile the same dependency was added in
commit
680b2c5ee3b53c627074192b3cf14416a24da6ea (the dependency fix
however was not mentioned in the commit message)
Bram [Wed, 17 Aug 2022 16:18:45 +0000 (18:18 +0200)]
Add extra scope block for switch case (fix g++)
Code in
9fdd7fc4796d89d16dceea42f2af91e4fde296ed broke the g++ builds,
Basically after the commit the code looked like:
switch (o->op_type) {
...
case OP_SASSIGN:
...
OP* rhs = cBINOPx(o)->op_first;
OP* lval = cBINOPx(o)->op_last;
...
break;
case OP_AASSIGN: {
g++ does not allow this and errors with:
peep.c:3897:14: error: jump to case label
3897 | case OP_AASSIGN: {
| ^~~~~~~~~~
peep.c:3844:17: note: crosses initialization of 'OP* lval'
3844 | OP* lval = cBINOPx(o)->op_last;
| ^~~~
peep.c:3843:17: note: crosses initialization of 'OP* rhs'
3843 | OP* rhs = cBINOPx(o)->op_first;
| ^~~
This happens because `rhs` and `lval` are not scoped in the case statement
so it could fall through to the next case.
The solution is to scope them which this commit now does by adding a
separate scope for `OP_SASSIGN` (similar to `OP_AASSIGN`).
Fixes #20108
Karl Williamson [Sun, 14 Aug 2022 18:23:34 +0000 (12:23 -0600)]
locale.c: Fix memory leak
Bram [Tue, 16 Aug 2022 10:07:36 +0000 (12:07 +0200)]
Add 'g++' build in the GitHub workflow
Also do a build with 'g++' since it can catch more errors.
(I accidentally broke builds with g++ and that commit got merged
on blead; smokers then reported build failures with g++ but it would
have been nicer if I knew this before it got merged.)
Paul "LeoNerd" Evans [Wed, 17 Aug 2022 13:10:39 +0000 (14:10 +0100)]
Add a PadnameREFCNT_inc() macro
Implemented as a static inline function call, so that it can return the
padname pointer itself. This would allow use in expressions such as
ptr->field = PadnameREFCNT_inc(pn);
That makes it similar to the familiar SvREFCNT_inc() macro.
Richard Leach [Wed, 17 Aug 2022 12:23:15 +0000 (12:23 +0000)]
Perldelta for GH#19943 (OP_PADSV_STORE)
The wording has been left generic, since other such optimizations
are in the PR queue and could just be tacked on the end later.
Reini Urban [Mon, 15 Aug 2022 22:18:51 +0000 (22:18 +0000)]
Filter-Util-Call: sync with CPAN version 1.63
James E Keenan [Mon, 15 Aug 2022 21:47:48 +0000 (21:47 +0000)]
CUSTOMIZED entry needs relative path from distribution
Per Porting/Maintainers.pl, "The path [to an entry in the CUSTOMIZED
element of a particular module's entry in %MODULES] should be relative
to the distribution directory." In this case, the distribution
directory is 'cpan/Filter-Util/', so we have to go up two levels in
order to position ourselves to see 'pod/perlfilter.pod'.
If not corrected, when running 'perl Porting/sync-with-cpan
Filter::Util::Call', we get this message:
error: pathspec 'Filter-Util-Call/pod/perlfilter.pod'
did not match any file(s) known to git"
Not surprisingly, we also get a failure in 't/porting/customized.t' if
we don't correct this.
Aristotle Pagaltzis [Sun, 14 Aug 2022 19:35:41 +0000 (19:35 +0000)]
Memoize: sync with CPAN version 1.10
From Changes:
1.10 Thu 11 Aug 2022
* No warning when NORMALIZER returns undef (RT #143921).
This was a backcompat breakage in 1.09
1.09 Sun 31 Jul 2022
* Skip for thread safety tests on Perl 5.10.0
* Drastically improved Memoized::NDBM_File EXISTS shim
* Minimum Perl version set at 5.6
* Removal of Memoize::ExpireTest and Memoize::ExpireFile
1.08 Thu 28 Jul 2022
* Skip for thread safety tests on unthreaded perls
1.07 Thu 28 Jul 2022
* Thread safety (RT #21707)
* Support for reference values in Memoize::Expire (RT #38344)
* Support for iterating tied Memoize::Expire hashes (RT #124507)
* Advisory locking in Memoize::Storable (RT #133859)
1.06 Wed 27 Jul 2022
* Deprecation of Memoize::SDBM_File
* Test fixes
1.05 Sun 24 Jul 2022
* Actual implementation of the HASH option of Memoize::Expire (RT #18171, Eric Brine)
* Vim modeline in Expire.pm (RT #38785, Moritz Lenz)
* Fix for minor memory leak in unmemoize() (RT #52819, Joshua I. Miller)
* Actual implementation of the 'nstore' option of Memoize::Storable (RT #77790, Jonathan Nieder)
* Fix for broken LIST_CACHE => MERGE option in memoize() (RT #91927, Slaven Rezic)
1.04 Fri 22 Jul 2022
* No removal of . from @INC during TIE module load.
The module load is not optional, so the . removal was mistaken
* Properly overridable $Memoize::Expire::DEBUG (RT #133860, Slaven Rezic).
Previously it would get reset the module was loaded
* Correct INSTALLDIRS for installation on pre-5.12 perls (RT #63935)
* Documentation fixes (David Steinbrunner, Dominic Hargreaves, Slaven Rezic, Glenn Fowler)
Committer:
* Remove CUSTOMIZED entry for Memoize in Porting/Maintainers.pl.
* Update upstream maintainer.
* On CPAN, Memoize now distributed as '.tar.gz'.
Richard Leach [Fri, 8 Jul 2022 14:52:19 +0000 (14:52 +0000)]
Implement OP_PADSV_STORE - combined sassign/padsv OP
This commit introduces a new OP to replace simple cases
of OP_SASSIGN and OP_PADSV.
For example, 'my $x = 1' is currently implemented as:
1 <;> nextstate(main 1 -e:1) v:{
2 <$> const(IV 1) s
3 <0> padsv[$x:1,2] sRM*/LVINTRO
4 <2> sassign vKS/2
But now will be turned into:
1 <;> nextstate(main 1 -e:1) v:{
2 <$> const(IV 1) s
3 <1> padsv_store[$x:1,2] vKMS/LVINTRO
This intended to be a transparent performance optimization.
It should be applicable for RHS optrees of varying complexity.
David Mitchell [Wed, 17 Aug 2022 08:20:22 +0000 (09:20 +0100)]
add missing =cut to sv.h
The just-added CvREFCOUNTED_ANYSV apidoc entry was missing the =cut
at the end.
Tony Cook [Mon, 15 Aug 2022 05:55:47 +0000 (15:55 +1000)]
prevent calling recursing into S_concat_pat() for an empty array
If AvARRAY() was NULL (and it is in one test) this would cause
undefined behaviour in the S_concat_pat() outer for loop.
Paul "LeoNerd" Evans [Tue, 9 Aug 2022 11:55:04 +0000 (12:55 +0100)]
Define a CvREFCOUNTED_ANYSV flag
If this flag is set, then the CvXSUBANY(cv).any_sv pointer will have its
reference count decremented when the CV itself is freed. This is useful
for XS extensions that wish to store extra data in here. Without this
flag, such extensions have to resort to using magic with a 'free'
function to perform this work.
Bram [Tue, 16 Aug 2022 09:21:13 +0000 (11:21 +0200)]
locale.c: Fix trial_locales declaration (fix g++)
Commit
e57aaf750b3d2461053eff690d0954bc8fb9d7f3 broke builds with g++
because the declaration of `trial_locales` was incorrect. It contained
an extra 'struct' keyword which is wrong.
Errors on g++:
locale.c:3539:12: error: using typedef-name 'trial_locales_struct' after 'struct'
struct trial_locales_struct trial_locales[5];
^
locale.c:3537:7: note: 'trial_locales_struct' has a previous declaration here
} trial_locales_struct;
(Build tested with gcc-12, g++-12, gcc-4.9, g++-4.9)
Yves Orton [Sat, 13 Aug 2022 12:59:16 +0000 (14:59 +0200)]
t/porting/authors.t - respect .mailmap in test
The logic in checkAUTHORS.pl internally uses the "Author:" data from
git log --pretty=fuller
which respects .mailmap entries, thus it is actually equivalent to
git log --pretty='format:%aN %aE'
whereas "%an %ae" will show the raw uncorrected values, this can then
make authors.t think there are more and different authors than there
actually are.
Karl Williamson [Sun, 14 Aug 2022 12:31:42 +0000 (06:31 -0600)]
locale.c: Clarify comment
Bram [Sun, 7 Aug 2022 13:22:29 +0000 (15:22 +0200)]
Cleanup 'use system default locale' on WIN32
The 'use system default locale' for WIN32 was messy...
During the 'determine fallback' process it would add the empty
string as 'trial_locale' to signal the code to try the system default on
windows.
This was handled by an extra if condition at the start of the loop
to special case this.
Clean all this up by determining the 'real' value for `trial_locale` and
add that in the list.
(Note: I did test this change on Win32 but the system default failback
currently doesn't work as expected due to GH #20054. The failure is
unrelated to this commit.)
Bram [Sun, 7 Aug 2022 13:22:24 +0000 (15:22 +0200)]
Set fallback_(desc|name) when adding trial_locales
When adding an extra item to the `trial_locales` list then already
set the 'fallback description' and 'fallback name' to show in the
warning message.
This makes the code easier since now the later code doesn't need to
determine the message again.
This commit doesn't alter the output of the warning message
when the locale settings are incorrect.