=item Is the implementation generic enough to be portable?
The worst patches make use of a system-specific features. It's highly
-unlikely that nonportable additions to the Perl language will be
+unlikely that non-portable additions to the Perl language will be
accepted.
=item Is the implementation tested?
=head2 Keeping in sync
The source code to the Perl interpreter, in its different versions, is
-kept in a repository managed by a revision control system ( which is
-currently the Perforce program, see http://perforce.com/ ). The
-pumpkings and a few others have access to the repository to check in
-changes. Periodically the pumpking for the development version of Perl
-will release a new version, so the rest of the porters can see what's
-changed. The current state of the main trunk of repository, and patches
-that describe the individual changes that have happened since the last
-public release are available at this location:
+kept in a repository managed by the git revision control system. The
+pumpkings and a few others have write access to the repository to check in
+changes.
- http://public.activestate.com/pub/apc/
- ftp://public.activestate.com/pub/apc/
+How to clone and use the git perl repository is described in L<perlrepository>.
-If you're looking for a particular change, or a change that affected
-a particular set of files, you may find the B<Perl Repository Browser>
-useful:
+You can also choose to use rsync to get a copy of the current source tree
+for the bleadperl branch and all maintenance branches:
- http://public.activestate.com/cgi-bin/perlbrowse
+ $ rsync -avz rsync://perl5.git.perl.org/perl-current .
+ $ rsync -avz rsync://perl5.git.perl.org/perl-5.12.x .
+ $ rsync -avz rsync://perl5.git.perl.org/perl-5.10.x .
+ $ rsync -avz rsync://perl5.git.perl.org/perl-5.8.x .
+ $ rsync -avz rsync://perl5.git.perl.org/perl-5.6.x .
+ $ rsync -avz rsync://perl5.git.perl.org/perl-5.005xx .
+
+(Add the C<--delete> option to remove leftover files)
+
+To get a full list of the available sync points:
+
+ $ rsync perl5.git.perl.org::
You may also want to subscribe to the perl5-changes mailing list to
receive a copy of each patch that gets submitted to the maintenance
state of evolution. You should expect it to be very buggy. Do B<not> use
it for any purpose other than testing and development.
-Keeping in sync with the most recent branch can be done in several ways,
-but the most convenient and reliable way is using B<rsync>, available at
-ftp://rsync.samba.org/pub/rsync/ . (You can also get the most recent
-branch by FTP.)
-
-If you choose to keep in sync using rsync, there are two approaches
-to doing so:
-
-=over 4
-
-=item rsync'ing the source tree
-
-Presuming you are in the directory where your perl source resides
-and you have rsync installed and available, you can "upgrade" to
-the bleadperl using:
-
- # rsync -avz rsync://public.activestate.com/perl-current/ .
-
-This takes care of updating every single item in the source tree to
-the latest applied patch level, creating files that are new (to your
-distribution) and setting date/time stamps of existing files to
-reflect the bleadperl status.
-
-Note that this will not delete any files that were in '.' before
-the rsync. Once you are sure that the rsync is running correctly,
-run it with the --delete and the --dry-run options like this:
-
- # rsync -avz --delete --dry-run rsync://public.activestate.com/perl-current/ .
-
-This will I<simulate> an rsync run that also deletes files not
-present in the bleadperl master copy. Observe the results from
-this run closely. If you are sure that the actual run would delete
-no files precious to you, you could remove the '--dry-run' option.
-
-You can than check what patch was the latest that was applied by
-looking in the file B<.patch>, which will show the number of the
-latest patch.
-
-If you have more than one machine to keep in sync, and not all of
-them have access to the WAN (so you are not able to rsync all the
-source trees to the real source), there are some ways to get around
-this problem.
-
-=over 4
-
-=item Using rsync over the LAN
-
-Set up a local rsync server which makes the rsynced source tree
-available to the LAN and sync the other machines against this
-directory.
-
-From http://rsync.samba.org/README.html :
-
- "Rsync uses rsh or ssh for communication. It does not need to be
- setuid and requires no special privileges for installation. It
- does not require an inetd entry or a daemon. You must, however,
- have a working rsh or ssh system. Using ssh is recommended for
- its security features."
-
-=item Using pushing over the NFS
-
-Having the other systems mounted over the NFS, you can take an
-active pushing approach by checking the just updated tree against
-the other not-yet synced trees. An example would be
-
- #!/usr/bin/perl -w
-
- use strict;
- use File::Copy;
-
- my %MF = map {
- m/(\S+)/;
- $1 => [ (stat $1)[2, 7, 9] ]; # mode, size, mtime
- } `cat MANIFEST`;
-
- my %remote = map { $_ => "/$_/pro/3gl/CPAN/perl-5.7.1" } qw(host1 host2);
-
- foreach my $host (keys %remote) {
- unless (-d $remote{$host}) {
- print STDERR "Cannot Xsync for host $host\n";
- next;
- }
- foreach my $file (keys %MF) {
- my $rfile = "$remote{$host}/$file";
- my ($mode, $size, $mtime) = (stat $rfile)[2, 7, 9];
- defined $size or ($mode, $size, $mtime) = (0, 0, 0);
- $size == $MF{$file}[1] && $mtime == $MF{$file}[2] and next;
- printf "%4s %-34s %8d %9d %8d %9d\n",
- $host, $file, $MF{$file}[1], $MF{$file}[2], $size, $mtime;
- unlink $rfile;
- copy ($file, $rfile);
- utime time, $MF{$file}[2], $rfile;
- chmod $MF{$file}[0], $rfile;
- }
- }
-
-though this is not perfect. It could be improved with checking
-file checksums before updating. Not all NFS systems support
-reliable utime support (when used over the NFS).
-
-=back
-
-=item rsync'ing the patches
-
-The source tree is maintained by the pumpking who applies patches to
-the files in the tree. These patches are either created by the
-pumpking himself using C<diff -c> after updating the file manually or
-by applying patches sent in by posters on the perl5-porters list.
-These patches are also saved and rsync'able, so you can apply them
-yourself to the source files.
-
-Presuming you are in a directory where your patches reside, you can
-get them in sync with
-
- # rsync -avz rsync://public.activestate.com/perl-current-diffs/ .
-
-This makes sure the latest available patch is downloaded to your
-patch directory.
-
-It's then up to you to apply these patches, using something like
-
- # last="`cat ../perl-current/.patch`.gz"
- # rsync -avz rsync://public.activestate.com/perl-current-diffs/ .
- # find . -name '*.gz' -newer $last -exec gzcat {} \; >blead.patch
- # cd ../perl-current
- # patch -p1 -N <../perl-current-diffs/blead.patch
-
-or, since this is only a hint towards how it works, use CPAN-patchaperl
-from Andreas König to have better control over the patching process.
-
-=back
-
-=head2 Why rsync the source tree
-
-=over 4
-
-=item It's easier to rsync the source tree
-
-Since you don't have to apply the patches yourself, you are sure all
-files in the source tree are in the right state.
-
-=item It's more reliable
-
-While both the rsync-able source and patch areas are automatically
-updated every few minutes, keep in mind that applying patches may
-sometimes mean careful hand-holding, especially if your version of
-the C<patch> program does not understand how to deal with new files,
-files with 8-bit characters, or files without trailing newlines.
-
-=back
-
-=head2 Why rsync the patches
-
-=over 4
-
-=item It's easier to rsync the patches
-
-If you have more than one machine that you want to keep in track with
-bleadperl, it's easier to rsync the patches only once and then apply
-them to all the source trees on the different machines.
-
-In case you try to keep in pace on 5 different machines, for which
-only one of them has access to the WAN, rsync'ing all the source
-trees should than be done 5 times over the NFS. Having
-rsync'ed the patches only once, I can apply them to all the source
-trees automatically. Need you say more ;-)
-
-=item It's a good reference
-
-If you do not only like to have the most recent development branch,
-but also like to B<fix> bugs, or extend features, you want to dive
-into the sources. If you are a seasoned perl core diver, you don't
-need no manuals, tips, roadmaps, perlguts.pod or other aids to find
-your way around. But if you are a starter, the patches may help you
-in finding where you should start and how to change the bits that
-bug you.
-
-The file B<Changes> is updated on occasions the pumpking sees as his
-own little sync points. On those occasions, he releases a tar-ball of
-the current source tree (i.e. perl@7582.tar.gz), which will be an
-excellent point to start with when choosing to use the 'rsync the
-patches' scheme. Starting with perl@7582, which means a set of source
-files on which the latest applied patch is number 7582, you apply all
-succeeding patches available from then on (7583, 7584, ...).
-
-You can use the patches later as a kind of search archive.
-
-=over 4
-
-=item Finding a start point
-
-If you want to fix/change the behaviour of function/feature Foo, just
-scan the patches for patches that mention Foo either in the subject,
-the comments, or the body of the fix. A good chance the patch shows
-you the files that are affected by that patch which are very likely
-to be the starting point of your journey into the guts of perl.
-
-=item Finding how to fix a bug
-
-If you've found I<where> the function/feature Foo misbehaves, but you
-don't know how to fix it (but you do know the change you want to
-make), you can, again, peruse the patches for similar changes and
-look how others apply the fix.
-
-=item Finding the source of misbehaviour
-
-When you keep in sync with bleadperl, the pumpking would love to
-I<see> that the community efforts really work. So after each of his
-sync points, you are to 'make test' to check if everything is still
-in working order. If it is, you do 'make ok', which will send an OK
-report to perlbug@perl.org. (If you do not have access to a mailer
-from the system you just finished successfully 'make test', you can
-do 'make okfile', which creates the file C<perl.ok>, which you can
-than take to your favourite mailer and mail yourself).
-
-But of course, as always, things will not always lead to a success
-path, and one or more test do not pass the 'make test'. Before
-sending in a bug report (using 'make nok' or 'make nokfile'), check
-the mailing list if someone else has reported the bug already and if
-so, confirm it by replying to that message. If not, you might want to
-trace the source of that misbehaviour B<before> sending in the bug,
-which will help all the other porters in finding the solution.
-
-Here the saved patches come in very handy. You can check the list of
-patches to see which patch changed what file and what change caused
-the misbehaviour. If you note that in the bug report, it saves the
-one trying to solve it, looking for that point.
-
-=back
-
-If searching the patches is too bothersome, you might consider using
-perl's bugtron to find more information about discussions and
-ramblings on posted bugs.
-
-If you want to get the best of both worlds, rsync both the source
-tree for convenience, reliability and ease and rsync the patches
-for reference.
-
-=back
-
-=head2 Working with the source
-
-Because you cannot use the Perforce client, you cannot easily generate
-diffs against the repository, nor will merges occur when you update
-via rsync. If you edit a file locally and then rsync against the
-latest source, changes made in the remote copy will I<overwrite> your
-local versions!
-
-The best way to deal with this is to maintain a tree of symlinks to
-the rsync'd source. Then, when you want to edit a file, you remove
-the symlink, copy the real file into the other tree, and edit it. You
-can then diff your edited file against the original to generate a
-patch, and you can safely update the original tree.
-
-Perl's F<Configure> script can generate this tree of symlinks for you.
-The following example assumes that you have used rsync to pull a copy
-of the Perl source into the F<perl-rsync> directory. In the directory
-above that one, you can execute the following commands:
-
- mkdir perl-dev
- cd perl-dev
- ../perl-rsync/Configure -Dmksymlinks -Dusedevel -D"optimize=-g"
-
-This will start the Perl configuration process. After a few prompts,
-you should see something like this:
-
- Symbolic links are supported.
-
- Checking how to test for symbolic links...
- Your builtin 'test -h' may be broken.
- Trying external '/usr/bin/test -h'.
- You can test for symbolic links with '/usr/bin/test -h'.
-
- Creating the symbolic links...
- (First creating the subdirectories...)
- (Then creating the symlinks...)
-
-The specifics may vary based on your operating system, of course.
-After you see this, you can abort the F<Configure> script, and you
-will see that the directory you are in has a tree of symlinks to the
-F<perl-rsync> directories and files.
-
-If you plan to do a lot of work with the Perl source, here are some
-Bourne shell script functions that can make your life easier:
-
- function edit {
- if [ -L $1 ]; then
- mv $1 $1.orig
- cp $1.orig $1
- vi $1
- else
- /bin/vi $1
- fi
- }
-
- function unedit {
- if [ -L $1.orig ]; then
- rm $1
- mv $1.orig $1
- fi
- }
-
-Replace "vi" with your favorite flavor of editor.
-
-Here is another function which will quickly generate a patch for the
-files which have been edited in your symlink tree:
-
- mkpatchorig() {
- local diffopts
- for f in `find . -name '*.orig' | sed s,^\./,,`
- do
- case `echo $f | sed 's,.orig$,,;s,.*\.,,'` in
- c) diffopts=-p ;;
- pod) diffopts='-F^=' ;;
- *) diffopts= ;;
- esac
- diff -du $diffopts $f `echo $f | sed 's,.orig$,,'`
- done
- }
-
-This function produces patches which include enough context to make
-your changes obvious. This makes it easier for the Perl pumpking(s)
-to review them when you send them to the perl5-porters list, and that
-means they're more likely to get applied.
-
-This function assumed a GNU diff, and may require some tweaking for
-other diff variants.
-
=head2 Perlbug administration
-There is a single remote administrative interface for modifying bug status,
-category, open issues etc. using the B<RT> I<bugtracker> system, maintained
-by I<Robert Spier>. Become an administrator, and close any bugs you can get
+There is a single remote administrative interface for modifying bug status,
+category, open issues etc. using the B<RT> bugtracker system, maintained
+by Robert Spier. Become an administrator, and close any bugs you can get
your sticky mitts on:
- http://rt.perl.org
-
-The bugtracker mechanism for B<perl5> bugs in particular is at:
-
- http://bugs6.perl.org/perlbug
+ http://bugs.perl.org/
To email the bug system administrators:
"perlbug-admin" <perlbug-admin@perl.org>
-
=head2 Submitting patches
Always submit patches to I<perl5-porters@perl.org>. If you're
patching a core module and there's an author listed, send the author a
copy (see L<Patching a core module>). This lets other porters review
your patch, which catches a surprising number of errors in patches.
-Either use the diff program (available in source code form from
-ftp://ftp.gnu.org/pub/gnu/ , or use Johan Vromans' I<makepatch>
-(available from I<CPAN/authors/id/JV/>). Unified diffs are preferred,
-but context diffs are accepted. Do not send RCS-style diffs or diffs
-without context lines. More information is given in the
-I<Porting/patching.pod> file in the Perl source distribution. Please
-patch against the latest B<development> version (e.g., if you're
-fixing a bug in the 5.005 track, patch against the latest 5.005_5x
-version). Only patches that survive the heat of the development
-branch get applied to maintenance versions.
+Please patch against the latest B<development> version. (e.g., even if
+you're fixing a bug in the 5.8 track, patch against the C<blead> branch in
+the git repository.)
+
+If changes are accepted, they are applied to the development branch. Then
+the maintenance pumpking decides which of those patches is to be
+backported to the maint branch. Only patches that survive the heat of the
+development branch get applied to maintenance versions.
Your patch should update the documentation and test suite. See
-L<Writing a test>.
+L<Writing a test>. If you have added or removed files in the distribution,
+edit the MANIFEST file accordingly, sort the MANIFEST file using
+C<make manisort>, and include those changes as part of your patch.
+
+Patching documentation also follows the same order: if accepted, a patch
+is first applied to B<development>, and if relevant then it's backported
+to B<maintenance>. (With an exception for some patches that document
+behaviour that only appears in the maintenance branch, but which has
+changed in the development version.)
To report a bug in Perl, use the program I<perlbug> which comes with
Perl (if you can't get Perl to work, send mail to the address
I<perlbug@perl.org> or I<perlbug@perl.com>). Reporting bugs through
I<perlbug> feeds into the automated bug-tracking system, access to
-which is provided through the web at http://bugs.perl.org/ . It
+which is provided through the web at http://rt.perl.org/rt3/ . It
often pays to check the archives of the perl5-porters mailing list to
see whether the bug you're reporting has been reported before, and if
so whether it was considered a bug. See above for the location of
The CPAN testers ( http://testers.cpan.org/ ) are a group of
volunteers who test CPAN modules on a variety of platforms. Perl
-Smokers ( http://archives.develooper.com/daily-build@perl.org/ )
-automatically tests Perl source releases on platforms with various
-configurations. Both efforts welcome volunteers.
+Smokers ( http://www.nntp.perl.org/group/perl.daily-build and
+http://www.nntp.perl.org/group/perl.daily-build.reports/ )
+automatically test Perl source releases on platforms with various
+configurations. Both efforts welcome volunteers. In order to get
+involved in smoke testing of the perl itself visit
+L<http://search.cpan.org/dist/Test-Smoke>. In order to start smoke
+testing CPAN modules visit L<http://search.cpan.org/dist/CPANPLUS-YACSmoke/>
+or L<http://search.cpan.org/dist/minismokebox/> or
+L<http://search.cpan.org/dist/CPAN-Reporter/>.
It's a good idea to read and lurk for a while before chipping in.
That way you'll get to see the dynamic of the conversations, learn the
best way to study it is to read it in conjunction with poking at Perl
source, and we'll do that later on.
-You might also want to look at Gisle Aas's illustrated perlguts -
-there's no guarantee that this will be absolutely up-to-date with the
-latest documentation in the Perl core, but the fundamentals will be
-right. ( http://gisle.aas.no/perl/illguts/ )
+Gisle Aas's "illustrated perlguts", also known as I<illguts>, has very
+helpful pictures:
+
+L<http://search.cpan.org/dist/illguts/>
=item L<perlxstut> and L<perlxs>
=item The perl5-porters FAQ
-This should be available from http://simon-cozens.org/writings/p5p-faq ;
-alternatively, you can get the FAQ emailed to you by sending mail to
-C<perl5-porters-faq@perl.org>. It contains hints on reading perl5-porters,
-information on how perl5-porters works and how Perl development in general
-works.
+This should be available from http://dev.perl.org/perl5/docs/p5p-faq.html .
+It contains hints on reading perl5-porters, information on how
+perl5-porters works and how Perl development in general works.
=back
=item Core modules
-Modules shipped as part of the Perl core live in the F<lib/> and F<ext/>
-subdirectories: F<lib/> is for the pure-Perl modules, and F<ext/>
-contains the core XS modules.
+Modules shipped as part of the Perl core live in various subdirectories, where
+two are dedicated to core-only modules, and two are for the dual-life modules
+which live on CPAN and may be maintained separately with respect to the Perl
+core:
+
+ lib/ is for pure-Perl modules, which exist in the core only.
+
+ ext/ is for XS extensions, and modules with special Makefile.PL
+ requirements, which exist in the core only.
+
+ cpan/ is for dual-life modules, where the CPAN module is
+ canonical (should be patched first).
+
+ dist/ is for dual-life modules, where the blead source is
+ canonical.
+
+For some dual-life modules it has not been discussed if the CPAN version or the
+blead source is canonical. Until that is done, those modules should be in
+F<cpan/>.
=item Tests
=item Configure
-The configure process is the way we make Perl portable across the
+The Configure process is the way we make Perl portable across the
myriad of operating systems it supports. Responsibility for the
-configure, build and installation process, as well as the overall
-portability of the core code rests with the configure pumpkin - others
-help out with individual operating systems.
+Configure, build and installation process, as well as the overall
+portability of the core code rests with the Configure pumpkin -
+others help out with individual operating systems.
+
+The three files that fall under his/her responsibility are Configure,
+config_h.SH, and Porting/Glossary (and a whole bunch of small related
+files that are less important here). The Configure pumpkin decides how
+patches to these are dealt with. Currently, the Configure pumpkin will
+accept patches in most common formats, even directly to these files.
+Other committers are allowed to commit to these files under the strict
+condition that they will inform the Configure pumpkin, either on IRC
+(if he/she happens to be around) or through (personal) e-mail.
The files involved are the operating system directories, (F<win32/>,
F<os2/>, F<vms/> and so on) the shell scripts which generate F<config.h>
and F<Makefile>, as well as the metaconfig files which generate
F<Configure>. (metaconfig isn't included in the core distribution.)
+See http://perl5.git.perl.org/metaconfig.git/blob/HEAD:/README for a
+description of the full process involved.
+
=item Interpreter
And of course, there's the core of the Perl interpreter itself. Let's
resembles the code found in L<perlembed>; most of the real action takes
place in F<perl.c>
+F<perlmain.c> is generated by L<writemain> from F<miniperlmain.c> at
+make time, so you should make perl to follow this along.
+
First, F<perlmain.c> allocates some memory and constructs a Perl
-interpreter:
+interpreter, along these lines:
1 PERL_SYS_INIT3(&argc,&argv,&env);
2
interpreter. It's quite a simple function, and the guts of it looks like
this:
- my_perl = (PerlInterpreter*)PerlMem_malloc(sizeof(PerlInterpreter));
+ my_perl = (PerlInterpreter*)PerlMem_malloc(sizeof(PerlInterpreter));
Here you see an example of Perl's system abstraction, which we'll see
later: C<PerlMem_malloc> is either your system's C<malloc>, or Perl's
own C<malloc> as defined in F<malloc.c> if you selected that option at
configure time.
-Next, in line 7, we construct the interpreter; this sets up all the
-special variables that Perl needs, the stacks, and so on.
+Next, in line 7, we construct the interpreter using perl_construct,
+also in F<perl.c>; this sets up all the special variables that Perl
+needs, the stacks, and so on.
Now we pass Perl the command line options, and tell it to go:
- exitstatus = perl_parse(my_perl, xs_init, argc, argv, (char **)NULL);
- if (!exitstatus) {
- exitstatus = perl_run(my_perl);
- }
+ exitstatus = perl_parse(my_perl, xs_init, argc, argv, (char **)NULL);
+ if (!exitstatus)
+ perl_run(my_perl);
+ exitstatus = perl_destruct(my_perl);
+
+ perl_free(my_perl);
C<perl_parse> is actually a wrapper around C<S_parse_body>, as defined
in F<perl.c>, which processes the command line options, sets up any
=item Exception handing
-Perl's exception handing (i.e. C<die> etc) is built on top of the low-level
+Perl's exception handing (i.e. C<die> etc.) is built on top of the low-level
C<setjmp()>/C<longjmp()> C-library functions. These basically provide a
way to capture the current PC and SP registers and later restore them; i.e.
a C<longjmp()> continues at the point in code where a previous C<setjmp()>
analysis of Java code, but later the cpd part of it was extended to
parse also C and C++.
-Download the pmd-X.y.jar from the SourceForge site, and then run
-it on source code thusly:
+Download the pmd-bin-X.Y.zip () from the SourceForge site, extract the
+pmd-X.Y.jar from it, and then run that on source code thusly:
java -cp pmd-X.Y.jar net.sourceforge.pmd.cpd.CPD --minimum-tokens 100 --files /some/where/src --language c > cpd.txt
The C<-Wall> is by default on.
-The C<-ansi> (and its sidekick, C<-pedantic>) would be nice to be
-on always, but unfortunately they are not safe on all platforms,
-they can for example cause fatal conflicts with the system headers
-(Solaris being a prime example). The C<cflags> frontend selects
-C<-ansi -pedantic> for the platforms where they are known to be safe.
+The C<-ansi> (and its sidekick, C<-pedantic>) would be nice to be on
+always, but unfortunately they are not safe on all platforms, they can
+for example cause fatal conflicts with the system headers (Solaris
+being a prime example). If Configure C<-Dgccansipedantic> is used,
+the C<cflags> frontend selects C<-ansi -pedantic> for the platforms
+where they are known to be safe.
Starting from Perl 5.9.4 the following extra flags are added:
=back
The following flags would be nice to have but they would first need
-their own Stygian stablemaster:
+their own Augean stablemaster:
=over 4
C<-Wstrict-prototypes>
-=item *
-
=back
The C<-Wtraditional> is another example of the annoying tendency of
-gcc to bundle a lot of warnings under one switch -- it would be
-impossible to deploy in practice because it would complain a lot -- but
+gcc to bundle a lot of warnings under one switch (it would be
+impossible to deploy in practice because it would complain a lot) but
it does contain some warnings that would be beneficial to have available
on their own, such as the warning about string constants inside macros
containing the macro arguments: this behaved differently pre-ANSI
You may find it helpful to have a "macro dictionary", which you can
produce by saying C<cpp -dM perl.c | sort>. Even then, F<cpp> won't
-recursively apply those macros for you.
+recursively apply those macros for you.
=head2 gdb macro support
F<dump.c>; these work a little like an internal
L<Devel::Peek|Devel::Peek>, but they also cover OPs and other structures
that you can't get at from Perl. Let's take an example. We'll use the
-C<$a = $b + $c> we used before, but give it a bit of context:
+C<$a = $b + $c> we used before, but give it a bit of context:
C<$b = "6XXXX"; $c = 2.3;>. Where's a good place to stop and poke around?
What about C<pp_add>, the function we examined earlier to implement the
arranges for two C<NV>s to be placed into C<left> and C<right> - let's
slightly expand it:
- #define dPOPTOPnnrl_ul NV right = POPn; \
- SV *leftsv = TOPs; \
- NV left = USE_LEFT(leftsv) ? SvNV(leftsv) : 0.0
+ #define dPOPTOPnnrl_ul NV right = POPn; \
+ SV *leftsv = TOPs; \
+ NV left = USE_LEFT(leftsv) ? SvNV(leftsv) : 0.0
C<POPn> takes the SV from the top of the stack and obtains its NV either
directly (if C<SvNOK> is set) or by calling the C<sv_2nv> function.
C<TOPs> takes the next SV from the top of the stack - yes, C<POPn> uses
C<TOPs> - but doesn't remove it. We then use C<SvNV> to get the NV from
-C<leftsv> in the same way as before - yes, C<POPn> uses C<SvNV>.
+C<leftsv> in the same way as before - yes, C<POPn> uses C<SvNV>.
Since we don't have an NV for C<$b>, we'll have to use C<sv_2nv> to
convert it. If we step again, we'll find ourselves there:
All right, we've now had a look at how to navigate the Perl sources and
some things you'll need to know when fiddling with them. Let's now get
on and create a simple patch. Here's something Larry suggested: if a
-C<U> is the first active format during a C<pack>, (for example,
+C<U> is the first active format during a C<pack>, (for example,
C<pack "U3C8", @stuff>) then the resulting string should be treated as
UTF-8 encoded.
+If you are working with a git clone of the Perl repository, you will want to
+create a branch for your changes. This will make creating a proper patch much
+simpler. See the L<perlrepository> for details on how to do this.
+
How do we prepare to fix this up? First we locate the code in question -
the C<pack> happens at runtime, so it's going to be in one of the F<pp>
files. Sure enough, C<pp_pack> is in F<pp.c>. Since we're going to be
The regression tests for each operator live in F<t/op/>, and so we
make a copy of F<t/op/pack.t> to F<t/op/pack.t~>. Now we can add our
tests to the end. First, we'll test that the C<U> does indeed create
-Unicode strings.
+Unicode strings.
t/op/pack.t has a sensible ok() function, but if it didn't we could
use the one from t/test.pl.
so instead of this:
- print 'not ' unless "1.20.300.4000" eq sprintf "%vd", pack("U*",1,20,300,4000);
+ print 'not ' unless "1.20.300.4000" eq sprintf "%vd",
+ pack("U*",1,20,300,4000);
print "ok $test\n"; $test++;
we can write the more sensible (see L<Test::More> for a full
explanation of is() and other testing functions).
- is( "1.20.300.4000", sprintf "%vd", pack("U*",1,20,300,4000),
- "U* produces unicode" );
+ is( "1.20.300.4000", sprintf "%vd", pack("U*",1,20,300,4000),
+ "U* produces Unicode" );
Now we'll test that we got that space-at-the-beginning business right:
is( "1.20.300.4000", sprintf "%vd", pack(" U*",1,20,300,4000),
- " with spaces at the beginning" );
+ " with spaces at the beginning" );
And finally we'll test that we don't make Unicode strings if C<U> is B<not>
the first active format:
isnt( v1.20.300.4000, sprintf "%vd", pack("C0U*",1,20,300,4000),
- "U* not first isn't unicode" );
+ "U* not first isn't Unicode" );
Mustn't forget to change the number of tests which appears at the top,
or else the automated tester will get confused. This will either look
If the pattern begins with a C<U>, the resulting string will be treated
as UTF-8-encoded Unicode. You can force UTF-8 encoding on in a string
with an initial C<U0>, and the bytes that follow will be interpreted as
- Unicode characters. If you don't want this to happen, you can begin your
- pattern with C<C0> (or anything else) to force Perl not to UTF-8 encode your
- string, and then follow this with a C<U*> somewhere in your pattern.
-
-All done. Now let's create the patch. F<Porting/patching.pod> tells us
-that if we're making major changes, we should copy the entire directory
-to somewhere safe before we begin fiddling, and then do
-
- diff -ruN old new > patch
-
-However, we know which files we've changed, and we can simply do this:
-
- diff -u pp.c~ pp.c > patch
- diff -u t/op/pack.t~ t/op/pack.t >> patch
- diff -u pod/perlfunc.pod~ pod/perlfunc.pod >> patch
-
-We end up with a patch looking a little like this:
-
- --- pp.c~ Fri Jun 02 04:34:10 2000
- +++ pp.c Fri Jun 16 11:37:25 2000
- @@ -4375,6 +4375,7 @@
- register I32 items;
- STRLEN fromlen;
- register char *pat = SvPVx(*++MARK, fromlen);
- + char *patcopy;
- register char *patend = pat + fromlen;
- register I32 len;
- I32 datumtype;
- @@ -4405,6 +4406,7 @@
- ...
-
-And finally, we submit it, with our rationale, to perl5-porters. Job
-done!
+ Unicode characters. If you don't want this to happen, you can begin
+ your pattern with C<C0> (or anything else) to force Perl not to UTF-8
+ encode your string, and then follow this with a C<U*> somewhere in your
+ pattern.
=head2 Patching a core module
conventions used in the perl source files. See L<perlstyle> for
details. Although most of the guidelines discussed seem to focus on
Perl code, rather than c, they all apply (except when they don't ;).
-See also I<Porting/patching.pod> file in the Perl source distribution
-for lots of details about both formatting and submitting patches of
-your changes.
+Also see L<perlrepository> for lots of details about both formatting and
+submitting patches of your changes.
Lastly, TEST TEST TEST TEST TEST any code before posting to p5p.
Test on as many platforms as you can find. Test as many perl
them out), but it basically works that way. Everything else lives in
F<t/>.
+Testing of warning messages is often separately done by using expect scripts in
+F<t/lib/warnings>. This is because much of the setup for them is already done
+for you.
+
+If you add a new test directory under F<t/>, it is imperative that you
+add that directory to F<t/HARNESS> and F<t/TEST>.
+
=over 3
=item F<t/base/>
here. There are still some bits and pieces hanging around in here
that need to be moved. Perhaps you could move them? Thanks!
+=item F<t/mro/>
+
+Tests for perl's method resolution order implementations
+(see L<mro>).
+
=item F<t/op/>
Tests for perl's built in functions that don't fit into any of the
other directories.
-=item F<t/pod/>
+=item F<t/re/>
-Tests for POD directives. There are still some tests for the Pod
-modules hanging around in here that need to be moved out into F<lib/>.
+Tests for regex related functions or behaviour. (These used to live
+in t/op).
=item F<t/run/>
decision of which to use depends on what part of the test suite you're
working on. This is a measure to prevent a high-level failure (such
as Config.pm breaking) from causing basic functionality tests to fail.
+If you write your own test, use the L<Test Anything Protocol|TAP>.
-=over 4
+=over 4
=item t/base t/comp
=back
When you say "make test" Perl uses the F<t/TEST> program to run the
-test suite (except under Win32 where it uses F<t/harness> instead.)
-All tests are run from the F<t/> directory, B<not> the directory
-which contains the test. This causes some problems with the tests
+test suite (except under Win32 where it uses F<t/harness> instead.)
+All tests are run from the F<t/> directory, B<not> the directory
+which contains the test. This causes some problems with the tests
in F<lib/>, so here's some opportunity for some patching.
You must be triply conscious of cross-platform concerns. This usually
=item minitest
Run F<miniperl> on F<t/base>, F<t/comp>, F<t/cmd>, F<t/run>, F<t/io>,
-F<t/op>, and F<t/uni> tests.
+F<t/op>, F<t/uni> and F<t/mro> tests.
=item test.valgrind check.valgrind utest.valgrind ucheck.valgrind
nmake test TEST_FILES="op/*.t"
nmake test TEST_SWITCHES="-torture" TEST_FILES="op/*.t"
+=item Parallel tests
+
+The core distribution can now run its regression tests in parallel on
+Unix-like platforms. Instead of running C<make test>, set C<TEST_JOBS> in
+your environment to the number of tests to run in parallel, and run
+C<make test_harness>. On a Bourne-like shell, this can be done as
+
+ TEST_JOBS=3 make test_harness # Run 3 tests in parallel
+
+An environment variable is used, rather than parallel make itself, because
+L<TAP::Harness> needs to be able to schedule individual non-conflicting test
+scripts itself, and there is no standard interface to C<make> utilities to
+interact with their job schedulers.
+
+Note that currently some test scripts may fail when run in parallel (most
+notably C<ext/IO/t/io_dir.t>). If necessary run just the failing scripts
+again sequentially and see if the failures go away.
=item test-notty test_notty
Sets PERL_SKIP_TTY_TEST to true before running normal test.
harness -v -torture -re LIST OF PATTERNS TO MATCH
If C<LIST OF FILES TO TEST> is omitted the file list is obtained from
-the manifest. The file list may include shell wildcards which will be
+the manifest. The file list may include shell wildcards which will be
expanded out.
=over 4
=item -v
-Run the tests under verbose mode so you can see what tests were run,
-and debug outbut.
+Run the tests under verbose mode so you can see what tests were run,
+and debug output.
=item -torture
=item -re LIST OF PATTERNS
-Filter the file list so that all the test files run match
+Filter the file list so that all the test files run match
/(LIST|OF|PATTERNS)/. Note that with this form the patterns
are joined by '|' and you cannot supply a list of files, instead
the test files are obtained from the MANIFEST.
except that the harnesses set up some environment variables that may
affect the execution of the test :
-=over 4
+=over 4
=item PERL_CORE=1
=back
+=head3 Other environment variables that may influence tests
+
+=over 4
+
+=item PERL_TEST_Net_Ping
+
+Setting this variable runs all the Net::Ping modules tests,
+otherwise some tests that interact with the outside world are skipped.
+See L<perl58delta>.
+
+=item PERL_TEST_NOVREXX
+
+Setting this variable skips the vrexx.t tests for OS2::REXX.
+
+=item PERL_TEST_NUMCONVERTS
+
+This sets a variable in op/numconvert.t.
+
+=back
+
+See also the documentation for the Test and Test::Harness modules,
+for more environment variables that affect testing.
+
=head2 Common problems when patching Perl source code
Perl source plays by ANSI C89 rules: no C99 (or C++) extensions. In
Compiling with threading (-Duseithreads) completely rewrites
the function prototypes of Perl. You better try your changes
-with that. Related to this is the difference between "Perl_"-less
+with that. Related to this is the difference between "Perl_-less"
and "Perl_-ly" APIs, for example:
Perl_sv_setiv(aTHX_ ...);
sv_setiv(...);
-The first one explicitly passes in the context, which is needed for
-e.g. threaded builds. The second one does that implicitly; do not get
-them mixed.
+The first one explicitly passes in the context, which is needed for e.g.
+threaded builds. The second one does that implicitly; do not get them
+mixed. If you are not passing in a aTHX_, you will need to do a dTHX
+(or a dVAR) as the first thing in the function.
See L<perlguts/"How multiple interpreters and concurrency are supported">
for further discussion about context.
Not compiling with -DDEBUGGING
The DEBUGGING define exposes more code to the compiler,
-therefore more ways for things to go wrong.
+therefore more ways for things to go wrong. You should try it.
+
+=item *
+
+Introducing (non-read-only) globals
+
+Do not introduce any modifiable globals, truly global or file static.
+They are bad form and complicate multithreading and other forms of
+concurrency. The right way is to introduce them as new interpreter
+variables, see F<intrpvar.h> (at the very end for binary compatibility).
+
+Introducing read-only (const) globals is okay, as long as you verify
+with e.g. C<nm libperl.a|egrep -v ' [TURtr] '> (if your C<nm> has
+BSD-style output) that the data you added really is read-only.
+(If it is, it shouldn't show up in the output of that command.)
+
+If you want to have static strings, make them constant:
+
+ static const char etc[] = "...";
+
+If you want to have arrays of constant strings, note carefully
+the right combination of C<const>s:
+
+ static const char * const yippee[] =
+ {"hi", "ho", "silver"};
+
+There is a way to completely hide any modifiable globals (they are all
+moved to heap), the compilation setting C<-DPERL_GLOBAL_STRUCT_PRIVATE>.
+It is not normally used, but can be used for testing, read more
+about it in L<perlguts/"Background and PERL_IMPLICIT_CONTEXT">.
=item *
The new shiny result of either genuine new functionality or your
arduous refactoring is now ready and correctly exported. So what
-could possibly be wrong?
+could possibly go wrong?
Maybe simply that your function did not need to be exported in the
first place. Perl has a long and not so glorious history of exporting
=back
-=head Portability problems
+=head2 Portability problems
The following are common causes of compilation and/or execution
failures, not common to Perl as such. The C FAQ is good bedtime
-reading. Please test your changes with as many C compilers as
-possible -- we will, anyway, and it's nice to save oneself from
-public embarrassment.
+reading. Please test your changes with as many C compilers and
+platforms as possible; we will, anyway, and it's nice to save
+oneself from public embarrassment.
+
+If using gcc, you can add the C<-std=c89> option which will hopefully
+catch most of these unportabilities. (However it might also catch
+incompatibilities in your system's header files.)
+
+Use the Configure C<-Dgccansipedantic> flag to enable the gcc
+C<-ansi -pedantic> flags which enforce stricter ANSI rules.
+
+If using the C<gcc -Wall> note that not all the possible warnings
+(like C<-Wunitialized>) are given unless you also compile with C<-O>.
+
+Note that if using gcc, starting from Perl 5.9.5 the Perl core source
+code files (the ones at the top level of the source code distribution,
+but not e.g. the extensions under ext/) are automatically compiled
+with as many as possible of the C<-std=c89>, C<-ansi>, C<-pedantic>,
+and a selection of C<-W> flags (see cflags.SH).
+
+Also study L<perlport> carefully to avoid any bad assumptions
+about the operating system, filesystems, and so forth.
+
+You may once in a while try a "make microperl" to see whether we
+can still compile Perl with just the bare minimum of interfaces.
+(See README.micro.)
+
+Do not assume an operating system indicates a certain compiler.
=over 4
{
IV i = (IV)p;
-Either are bad, and broken, and unportable. Use the PTR2IV()
+Both are bad, and broken, and unportable. Use the PTR2IV()
macro that does it right. (Likewise, there are PTR2UV(), PTR2NV(),
INT2PTR(), and NUM2PTR().)
=item *
+Casting between data function pointers and data pointers
+
Technically speaking casting between function pointers and data
pointers is unportable and undefined, but practically speaking
it seems to work, but you should use the FPTR2DPTR() and DPTR2FPTR()
-macros.
+macros. Sometimes you can also play games with unions.
=item *
shorts are 64 bits. This is all legal according to the C standard.
(In other words, "long long" is not a portable way to specify 64 bits,
and "long long" is not even guaranteed to be any wider than "long".)
-Use definitions like IVSIZE, I32SIZE, and so forth.
+
+Instead, use the definitions IV, UV, IVSIZE, I32SIZE, and so forth.
+Avoid things like I32 because they are B<not> guaranteed to be
+I<exactly> 32 bits, they are I<at least> 32 bits, nor are they
+guaranteed to be B<int> or B<long>. If you really explicitly need
+64-bit variables, use I64 and U64, but only if guarded by HAS_QUAD.
=item *
Assuming one can dereference any type of pointer for any type of data
char *p = ...;
- long pony = *p;
+ long pony = *p; /* BAD */
Many platforms, quite rightly so, will give you a core dump instead
of a pony if the p happens not be correctly aligned.
Lvalue casts
- (int)*p = ...;
+ (int)*p = ...; /* BAD */
Simply not portable. Get your lvalue to be of the right type,
-or maybe use temporary variables.
+or maybe use temporary variables, or dirty tricks with unions.
=item *
-Using //-comments
+Assume B<anything> about structs (especially the ones you
+don't control, like the ones coming from the system headers)
- // This function bamfoodles the zorklator.
+=over 8
-That is C99 or C++. Perl is C89. Using the //-comments is silently
-allowed by many C compilers but cranking up the ANSI strictness (which
-we like to do) causes the compilation to fail.
+=item *
+
+That a certain field exists in a struct
=item *
-Mixing declarations and code
+That no other fields exist besides the ones you know of
- void zorklator()
- {
- int n = 3;
- set_zorkmids(n);
- int q = 4;
+=item *
-That is C99 or C++. Some compilers allow that, but you shouldn't.
+That a field is of certain signedness, sizeof, or type
=item *
-Mixing signed char pointers with unsigned char pointers
+That the fields are in a certain order
- int foo(char *s) { ... }
- ...
- unsigned char *t = ...; /* Or U8* t = ... */
- foo(t);
+=over 8
-While this is legal practice, it is certainly dubious, and downright
-fatal in at least one platform: for example VMS cc considers this a
-fatal error. One cause for people often making this mistake is that a
-"naked char" and therefore deferencing a "naked char pointer" have an
-undefined sign: it depends on the compilers and the platform whether
-the result is signed or unsigned.
+=item *
+
+While C guarantees the ordering specified in the struct definition,
+between different platforms the definitions might differ
+
+=back
=item *
-Macros that have string constants and their arguments as substrings of
-the string constants
+That the sizeof(struct) or the alignments are the same everywhere
- #define FOO(n) printf("number = %d\n", n)
- FOO(10);
+=over 8
-Pre-ANSI semantics for that was equivalent to
+=item *
- printf("10umber = %d\10");
+There might be padding bytes between the fields to align the fields -
+the bytes can be anything
+
+=item *
-which is probably not what you were expecting. Unfortunately at
-least one C compiler does real backward compatibility here, in AIX
-that is what still happens even though the rest of the AIX compiler
-is very happily C89.
+Structs are required to be aligned to the maximum alignment required
+by the fields - which for native types is for usually equivalent to
+sizeof() of the field
=back
-=head2 Security problems
+=back
-Last but not least, here are various tips for safer coding.
+=item *
-=over 4
+Assuming the character set is ASCIIish
+
+Perl can compile and run under EBCDIC platforms. See L<perlebcdic>.
+This is transparent for the most part, but because the character sets
+differ, you shouldn't use numeric (decimal, octal, nor hex) constants
+to refer to characters. You can safely say 'A', but not 0x41.
+You can safely say '\n', but not \012.
+If a character doesn't have a trivial input form, you can
+create a #define for it in both C<utfebcdic.h> and C<utf8.h>, so that
+it resolves to different values depending on the character set being used.
+(There are three different EBCDIC character sets defined in C<utfebcdic.h>,
+so it might be best to insert the #define three times in that file.)
+
+Also, the range 'A' - 'Z' in ASCII is an unbroken sequence of 26 upper case
+alphabetic characters. That is not true in EBCDIC. Nor for 'a' to 'z'.
+But '0' - '9' is an unbroken range in both systems. Don't assume anything
+about other ranges.
+
+Many of the comments in the existing code ignore the possibility of EBCDIC,
+and may be wrong therefore, even if the code works.
+This is actually a tribute to the successful transparent insertion of being
+able to handle EBCDIC without having to change pre-existing code.
+
+UTF-8 and UTF-EBCDIC are two different encodings used to represent Unicode
+code points as sequences of bytes. Macros
+with the same names (but different definitions)
+in C<utf8.h> and C<utfebcdic.h>
+are used to allow the calling code to think that there is only one such
+encoding.
+This is almost always referred to as C<utf8>, but it means the EBCDIC version
+as well. Again, comments in the code may well be wrong even if the code itself
+is right.
+For example, the concept of C<invariant characters> differs between ASCII and
+EBCDIC.
+On ASCII platforms, only characters that do not have the high-order
+bit set (i.e. whose ordinals are strict ASCII, 0 - 127)
+are invariant, and the documentation and comments in the code
+may assume that,
+often referring to something like, say, C<hibit>.
+The situation differs and is not so simple on EBCDIC machines, but as long as
+the code itself uses the C<NATIVE_IS_INVARIANT()> macro appropriately, it
+works, even if the comments are wrong.
=item *
-Do not use gets()
+Assuming the character set is just ASCII
-Or we will publicly ridicule you. Seriously.
+ASCII is a 7 bit encoding, but bytes have 8 bits in them. The 128 extra
+characters have different meanings depending on the locale. Absent a locale,
+currently these extra characters are generally considered to be unassigned,
+and this has presented some problems.
+This is being changed starting in 5.12 so that these characters will
+be considered to be Latin-1 (ISO-8859-1).
=item *
-Do not use strcpy() or strcat()
+Mixing #define and #ifdef
+
+ #define BURGLE(x) ... \
+ #ifdef BURGLE_OLD_STYLE /* BAD */
+ ... do it the old way ... \
+ #else
+ ... do it the new way ... \
+ #endif
-While some uses of these still linger in the Perl source code,
-we have inspected them for safety and are very, very ashamed of them,
-and plan to get rid of them. In places where there are strlcpy()
-and strlcat() we prefer to use them, and there is a plan to integrate
-the strlcpy/strlcat implementation of INN.
+You cannot portably "stack" cpp directives. For example in the above
+you need two separate BURGLE() #defines, one for each #ifdef branch.
=item *
-Do not use sprintf() or vsprintf()
+Adding non-comment stuff after #endif or #else
-Use my_snprintf() and my_vnsprintf() instead, which will try to use
-snprintf() and vsnprintf() if those safer APIs are available.
+ #ifdef SNOSH
+ ...
+ #else !SNOSH /* BAD */
+ ...
+ #endif SNOSH /* BAD */
-=back
+The #endif and #else cannot portably have anything non-comment after
+them. If you want to document what is going (which is a good idea
+especially if the branches are long), use (C) comments:
-=head2 Common problems when patching Perl source code
+ #ifdef SNOSH
+ ...
+ #else /* !SNOSH */
+ ...
+ #endif /* SNOSH */
-Perl source plays by ANSI C89 rules: no C99 (or C++) extensions. In
-some cases we have to take pre-ANSI requirements into consideration.
-You don't care about some particular platform having broken Perl?
-I hear there is still a strong demand for J2EE programmers.
+The gcc option C<-Wendif-labels> warns about the bad variant
+(by default on starting from Perl 5.9.4).
-=head2 Perl environment problems
+=item *
-=over 4
+Having a comma after the last element of an enum list
-=item *
+ enum color {
+ CERULEAN,
+ CHARTREUSE,
+ CINNABAR, /* BAD */
+ };
-Not compiling with threading
+is not portable. Leave out the last comma.
-Compiling with threading (-Duseithreads) completely rewrites
-the function prototypes of Perl. You better try your changes
-with that. Related to this is the difference between "Perl_"-less
-and "Perl_-ly" APIs, for example:
+Also note that whether enums are implicitly morphable to ints
+varies between compilers, you might need to (int).
- Perl_sv_setiv(aTHX_ ...);
- sv_setiv(...);
+=item *
-The first one explicitly passes in the context, which is needed for
-e.g. threaded builds. The second one does that implicitly; do not get
-them mixed.
+Using //-comments
-See L<perlguts/"How multiple interpreters and concurrency are supported">
-for further discussion about context.
+ // This function bamfoodles the zorklator. /* BAD */
+
+That is C99 or C++. Perl is C89. Using the //-comments is silently
+allowed by many C compilers but cranking up the ANSI C89 strictness
+(which we like to do) causes the compilation to fail.
=item *
-Not compiling with -DDEBUGGING
+Mixing declarations and code
-The DEBUGGING define exposes more code to the compiler,
-therefore more ways for things to go wrong.
+ void zorklator()
+ {
+ int n = 3;
+ set_zorkmids(n); /* BAD */
+ int q = 4;
+
+That is C99 or C++. Some C compilers allow that, but you shouldn't.
+
+The gcc option C<-Wdeclaration-after-statements> scans for such problems
+(by default on starting from Perl 5.9.4).
=item *
-Not exporting your new function
+Introducing variables inside for()
-Some platforms (Win32, AIX, VMS, OS/2, to name a few) require any
-function that is part of the public API (the shared Perl library)
-to be explicitly marked as exported. See the discussion about
-F<embed.pl> in L<perlguts>.
+ for(int i = ...; ...; ...) { /* BAD */
+
+That is C99 or C++. While it would indeed be awfully nice to have that
+also in C89, to limit the scope of the loop variable, alas, we cannot.
=item *
-Exporting your new function
+Mixing signed char pointers with unsigned char pointers
-The new shiny result of either genuine new functionality or your
-arduous refactoring is now ready and correctly exported. So what
-could possibly be wrong?
+ int foo(char *s) { ... }
+ ...
+ unsigned char *t = ...; /* Or U8* t = ... */
+ foo(t); /* BAD */
-Maybe simply that your function did not need to be exported in the
-first place. Perl has a long and not so glorious history of exporting
-functions that it should not have.
+While this is legal practice, it is certainly dubious, and downright
+fatal in at least one platform: for example VMS cc considers this a
+fatal error. One cause for people often making this mistake is that a
+"naked char" and therefore dereferencing a "naked char pointer" have
+an undefined signedness: it depends on the compiler and the flags of
+the compiler and the underlying platform whether the result is signed
+or unsigned. For this very same reason using a 'char' as an array
+index is bad.
-If the function is used only inside one source code file, make it
-static. See the discussion about F<embed.pl> in L<perlguts>.
+=item *
-If the function is used across several files, but intended only for
-Perl's internal use (and this should be the common case), do not
-export it to the public API. See the discussion about F<embed.pl>
-in L<perlguts>.
+Macros that have string constants and their arguments as substrings of
+the string constants
-=back
+ #define FOO(n) printf("number = %d\n", n) /* BAD */
+ FOO(10);
-=head Portability problems
+Pre-ANSI semantics for that was equivalent to
-The following are common causes of compilation and/or execution
-failures, not common to Perl as such. The C FAQ is good bedtime
-reading. Please test your changes with as many C compilers as
-possible -- we will, anyway, and it's nice to save oneself from
-public embarrassment.
+ printf("10umber = %d\10");
-=over 4
+which is probably not what you were expecting. Unfortunately at least
+one reasonably common and modern C compiler does "real backward
+compatibility" here, in AIX that is what still happens even though the
+rest of the AIX compiler is very happily C89.
=item *
-Casting pointers to integers or casting integers to pointers
+Using printf formats for non-basic C types
- void castaway(U8* p)
- {
- IV i = p;
+ IV i = ...;
+ printf("i = %d\n", i); /* BAD */
-or
+While this might by accident work in some platform (where IV happens
+to be an C<int>), in general it cannot. IV might be something larger.
+Even worse the situation is with more specific types (defined by Perl's
+configuration step in F<config.h>):
- void castaway(U8* p)
- {
- IV i = (IV)p;
+ Uid_t who = ...;
+ printf("who = %d\n", who); /* BAD */
-Either are bad, and broken, and unportable. Use the PTR2IV()
-macro that does it right. (Likewise, there are PTR2UV(), PTR2NV(),
-INT2PTR(), and NUM2PTR().)
+The problem here is that Uid_t might be not only not C<int>-wide
+but it might also be unsigned, in which case large uids would be
+printed as negative values.
-=item *
+There is no simple solution to this because of printf()'s limited
+intelligence, but for many types the right format is available as
+with either 'f' or '_f' suffix, for example:
-Technically speaking casting between function pointers and data
-pointers is unportable and undefined, but practically speaking
-it seems to work, but you should use the FPTR2DPTR() and DPTR2FPTR()
-macros.
+ IVdf /* IV in decimal */
+ UVxf /* UV is hexadecimal */
-=item *
+ printf("i = %"IVdf"\n", i); /* The IVdf is a string constant. */
-Assuming sizeof(int) == sizeof(long)
+ Uid_t_f /* Uid_t in decimal */
-There are platforms where longs are 64 bits, and platforms where ints
-are 64 bits, and while we are out to shock you, even platforms where
-shorts are 64 bits. This is all legal according to the C standard.
-(In other words, "long long" is not a portable way to specify 64 bits,
-and "long long" is not even guaranteed to be any wider than "long".)
-Use definitions like IVSIZE, I32SIZE, and so forth.
+ printf("who = %"Uid_t_f"\n", who);
-=item *
+Or you can try casting to a "wide enough" type:
-Assuming one can dereference any type of pointer for any type of data
+ printf("i = %"IVdf"\n", (IV)something_very_small_and_signed);
- char *p = ...;
- long pony = *p;
+Also remember that the C<%p> format really does require a void pointer:
-Many platforms, quite rightly so, will give you a core dump instead
-of a pony if the p happens not be correctly aligned.
+ U8* p = ...;
+ printf("p = %p\n", (void*)p);
+
+The gcc option C<-Wformat> scans for such problems.
=item *
-Lvalue casts
+Blindly using variadic macros
- (int)*p = ...;
+gcc has had them for a while with its own syntax, and C99 brought
+them with a standardized syntax. Don't use the former, and use
+the latter only if the HAS_C99_VARIADIC_MACROS is defined.
-Simply not portable. Get your lvalue to be of the right type,
-or maybe use temporary variables.
+=item *
+
+Blindly passing va_list
+
+Not all platforms support passing va_list to further varargs (stdarg)
+functions. The right thing to do is to copy the va_list using the
+Perl_va_copy() if the NEED_VA_COPY is defined.
=item *
-Using //-comments
+Using gcc statement expressions
- // This function bamfoodles the zorklator.
+ val = ({...;...;...}); /* BAD */
-That is C99 or C++. Perl is C89. Using the //-comments is silently
-allowed by many C compilers but cranking up the ANSI strictness (which
-we like to do) causes the compilation to fail.
+While a nice extension, it's not portable. The Perl code does
+admittedly use them if available to gain some extra speed
+(essentially as a funky form of inlining), but you shouldn't.
=item *
-Mixing declarations and code
+Binding together several statements in a macro
- void zorklator()
- {
- int n = 3;
- set_zorkmids(n);
- int q = 4;
+Use the macros STMT_START and STMT_END.
-That is C99 or C++. Some compilers allow that, but you shouldn't.
+ STMT_START {
+ ...
+ } STMT_END
=item *
-Mixing signed char pointers with unsigned char pointers
+Testing for operating systems or versions when should be testing for features
+
+ #ifdef __FOONIX__ /* BAD */
+ foo = quux();
+ #endif
+
+Unless you know with 100% certainty that quux() is only ever available
+for the "Foonix" operating system B<and> that is available B<and>
+correctly working for B<all> past, present, B<and> future versions of
+"Foonix", the above is very wrong. This is more correct (though still
+not perfect, because the below is a compile-time check):
+
+ #ifdef HAS_QUUX
+ foo = quux();
+ #endif
+
+How does the HAS_QUUX become defined where it needs to be? Well, if
+Foonix happens to be Unixy enough to be able to run the Configure
+script, and Configure has been taught about detecting and testing
+quux(), the HAS_QUUX will be correctly defined. In other platforms,
+the corresponding configuration step will hopefully do the same.
+
+In a pinch, if you cannot wait for Configure to be educated,
+or if you have a good hunch of where quux() might be available,
+you can temporarily try the following:
+
+ #if (defined(__FOONIX__) || defined(__BARNIX__))
+ # define HAS_QUUX
+ #endif
- int foo(char *s) { ... }
...
- unsigned char *t = ...; /* Or U8* t = ... */
- foo(t);
-While this is legal practice, it is certainly dubious, and downright
-fatal in at least one platform: for example VMS cc considers this a
-fatal error. One cause for people often making this mistake is that a
-"naked char" and therefore deferencing a "naked char pointer" have an
-undefined sign: it depends on the compilers and the platform whether
-the result is signed or unsigned.
+ #ifdef HAS_QUUX
+ foo = quux();
+ #endif
-=item *
+But in any case, try to keep the features and operating systems separate.
-Macros that have string constants and their arguments as substrings of
-the string constants
+=back
- #define FOO(n) printf("number = %d\n", n)
- FOO(10);
+=head2 Problematic System Interfaces
-Pre-ANSI semantics for that was equivalent to
+=over 4
- printf("10umber = %d\10");
+=item *
+
+malloc(0), realloc(0), calloc(0, 0) are non-portable. To be portable
+allocate at least one byte. (In general you should rarely need to
+work at this low level, but instead use the various malloc wrappers.)
+
+=item *
-which is probably not what you were expecting. Unfortunately at
-least one C compiler does real backward compatibility here, in AIX
-that is what still happens even though the rest of the AIX compiler
-is very happily C89.
+snprintf() - the return type is unportable. Use my_snprintf() instead.
=back
=item *
-Do not use strcpy() or strcat()
+Do not use strcpy() or strcat() or strncpy() or strncat()
-Where there still linger some uses of these in the Perl source code,
-we have inspected them for safety and are very, very ashamed of them,
-and plan to get rid of them. In places where there are strlcpy()
-and strlcat() we prefer to use them, and there is a plan to integrate
-the strlcpy/strlcat implementation of INN.
+Use my_strlcpy() and my_strlcat() instead: they either use the native
+implementation, or Perl's own implementation (borrowed from the public
+domain implementation of INN).
=item *
Do not use sprintf() or vsprintf()
-Use my_snprintf() and my_vnsprintf() instead, which will try to use
-snprintf() and vsnprintf() if those safer APIs are available.
+If you really want just plain byte strings, use my_snprintf()
+and my_vsnprintf() instead, which will try to use snprintf() and
+vsnprintf() if those safer APIs are available. If you want something
+fancier than a plain byte string, use SVs and Perl_sv_catpvf().
=back
minutes become hours. For example as of Perl 5.8.1, the
ext/Encode/t/Unicode.t takes extraordinarily long to complete under
e.g. Purify, Third Degree, and valgrind. Under valgrind it takes more
-than six hours, even on a snappy computer-- the said test must be
+than six hours, even on a snappy computer. The said test must be
doing something that is quite unfriendly for memory debuggers. If you
don't feel like waiting, that you can simply kill away the perl
process.
B<NOTE 2>: To minimize the number of memory leak false alarms (see
-L</PERL_DESTRUCT_LEVEL> for more information), you have to have
-environment variable PERL_DESTRUCT_LEVEL set to 2. The F<TEST>
-and harness scripts do that automatically. But if you are running
-some of the tests manually-- for csh-like shells:
+L</PERL_DESTRUCT_LEVEL> for more information), you have to set the
+environment variable PERL_DESTRUCT_LEVEL to 2.
+
+For csh-like shells:
setenv PERL_DESTRUCT_LEVEL 2
-and for Bourne-type shells:
+For Bourne-type shells:
PERL_DESTRUCT_LEVEL=2
export PERL_DESTRUCT_LEVEL
-or in UNIXy environments you can also use the C<env> command:
+In Unixy environments you can also use the C<env> command:
env PERL_DESTRUCT_LEVEL=2 valgrind ./perl -Ilib ...
is a good sign of these. Fixing these leaks is non-trivial,
unfortunately, but they must be fixed eventually.
+B<NOTE 4>: L<DynaLoader> will not clean up after itself completely
+unless Perl is built with the Configure option
+C<-Accflags=-DDL_UNLOAD_ALL_AT_EXIT>.
+
=head2 Rational Software's Purify
Purify is a commercial tool that is helpful in identifying
Once you've compiled a perl suitable for Purify'ing, then you
can just:
- make pureperl
+ make pureperl
which creates a binary named 'pureperl' that has been Purify'ed.
This binary is used in place of the standard 'perl' binary
make pureperl
cd t
- ../pureperl -I../lib harness
+ ../pureperl -I../lib harness
which would run Perl on test.pl and report any memory problems.
You should add -DPURIFY to the DEFINES line so the DEFINES
line looks something like:
- DEFINES = -DWIN32 -D_CONSOLE -DNO_STRICT $(CRYPT_FLAG) -DPURIFY=1
+ DEFINES = -DWIN32 -D_CONSOLE -DNO_STRICT $(CRYPT_FLAG) -DPURIFY=1
to disable Perl's arena memory allocation functions, as
well as to force use of memory allocation functions derived
cd win32
make
cd ../t
- purify ../perl -I../lib harness
+ purify ../perl -I../lib harness
which would instrument Perl in memory, run Perl on test.pl,
then finally report any memory problems.
=head2 valgrind
The excellent valgrind tool can be used to find out both memory leaks
-and illegal memory accesses. As of August 2003 it unfortunately works
-only on x86 (ELF) Linux. The special "test.valgrind" target can be used
-to run the tests under valgrind. Found errors and memory leaks are
-logged in files named F<test.valgrind>.
+and illegal memory accesses. As of version 3.3.0, Valgrind only
+supports Linux on x86, x86-64 and PowerPC. The special "test.valgrind"
+target can be used to run the tests under valgrind. Found errors
+and memory leaks are logged in files named F<testfile.valgrind>.
+
+Valgrind also provides a cachegrind tool, invoked on perl as:
+
+ VG_OPTS=--tool=cachegrind make test.valgrind
As system libraries (most notably glibc) are also triggering errors,
valgrind allows to suppress such errors using suppression files. The
converts C<new_SV()> from a macro into a real function, so you can use
your favourite debugger to discover where those pesky SVs were allocated.
+If you see that you're leaking memory at runtime, but neither valgrind
+nor C<-DDEBUG_LEAKING_SCALARS> will find anything, you're probably
+leaking SVs that are still reachable and will be properly cleaned up
+during destruction of the interpreter. In such cases, using the C<-Dm>
+switch can point you to the source of the leak. If the executable was
+built with C<-DDEBUG_LEAKING_SCALARS>, C<-Dm> will output SV allocations
+in addition to memory allocations. Each SV allocation has a distinct
+serial number that will be written on creation and destruction of the SV.
+So if you're executing the leaking code in a loop, you need to look for
+SVs that are created, but never destroyed between each cycle. If such an
+SV is found, set a conditional breakpoint within C<new_SV()> and make it
+break only when C<PL_sv_serial> is equal to the serial number of the
+leaking SV. Then you will catch the interpreter in exactly the state
+where the leaking SV is allocated, which is sufficient in many cases to
+find the source of the leak.
+
+As C<-Dm> is using the PerlIO layer for output, it will by itself
+allocate quite a bunch of SVs, which are hidden to avoid recursion.
+You can bypass the PerlIO layer if you use the SV logging provided
+by C<-DPERL_MEM_LOG> instead.
+
=head2 PERL_MEM_LOG
-If compiled with C<-DPERL_MEM_LOG>, all Newx() and Renew() allocations
-and Safefree() in the Perl core go through logging functions, which is
-handy for breakpoint setting. If also compiled with C<-DPERL_MEM_LOG_STDERR>,
-the allocations and frees are logged to STDERR (or more precisely, to the
-file descriptor 2) in these logging functions, with the calling source code
-file and line number (and C function name, if supported by the C compiler).
+If compiled with C<-DPERL_MEM_LOG>, both memory and SV allocations go
+through logging functions, which is handy for breakpoint setting.
+
+Unless C<-DPERL_MEM_LOG_NOIMPL> is also compiled, the logging
+functions read $ENV{PERL_MEM_LOG} to determine whether to log the
+event, and if so how:
-This logging is somewhat similar to C<-Dm> but independent of C<-DDEBUGGING>,
-and at a higher level (the C<-Dm> is directly at the point of C<malloc()>,
-while the C<PERL_MEM_LOG> is at the level of C<New()>).
+ $ENV{PERL_MEM_LOG} =~ /m/ Log all memory ops
+ $ENV{PERL_MEM_LOG} =~ /s/ Log all SV ops
+ $ENV{PERL_MEM_LOG} =~ /t/ include timestamp in Log
+ $ENV{PERL_MEM_LOG} =~ /^(\d+)/ write to FD given (default is 2)
+
+Memory logging is somewhat similar to C<-Dm> but is independent of
+C<-DDEBUGGING>, and at a higher level; all uses of Newx(), Renew(),
+and Safefree() are logged with the caller's source code file and line
+number (and C function name, if supported by the C compiler). In
+contrast, C<-Dm> is directly at the point of C<malloc()>. SV logging
+is similar.
+
+Since the logging doesn't use PerlIO, all SV allocations are logged
+and no extra SV allocations are introduced by enabling the logging.
+If compiled with C<-DDEBUG_LEAKING_SCALARS>, the serial number for
+each SV allocation is also logged.
=head2 Profiling
=head2 Gprof Profiling
-gprof is a profiling tool available in many UNIX platforms,
+gprof is a profiling tool available in many Unix platforms,
it uses F<statistical time-sampling>.
You can build a profiled version of perl called "perl.gprof" by
For more detailed explanation of the available commands and output
formats, see your own local documentation of gprof.
+quick hint:
+
+ $ sh Configure -des -Dusedevel -Doptimize='-pg' && make perl.gprof
+ $ ./perl.gprof someprog # creates gmon.out in current directory
+ $ gprof ./perl.gprof > out
+ $ view out
+
=head2 GCC gcov Profiling
Starting from GCC 3.0 I<basic block profiling> is officially available
http://gcc.gnu.org/onlinedocs/gcc-3.0/gcc_8.html#SEC132
+quick hint:
+
+ $ sh Configure -des -Dusedevel -Doptimize='-g' \
+ -Accflags='-fprofile-arcs -ftest-coverage' \
+ -Aldflags='-fprofile-arcs -ftest-coverage' && make perl.gcov
+ $ rm -f regexec.c.gcov regexec.gcda
+ $ ./perl.gcov
+ $ gcov regexec.c
+ $ view regexec.c.gcov
+
=head2 Pixie Profiling
Pixie is a profiling tool available on IRIX and Tru64 (aka Digital
or 0xEFEFEFEF, you may be seeing the effect of the Poison() macros,
see L<perlclib>.
+=item *
+
+Under ithreads the optree is read only. If you want to enforce this, to check
+for write accesses from buggy code, compile with C<-DPL_OP_SLAB_ALLOC> to
+enable the OP slab allocator and C<-DPERL_DEBUG_READONLY_OPS> to enable code
+that allocates op memory via C<mmap>, and sets it read-only at run time.
+Any write access to an op results in a C<SIGBUS> and abort.
+
+This code is intended for development only, and may not be portable even to
+all Unix variants. Also, it is an 80% solution, in that it isn't able to make
+all ops read only. Specifically it
+
+=over
+
+=item 1
+
+Only sets read-only on all slabs of ops at C<CHECK> time, hence ops allocated
+later via C<require> or C<eval> will be re-write
+
+=item 2
+
+Turns an entire slab of ops read-write if the refcount of any op in the slab
+needs to be decreased.
+
+=item 3
+
+Turns an entire slab of ops read-write if any op from the slab is freed.
+
+=back
+
+It's not possible to turn the slabs to read-only after an action requiring
+read-write access, as either can happen during op tree building time, so
+there may still be legitimate write access.
+
+However, as an 80% solution it is still effective, as currently it catches
+a write access during the generation of F<Config.pm>, which means that we
+can't yet build F<perl> with this enabled.
+
=back
+
=head1 CONCLUSION
We've had a brief look around the Perl source, how to maintain quality
I'd now suggest you read over those references again, and then, as soon
as possible, get your hands dirty. The best way to learn is by doing,
-so:
+so:
=over 3
=back
-If you can do these things, you've started on the long road to Perl porting.
+If you can do these things, you've started on the long road to Perl porting.
Thanks for wanting to help make Perl better - and happy hacking!
+=head2 Metaphoric Quotations
+
+If you recognized the quote about the Road above, you're in luck.
+
+Most software projects begin each file with a literal description of each
+file's purpose. Perl instead begins each with a literary allusion to that
+file's purpose.
+
+Like chapters in many books, all top-level Perl source files (along with a
+few others here and there) begin with an epigramic inscription that alludes,
+indirectly and metaphorically, to the material you're about to read.
+
+Quotations are taken from writings of J.R.R Tolkien pertaining to his
+Legendarium, almost always from I<The Lord of the Rings>. Chapters and
+page numbers are given using the following editions:
+
+=over 4
+
+=item *
+
+I<The Hobbit>, by J.R.R. Tolkien. The hardcover, 70th-anniversary
+edition of 2007 was used, published in the UK by Harper Collins Publishers
+and in the US by the Houghton Mifflin Company.
+
+=item *
+
+I<The Lord of the Rings>, by J.R.R. Tolkien. The hardcover,
+50th-anniversary edition of 2004 was used, published in the UK by Harper
+Collins Publishers and in the US by the Houghton Mifflin Company.
+
+=item *
+
+I<The Lays of Beleriand>, by J.R.R. Tolkien and published posthumously by his
+son and literary executor, C.J.R. Tolkien, being the 3rd of the 12 volumes
+in Christopher's mammoth I<History of Middle Earth>. Page numbers derive
+from the hardcover edition, first published in 1983 by George Allen &
+Unwin; no page numbers changed for the special 3-volume omnibus edition of
+2002 or the various trade-paper editions, all again now by Harper Collins
+or Houghton Mifflin.
+
+=back
+
+Other JRRT books fair game for quotes would thus include I<The Adventures of
+Tom Bombadil>, I<The Silmarillion>, I<Unfinished Tales>, and I<The Tale of
+the Children of Hurin>, all but the first posthumously assembled by CJRT.
+But I<The Lord of the Rings> itself is perfectly fine and probably best to
+quote from, provided you can find a suitable quote there.
+
+So if you were to supply a new, complete, top-level source file to add to
+Perl, you should conform to this peculiar practice by yourself selecting an
+appropriate quotation from Tolkien, retaining the original spelling and
+punctuation and using the same format the rest of the quotes are in.
+Indirect and oblique is just fine; remember, it's a metaphor, so being meta
+is, after all, what it's for.
+
=head1 AUTHOR
This document was written by Nathan Torkington, and is maintained by
the perl5-porters mailing list.
+=head1 SEE ALSO
+
+L<perlrepository>
https://perl5.git.perl.org / perl5.git / blobdiff