Over this group of porters presides Larry Wall. He has the final word
in what does and does not change in the Perl language. Various
-releases of Perl are shepherded by a ``pumpking'', a porter
-responsible for gathering patches, deciding on a patch-by-patch
+releases of Perl are shepherded by a "pumpking", a porter
+responsible for gathering patches, deciding on a patch-by-patch,
feature-by-feature basis what will and will not go into the release.
For instance, Gurusamy Sarathy was the pumpking for the 5.6 release of
-Perl, and Jarkko Hietaniemi is the pumpking for the 5.8 release, and
-Hugo van der Sanden will be the pumpking for the 5.10 release.
+Perl, and Jarkko Hietaniemi was the pumpking for the 5.8 release, and
+Rafael Garcia-Suarez holds the pumpking crown for the 5.10 release.
In addition, various people are pumpkings for different things. For
-instance, Andy Dougherty and Jarkko Hietaniemi share the I<Configure>
-pumpkin.
+instance, Andy Dougherty and Jarkko Hietaniemi did a grand job as the
+I<Configure> pumpkin up till the 5.8 release. For the 5.10 release
+H.Merijn Brand took over.
Larry sees Perl development along the lines of the US government:
there's the Legislature (the porters), the Executive branch (the
or would it be broadly useful? Sometimes, instead of adding a feature
with a tight focus, the porters might decide to wait until someone
implements the more generalized feature. For instance, instead of
-implementing a ``delayed evaluation'' feature, the porters are waiting
+implementing a "delayed evaluation" feature, the porters are waiting
for a macro system that would permit delayed evaluation and much more.
=item Does it potentially introduce new bugs?
=item Is there another way to do it?
-Larry said ``Although the Perl Slogan is I<There's More Than One Way
-to Do It>, I hesitate to make 10 ways to do something''. This is a
+Larry said "Although the Perl Slogan is I<There's More Than One Way
+to Do It>, I hesitate to make 10 ways to do something". This is a
tricky heuristic to navigate, though--one man's essential addition is
another man's pointless cruft.
Working code is always preferred to pie-in-the-sky ideas. A patch to
add a feature stands a much higher chance of making it to the language
than does a random feature request, no matter how fervently argued the
-request might be. This ties into ``Will it be useful?'', as the fact
+request might be. This ties into "Will it be useful?", as the fact
that someone took the time to make the patch demonstrates a strong
desire for the feature.
=back
-If you're on the list, you might hear the word ``core'' bandied
-around. It refers to the standard distribution. ``Hacking on the
-core'' means you're changing the C source code to the Perl
-interpreter. ``A core module'' is one that ships with Perl.
+If you're on the list, you might hear the word "core" bandied
+around. It refers to the standard distribution. "Hacking on the
+core" means you're changing the C source code to the Perl
+interpreter. "A core module" is one that ships with Perl.
=head2 Keeping in sync
that describe the individual changes that have happened since the last
public release are available at this location:
- ftp://ftp.linux.activestate.com/pub/staff/gsar/APC/
+ http://public.activestate.com/pub/apc/
+ ftp://public.activestate.com/pub/apc/
+
+If you're looking for a particular change, or a change that affected
+a particular set of files, you may find the B<Perl Repository Browser>
+useful:
+
+ http://public.activestate.com/cgi-bin/perlbrowse
+
+You may also want to subscribe to the perl5-changes mailing list to
+receive a copy of each patch that gets submitted to the maintenance
+and development "branches" of the perl repository. See
+http://lists.perl.org/ for subscription information.
If you are a member of the perl5-porters mailing list, it is a good
thing to keep in touch with the most recent changes. If not only to
=item rsync'ing the source tree
Presuming you are in the directory where your perl source resides
-and you have rsync installed and available, you can `upgrade' to
+and you have rsync installed and available, you can "upgrade" to
the bleadperl using:
- # rsync -avz rsync://ftp.linux.activestate.com/perl-current/ .
+ # rsync -avz rsync://public.activestate.com/perl-current/ .
This takes care of updating every single item in the source tree to
the latest applied patch level, creating files that are new (to your
the rsync. Once you are sure that the rsync is running correctly,
run it with the --delete and the --dry-run options like this:
- # rsync -avz --delete --dry-run rsync://ftp.linux.activestate.com/perl-current/ .
+ # rsync -avz --delete --dry-run rsync://public.activestate.com/perl-current/ .
This will I<simulate> an rsync run that also deletes files not
present in the bleadperl master copy. Observe the results from
Presuming you are in a directory where your patches reside, you can
get them in sync with
- # rsync -avz rsync://ftp.linux.activestate.com/perl-current-diffs/ .
+ # rsync -avz rsync://public.activestate.com/perl-current-diffs/ .
This makes sure the latest available patch is downloaded to your
patch directory.
It's then up to you to apply these patches, using something like
- # last=`ls -t *.gz | sed q`
- # rsync -avz rsync://ftp.linux.activestate.com/perl-current-diffs/ .
+ # last="`cat ../perl-current/.patch`.gz"
+ # rsync -avz rsync://public.activestate.com/perl-current-diffs/ .
# find . -name '*.gz' -newer $last -exec gzcat {} \; >blead.patch
# cd ../perl-current
# patch -p1 -N <../perl-current-diffs/blead.patch
Since you don't have to apply the patches yourself, you are sure all
files in the source tree are in the right state.
-=item It's more recent
-
-According to Gurusamy Sarathy:
-
- "... The rsync mirror is automatic and syncs with the repository
- every five minutes.
-
- "Updating the patch area still requires manual intervention
- (with all the goofiness that implies, which you've noted) and
- is typically on a daily cycle. Making this process automatic
- is on my tuit list, but don't ask me when."
-
=item It's more reliable
-Well, since the patches are updated by hand, I don't have to say any
-more ... (see Sarathy's remark).
+While both the rsync-able source and patch areas are automatically
+updated every few minutes, keep in mind that applying patches may
+sometimes mean careful hand-holding, especially if your version of
+the C<patch> program does not understand how to deal with new files,
+files with 8-bit characters, or files without trailing newlines.
=back
I<see> that the community efforts really work. So after each of his
sync points, you are to 'make test' to check if everything is still
in working order. If it is, you do 'make ok', which will send an OK
-report to perlbug@perl.org. (If you do not have access to a mailer
+report to I<perlbug@perl.org>. (If you do not have access to a mailer
from the system you just finished successfully 'make test', you can
do 'make okfile', which creates the file C<perl.ok>, which you can
than take to your favourite mailer and mail yourself).
=back
+=head2 Working with the source
+
+Because you cannot use the Perforce client, you cannot easily generate
+diffs against the repository, nor will merges occur when you update
+via rsync. If you edit a file locally and then rsync against the
+latest source, changes made in the remote copy will I<overwrite> your
+local versions!
+
+The best way to deal with this is to maintain a tree of symlinks to
+the rsync'd source. Then, when you want to edit a file, you remove
+the symlink, copy the real file into the other tree, and edit it. You
+can then diff your edited file against the original to generate a
+patch, and you can safely update the original tree.
+
+Perl's F<Configure> script can generate this tree of symlinks for you.
+The following example assumes that you have used rsync to pull a copy
+of the Perl source into the F<perl-rsync> directory. In the directory
+above that one, you can execute the following commands:
+
+ mkdir perl-dev
+ cd perl-dev
+ ../perl-rsync/Configure -Dmksymlinks -Dusedevel -D"optimize=-g"
+
+This will start the Perl configuration process. After a few prompts,
+you should see something like this:
+
+ Symbolic links are supported.
+
+ Checking how to test for symbolic links...
+ Your builtin 'test -h' may be broken.
+ Trying external '/usr/bin/test -h'.
+ You can test for symbolic links with '/usr/bin/test -h'.
+
+ Creating the symbolic links...
+ (First creating the subdirectories...)
+ (Then creating the symlinks...)
+
+The specifics may vary based on your operating system, of course.
+After you see this, you can abort the F<Configure> script, and you
+will see that the directory you are in has a tree of symlinks to the
+F<perl-rsync> directories and files.
+
+If you plan to do a lot of work with the Perl source, here are some
+Bourne shell script functions that can make your life easier:
+
+ function edit {
+ if [ -L $1 ]; then
+ mv $1 $1.orig
+ cp $1.orig $1
+ vi $1
+ else
+ vi $1
+ fi
+ }
-=head2 Perlbug remote interface
-
-=over 4
-
-There are three (3) remote administrative interfaces for modifying bug
-status, category, etc. In all cases an admin must be first registered
-with the Perlbug database by sending an email request to
-richard@perl.org or bugmongers@perl.org.
-
-The main requirement is the willingness to classify, (with the
-emphasis on closing where possible :), outstanding bugs. Further
-explanation can be garnered from the web at http://bugs.perl.org/ , or
-by asking on the admin mailing list at: bugmongers@perl.org
-
-For more info on the web see
-
- http://bugs.perl.org/perlbug.cgi?req=spec
-
-=item 1 http://bugs.perl.org
-
-Login via the web, (remove B<admin/> if only browsing), where interested Cc's, tests, patches and change-ids, etc. may be assigned.
-
- http://bugs.perl.org/admin/index.html
-
-
-=item 2 bugdb@perl.org
-
-Where the subject line is used for commands:
-
- To: bugdb@perl.org
- Subject: -a close bugid1 bugid2 aix install
-
- To: bugdb@perl.org
- Subject: -h
-
-
-=item 3 commands_and_bugdids@bugs.perl.org
-
-Where the address itself is the source for the commands:
+ function unedit {
+ if [ -L $1.orig ]; then
+ rm $1
+ mv $1.orig $1
+ fi
+ }
- To: close_bugid1_bugid2_aix@bugs.perl.org
+Replace "vi" with your favorite flavor of editor.
+
+Here is another function which will quickly generate a patch for the
+files which have been edited in your symlink tree:
+
+ mkpatchorig() {
+ local diffopts
+ for f in `find . -name '*.orig' | sed s,^\./,,`
+ do
+ case `echo $f | sed 's,.orig$,,;s,.*\.,,'` in
+ c) diffopts=-p ;;
+ pod) diffopts='-F^=' ;;
+ *) diffopts= ;;
+ esac
+ diff -du $diffopts $f `echo $f | sed 's,.orig$,,'`
+ done
+ }
- To: help@bugs.perl.org
+This function produces patches which include enough context to make
+your changes obvious. This makes it easier for the Perl pumpking(s)
+to review them when you send them to the perl5-porters list, and that
+means they're more likely to get applied.
+This function assumed a GNU diff, and may require some tweaking for
+other diff variants.
-=item notes, patches, tests
+=head2 Perlbug administration
-For patches and tests, the message body is assigned to the appropriate bug/s and forwarded to p5p for their attention.
+There is a single remote administrative interface for modifying bug status,
+category, open issues etc. using the B<RT> bugtracker system, maintained
+by Robert Spier. Become an administrator, and close any bugs you can get
+your sticky mitts on:
- To: test_<bugid1>_aix_close@bugs.perl.org
- Subject: this is a test for the (now closed) aix bug
+ http://bugs.perl.org/
- Test is the body of the mail
+To email the bug system administrators:
-=back
+ "perlbug-admin" <perlbug-admin@perl.org>
=head2 Submitting patches
but context diffs are accepted. Do not send RCS-style diffs or diffs
without context lines. More information is given in the
I<Porting/patching.pod> file in the Perl source distribution. Please
-patch against the latest B<development> version (e.g., if you're
-fixing a bug in the 5.005 track, patch against the latest 5.005_5x
-version). Only patches that survive the heat of the development
+patch against the latest B<development> version. (e.g., even if you're
+fixing a bug in the 5.8 track, patch against the latest B<development>
+version rsynced from rsync://public.activestate.com/perl-current/ )
+
+If changes are accepted, they are applied to the development branch. Then
+the 5.8 pumpking decides which of those patches is to be backported to the
+maint branch. Only patches that survive the heat of the development
branch get applied to maintenance versions.
Your patch should update the documentation and test suite. See
-L<Writing a test>.
+L<Writing a test>. If you have added or removed files in the distribution,
+edit the MANIFEST file accordingly, sort the MANIFEST file using
+C<make manisort>, and include those changes as part of your patch.
+
+Patching documentation also follows the same order: if accepted, a patch
+is first applied to B<development>, and if relevant then it's backported
+to B<maintenance>. (With an exception for some patches that document
+behaviour that only appears in the maintenance branch, but which has
+changed in the development version.)
To report a bug in Perl, use the program I<perlbug> which comes with
Perl (if you can't get Perl to work, send mail to the address
I<perlbug@perl.org> or I<perlbug@perl.com>). Reporting bugs through
I<perlbug> feeds into the automated bug-tracking system, access to
-which is provided through the web at http://bugs.perl.org/ . It
+which is provided through the web at http://rt.perl.org/rt3/ . It
often pays to check the archives of the perl5-porters mailing list to
see whether the bug you're reporting has been reported before, and if
so whether it was considered a bug. See above for the location of
The CPAN testers ( http://testers.cpan.org/ ) are a group of
volunteers who test CPAN modules on a variety of platforms. Perl
-Smokers ( http://archives.develooper.com/daily-build@perl.org/ )
-automatically tests Perl source releases on platforms with various
-configurations. Both efforts welcome volunteers.
+Smokers ( http://www.nntp.perl.org/group/perl.daily-build and
+http://www.nntp.perl.org/group/perl.daily-build.reports/ )
+automatically test Perl source releases on platforms with various
+configurations. Both efforts welcome volunteers. In order to get
+involved in smoke testing of the perl itself visit
+L<http://search.cpan.org/dist/Test-Smoke>. In order to start smoke
+testing CPAN modules visit L<http://search.cpan.org/dist/CPAN-YACSmoke/>
+or L<http://search.cpan.org/dist/POE-Component-CPAN-YACSmoke/> or
+L<http://search.cpan.org/dist/CPAN-Reporter/>.
It's a good idea to read and lurk for a while before chipping in.
That way you'll get to see the dynamic of the conversations, learn the
=item The perl5-porters FAQ
-This is posted to perl5-porters at the beginning on every month, and
-should be available from http://perlhacker.org/p5p-faq ; alternatively,
-you can get the FAQ emailed to you by sending mail to
-C<perl5-porters-faq@perl.org>. It contains hints on reading
-perl5-porters, information on how perl5-porters works and how Perl
-development in general works.
+This should be available from http://dev.perl.org/perl5/docs/p5p-faq.html .
+It contains hints on reading perl5-porters, information on how
+perl5-porters works and how Perl development in general works.
=back
C<yyparse>, the parser, lives in F<perly.c>, although you're better off
reading the original YACC input in F<perly.y>. (Yes, Virginia, there
B<is> a YACC grammar for Perl!) The job of the parser is to take your
-code and `understand' it, splitting it into sentences, deciding which
+code and "understand" it, splitting it into sentences, deciding which
operands go with which operators and so on.
The parser is nobly assisted by the lexer, which chunks up your input
execution if required.
The actual functions called are known as PP code, and they're spread
-between four files: F<pp_hot.c> contains the `hot' code, which is most
+between four files: F<pp_hot.c> contains the "hot" code, which is most
often used and highly optimized, F<pp_sys.c> contains all the
system-specific functions, F<pp_ctl.c> contains the functions which
implement control structures (C<if>, C<while> and the like) and F<pp.c>
contains everything else. These are, if you like, the C code for Perl's
built-in functions and operators.
+Note that each C<pp_> function is expected to return a pointer to the next
+op. Calls to perl subs (and eval blocks) are handled within the same
+runops loop, and do not consume extra space on the C stack. For example,
+C<pp_entersub> and C<pp_entertry> just push a C<CxSUB> or C<CxEVAL> block
+struct onto the context stack which contain the address of the op
+following the sub call or eval. They then return the first op of that sub
+or eval block, and so execution continues of that sub or block. Later, a
+C<pp_leavesub> or C<pp_leavetry> op pops the C<CxSUB> or C<CxEVAL>,
+retrieves the return op from it, and returns it.
+
+=item Exception handing
+
+Perl's exception handing (i.e. C<die> etc) is built on top of the low-level
+C<setjmp()>/C<longjmp()> C-library functions. These basically provide a
+way to capture the current PC and SP registers and later restore them; i.e.
+a C<longjmp()> continues at the point in code where a previous C<setjmp()>
+was done, with anything further up on the C stack being lost. This is why
+code should always save values using C<SAVE_FOO> rather than in auto
+variables.
+
+The perl core wraps C<setjmp()> etc in the macros C<JMPENV_PUSH> and
+C<JMPENV_JUMP>. The basic rule of perl exceptions is that C<exit>, and
+C<die> (in the absence of C<eval>) perform a C<JMPENV_JUMP(2)>, while
+C<die> within C<eval> does a C<JMPENV_JUMP(3)>.
+
+At entry points to perl, such as C<perl_parse()>, C<perl_run()> and
+C<call_sv(cv, G_EVAL)> each does a C<JMPENV_PUSH>, then enter a runops
+loop or whatever, and handle possible exception returns. For a 2 return,
+final cleanup is performed, such as popping stacks and calling C<CHECK> or
+C<END> blocks. Amongst other things, this is how scope cleanup still
+occurs during an C<exit>.
+
+If a C<die> can find a C<CxEVAL> block on the context stack, then the
+stack is popped to that level and the return op in that block is assigned
+to C<PL_restartop>; then a C<JMPENV_JUMP(3)> is performed. This normally
+passes control back to the guard. In the case of C<perl_run> and
+C<call_sv>, a non-null C<PL_restartop> triggers re-entry to the runops
+loop. The is the normal way that C<die> or C<croak> is handled within an
+C<eval>.
+
+Sometimes ops are executed within an inner runops loop, such as tie, sort
+or overload code. In this case, something like
+
+ sub FETCH { eval { die } }
+
+would cause a longjmp right back to the guard in C<perl_run>, popping both
+runops loops, which is clearly incorrect. One way to avoid this is for the
+tie code to do a C<JMPENV_PUSH> before executing C<FETCH> in the inner
+runops loop, but for efficiency reasons, perl in fact just sets a flag,
+using C<CATCH_SET(TRUE)>. The C<pp_require>, C<pp_entereval> and
+C<pp_entertry> ops check this flag, and if true, they call C<docatch>,
+which does a C<JMPENV_PUSH> and starts a new runops level to execute the
+code, rather than doing it on the current loop.
+
+As a further optimisation, on exit from the eval block in the C<FETCH>,
+execution of the code following the block is still carried on in the inner
+loop. When an exception is raised, C<docatch> compares the C<JMPENV>
+level of the C<CxEVAL> with C<PL_top_env> and if they differ, just
+re-throws the exception. In this way any inner loops get popped.
+
+Here's an example.
+
+ 1: eval { tie @a, 'A' };
+ 2: sub A::TIEARRAY {
+ 3: eval { die };
+ 4: die;
+ 5: }
+
+To run this code, C<perl_run> is called, which does a C<JMPENV_PUSH> then
+enters a runops loop. This loop executes the eval and tie ops on line 1,
+with the eval pushing a C<CxEVAL> onto the context stack.
+
+The C<pp_tie> does a C<CATCH_SET(TRUE)>, then starts a second runops loop
+to execute the body of C<TIEARRAY>. When it executes the entertry op on
+line 3, C<CATCH_GET> is true, so C<pp_entertry> calls C<docatch> which
+does a C<JMPENV_PUSH> and starts a third runops loop, which then executes
+the die op. At this point the C call stack looks like this:
+
+ Perl_pp_die
+ Perl_runops # third loop
+ S_docatch_body
+ S_docatch
+ Perl_pp_entertry
+ Perl_runops # second loop
+ S_call_body
+ Perl_call_sv
+ Perl_pp_tie
+ Perl_runops # first loop
+ S_run_body
+ perl_run
+ main
+
+and the context and data stacks, as shown by C<-Dstv>, look like:
+
+ STACK 0: MAIN
+ CX 0: BLOCK =>
+ CX 1: EVAL => AV() PV("A"\0)
+ retop=leave
+ STACK 1: MAGIC
+ CX 0: SUB =>
+ retop=(null)
+ CX 1: EVAL => *
+ retop=nextstate
+
+The die pops the first C<CxEVAL> off the context stack, sets
+C<PL_restartop> from it, does a C<JMPENV_JUMP(3)>, and control returns to
+the top C<docatch>. This then starts another third-level runops level,
+which executes the nextstate, pushmark and die ops on line 4. At the point
+that the second C<pp_die> is called, the C call stack looks exactly like
+that above, even though we are no longer within an inner eval; this is
+because of the optimization mentioned earlier. However, the context stack
+now looks like this, ie with the top CxEVAL popped:
+
+ STACK 0: MAIN
+ CX 0: BLOCK =>
+ CX 1: EVAL => AV() PV("A"\0)
+ retop=leave
+ STACK 1: MAGIC
+ CX 0: SUB =>
+ retop=(null)
+
+The die on line 4 pops the context stack back down to the CxEVAL, leaving
+it as:
+
+ STACK 0: MAIN
+ CX 0: BLOCK =>
+
+As usual, C<PL_restartop> is extracted from the C<CxEVAL>, and a
+C<JMPENV_JUMP(3)> done, which pops the C stack back to the docatch:
+
+ S_docatch
+ Perl_pp_entertry
+ Perl_runops # second loop
+ S_call_body
+ Perl_call_sv
+ Perl_pp_tie
+ Perl_runops # first loop
+ S_run_body
+ perl_run
+ main
+
+In this case, because the C<JMPENV> level recorded in the C<CxEVAL>
+differs from the current one, C<docatch> just does a C<JMPENV_JUMP(3)>
+and the C stack unwinds to:
+
+ perl_run
+ main
+
+Because C<PL_restartop> is non-null, C<run_body> starts a new runops loop
+and execution continues.
+
=back
=head2 Internal Variable Types
Line 13 manipulates the flags; since we've changed the PV, any IV or NV
values will no longer be valid: if we have C<$a=10; $a.="6";> we don't
-want to use the old IV of 10. C<SvPOK_only_utf8> is a special UTF8-aware
+want to use the old IV of 10. C<SvPOK_only_utf8> is a special UTF-8-aware
version of C<SvPOK_only>, a macro which turns off the IOK and NOK flags
and turns on POK. The final C<SvTAINT> is a macro which launders tainted
data if taint mode is turned on.
The easiest way to examine the op tree is to stop Perl after it has
finished parsing, and get it to dump out the tree. This is exactly what
-the compiler backends L<B::Terse|B::Terse> and L<B::Debug|B::Debug> do.
+the compiler backends L<B::Terse|B::Terse>, L<B::Concise|B::Concise>
+and L<B::Debug|B::Debug> do.
Let's have a look at how Perl sees C<$a = $b + $c>:
fed certain things by the tokeniser, which generally end up in upper
case. Here, C<ADDOP>, is provided when the tokeniser sees C<+> in your
code. C<ASSIGNOP> is provided when C<=> is used for assigning. These are
-`terminal symbols', because you can't get any simpler than them.
+"terminal symbols", because you can't get any simpler than them.
The grammar, lines one and three of the snippet above, tells you how to
-build up more complex forms. These complex forms, `non-terminal symbols'
+build up more complex forms. These complex forms, "non-terminal symbols"
are generally placed in lower case. C<term> here is a non-terminal
symbol, representing a single expression.
C<newBINOP>, a function in F<op.c>, is the op type. It's an addition
operator, so we want the type to be C<ADDOP>. We could specify this
directly, but it's right there as the second token in the input, so we
-use C<$2>. The second parameter is the op's flags: 0 means `nothing
-special'. Then the things to add: the left and right hand side of our
+use C<$2>. The second parameter is the op's flags: 0 means "nothing
+special". Then the things to add: the left and right hand side of our
expression, in scalar context.
=head2 Stacks
=item Mark stack
-I say `your portion of the stack' above because PP code doesn't
+I say "your portion of the stack" above because PP code doesn't
necessarily get the whole stack to itself: if your function calls
another function, you'll only want to expose the arguments aimed for the
called function, and not (necessarily) let it get at your own data. The
-way we do this is to have a `virtual' bottom-of-stack, exposed to each
+way we do this is to have a "virtual" bottom-of-stack, exposed to each
function. The mark stack keeps bookmarks to locations in the argument
stack usable by each function. For instance, when dealing with a tied
-variable, (internally, something with `P' magic) Perl has to call
+variable, (internally, something with "P" magic) Perl has to call
methods for accesses to the tied variables. However, we need to separate
the arguments exposed to the method to the argument exposed to the
-original function - the store or fetch or whatever it may be. Here's how
-the tied C<push> is implemented; see C<av_push> in F<av.c>:
+original function - the store or fetch or whatever it may be. Here's
+
roughly how the tied C<push> is implemented; see C<av_push> in F<av.c>:
1 PUSHMARK(SP);
2 EXTEND(SP,2);
6 ENTER;
7 call_method("PUSH", G_SCALAR|G_DISCARD);
8 LEAVE;
- 9 POPSTACK;
-
-The lines which concern the mark stack are the first, fifth and last
-lines: they save away, restore and remove the current position of the
-argument stack.
Let's examine the whole implementation, for practice:
5 PUTBACK;
-Next we tell Perl to make the change to the global stack pointer: C<dSP>
-only gave us a local copy, not a reference to the global.
+Next we tell Perl to update the global stack pointer from our internal
+variable: C<dSP> only gave us a local copy, not a reference to the global.
6 ENTER;
7 call_method("PUSH", G_SCALAR|G_DISCARD);
To actually do the magic method call, we have to call a subroutine in
Perl space: C<call_method> takes care of that, and it's described in
L<perlcall>. We call the C<PUSH> method in scalar context, and we're
-going to discard its return value.
-
- 9 POPSTACK;
-
-Finally, we remove the value we placed on the mark stack, since we
-don't need it any more.
+going to discard its return value. The call_method() function
+removes the top element of the mark stack, so there is nothing for
+the caller to clean up.
=item Save stack
to L<perlguts/Background and PERL_IMPLICIT_CONTEXT> for information on
the C<[pad]THX_?> macros.
+=head2 The .i Targets
+
+You can expand the macros in a F<foo.c> file by saying
+
+ make foo.i
+
+which will expand the macros using cpp. Don't be scared by the results.
+
+=head1 SOURCE CODE STATIC ANALYSIS
+
+Various tools exist for analysing C source code B<statically>, as
+opposed to B<dynamically>, that is, without executing the code.
+It is possible to detect resource leaks, undefined behaviour, type
+mismatches, portability problems, code paths that would cause illegal
+memory accesses, and other similar problems by just parsing the C code
+and looking at the resulting graph, what does it tell about the
+execution and data flows. As a matter of fact, this is exactly
+how C compilers know to give warnings about dubious code.
+
+=head2 lint, splint
+
+The good old C code quality inspector, C<lint>, is available in
+several platforms, but please be aware that there are several
+different implementations of it by different vendors, which means that
+the flags are not identical across different platforms.
+
+There is a lint variant called C<splint> (Secure Programming Lint)
+available from http://www.splint.org/ that should compile on any
+Unix-like platform.
+
+There are C<lint> and <splint> targets in Makefile, but you may have
+to diddle with the flags (see above).
+
+=head2 Coverity
+
+Coverity (http://www.coverity.com/) is a product similar to lint and
+as a testbed for their product they periodically check several open
+source projects, and they give out accounts to open source developers
+to the defect databases.
+
+=head2 cpd (cut-and-paste detector)
+
+The cpd tool detects cut-and-paste coding. If one instance of the
+cut-and-pasted code changes, all the other spots should probably be
+changed, too. Therefore such code should probably be turned into a
+subroutine or a macro.
+
+cpd (http://pmd.sourceforge.net/cpd.html) is part of the pmd project
+(http://pmd.sourceforge.net/). pmd was originally written for static
+analysis of Java code, but later the cpd part of it was extended to
+parse also C and C++.
+
+Download the pmd-bin-X.Y.zip () from the SourceForge site, extract the
+pmd-X.Y.jar from it, and then run that on source code thusly:
+
+ java -cp pmd-X.Y.jar net.sourceforge.pmd.cpd.CPD --minimum-tokens 100 --files /some/where/src --language c > cpd.txt
+
+You may run into memory limits, in which case you should use the -Xmx option:
+
+ java -Xmx512M ...
+
+=head2 gcc warnings
+
+Though much can be written about the inconsistency and coverage
+problems of gcc warnings (like C<-Wall> not meaning "all the
+warnings", or some common portability problems not being covered by
+C<-Wall>, or C<-ansi> and C<-pedantic> both being a poorly defined
+collection of warnings, and so forth), gcc is still a useful tool in
+keeping our coding nose clean.
+
+The C<-Wall> is by default on.
+
+The C<-ansi> (and its sidekick, C<-pedantic>) would be nice to be on
+always, but unfortunately they are not safe on all platforms, they can
+for example cause fatal conflicts with the system headers (Solaris
+being a prime example). If Configure C<-Dgccansipedantic> is used,
+the C<cflags> frontend selects C<-ansi -pedantic> for the platforms
+where they are known to be safe.
+
+Starting from Perl 5.9.4 the following extra flags are added:
+
+=over 4
+
+=item *
+
+C<-Wendif-labels>
+
+=item *
+
+C<-Wextra>
+
+=item *
+
+C<-Wdeclaration-after-statement>
+
+=back
+
+The following flags would be nice to have but they would first need
+their own Stygian stablemaster:
+
+=over 4
+
+=item *
+
+C<-Wpointer-arith>
+
+=item *
+
+C<-Wshadow>
+
+=item *
+
+C<-Wstrict-prototypes>
+
+=back
+
+The C<-Wtraditional> is another example of the annoying tendency of
+gcc to bundle a lot of warnings under one switch -- it would be
+impossible to deploy in practice because it would complain a lot -- but
+it does contain some warnings that would be beneficial to have available
+on their own, such as the warning about string constants inside macros
+containing the macro arguments: this behaved differently pre-ANSI
+than it does in ANSI, and some C compilers are still in transition,
+AIX being an example.
+
+=head2 Warnings of other C compilers
+
+Other C compilers (yes, there B<are> other C compilers than gcc) often
+have their "strict ANSI" or "strict ANSI with some portability extensions"
+modes on, like for example the Sun Workshop has its C<-Xa> mode on
+(though implicitly), or the DEC (these days, HP...) has its C<-std1>
+mode on.
+
+=head2 DEBUGGING
+
+You can compile a special debugging version of Perl, which allows you
+to use the C<-D> option of Perl to tell more about what Perl is doing.
+But sometimes there is no alternative than to dive in with a debugger,
+either to see the stack trace of a core dump (very useful in a bug
+report), or trying to figure out what went wrong before the core dump
+happened, or how did we end up having wrong or unexpected results.
+
=head2 Poking at Perl
To really poke around with Perl, you'll probably want to build Perl for
make
C<-g> is a flag to the C compiler to have it produce debugging
-information which will allow us to step through a running program.
+information which will allow us to step through a running program,
+and to see in which C function we are at (without the debugging
+information we might see only the numerical addresses of the functions,
+which is not very helpful).
+
F<Configure> will also turn on the C<DEBUGGING> compilation symbol which
enables all the internal debugging code in Perl. There are a whole bunch
of things you can debug with this: L<perlrun> lists them all, and the
=item *
-We'll use C<gdb> for our examples here; the principles will apply to any
-debugger, but check the manual of the one you're using.
+We'll use C<gdb> for our examples here; the principles will apply to
+any debugger (many vendors call their debugger C<dbx>), but check the
+manual of the one you're using.
=back
gdb ./perl
+Or if you have a core dump:
+
+ gdb ./perl core
+
You'll want to do that in your Perl source tree so the debugger can read
the source code. You should see the copyright message, followed by the
prompt.
=item print
Execute the given C code and print its results. B<WARNING>: Perl makes
-heavy use of macros, and F<gdb> is not aware of macros. You'll have to
-substitute them yourself. So, for instance, you can't say
+heavy use of macros, and F<gdb> does not necessarily support macros
+(see later L</"gdb macro support">). You'll have to substitute them
+yourself, or to invoke cpp on the source code files
+(see L</"The .i Targets">)
+So, for instance, you can't say
print SvPV_nolen(sv)
print Perl_sv_2pv_nolen(sv)
+=back
+
You may find it helpful to have a "macro dictionary", which you can
produce by saying C<cpp -dM perl.c | sort>. Even then, F<cpp> won't
-recursively apply the macros for you.
+recursively apply those macros for you.
-=back
+=head2 gdb macro support
+
+Recent versions of F<gdb> have fairly good macro support, but
+in order to use it you'll need to compile perl with macro definitions
+included in the debugging information. Using F<gcc> version 3.1, this
+means configuring with C<-Doptimize=-g3>. Other compilers might use a
+different switch (if they support debugging macros at all).
=head2 Dumping Perl Data Structures
F<dump.c>; these work a little like an internal
L<Devel::Peek|Devel::Peek>, but they also cover OPs and other structures
that you can't get at from Perl. Let's take an example. We'll use the
-C<$a = $b + $c> we used before, but give it a bit of context:
+C<$a = $b + $c> we used before, but give it a bit of context:
C<$b = "6XXXX"; $c = 2.3;>. Where's a good place to stop and poke around?
What about C<pp_add>, the function we examined earlier to implement the
directly (if C<SvNOK> is set) or by calling the C<sv_2nv> function.
C<TOPs> takes the next SV from the top of the stack - yes, C<POPn> uses
C<TOPs> - but doesn't remove it. We then use C<SvNV> to get the NV from
-C<leftsv> in the same way as before - yes, C<POPn> uses C<SvNV>.
+C<leftsv> in the same way as before - yes, C<POPn> uses C<SvNV>.
Since we don't have an NV for C<$b>, we'll have to use C<sv_2nv> to
convert it. If we step again, we'll find ourselves there:
All right, we've now had a look at how to navigate the Perl sources and
some things you'll need to know when fiddling with them. Let's now get
on and create a simple patch. Here's something Larry suggested: if a
-C<U> is the first active format during a C<pack>, (for example,
+C<U> is the first active format during a C<pack>, (for example,
C<pack "U3C8", @stuff>) then the resulting string should be treated as
-UTF8 encoded.
+UTF-8 encoded.
How do we prepare to fix this up? First we locate the code in question -
the C<pack> happens at runtime, so it's going to be in one of the F<pp>
while (pat < patend) {
Now if we see a C<U> which was at the start of the string, we turn on
-the UTF8 flag for the output SV, C<cat>:
+the C<UTF8> flag for the output SV, C<cat>:
+ if (datumtype == 'U' && pat==patcopy+1)
+ SvUTF8_on(cat);
The regression tests for each operator live in F<t/op/>, and so we
make a copy of F<t/op/pack.t> to F<t/op/pack.t~>. Now we can add our
tests to the end. First, we'll test that the C<U> does indeed create
-Unicode strings.
+Unicode strings.
t/op/pack.t has a sensible ok() function, but if it didn't we could
use the one from t/test.pl.
we can write the more sensible (see L<Test::More> for a full
explanation of is() and other testing functions).
- is( "1.20.300.4000", sprintf "%vd", pack("U*",1,20,300,4000),
- "U* produces unicode" );
+ is( "1.20.300.4000", sprintf "%vd", pack("U*",1,20,300,4000),
+ "U* produces Unicode" );
Now we'll test that we got that space-at-the-beginning business right:
the first active format:
isnt( v1.20.300.4000, sprintf "%vd", pack("C0U*",1,20,300,4000),
- "U* not first isn't unicode" );
+ "U* not first isn't Unicode" );
Mustn't forget to change the number of tests which appears at the top,
or else the automated tester will get confused. This will either look
=item *
If the pattern begins with a C<U>, the resulting string will be treated
- as Unicode-encoded. You can force UTF8 encoding on in a string with an
- initial C<U0>, and the bytes that follow will be interpreted as Unicode
- characters. If you don't want this to happen, you can begin your pattern
- with C<C0> (or anything else) to force Perl not to UTF8 encode your
+ as UTF-8-encoded Unicode. You can force UTF-8 encoding on in a string
+ with an initial C<U0>, and the bytes that follow will be interpreted as
+ Unicode characters. If you don't want this to happen, you can begin your
+ pattern with C<C0> (or anything else) to force Perl not to UTF-8 encode your
string, and then follow this with a C<U*> somewhere in your pattern.
All done. Now let's create the patch. F<Porting/patching.pod> tells us
maintainer keep the CPAN version in sync with the core version without
constantly scanning p5p.
+The list of maintainers of core modules is usefully documented in
+F<Porting/Maintainers.pl>.
+
=head2 Adding a new function to the core
If, as part of a patch to fix a bug, or just because you have an
Testing features of how perl actually runs, including exit codes and
handling of PERL* environment variables.
+=item F<t/uni/>
+
+Tests for the core support of Unicode.
+
+=item F<t/win32/>
+
+Windows-specific tests.
+
+=item F<t/x2p>
+
+A test suite for the s2p converter.
+
=back
The core uses the same testing style as the rest of Perl, a simple
working on. This is a measure to prevent a high-level failure (such
as Config.pm breaking) from causing basic functionality tests to fail.
-=over 4
+=over 4
=item t/base t/comp
=back
When you say "make test" Perl uses the F<t/TEST> program to run the
-test suite. All tests are run from the F<t/> directory, B<not> the
-directory which contains the test. This causes some problems with the
-tests in F<lib/>, so here's some opportunity for some patching.
+test suite (except under Win32 where it uses F<t/harness> instead.)
+All tests are run from the F<t/> directory, B<not> the directory
+which contains the test. This causes some problems with the tests
+in F<lib/>, so here's some opportunity for some patching.
You must be triply conscious of cross-platform concerns. This usually
boils down to using File::Spec and avoiding things like C<fork()> and
There are various special make targets that can be used to test Perl
slightly differently than the standard "test" target. Not all them
are expected to give a 100% success rate. Many of them have several
-aliases.
+aliases, and many of them are not available on certain operating
+systems.
=over 4
=item coretest
-Run F<perl> on all but F<lib/*> tests.
+Run F<perl> on all core tests (F<t/*> and F<lib/[a-z]*> pragma tests).
+
+(Not available on Win32)
=item test.deparse
-Run all the tests through the B::Deparse. Not all tests will succeed.
+Run all the tests through B::Deparse. Not all tests will succeed.
+
+(Not available on Win32)
+
+=item test.taintwarn
+
+Run all tests with the B<-t> command-line switch. Not all tests
+are expected to succeed (until they're specifically fixed, of course).
+
+(Not available on Win32)
=item minitest
Run F<miniperl> on F<t/base>, F<t/comp>, F<t/cmd>, F<t/run>, F<t/io>,
F<t/op>, and F<t/uni> tests.
+=item test.valgrind check.valgrind utest.valgrind ucheck.valgrind
+
+(Only in Linux) Run all the tests using the memory leak + naughty
+memory access tool "valgrind". The log files will be named
+F<testname.valgrind>.
+
=item test.third check.third utest.third ucheck.third
(Only in Tru64) Run all the tests using the memory leak + naughty
memory access tool "Third Degree". The log files will be named
-F<perl3.log.testname>.
+F<perl.3log.testname>.
=item test.torture torturetest
Run all the usual tests and some extra tests. As of Perl 5.8.0 the
-only extra tests are Abigail's JAPHs, t/japh/abigail.t.
+only extra tests are Abigail's JAPHs, F<t/japh/abigail.t>.
You can also run the torture test with F<t/harness> by giving
C<-torture> argument to F<t/harness>.
Run all the tests with -Mutf8. Not all tests will succeed.
+(Not available on Win32)
+
+=item minitest.utf16 test.utf16
+
+Runs the tests with UTF-16 encoded scripts, encoded with different
+versions of this encoding.
+
+C<make utest.utf16> runs the test suite with a combination of C<-utf8> and
+C<-utf16> arguments to F<t/TEST>.
+
+(Not available on Win32)
+
+=item test_harness
+
+Run the test suite with the F<t/harness> controlling program, instead of
+F<t/TEST>. F<t/harness> is more sophisticated, and uses the
+L<Test::Harness> module, thus using this test target supposes that perl
+mostly works. The main advantage for our purposes is that it prints a
+detailed summary of failed tests at the end. Also, unlike F<t/TEST>, it
+doesn't redirect stderr to stdout.
+
+Note that under Win32 F<t/harness> is always used instead of F<t/TEST>, so
+there is no special "test_harness" target.
+
+Under Win32's "test" target you may use the TEST_SWITCHES and TEST_FILES
+environment variables to control the behaviour of F<t/harness>. This means
+you can say
+
+ nmake test TEST_FILES="op/*.t"
+ nmake test TEST_SWITCHES="-torture" TEST_FILES="op/*.t"
+
+=item test-notty test_notty
+
+Sets PERL_SKIP_TTY_TEST to true before running normal test.
+
+=back
+
+=head2 Running tests by hand
+
+You can run part of the test suite by hand by using one the following
+commands from the F<t/> directory :
+
+ ./perl -I../lib TEST list-of-.t-files
+
+or
+
+ ./perl -I../lib harness list-of-.t-files
+
+(if you don't specify test scripts, the whole test suite will be run.)
+
+=head3 Using t/harness for testing
+
+If you use C<harness> for testing you have several command line options
+available to you. The arguments are as follows, and are in the order
+that they must appear if used together.
+
+ harness -v -torture -re=pattern LIST OF FILES TO TEST
+ harness -v -torture -re LIST OF PATTERNS TO MATCH
+
+If C<LIST OF FILES TO TEST> is omitted the file list is obtained from
+the manifest. The file list may include shell wildcards which will be
+expanded out.
+
+=over 4
+
+=item -v
+
+Run the tests under verbose mode so you can see what tests were run,
+and debug outbut.
+
+=item -torture
+
+Run the torture tests as well as the normal set.
+
+=item -re=PATTERN
+
+Filter the file list so that all the test files run match PATTERN.
+Note that this form is distinct from the B<-re LIST OF PATTERNS> form below
+in that it allows the file list to be provided as well.
+
+=item -re LIST OF PATTERNS
+
+Filter the file list so that all the test files run match
+/(LIST|OF|PATTERNS)/. Note that with this form the patterns
+are joined by '|' and you cannot supply a list of files, instead
+the test files are obtained from the MANIFEST.
+
+=back
+
+You can run an individual test by a command similar to
+
+ ./perl -I../lib patho/to/foo.t
+
+except that the harnesses set up some environment variables that may
+affect the execution of the test :
+
+=over 4
+
+=item PERL_CORE=1
+
+indicates that we're running this test part of the perl core test suite.
+This is useful for modules that have a dual life on CPAN.
+
+=item PERL_DESTRUCT_LEVEL=2
+
+is set to 2 if it isn't set already (see L</PERL_DESTRUCT_LEVEL>)
+
+=item PERL
+
+(used only by F<t/TEST>) if set, overrides the path to the perl executable
+that should be used to run the tests (the default being F<./perl>).
+
+=item PERL_SKIP_TTY_TEST
+
+if set, tells to skip the tests that need a terminal. It's actually set
+automatically by the Makefile, but can also be forced artificially by
+running 'make test_notty'.
+
+=back
+
+=head2 Common problems when patching Perl source code
+
+Perl source plays by ANSI C89 rules: no C99 (or C++) extensions. In
+some cases we have to take pre-ANSI requirements into consideration.
+You don't care about some particular platform having broken Perl?
+I hear there is still a strong demand for J2EE programmers.
+
+=head2 Perl environment problems
+
+=over 4
+
+=item *
+
+Not compiling with threading
+
+Compiling with threading (-Duseithreads) completely rewrites
+the function prototypes of Perl. You better try your changes
+with that. Related to this is the difference between "Perl_-less"
+and "Perl_-ly" APIs, for example:
+
+ Perl_sv_setiv(aTHX_ ...);
+ sv_setiv(...);
+
+The first one explicitly passes in the context, which is needed for e.g.
+threaded builds. The second one does that implicitly; do not get them
+mixed. If you are not passing in a aTHX_, you will need to do a dTHX
+(or a dVAR) as the first thing in the function.
+
+See L<perlguts/"How multiple interpreters and concurrency are supported">
+for further discussion about context.
+
+=item *
+
+Not compiling with -DDEBUGGING
+
+The DEBUGGING define exposes more code to the compiler,
+therefore more ways for things to go wrong. You should try it.
+
+=item *
+
+Introducing (non-read-only) globals
+
+Do not introduce any modifiable globals, truly global or file static.
+They are bad form and complicate multithreading and other forms of
+concurrency. The right way is to introduce them as new interpreter
+variables, see F<intrpvar.h> (at the very end for binary compatibility).
+
+Introducing read-only (const) globals is okay, as long as you verify
+with e.g. C<nm libperl.a|egrep -v ' [TURtr] '> (if your C<nm> has
+BSD-style output) that the data you added really is read-only.
+(If it is, it shouldn't show up in the output of that command.)
+
+If you want to have static strings, make them constant:
+
+ static const char etc[] = "...";
+
+If you want to have arrays of constant strings, note carefully
+the right combination of C<const>s:
+
+ static const char * const yippee[] =
+ {"hi", "ho", "silver"};
+
+There is a way to completely hide any modifiable globals (they are all
+moved to heap), the compilation setting C<-DPERL_GLOBAL_STRUCT_PRIVATE>.
+It is not normally used, but can be used for testing, read more
+about it in L<perlguts/"Background and PERL_IMPLICIT_CONTEXT">.
+
+=item *
+
+Not exporting your new function
+
+Some platforms (Win32, AIX, VMS, OS/2, to name a few) require any
+function that is part of the public API (the shared Perl library)
+to be explicitly marked as exported. See the discussion about
+F<embed.pl> in L<perlguts>.
+
+=item *
+
+Exporting your new function
+
+The new shiny result of either genuine new functionality or your
+arduous refactoring is now ready and correctly exported. So what
+could possibly go wrong?
+
+Maybe simply that your function did not need to be exported in the
+first place. Perl has a long and not so glorious history of exporting
+functions that it should not have.
+
+If the function is used only inside one source code file, make it
+static. See the discussion about F<embed.pl> in L<perlguts>.
+
+If the function is used across several files, but intended only for
+Perl's internal use (and this should be the common case), do not
+export it to the public API. See the discussion about F<embed.pl>
+in L<perlguts>.
+
+=back
+
+=head2 Portability problems
+
+The following are common causes of compilation and/or execution
+failures, not common to Perl as such. The C FAQ is good bedtime
+reading. Please test your changes with as many C compilers and
+platforms as possible -- we will, anyway, and it's nice to save
+oneself from public embarrassment.
+
+If using gcc, you can add the C<-std=c89> option which will hopefully
+catch most of these unportabilities. (However it might also catch
+incompatibilities in your system's header files.)
+
+Use the Configure C<-Dgccansipedantic> flag to enable the gcc
+C<-ansi -pedantic> flags which enforce stricter ANSI rules.
+
+If using the C<gcc -Wall> note that not all the possible warnings
+(like C<-Wunitialized>) are given unless you also compile with C<-O>.
+
+Note that if using gcc, starting from Perl 5.9.5 the Perl core source
+code files (the ones at the top level of the source code distribution,
+but not e.g. the extensions under ext/) are automatically compiled
+with as many as possible of the C<-std=c89>, C<-ansi>, C<-pedantic>,
+and a selection of C<-W> flags (see cflags.SH).
+
+Also study L<perlport> carefully to avoid any bad assumptions
+about the operating system, filesystems, and so forth.
+
+You may once in a while try a "make microperl" to see whether we
+can still compile Perl with just the bare minimum of interfaces.
+(See README.micro.)
+
+Do not assume an operating system indicates a certain compiler.
+
+=over 4
+
+=item *
+
+Casting pointers to integers or casting integers to pointers
+
+ void castaway(U8* p)
+ {
+ IV i = p;
+
+or
+
+ void castaway(U8* p)
+ {
+ IV i = (IV)p;
+
+Both are bad, and broken, and unportable. Use the PTR2IV()
+macro that does it right. (Likewise, there are PTR2UV(), PTR2NV(),
+INT2PTR(), and NUM2PTR().)
+
+=item *
+
+Casting between data function pointers and data pointers
+
+Technically speaking casting between function pointers and data
+pointers is unportable and undefined, but practically speaking
+it seems to work, but you should use the FPTR2DPTR() and DPTR2FPTR()
+macros. Sometimes you can also play games with unions.
+
+=item *
+
+Assuming sizeof(int) == sizeof(long)
+
+There are platforms where longs are 64 bits, and platforms where ints
+are 64 bits, and while we are out to shock you, even platforms where
+shorts are 64 bits. This is all legal according to the C standard.
+(In other words, "long long" is not a portable way to specify 64 bits,
+and "long long" is not even guaranteed to be any wider than "long".)
+
+Instead, use the definitions IV, UV, IVSIZE, I32SIZE, and so forth.
+Avoid things like I32 because they are B<not> guaranteed to be
+I<exactly> 32 bits, they are I<at least> 32 bits, nor are they
+guaranteed to be B<int> or B<long>. If you really explicitly need
+64-bit variables, use I64 and U64, but only if guarded by HAS_QUAD.
+
+=item *
+
+Assuming one can dereference any type of pointer for any type of data
+
+ char *p = ...;
+ long pony = *p; /* BAD */
+
+Many platforms, quite rightly so, will give you a core dump instead
+of a pony if the p happens not be correctly aligned.
+
+=item *
+
+Lvalue casts
+
+ (int)*p = ...; /* BAD */
+
+Simply not portable. Get your lvalue to be of the right type,
+or maybe use temporary variables, or dirty tricks with unions.
+
+=item *
+
+Assume B<anything> about structs (especially the ones you
+don't control, like the ones coming from the system headers)
+
+=over 8
+
+=item *
+
+That a certain field exists in a struct
+
+=item *
+
+That no other fields exist besides the ones you know of
+
+=item *
+
+That a field is of certain signedness, sizeof, or type
+
+=item *
+
+That the fields are in a certain order
+
+=over 8
+
+=item *
+
+While C guarantees the ordering specified in the struct definition,
+between different platforms the definitions might differ
+
+=back
+
+=item *
+
+That the sizeof(struct) or the alignments are the same everywhere
+
+=over 8
+
+=item *
+
+There might be padding bytes between the fields to align the fields -
+the bytes can be anything
+
+=item *
+
+Structs are required to be aligned to the maximum alignment required
+by the fields - which for native types is for usually equivalent to
+sizeof() of the field
+
+=back
+
+=back
+
+=item *
+
+Mixing #define and #ifdef
+
+ #define BURGLE(x) ... \
+ #ifdef BURGLE_OLD_STYLE /* BAD */
+ ... do it the old way ... \
+ #else
+ ... do it the new way ... \
+ #endif
+
+You cannot portably "stack" cpp directives. For example in the above
+you need two separate BURGLE() #defines, one for each #ifdef branch.
+
+=item *
+
+Adding stuff after #endif or #else
+
+ #ifdef SNOSH
+ ...
+ #else !SNOSH /* BAD */
+ ...
+ #endif SNOSH /* BAD */
+
+The #endif and #else cannot portably have anything non-comment after
+them. If you want to document what is going (which is a good idea
+especially if the branches are long), use (C) comments:
+
+ #ifdef SNOSH
+ ...
+ #else /* !SNOSH */
+ ...
+ #endif /* SNOSH */
+
+The gcc option C<-Wendif-labels> warns about the bad variant
+(by default on starting from Perl 5.9.4).
+
+=item *
+
+Having a comma after the last element of an enum list
+
+ enum color {
+ CERULEAN,
+ CHARTREUSE,
+ CINNABAR, /* BAD */
+ };
+
+is not portable. Leave out the last comma.
+
+Also note that whether enums are implicitly morphable to ints
+varies between compilers, you might need to (int).
+
+=item *
+
+Using //-comments
+
+ // This function bamfoodles the zorklator. /* BAD */
+
+That is C99 or C++. Perl is C89. Using the //-comments is silently
+allowed by many C compilers but cranking up the ANSI C89 strictness
+(which we like to do) causes the compilation to fail.
+
+=item *
+
+Mixing declarations and code
+
+ void zorklator()
+ {
+ int n = 3;
+ set_zorkmids(n); /* BAD */
+ int q = 4;
+
+That is C99 or C++. Some C compilers allow that, but you shouldn't.
+
+The gcc option C<-Wdeclaration-after-statements> scans for such problems
+(by default on starting from Perl 5.9.4).
+
+=item *
+
+Introducing variables inside for()
+
+ for(int i = ...; ...; ...) { /* BAD */
+
+That is C99 or C++. While it would indeed be awfully nice to have that
+also in C89, to limit the scope of the loop variable, alas, we cannot.
+
+=item *
+
+Mixing signed char pointers with unsigned char pointers
+
+ int foo(char *s) { ... }
+ ...
+ unsigned char *t = ...; /* Or U8* t = ... */
+ foo(t); /* BAD */
+
+While this is legal practice, it is certainly dubious, and downright
+fatal in at least one platform: for example VMS cc considers this a
+fatal error. One cause for people often making this mistake is that a
+"naked char" and therefore dereferencing a "naked char pointer" have
+an undefined signedness: it depends on the compiler and the flags of
+the compiler and the underlying platform whether the result is signed
+or unsigned. For this very same reason using a 'char' as an array
+index is bad.
+
+=item *
+
+Macros that have string constants and their arguments as substrings of
+the string constants
+
+ #define FOO(n) printf("number = %d\n", n) /* BAD */
+ FOO(10);
+
+Pre-ANSI semantics for that was equivalent to
+
+ printf("10umber = %d\10");
+
+which is probably not what you were expecting. Unfortunately at least
+one reasonably common and modern C compiler does "real backward
+compatibility" here, in AIX that is what still happens even though the
+rest of the AIX compiler is very happily C89.
+
+=item *
+
+Using printf formats for non-basic C types
+
+ IV i = ...;
+ printf("i = %d\n", i); /* BAD */
+
+While this might by accident work in some platform (where IV happens
+to be an C<int>), in general it cannot. IV might be something larger.
+Even worse the situation is with more specific types (defined by Perl's
+configuration step in F<config.h>):
+
+ Uid_t who = ...;
+ printf("who = %d\n", who); /* BAD */
+
+The problem here is that Uid_t might be not only not C<int>-wide
+but it might also be unsigned, in which case large uids would be
+printed as negative values.
+
+There is no simple solution to this because of printf()'s limited
+intelligence, but for many types the right format is available as
+with either 'f' or '_f' suffix, for example:
+
+ IVdf /* IV in decimal */
+ UVxf /* UV is hexadecimal */
+
+ printf("i = %"IVdf"\n", i); /* The IVdf is a string constant. */
+
+ Uid_t_f /* Uid_t in decimal */
+
+ printf("who = %"Uid_t_f"\n", who);
+
+Or you can try casting to a "wide enough" type:
+
+ printf("i = %"IVdf"\n", (IV)something_very_small_and_signed);
+
+Also remember that the C<%p> format really does require a void pointer:
+
+ U8* p = ...;
+ printf("p = %p\n", (void*)p);
+
+The gcc option C<-Wformat> scans for such problems.
+
+=item *
+
+Blindly using variadic macros
+
+gcc has had them for a while with its own syntax, and C99 brought
+them with a standardized syntax. Don't use the former, and use
+the latter only if the HAS_C99_VARIADIC_MACROS is defined.
+
+=item *
+
+Blindly passing va_list
+
+Not all platforms support passing va_list to further varargs (stdarg)
+functions. The right thing to do is to copy the va_list using the
+Perl_va_copy() if the NEED_VA_COPY is defined.
+
+=item *
+
+Using gcc statement expressions
+
+ val = ({...;...;...}); /* BAD */
+
+While a nice extension, it's not portable. The Perl code does
+admittedly use them if available to gain some extra speed
+(essentially as a funky form of inlining), but you shouldn't.
+
+=item *
+
+Binding together several statements
+
+Use the macros STMT_START and STMT_END.
+
+ STMT_START {
+ ...
+ } STMT_END
+
+=item *
+
+Testing for operating systems or versions when should be testing for features
+
+ #ifdef __FOONIX__ /* BAD */
+ foo = quux();
+ #endif
+
+Unless you know with 100% certainty that quux() is only ever available
+for the "Foonix" operating system B<and> that is available B<and>
+correctly working for B<all> past, present, B<and> future versions of
+"Foonix", the above is very wrong. This is more correct (though still
+not perfect, because the below is a compile-time check):
+
+ #ifdef HAS_QUUX
+ foo = quux();
+ #endif
+
+How does the HAS_QUUX become defined where it needs to be? Well, if
+Foonix happens to be UNIXy enought to be able to run the Configure
+script, and Configure has been taught about detecting and testing
+quux(), the HAS_QUUX will be correctly defined. In other platforms,
+the corresponding configuration step will hopefully do the same.
+
+In a pinch, if you cannot wait for Configure to be educated,
+or if you have a good hunch of where quux() might be available,
+you can temporarily try the following:
+
+ #if (defined(__FOONIX__) || defined(__BARNIX__))
+ # define HAS_QUUX
+ #endif
+
+ ...
+
+ #ifdef HAS_QUUX
+ foo = quux();
+ #endif
+
+But in any case, try to keep the features and operating systems separate.
+
+=back
+
+=head2 Problematic System Interfaces
+
+=over 4
+
+=item *
+
+malloc(0), realloc(0), calloc(0, 0) are nonportable. To be portable
+allocate at least one byte. (In general you should rarely need to
+work at this low level, but instead use the various malloc wrappers.)
+
+=item *
+
+snprintf() - the return type is unportable. Use my_snprintf() instead.
+
+=back
+
+=head2 Security problems
+
+Last but not least, here are various tips for safer coding.
+
+=over 4
+
+=item *
+
+Do not use gets()
+
+Or we will publicly ridicule you. Seriously.
+
+=item *
+
+Do not use strcpy() or strcat() or strncpy() or strncat()
+
+Use my_strlcpy() and my_strlcat() instead: they either use the native
+implementation, or Perl's own implementation (borrowed from the public
+domain implementation of INN).
+
+=item *
+
+Do not use sprintf() or vsprintf()
+
+If you really want just plain byte strings, use my_snprintf()
+and my_vsnprintf() instead, which will try to use snprintf() and
+vsnprintf() if those safer APIs are available. If you want something
+fancier than a plain byte string, use SVs and Perl_sv_catpvf().
+
=back
=head1 EXTERNAL TOOLS FOR DEBUGGING PERL
meant as a guide to interfacing these tools with Perl, not
as any kind of guide to the use of the tools themselves.
+B<NOTE 1>: Running under memory debuggers such as Purify, valgrind, or
+Third Degree greatly slows down the execution: seconds become minutes,
+minutes become hours. For example as of Perl 5.8.1, the
+ext/Encode/t/Unicode.t takes extraordinarily long to complete under
+e.g. Purify, Third Degree, and valgrind. Under valgrind it takes more
+than six hours, even on a snappy computer-- the said test must be
+doing something that is quite unfriendly for memory debuggers. If you
+don't feel like waiting, that you can simply kill away the perl
+process.
+
+B<NOTE 2>: To minimize the number of memory leak false alarms (see
+L</PERL_DESTRUCT_LEVEL> for more information), you have to have
+environment variable PERL_DESTRUCT_LEVEL set to 2. The F<TEST>
+and harness scripts do that automatically. But if you are running
+some of the tests manually-- for csh-like shells:
+
+ setenv PERL_DESTRUCT_LEVEL 2
+
+and for Bourne-type shells:
+
+ PERL_DESTRUCT_LEVEL=2
+ export PERL_DESTRUCT_LEVEL
+
+or in UNIXy environments you can also use the C<env> command:
+
+ env PERL_DESTRUCT_LEVEL=2 valgrind ./perl -Ilib ...
+
+B<NOTE 3>: There are known memory leaks when there are compile-time
+errors within eval or require, seeing C<S_doeval> in the call stack
+is a good sign of these. Fixing these leaks is non-trivial,
+unfortunately, but they must be fixed eventually.
+
+B<NOTE 4>: L<DynaLoader> will not clean up after itself completely
+unless Perl is built with the Configure option
+C<-Accflags=-DDL_UNLOAD_ALL_AT_EXIT>.
+
=head2 Rational Software's Purify
Purify is a commercial tool that is helpful in identifying
optimal testing with Purify. Purify is available under
Windows NT, Solaris, HP-UX, SGI, and Siemens Unix.
-The only currently known leaks happen when there are
-compile-time errors within eval or require. (Fixing these
-is non-trivial, unfortunately, but they must be fixed
-eventually.)
-
=head2 Purify on Unix
On Unix, Purify creates a new Perl binary. To get the most
Once you've compiled a perl suitable for Purify'ing, then you
can just:
- make pureperl
+ make pureperl
which creates a binary named 'pureperl' that has been Purify'ed.
This binary is used in place of the standard 'perl' binary
when you want to debug Perl memory problems.
-To minimize the number of memory leak false alarms
-(see L</PERL_DESTRUCT_LEVEL>), set environment variable
-PERL_DESTRUCT_LEVEL to 2.
-
- setenv PERL_DESTRUCT_LEVEL 2
-
-In Bourne-type shells:
-
- PERL_DESTRUCT_LEVEL=2
- export PERL_DESTRUCT_LEVEL
-
As an example, to show any memory leaks produced during the
standard Perl testset you would create and run the Purify'ed
perl as:
make pureperl
cd t
- ../pureperl -I../lib harness
+ ../pureperl -I../lib harness
which would run Perl on test.pl and report any memory problems.
You should add -DPURIFY to the DEFINES line so the DEFINES
line looks something like:
- DEFINES = -DWIN32 -D_CONSOLE -DNO_STRICT $(CRYPT_FLAG) -DPURIFY=1
+ DEFINES = -DWIN32 -D_CONSOLE -DNO_STRICT $(CRYPT_FLAG) -DPURIFY=1
to disable Perl's arena memory allocation functions, as
well as to force use of memory allocation functions derived
cd win32
make
cd ../t
- purify ../perl -I../lib harness
+ purify ../perl -I../lib harness
which would instrument Perl in memory, run Perl on test.pl,
then finally report any memory problems.
-B<NOTE>: as of Perl 5.8.0, the ext/Encode/t/Unicode.t takes
-extraordinarily long (hours?) to complete under Purify. It has been
-theorized that it would eventually finish, but nobody has so far been
-patient enough :-) (This same extreme slowdown has been seen also with
-the Third Degree tool, so the said test must be doing something that
-is quite unfriendly for memory debuggers.) It is suggested that you
-simply kill away that testing process.
+=head2 valgrind
+
+The excellent valgrind tool can be used to find out both memory leaks
+and illegal memory accesses. As of August 2003 it unfortunately works
+only on x86 (ELF) Linux. The special "test.valgrind" target can be used
+to run the tests under valgrind. Found errors and memory leaks are
+logged in files named F<testfile.valgrind>.
+
+Valgrind also provides a cachegrind tool, invoked on perl as:
+
+ VG_OPTS=--tool=cachegrind make test.valgrind
+
+As system libraries (most notably glibc) are also triggering errors,
+valgrind allows to suppress such errors using suppression files. The
+default suppression file that comes with valgrind already catches a lot
+of them. Some additional suppressions are defined in F<t/perl.supp>.
+
+To get valgrind and for more information see
+
+ http://developer.kde.org/~sewardj/
=head2 Compaq's/Digital's/HP's Third Degree
=head2 PERL_DESTRUCT_LEVEL
-If you want to run any of the tests yourself manually using the
-pureperl or perl.third executables, please note that by default
-perl B<does not> explicitly cleanup all the memory it has allocated
-(such as global memory arenas) but instead lets the exit() of
-the whole program "take care" of such allocations, also known
-as "global destruction of objects".
+If you want to run any of the tests yourself manually using e.g.
+valgrind, or the pureperl or perl.third executables, please note that
+by default perl B<does not> explicitly cleanup all the memory it has
+allocated (such as global memory arenas) but instead lets the exit()
+of the whole program "take care" of such allocations, also known as
+"global destruction of objects".
There is a way to tell perl to do complete cleanup: set the
environment variable PERL_DESTRUCT_LEVEL to a non-zero value.
(Note: the mod_perl apache module uses also this environment variable
for its own purposes and extended its semantics. Refer to the mod_perl
-documentation for more information.)
+documentation for more information. Also, spawned threads do the
+equivalent of setting this variable to the value 1.)
+
+If, at the end of a run you get the message I<N scalars leaked>, you can
+recompile with C<-DDEBUG_LEAKING_SCALARS>, which will cause the addresses
+of all those leaked SVs to be dumped along with details as to where each
+SV was originally allocated. This information is also displayed by
+Devel::Peek. Note that the extra details recorded with each SV increases
+memory usage, so it shouldn't be used in production environments. It also
+converts C<new_SV()> from a macro into a real function, so you can use
+your favourite debugger to discover where those pesky SVs were allocated.
+
+=head2 PERL_MEM_LOG
+
+If compiled with C<-DPERL_MEM_LOG>, all Newx() and Renew() allocations
+and Safefree() in the Perl core go through logging functions, which is
+handy for breakpoint setting. If also compiled with C<-DPERL_MEM_LOG_STDERR>,
+the allocations and frees are logged to STDERR (or more precisely, to the
+file descriptor 2) in these logging functions, with the calling source code
+file and line number (and C function name, if supported by the C compiler).
+
+This logging is somewhat similar to C<-Dm> but independent of C<-DDEBUGGING>,
+and at a higher level (the C<-Dm> is directly at the point of C<malloc()>,
+while the C<PERL_MEM_LOG> is at the level of C<New()>).
=head2 Profiling
For more detailed explanation of the available commands and output
formats, see your own local documentation of gprof.
+quick hint:
+
+ $ sh Configure -des -Dusedevel -Doptimize='-g' -Accflags='-pg' -Aldflags='-pg' && make
+ $ ./perl someprog # creates gmon.out in current directory
+ $ gprof perl > out
+ $ view out
+
=head2 GCC gcov Profiling
Starting from GCC 3.0 I<basic block profiling> is officially available
http://gcc.gnu.org/onlinedocs/gcc-3.0/gcc_8.html#SEC132
+quick hint:
+
+ $ sh Configure -des -Doptimize='-g' -Accflags='-fprofile-arcs -ftest-coverage' \
+ -Aldflags='-fprofile-arcs -ftest-coverage' && make perl.gcov
+ $ rm -f regexec.c.gcov regexec.gcda
+ $ ./perl.gcov
+ $ gcov regexec.c
+ $ view regexec.c.gcov
+
=head2 Pixie Profiling
Pixie is a profiling tool available on IRIX and Tru64 (aka Digital
=item *
-If you see in a debugger a memory area mysteriously full of 0xabababab,
-you may be seeing the effect of the Poison() macro, see L<perlclib>.
+If you see in a debugger a memory area mysteriously full of 0xABABABAB
+or 0xEFEFEFEF, you may be seeing the effect of the Poison() macros,
+see L<perlclib>.
+
+=item *
+
+Under ithreads the optree is read only. If you want to enforce this, to check
+for write accesses from buggy code, compile with C<-DPL_OP_SLAB_ALLOC> to
+enable the OP slab allocator and C<-DPERL_DEBUG_READONLY_OPS> to enable code
+that allocates op memory via C<mmap>, and sets it read-only at run time.
+Any write access to an op results in a C<SIGBUS> and abort.
+
+This code is intended for development only, and may not be portable even to
+all Unix variants. Also, it is an 80% solution, in that it isn't able to make
+all ops read only. Specifically it
+
+=over
+
+=item 1
+
+Only sets read-only on all slabs of ops at C<CHECK> time, hence ops allocated
+later via C<require> or C<eval> will be re-write
+
+=item 2
+
+Turns an entire slab of ops read-write if the refcount of any op in the slab
+needs to be decreased.
+
+=item 3
+
+Turns an entire slab of ops read-write if any op from the slab is freed.
=back
-=head2 CONCLUSION
+It's not possible to turn the slabs to read-only after an action requiring
+read-write access, as either can happen during op tree building time, so
+there may still be legitimate write access.
+
+However, as an 80% solution it is still effective, as currently it catches
+a write access during the generation of F<Config.pm>, which means that we
+can't yet build F<perl> with this enabled.
+
+=back
-We've had a brief look around the Perl source, an overview of the stages
-F<perl> goes through when it's running your code, and how to use a
-debugger to poke at the Perl guts. We took a very simple problem and
-demonstrated how to solve it fully - with documentation, regression
-tests, and finally a patch for submission to p5p. Finally, we talked
-about how to use external tools to debug and test Perl.
+
+=head1 CONCLUSION
+
+We've had a brief look around the Perl source, how to maintain quality
+of the source code, an overview of the stages F<perl> goes through
+when it's running your code, how to use debuggers to poke at the Perl
+guts, and finally how to analyse the execution of Perl. We took a very
+simple problem and demonstrated how to solve it fully - with
+documentation, regression tests, and finally a patch for submission to
+p5p. Finally, we talked about how to use external tools to debug and
+test Perl.
I'd now suggest you read over those references again, and then, as soon
as possible, get your hands dirty. The best way to learn is by doing,
-so:
+so:
=over 3
=back
-If you can do these things, you've started on the long road to Perl porting.
+If you can do these things, you've started on the long road to Perl porting.
Thanks for wanting to help make Perl better - and happy hacking!
=head1 AUTHOR
This document was written by Nathan Torkington, and is maintained by
the perl5-porters mailing list.
-
https://perl5.git.perl.org / perl5.git / blobdiff