=head2 Keeping in sync
The source code to the Perl interpreter, in its different versions, is
-kept in a repository managed by a revision control system ( which is
-currently the Perforce program, see http://perforce.com/ ). The
-pumpkings and a few others have access to the repository to check in
-changes. Periodically the pumpking for the development version of Perl
-will release a new version, so the rest of the porters can see what's
-changed. The current state of the main trunk of repository, and patches
-that describe the individual changes that have happened since the last
-public release are available at this location:
+kept in a repository managed by the git revision control system. The
+pumpkings and a few others have write access to the repository to check in
+changes.
- http://public.activestate.com/pub/apc/
- ftp://public.activestate.com/pub/apc/
+How to clone and use the git perl repository is described in L<perlrepository>.
-If you're looking for a particular change, or a change that affected
-a particular set of files, you may find the B<Perl Repository Browser>
-useful:
+You can also choose to use rsync to get a copy of the current source tree
+for the bleadperl branch and all maintenance branches :
- http://public.activestate.com/cgi-bin/perlbrowse
+ $ rsync -avz rsync://perl5.git.perl.org/APC/perl-current .
+ $ rsync -avz rsync://perl5.git.perl.org/APC/perl-5.10.x .
+ $ rsync -avz rsync://perl5.git.perl.org/APC/perl-5.8.x .
+ $ rsync -avz rsync://perl5.git.perl.org/APC/perl-5.6.x .
+ $ rsync -avz rsync://perl5.git.perl.org/APC/perl-5.005xx .
+
+(Add the C<--delete> option to remove leftover files)
You may also want to subscribe to the perl5-changes mailing list to
receive a copy of each patch that gets submitted to the maintenance
state of evolution. You should expect it to be very buggy. Do B<not> use
it for any purpose other than testing and development.
-Keeping in sync with the most recent branch can be done in several ways,
-but the most convenient and reliable way is using B<rsync>, available at
-ftp://rsync.samba.org/pub/rsync/ . (You can also get the most recent
-branch by FTP.)
-
-If you choose to keep in sync using rsync, there are two approaches
-to doing so:
-
-=over 4
-
-=item rsync'ing the source tree
-
-Presuming you are in the directory where your perl source resides
-and you have rsync installed and available, you can "upgrade" to
-the bleadperl using:
-
- # rsync -avz rsync://public.activestate.com/perl-current/ .
-
-This takes care of updating every single item in the source tree to
-the latest applied patch level, creating files that are new (to your
-distribution) and setting date/time stamps of existing files to
-reflect the bleadperl status.
-
-Note that this will not delete any files that were in '.' before
-the rsync. Once you are sure that the rsync is running correctly,
-run it with the --delete and the --dry-run options like this:
-
- # rsync -avz --delete --dry-run rsync://public.activestate.com/perl-current/ .
-
-This will I<simulate> an rsync run that also deletes files not
-present in the bleadperl master copy. Observe the results from
-this run closely. If you are sure that the actual run would delete
-no files precious to you, you could remove the '--dry-run' option.
-
-You can than check what patch was the latest that was applied by
-looking in the file B<.patch>, which will show the number of the
-latest patch.
-
-If you have more than one machine to keep in sync, and not all of
-them have access to the WAN (so you are not able to rsync all the
-source trees to the real source), there are some ways to get around
-this problem.
-
-=over 4
-
-=item Using rsync over the LAN
-
-Set up a local rsync server which makes the rsynced source tree
-available to the LAN and sync the other machines against this
-directory.
-
-From http://rsync.samba.org/README.html :
-
- "Rsync uses rsh or ssh for communication. It does not need to be
- setuid and requires no special privileges for installation. It
- does not require an inetd entry or a daemon. You must, however,
- have a working rsh or ssh system. Using ssh is recommended for
- its security features."
-
-=item Using pushing over the NFS
-
-Having the other systems mounted over the NFS, you can take an
-active pushing approach by checking the just updated tree against
-the other not-yet synced trees. An example would be
-
- #!/usr/bin/perl -w
-
- use strict;
- use File::Copy;
-
- my %MF = map {
- m/(\S+)/;
- $1 => [ (stat $1)[2, 7, 9] ]; # mode, size, mtime
- } `cat MANIFEST`;
-
- my %remote = map { $_ => "/$_/pro/3gl/CPAN/perl-5.7.1" } qw(host1 host2);
-
- foreach my $host (keys %remote) {
- unless (-d $remote{$host}) {
- print STDERR "Cannot Xsync for host $host\n";
- next;
- }
- foreach my $file (keys %MF) {
- my $rfile = "$remote{$host}/$file";
- my ($mode, $size, $mtime) = (stat $rfile)[2, 7, 9];
- defined $size or ($mode, $size, $mtime) = (0, 0, 0);
- $size == $MF{$file}[1] && $mtime == $MF{$file}[2] and next;
- printf "%4s %-34s %8d %9d %8d %9d\n",
- $host, $file, $MF{$file}[1], $MF{$file}[2], $size, $mtime;
- unlink $rfile;
- copy ($file, $rfile);
- utime time, $MF{$file}[2], $rfile;
- chmod $MF{$file}[0], $rfile;
- }
- }
-
-though this is not perfect. It could be improved with checking
-file checksums before updating. Not all NFS systems support
-reliable utime support (when used over the NFS).
-
-=back
-
-=item rsync'ing the patches
-
-The source tree is maintained by the pumpking who applies patches to
-the files in the tree. These patches are either created by the
-pumpking himself using C<diff -c> after updating the file manually or
-by applying patches sent in by posters on the perl5-porters list.
-These patches are also saved and rsync'able, so you can apply them
-yourself to the source files.
-
-Presuming you are in a directory where your patches reside, you can
-get them in sync with
-
- # rsync -avz rsync://public.activestate.com/perl-current-diffs/ .
-
-This makes sure the latest available patch is downloaded to your
-patch directory.
-
-It's then up to you to apply these patches, using something like
-
- # last="`cat ../perl-current/.patch`.gz"
- # rsync -avz rsync://public.activestate.com/perl-current-diffs/ .
- # find . -name '*.gz' -newer $last -exec gzcat {} \; >blead.patch
- # cd ../perl-current
- # patch -p1 -N <../perl-current-diffs/blead.patch
-
-or, since this is only a hint towards how it works, use CPAN-patchaperl
-from Andreas König to have better control over the patching process.
-
-=back
-
-=head2 Why rsync the source tree
-
-=over 4
-
-=item It's easier to rsync the source tree
-
-Since you don't have to apply the patches yourself, you are sure all
-files in the source tree are in the right state.
-
-=item It's more reliable
-
-While both the rsync-able source and patch areas are automatically
-updated every few minutes, keep in mind that applying patches may
-sometimes mean careful hand-holding, especially if your version of
-the C<patch> program does not understand how to deal with new files,
-files with 8-bit characters, or files without trailing newlines.
-
-=back
-
-=head2 Why rsync the patches
-
-=over 4
-
-=item It's easier to rsync the patches
-
-If you have more than one machine that you want to keep in track with
-bleadperl, it's easier to rsync the patches only once and then apply
-them to all the source trees on the different machines.
-
-In case you try to keep in pace on 5 different machines, for which
-only one of them has access to the WAN, rsync'ing all the source
-trees should than be done 5 times over the NFS. Having
-rsync'ed the patches only once, I can apply them to all the source
-trees automatically. Need you say more ;-)
-
-=item It's a good reference
-
-If you do not only like to have the most recent development branch,
-but also like to B<fix> bugs, or extend features, you want to dive
-into the sources. If you are a seasoned perl core diver, you don't
-need no manuals, tips, roadmaps, perlguts.pod or other aids to find
-your way around. But if you are a starter, the patches may help you
-in finding where you should start and how to change the bits that
-bug you.
-
-The file B<Changes> is updated on occasions the pumpking sees as his
-own little sync points. On those occasions, he releases a tar-ball of
-the current source tree (i.e. perl@7582.tar.gz), which will be an
-excellent point to start with when choosing to use the 'rsync the
-patches' scheme. Starting with perl@7582, which means a set of source
-files on which the latest applied patch is number 7582, you apply all
-succeeding patches available from then on (7583, 7584, ...).
-
-You can use the patches later as a kind of search archive.
-
-=over 4
-
-=item Finding a start point
-
-If you want to fix/change the behaviour of function/feature Foo, just
-scan the patches for patches that mention Foo either in the subject,
-the comments, or the body of the fix. A good chance the patch shows
-you the files that are affected by that patch which are very likely
-to be the starting point of your journey into the guts of perl.
-
-=item Finding how to fix a bug
-
-If you've found I<where> the function/feature Foo misbehaves, but you
-don't know how to fix it (but you do know the change you want to
-make), you can, again, peruse the patches for similar changes and
-look how others apply the fix.
-
-=item Finding the source of misbehaviour
-
-When you keep in sync with bleadperl, the pumpking would love to
-I<see> that the community efforts really work. So after each of his
-sync points, you are to 'make test' to check if everything is still
-in working order. If it is, you do 'make ok', which will send an OK
-report to I<perlbug@perl.org>. (If you do not have access to a mailer
-from the system you just finished successfully 'make test', you can
-do 'make okfile', which creates the file C<perl.ok>, which you can
-than take to your favourite mailer and mail yourself).
-
-But of course, as always, things will not always lead to a success
-path, and one or more test do not pass the 'make test'. Before
-sending in a bug report (using 'make nok' or 'make nokfile'), check
-the mailing list if someone else has reported the bug already and if
-so, confirm it by replying to that message. If not, you might want to
-trace the source of that misbehaviour B<before> sending in the bug,
-which will help all the other porters in finding the solution.
-
-Here the saved patches come in very handy. You can check the list of
-patches to see which patch changed what file and what change caused
-the misbehaviour. If you note that in the bug report, it saves the
-one trying to solve it, looking for that point.
-
-=back
-
-If searching the patches is too bothersome, you might consider using
-perl's bugtron to find more information about discussions and
-ramblings on posted bugs.
-
-If you want to get the best of both worlds, rsync both the source
-tree for convenience, reliability and ease and rsync the patches
-for reference.
-
-=back
-
-=head2 Working with the source
-
-Because you cannot use the Perforce client, you cannot easily generate
-diffs against the repository, nor will merges occur when you update
-via rsync. If you edit a file locally and then rsync against the
-latest source, changes made in the remote copy will I<overwrite> your
-local versions!
-
-The best way to deal with this is to maintain a tree of symlinks to
-the rsync'd source. Then, when you want to edit a file, you remove
-the symlink, copy the real file into the other tree, and edit it. You
-can then diff your edited file against the original to generate a
-patch, and you can safely update the original tree.
-
-Perl's F<Configure> script can generate this tree of symlinks for you.
-The following example assumes that you have used rsync to pull a copy
-of the Perl source into the F<perl-rsync> directory. In the directory
-above that one, you can execute the following commands:
-
- mkdir perl-dev
- cd perl-dev
- ../perl-rsync/Configure -Dmksymlinks -Dusedevel -D"optimize=-g"
-
-This will start the Perl configuration process. After a few prompts,
-you should see something like this:
-
- Symbolic links are supported.
-
- Checking how to test for symbolic links...
- Your builtin 'test -h' may be broken.
- Trying external '/usr/bin/test -h'.
- You can test for symbolic links with '/usr/bin/test -h'.
-
- Creating the symbolic links...
- (First creating the subdirectories...)
- (Then creating the symlinks...)
-
-The specifics may vary based on your operating system, of course.
-After you see this, you can abort the F<Configure> script, and you
-will see that the directory you are in has a tree of symlinks to the
-F<perl-rsync> directories and files.
-
-If you plan to do a lot of work with the Perl source, here are some
-Bourne shell script functions that can make your life easier:
-
- function edit {
- if [ -L $1 ]; then
- mv $1 $1.orig
- cp $1.orig $1
- vi $1
- else
- vi $1
- fi
- }
-
- function unedit {
- if [ -L $1.orig ]; then
- rm $1
- mv $1.orig $1
- fi
- }
-
-Replace "vi" with your favorite flavor of editor.
-
-Here is another function which will quickly generate a patch for the
-files which have been edited in your symlink tree:
-
- mkpatchorig() {
- local diffopts
- for f in `find . -name '*.orig' | sed s,^\./,,`
- do
- case `echo $f | sed 's,.orig$,,;s,.*\.,,'` in
- c) diffopts=-p ;;
- pod) diffopts='-F^=' ;;
- *) diffopts= ;;
- esac
- diff -du $diffopts $f `echo $f | sed 's,.orig$,,'`
- done
- }
-
-This function produces patches which include enough context to make
-your changes obvious. This makes it easier for the Perl pumpking(s)
-to review them when you send them to the perl5-porters list, and that
-means they're more likely to get applied.
-
-This function assumed a GNU diff, and may require some tweaking for
-other diff variants.
-
=head2 Perlbug administration
There is a single remote administrative interface for modifying bug status,
patching a core module and there's an author listed, send the author a
copy (see L<Patching a core module>). This lets other porters review
your patch, which catches a surprising number of errors in patches.
-Either use the diff program (available in source code form from
-ftp://ftp.gnu.org/pub/gnu/ , or use Johan Vromans' I<makepatch>
-(available from I<CPAN/authors/id/JV/>). Unified diffs are preferred,
-but context diffs are accepted. Do not send RCS-style diffs or diffs
-without context lines. More information is given in the
-I<Porting/patching.pod> file in the Perl source distribution. Please
-patch against the latest B<development> version. (e.g., even if you're
-fixing a bug in the 5.8 track, patch against the latest B<development>
-version rsynced from rsync://public.activestate.com/perl-current/ )
+Please patch against the latest B<development> version. (e.g., even if
+you're fixing a bug in the 5.8 track, patch against the C<blead> branch in
+the git repository.)
If changes are accepted, they are applied to the development branch. Then
-the 5.8 pumpking decides which of those patches is to be backported to the
-maint branch. Only patches that survive the heat of the development
-branch get applied to maintenance versions.
+the maintenance pumpking decides which of those patches is to be
+backported to the maint branch. Only patches that survive the heat of the
+development branch get applied to maintenance versions.
Your patch should update the documentation and test suite. See
L<Writing a test>. If you have added or removed files in the distribution,
resembles the code found in L<perlembed>; most of the real action takes
place in F<perl.c>
+F<perlmain.c> is generated by L<writemain> from F<miniperlmain.c> at
+make time, so you should make perl to follow this along.
+
First, F<perlmain.c> allocates some memory and constructs a Perl
-interpreter:
+interpreter, along these lines:
1 PERL_SYS_INIT3(&argc,&argv,&env);
2
own C<malloc> as defined in F<malloc.c> if you selected that option at
configure time.
-Next, in line 7, we construct the interpreter; this sets up all the
-special variables that Perl needs, the stacks, and so on.
+Next, in line 7, we construct the interpreter using perl_construct,
+also in F<perl.c>; this sets up all the special variables that Perl
+needs, the stacks, and so on.
Now we pass Perl the command line options, and tell it to go:
exitstatus = perl_parse(my_perl, xs_init, argc, argv, (char **)NULL);
- if (!exitstatus) {
- exitstatus = perl_run(my_perl);
- }
+ if (!exitstatus)
+ perl_run(my_perl);
+ exitstatus = perl_destruct(my_perl);
+
+ perl_free(my_perl);
C<perl_parse> is actually a wrapper around C<S_parse_body>, as defined
in F<perl.c>, which processes the command line options, sets up any
=item Exception handing
-Perl's exception handing (i.e. C<die> etc) is built on top of the low-level
+Perl's exception handing (i.e. C<die> etc.) is built on top of the low-level
C<setjmp()>/C<longjmp()> C-library functions. These basically provide a
way to capture the current PC and SP registers and later restore them; i.e.
a C<longjmp()> continues at the point in code where a previous C<setjmp()>
=back
The following flags would be nice to have but they would first need
-their own Stygian stablemaster:
+their own Augean stablemaster:
=over 4
here. There are still some bits and pieces hanging around in here
that need to be moved. Perhaps you could move them? Thanks!
+=item F<t/mro/>
+
+Tests for perl's method resolution order implementations
+(see L<mro>).
+
=item F<t/op/>
Tests for perl's built in functions that don't fit into any of the
=item minitest
Run F<miniperl> on F<t/base>, F<t/comp>, F<t/cmd>, F<t/run>, F<t/io>,
-F<t/op>, and F<t/uni> tests.
+F<t/op>, F<t/uni> and F<t/mro> tests.
=item test.valgrind check.valgrind utest.valgrind ucheck.valgrind
=item -v
Run the tests under verbose mode so you can see what tests were run,
-and debug outbut.
+and debug output.
=item -torture
=item *
+Assuming the character set is ASCIIish
+
+Perl can compile and run under EBCDIC platforms. See L<perlebcdic>.
+This is transparent for the most part, but because the character sets
+differ, you shouldn't use numeric (decimal, octal, nor hex) constants
+to refer to characters. You can safely say 'A', but not 0x41.
+You can safely say '\n', but not \012.
+If a character doesn't have a trivial input form, you can
+create a #define for it in both C<utfebcdic.h> and C<utf8.h>, so that
+it resolves to different values depending on the character set being used.
+(There are three different EBCDIC character sets defined in C<utfebcdic.h>,
+so it might be best to insert the #define three times in that file.)
+
+Also, the range 'A' - 'Z' in ASCII is an unbroken sequence of 26 upper case
+alphabetic characters. That is not true in EBCDIC. Nor for 'a' to 'z'.
+But '0' - '9' is an unbroken range in both systems. Don't assume anything
+about other ranges.
+
+Many of the comments in the existing code ignore the possibility of EBCDIC,
+and may be wrong therefore, even if the code works.
+This is actually a tribute to the successful transparent insertion of being
+able to handle EBCDIC without having to change pre-existing code.
+
+UTF-8 and UTF-EBCDIC are two different encodings used to represent Unicode
+code points as sequences of bytes. Macros
+with the same names (but different definitions)
+in C<utf8.h> and C<utfebcdic.h>
+are used to allow the calling code to think that there is only one such
+encoding.
+This is almost always referred to as C<utf8>, but it means the EBCDIC version
+as well. Again, comments in the code may well be wrong even if the code itself
+is right.
+For example, the concept of C<invariant characters> differs between ASCII and
+EBCDIC.
+On ASCII platforms, only characters that do not have the high-order
+bit set (i.e. whose ordinals are strict ASCII, 0 - 127)
+are invariant, and the documentation and comments in the code
+may assume that,
+often referring to something like, say, C<hibit>.
+The situation differs and is not so simple on EBCDIC machines, but as long as
+the code itself uses the C<NATIVE_IS_INVARIANT()> macro appropriately, it
+works, even if the comments are wrong.
+
+=item *
+
+Assuming the character set is just ASCII
+
+ASCII is a 7 bit encoding, but bytes have 8 bits in them. The 128 extra
+characters have different meanings depending on the locale. Absent a locale,
+currently these extra characters are generally considered to be unassigned,
+and this has presented some problems.
+This is scheduled to be changed in 5.12 so that these characters will
+be considered to be Latin-1 (ISO-8859-1).
+
+=item *
+
Mixing #define and #ifdef
#define BURGLE(x) ... \
=item *
-Adding stuff after #endif or #else
+Adding non-comment stuff after #endif or #else
#ifdef SNOSH
...
=item *
-Binding together several statements
+Binding together several statements in a macro
Use the macros STMT_START and STMT_END.
=head2 valgrind
The excellent valgrind tool can be used to find out both memory leaks
-and illegal memory accesses. As of August 2003 it unfortunately works
-only on x86 (ELF) Linux. The special "test.valgrind" target can be used
-to run the tests under valgrind. Found errors and memory leaks are
-logged in files named F<testfile.valgrind>.
+and illegal memory accesses. As of version 3.3.0, Valgrind only
+supports Linux on x86, x86-64 and PowerPC. The special "test.valgrind"
+target can be used to run the tests under valgrind. Found errors
+and memory leaks are logged in files named F<testfile.valgrind>.
Valgrind also provides a cachegrind tool, invoked on perl as:
converts C<new_SV()> from a macro into a real function, so you can use
your favourite debugger to discover where those pesky SVs were allocated.
+If you see that you're leaking memory at runtime, but neither valgrind
+nor C<-DDEBUG_LEAKING_SCALARS> will find anything, you're probably
+leaking SVs that are still reachable and will be properly cleaned up
+during destruction of the interpreter. In such cases, using the C<-Dm>
+switch can point you to the source of the leak. If the executable was
+built with C<-DDEBUG_LEAKING_SCALARS>, C<-Dm> will output SV allocations
+in addition to memory allocations. Each SV allocation has a distinct
+serial number that will be written on creation and destruction of the SV.
+So if you're executing the leaking code in a loop, you need to look for
+SVs that are created, but never destroyed between each cycle. If such an
+SV is found, set a conditional breakpoint within C<new_SV()> and make it
+break only when C<PL_sv_serial> is equal to the serial number of the
+leaking SV. Then you will catch the interpreter in exactly the state
+where the leaking SV is allocated, which is sufficient in many cases to
+find the source of the leak.
+
+As C<-Dm> is using the PerlIO layer for output, it will by itself
+allocate quite a bunch of SVs, which are hidden to avoid recursion.
+You can bypass the PerlIO layer if you use the SV logging provided
+by C<-DPERL_MEM_LOG> instead.
+
=head2 PERL_MEM_LOG
If compiled with C<-DPERL_MEM_LOG>, all Newx() and Renew() allocations
and at a higher level (the C<-Dm> is directly at the point of C<malloc()>,
while the C<PERL_MEM_LOG> is at the level of C<New()>).
+In addition to memory allocations, SV allocations will be logged, just as
+with C<-Dm>. However, since the logging doesn't use PerlIO, all SV allocations
+are logged and no extra SV allocations are introduced by enabling the logging.
+If compiled with C<-DDEBUG_LEAKING_SCALARS>, the serial number for each SV
+allocation is also logged.
+
+You can control the logging from your environment if you compile with
+C<-DPERL_MEM_LOG_ENV>. Then you need to explicitly set C<PERL_MEM_LOG> and/or
+C<PERL_SV_LOG> to a non-zero value to enable logging of memory and/or SV
+allocations.
+
=head2 Profiling
Depending on your platform there are various of profiling Perl.
If you can do these things, you've started on the long road to Perl porting.
Thanks for wanting to help make Perl better - and happy hacking!
+=head2 Metaphoric Quotations
+
+If you recognized the quote about the Road above, you're in luck.
+
+Most software projects begin each file with a literal description of each
+file's purpose. Perl instead begins each with a literary allusion to that
+file's purpose.
+
+Like chapters in many books, all top-level Perl source files (along with a
+few others here and there) begin with an epigramic inscription that alludes,
+indirectly and metaphorically, to the material you're about to read.
+
+Quotations are taken from writings of J.R.R Tolkien pertaining to his
+Legendarium, almost always from I<The Lord of the Rings>. Chapters and
+page numbers are given using the following editions:
+
+=over 4
+
+=item *
+
+I<The Hobbit>, by J.R.R. Tolkien. The hardcover, 70th-anniversary
+edition of 2007 was used, published in the UK by Harper Collins Publishers
+and in the US by the Houghton Mifflin Company.
+
+=item *
+
+I<The Lord of the Rings>, by J.R.R. Tolkien. The hardcover,
+50th-anniversary edition of 2004 was used, published in the UK by Harper
+Collins Publishers and in the US by the Houghton Mifflin Company.
+
+=item *
+
+I<The Lays of Beleriand>, by J.R.R. Tolkien and published posthumously by his
+son and literary executor, C.J.R. Tolkien, being the 3rd of the 12 volumes
+in Christopher's mammoth I<History of Middle Earth>. Page numbers derive
+from the hardcover edition, first published in 1983 by George Allen &
+Unwin; no page numbers changed for the special 3-volume omnibus edition of
+2002 or the various trade-paper editions, all again now by Harper Collins
+or Houghton Mifflin.
+
+=back
+
+Other JRRT books fair game for quotes would thus include I<The Adventures of
+Tom Bombadil>, I<The Silmarillion>, I<Unfinished Tales>, and I<The Tale of
+the Children of Hurin>, all but the first posthumously assembled by CJRT.
+But I<The Lord of the Rings> itself is perfectly fine and probably best to
+quote from, provided you can find a suitable quote there.
+
+So if you were to supply a new, complete, top-level source file to add to
+Perl, you should conform to this peculiar practice by yourself selecting an
+appropriate quotation from Tolkien, retaining the original spelling and
+punctuation and using the same format the rest of the quotes are in.
+Indirect and oblique is just fine; remember, it's a metaphor, so being meta
+is, after all, what it's for.
+
=head1 AUTHOR
This document was written by Nathan Torkington, and is maintained by
the perl5-porters mailing list.
+
+=head1 SEE ALSO
+
+L<perlrepository>
https://perl5.git.perl.org / perl5.git / blobdiff