Porting/sync-with-cpan: handle unwritable files in tarballs

[perl5.git] / Porting / release_managers_guide.pod

=encoding utf8

=head1 NAME

release_managers_guide - Releasing a new version of perl 5.x

Note that things change at each release, so there may be new things not
covered here, or tools may need updating.

=head1 MAKING A CHECKLIST

If you are preparing to do a release, you can run the
F<Porting/make-rmg-checklist> script to generate a new version of this
document that starts with a checklist for your release.

This script is run as:

    perl Porting/make-rmg-checklist \
        --type [BLEAD-POINT or MAINT or ...] > /tmp/rmg.pod

You can also pass the C<--html> flag to generate an HTML document instead of
POD.

    perl Porting/make-rmg-checklist --html \
        --type [BLEAD-POINT or MAINT or ...] > /tmp/rmg.html

=head1 SYNOPSIS

This document describes the series of tasks required - some automatic, some
manual - to produce a perl release of some description, be that a release
candidate, or final, numbered release of maint or blead.

The release process has traditionally been executed by the current
pumpking. Blead releases from 5.11.0 forward are made each month on the
20th by a non-pumpking release engineer.  The release engineer roster
and schedule can be found in Porting/release_schedule.pod.

This document both helps as a check-list for the release engineer
and is a base for ideas on how the various tasks could be automated
or distributed.

The checklist of a typical release cycle is as follows:

    (5.10.1 is released, and post-release actions have been done)

    ...time passes...

    a few weeks before the release, a number of steps are performed,
	including bumping the version to 5.10.2

    ...a few weeks pass...

    perl-5.10.2-RC1 is released

    perl-5.10.2 is released

    post-release actions are performed, including creating new
	perldelta.pod

    ... the cycle continues ...

=head1 DETAILS

Some of the tasks described below apply to all four types of
release of Perl. (blead, RC, final release of maint, final
release of blead). Some of these tasks apply only to a subset
of these release types.  If a step does not apply to a given
type of release, you will see a notation to that effect at
the beginning of the step.

=head2 Release types

=over 4

=item Release Candidate (RC)

A release candidate is an attempt to produce a tarball that is a close as
possible to the final release. Indeed, unless critical faults are found
during the RC testing, the final release will be identical to the RC
barring a few minor fixups (updating the release date in F<perlhist.pod>,
removing the RC status from F<patchlevel.h>, etc). If faults are found,
then the fixes should be put into a new release candidate, never directly
into a final release.


=item Stable/Maint release (MAINT).

A release with an even version number, and subversion number > 0, such as
5.14.1 or 5.14.2.

At this point you should have a working release candidate with few or no
changes since.

It's essentially the same procedure as for making a release candidate, but
with a whole bunch of extra post-release steps.

Note that for a maint release there are two versions of this guide to
consider: the one in the maint branch, and the one in blead. Which one to
use is a fine judgement. The blead one will be most up-to-date, while
it might describe some steps or new tools that aren't applicable to older
maint branches. It is probably best to review both versions of this
document, but to most closely follow the steps in the maint version.

=item A blead point release (BLEAD-POINT)

A release with an odd version number, such as 5.15.0 or 5.15.1.

This isn't for production, so it has less stability requirements than for
other release types, and isn't preceded by RC releases. Other than that,
it is similar to a MAINT release.

=item Blead final release (BLEAD-FINAL)

A release with an even version number, and subversion number == 0, such as
5.14.0. That is to say, it's the big new release once per year.

It's essentially the same procedure as for making a release candidate, but
with a whole bunch of extra post-release steps, even more than for MAINT.

=back

=for checklist begin

=head2 Prerequisites

Before you can make an official release of perl, there are a few
hoops you need to jump through:

=head3 PAUSE account with pumpkin status

Make sure you have a PAUSE account suitable for uploading a perl release.
If you don't have a PAUSE account, then request one:

    https://pause.perl.org/pause/query?ACTION=request_id

Check that your account is allowed to upload perl distros: go to
L<https://pause.perl.org/pause/authenquery?ACTION=who_pumpkin> and check that
your PAUSE ID is listed there.  If not, ask Andreas KE<0xf6>nig to add your ID
to the list of people allowed to upload something called perl.  You can find
Andreas' email address at:

    https://pause.perl.org/pause/query?ACTION=pause_04imprint

=head3 rt.perl.org update access

Make sure you have permission to close tickets on L<http://rt.perl.org/>
so you can respond to bug report as necessary during your stint.  If you
don't, make an account (if you don't have one) and contact the pumpking
with your username to get ticket-closing permission.

=head3 git checkout and commit bit

You will need a working C<git> installation, checkout of the perl
git repository and perl commit bit.  For information about working
with perl and git, see F<pod/perlgit.pod>.

If you are not yet a perl committer, you won't be able to make a
release.  Have a chat with whichever evil perl porter tried to talk
you into the idea in the first place to figure out the best way to
resolve the issue.

=head3 web-based file share

You will need to be able to share tarballs with #p5p members for
pre-release testing, and you may wish to upload to PAUSE via URL.
Make sure you have a way of sharing files, such as a web server or
file-sharing service.

Porters have access to the "dromedary" server (users.perl5.git.perl.org),
which has a F<public_html> directory to share files with.
(L<http://users.perl5.git.perl.org/~username/perl-5.xx.y.tar.gz>)

If you use Dropbox, you can append "raw=1" as a parameter to their usual
sharing link to allow direct download (albeit with redirects).

=head3 git clone of https://github.com/perlorg/perlweb

For updating the L<http://dev.perl.org> web pages, either a Github account or
sweet-talking somebody with a Github account into obedience is needed. This
is only needed on the day of the release or shortly afterwards.

=head3 Quotation for release announcement epigraph

You will need a quotation to use as an epigraph to your release announcement.

=head3 Install the previous version of perl

During the testing phase of the release you have created, you will be
asked to compare the installed files with a previous install. Save yourself
some time on release day, and have a (clean) install of the previous
version ready.

=head2 Building a release - advance actions

The work of building a release candidate for an even numbered release
(BLEAD-FINAL) of perl generally starts several weeks before the first
release candidate.  Some of the following steps should be done regularly,
but all I<must> be done in the run up to a release.

=head3 dual-life CPAN module synchronisation

To see which core distro versions differ from the current CPAN versions:

    $ ./perl -Ilib Porting/core-cpan-diff -x -a

However, this only checks whether the version recorded in
F<Porting/Maintainers.pl> differs from the latest on CPAN.  It doesn't tell you
if the code itself has diverged from CPAN.

You can also run an actual diff of the contents of the modules, comparing core
to CPAN, to ensure that there were no erroneous/extraneous changes that need to
be dealt with. You do this by not passing the C<-x> option:

    $ ./perl -Ilib Porting/core-cpan-diff -a -o /tmp/corediffs

Passing C<-u cpan> will probably be helpful, since it limits the search to
distributions with 'cpan' upstream source.  (It's OK for blead upstream to
differ from CPAN because those dual-life releases usually come I<after> perl
is released.)

See also the C<-d> and C<-v> options for more detail (and the C<-u> option as
mentioned above).  You'll probably want to use the C<-c cachedir> option to
avoid repeated CPAN downloads and may want to use C<-m file:///mirror/path> if
you made a local CPAN mirror. Note that a minicpan mirror won't actually work,
but can provide a good first pass to quickly get a list of modules which
definitely haven't changed, to avoid having to download absolutely everything.

For a BLEAD-POINT or BLEAD-FINAL release with 'cpan' upstream, if a CPAN
release appears to be ahead of blead, then consider updating it (or asking the
relevant porter to do so). (However, if this is a BLEAD-FINAL release or one of
the last BLEAD-POINT releases before it and hence blead is in some kind of
"code freeze" state (e.g. the sequence might be "contentious changes freeze",
then "user-visible changes freeze" and finally "full code freeze") then any
CPAN module updates must be subject to the same restrictions, so it may not be
possible to update all modules until after the BLEAD-FINAL release.) If blead
contains edits to a 'cpan' upstream module, this is naughty but sometimes
unavoidable to keep blead tests passing. Make sure the affected file has a
CUSTOMIZED entry in F<Porting/Maintainers.pl>.

If you are making a MAINT release, run C<core-cpan-diff> on both blead and
maint, then diff the two outputs. Compare this with what you expect, and if
necessary, fix things up. For example, you might think that both blead
and maint are synchronised with a particular CPAN module, but one might
have some extra changes.

=head3 How to sync a CPAN module with a cpanE<sol> distro

=over 4

=item *

Fetch the most recent version from CPAN.

=item *

Unpack the retrieved tarball. Rename the old directory; rename the new
directory to the original name.

=item *

Restore any F<.gitignore> file. This can be done by issuing
C<git checkout .gitignore> in the F<cpan/Distro> directory.

=item *

Remove files we do not need. That is, remove any files that match the
entries in C<@IGNORABLE> in F<Porting/Maintainers.pl>, and anything that
matches the C<EXCLUDED> section of the distro's entry in the C<%Modules>
hash.

=item *

Restore any files mentioned in the C<CUSTOMIZED> section, using
C<git checkout>. Make any new customizations if necessary. Also,
restore any files that are mentioned in C<@IGNORE>, but were checked
into the repository anyway.

=item *

For any new files in the distro, determine whether they are needed.
If not, delete them, and list them in either C<EXCLUDED> or C<@IGNORABLE>.
Otherwise, add them to C<MANIFEST>, and run C<git add> to add the files
to the repository.

=item *

For any files that are gone, remove them from C<MANIFEST>, and use
C<git rm> to tell git the files will be gone.

=item *

If the C<MANIFEST> file was changed in any of the previous steps, run
C<perl Porting/manisort --output MANIFEST.sort; mv MANIFEST.sort MANIFEST>.

=item *

For any files that have an execute bit set, either remove the execute
bit, or edit F<Porting/exec-bit.txt>

=item *

Run C<make> (or C<nmake> on Windows), see if C<perl> compiles.

=item *

Run the tests for the package.

=item *

Run the tests in F<t/porting> (C<make test_porting>).

=item *

Update the C<DISTRIBUTION> entry in F<Porting/Maintainers.pl>.

=item *

Run a full configure/build/test cycle.

=item *

If everything is ok, commit the changes.

=back

For entries with a non-simple C<FILES> section, or with a C<MAP>, you
may have to take more steps than listed above.

F<Porting/sync-with-cpan> is a script that automates most of the steps
above; but see the comments at the beginning of the file.  In particular,
it has not yet been exercised on Windows, but will certainly require a set
of Unix tools such as Cygwin, and steps that run C<make> will need to run
C<nmake> instead.

=head3 dual-life CPAN module stability

Ensure dual-life CPAN modules are stable, which comes down to:

   for each module that fails its regression tests on $current
       did it fail identically on $previous?
       if yes, "SEP" (Somebody Else's Problem)
       else work out why it failed (a bisect is useful for this)

   attempt to group failure causes

   for each failure cause
       is that a regression?
       if yes, figure out how to fix it
           (more code? revert the code that broke it)
       else
           (presumably) it's relying on something un-or-under-documented
           should the existing behaviour stay?
               yes - goto "regression"
               no - note it in perldelta as a significant bugfix
               (also, try to inform the module's author)

=head3 monitor smoke tests for failures

Similarly, monitor the smoking of core tests, and try to fix.  See
L<http://smoke.procura.nl/index.html>, L<http://perl5.test-smoke.org/>
and L<http://perl.develop-help.com> for a summary. See also
L<http://www.nntp.perl.org/group/perl.daily-build.reports/> which has
the raw reports.

Similarly, monitor the smoking of perl for compiler warnings, and try to
fix.

=for checklist skip BLEAD-POINT

=head3 monitor CPAN testers for failures

For any release except a BLEAD-POINT: Examine the relevant analysis report(s)
at http://analysis.cpantesters.org/beforemaintrelease to see how the impending
release is performing compared to previous releases with regard to building
and testing CPAN modules.

That page accepts a query parameter, C<pair> that takes a pair of
colon-delimited versions to use for comparison.  For example:

http://analysis.cpantesters.org/beforemaintrelease?pair=5.20.2:5.22.0%20RC1

=head3 update perldelta

Get perldelta in a mostly finished state.

Read  F<Porting/how_to_write_a_perldelta.pod>, and try to make sure that
every section it lists is, if necessary, populated and complete. Copy
edit the whole document.

You won't be able to automatically fill in the "Updated Modules" section until
after Module::CoreList is updated (as described below in
L<"update Module::CoreList">).

=head3 Bump the version number

Do not do this yet for a BLEAD-POINT release! You will do this at the end of
the release process.

Increase the version number (e.g. from 5.12.0 to 5.12.1).

For a release candidate for a stable perl, this should happen a week or two
before the first release candidate to allow sufficient time for testing and
smoking with the target version built into the perl executable. For
subsequent release candidates and the final release, it is not necessary to
bump the version further.

There is a tool to semi-automate this process:

    $ ./perl -Ilib Porting/bump-perl-version -i 5.10.0 5.10.1

Remember that this tool is largely just grepping for '5.10.0' or whatever,
so it will generate false positives. Be careful not change text like
"this was fixed in 5.10.0"!

Use git status and git diff to select changes you want to keep.

Be particularly careful with F<INSTALL>, which contains a mixture of
C<5.10.0>-type strings, some of which need bumping on every release, and
some of which need to be left unchanged.
See below in L<"update INSTALL"> for more details.

For the first RC release leading up to a BLEAD-FINAL release, update the
description of which releases are now "officially" supported in
F<pod/perlpolicy.pod>.

When doing a BLEAD-POINT or BLEAD-FINAL release, also make sure the
C<PERL_API_*> constants in F<patchlevel.h> are in sync with the version
you're releasing, unless you're absolutely sure the release you're about to
make is 100% binary compatible to an earlier release. When releasing a MAINT
perl version, the C<PERL_API_*> constants C<MUST NOT> be changed as we aim
to guarantee binary compatibility in maint branches.

After editing, regenerate uconfig.h (this must be run on a system with a
/bin/sh available):

 $ perl regen/uconfig_h.pl

This might not cause any new changes.

You may also need to regen opcodes:

 $ ./perl -Ilib regen/opcode.pl

Test your changes:

 $ git clean -xdf   # careful if you don't have local files to keep!
 $ ./Configure -des -Dusedevel
 $ make
 $ make test

Do note that at this stage, porting tests will fail. They will continue
to fail until you've updated Module::CoreList, as described below.

Commit your changes:

 $ git status
 $ git diff
 B<review the delta carefully>

 $ git commit -a -m 'Bump the perl version in various places for 5.x.y'

At this point you may want to compare the commit with a previous bump to
see if they look similar.  See commit f7cf42bb69 for an example of a
previous version bump.

When the version number is bumped, you should also update Module::CoreList
(as described below in L<"update Module::CoreList">) to reflect the new
version number.

=head3 update INSTALL

Review and update INSTALL to account for the change in version number.
The lines in F<INSTALL> about "is not binary compatible with" may require a
correct choice of earlier version to declare incompatibility with. These are
in the "Changes and Incompatibilities" and "Coexistence with earlier versions
of perl 5" sections.

Be particularly careful with the section "Upgrading from 5.X.Y or earlier".
The "X.Y" needs to be changed to the most recent version that we are
I<not> binary compatible with.

For MAINT and BLEAD-FINAL releases, this needs to refer to the last
release in the previous development cycle (so for example, for a 5.14.x
release, this would be 5.13.11).

For BLEAD-POINT releases, it needs to refer to the previous BLEAD-POINT
release (so for 5.15.3 this would be 5.15.2).  If the last release manager
followed instructions, this should have already been done after the last
blead release, so you may find nothing to do here.

=head3 Check copyright years

Check that the copyright years are up to date by running:

    $ ./perl t/porting/copyright.t --now

Remedy any test failures by editing README or perl.c accordingly (search for
the "Copyright"). If updating perl.c, check if the file's own copyright date in
the C comment at the top needs updating, as well as the one printed by C<-v>.

=head3 Check more build configurations

Try running the full test suite against multiple Perl configurations. Here are
some sets of Configure flags you can try:

=over 4

=item *

C<-Duseshrplib -Dusesitecustomize>

=item *

C<-Duserelocatableinc>

=item *

C<-Dusethreads>

=back

If you have multiple compilers on your machine, you might also consider
compiling with C<-Dcc=$other_compiler>.

=head3 update perlport

L<perlport> has a section currently named I<Supported Platforms> that
indicates which platforms are known to build in the current release.
If necessary update the list and the indicated version number.

=head3 check a readonly build

Even before other prep work, follow the steps in  L<build the tarball> and test
it locally.  Because a perl source tarballs sets many files read-only, it could
test differently than tests run from the repository.  After you're sure
permissions aren't a problem, delete the generated directory and tarballs.

=head2 Building a release - on the day

This section describes the actions required to make a release
that are performed near to, or on the actual release day.

=head3 re-check earlier actions

Review all the actions in the previous section,
L<"Building a release - advance actions"> to ensure they are all done and
up-to-date.

=head3 create a release branch

For BLEAD-POINT releases, making a release from a release branch avoids the
need to freeze blead during the release. This is less important for
BLEAD-FINAL, MAINT, and RC releases, since blead will already be frozen in
those cases. Create the branch by running

    git checkout -b release-5.xx.yy

=head3 build a clean perl

Make sure you have a gitwise-clean perl directory (no modified files,
unpushed commits etc):

    $ git status
    $ git clean -dxf

then configure and build perl so that you have a Makefile and porting tools:

    $ ./Configure -Dusedevel -des && make

=head3 Check module versions

For each Perl release since the previous release of the current branch, check
for modules that have identical version numbers but different contents by
running:

    $ ./perl -Ilib Porting/cmpVERSION.pl --tag=v5.X.YY

(This is done automatically by F<t/porting/cmp_version.t> for the previous
release of the current branch, but not for any releases from other branches.)

Any modules that fail will need a version bump, plus a nudge to the upstream
maintainer for 'cpan' upstream modules.

=head3 update Module::CoreList

=head4 Bump Module::CoreList* $VERSIONs

If necessary, bump C<$Module::CoreList::VERSION> (there's no need to do this
for every RC; in RC1, bump the version to a new clean number that will
appear in the final release, and leave as-is for the later RCs and final).
It may also happen that C<Module::CoreList> has been modified in blead, and
hence has a new version number already.  (But make sure it is not the same
number as a CPAN release.)

C<$Module::CoreList::TieHashDelta::VERSION> and
C<$Module::CoreList::Utils::VERSION> should always be equal to
C<$Module::CoreList::VERSION>. If necessary, bump those two versions to match
before proceeding.

The files to modify are:

=over 4

=item *

F<dist/Module-CoreList/lib/Module/CoreList.pm>

=item *

F<dist/Module-CoreList/lib/Module/CoreList/Utils.pm>

=item *

F<dist/Module-CoreList/lib/Module/CoreList/TieHashDelta.pm>

=back

=head4 Update C<Module::CoreList> with module version data for the new release.

Note that if this is a MAINT release, you should run the following actions
from the maint branch, but commit the C<CoreList.pm> changes in
I<blead> and subsequently cherry-pick any releases since the last
maint release and then your recent commit.  XXX need a better example

[ Note that the procedure for handling Module::CoreList in maint branches
is a bit complex, and the RMG currently don't describe a full and
workable approach. The main issue is keeping Module::CoreList
and its version number synchronised across all maint branches, blead and
CPAN, while having to bump its version number for every RC release.
See this brief p5p thread:

    Message-ID: <20130311174402.GZ2294@iabyn.com>

If you can devise a workable system, feel free to try it out, and to
update the RMG accordingly!

DAPM May 2013 ]

F<corelist.pl> uses ftp.funet.fi to verify information about dual-lived
modules on CPAN. It can use a full, local CPAN mirror and/or fall back
on HTTP::Tiny to fetch package metadata remotely.

(If you'd prefer to have a full CPAN mirror, see
http://www.cpan.org/misc/cpan-faq.html#How_mirror_CPAN)

Then change to your perl checkout, and if necessary,

    $ make

Then, If you have a local CPAN mirror, run:

    $ ./perl -Ilib Porting/corelist.pl ~/my-cpan-mirror

Otherwise, run:

    $ ./perl -Ilib Porting/corelist.pl cpan

This will chug for a while, possibly reporting various warnings about
badly-indexed CPAN modules unrelated to the modules actually in core.
Assuming all goes well, it will update
F<dist/Module-CoreList/lib/Module/CoreList.pm> and possibly
F<dist/Module-CoreList/lib/Module/CoreList/Utils.pm>.

Check those files over carefully:

    $ git diff dist/Module-CoreList/lib/Module/CoreList.pm
    $ git diff dist/Module-CoreList/lib/Module/CoreList/Utils.pm

=head4 Bump version in Module::CoreList F<Changes>

Also edit Module::CoreList's new version number in its F<Changes> file.
This file is F<dist/Module-CoreList/Changes>.

=head4 Add Module::CoreList version bump to perldelta

Add a perldelta entry for the new Module::CoreList version. You only
need to do this if you want to add notes about the changes included
with this version of Module::CoreList. Otherwise, its version bump
will be automatically filled in below in L<finalize perldelta>.

=for checklist skip RC

=head4 Update C<%Module::CoreList::released>

For any release except an RC: Update this version's entry in the C<%released>
hash with today's date.

=head4 Commit Module::CoreList changes

Finally, commit the new version of Module::CoreList:
(unless this is for MAINT; in which case commit it to blead first, then
cherry-pick it back).

    $ git commit -m 'Update Module::CoreList for 5.x.y' \
        dist/Module-CoreList/Changes \
        dist/Module-CoreList/lib/Module/CoreList.pm \
        dist/Module-CoreList/lib/Module/CoreList/Utils.pm

=head4 Rebuild and test

Build and test to get the changes into the currently built lib directory and to
ensure all tests are passing.

=head3 finalize perldelta

Finalize the perldelta.  In particular, fill in the Acknowledgements
section, which can be generated with something like:

    $ perl Porting/acknowledgements.pl v5.15.0..HEAD

Fill in the "New/Updated Modules" sections now that Module::CoreList is
updated:

    $ ./perl -Ilib Porting/corelist-perldelta.pl \
        --mode=update pod/perldelta.pod

For a MAINT release use something like this instead:

    $ ./perl -Ilib Porting/corelist-perldelta.pl 5.020001 5.020002 \
        --mode=update pod/perldelta.pod

Ideally, also fill in a summary of the major changes to each module for which
an entry has been added by F<corelist-perldelta.pl>.

Re-read the perldelta to try to find any embarrassing typos and thinkos;
remove any C<TODO> or C<XXX> flags; update the "Known Problems" section
with any serious issues for which fixes are not going to happen now; and
run through pod and spell checkers, e.g.

    $ podchecker -warnings -warnings pod/perldelta.pod
    $ spell pod/perldelta.pod

Also, you may want to generate and view an HTML version of it to check
formatting, e.g.

    $ ./perl -Ilib ext/Pod-Html/bin/pod2html pod/perldelta.pod > \
        /tmp/perldelta.html

Another good HTML preview option is http://search.cpan.org/pod2html

If you make changes, be sure to commit them.

=for checklist skip BLEAD-POINT MAINT RC

=head3 remove stale perldeltas

For the first RC release that is ONLY for a BLEAD-FINAL, the perldeltas
from the BLEAD-POINT releases since the previous BLEAD-FINAL should have
now been consolidated into the current perldelta, and hence are now just
useless clutter.  They can be removed using:

    $ git rm <file1> <file2> ...

For example, for RC0 of 5.16.0:

    $ cd pod
    $ git rm perldelta515*.pod

=for checklist skip BLEAD-FINAL BLEAD-POINT

=head3 add recent perldeltas

For the first RC for a MAINT release, copy in any recent perldeltas from
blead that have been added since the last release on this branch. This
should include any recent maint releases on branches older than your one,
but not newer. For example if you're producing a 5.14.x release, copy any
perldeltas from recent 5.10.x, 5.12.x etc maint releases, but not from
5.16.x or higher. Remember to

    $ git add <file1> <file2> ...

=head3 update and commit perldelta files

If you have added or removed any perldelta files via the previous two
steps, then edit F<pod/perl.pod> to add/remove them from its table of
contents, then run F<Porting/pod_rules.pl> to propagate your changes there
into all the other files that mention them (including F<MANIFEST>). You'll
need to C<git add> the files that it changes.

Then build a clean perl and do a full test

    $ git status
    $ git clean -dxf
    $ ./Configure -Dusedevel -des
    $ make
    $ make test

Once all tests pass, commit your changes.

=head3 build a clean perl

If you skipped the previous step (adding/removing perldeltas),
again, make sure you have a gitwise-clean perl directory (no modified files,
unpushed commits etc):

    $ git status
    $ git clean -dxf

then configure and build perl so that you have a Makefile and porting tools:

    $ ./Configure -Dusedevel -des && make

=for checklist skip BLEAD-FINAL BLEAD-POINT

=head3 synchronise from blead's perlhist.pod

For the first RC for a MAINT release, copy in the latest
F<pod/perlhist.pod> from blead; this will include details of newer
releases in all branches. In theory, blead's version should be a strict
superset of the one in this branch, but it's probably safest to examine the
changes first, to ensure that there's nothing in this branch that was
forgotten from blead. An easy way to do that is with C<< git checkout -p >>,
to selectively apply any changes from the blead version to your current
branch:

    $ git fetch origin
    $ git checkout -p origin/blead pod/perlhist.pod
    $ git commit -m 'sync perlhist from blead' pod/perlhist.pod

=head3 update perlhist.pod

Add an entry to F<pod/perlhist.pod> with the release date, e.g.:

    David    5.10.1       2009-Aug-06

List yourself in the left-hand column, and if this is the first release
that you've ever done, make sure that your name is listed in the section
entitled C<THE KEEPERS OF THE PUMPKIN>.

I<If you're making a BLEAD-FINAL release>, also update the "SELECTED
RELEASE SIZES" section with the output of
F<Porting/perlhist_calculate.pl>.

Be sure to commit your changes:

    $ git commit -m 'add new release to perlhist' pod/perlhist.pod

=for checklist skip BLEAD-POINT

=head3 update patchlevel.h

I<You MUST SKIP this step for a BLEAD-POINT release>

Update F<patchlevel.h> to add a C<-RC1>-or-whatever string; or, if this is
a final release, remove it. For example:

     static const char * const local_patches[] = {
             NULL
    +        ,"RC1"
     #ifdef PERL_GIT_UNCOMMITTED_CHANGES
             ,"uncommitted-changes"
     #endif

Be sure to commit your change:

    $ git commit -m 'bump version to RCnnn' patchlevel.h

=head3 run makemeta to update META files

    $ ./perl -Ilib Porting/makemeta

Be sure to commit any changes (if applicable):

    $ git status   # any changes?
    $ git commit -m 'Update META files' META.*

=head3 build, test and check a fresh perl

Build perl, then make sure it passes its own test suite, and installs:

    $ git clean -xdf
    $ ./Configure -des -Dprefix=/tmp/perl-5.x.y-pretest

    # or if it's an odd-numbered version:
    $ ./Configure -des -Dusedevel -Dprefix=/tmp/perl-5.x.y-pretest

    $ make test install

Check that the output of C</tmp/perl-5.x.y-pretest/bin/perl -v> and
C</tmp/perl-5.x.y-pretest/bin/perl -V> are as expected,
especially as regards version numbers, patch and/or RC levels, and @INC
paths. Note that as they have been built from a git working
directory, they will still identify themselves using git tags and
commits. (Note that for an odd-numbered version, perl will install
itself as C<perl5.x.y>). C<perl -v> will identify itself as:

 This is perl 5, version X, subversion Y (v5.X.Y (v5.X.Z-NNN-gdeadbeef))

where 5.X.Z is the latest tag, NNN the number of commits since this tag,
and C<< deadbeef >> commit of that tag.

Then delete the temporary installation.

=head3 create the release tag

Create the tag identifying this release (e.g.):

    $ git tag v5.11.0 -m "First release of the v5.11 series!"

It is B<VERY> important that from this point forward, you not push
your git changes to the Perl master repository.  If anything goes
wrong before you publish your newly-created tag, you can delete
and recreate it.  Once you push your tag, we're stuck with it
and you'll need to use a new version number for your release.

=head3 build the tarball

Before you run the following, you might want to install 7-Zip (the
C<p7zip-full> package under Debian or the C<p7zip> port on MacPorts) or
the AdvanceCOMP suite (e.g. the C<advancecomp> package under Debian,
or the C<advancecomp> port on macports - 7-Zip on Windows is the
same code as AdvanceCOMP, so Windows users get the smallest files
first time). These compress about 5% smaller than gzip and bzip2.
Over the lifetime of your distribution this will save a lot of
people a small amount of download time and disk space, which adds
up.

In order to produce the C<xz> tarball, XZ Utils are required. The C<xz>
utility is included with most modern UNIX-type operating systems and
is available for Cygwin. A Windows port is available from
L<http://tukaani.org/xz/>.

B<IMPORTANT>: if you are on OS X, you must export C<COPYFILE_DISABLE=1>
to prevent OS X resource files from being included in your tarball. After
creating the tarball following the instructions below, inspect it to ensure
you don't have files like F<._foobar>.

Create a tarball. Use the C<-s> option to specify a suitable suffix for
the tarball and directory name:

 $ cd root/of/perl/tree
 $ make distclean       # make sure distclean works
 $ git clean -xdf       # make sure perl and git agree on files
                        # git clean should not output anything!
 $ git status           # and there's nothing lying around

 $ perl Porting/makerel -bx -s RC1            # for a release candidate
 $ perl Porting/makerel -bx                   # for the release itself

This creates the  directory F<../perl-x.y.z-RC1> or similar, copies all
the MANIFEST files into it, sets the correct permissions on them, then
tars it up as F<../perl-x.y.z-RC1.tar.gz>. With C<-b>, it also creates a
C<tar.bz2> file. The C<-x> also produces a C<tar.xz> file.

If you're getting your tarball suffixed with -uncommitted and you're sure
your changes were all committed, you can override the suffix with:

    $ perl Porting/makerel -b -s ''

XXX if we go for extra tags and branches stuff, then add the extra details
here

Finally, clean up the temporary directory, e.g.

    $ rm -rf ../perl-x.y.z-RC1

=head3 test the tarball

Once you have a tarball it's time to test the tarball (not the repository).

=head4 Copy the tarball to a web server

Copy the tarballs (.gz and possibly .bz2 and .xz) to a web server somewhere you
have access to.

=head4 Download the tarball to another machine and unpack it

Download the tarball to some other machine. For a release candidate,
you really want to test your tarball on two or more different platforms
and architectures.

=head4 Ask #p5p to test the tarball on different platforms

Once you've verified the tarball can be downloaded and unpacked,
ask the #p5p IRC channel on irc.perl.org for volunteers to test the
tarballs on whatever platforms they can.

If you're not confident in the tarball, you can defer this step until after
your own tarball testing, below.

=head4 Check that F<Configure> works

Check that basic configuration and tests work on each test machine:

    $ ./Configure -des && make all test

    # Or for a development release:
    $ ./Configure -Dusedevel -des && make all test

=head4 Run the test harness and install

Check that the test harness and install work on each test machine:

 $ make distclean
 $ ./Configure -des -Dprefix=/install/path && make all test_harness install
 $ cd /install/path

=head4 Check C<perl -v> and C<perl -V>

Check that the output of C<perl -v> and C<perl -V> are as expected,
especially as regards version numbers, patch and/or RC levels, and @INC
paths.

Note that the results may be different without a F<.git/> directory,
which is why you should test from the tarball.

=head4 Run the Installation Verification Procedure utility

    $ ./perl utils/perlivp
    ...
    All tests successful.
    $

=head4 Compare the installed paths to the last release

Compare the pathnames of all installed files with those of the previous
release (i.e. against the last installed tarball on this branch which you
have previously verified using this same procedure). In particular, look
for files in the wrong place, or files no longer included which should be.
For example, suppose the about-to-be-released version is 5.10.1 and the
previous is 5.10.0:

    cd installdir-5.10.0/
    find . -type f | perl -pe's/5\.10\.0/5.10.1/g' | sort > /tmp/f1
    cd installdir-5.10.1/
    find . -type f | sort > /tmp/f2
    diff -u /tmp/f[12]

=head4 Bootstrap the CPAN client

Bootstrap the CPAN client on the clean install:

    $ bin/cpan

    # Or, perhaps:
    $ bin/cpan5.xx.x

=head4 Install the Inline module with CPAN and test it

Try installing a popular CPAN module that's reasonably complex and that
has dependencies; for example:

    CPAN> install Inline::C
    CPAN> quit

Check that your perl can run this:

    $ bin/perl -lwe "use Inline C => q[int f() { return 42;}]; print f"
    42
    $

=head4 Make sure that perlbug works

Test L<perlbug> with the following:

    $ bin/perlbug
    ...
    Subject: test bug report
    Local perl administrator [yourself]:
    Editor [vi]:
    Module:
    Category [core]:
    Severity [low]:
    (edit report)
    Action (Send/Display/Edit/Subject/Save to File): f
    Name of file to save message in [perlbug.rep]:
    Action (Send/Display/Edit/Subject/Save to File): q

and carefully examine the output (in F<perlbug.rep]>), especially
the "Locally applied patches" section. If everything appears okay, then
delete the file, and try it again, this time actually submitting the bug
report. Check that it shows up, then remember to close it!

=for checklist skip BLEAD-POINT

=head3 monitor smokes

XXX This is probably irrelevant if working on a release branch, though
MAINT or RC might want to push a smoke branch and wait.

Wait for the smoke tests to catch up with the commit which this release is
based on (or at least the last commit of any consequence).

Then check that the smoke tests pass (particularly on Win32). If not, go
back and fix things.

Note that for I<BLEAD-POINT> releases this may not be practical. It takes a
long time for the smokers to catch up, especially the Win32
smokers. This is why we have a RC cycle for I<MAINT> and I<BLEAD-FINAL>
releases, but for I<BLEAD-POINT> releases sometimes the best you can do is
to plead with people on IRC to test stuff on their platforms, fire away,
and then hope for the best.

=head3 upload to PAUSE

Once smoking is okay, upload it to PAUSE. This is the point of no return.
If anything goes wrong after this point, you will need to re-prepare
a new release with a new minor version or RC number.

    https://pause.perl.org/

(Login, then select 'Upload a file to CPAN')

If your workstation is not connected to a high-bandwidth,
high-reliability connection to the Internet, you should probably use the
"GET URL" feature (rather than "HTTP UPLOAD") to have PAUSE retrieve the
new release from wherever you put it for testers to find it.  This will
eliminate anxious gnashing of teeth while you wait to see if your
15 megabyte HTTP upload successfully completes across your slow, twitchy
cable modem.

You can make use of your home directory on dromedary for
this purpose: F<http://users.perl5.git.perl.org/~USERNAME> maps to
F</home/USERNAME/public_html>, where F<USERNAME> is your login account
on dromedary.

I<Remember>: if your upload is partially successful, you
may need to contact a PAUSE administrator or even bump the version of perl.

Upload the .gz, .xz, and .bz2 versions of the tarball.

Do not proceed any further until you are sure that your tarballs are on CPAN.
Check your authors directory www.cpan.org (the globally balanced "fast"
mirror) to confirm that your uploads have been successful.

=for checklist skip RC BLEAD-POINT

=head3 wait for indexing

I<You MUST SKIP this step for RC and BLEAD-POINT>

Wait until you receive notification emails from the PAUSE indexer
confirming that your uploads have been received.  IMPORTANT -- you will
probably get an email that indexing has failed, due to module permissions.
This is considered normal.

=for checklist skip BLEAD-POINT

=head3 disarm patchlevel.h

I<You MUST SKIP this step for BLEAD-POINT release>

Disarm the F<patchlevel.h> change; for example,

     static const char * const local_patches[] = {
             NULL
    -        ,"RC1"
     #ifdef PERL_GIT_UNCOMMITTED_CHANGES
             ,"uncommitted-changes"
     #endif

Be sure to commit your change:

    $ git commit -m 'disarm RCnnn bump' patchlevel.h

=head3 announce to p5p

Mail p5p to announce your new release, with a quote you prepared earlier.

Use the template at Porting/release_announcement_template.txt

Send a carbon copy to C<noc@metacpan.org>

=head3 merge release branch back to blead

Merge the (local) release branch back into master now, and delete it.

    git checkout blead
    git pull
    git merge release-5.xx.yy
    git push
    git branch -d release-5.xx.yy

Note: The merge will create a merge commit if other changes have been pushed
to blead while you've been working on your release branch. Do NOT rebase your
branch to avoid the merge commit (as you might normally do when merging a
small branch into blead) since doing so will invalidate the tag that you
created earlier.

=head3 publish the release tag

Now that you've shipped the new perl release to PAUSE and pushed your changes
to the Perl master repository, it's time to publish the tag you created
earlier too (e.g.):

    $ git push origin tag v5.11.0

=head3 update epigraphs.pod

Add your quote to F<Porting/epigraphs.pod> and commit it.
You can include the customary link to the release announcement even before your
message reaches the web-visible archives by looking for the X-List-Archive
header in your message after receiving it back via perl5-porters.

=head3 blog about your epigraph

If you have a blog, please consider writing an entry in your blog explaining
why you chose that particular quote for your epigraph.

=for checklist skip RC

=head3 Release schedule

I<You MUST SKIP this step for RC>

Tick the entry for your release in F<Porting/release_schedule.pod>.

=for checklist skip RC

=head3 Module::CoreList nagging

I<You MUST SKIP this step for RC>

Remind the current maintainer of C<Module::CoreList> to push a new release
to CPAN.

=for checklist skip RC

=head3 new perldelta

I<You MUST SKIP this step for RC>

Create a new perldelta.

=over 4

=item *

Confirm that you have a clean checkout with no local changes.

=item *

Run F<Porting/new-perldelta.pl>

=item *

Run the C<git add> commands it outputs to add new and modified files.

=item *

Verify that the build still works, by running C<./Configure> and
C<make test_porting>. (On Win32 use the appropriate make utility).

=item *

If F<t/porting/podcheck.t> spots errors in the new F<pod/perldelta.pod>,
run C<./perl -MTestInit t/porting/podcheck.t | less> for more detail.
Skip to the end of its test output to see the options it offers you.

=item *

When C<make test_porting> passes, commit the new perldelta.

=back

At this point you may want to compare the commit with a previous bump to
see if they look similar.  See commit ba03bc34a4 for an example of a
previous version bump.

=for checklist skip MAINT RC

=head3 bump version

I<You MUST SKIP this step for RC and MAINT>

If this was a BLEAD-FINAL release (i.e. the first release of a new maint
series, 5.x.0 where x is even), then bump the version in the blead branch
in git, e.g. 5.12.0 to 5.13.0.

First, add a new feature bundle to F<regen/feature.pl>, initially by just
copying the exiting entry, and bump the file's $VERSION (after the __END__
marker); e.g.

	 "5.14" => [qw(switch say state unicode_strings)],
    +    "5.15" => [qw(switch say state unicode_strings)],

Run F<regen/feature.pl> to propagate the changes to F<lib/feature.pm>.

Then follow the section L<"Bump the version number"> to bump the version
in the remaining files and test and commit.

If this was a BLEAD-POINT release, then just follow the section
L<"Bump the version number">.

After bumping the version, follow the section L<"update INSTALL"> to
ensure all version number references are correct.

(Note: The version is NOT bumped immediately after a MAINT release in order
to avoid confusion and wasted time arising from bug reports relating to
"intermediate versions" such as 5.20.1-and-a-bit: If the report is caused
by a bug that gets fixed in 5.20.2 and this intermediate version already
calls itself 5.20.2 then much time can be wasted in figuring out why there
is a failure from something that "should have been fixed". If the bump is
late then there is a much smaller window of time for such confusing bug
reports to arise. (The opposite problem -- trying to figure out why there
*is* a bug in something calling itself 5.20.1 when in fact the bug was
introduced later -- shouldn't arise for MAINT releases since they should,
in theory, only contain bug fixes but never regressions.))

=head3 clean build and test

Run a clean build and test to make sure nothing obvious is broken.

In particular, F<Porting/perldelta_template.pod> is intentionally exempted
from podchecker tests, to avoid false positives about placeholder text.
However, once it's copied to F<pod/perldelta.pod> the contents can now
cause test failures. Problems should be resolved by doing one of the
following:

=over

=item 1

Replace placeholder text with correct text.

=item 2

If the problem is from a broken placeholder link, you can add it to the
array C<@perldelta_ignore_links> in F<t/porting/podcheck.t>.  Lines
containing such links should be marked with C<XXX> so that they get
cleaned up before the next release.

=item 3

Following the instructions output by F<t/porting/podcheck.t> on how to
update its exceptions database.

=back

=head3 push commits

Finally, push any commits done above.

    $ git push origin ....

=for checklist skip BLEAD-POINT MAINT RC

=head3 create maint branch

I<You MUST SKIP this step for RC, BLEAD-POINT, MAINT>

If this was a BLEAD-FINAL release (i.e. the first release of a new maint
series, 5.x.0 where x is even), then create a new maint branch based on
the commit tagged as the current release.

Assuming you're using git 1.7.x or newer:

    $ git checkout -b maint-5.12 v5.12.0
    $ git push origin -u maint-5.12


=for checklist skip BLEAD-POINT MAINT RC

=head3 make the maint branch available in the APC

Clone the new branch into /srv/gitcommon/branches on camel so the APC will
receive its changes.

    $ git clone --branch maint-5.14 /gitroot/perl.git \
    ?  /srv/gitcommon/branches/perl-5.14.x
    $ chmod -R g=u /srv/gitcommon/branches/perl-5.14.x

And nag the sysadmins to make this directory available via rsync.

XXX Who are the sysadmins?  Contact info?

=for checklist skip BLEAD-POINT RC

=head3 copy perldelta.pod to blead

I<You MUST SKIP this step for RC, BLEAD-POINT>

Copy the perldelta.pod for this release into blead; for example:

 $ cd ..../blead
 $ cp -i ../5.10.x/pod/perldelta.pod pod/perl5101delta.pod  #for example
 $ git add pod/perl5101delta.pod

Don't forget to set the NAME correctly in the new file (e.g. perl5101delta
rather than perldelta).

Edit F<pod/perl.pod> to add an entry for the file, e.g.:

    perl5101delta		Perl changes in version 5.10.1

Then rebuild various files:

    $ perl Porting/pod_rules.pl

Finally, commit and push:

    $ git commit -a -m 'add perlXXXdelta'
    $ git push origin ....

=for checklist skip BLEAD-POINT

=head3 copy perlhist.pod entries to blead

Make sure any recent F<pod/perlhist.pod> entries are copied to
F<perlhist.pod> on blead.  e.g.

    5.8.9         2008-Dec-14

=head3 Relax!

I<You MUST RETIRE to your preferred PUB, CAFE or SEASIDE VILLA for some
much-needed rest and relaxation>.

Thanks for releasing perl!

=head2 Building a release - the day after

=for checklist skip BLEAD-FINAL, MAINT, RC

=head3 update Module::CoreList

I<After a BLEAD-POINT release only>

After Module::CoreList has shipped to CPAN by the maintainer, update
Module::CoreList in the source so that it reflects the new blead
version number:

=over 4

=item *

Update F<Porting/Maintainers.pl> to list the new DISTRIBUTION on CPAN,
which should be identical to what is currently in blead.

=item *

Bump the $VERSION in F<dist/Module-CoreList/lib/Module/CoreList.pm>,
F<dist/Module-CoreList/lib/Module/CoreList/TieHashDelta.pm> and
F<dist/Module-CoreList/lib/Module/CoreList/Utils.pm>.

=item *

If you have a local CPAN mirror, run:

    $ ./perl -Ilib Porting/corelist.pl ~/my-cpan-mirror

Otherwise, run:

    $ ./perl -Ilib Porting/corelist.pl cpan

This will update F<dist/Module-CoreList/lib/Module/CoreList.pm> and
F<dist/Module-CoreList/lib/Module/CoreList/Utils.pm> as it did before,
but this time adding new sections for the next BLEAD-POINT release.

=item *

Add the new $Module::CoreList::VERSION to
F<dist/Module-CoreList/Changes>.

=item *

Update F<pod/perldelta.pod> to mention the upgrade to Module::CoreList.

=item *

Remake perl to get your changed .pm files propagated into F<lib/> and
then run at least the F<dist/Module-CoreList/t/*.t> tests and the
test_porting makefile target to check that they're ok.

=item *

Run

 $ ./perl -Ilib -MModule::CoreList \
    -le 'print Module::CoreList->find_version($]) ? "ok" : "not ok"'

and check that it outputs "ok" to prove that Module::CoreList now knows
about blead's current version.

=item *

Commit and push your changes.

=back

=head3 check tarball availability

Check various website entries to make sure the that tarball has appeared
and is properly indexed:

=over 4

=item *

Check your author directory under L<http://www.cpan.org/authors/id/>
to ensure that the tarballs are available on the website.

=item *

Check F</src> on CPAN (on a fast mirror) to ensure that links to
the new tarballs have appeared: There should be links in F</src/5.0>
(which is accumulating all new versions), and (for BLEAD-FINAL and
MAINT only) an appropriate mention in F</src/README.html> (which describes
the latest versions in each stable branch, with links).

The F</src/5.0> links should appear automatically, some hours after upload.
If they don't, or the F</src> description is inadequate,
ask Ask <ask@perl.org>.

=item *

Check L<http://www.cpan.org/src/> to ensure that the F</src> updates
have been correctly mirrored to the website.
If they haven't, ask Ask <ask@perl.org>.

=item *

Check L<http://search.cpan.org> to see if it has indexed the distribution.
It should be visible at a URL like C<http://search.cpan.org/dist/perl-5.10.1/>.

=back

=for checklist skip RC

=head3 update dev.perl.org

I<You MUST SKIP this step for a RC release>

In your C<perlweb> repository, link to the new release.  For a new
latest-maint release, edit F<docs/shared/tpl/stats.html>.  Otherwise,
edit F<docs/dev/perl5/index.html>.

Then make a pull request to Leo Lapworth.  If this fails for some reason
and you cannot cajole anybody else into submitting that change, you can
mail Leo as last resort.

This repository can be found on L<github|https://github.com/perlorg/perlweb>.

=head3 update release manager's guide

Go over your notes from the release (you did take some, right?) and update
F<Porting/release_managers_guide.pod> with any fixes or information that
will make life easier for the next release manager.

=for checklist end

=head1 SOURCE

Based on
http://www.xray.mpe.mpg.de/mailing-lists/perl5-porters/2009-05/msg00608.html,
plus a whole bunch of other sources, including private correspondence.

=cut