remove another duplicated section

[perl5.git] / pod / perlgit.pod

=encoding utf8

=for comment
Consistent formatting of this file is achieved with:
  perl ./Porting/podtidy pod/perlgit.pod

=head1 NAME

perlgit - Detailed information about git and the Perl repository

=head1 DESCRIPTION

This document provides details on using git to develop Perl. If you are
just interested in working on a quick patch, see L<perlhack> first.
This document is intended for people who are regular contributors to
Perl, including those with write access to the git repository.

=head1 CLONING THE REPOSITORY

All of Perl's source code is kept centrally in a Git repository at
I<perl5.git.perl.org>.

You can make a read-only clone of the repository by running:

  % git clone git://perl5.git.perl.org/perl.git perl

This uses the git protocol (port 9418).

If you cannot use the git protocol for firewall reasons, you can also
clone via http, though this is much slower:

  % git clone http://perl5.git.perl.org/perl.git perl

=head1 WORKING WITH THE REPOSITORY

Once you have changed into the repository directory, you can inspect
it. After a clone the repository will contain a single local branch,
which will be the current branch as well, as indicated by the asterisk.

  % git branch
  * blead

Using the -a switch to C<branch> will also show the remote tracking
branches in the repository:

  % git branch -a
  * blead
    origin/HEAD
    origin/blead
  ...

The branches that begin with "origin" correspond to the "git remote"
that you cloned from (which is named "origin"). Each branch on the
remote will be exactly tracked by theses branches. You should NEVER do
work on these remote tracking branches. You only ever do work in a
local branch. Local branches can be configured to automerge (on pull)
from a designated remote tracking branch. This is the case with the
default branch C<blead> which will be configured to merge from the
remote tracking branch C<origin/blead>.

You can see recent commits:

  % git log

And pull new changes from the repository, and update your local
repository (must be clean first)

  % git pull

Assuming we are on the branch C<blead> immediately after a pull, this
command would be more or less equivalent to:

  % git fetch
  % git merge origin/blead

In fact if you want to update your local repository without touching
your working directory you do:

  % git fetch

And if you want to update your remote-tracking branches for all defined
remotes simultaneously you can do

  % git remote update

Neither of these last two commands will update your working directory,
however both will update the remote-tracking branches in your
repository.

To make a local branch of a remote branch:

  % git checkout -b maint-5.10 origin/maint-5.10

To switch back to blead:

  % git checkout blead

=head2 Finding out your status

The most common git command you will use will probably be

  % git status

This command will produce as output a description of the current state
of the repository, including modified files and unignored untracked
files, and in addition it will show things like what files have been
staged for the next commit, and usually some useful information about
how to change things. For instance the following:

  $ git status
  # On branch blead
  # Your branch is ahead of 'origin/blead' by 1 commit.
  #
  # Changes to be committed:
  #   (use "git reset HEAD <file>..." to unstage)
  #
  #       modified:   pod/perlgit.pod
  #
  # Changed but not updated:
  #   (use "git add <file>..." to update what will be committed)
  #
  #       modified:   pod/perlgit.pod
  #
  # Untracked files:
  #   (use "git add <file>..." to include in what will be committed)
  #
  #       deliberate.untracked

This shows that there were changes to this document staged for commit,
and that there were further changes in the working directory not yet
staged. It also shows that there was an untracked file in the working
directory, and as you can see shows how to change all of this. It also
shows that there is one commit on the working branch C<blead> which has
not been pushed to the C<origin> remote yet. B<NOTE>: that this output
is also what you see as a template if you do not provide a message to
C<git commit>.

=head2 Patch workflow

First, please read L<perlhack> for details on hacking the Perl core.
That document covers many details on how to create a good patch.

If you already have a Perl repository, you should ensure that you're on
the I<blead> branch, and your repository is up to date:

  % git checkout blead
  % git pull

It's preferable to patch against the latest blead version, since this
is where new development occurs for all changes other than critical bug
fixes. Critical bug fix patches should be made against the relevant
maint branches, or should be submitted with a note indicating all the
branches where the fix should be applied.

Now that we have everything up to date, we need to create a temporary
new branch for these changes and switch into it:

  % git checkout -b orange

which is the short form of

  % git branch orange
  % git checkout orange

Creating a topic branch makes it easier for the maintainers to rebase
or merge back into the master blead for a more linear history. If you
don't work on a topic branch the maintainer has to manually cherry pick
your changes onto blead before they can be applied.

That'll get you scolded on perl5-porters, so don't do that. Be Awesome.

Then make your changes. For example, if Leon Brocard changes his name
to Orange Brocard, we should change his name in the AUTHORS file:

  % perl -pi -e 's{Leon Brocard}{Orange Brocard}' AUTHORS

You can see what files are changed:

  % git status
  # On branch orange
  # Changes to be committed:
  #   (use "git reset HEAD <file>..." to unstage)
  #
  #    modified:   AUTHORS
  #

And you can see the changes:

  % git diff
  diff --git a/AUTHORS b/AUTHORS
  index 293dd70..722c93e 100644
  --- a/AUTHORS
  +++ b/AUTHORS
  @@ -541,7 +541,7 @@    Lars Hecking                   <lhecking@nmrc.ucc.ie>
   Laszlo Molnar                  <laszlo.molnar@eth.ericsson.se>
   Leif Huhn                      <leif@hale.dkstat.com>
   Len Johnson                    <lenjay@ibm.net>
  -Leon Brocard                   <acme@astray.com>
  +Orange Brocard                 <acme@astray.com>
   Les Peters                     <lpeters@aol.net>
   Lesley Binks                   <lesley.binks@gmail.com>
   Lincoln D. Stein               <lstein@cshl.org>

Now commit your change locally:

  % git commit -a -m 'Rename Leon Brocard to Orange Brocard'
  Created commit 6196c1d: Rename Leon Brocard to Orange Brocard
   1 files changed, 1 insertions(+), 1 deletions(-)

The C<-a> option is used to include all files that git tracks that you
have changed. If at this time, you only want to commit some of the
files you have worked on, you can omit the C<-a> and use the command
C<S<git add I<FILE ...>>> before doing the commit. C<S<git add
--interactive>> allows you to even just commit portions of files
instead of all the changes in them.

The C<-m> option is used to specify the commit message. If you omit it,
git will open a text editor for you to compose the message
interactively. This is useful when the changes are more complex than
the sample given here, and, depending on the editor, to know that the
first line of the commit message doesn't exceed the 50 character legal
maximum.

Once you've finished writing your commit message and exited your
editor, git will write your change to disk and tell you something like
this:

  Created commit daf8e63: explain git status and stuff about remotes
   1 files changed, 83 insertions(+), 3 deletions(-)

If you re-run C<git status>, you should see something like this:

  % git status
  # On branch blead
  # Your branch is ahead of 'origin/blead' by 2 commits.
  #
  # Untracked files:
  #   (use "git add <file>..." to include in what will be committed)
  #
  #       deliberate.untracked
  nothing added to commit but untracked files present (use "git add" to track)

When in doubt, before you do anything else, check your status and read
it carefully, many questions are answered directly by the git status
output.

You can examine your last commit with:

  % git show HEAD

and if you are not happy with either the description or the patch
itself you can fix it up by editing the files once more and then issue:

  % git commit -a --amend

Now you should create a patch file for all your local changes:

  % git format-patch -M origin..
  0001-Rename-Leon-Brocard-to-Orange-Brocard.patch

You should now send an email to to
L<perlbug@perl.org|mailto:perlbug@perl.org> with a description of your
changes, and include this patch file as an attachment. In addition to
being tracked by RT, mail to perlbug will automatically be forwarded to
perl5-porters (with manual moderation, so please be patient). You
should only send patches to
L<perl5-porters@perl.org|mailto:perl5-porters@perl.org> directly if the
patch is not ready to be applied, but intended for discussion.

See the next section for how to configure and use git to send these
emails for you.

If you want to delete your temporary branch, you may do so with:

  % git checkout blead
  % git branch -d orange
  error: The branch 'orange' is not an ancestor of your current HEAD.
  If you are sure you want to delete it, run 'git branch -D orange'.
  % git branch -D orange
  Deleted branch orange.

=head2 Committing your changes

Assuming that you'd like to commit all the changes you've made as a
single atomic unit, run this command:

   % git commit -a

(That C<-a> tells git to add every file you've changed to this commit.
New files aren't automatically added to your commit when you use
C<commit -a> If you want to add files or to commit some, but not all of
your changes, have a look at the documentation for C<git add>.)

Git will start up your favorite text editor, so that you can craft a
commit message for your change. See L<perlhack/Commit message> for more
information about what makes a good commit message.

Once you've finished writing your commit message and exited your
editor, git will write your change to disk and tell you something like
this:

  Created commit daf8e63: explain git status and stuff about remotes
   1 files changed, 83 insertions(+), 3 deletions(-)

If you re-run C<git status>, you should see something like this:

  % git status
  # On branch blead
  # Your branch is ahead of 'origin/blead' by 2 commits.
  #
  # Untracked files:
  #   (use "git add <file>..." to include in what will be committed)
  #
  #       deliberate.untracked
  nothing added to commit but untracked files present (use "git add" to track)

When in doubt, before you do anything else, check your status and read
it carefully, many questions are answered directly by the git status
output.

=head2 Using git to send patch emails

Please read L<perlhack> first in order to figure out where your patches
should be sent.

In your ~/git/perl repository, set the destination email to perl's bug
tracker:

  $ git config sendemail.to perlbug@perl.org

Or maybe perl5-porters:

  $ git config sendemail.to perl5-porters@perl.org

Then you can use git directly to send your patch emails:

  $ git send-email 0001-Rename-Leon-Brocard-to-Orange-Brocard.patch

You may need to set some configuration variables for your particular
email service provider. For example, to set your global git config to
send email via a gmail account:

  $ git config --global sendemail.smtpserver smtp.gmail.com
  $ git config --global sendemail.smtpssl 1
  $ git config --global sendemail.smtpuser YOURUSERNAME@gmail.com

With this configuration, you will be prompted for your gmail password
when you run 'git send-email'. You can also configure
C<sendemail.smtppass> with your password if you don't care about having
your password in the .gitconfig file.

=head2 A note on derived files

Be aware that many files in the distribution are derivative--avoid
patching them, because git won't see the changes to them, and the build
process will overwrite them. Patch the originals instead. Most
utilities (like perldoc) are in this category, i.e. patch
F<utils/perldoc.PL> rather than F<utils/perldoc>. Similarly, don't
create patches for files under $src_root/ext from their copies found in
$install_root/lib. If you are unsure about the proper location of a
file that may have gotten copied while building the source
distribution, consult the C<MANIFEST>.

=head2 Cleaning a working directory

The command C<git clean> can with varying arguments be used as a
replacement for C<make clean>.

To reset your working directory to a pristine condition you can do:

  % git clean -dxf

However, be aware this will delete ALL untracked content. You can use

  % git clean -Xf

to remove all ignored untracked files, such as build and test
byproduct, but leave any  manually created files alone.

If you only want to cancel some uncommitted edits, you can use C<git
checkout> and give it a list of files to be reverted, or C<git checkout
-f> to revert them all.

If you want to cancel one or several commits, you can use C<git reset>.

=head2 Bisecting

C<git> provides a built-in way to determine, with a binary search in
the history, which commit should be blamed for introducing a given bug.

Suppose that we have a script F<~/testcase.pl> that exits with C<0>
when some behaviour is correct, and with C<1> when it's faulty. You
need an helper script that automates building C<perl> and running the
testcase:

  % cat ~/run
  #!/bin/sh
  git clean -dxf

  # If you get './makedepend: 1: Syntax error: Unterminated quoted
  # string' when bisecting versions of perl older than 5.9.5 this hack
  # will work around the bug in makedepend.SH which was fixed in
  # version 96a8704c. Make sure to comment out `git checkout makedepend.SH'
  # below too.
  git show blead:makedepend.SH > makedepend.SH

  # If you can use ccache, add -Dcc=ccache\ gcc -Dld=gcc to the Configure line
  # if Encode is not needed for the test, you can speed up the bisect by
  # excluding it from the runs with -Dnoextensions=Encode
  sh Configure -des -Dusedevel -Doptimize="-g"
  test -f config.sh || exit 125
  # Correct makefile for newer GNU gcc
  perl -ni -we 'print unless /<(?:built-in|command)/' makefile x2p/makefile
  # if you just need miniperl, replace test_prep with miniperl
  make test_prep
  [ -x ./perl ] || exit 125
  ./perl -Ilib ~/testcase.pl
  ret=$?
  [ $ret -gt 127 ] && ret=127
  # git checkout makedepend.SH
  git clean -dxf
  exit $ret

This script may return C<125> to indicate that the corresponding commit
should be skipped. Otherwise, it returns the status of
F<~/testcase.pl>.

You first enter in bisect mode with:

  % git bisect start

For example, if the bug is present on C<HEAD> but wasn't in 5.10.0,
C<git> will learn about this when you enter:

  % git bisect bad
  % git bisect good perl-5.10.0
  Bisecting: 853 revisions left to test after this

This results in checking out the median commit between C<HEAD> and
C<perl-5.10.0>. You can then run the bisecting process with:

  % git bisect run ~/run

When the first bad commit is isolated, C<git bisect> will tell you so:

  ca4cfd28534303b82a216cfe83a1c80cbc3b9dc5 is first bad commit
  commit ca4cfd28534303b82a216cfe83a1c80cbc3b9dc5
  Author: Dave Mitchell <davem@fdisolutions.com>
  Date:   Sat Feb 9 14:56:23 2008 +0000

      [perl #49472] Attributes + Unknown Error
      ...

  bisect run success

You can peek into the bisecting process with C<git bisect log> and
C<git bisect visualize>. C<git bisect reset> will get you out of bisect
mode.

Please note that the first C<good> state must be an ancestor of the
first C<bad> state. If you want to search for the commit that I<solved>
some bug, you have to negate your test case (i.e. exit with C<1> if OK
and C<0> if not) and still mark the lower bound as C<good> and the
upper as C<bad>. The "first bad commit" has then to be understood as
the "first commit where the bug is solved".

C<git help bisect> has much more information on how you can tweak your
binary searches.

=head1 Topic branches and rewriting history

Individual committers should create topic branches under
B<yourname>/B<some_descriptive_name>. Other committers should check
with a topic branch's creator before making any change to it.

The simplest way to create a remote topic branch that works on all
versions of git is to push the current head as a new branch on the
remote, then check it out locally:

  $ branch="$yourname/$some_descriptive_name"
  $ git push origin HEAD:$branch
  $ git checkout -b $branch origin/$branch

Users of git 1.7 or newer can do it in a more obvious manner:

  $ branch="$yourname/$some_descriptive_name"
  $ git checkout -b $branch
  $ git push origin -u $branch

If you are not the creator of B<yourname>/B<some_descriptive_name>, you
might sometimes find that the original author has edited the branch's
history. There are lots of good reasons for this. Sometimes, an author
might simply be rebasing the branch onto a newer source point.
Sometimes, an author might have found an error in an early commit which
they wanted to fix before merging the branch to blead.

Currently the master repository is configured to forbid
non-fast-forward merges. This means that the branches within can not be
rebased and pushed as a single step.

The only way you will ever be allowed to rebase or modify the history
of a pushed branch is to delete it and push it as a new branch under
the same name. Please think carefully about doing this. It may be
better to sequentially rename your branches so that it is easier for
others working with you to cherry-pick their local changes onto the new
version. (XXX: needs explanation).

If you want to rebase a personal topic branch, you will have to delete
your existing topic branch and push as a new version of it. You can do
this via the following formula (see the explanation about C<refspec>'s
in the git push documentation for details) after you have rebased your
branch:

   # first rebase
   $ git checkout $user/$topic
   $ git fetch
   $ git rebase origin/blead

   # then "delete-and-push"
   $ git push origin :$user/$topic
   $ git push origin $user/$topic

B<NOTE:> it is forbidden at the repository level to delete any of the
"primary" branches. That is any branch matching
C<m!^(blead|maint|perl)!>. Any attempt to do so will result in git
producing an error like this:

    $ git push origin :blead
    *** It is forbidden to delete blead/maint branches in this repository
    error: hooks/update exited with error code 1
    error: hook declined to update refs/heads/blead
    To ssh://perl5.git.perl.org/perl
     ! [remote rejected] blead (hook declined)
     error: failed to push some refs to 'ssh://perl5.git.perl.org/perl'

As a matter of policy we do B<not> edit the history of the blead and
maint-* branches. If a typo (or worse) sneaks into a commit to blead or
maint-*, we'll fix it in another commit. The only types of updates
allowed on these branches are "fast-forward's", where all history is
preserved.

Annotated tags in the canonical perl.git repository will never be
deleted or modified. Think long and hard about whether you want to push
a local tag to perl.git before doing so. (Pushing unannotated tags is
not allowed.)

=head2 Grafts

The perl history contains one mistake which was not caught in the
conversion: a merge was recorded in the history between blead and
maint-5.10 where no merge actually occurred. Due to the nature of git,
this is now impossible to fix in the public repository. You can remove
this mis-merge locally by adding the following line to your
C<.git/info/grafts> file:

  296f12bbbbaa06de9be9d09d3dcf8f4528898a49 434946e0cb7a32589ed92d18008aaa1d88515930

It is particularly important to have this graft line if any bisecting
is done in the area of the "merge" in question.

=head1 WRITE ACCESS TO THE GIT REPOSITORY

Once you have write access, you will need to modify the URL for the
origin remote to enable pushing. Edit F<.git/config> with the
git-config(1) command:

  % git config remote.origin.url ssh://perl5.git.perl.org/perl.git

You can also set up your user name and e-mail address. Most people do
this once globally in their F<~/.gitconfig> by doing something like:

  % git config --global user.name "Ævar Arnfjörð Bjarmason"
  % git config --global user.email avarab@gmail.com

However if you'd like to override that just for perl then execute then
execute something like the following in F<perl>:

  % git config user.email avar@cpan.org

It is also possible to keep C<origin> as a git remote, and add a new
remote for ssh access:

  % git remote add camel perl5.git.perl.org:/perl.git

This allows you to update your local repository by pulling from
C<origin>, which is faster and doesn't require you to authenticate, and
to push your changes back with the C<camel> remote:

  % git fetch camel
  % git push camel

The C<fetch> command just updates the C<camel> refs, as the objects
themselves should have been fetched when pulling from C<origin>.

=head1 Accepting a patch

If you have received a patch file generated using the above section,
you should try out the patch.

First we need to create a temporary new branch for these changes and
switch into it:

  % git checkout -b experimental

Patches that were formatted by C<git format-patch> are applied with
C<git am>:

  % git am 0001-Rename-Leon-Brocard-to-Orange-Brocard.patch
  Applying Rename Leon Brocard to Orange Brocard

If just a raw diff is provided, it is also possible use this two-step
process:

  % git apply bugfix.diff
  % git commit -a -m "Some fixing" --author="That Guy <that.guy@internets.com>"

Now we can inspect the change:

  % git show HEAD
  commit b1b3dab48344cff6de4087efca3dbd63548ab5e2
  Author: Leon Brocard <acme@astray.com>
  Date:   Fri Dec 19 17:02:59 2008 +0000

    Rename Leon Brocard to Orange Brocard

  diff --git a/AUTHORS b/AUTHORS
  index 293dd70..722c93e 100644
  --- a/AUTHORS
  +++ b/AUTHORS
  @@ -541,7 +541,7 @@ Lars Hecking                        <lhecking@nmrc.ucc.ie>
   Laszlo Molnar                  <laszlo.molnar@eth.ericsson.se>
   Leif Huhn                      <leif@hale.dkstat.com>
   Len Johnson                    <lenjay@ibm.net>
  -Leon Brocard                   <acme@astray.com>
  +Orange Brocard                 <acme@astray.com>
   Les Peters                     <lpeters@aol.net>
   Lesley Binks                   <lesley.binks@gmail.com>
   Lincoln D. Stein               <lstein@cshl.org>

If you are a committer to Perl and you think the patch is good, you can
then merge it into blead then push it out to the main repository:

  % git checkout blead
  % git merge experimental
  % git push

If you want to delete your temporary branch, you may do so with:

  % git checkout blead
  % git branch -d experimental
  error: The branch 'experimental' is not an ancestor of your current HEAD.
  If you are sure you want to delete it, run 'git branch -D experimental'.
  % git branch -D experimental
  Deleted branch experimental.

=head2 Committing to blead

The 'blead' branch will become the next production release of Perl.

Before pushing I<any> local change to blead, it's incredibly important
that you do a few things, lest other committers come after you with
pitchforks and torches:

=over

=item *

Make sure you have a good commit message. See L<perlhack/Commit
message> for details.

=item *

Run the test suite. You might not think that one typo fix would break a
test file. You'd be wrong. Here's an example of where not running the
suite caused problems. A patch was submitted that added a couple of
tests to an existing .t. It couldn't possibly affect anything else, so
no need to test beyond the single affected .t, right?  But, the
submitter's email address had changed since the last of their
submissions, and this caused other tests to fail. Running the test
target given in the next item would have caught this problem.

=item *

If you don't run the full test suite, at least C<make test_porting>.
This will run basic sanity checks. To see which sanity checks, have a
look in F<t/porting>.

=item *

If you make any changes that affect miniperl or core routines that have
different code paths for miniperl, be sure to run C<make minitest>.
This will catch problems that even the full test suite will not catch
because it runs a subset of tests under miniperl rather than perl.

=back

=head2 Committing to maintenance versions

Maintenance versions should only be altered to add critical bug fixes,
see L<perlpolicy>.

To commit to a maintenance version of perl, you need to create a local
tracking branch:

  % git checkout --track -b maint-5.005 origin/maint-5.005

This creates a local branch named C<maint-5.005>, which tracks the
remote branch C<origin/maint-5.005>. Then you can pull, commit, merge
and push as before.

You can also cherry-pick commits from blead and another branch, by
using the C<git cherry-pick> command. It is recommended to use the
B<-x> option to C<git cherry-pick> in order to record the SHA1 of the
original commit in the new commit message.

Before pushing any change to a maint version, make sure you've
satisfied the steps in L</Committing to blead> above.

=head2 Merging from a branch via GitHub

While we don't encourage the submission of patches via GitHub, that
will still happen. Here is a guide to merging patches from a GitHub
repository.

  % git remote add avar git://github.com/avar/perl.git
  % git fetch avar

Now you can see the differences between the branch and blead:

  % git diff avar/orange

And you can see the commits:

  % git log avar/orange

If you approve of a specific commit, you can cherry pick it:

  % git cherry-pick 0c24b290ae02b2ab3304f51d5e11e85eb3659eae

Or you could just merge the whole branch if you like it all:

  % git merge avar/orange

And then push back to the repository:

  % git push

=head2 A note on camel and dromedary

The committers have SSH access to the two servers that serve
C<perl5.git.perl.org>. One is C<perl5.git.perl.org> itself (I<camel>),
which is the 'master' repository. The second one is
C<users.perl5.git.perl.org> (I<dromedary>), which can be used for
general testing and development. Dromedary syncs the git tree from
camel every few minutes, you should not push there. Both machines also
have a full CPAN mirror in /srv/CPAN, please use this. To share files
with the general public, dromedary serves your ~/public_html/ as
C<http://users.perl5.git.perl.org/~yourlogin/>

These hosts have fairly strict firewalls to the outside. Outgoing, only
rsync, ssh and git are allowed. For http and ftp, you can use
http://webproxy:3128 as proxy. Incoming, the firewall tries to detect
attacks and blocks IP addresses with suspicious activity. This
sometimes (but very rarely) has false positives and you might get
blocked. The quickest way to get unblocked is to notify the admins.

These two boxes are owned, hosted, and operated by booking.com. You can
reach the sysadmins in #p5p on irc.perl.org or via mail to
C<perl5-porters@perl.org>.