This is a live mirror of the Perl 5 development currently hosted at https://github.com/perl/perl5
Additional reason for ssh passphrase failures.
[perl5.git] / Porting / repository.pod
CommitLineData
0c97a5ed
JH
1=head1 NAME
2
3repository - Using the Perl repository
4
0cfb3454
GS
5=head1 Synopsis
6
7First, we assume here that you have already decided that you will
8need B<write> access to the repository. If all you need is B<read>
9access, there are much better ways to access the most current state of
10the perl repository, or explore individual files and patches therein.
11See L<perlhack> for details.
12
13This document describes what a Perl Porter needs to do to start using
14the Perl repository.
0c97a5ed
JH
15
16=head1 Prerequisites
17
18You'll need to get hold of the following software.
19
20=over 4
21
22=item Perforce
23
24Download a perforce client from:
25
26 http://www.perforce.com/perforce/loadprog.html
27
28You'll probably also want to look at:
29
30 http://www.perforce.com/perforce/technical.html
31
32where you can look at or download its documentation.
33
34=item ssh
35
36If you don't already have access to an ssh client, then look at its
37home site C<http://www.cs.hut.fi/ssh> which mentions ftp sites from
38which it's available. You only need to build the client parts (ssh
39and ssh-keygen should suffice).
40
d457d97a 41If you're on Windows then you might like to obtain Cygwin from:
dbb93a7d 42
d457d97a 43 http://cygwin.com/
dbb93a7d 44
d457d97a
SH
45which contains an ssh client. (MSYS also contains an ssh client
46but it seems to time-out and disconnect from the server and doesn't
47understand the ServerAliveInterval setting described later that can
48be used to stop Cygwin's ssh client from doing this.)
dbb93a7d
SH
49
50Alternatively, the "plink" program, part of PuTTY:
51
52 http://www.chiark.greenend.org.uk/~sgtatham/putty/
53
54should also work fine for Windows users.
55
0c97a5ed
JH
56=back
57
58=head1 Creating an SSH Key Pair
59
60If you already use ssh and want to use the same key pair for perl
61repository access then you can skip the rest of this section.
62Otherwise, generate an ssh key pair for use with the repository
63by typing the command
64
65 ssh-keygen
66
67After generating a key pair and testing it, ssh-keygen will ask you
68to enter a filename in which to save the key. The default it offers
69will be the file F<~/.ssh/identity> which is suitable unless you
70particularly want to keep separate ssh identities for some reason.
71If so, you could save the perl repository private key in the file
72F<~/.ssh/perl>, for example, but I will use the standard filename
73in the remainder of the examples of this document.
74
75After typing in the filename, it will prompt you to type in a
76passphrase. The private key will itself be encrypted so that it is
77usable only when that passphrase is typed. (When using ssh, you will
78be prompted when it requires a pass phrase to unlock a private key.)
79If you provide a blank passphrase then no passphrase will be needed
80to unlock the key and, as a consequence, anyone who gains access to
81the key file gains access to accounts protected with that key
82(barring additional configuration to restrict access by IP address).
83
84When you have typed the passphrase in twice, ssh-keygen will confirm
85where it has saved the private key (in the filename you gave and
86with permissions set to be only readable by you), what your public
87key is (don't worry: you don't need to memorise it) and where it
88has saved the corresponding public key. The public key is saved in
89a filename corresponding to your private key's filename but with
90".pub" appended, usually F<~/.ssh/identity.pub>. That public key
91can be (but need not be) world readable. It is not used by your
92own system at all.
93
dbb93a7d
SH
94Note that the above process creates a key pair for ssh protocol 1.
95You can request ssh protocol 2 (RSA) instead if you prefer (if your
96particular ssh client supports it), via the command
97
98 ssh-keygen -t rsa
99
100This will create private/public identity files called F<~/.ssh/id_rsa>
101and F<~/.ssh/id_rsa.pub> respectively. Protocol 2 offers a higher
102level of security than protocol 1. This is not required for access to
103the Perl repository -- ssh is used for authentication rather than
104encryption (the Perl sources are open anyway) -- but either protocol
105is supported by the server.
106
d457d97a
SH
107B<IMPORTANT NOTE FOR CYGWIN USERS:> In order to make the private key
108files only readable by you you must include the string "ntea" in the
109"CYGWIN" environment variable in the shell used to run C<chmod(1)>,
110and in the shell used to run the ssh client itself later. If "CYGWIN"
111doesn't contain "ntea" then it will appear to the ssh client that the
112file permissions are not set correctly, in which case the files will be
113ignored and you won't be able to connect.
114
0c97a5ed
JH
115=head1 Notifying the Repository Keeper
116
117Mail the contents of that public key file to the keeper of the perl
118repository (see L</Contact Information> below).
119When the key is added to the repository host's configuration file,
120you will be able to connect to it with ssh by using the corresponding
121private key file (after unlocking it with your chosen passphrase).
122
dbb93a7d
SH
123There is no harm in creating both protocol 1 and protocol 2 keys and
124mailing them both in. That way you'll be able to connect using either
125protocol, which may be useful if you later find yourself using a client
126that only supports one or the other protocol.
127
0c97a5ed
JH
128=head1 Connecting to the Repository
129
130Connections to the repository are made by using ssh to provide a
131TCP "tunnel" rather than by using ssh to login to or invoke any
dbb93a7d
SH
132ordinary commands on the repository.
133
134The ssh (secure shell) protocol runs over port number 22, so if you
135have a firewall installed at the client end then you must ensure that
136it is configured to allow you to make an outgoing connection to port 22
137on sickle.activestate.com.
138
139When you want to start a session using the repository, use the command:
0c97a5ed 140
4bc101a5 141 ssh -l perlrep -f -q -x -L 1666:127.0.0.1:1666 sickle.activestate.com foo
0c97a5ed 142
dbb93a7d
SH
143If you are not using the default filename of F<~/.ssh/identity> or
144F<~/.ssh/id_rsa> to hold your perl repository private key then you'll
145need to add the option B<-i filename> to tell ssh where it is. Unless
146you chose a blank passphrase for that private key, ssh will prompt you
147for the passphrase to unlock that key. Then ssh will fork and put itself
0c97a5ed 148in the background, returning you (silently) to your shell prompt.
dbb93a7d
SH
149
150Note that the first time you connect you may see a message like
151"The authenticity of host 'sickle.activestate.com' can't be established,"
152and asking you if you want to continue. Just answer yes and sickle's
153details will be cached in a F<known_hosts> or F<known_hosts2> file. You
154will not see that message again unless you delete the cache file.
155
0c97a5ed
JH
156The tunnel for repository access is now ready for use.
157
158For the sake of completeness (and for the case where the chosen
159port of 1666 is already in use on your machine), I'll briefly
160describe what all those ssh arguments are for.
161
162=over 4
163
4bc101a5 164=item B<-l perlrep>
0c97a5ed 165
4bc101a5
JH
166Use a remote username of perlrep. (The account on the repository which
167provides the end-point of the ssh tunnel is named "perlrep".)
0c97a5ed
JH
168
169=item B<-f>
170
171Tells ssh to fork and remain running in the background. Since ssh
172is only being used for its tunnelling capabilities, the command
173that ssh runs never does any I/O and can sit silently in the
174background.
175
176=item B<-q>
177
178Tells ssh to be quiet. Without this option, ssh will output a
179message each time you use a p4 command (since each p4 command
180tunnels over the ssh connection to reach the repository).
181
182=item B<-x>
183
184Tells ssh not to bother to set up a tunnel for X11 connections.
185The repository doesn't allow this anyway.
186
187=item B<-L 1666:127.0.0.1:1666>
188
189This is the important option. It tells ssh to listen out for
190connections made to port 1666 on your local machine. When such
191a connection is made, the ssh client tells the remote side
192(the corresponding ssh daemon on the repository) to make a
193connection to IP address 127.0.0.1, port 1666. Data flowing
194along that connection is tunnelled over the ssh connection
195(encrypted). The perforce daemon running on the repository
196only accepts connections from localhost and that is exactly
197where ssh-tunnelled connections appear to come from.
198
199If port 1666 is already in use on your machine then you can
200choose any non-privileged port (a number between 1024 and 65535)
201which happens to be free on your machine. It's the first of the
202three colon separated values that you should change. Picking
203port 2345 would mean changing the option to
204B<-L 2345:127.0.0.1:1666>. Whatever port number you choose should
205be used for the value of the P4PORT environment variable (q.v.).
206
207=item sickle.activestate.com
208
4bc101a5 209This is the canonical name of the host on which the perl repository
1cb8ef30 210resides.
0c97a5ed
JH
211
212=item foo
213
214This is a dummy place holder argument. Without an argument
215here, ssh will try to perform an interactive login to the
216repository which is not allowed. Ordinarily, this argument
217is for the one-off command which is to be executed on the
218remote host. However, the repository's ssh configuration
219file uses the "command=" option to force a particular
220command to run so the actual value of the argument is
221ignored. The command that's actually run merely pauses and
222waits for the ssh connection to drop, then exits.
223
224=back
225
226=head1 Problems
227
228You should normally get a prompt that asks for the passphrase
229for your RSA key when you connect with the ssh command shown
230above. If you see a prompt that looks like:
231
232 perlrep@sickle.activestate.com's password:
233
dbb93a7d
SH
234Then you either don't have a F<~/.ssh/identity> or F<~/.ssh/id_rsa>
235file corresponding to your public key, or that file is not readable.
7dda06b0
SP
236Fix the problem and try again. Alternatively, some ssh implementations
237will fail to verify your RSA key if the key if readable by others.
238Just lower the permissions to make the key readable to yourself.
0c97a5ed 239
dbb93a7d
SH
240If you only had the public key file for one protocol installed at the
241server end then make sure your client is using the corresponding
242protocol. An ssh client that supports protocol 2 will probably choose
243that by default, which will fail if the server end only has your public
244key file for protocol 1. Some ssh clients have "-1" and "-2" arguments
245to force which protocol to use.
246
247The "-v" (verbose) flag can be useful for seeing what protocol your
248client is actually trying to connect with, and for spotting any other
249problems. The flag can be specified multiple times to increase
250verbosity. Note that specifying the "-q" flag as well might override
251your request for verbose output, so drop the "-q" flag when trying this.
252
d457d97a
SH
253If you're using the Cygwin ssh client on Windows then you will probably
254find that the connection times out after a short period of inactivity.
255You will have to keep re-entering your passphrase to reconnect, which
256gets annoying after a while. In order to prevent these time-outs from
257happening place the following two lines in the file F<~/.ssh/config>:
258
259 Host sickle.activestate.com
260 ServerAliveInterval 120
261
262This causes the ssh client to send a message to the server every 120
263seconds to check that the server is still alive. The client will not
264disconnect unless "ServerAliveCountMax" many of these messages go
265unanswered. Run C<man ssh_config> for more details. Note also that
266this option applies to protocol version 2 only.
267
0c97a5ed
JH
268=head1 Using the Perforce Client
269
270Remember to read the documentation for Perforce. You need
271to make sure that three environment variable are set
272correctly before using the p4 client with the perl repository.
273
274=over 4
275
276=item P4PORT
277
278Set this to localhost:1666 (the port for your ssh client to listen on)
279unless that port is already in use on your host. If it is, see
280the section above on the B<-L 1666:127.0.0.1:1666> option to ssh.
281
282=item P4CLIENT
283
284The value of this is the name by which Perforce knows your
2f6eead3 285host's workspace. You need to pick a name (normally, your
83d05790 286Perforce username, a dash, and your host's short name)
0c97a5ed 287when you first start using the perl repository and then
2f6eead3
RG
288stick with it.
289
290Perforce keeps track of the files you have on your machine. It
291does this through your client. When you first sync a version of a
292file, the file comes from the server to your machine. If you sync
293the same file again the server does nothing because it
294knows you already have the file.
295
296You should NOT use the same client on different machines. If you do
297you probably won't get the files you expect, and may end up with
298nasty corruption. Perforce allows you to have as many clients as
299you want. For example, sally-home, sally-openbsd, sally-laptop.
300
301Also, never change the client's root and view at the same time.
302See C<http://www.perforce.com/perforce/doc.002/manuals/p4guide/04_details.html#1048341>
303
0c97a5ed 304If you have multiple hosts sharing the same directory structure
2f6eead3
RG
305via NFS then you may be able to get away with only one client name,
306but be careful.
0c97a5ed
JH
307
308The C<p4 clients> command lists all currently known clients.
309
310=item P4USER
311
312This is the username by which perforce knows you. Use your
313username if you have a well known or obvious one or else pick
314a new one which other perl5-porters will recognise. There is
2f6eead3
RG
315a licence limit on the number of these usernames, so be sure not
316to use more than one.
317
318It is very important to set a password for your Perforce username,
319or else anyone can impersonate you. Use the C<p4 passwd> command
320to do this. Once a password is set for your account, you'll need
321to tell Perforce what it is. You can do this by setting the
322environment variable P4PASSWD, or you can use the C<-P> flag
323with the C<p4> command.
324
325There are a few techniques you can use to avoid having to either
326set an environment variable or type the password on every command.
327One is to create a shell alias, for example, in bash, add something like
328 alias p4='p4 -P secret'
329to your F<.bash_profile> file. Another way is to create a small shell
330script, for example
331 #!/bin/bash
332 p4 -P secret $@
333And use this instead of running C<p4> directly.
334
335With either of these, be sure the file containing your password
336(the F<.bash_profile> or shell script file) is only readable by you.
0c97a5ed
JH
337
338The C<p4 users> command lists all currently known users.
339
340=back
341
dbb93a7d
SH
342Note that on Windows P4PORT and P4USER are requested when installing
343Perforce. They are stored in the registry, so they do not need to be
344set in the environment.
345
0c97a5ed
JH
346Once these three environment variables are set, you can use the
347perforce p4 client exactly as described in its documentation.
dbb93a7d 348
0c97a5ed 349After setting these variables and connecting to the repository
83d05790 350for the first time, you should use the C<p4 user> command to
dbb93a7d
SH
351set a valid email address for yourself. Messages to the commit list
352are sent (faked) from whatever email address you set here.
353
354Also use the C<p4 client> command to specify your workspace
355specifications for each individual client from which you will interact
356with the repository. The P4CLIENT environment variable, of course,
357needs to be set to one of these client workspace names.
0c97a5ed
JH
358
359=head1 Ending a Repository Session
360
361When you have finished a session using the repository, you
362should kill off the ssh client process to break the tunnel.
363Since ssh forked itself into the background, you'll need to use
364something like ps with the appropriate options to find the ssh
365process and then kill it manually. The default signal of
366SIGTERM is fine.
367
368=head1 Overview of the Repository
369
370Please read at least the introductory sections of the Perforce
371User Guide (and perhaps the Quick Start Guide as well) before
372reading this section.
373
374Every repository user typically "owns" a "branch" of the mainline
375code in the repository. They hold the "pumpkin" for things in this
376area, and are usually the only user who will modify files there.
377This is not strictly enforced in order to allow the flexibility
378of other users stealing the pumpkin for short periods with the
379owner's permission.
380
dbb93a7d 381Here is (part of) the current structure of the repository:
0c97a5ed
JH
382
383 /----+-----perl - Mainline development (bleadperl)
f704d51e 384 +-----perlio - PerlIO Pumpkin's Perl
0c97a5ed 385 +-----vmsperl - VMS Pumpkin's Perl
d7f8936a 386 +-----maint-5.004------perl - Maintenance branches
0c97a5ed 387 +-----maint-5.005------perl
dbb93a7d
SH
388 +-----maint-5.6--------perl
389 +-----maint-5.8--------perl
390 +-----pureperl---------pureperl
0c97a5ed
JH
391
392Perforce uses a branching model that simply tracks relationships
393between files. It does not care about directories at all, so
394any file can be a branch of any other file--the fully qualified
395depot path name (of the form //depot/foo/bar.c) uniquely determines
396a file for the purpose of establishing branching relationships.
397Since a branch usually involves hundreds of files, such relationships
398are typically specified en masse using a branch map (try `p4 help branch`).
399`p4 branches` lists the existing branches that have been set up.
400`p4 branch -o branchname` can be used to view the map for a particular
401branch, if you want to determine the ancestor for a particular set of
402files.
403
404The mainline (aka "trunk") code in the Perl repository is under
405"//depot/perl/...". Most branches typically map its entire
406contents under a directory that goes by the same name as the branch
f704d51e
JH
407name. Thus the contents of the perlio branch are to be found
408in //depot/perlio.
0c97a5ed
JH
409
410Run `p4 client` to specify how the repository contents should map to
411your local disk. Most users will typically have a client map that
412includes at least their entire branch and the contents of the mainline.
413
414Run `p4 changes -l -m10` to check on the activity in the repository.
415//depot/perl/Porting/genlog is useful to get an annotated changelog
416that shows files and branches. You can use this listing to determine
417if there are any changes in the mainline that you need to merge into
418your own branch. A typical merging session looks like this:
419
f704d51e
JH
420 % cd ~/p4view/perlio
421 % p4 integrate -b perlio # to bring parent changes into perlio
422 % p4 resolve -am ./... # auto merge the changes
0c97a5ed
JH
423 % p4 resolve ./... # manual merge conflicting changes
424 % p4 submit ./... # check in
425
f704d51e 426If the owner of the mainline wants to bring the changes in perlio
0c97a5ed
JH
427back into the mainline, they do:
428
f704d51e 429 % p4 integrate -r -b perlio
0c97a5ed
JH
430 ...
431
432Generating a patch for change#42 is done as follows:
433
a2c6387b 434 % p4genpatch 42 > change-42.patch
0c97a5ed 435
a2c6387b 436F<p4genpatch> is to be found in //depot/perl/Porting/.
f704d51e
JH
437
438The usual routine to apply a patch is
439
440 % p4 edit file.c file.h
441 % patch < patch.txt
442
443(any necessary, re-Configure, make regen_headers, make clean, etc, here)
444
445 % make all test
446
447(preferably make all test in several platforms and under several
448different Configurations)
449
450 % while unhappy
451 do
452 $EDITOR
453 make all test
454 done
455 % p4 submit
456
457Other useful Perforce commands
458
459 % p4 describe -du 12345 # show change 12345
460
461Note: the output of "p4 describe" is not in proper diff format, use
a2c6387b 462the F<Porting/p4genpatch> to get a diff-compatible format.
44897049
HS
463(Note that it may be easier to get one already prepared: grep
464L<perlhack> for APC, and append eg "/diffs/12345.gz" to one of the
465URLs to get a usable patch.)
f704d51e
JH
466
467 % p4 diff -se ./... # have I modified something but forgotten
468 # to "p4 edit", easy faux pas with autogenerated
469 # files like proto.h, or if one forgets to
470 # look carefully which files a patch modifies
471 % p4 sync file.h # if someone else has modified file.h
472 % p4 opened # which files are opened (p4 edit) by me
473 % p4 opened -a # which files are opened by anybody
474 % p4 diff -du file.c # what changes have I done
475 % p4 revert file.h # never mind my changes
476 % p4 sync -f argh.c # forcibly synchronize your file
477 # from the repository
478 % p4 diff -sr | p4 -x - revert
479 # throw away (opened but) unchanged files
480 # (in Perforce it's a little bit too easy
481 # to checkin unchanged files)
482
483Integrate patch 12345 from the mainline to the maint-5.6 branch:
484(you have to in the directory that has both the mainline and
485the maint-5.6/perl as subdirectories)
486
487 % p4 integrate -d perl/...@12345,12345 maint-5.6/perl/...
488
489Integrate patches 12347-12350 from the perlio branch to the mainline:
490
491 % p4 integrate -d perlio/...@12347,12350 perl/...
0c97a5ed
JH
492
493=head1 Contact Information
494
dbb93a7d 495The mail alias E<lt>perl-repository-keepers@perl.orgE<gt> can be used to reach
b09defb6 496all current users of the repository.
0c97a5ed 497
4abe7572
NC
498The repository keeper is currently Philippe M. Chiasson
499E<lt>gozer@ActiveState.comE<gt>.
0c97a5ed
JH
500
501=head1 AUTHORS
502
dbb93a7d
SH
503Malcolm Beattie, E<lt>mbeattie@sable.ox.ac.ukE<gt>, 24 June 1997.
504
505Gurusamy Sarathy, E<lt>gsar@activestate.comE<gt>, 8 May 1999.
0c97a5ed 506
dbb93a7d 507Slightly updated by Simon Cozens, E<lt>simon@brecon.co.ukE<gt>, 3 July 2000.
0c97a5ed 508
dbb93a7d 509More updates by Jarkko Hietaniemi, E<lt>jhi@iki.fiE<gt>, 28 June 2001.
f704d51e 510
dbb93a7d 511Perforce clarifications by Randall Gellens, E<lt>rcg@users.sourceforge.netE<gt>, 12 July 2001.
0c97a5ed 512
d457d97a
SH
513Windows-related updates by Steve Hay E<lt>shay@cpan.orgE<gt>, 23 July 2004
514and 08 Aug 2005.
2f6eead3 515
0c97a5ed 516=cut