Clarify side-by-side checkouts.

[metaconfig.git] / README

	Jarkko's How to build Configure tweaked by Nick and Merijn.

The Configure script and config_h.SH file in the Perl distribution are
generated by a program called metaconfig.  Metaconfig was originally 
written by Larry Wall, and was subsequently enhanced and maintained
by Raphael Manfredi. The binary that invokes the generation of the
Configure file is called mconfig.

As sort order and filenaming are vital in this process, make sure you
are working on a case-sensitive file system! (Case preserving is not
sufficient).

You have presumably obtained the metaconfig from the repository e.g.

  $ git clone github.com:perl5-metaconfig/metaconfig metaconfig

When working with metaconfig you will generally have two git checkouts
next to each other:  (1) this metaconfig checkout; and (2) a checkout of
the Perl 5 source code in which you will generate a new Configure
script. In this README, we will refer to these directories as the
'metaconfig' directory and the 'perl' directory.

Since these two directories are normally next to each other, so ../perl
will get you to perl and ../perl/../metaconfig will get you back here.
You should establish a symbolic link to the checkout in which Configure
is generated such as this:

  $ cd metaconfig
  $ ln -s ../perl perl

Contents of this directory:

    README:	This file.
    U:		Metaconfig units used for building Perl's Configure
    U.check:	Sample directory used for testing new metaconfig units.
    		see U.check/README for more information.
    dist-git:
		a git clone of "dist". Optionally present. See (a) below.
		This is where dist/meta resides as of 2016-04-01
    dist:
		The folder where the original units from dist are in.
		These may differ from dist-git, as upstream also moves
		on and develops.

(a) You need to have dist installed so that you have metalint and metaconfig
    in your $PATH. As dist/meta binaries are now included in the git checkout,
    you do NOT need to install dist/meta itself.

    If you also want to play with or compare to the original meta/dist, you
    can checkout that too.

    The dist version used for perl is dist-3.5-20 in this directory, which is
    a slightly modified version of the original, which you can get at GITHUB
    repository https://github.com/rmanfredi/dist.git. If you'd like to keep
    up to date with changes in dist, you can use git to create your own clone.
    For git, that would be something like:

    $ git clone https://github.com/rmanfredi/dist.git dist-git

    Unsurprisingly 'dist' uses (its) Configure to generate itself:

    $ cd dist-3.5-20	# or dist-git
    $ chmod -R +w .     # We have derived files in git :-(
    $ ./Configure
    $ make
    $ make install

    After make install, remove lib/U/d_debugging.U in your target lib, as perl
    uses its own way to set/define debugging (see INSTALL)

    dist's 'Configure' is similar to perl's but perhaps not quite as polished.

    There are some perl specific "dist units" in the 'U' directory.
    The U directory also contains some patches to 'dist' which have already
    been applied to dist-3.5-20 directory.
    We have not yet arranged for metaconfig to use perl's versions of the
    'units' by default so you need some housekeeping in the perl directory...

    Then add metaconfig/bin to your $PATH or create aliases like

    $ export MC5=/your/path/to/metaconfig
    $ alias ml="perl $MC5/bin/mlint -O"
    $ alias mc="perl $MC5/bin/mconfig -m -O"

    examples in the rest of this README will just refer to mlint and mconfig
    as if they appear in your $PATH

(aa)

    If you plan to make changes to mconfig or mlint locally (and you might
    want to, as both are written for perl4), consider installing mconfig and
    mlint from the cmon subdirectory into your $PATH too. These are the
    non-autoloading versions and can easily be changed. As these are used by
    all team members, please communicate changes on github first.

(b) You need to be in a/the Perl directory, which you created the symbolic
    link for in preparation. In this working directory, you need symbolic
    links too, which are already known to perl itself to ignore. Assuming
    you have metaconfig and perl side by side on the same level:
     1) have a symlink to ../metaconfig/U called U
     2) have a symlink to ../metaconfig/.package called .package
     3) have a symlink to MANIFEST called MANIFEST.new
     4) chmod +w Configure config_h.SH Porting/Glossary Porting/config*

(c) Write the new unit as U/perl/d_bar.U ('perl' can also be 'modified',
    'compline' or any other existing folder, except for 'all'). Choose
    the best appropriate subdir of U.  See U/README for a description of
    the various subdirectories.)

(d) Run "mlint -O" to see nits: as opposed to lint, the gripings of mlint
    are usually serious and need fixing

    Without -O, exceptions are lots of
      Your private U/modified/issymlink.U overrides the public one.
    due to the perl special units

    and

    "End.U": stale ?MAKE: dependency '$W'.

    which is apparently normal ...

-- the next steps are in the perl folder

(e) chmod +w Configure config_h.SH

    mlint and mconfig will probly die when these are read-only

(f) mconfig -m -O to regenerate Configure and config_h.SH

    Make *sure* your mconfig is the correct one in your $PATH, as the mono-web
    package will install /usr/bin/mconfig which will do something completely
    different.

(g) metaconfig does not deal with depends in config_h.SH, so some
    reorganization is needed.

    $ cd perl
    $ perl Porting/config_h.SH

    will fix the ordering
    
(h) The messy semi-automated part is that the knowledge of the new symbol
    needs to be propagated to non-Configure lands like Win32, WinCE, Netware,
    VMS, VOS, EPOC, ...  see previous Configure changes to see which are these
    heathen lands.  Files to take care of are
    {win32,wince,NetWare}/config_[hH]*, (Win32, WinCE, NetWare),
    configure.com (VMS), epoc/config.sh (EPOC).  Depending on the kind of
    patch djgpp/config* might also need adjusting (for example when
    adding/changing the list of extensions)

    Most can be checked and updated by a tool Nicholas provided:

    $ cd perl
    $ perl Porting/Porting/checkcfgvar.pl

    and if it shows differences,

    $ perl Porting/checkcfgvar.pl --regen --default=define

    (of course "define" can also be "undef" based on the changes you made

    For Win32 the process is semi-automated - if you have a Win32
    machine to run dmake on ...

(i) Check if U/mkglossary (right near the top) points to where you keep
    dist's standard metaconfig units as well as your perl-specific ones.

(j) Run the perl build chain

    $ cd perl
    $ make veryclean
    $ ./Configure -Duse64bitall -Dusethreads -Dusedevel -des

    The dependency for uconfig.h isn't carved in stone, so you might
    need to regenerate it

    $ perl regen/uconfig_h.pl

    Then make and make test or make test_harness

    $ make -j12
    $ env TEST_JOBS=13 make test_harness

    Before you start committing, make sure that the other developers
    are happy and run

    $ make test_porting

    again

(k) Run U/mksample to freshen the Porting/config* and Porting/Glossary.
    Adjust the various compile-time options (e.g. 64bit, threads) as
    you see fit.
    You can skip this phase, it's not essential, just good housekeeping.

    Most of this only works if you have run the core-tests with the new
    generated files

    You should at least check

    $ perl U/mkgloss.pl | diff Porting/Glossary -

    This will show two warnings that you can ignore:

    U/mkglossary: couldn't find libdb_needs_pthread
    U/mkglossary: couldn't find libdirs

    all other things need a review

-- the next steps are in the metaconfig folder again

(l) git add U/perl/foo/bar.U when you are ready ...

(m) git commit -m "Your commit description"

(n) When all patches are applied, tested and committed, and you are happy,
    git push