This is a live mirror of the Perl 5 development currently hosted at https://github.com/perl/perl5
Backport some commits from Zefram
[metaconfig.git] / README
CommitLineData
496e2020
KW
1 Jarkko's How to build Configure tweaked by Nick and Merijn, and now
2 maintained by perl5-metaconfig
2eb0fc7c
MB
3
4The Configure script and config_h.SH file in the Perl distribution are
496e2020 5generated by a program called metaconfig. Metaconfig was originally
2eb0fc7c 6written by Larry Wall, and was subsequently enhanced and maintained
92a8625a
MBT
7by Raphael Manfredi. The binary that invokes the generation of the
8Configure file is called mconfig.
9
10As sort order and filenaming are vital in this process, make sure you
11are working on a case-sensitive file system! (Case preserving is not
12sufficient).
13b7c768
NIS
13
14You have presumably obtained the metaconfig from the repository e.g.
6475898c 15
b3647673 16 $ git clone github.com:perl5-metaconfig/metaconfig metaconfig
13b7c768 17
bb6fc91c
JK
18When working with metaconfig you will generally have two git checkouts
19next to each other: (1) this metaconfig checkout; and (2) a checkout of
20the Perl 5 source code in which you will generate a new Configure
21script. In this README, we will refer to these directories as the
22'metaconfig' directory and the 'perl' directory.
23
24Since these two directories are normally next to each other, so ../perl
25will get you to perl and ../perl/../metaconfig will get you back here.
26You should establish a symbolic link to the checkout in which Configure
27is generated such as this:
b3647673
MBT
28
29 $ cd metaconfig
30 $ ln -s ../perl perl
13b7c768 31
496e2020
KW
32We will do the reverse symlinks later.
33
2eb0fc7c
MB
34Contents of this directory:
35
36 README: This file.
07438c4f 37 U: Metaconfig units used for building Perl's Configure
e412846d
MBT
38 dist-git:
39 a git clone of "dist". Optionally present. See (a) below.
40 This is where dist/meta resides as of 2016-04-01
e00c5fdc 41 dist:
b3647673
MBT
42 The folder where the original units from dist are in.
43 These may differ from dist-git, as upstream also moves
44 on and develops.
2eb0fc7c 45
5268aa6e
DH
46Development workflow:
47
0a071d90
MBT
48(a) In order to assemble Configure from its units, you need mlint/metaline and
49 mconfig/metaconfig from the "dist" package installed and available in your
50 $PATH. You can either use the version that comes with your OS (Debian ships
51 it) or the versions that are included in this checkout: just add the full
52 name of this folder/bin to your $PATH. If you are not planning to analyse
53 differences of the current state with upstream dist, you can skip the rest
54 op step (a) now.
1bf51bf8 55
b3647673
MBT
56 If you also want to play with or compare to the original meta/dist, you
57 can checkout that too.
1bf51bf8 58
e00c5fdc 59 The dist version used for perl is dist-3.5-20 in this directory, which is
e412846d
MBT
60 a slightly modified version of the original, which you can get at GITHUB
61 repository https://github.com/rmanfredi/dist.git. If you'd like to keep
62 up to date with changes in dist, you can use git to create your own clone.
63 For git, that would be something like:
1e695180 64
e412846d 65 $ git clone https://github.com/rmanfredi/dist.git dist-git
1e695180 66
13b7c768
NIS
67 Unsurprisingly 'dist' uses (its) Configure to generate itself:
68
e412846d 69 $ cd dist-3.5-20 # or dist-git
1e695180
MB
70 $ chmod -R +w . # We have derived files in git :-(
71 $ ./Configure
72 $ make
73 $ make install
13b7c768 74
e00c5fdc 75 After make install, remove lib/U/d_debugging.U in your target lib, as perl
07438c4f 76 uses its own way to set/define debugging (see INSTALL)
e00c5fdc 77
e00c5fdc 78 dist's 'Configure' is similar to perl's but perhaps not quite as polished.
13b7c768
NIS
79
80 There are some perl specific "dist units" in the 'U' directory.
e00c5fdc
MB
81 The U directory also contains some patches to 'dist' which have already
82 been applied to dist-3.5-20 directory.
496e2020
KW
83
84(aa) We have not yet arranged for metaconfig to use perl's versions of the
e00c5fdc
MB
85 'units' by default so you need some housekeeping in the perl directory...
86
b3647673
MBT
87 Then add metaconfig/bin to your $PATH or create aliases like
88
89 $ export MC5=/your/path/to/metaconfig
90 $ alias ml="perl $MC5/bin/mlint -O"
3239fe17 91 $ alias mc="perl $MC5/bin/mconfig -m -O -X .metaconf-exclusions.txt"
b3647673
MBT
92
93 examples in the rest of this README will just refer to mlint and mconfig
94 as if they appear in your $PATH
95
496e2020 96(aaa)
352aa7ee 97
b3647673 98 If you plan to make changes to mconfig or mlint locally (and you might
92a8625a
MBT
99 want to, as both are written for perl4), consider installing mconfig and
100 mlint from the cmon subdirectory into your $PATH too. These are the
b3647673
MBT
101 non-autoloading versions and can easily be changed. As these are used by
102 all team members, please communicate changes on github first.
103
496e2020
KW
104(b) You need to be in the 'perl' checkout directory, which you created the
105 symbolic link to, in preparation. In this working directory, you need
106 symbolic links too, which are already known to perl itself to ignore.
107 Assuming you have metaconfig and perl side by side on the same level:
108 ln -s ../metaconfig/U U
109 ln -s ../metaconfig/.package .package
110 ln -s MANIFEST MANIFEST.new
111 chmod +w Configure config_h.SH Porting/Glossary Porting/config*
112
113(c) Create a new file for the new unit as U/foo/d_bar.U
114 ('foo' is one of the existing folders in U except for 'all'. It most
115 likely will be 'perl', but it could also be 'modified', 'compline' or any
116 other existing folder). Choose the best appropriate subdir of U. See
117 U/README for a description of the various subdirectories.) You should
118 choose the closest existing unit file as a starting point, and first copy
119 it to the new file. For example, the unit for seeing if strtold_l() exists
120 was created as U/threads/d_strtold_l.U, copied from perl/d_strtold.U, then
121 adjusted. It goes under 'threads' because it is used only on threaded
122 perls.
13b7c768 123
b3647673
MBT
124(d) Run "mlint -O" to see nits: as opposed to lint, the gripings of mlint
125 are usually serious and need fixing
13b7c768 126
b3647673 127 Without -O, exceptions are lots of
cb69034d 128 Your private U/modified/issymlink.U overrides the public one.
13b7c768
NIS
129 due to the perl special units
130
131 and
132
133 "End.U": stale ?MAKE: dependency '$W'.
134
135 which is apparently normal ...
136
496e2020
KW
137-- the next steps are in the perl folder, though the instructions below include
138 a 'cd perl' at each step, as a reminder. If you already are in 'perl',
139 disregard the reminders.
13b7c768 140
496e2020
KW
141(e) There is a chicken and egg problem for newly created units. To get around
142 this, for such a unit, edit the file metaconfig.h and add to the comment
143 the appropropriate name. To continue the example above, we would add the
144 string HAS_STRTOLD_L at the end of the comment. This can be removed once
145 the code base has actual uses of the unit.
b3647673 146
3239fe17
AC
147(f) "mconfig -m -O -X .metaconf-exclusions.txt" to regenerate Configure and
148 config_h.SH
92a8625a
MBT
149
150 Make *sure* your mconfig is the correct one in your $PATH, as the mono-web
151 package will install /usr/bin/mconfig which will do something completely
152 different.
13b7c768 153
b15dd5ec
MB
154(g) metaconfig does not deal with depends in config_h.SH, so some
155 reorganization is needed.
156
b3647673 157 $ cd perl
430b74fe 158 $ perl Porting/config_h.pl
b15dd5ec
MB
159
160 will fix the ordering
496e2020 161
b3647673 162(h) The messy semi-automated part is that the knowledge of the new symbol
974341ad 163 needs to be propagated to non-Configure lands like Win32, WinCE, Netware,
65c1af4d 164 VMS, VOS, ... see previous Configure changes to see which are these
974341ad
JH
165 heathen lands. Files to take care of are
166 {win32,wince,NetWare}/config_[hH]*, (Win32, WinCE, NetWare),
65c1af4d
KW
167 configure.com (VMS). Depending on the kind of patch djgpp/config* might
168 also need adjusting (for example when adding/changing the list of
169 extensions)
b3647673
MBT
170
171 Most can be checked and updated by a tool Nicholas provided:
172
3a12a5f5
KW
173 $ cd perl
174 $ perl Porting/checkcfgvar.pl
b3647673 175
3a12a5f5 176 and if it shows differences, use one of:
b3647673 177
3a12a5f5
KW
178 $ perl Porting/checkcfgvar.pl --regen --default=undef
179 $ perl Porting/checkcfgvar.pl --regen --default=define
b3647673 180
3a12a5f5
KW
181 based on the changes you made. For safety, probes should probably be
182 'undef', whereas some other things unconditionally should default to
183 'define'. For example, 'default_inc_excludes_dot' should be 'define'
184 except in very limited circumstances, because it closes a security hole.
13b7c768 185
c415f587
KW
186 For Win32 the process is semi-automated. You have to have a Win32
187 machine to run dmake on to complete the process, but that can be done
188 later by someone with such access.
496e2020 189
44d4b8e4
KW
190 For VMS, ('configure.com'), it may be best to add the units as 'undef' and
191 let the VMS experts deal with them later. However, you can set them to
192 'define' if they are non-tricky (such as being basic functions having
193 standard signatures across architectures), and are in the oldest release of
194 VMS that perl can be compiled on, which is 7.3-2. Appendix A of "HP C
195 Run-Time Library Reference Manual for OpenVMS Systems" gives you that
196 information. As of October 2017, the latest version online is available
197 at: http://h41379.www4.hpe.com/doc/84final/5763/5763profile.html
198
199 In configure.com, if there is an existing probe that is essentially the
200 same (except for the names) as the one you're adding, you can copy, paste,
201 and adjust to create a new one, but note that it's easy to run afoul of the
202 quoting rules in configure.com. New probed-for units likely will require
203 at least 2 groups of changes.
204
205 Rerun checkcfgvar.pl until you've fixed everything it finds.
13b7c768 206
b3647673 207(i) Check if U/mkglossary (right near the top) points to where you keep
2eb0fc7c 208 dist's standard metaconfig units as well as your perl-specific ones.
13b7c768 209
b3647673 210(j) Run the perl build chain
cf3c5365 211
b3647673 212 $ cd perl
496e2020 213 $ make veryclean # Only if Configure already has been run
b3647673 214 $ ./Configure -Duse64bitall -Dusethreads -Dusedevel -des
b3647673 215 $ perl regen/uconfig_h.pl
cf3c5365 216
b3647673 217 Then make and make test or make test_harness
cf3c5365 218
b3647673
MBT
219 $ make -j12
220 $ env TEST_JOBS=13 make test_harness
cf3c5365 221
0a071d90 222 Before you start committing, make sure that
cf3c5365 223
b3647673
MBT
224 $ make test_porting
225
0a071d90 226 still passes
b3647673 227
496e2020 228(k) Optionally, run Porting/mksample to freshen the Porting/config*.
b3647673
MBT
229 Adjust the various compile-time options (e.g. 64bit, threads) as
230 you see fit.
496e2020 231 You can skip this step, it's not essential, just good housekeeping.
b3647673
MBT
232
233 Most of this only works if you have run the core-tests with the new
234 generated files
235
82e3f9e4
DH
236(kk) Run U/mkgloss.pl to freshen Porting/Glossary
237
b3647673
MBT
238 You should at least check
239
240 $ perl U/mkgloss.pl | diff Porting/Glossary -
241
242 This will show two warnings that you can ignore:
243
244 U/mkglossary: couldn't find libdb_needs_pthread
245 U/mkglossary: couldn't find libdirs
246
247 all other things need a review
6475898c
MB
248
249-- the next steps are in the metaconfig folder again
250
496e2020 251(l) git add U/foo/bar.U when you are ready ...
6475898c
MB
252
253(m) git commit -m "Your commit description"
13b7c768 254
6475898c
MB
255(n) When all patches are applied, tested and committed, and you are happy,
256 git push
b69b99d9 257
5268aa6e
DH
258References:
259
260Documentation on 'dist' may be found at these locations:
261https://github.com/rmanfredi/dist/blob/master/mcon/man/mconfig.SH
262https://manpages.debian.org/stretch/dist/metaconfig.1.en.html
263
264Git tags:
265
266Tags are maintained in this git repository mapping the version of the
267units that were used for the Configure in a given release of perl,
268named simply after the version of perl in question (for example, at
269the time of writing the current stable release is 5.26.1). This provides
270a stable reference for downstreams wishing to import the metaconfig units
271into their own packaging. Therefore, at minimum tags for each stable
272release should be made (adding tags for development releases being an
273optional extra).