Commit | Line | Data |
---|---|---|
496e2020 KW |
1 | Jarkko's How to build Configure tweaked by Nick and Merijn, and now |
2 | maintained by perl5-metaconfig | |
2eb0fc7c MB |
3 | |
4 | The Configure script and config_h.SH file in the Perl distribution are | |
496e2020 | 5 | generated by a program called metaconfig. Metaconfig was originally |
2eb0fc7c | 6 | written by Larry Wall, and was subsequently enhanced and maintained |
92a8625a MBT |
7 | by Raphael Manfredi. The binary that invokes the generation of the |
8 | Configure file is called mconfig. | |
9 | ||
10 | As sort order and filenaming are vital in this process, make sure you | |
11 | are working on a case-sensitive file system! (Case preserving is not | |
12 | sufficient). | |
13b7c768 NIS |
13 | |
14 | You have presumably obtained the metaconfig from the repository e.g. | |
6475898c | 15 | |
b3647673 | 16 | $ git clone github.com:perl5-metaconfig/metaconfig metaconfig |
13b7c768 | 17 | |
bb6fc91c JK |
18 | When working with metaconfig you will generally have two git checkouts |
19 | next to each other: (1) this metaconfig checkout; and (2) a checkout of | |
20 | the Perl 5 source code in which you will generate a new Configure | |
21 | script. In this README, we will refer to these directories as the | |
22 | 'metaconfig' directory and the 'perl' directory. | |
23 | ||
24 | Since these two directories are normally next to each other, so ../perl | |
25 | will get you to perl and ../perl/../metaconfig will get you back here. | |
26 | You should establish a symbolic link to the checkout in which Configure | |
27 | is generated such as this: | |
b3647673 MBT |
28 | |
29 | $ cd metaconfig | |
30 | $ ln -s ../perl perl | |
13b7c768 | 31 | |
496e2020 KW |
32 | We will do the reverse symlinks later. |
33 | ||
2eb0fc7c MB |
34 | Contents of this directory: |
35 | ||
36 | README: This file. | |
07438c4f | 37 | U: Metaconfig units used for building Perl's Configure |
e412846d MBT |
38 | dist-git: |
39 | a git clone of "dist". Optionally present. See (a) below. | |
40 | This is where dist/meta resides as of 2016-04-01 | |
e00c5fdc | 41 | dist: |
b3647673 MBT |
42 | The folder where the original units from dist are in. |
43 | These may differ from dist-git, as upstream also moves | |
44 | on and develops. | |
2eb0fc7c | 45 | |
5268aa6e DH |
46 | Development workflow: |
47 | ||
0a071d90 MBT |
48 | (a) In order to assemble Configure from its units, you need mlint/metaline and |
49 | mconfig/metaconfig from the "dist" package installed and available in your | |
50 | $PATH. You can either use the version that comes with your OS (Debian ships | |
51 | it) or the versions that are included in this checkout: just add the full | |
52 | name of this folder/bin to your $PATH. If you are not planning to analyse | |
53 | differences of the current state with upstream dist, you can skip the rest | |
54 | op step (a) now. | |
1bf51bf8 | 55 | |
b3647673 MBT |
56 | If you also want to play with or compare to the original meta/dist, you |
57 | can checkout that too. | |
1bf51bf8 | 58 | |
e00c5fdc | 59 | The dist version used for perl is dist-3.5-20 in this directory, which is |
e412846d MBT |
60 | a slightly modified version of the original, which you can get at GITHUB |
61 | repository https://github.com/rmanfredi/dist.git. If you'd like to keep | |
62 | up to date with changes in dist, you can use git to create your own clone. | |
63 | For git, that would be something like: | |
1e695180 | 64 | |
e412846d | 65 | $ git clone https://github.com/rmanfredi/dist.git dist-git |
1e695180 | 66 | |
13b7c768 NIS |
67 | Unsurprisingly 'dist' uses (its) Configure to generate itself: |
68 | ||
e412846d | 69 | $ cd dist-3.5-20 # or dist-git |
1e695180 MB |
70 | $ chmod -R +w . # We have derived files in git :-( |
71 | $ ./Configure | |
72 | $ make | |
73 | $ make install | |
13b7c768 | 74 | |
e00c5fdc | 75 | After make install, remove lib/U/d_debugging.U in your target lib, as perl |
07438c4f | 76 | uses its own way to set/define debugging (see INSTALL) |
e00c5fdc | 77 | |
e00c5fdc | 78 | dist's 'Configure' is similar to perl's but perhaps not quite as polished. |
13b7c768 NIS |
79 | |
80 | There are some perl specific "dist units" in the 'U' directory. | |
e00c5fdc MB |
81 | The U directory also contains some patches to 'dist' which have already |
82 | been applied to dist-3.5-20 directory. | |
496e2020 KW |
83 | |
84 | (aa) We have not yet arranged for metaconfig to use perl's versions of the | |
e00c5fdc MB |
85 | 'units' by default so you need some housekeeping in the perl directory... |
86 | ||
b3647673 MBT |
87 | Then add metaconfig/bin to your $PATH or create aliases like |
88 | ||
89 | $ export MC5=/your/path/to/metaconfig | |
90 | $ alias ml="perl $MC5/bin/mlint -O" | |
3239fe17 | 91 | $ alias mc="perl $MC5/bin/mconfig -m -O -X .metaconf-exclusions.txt" |
b3647673 MBT |
92 | |
93 | examples in the rest of this README will just refer to mlint and mconfig | |
94 | as if they appear in your $PATH | |
95 | ||
496e2020 | 96 | (aaa) |
352aa7ee | 97 | |
b3647673 | 98 | If you plan to make changes to mconfig or mlint locally (and you might |
92a8625a MBT |
99 | want to, as both are written for perl4), consider installing mconfig and |
100 | mlint from the cmon subdirectory into your $PATH too. These are the | |
b3647673 MBT |
101 | non-autoloading versions and can easily be changed. As these are used by |
102 | all team members, please communicate changes on github first. | |
103 | ||
496e2020 KW |
104 | (b) You need to be in the 'perl' checkout directory, which you created the |
105 | symbolic link to, in preparation. In this working directory, you need | |
106 | symbolic links too, which are already known to perl itself to ignore. | |
107 | Assuming you have metaconfig and perl side by side on the same level: | |
108 | ln -s ../metaconfig/U U | |
109 | ln -s ../metaconfig/.package .package | |
110 | ln -s MANIFEST MANIFEST.new | |
111 | chmod +w Configure config_h.SH Porting/Glossary Porting/config* | |
112 | ||
113 | (c) Create a new file for the new unit as U/foo/d_bar.U | |
114 | ('foo' is one of the existing folders in U except for 'all'. It most | |
115 | likely will be 'perl', but it could also be 'modified', 'compline' or any | |
116 | other existing folder). Choose the best appropriate subdir of U. See | |
117 | U/README for a description of the various subdirectories.) You should | |
118 | choose the closest existing unit file as a starting point, and first copy | |
119 | it to the new file. For example, the unit for seeing if strtold_l() exists | |
120 | was created as U/threads/d_strtold_l.U, copied from perl/d_strtold.U, then | |
121 | adjusted. It goes under 'threads' because it is used only on threaded | |
122 | perls. | |
13b7c768 | 123 | |
b3647673 MBT |
124 | (d) Run "mlint -O" to see nits: as opposed to lint, the gripings of mlint |
125 | are usually serious and need fixing | |
13b7c768 | 126 | |
b3647673 | 127 | Without -O, exceptions are lots of |
cb69034d | 128 | Your private U/modified/issymlink.U overrides the public one. |
13b7c768 NIS |
129 | due to the perl special units |
130 | ||
131 | and | |
132 | ||
133 | "End.U": stale ?MAKE: dependency '$W'. | |
134 | ||
135 | which is apparently normal ... | |
136 | ||
496e2020 KW |
137 | -- the next steps are in the perl folder, though the instructions below include |
138 | a 'cd perl' at each step, as a reminder. If you already are in 'perl', | |
139 | disregard the reminders. | |
13b7c768 | 140 | |
496e2020 KW |
141 | (e) There is a chicken and egg problem for newly created units. To get around |
142 | this, for such a unit, edit the file metaconfig.h and add to the comment | |
143 | the appropropriate name. To continue the example above, we would add the | |
144 | string HAS_STRTOLD_L at the end of the comment. This can be removed once | |
145 | the code base has actual uses of the unit. | |
b3647673 | 146 | |
3239fe17 AC |
147 | (f) "mconfig -m -O -X .metaconf-exclusions.txt" to regenerate Configure and |
148 | config_h.SH | |
92a8625a MBT |
149 | |
150 | Make *sure* your mconfig is the correct one in your $PATH, as the mono-web | |
151 | package will install /usr/bin/mconfig which will do something completely | |
152 | different. | |
13b7c768 | 153 | |
b15dd5ec MB |
154 | (g) metaconfig does not deal with depends in config_h.SH, so some |
155 | reorganization is needed. | |
156 | ||
b3647673 | 157 | $ cd perl |
430b74fe | 158 | $ perl Porting/config_h.pl |
b15dd5ec MB |
159 | |
160 | will fix the ordering | |
496e2020 | 161 | |
b3647673 | 162 | (h) The messy semi-automated part is that the knowledge of the new symbol |
974341ad | 163 | needs to be propagated to non-Configure lands like Win32, WinCE, Netware, |
65c1af4d | 164 | VMS, VOS, ... see previous Configure changes to see which are these |
974341ad JH |
165 | heathen lands. Files to take care of are |
166 | {win32,wince,NetWare}/config_[hH]*, (Win32, WinCE, NetWare), | |
65c1af4d KW |
167 | configure.com (VMS). Depending on the kind of patch djgpp/config* might |
168 | also need adjusting (for example when adding/changing the list of | |
169 | extensions) | |
b3647673 MBT |
170 | |
171 | Most can be checked and updated by a tool Nicholas provided: | |
172 | ||
3a12a5f5 KW |
173 | $ cd perl |
174 | $ perl Porting/checkcfgvar.pl | |
b3647673 | 175 | |
3a12a5f5 | 176 | and if it shows differences, use one of: |
b3647673 | 177 | |
3a12a5f5 KW |
178 | $ perl Porting/checkcfgvar.pl --regen --default=undef |
179 | $ perl Porting/checkcfgvar.pl --regen --default=define | |
b3647673 | 180 | |
3a12a5f5 KW |
181 | based on the changes you made. For safety, probes should probably be |
182 | 'undef', whereas some other things unconditionally should default to | |
183 | 'define'. For example, 'default_inc_excludes_dot' should be 'define' | |
184 | except in very limited circumstances, because it closes a security hole. | |
13b7c768 | 185 | |
c415f587 KW |
186 | For Win32 the process is semi-automated. You have to have a Win32 |
187 | machine to run dmake on to complete the process, but that can be done | |
188 | later by someone with such access. | |
496e2020 | 189 | |
44d4b8e4 KW |
190 | For VMS, ('configure.com'), it may be best to add the units as 'undef' and |
191 | let the VMS experts deal with them later. However, you can set them to | |
192 | 'define' if they are non-tricky (such as being basic functions having | |
193 | standard signatures across architectures), and are in the oldest release of | |
194 | VMS that perl can be compiled on, which is 7.3-2. Appendix A of "HP C | |
195 | Run-Time Library Reference Manual for OpenVMS Systems" gives you that | |
196 | information. As of October 2017, the latest version online is available | |
197 | at: http://h41379.www4.hpe.com/doc/84final/5763/5763profile.html | |
198 | ||
199 | In configure.com, if there is an existing probe that is essentially the | |
200 | same (except for the names) as the one you're adding, you can copy, paste, | |
201 | and adjust to create a new one, but note that it's easy to run afoul of the | |
202 | quoting rules in configure.com. New probed-for units likely will require | |
203 | at least 2 groups of changes. | |
204 | ||
205 | Rerun checkcfgvar.pl until you've fixed everything it finds. | |
13b7c768 | 206 | |
b3647673 | 207 | (i) Check if U/mkglossary (right near the top) points to where you keep |
2eb0fc7c | 208 | dist's standard metaconfig units as well as your perl-specific ones. |
13b7c768 | 209 | |
b3647673 | 210 | (j) Run the perl build chain |
cf3c5365 | 211 | |
b3647673 | 212 | $ cd perl |
496e2020 | 213 | $ make veryclean # Only if Configure already has been run |
b3647673 | 214 | $ ./Configure -Duse64bitall -Dusethreads -Dusedevel -des |
b3647673 | 215 | $ perl regen/uconfig_h.pl |
cf3c5365 | 216 | |
b3647673 | 217 | Then make and make test or make test_harness |
cf3c5365 | 218 | |
b3647673 MBT |
219 | $ make -j12 |
220 | $ env TEST_JOBS=13 make test_harness | |
cf3c5365 | 221 | |
0a071d90 | 222 | Before you start committing, make sure that |
cf3c5365 | 223 | |
b3647673 MBT |
224 | $ make test_porting |
225 | ||
0a071d90 | 226 | still passes |
b3647673 | 227 | |
496e2020 | 228 | (k) Optionally, run Porting/mksample to freshen the Porting/config*. |
b3647673 MBT |
229 | Adjust the various compile-time options (e.g. 64bit, threads) as |
230 | you see fit. | |
496e2020 | 231 | You can skip this step, it's not essential, just good housekeeping. |
b3647673 MBT |
232 | |
233 | Most of this only works if you have run the core-tests with the new | |
234 | generated files | |
235 | ||
82e3f9e4 DH |
236 | (kk) Run U/mkgloss.pl to freshen Porting/Glossary |
237 | ||
b3647673 MBT |
238 | You should at least check |
239 | ||
240 | $ perl U/mkgloss.pl | diff Porting/Glossary - | |
241 | ||
242 | This will show two warnings that you can ignore: | |
243 | ||
244 | U/mkglossary: couldn't find libdb_needs_pthread | |
245 | U/mkglossary: couldn't find libdirs | |
246 | ||
247 | all other things need a review | |
6475898c MB |
248 | |
249 | -- the next steps are in the metaconfig folder again | |
250 | ||
496e2020 | 251 | (l) git add U/foo/bar.U when you are ready ... |
6475898c MB |
252 | |
253 | (m) git commit -m "Your commit description" | |
13b7c768 | 254 | |
6475898c MB |
255 | (n) When all patches are applied, tested and committed, and you are happy, |
256 | git push | |
b69b99d9 | 257 | |
5268aa6e DH |
258 | References: |
259 | ||
260 | Documentation on 'dist' may be found at these locations: | |
261 | https://github.com/rmanfredi/dist/blob/master/mcon/man/mconfig.SH | |
262 | https://manpages.debian.org/stretch/dist/metaconfig.1.en.html | |
263 | ||
264 | Git tags: | |
265 | ||
266 | Tags are maintained in this git repository mapping the version of the | |
267 | units that were used for the Configure in a given release of perl, | |
268 | named simply after the version of perl in question (for example, at | |
269 | the time of writing the current stable release is 5.26.1). This provides | |
270 | a stable reference for downstreams wishing to import the metaconfig units | |
271 | into their own packaging. Therefore, at minimum tags for each stable | |
272 | release should be made (adding tags for development releases being an | |
273 | optional extra). |