| 1 | Jarkko's How to build Configure tweaked by Nick and Merijn, and now |
| 2 | maintained by perl5-metaconfig |
| 3 | |
| 4 | The Configure script and config_h.SH file in the Perl distribution are |
| 5 | generated by a program called metaconfig. Metaconfig was originally |
| 6 | written by Larry Wall, and was subsequently enhanced and maintained |
| 7 | by Raphael Manfredi. The binary that invokes the generation of the |
| 8 | Configure file is called mconfig. |
| 9 | |
| 10 | As sort order and filenaming are vital in this process, make sure you |
| 11 | are working on a case-sensitive file system! (Case preserving is not |
| 12 | sufficient). |
| 13 | |
| 14 | You have presumably obtained the metaconfig from the repository e.g. |
| 15 | |
| 16 | $ git clone git@github.com:Perl/metaconfig metaconfig |
| 17 | |
| 18 | When working with metaconfig you will generally have two git checkouts |
| 19 | next to each other: (1) this metaconfig checkout; and (2) a checkout of |
| 20 | the Perl source code in which you will generate a new Configure script. In this |
| 21 | README, we will refer to these directories as the 'metaconfig' directory and |
| 22 | the 'perl' directory. |
| 23 | |
| 24 | Since these two directories are normally next to each other, so ../perl |
| 25 | will get you to perl and ../perl/../metaconfig will get you back here. |
| 26 | You should establish a symbolic link to the checkout in which Configure |
| 27 | is generated such as this: |
| 28 | |
| 29 | $ cd metaconfig |
| 30 | $ ln -s ../perl perl |
| 31 | |
| 32 | We will do the reverse symlinks later. |
| 33 | |
| 34 | Contents of this directory: |
| 35 | |
| 36 | README: This file. |
| 37 | U: Metaconfig units used for building Perl's Configure |
| 38 | dist-git: |
| 39 | a git clone of "dist". Optionally present. See (a) below. |
| 40 | This is where dist/meta resides as of 2016-04-01 |
| 41 | dist: |
| 42 | The folder where the original units from dist are in. |
| 43 | These may differ from dist-git, as upstream also moves |
| 44 | on and develops. |
| 45 | |
| 46 | Development workflow: |
| 47 | |
| 48 | (a) In order to assemble Configure from its units, you need mlint/metalint and |
| 49 | mconfig/metaconfig from the "dist" package installed and available in your |
| 50 | $PATH. You can either use the version that comes with your OS (Debian ships |
| 51 | it) or the versions that are included in this checkout: just add the full |
| 52 | name of this folder/bin to your $PATH. If you are not planning to analyse |
| 53 | differences of the current state with upstream dist, you can skip the rest |
| 54 | of step (a) now. |
| 55 | |
| 56 | If you also want to play with or compare to the original meta/dist, you |
| 57 | can checkout that too. |
| 58 | |
| 59 | The dist version used for perl is dist-3.5-20 in this directory, which is |
| 60 | a slightly modified version of the original, which you can get at GITHUB |
| 61 | repository https://github.com/rmanfredi/dist.git. If you'd like to keep |
| 62 | up to date with changes in dist, you can use git to create your own clone. |
| 63 | For git, that would be something like: |
| 64 | |
| 65 | $ git clone https://github.com/rmanfredi/dist.git dist-git |
| 66 | |
| 67 | Unsurprisingly 'dist' uses (its) Configure to generate itself: |
| 68 | |
| 69 | $ cd dist-3.5-20 # or dist-git |
| 70 | $ chmod -R +w . # We have derived files in git :-( |
| 71 | $ ./Configure |
| 72 | $ make |
| 73 | $ make install |
| 74 | |
| 75 | After make install, remove lib/U/d_debugging.U in your target lib, as perl |
| 76 | uses its own way to set/define debugging (see INSTALL) |
| 77 | |
| 78 | dist's 'Configure' is similar to perl's but perhaps not quite as polished. |
| 79 | |
| 80 | There are some perl specific "dist units" in the 'U' directory. |
| 81 | The U directory also contains some patches to 'dist' which have already |
| 82 | been applied to dist-3.5-20 directory. |
| 83 | |
| 84 | (aa) We have not yet arranged for metaconfig to use perl's versions of the |
| 85 | 'units' by default so you need some housekeeping in the perl directory... |
| 86 | |
| 87 | Add metaconfig/bin to your $PATH or create aliases like |
| 88 | |
| 89 | $ export MC5=/your/path/to/metaconfig |
| 90 | $ alias ml="perl $MC5/bin/mlint -O" |
| 91 | $ alias mc="perl $MC5/bin/mconfig -m -O" |
| 92 | |
| 93 | examples in the rest of this README will just refer to mlint and mconfig |
| 94 | as if they appear in your $PATH |
| 95 | |
| 96 | (aaa) |
| 97 | |
| 98 | If you plan to make changes to mconfig or mlint locally (and you might |
| 99 | want to, as both are written for perl4), consider installing mconfig and |
| 100 | mlint from the cmon subdirectory into your $PATH too. These are the |
| 101 | non-autoloading versions and can easily be changed. As these are used by |
| 102 | all team members, please communicate changes on github first. |
| 103 | |
| 104 | (b) You need to be in the 'perl' checkout directory, which you created the |
| 105 | symbolic link to, in preparation. In this working directory, you need |
| 106 | symbolic links too, which are already known to perl itself to ignore. |
| 107 | Assuming you have metaconfig and perl side by side on the same level: |
| 108 | ln -s ../metaconfig/U U |
| 109 | ln -s ../metaconfig/.package .package |
| 110 | ln -s MANIFEST MANIFEST.new |
| 111 | chmod +w Configure config_h.SH Porting/Glossary Porting/config* |
| 112 | |
| 113 | (c) Create a new file for the unit as U/foo/d_bar.U |
| 114 | ('foo' is one of the existing folders in U except for 'all'. If you are |
| 115 | modifying a unit already in dist, simply copy the dist version to |
| 116 | 'modified' as a starting point. Otherwise, create a new file in one of the |
| 117 | other directories. It most likely will be 'perl', but it could also be |
| 118 | 'compline' or any other existing folder). Choose the best appropriate |
| 119 | subdir of U. See U/README for a description of the various subdirectories. |
| 120 | You should choose the closest existing unit file as a starting point, and |
| 121 | first copy it to the new file. For example, the unit for seeing if |
| 122 | strtold_l() exists was created as U/threads/d_strtold_l.U, copied from |
| 123 | perl/d_strtold.U, then adjusted. It goes under 'threads' because it is |
| 124 | used only on threaded perls. |
| 125 | |
| 126 | (d) Run "mlint -O" to see nits: as opposed to lint, the gripings of mlint |
| 127 | are usually serious and need fixing |
| 128 | |
| 129 | Without -O, exceptions are lots of |
| 130 | Your private U/modified/issymlink.U overrides the public one. |
| 131 | due to the perl special units |
| 132 | |
| 133 | and |
| 134 | |
| 135 | "End.U": stale ?MAKE: dependency '$W'. |
| 136 | |
| 137 | which is apparently normal ... |
| 138 | |
| 139 | -- the next steps are in the perl folder, though the instructions below include |
| 140 | a 'cd perl' at each step, as a reminder. If you already are in 'perl', |
| 141 | disregard the reminders. |
| 142 | |
| 143 | (e) There is a chicken and egg problem for newly created units. To get around |
| 144 | this, for such a unit, edit the file metaconfig.h and add to the comment |
| 145 | the appropropriate name. To continue the example above, we would add the |
| 146 | string HAS_STRTOLD_L at the end of the comment. This can be removed once |
| 147 | the code base has actual uses of the unit. |
| 148 | |
| 149 | (f) "mconfig -m -O" to regenerate Configure and config_h.SH |
| 150 | |
| 151 | Make *sure* your mconfig is the correct one in your $PATH, as the mono-web |
| 152 | package will install /usr/bin/mconfig which will do something completely |
| 153 | different. |
| 154 | |
| 155 | (g) metaconfig does not deal with depends in config_h.SH, so some |
| 156 | reorganization is needed. |
| 157 | |
| 158 | $ cd perl |
| 159 | $ perl Porting/config_h.pl |
| 160 | |
| 161 | will fix the ordering |
| 162 | |
| 163 | (h) The messy semi-automated part is that the knowledge of the new symbol |
| 164 | needs to be propagated to non-Configure lands like Win32, WinCE, Netware, |
| 165 | VMS, VOS, ... see previous Configure changes to see which are these |
| 166 | heathen lands. Files to take care of are |
| 167 | {win32,wince,NetWare}/config_[hH]*, (Win32, WinCE, NetWare), |
| 168 | configure.com (VMS). Depending on the kind of patch djgpp/config* might |
| 169 | also need adjusting (for example when adding/changing the list of |
| 170 | extensions) |
| 171 | |
| 172 | Most can be checked and updated by a tool Nicholas provided: |
| 173 | |
| 174 | $ cd perl |
| 175 | $ perl Porting/checkcfgvar.pl |
| 176 | |
| 177 | and if it shows differences, use one of: |
| 178 | |
| 179 | $ perl Porting/checkcfgvar.pl --regen --default=undef |
| 180 | $ perl Porting/checkcfgvar.pl --regen --default=define |
| 181 | |
| 182 | based on the changes you made. For safety, probes should probably be |
| 183 | 'undef', whereas some other things unconditionally should default to |
| 184 | 'define'. For example, 'default_inc_excludes_dot' should be 'define' |
| 185 | except in very limited circumstances, because it closes a security hole. |
| 186 | |
| 187 | For Win32 the process is semi-automated. You have to have a Win32 |
| 188 | machine to run dmake on to complete the process, but that can be done |
| 189 | later by someone with such access. |
| 190 | |
| 191 | For VMS, ('configure.com'), it may be best to add the units as 'undef' and |
| 192 | let the VMS experts deal with them later. However, you can set them to |
| 193 | 'define' if they are non-tricky (such as being basic functions having |
| 194 | standard signatures across architectures), and are in the oldest release of |
| 195 | VMS that perl can be compiled on, which is 7.3-2. Appendix A of "HP C |
| 196 | Run-Time Library Reference Manual for OpenVMS Systems" gives you that |
| 197 | information. As of October 2017, the latest version online is available |
| 198 | at: http://h41379.www4.hpe.com/doc/84final/5763/5763profile.html |
| 199 | |
| 200 | In configure.com, if there is an existing probe that is essentially the |
| 201 | same (except for the names) as the one you're adding, you can copy, paste, |
| 202 | and adjust to create a new one, but note that it's easy to run afoul of the |
| 203 | quoting rules in configure.com. New probed-for units likely will require |
| 204 | at least 2 groups of changes. |
| 205 | |
| 206 | Rerun checkcfgvar.pl until you've fixed everything it finds. |
| 207 | |
| 208 | (i) Check if U/mkglossary (right near the top) points to where you keep |
| 209 | dist's standard metaconfig units as well as your perl-specific ones. |
| 210 | |
| 211 | (j) Run the perl build chain |
| 212 | |
| 213 | $ cd perl |
| 214 | $ make veryclean # Only if Configure already has been run |
| 215 | $ ./Configure -Duse64bitall -Dusethreads -Dusedevel -des |
| 216 | $ perl regen/uconfig_h.pl |
| 217 | |
| 218 | Then make and make test or make test_harness |
| 219 | |
| 220 | $ make -j12 |
| 221 | $ env TEST_JOBS=13 make test_harness |
| 222 | |
| 223 | Before you start committing, make sure that |
| 224 | |
| 225 | $ make test_porting |
| 226 | |
| 227 | still passes |
| 228 | |
| 229 | (k) Optionally, run Porting/mksample to freshen the Porting/config*. |
| 230 | Adjust the various compile-time options (e.g. 64bit, threads) as |
| 231 | you see fit. |
| 232 | You can skip this step, it's not essential, just good housekeeping. |
| 233 | |
| 234 | Most of this only works if you have run the core-tests with the new |
| 235 | generated files |
| 236 | |
| 237 | (kk) Run U/mkgloss.pl to freshen Porting/Glossary |
| 238 | |
| 239 | You should at least check |
| 240 | |
| 241 | $ perl U/mkgloss.pl | diff Porting/Glossary - |
| 242 | |
| 243 | This will show two warnings that you can ignore: |
| 244 | |
| 245 | U/mkglossary: couldn't find libdb_needs_pthread |
| 246 | U/mkglossary: couldn't find libdirs |
| 247 | |
| 248 | all other things need a review |
| 249 | |
| 250 | -- the next steps are in the metaconfig folder again |
| 251 | |
| 252 | (l) git add U/foo/bar.U when you are ready ... |
| 253 | |
| 254 | (m) git commit -m "Your commit description" |
| 255 | |
| 256 | (n) When all patches are applied, tested and committed, and you are happy, |
| 257 | git push |
| 258 | |
| 259 | References: |
| 260 | |
| 261 | Documentation on 'dist' may be found at these locations: |
| 262 | https://github.com/rmanfredi/dist/blob/master/mcon/man/mconfig.SH |
| 263 | https://manpages.debian.org/stretch/dist/metaconfig.1.en.html |
| 264 | |
| 265 | Git tags: |
| 266 | |
| 267 | Tags are maintained in this git repository mapping the version of the |
| 268 | units that were used for the Configure in a given release of perl, |
| 269 | named simply after the version of perl in question (for example, at |
| 270 | the time of writing the current stable release is 5.26.1). This provides |
| 271 | a stable reference for downstreams wishing to import the metaconfig units |
| 272 | into their own packaging. Therefore, at minimum tags for each stable |
| 273 | release should be made (adding tags for development releases being an |
| 274 | optional extra). |