This is a live mirror of the Perl 5 development currently hosted at https://github.com/perl/perl5
HACKERS: Update
[perl5.git] / dist / Devel-PPPort / HACKERS
CommitLineData
adfe19db
MHM
1=head1 NAME
2
3HACKERS - Devel::PPPort internals for hackers
4
5=head1 SYNOPSIS
6
7So you probably want to hack C<Devel::PPPort>?
8
9Well, here's some information to get you started with what's
10lying around in this distribution.
11
12=head1 DESCRIPTION
13
b2049988 14=head2 How to build 366 versions of Perl
adfe19db
MHM
15
16C<Devel::PPPort> supports Perl versions between 5.003 and bleadperl.
17To guarantee this support, I need some of these versions on my
b2049988 18machine. I currently have 366 different Perl version/configuration
adfe19db
MHM
19combinations installed on my laptop.
20
21As many of the old Perl distributions need patching to compile
b2049988 22cleanly on newer systems (and because building 366 Perls by hand
adfe19db
MHM
23just isn't fun), I wrote a tool to build all the different
24versions and configurations. You can find it in F<devel/buildperl.pl>.
25It can currently build the following Perl releases:
26
27 5.003
28 5.004 - 5.004_05
29 5.005 - 5.005_04
30 5.6.x
31 5.7.x
32 5.8.x
33 5.9.x
b2049988 34 5.1x.x
f6db508c
KW
35 5.2x.x
36 5.3x.x
adfe19db
MHM
37
38=head2 Fully automatic API checks
39
40Knowing which parts of the API are not backwards compatible and
41probably need C<Devel::PPPort> support is another problem that's
42not easy to deal with manually. If you run
43
44 perl Makefile.PL --with-apicheck
45
46a C file is generated by F<parts/apicheck.pl> that is compiled
47and linked with C<Devel::PPPort>. This C file has the purpose of
48using each of the public API functions/macros once.
49
5cde1e48
KW
50The required information is derived from F<parts/embed.fnc> (just
51a copy of bleadperl's F<embed.fnc>), F<parts/apidoc.fnc> (which
adfe19db 52is generated by F<devel/mkapidoc.sh> and simply collects the rest
679ad62d 53of the apidoc entries spread over the Perl source code) and
5cde1e48 54F<parts/ppport.fnc> (which lists all API provided purely by
679ad62d 55Devel::PPPort).
5cde1e48 56The generated C file F<apicheck.c> is currently about 500k in size
adfe19db
MHM
57and takes quite a while to compile.
58
5cde1e48 59Usually, F<apicheck.c> won't compile with older perls. And even if
adfe19db
MHM
60it compiles, there's still a good chance of the dynamic linker
61failing at C<make test> time. But that's on purpose!
62
63We can use these failures to find changes in the API automatically.
64The two Perl scripts F<devel/mktodo> and F<devel/mktodo.pl>
65repeatedly run C<Devel::PPPort> with the apicheck code through
66all different versions of perl. Scanning the output of the compiler
67and the dynamic linker for errors, the files in F<parts/todo/> are
68generated. These files list all parts of the public API that don't
69work with less than a certain version of Perl.
70
71This information is in turn used by F<parts/apicheck.pl> to mask
72API calls in the generated C file for these versions, so the
73process can be stopped by the time F<apicheck.c> compiles cleanly
ba120f6f
MHM
74and the dynamic linker is happy. (Actually, this process may generate
75false positives, so by default each API call is checked once more
76afterwards.)
adfe19db 77
5cde1e48 78Running F<devel/mktodo> takes about an hour, depending of course
ba120f6f
MHM
79on the machine you're running it on. If you run it with
80the C<--nocheck> option, it won't recheck the API calls that failed
81in the compilation stage and it'll take significantly less time.
82Running with C<--nocheck> should usually be safe.
adfe19db 83
5cde1e48 84When running F<devel/mktodo> with the C<--base> option, it will
adfe19db
MHM
85generate the I<baseline> todo files by disabling all functionality
86provided by C<Devel::PPPort>. These are required for implementing
5cde1e48 87the C<--compat-version> option of the F<ppport.h> script. The
adfe19db
MHM
88baseline todo files hold the information about which version of
89Perl lacks a certain part of the API.
90
91However, only the documented public API can be checked this way.
92And since C<Devel::PPPort> provides more macros, these would not be
93affected by C<--compat-version>. It's the job of F<devel/scanprov>
94to figure out the baseline information for all remaining provided
95macros by scanning the include files in the F<CORE> directory of
96various Perl versions.
97
ba120f6f
MHM
98The whole process isn't platform independent. It has currently been
99tested only under Linux, and it definitely requires at least C<gcc> and
100the C<nm> utility.
101
102It's not very often that one has to regenerate the baseline and todo
103files. If you have to, you can either run F<devel/regenerate> or just
104execute the following steps by hand:
adfe19db
MHM
105
106=over 4
107
108=item *
109
110You need a whole bunch of different Perls. The more, the better.
111You can use F<devel/buildperl.pl> to build them. I keep my perls
112in F</tmp/perl>, so most of the tools take this as a default.
113
114=item *
115
56093a11
MHM
116You also need a freshly built bleadperl that is in the path under
117exactly this name. (The name of the executable is currently hardcoded
118in F<devel/mktodo> and F<devel/scanprov>.)
119
120=item *
121
adfe19db
MHM
122Remove all existing todo files in the F<parts/base> and
123F<parts/todo> directories.
124
125=item *
126
127Update the API information. Copy the latest F<embed.fnc> file from
128bleadperl to the F<parts> directory and run F<devel/mkapidoc.sh> to
129collect the remaining information in F<parts/apidoc.fnc>.
130
131=item *
132
133Build the new baseline by running
134
135 perl devel/mktodo --base
136
137in the root directory of the distribution. When it's finished,
138move all files from the F<parts/todo> directory to F<parts/base>.
139
140=item *
141
142Build the new todo files by running
143
144 perl devel/mktodo
145
146in the root directory of the distribution.
147
148=item *
149
150Finally, add the remaining baseline information by running
151
4a582685 152 perl Makefile.PL && make
49ef49fe 153 perl devel/scanprov --mode=write
adfe19db
MHM
154
155=back
156
157=head2 Implementation
158
159Residing in F<parts/inc/> is the "heart" of C<Devel::PPPort>. Each
160of the files implements a part of the supported API, along with
161hints, dependency information, XS code and tests.
162The files are in a POD-like format that is parsed using the
163functions in F<parts/ppptools.pl>.
164
3bfe1c68 165The scripts F<PPPort_pm.PL>, F<RealPPPort_xs.PL> and F<mktests.PL> all
adfe19db 166use the information in F<parts/inc/> to generate the main module
1d088ed8 167F<PPPort.pm>, the XS code in F<RealPPPort.xs> and various test files
adfe19db
MHM
168in F<t/>.
169
f6db508c
KW
170You can get extra information from F<PPPort_pm.PL> by setting the environment
171variable C<DPPP_CHECK_LEVEL> to 1 or 2.
172
adfe19db 173All of these files could be generated on the fly while building
5cde1e48 174C<Devel::PPPort>, but not having the tests in F<t/> will confuse
1d088ed8
MHM
175TEST/harness in the core. Not having F<PPPort.pm> will be bad for
176viewing the docs on C<search.cpan.org>. So unfortunately, it's
177unavoidable to put some redundancy into the package.
adfe19db
MHM
178
179=head2 Adding stuff to Devel::PPPort
180
181First, check if the code you plan to add fits into one of the
182existing files in F<parts/inc/>. If not, just start a new one and
183remember to include it from within F<PPPort_pm.PL>.
184
185Each file holds all relevant data for implementing a certain part
186of the API:
187
188=over 2
189
190=item *
191
192A list of the provided API in the C<=provides> section.
193
194=item *
195
196The implementation to add to F<ppport.h> in the C<=implementation>
197section.
198
9712ab2b
KW
199Here you can add additional information for a given item that will be displayed
200when F<ppport.h> is run. If your item is named C<foo>, you add a
201comment like so:
202
203 /* Hint: foo
204 paragraphs of stuff about foo you want to have
205 shown when ppport.h outputs something about foo
206 */
207
208This will cause S<C<perl ppport.h>> to display this hint when it outputs
209something about C<foo>.
210
211A more serious caution about C<foo> can be displayed by instead saying
212
213 /* Warning: foo
214 paragraphs of stuff about foo you want to have
215 shown when ppport.h outputs something about foo
216 */
217
adfe19db
MHM
218=item *
219
220The code required to add to PPPort.xs for testing the implementation.
221This code goes into the C<=xshead>, C<=xsinit>, C<=xsmisc>, C<=xsboot>
0d0f8426 222and C<=xsubs> section. Have a look at the template at the bottom
3bfe1c68 223of F<RealPPPort_xs.PL> to see where the code ends up.
adfe19db
MHM
224
225=item *
226
227The tests in the C<=tests> section. Remember not to use any fancy
f6db508c
KW
228modules or syntax elements, as the test code needs to be able to run
229with Perl 5.003. (This is because Devel::PPPort itself will run all test files
230in the process of generating the information about when a feature came into
231existence.) This means, for example
232
233=over
234
235=item C<my> isn't supported in C<for>-loops
adfe19db 236
0d0f8426 237 for my $x (1, 2, 3) { } # won't work with 5.003
adfe19db 238
f6db508c
KW
239Instead declare C<$x> just before the statement
240
241=item The postfix for statement modifier isn't supported
242
243 foo for 1..2
244
245won't compile. Instead enclose C<foo> in a loop.
246
247=item You can't use plain C<qr//>
248
249Instead, wrap it in a string eval C<eval "qr//">, and be sure it's skipped at
250execution time on perls earlier than 5.005
251
252=back
253
0d0f8426
MHM
254You can use C<ok()> to report success or failure:
255
f6db508c
KW
256 ok($got, $expected, 'name');
257 ok($got == 42); # Doesn't give good runtime diagnostics
0d0f8426 258
f6db508c
KW
259 ok($got, eval "qr/foo/", 'name') # But don't execute this statement
260 # on perls earlier than 5.005
adfe19db
MHM
261
262=back
263
264It's usually the best approach to just copy an existing file and
265use it as a template.
266
0d0f8426
MHM
267=head2 Implementation Hints
268
269In the C<=implementation> section, you can use
270
271 __UNDEFINED__ macro some definition
272
273instead of
274
275 #ifndef macro
276 # define macro some definition
277 #endif
278
279The macro can have optional arguments and the definition can even
280span multiple lines, like in
281
282 __UNDEFINED__ SvMAGIC_set(sv, val) \
283 STMT_START { assert(SvTYPE(sv) >= SVt_PVMG); \
284 (((XPVMG*) SvANY(sv))->xmg_magic = (val)); } STMT_END
285
960a24c4
KW
286This usually makes the code more compact and readable. And you only have to add
287the line C<__UNDEFINED__> to the C<=provides> section to get all macros
288implemented this way to be imported into this section, so they all get
289documented as being provided.
0d0f8426 290
3451ddaf
P
291If you need to define a helper macro which is not part of C<Devel::PPPort> API
292and its usage is only for definition of other C<Devel::PPPort> macros then use
293the C<D_PPP_> prefix for this macro name (e.g. C<D_PPP_SVPV_NOLEN_LP_ARG>).
294Some C<Devel::PPPort> scripts would report a warning when there is defined a
295macro which is not part of Perl public API, except those macros which start
296with the C<D_PPP_> prefix.
297
f6db508c 298Version checking can be tricky to get correct.
0d0f8426
MHM
299You can use
300
301 #if { VERSION < 5.9.3 }
302
303instead of
304
305 #if ((PERL_VERSION < 9) || (PERL_VERSION == 9 && PERL_SUBVERSION < 3))
306
307The version number can be either of the new form C<5.x.x> or of the older
308form C<5.00x_yy>. Both are translated into the correct preprocessor
309statements. It is also possible to combine this with other statements:
310
311 #if { VERSION >= 5.004 } && !defined(sv_vcatpvf)
b2049988 312 /* a */
0d0f8426
MHM
313 #elif { VERSION < 5.004_63 } && { VERSION != 5.004_05 }
314 /* b */
315 #endif
316
317This not only works in the C<=implementation> section, but also in
318the C<=xsubs>, C<=xsinit>, C<=xsmisc>, C<=xshead> and C<=xsboot> sections.
319
adfe19db
MHM
320=head2 Testing
321
322To automatically test C<Devel::PPPort> with lots of different Perl
323versions, you can use the F<soak> script. Just pass it a list of
324all Perl binaries you want to test.
325
326=head2 Special Makefile targets
327
4a582685 328You can use
adfe19db
MHM
329
330 make regen
331
4a582685
NC
332to regenerate all of the autogenerated files. To get rid of all
333generated files (except for F<parts/todo/*> and F<parts/base/*>),
334use
adfe19db
MHM
335
336 make purge_all
337
338That's it.
339
0d0f8426
MHM
340=head2 Submitting Patches
341
342If you've added some functionality to C<Devel::PPPort>, please
c601e8c7
SA
343consider submitting a patch with your work to P5P by sending a mail
344L<perlbug@perl.org|mailto:perlbug@perl.org>.
0d0f8426
MHM
345
346When submitting patches, please only add the relevant changes
347and don't include the differences of the generated files. You
348can use the C<purge_all> target to delete all autogenerated
349files.
350
56093a11
MHM
351=head2 Integrating into the Perl core
352
353When integrating this module into the Perl core, be sure to
354remove the following files from the distribution. They are
355either not needed or generated on the fly when building this
356module in the core:
357
358 MANIFEST
359 META.yml
360 PPPort.pm
361
adfe19db
MHM
362=head1 COPYRIGHT
363
b2049988 364Version 3.x, Copyright (C) 2004-2013, Marcus Holland-Moritz.
adfe19db
MHM
365
366Version 2.x, Copyright (C) 2001, Paul Marquess.
367
368Version 1.x, Copyright (C) 1999, Kenneth Albanowski.
369
370This program is free software; you can redistribute it and/or
371modify it under the same terms as Perl itself.
372
373=head1 SEE ALSO
374
5cde1e48 375See F<ppport.h> and F<devel/regenerate>.
adfe19db
MHM
376
377=cut