This is a live mirror of the Perl 5 development currently hosted at https://github.com/perl/perl5
HACKERS: Add info about Hints and Warnings
[perl5.git] / dist / Devel-PPPort / HACKERS
CommitLineData
adfe19db
MHM
1=head1 NAME
2
3HACKERS - Devel::PPPort internals for hackers
4
5=head1 SYNOPSIS
6
7So you probably want to hack C<Devel::PPPort>?
8
9Well, here's some information to get you started with what's
10lying around in this distribution.
11
12=head1 DESCRIPTION
13
b2049988 14=head2 How to build 366 versions of Perl
adfe19db
MHM
15
16C<Devel::PPPort> supports Perl versions between 5.003 and bleadperl.
17To guarantee this support, I need some of these versions on my
b2049988 18machine. I currently have 366 different Perl version/configuration
adfe19db
MHM
19combinations installed on my laptop.
20
21As many of the old Perl distributions need patching to compile
b2049988 22cleanly on newer systems (and because building 366 Perls by hand
adfe19db
MHM
23just isn't fun), I wrote a tool to build all the different
24versions and configurations. You can find it in F<devel/buildperl.pl>.
25It can currently build the following Perl releases:
26
27 5.003
28 5.004 - 5.004_05
29 5.005 - 5.005_04
30 5.6.x
31 5.7.x
32 5.8.x
33 5.9.x
b2049988 34 5.1x.x
adfe19db
MHM
35
36=head2 Fully automatic API checks
37
38Knowing which parts of the API are not backwards compatible and
39probably need C<Devel::PPPort> support is another problem that's
40not easy to deal with manually. If you run
41
42 perl Makefile.PL --with-apicheck
43
44a C file is generated by F<parts/apicheck.pl> that is compiled
45and linked with C<Devel::PPPort>. This C file has the purpose of
46using each of the public API functions/macros once.
47
5cde1e48
KW
48The required information is derived from F<parts/embed.fnc> (just
49a copy of bleadperl's F<embed.fnc>), F<parts/apidoc.fnc> (which
adfe19db 50is generated by F<devel/mkapidoc.sh> and simply collects the rest
679ad62d 51of the apidoc entries spread over the Perl source code) and
5cde1e48 52F<parts/ppport.fnc> (which lists all API provided purely by
679ad62d 53Devel::PPPort).
5cde1e48 54The generated C file F<apicheck.c> is currently about 500k in size
adfe19db
MHM
55and takes quite a while to compile.
56
5cde1e48 57Usually, F<apicheck.c> won't compile with older perls. And even if
adfe19db
MHM
58it compiles, there's still a good chance of the dynamic linker
59failing at C<make test> time. But that's on purpose!
60
61We can use these failures to find changes in the API automatically.
62The two Perl scripts F<devel/mktodo> and F<devel/mktodo.pl>
63repeatedly run C<Devel::PPPort> with the apicheck code through
64all different versions of perl. Scanning the output of the compiler
65and the dynamic linker for errors, the files in F<parts/todo/> are
66generated. These files list all parts of the public API that don't
67work with less than a certain version of Perl.
68
69This information is in turn used by F<parts/apicheck.pl> to mask
70API calls in the generated C file for these versions, so the
71process can be stopped by the time F<apicheck.c> compiles cleanly
ba120f6f
MHM
72and the dynamic linker is happy. (Actually, this process may generate
73false positives, so by default each API call is checked once more
74afterwards.)
adfe19db 75
5cde1e48 76Running F<devel/mktodo> takes about an hour, depending of course
ba120f6f
MHM
77on the machine you're running it on. If you run it with
78the C<--nocheck> option, it won't recheck the API calls that failed
79in the compilation stage and it'll take significantly less time.
80Running with C<--nocheck> should usually be safe.
adfe19db 81
5cde1e48 82When running F<devel/mktodo> with the C<--base> option, it will
adfe19db
MHM
83generate the I<baseline> todo files by disabling all functionality
84provided by C<Devel::PPPort>. These are required for implementing
5cde1e48 85the C<--compat-version> option of the F<ppport.h> script. The
adfe19db
MHM
86baseline todo files hold the information about which version of
87Perl lacks a certain part of the API.
88
89However, only the documented public API can be checked this way.
90And since C<Devel::PPPort> provides more macros, these would not be
91affected by C<--compat-version>. It's the job of F<devel/scanprov>
92to figure out the baseline information for all remaining provided
93macros by scanning the include files in the F<CORE> directory of
94various Perl versions.
95
ba120f6f
MHM
96The whole process isn't platform independent. It has currently been
97tested only under Linux, and it definitely requires at least C<gcc> and
98the C<nm> utility.
99
100It's not very often that one has to regenerate the baseline and todo
101files. If you have to, you can either run F<devel/regenerate> or just
102execute the following steps by hand:
adfe19db
MHM
103
104=over 4
105
106=item *
107
108You need a whole bunch of different Perls. The more, the better.
109You can use F<devel/buildperl.pl> to build them. I keep my perls
110in F</tmp/perl>, so most of the tools take this as a default.
111
112=item *
113
56093a11
MHM
114You also need a freshly built bleadperl that is in the path under
115exactly this name. (The name of the executable is currently hardcoded
116in F<devel/mktodo> and F<devel/scanprov>.)
117
118=item *
119
adfe19db
MHM
120Remove all existing todo files in the F<parts/base> and
121F<parts/todo> directories.
122
123=item *
124
125Update the API information. Copy the latest F<embed.fnc> file from
126bleadperl to the F<parts> directory and run F<devel/mkapidoc.sh> to
127collect the remaining information in F<parts/apidoc.fnc>.
128
129=item *
130
131Build the new baseline by running
132
133 perl devel/mktodo --base
134
135in the root directory of the distribution. When it's finished,
136move all files from the F<parts/todo> directory to F<parts/base>.
137
138=item *
139
140Build the new todo files by running
141
142 perl devel/mktodo
143
144in the root directory of the distribution.
145
146=item *
147
148Finally, add the remaining baseline information by running
149
4a582685 150 perl Makefile.PL && make
49ef49fe 151 perl devel/scanprov --mode=write
adfe19db
MHM
152
153=back
154
155=head2 Implementation
156
157Residing in F<parts/inc/> is the "heart" of C<Devel::PPPort>. Each
158of the files implements a part of the supported API, along with
159hints, dependency information, XS code and tests.
160The files are in a POD-like format that is parsed using the
161functions in F<parts/ppptools.pl>.
162
3bfe1c68 163The scripts F<PPPort_pm.PL>, F<RealPPPort_xs.PL> and F<mktests.PL> all
adfe19db 164use the information in F<parts/inc/> to generate the main module
1d088ed8 165F<PPPort.pm>, the XS code in F<RealPPPort.xs> and various test files
adfe19db
MHM
166in F<t/>.
167
168All of these files could be generated on the fly while building
5cde1e48 169C<Devel::PPPort>, but not having the tests in F<t/> will confuse
1d088ed8
MHM
170TEST/harness in the core. Not having F<PPPort.pm> will be bad for
171viewing the docs on C<search.cpan.org>. So unfortunately, it's
172unavoidable to put some redundancy into the package.
adfe19db
MHM
173
174=head2 Adding stuff to Devel::PPPort
175
176First, check if the code you plan to add fits into one of the
177existing files in F<parts/inc/>. If not, just start a new one and
178remember to include it from within F<PPPort_pm.PL>.
179
180Each file holds all relevant data for implementing a certain part
181of the API:
182
183=over 2
184
185=item *
186
187A list of the provided API in the C<=provides> section.
188
189=item *
190
191The implementation to add to F<ppport.h> in the C<=implementation>
192section.
193
9712ab2b
KW
194Here you can add additional information for a given item that will be displayed
195when F<ppport.h> is run. If your item is named C<foo>, you add a
196comment like so:
197
198 /* Hint: foo
199 paragraphs of stuff about foo you want to have
200 shown when ppport.h outputs something about foo
201 */
202
203This will cause S<C<perl ppport.h>> to display this hint when it outputs
204something about C<foo>.
205
206A more serious caution about C<foo> can be displayed by instead saying
207
208 /* Warning: foo
209 paragraphs of stuff about foo you want to have
210 shown when ppport.h outputs something about foo
211 */
212
adfe19db
MHM
213=item *
214
215The code required to add to PPPort.xs for testing the implementation.
216This code goes into the C<=xshead>, C<=xsinit>, C<=xsmisc>, C<=xsboot>
0d0f8426 217and C<=xsubs> section. Have a look at the template at the bottom
3bfe1c68 218of F<RealPPPort_xs.PL> to see where the code ends up.
adfe19db
MHM
219
220=item *
221
222The tests in the C<=tests> section. Remember not to use any fancy
223modules or syntax elements, as the test code should be able to run
224with Perl 5.003, which, for example, doesn't support C<my> in
225C<for>-loops:
226
0d0f8426 227 for my $x (1, 2, 3) { } # won't work with 5.003
adfe19db 228
0d0f8426
MHM
229You can use C<ok()> to report success or failure:
230
231 ok($got == 42);
232 ok($got, $expected);
233
234Regular expressions are not supported as the second argument to C<ok>,
235because older perls do not support the C<qr> operator.
adfe19db
MHM
236
237=back
238
239It's usually the best approach to just copy an existing file and
240use it as a template.
241
0d0f8426
MHM
242=head2 Implementation Hints
243
244In the C<=implementation> section, you can use
245
246 __UNDEFINED__ macro some definition
247
248instead of
249
250 #ifndef macro
251 # define macro some definition
252 #endif
253
254The macro can have optional arguments and the definition can even
255span multiple lines, like in
256
257 __UNDEFINED__ SvMAGIC_set(sv, val) \
258 STMT_START { assert(SvTYPE(sv) >= SVt_PVMG); \
259 (((XPVMG*) SvANY(sv))->xmg_magic = (val)); } STMT_END
260
960a24c4
KW
261This usually makes the code more compact and readable. And you only have to add
262the line C<__UNDEFINED__> to the C<=provides> section to get all macros
263implemented this way to be imported into this section, so they all get
264documented as being provided.
0d0f8426 265
3451ddaf
P
266If you need to define a helper macro which is not part of C<Devel::PPPort> API
267and its usage is only for definition of other C<Devel::PPPort> macros then use
268the C<D_PPP_> prefix for this macro name (e.g. C<D_PPP_SVPV_NOLEN_LP_ARG>).
269Some C<Devel::PPPort> scripts would report a warning when there is defined a
270macro which is not part of Perl public API, except those macros which start
271with the C<D_PPP_> prefix.
272
0d0f8426
MHM
273Version checking can be tricky if you want to do it correct.
274You can use
275
276 #if { VERSION < 5.9.3 }
277
278instead of
279
280 #if ((PERL_VERSION < 9) || (PERL_VERSION == 9 && PERL_SUBVERSION < 3))
281
282The version number can be either of the new form C<5.x.x> or of the older
283form C<5.00x_yy>. Both are translated into the correct preprocessor
284statements. It is also possible to combine this with other statements:
285
286 #if { VERSION >= 5.004 } && !defined(sv_vcatpvf)
b2049988 287 /* a */
0d0f8426
MHM
288 #elif { VERSION < 5.004_63 } && { VERSION != 5.004_05 }
289 /* b */
290 #endif
291
292This not only works in the C<=implementation> section, but also in
293the C<=xsubs>, C<=xsinit>, C<=xsmisc>, C<=xshead> and C<=xsboot> sections.
294
adfe19db
MHM
295=head2 Testing
296
297To automatically test C<Devel::PPPort> with lots of different Perl
298versions, you can use the F<soak> script. Just pass it a list of
299all Perl binaries you want to test.
300
301=head2 Special Makefile targets
302
4a582685 303You can use
adfe19db
MHM
304
305 make regen
306
4a582685
NC
307to regenerate all of the autogenerated files. To get rid of all
308generated files (except for F<parts/todo/*> and F<parts/base/*>),
309use
adfe19db
MHM
310
311 make purge_all
312
313That's it.
314
0d0f8426
MHM
315=head2 Submitting Patches
316
317If you've added some functionality to C<Devel::PPPort>, please
c601e8c7
SA
318consider submitting a patch with your work to P5P by sending a mail
319L<perlbug@perl.org|mailto:perlbug@perl.org>.
0d0f8426
MHM
320
321When submitting patches, please only add the relevant changes
322and don't include the differences of the generated files. You
323can use the C<purge_all> target to delete all autogenerated
324files.
325
56093a11
MHM
326=head2 Integrating into the Perl core
327
328When integrating this module into the Perl core, be sure to
329remove the following files from the distribution. They are
330either not needed or generated on the fly when building this
331module in the core:
332
333 MANIFEST
334 META.yml
335 PPPort.pm
336
adfe19db
MHM
337=head1 COPYRIGHT
338
b2049988 339Version 3.x, Copyright (C) 2004-2013, Marcus Holland-Moritz.
adfe19db
MHM
340
341Version 2.x, Copyright (C) 2001, Paul Marquess.
342
343Version 1.x, Copyright (C) 1999, Kenneth Albanowski.
344
345This program is free software; you can redistribute it and/or
346modify it under the same terms as Perl itself.
347
348=head1 SEE ALSO
349
5cde1e48 350See F<ppport.h> and F<devel/regenerate>.
adfe19db
MHM
351
352=cut