This is a live mirror of the Perl 5 development currently hosted at https://github.com/perl/perl5
make require report too many open files error
[perl5.git] / ext / Devel / PPPort / HACKERS
CommitLineData
adfe19db
MHM
1=head1 NAME
2
3HACKERS - Devel::PPPort internals for hackers
4
5=head1 SYNOPSIS
6
7So you probably want to hack C<Devel::PPPort>?
8
9Well, here's some information to get you started with what's
10lying around in this distribution.
11
12=head1 DESCRIPTION
13
f2ab5a41 14=head2 How to build 105 versions of Perl
adfe19db
MHM
15
16C<Devel::PPPort> supports Perl versions between 5.003 and bleadperl.
17To guarantee this support, I need some of these versions on my
f2ab5a41 18machine. I currently have 105 different Perl version/configuration
adfe19db
MHM
19combinations installed on my laptop.
20
21As many of the old Perl distributions need patching to compile
f2ab5a41 22cleanly on newer systems (and because building 105 Perls by hand
adfe19db
MHM
23just isn't fun), I wrote a tool to build all the different
24versions and configurations. You can find it in F<devel/buildperl.pl>.
25It can currently build the following Perl releases:
26
27 5.003
28 5.004 - 5.004_05
29 5.005 - 5.005_04
30 5.6.x
31 5.7.x
32 5.8.x
33 5.9.x
34
35=head2 Fully automatic API checks
36
37Knowing which parts of the API are not backwards compatible and
38probably need C<Devel::PPPort> support is another problem that's
39not easy to deal with manually. If you run
40
41 perl Makefile.PL --with-apicheck
42
43a C file is generated by F<parts/apicheck.pl> that is compiled
44and linked with C<Devel::PPPort>. This C file has the purpose of
45using each of the public API functions/macros once.
46
47The required information is derived from C<parts/embed.fnc> (just
48a copy of bleadperl's C<embed.fnc>) and C<parts/apidoc.fnc> (which
49is generated by F<devel/mkapidoc.sh> and simply collects the rest
50of the apidoc entries spread over the Perl source code).
51The generated C file C<apicheck.c> is currently about 500k in size
52and takes quite a while to compile.
53
54Usually, C<apicheck.c> won't compile with older perls. And even if
55it compiles, there's still a good chance of the dynamic linker
56failing at C<make test> time. But that's on purpose!
57
58We can use these failures to find changes in the API automatically.
59The two Perl scripts F<devel/mktodo> and F<devel/mktodo.pl>
60repeatedly run C<Devel::PPPort> with the apicheck code through
61all different versions of perl. Scanning the output of the compiler
62and the dynamic linker for errors, the files in F<parts/todo/> are
63generated. These files list all parts of the public API that don't
64work with less than a certain version of Perl.
65
66This information is in turn used by F<parts/apicheck.pl> to mask
67API calls in the generated C file for these versions, so the
68process can be stopped by the time F<apicheck.c> compiles cleanly
69and the dynamic linker is happy. (Actually, this process generates
70false positives, so each API call is checked once more afterwards.)
71
72Running C<devel/mktodo> takes a couple of hours.
73
74When running C<devel/mktodo> with the C<--base> option, it will
75generate the I<baseline> todo files by disabling all functionality
76provided by C<Devel::PPPort>. These are required for implementing
77the C<--compat-version> option of the C<ppport.h> script. The
78baseline todo files hold the information about which version of
79Perl lacks a certain part of the API.
80
81However, only the documented public API can be checked this way.
82And since C<Devel::PPPort> provides more macros, these would not be
83affected by C<--compat-version>. It's the job of F<devel/scanprov>
84to figure out the baseline information for all remaining provided
85macros by scanning the include files in the F<CORE> directory of
86various Perl versions.
87
88It's not very often that one has to regenerate the baseline and
89todo files, and the process hasn't been automated yet, but it's
90basically only the following steps:
91
92=over 4
93
94=item *
95
96You need a whole bunch of different Perls. The more, the better.
97You can use F<devel/buildperl.pl> to build them. I keep my perls
98in F</tmp/perl>, so most of the tools take this as a default.
99
100=item *
101
102Remove all existing todo files in the F<parts/base> and
103F<parts/todo> directories.
104
105=item *
106
107Update the API information. Copy the latest F<embed.fnc> file from
108bleadperl to the F<parts> directory and run F<devel/mkapidoc.sh> to
109collect the remaining information in F<parts/apidoc.fnc>.
110
111=item *
112
113Build the new baseline by running
114
115 perl devel/mktodo --base
116
117in the root directory of the distribution. When it's finished,
118move all files from the F<parts/todo> directory to F<parts/base>.
119
120=item *
121
122Build the new todo files by running
123
124 perl devel/mktodo
125
126in the root directory of the distribution.
127
128=item *
129
130Finally, add the remaining baseline information by running
131
4a582685
NC
132 perl Makefile.PL && make
133 perl devel/scanprov write
adfe19db
MHM
134
135=back
136
0c96388f
MHM
137Alternatively, you can try to use the F<devel/regenerate> script.
138
adfe19db
MHM
139=head2 Implementation
140
141Residing in F<parts/inc/> is the "heart" of C<Devel::PPPort>. Each
142of the files implements a part of the supported API, along with
143hints, dependency information, XS code and tests.
144The files are in a POD-like format that is parsed using the
145functions in F<parts/ppptools.pl>.
146
147The scripts F<PPPort_pm.PL>, F<PPPort_xs.PL> and F<mktests.PL> all
148use the information in F<parts/inc/> to generate the main module
149F<PPPort.pm>, the XS code in F<PPPort.xs> and various test files
150in F<t/>.
151
152All of these files could be generated on the fly while building
153C<Devel::PPPort>, but not having the tests in C<t/> and not having
154F<PPPort.xs> will confuse Configure and TEST/harness in the core.
155Not having F<PPPort.pm> will be bad for viewing the docs on
156C<search.cpan.org>. So unfortunately, it's unavoidable to put
157some redundancy into the package.
158
159=head2 Adding stuff to Devel::PPPort
160
161First, check if the code you plan to add fits into one of the
162existing files in F<parts/inc/>. If not, just start a new one and
163remember to include it from within F<PPPort_pm.PL>.
164
165Each file holds all relevant data for implementing a certain part
166of the API:
167
168=over 2
169
170=item *
171
172A list of the provided API in the C<=provides> section.
173
174=item *
175
176The implementation to add to F<ppport.h> in the C<=implementation>
177section.
178
179=item *
180
181The code required to add to PPPort.xs for testing the implementation.
182This code goes into the C<=xshead>, C<=xsinit>, C<=xsmisc>, C<=xsboot>
0d0f8426
MHM
183and C<=xsubs> section. Have a look at the template at the bottom
184of F<PPPort_xs.PL> to see where the code ends up.
adfe19db
MHM
185
186=item *
187
188The tests in the C<=tests> section. Remember not to use any fancy
189modules or syntax elements, as the test code should be able to run
190with Perl 5.003, which, for example, doesn't support C<my> in
191C<for>-loops:
192
0d0f8426 193 for my $x (1, 2, 3) { } # won't work with 5.003
adfe19db 194
0d0f8426
MHM
195You can use C<ok()> to report success or failure:
196
197 ok($got == 42);
198 ok($got, $expected);
199
200Regular expressions are not supported as the second argument to C<ok>,
201because older perls do not support the C<qr> operator.
adfe19db
MHM
202
203=back
204
205It's usually the best approach to just copy an existing file and
206use it as a template.
207
0d0f8426
MHM
208=head2 Implementation Hints
209
210In the C<=implementation> section, you can use
211
212 __UNDEFINED__ macro some definition
213
214instead of
215
216 #ifndef macro
217 # define macro some definition
218 #endif
219
220The macro can have optional arguments and the definition can even
221span multiple lines, like in
222
223 __UNDEFINED__ SvMAGIC_set(sv, val) \
224 STMT_START { assert(SvTYPE(sv) >= SVt_PVMG); \
225 (((XPVMG*) SvANY(sv))->xmg_magic = (val)); } STMT_END
226
227This usually makes the code more compact and readable. And you
228only have to add C<__UNDEFINED__> to the C<=provided> section.
229
230Version checking can be tricky if you want to do it correct.
231You can use
232
233 #if { VERSION < 5.9.3 }
234
235instead of
236
237 #if ((PERL_VERSION < 9) || (PERL_VERSION == 9 && PERL_SUBVERSION < 3))
238
239The version number can be either of the new form C<5.x.x> or of the older
240form C<5.00x_yy>. Both are translated into the correct preprocessor
241statements. It is also possible to combine this with other statements:
242
243 #if { VERSION >= 5.004 } && !defined(sv_vcatpvf)
244 /* a */
245 #elif { VERSION < 5.004_63 } && { VERSION != 5.004_05 }
246 /* b */
247 #endif
248
249This not only works in the C<=implementation> section, but also in
250the C<=xsubs>, C<=xsinit>, C<=xsmisc>, C<=xshead> and C<=xsboot> sections.
251
adfe19db
MHM
252=head2 Testing
253
254To automatically test C<Devel::PPPort> with lots of different Perl
255versions, you can use the F<soak> script. Just pass it a list of
256all Perl binaries you want to test.
257
258=head2 Special Makefile targets
259
4a582685 260You can use
adfe19db
MHM
261
262 make regen
263
4a582685
NC
264to regenerate all of the autogenerated files. To get rid of all
265generated files (except for F<parts/todo/*> and F<parts/base/*>),
266use
adfe19db
MHM
267
268 make purge_all
269
270That's it.
271
0d0f8426
MHM
272=head2 Submitting Patches
273
274If you've added some functionality to C<Devel::PPPort>, please
275consider submitting a patch with your work to either the author
276(E<lt>mhx@cpan.orgE<gt>) or to the CPAN Request Tracker at
277L<http://rt.cpan.org>.
278
279When submitting patches, please only add the relevant changes
280and don't include the differences of the generated files. You
281can use the C<purge_all> target to delete all autogenerated
282files.
283
adfe19db
MHM
284=head1 COPYRIGHT
285
0d0f8426 286Version 3.x, Copyright (C) 2004-2006, Marcus Holland-Moritz.
adfe19db
MHM
287
288Version 2.x, Copyright (C) 2001, Paul Marquess.
289
290Version 1.x, Copyright (C) 1999, Kenneth Albanowski.
291
292This program is free software; you can redistribute it and/or
293modify it under the same terms as Perl itself.
294
295=head1 SEE ALSO
296
297See L<ppport.h>.
298
299=cut
300