Commit | Line | Data |
---|---|---|
0061d4fa | 1 | |
04c692a8 DR |
2 | =encoding utf8 |
3 | ||
4 | =for comment | |
5 | Consistent formatting of this file is achieved with: | |
6 | perl ./Porting/podtidy pod/perlhacktips.pod | |
7 | ||
8 | =head1 NAME | |
9 | ||
10 | perlhacktips - Tips for Perl core C code hacking | |
11 | ||
12 | =head1 DESCRIPTION | |
13 | ||
14 | This document will help you learn the best way to go about hacking on | |
9b22382a | 15 | the Perl core C code. It covers common problems, debugging, profiling, |
04c692a8 DR |
16 | and more. |
17 | ||
18 | If you haven't read L<perlhack> and L<perlhacktut> yet, you might want | |
19 | to do that first. | |
20 | ||
21 | =head1 COMMON PROBLEMS | |
22 | ||
a66ca998 NC |
23 | Perl source now permits some specific C99 features which we know are |
24 | supported by all platforms, but mostly plays by ANSI C89 rules. | |
04c692a8 DR |
25 | You don't care about some particular platform having broken Perl? I |
26 | hear there is still a strong demand for J2EE programmers. | |
27 | ||
28 | =head2 Perl environment problems | |
29 | ||
30 | =over 4 | |
31 | ||
32 | =item * | |
33 | ||
34 | Not compiling with threading | |
35 | ||
36 | Compiling with threading (-Duseithreads) completely rewrites the | |
9b22382a | 37 | function prototypes of Perl. You better try your changes with that. |
04c692a8 DR |
38 | Related to this is the difference between "Perl_-less" and "Perl_-ly" |
39 | APIs, for example: | |
40 | ||
41 | Perl_sv_setiv(aTHX_ ...); | |
42 | sv_setiv(...); | |
43 | ||
44 | The first one explicitly passes in the context, which is needed for | |
9b22382a FC |
45 | e.g. threaded builds. The second one does that implicitly; do not get |
46 | them mixed. If you are not passing in a aTHX_, you will need to do a | |
c91f661c | 47 | dTHX as the first thing in the function. |
04c692a8 DR |
48 | |
49 | See L<perlguts/"How multiple interpreters and concurrency are | |
50 | supported"> for further discussion about context. | |
51 | ||
52 | =item * | |
53 | ||
54 | Not compiling with -DDEBUGGING | |
55 | ||
56 | The DEBUGGING define exposes more code to the compiler, therefore more | |
9b22382a | 57 | ways for things to go wrong. You should try it. |
04c692a8 DR |
58 | |
59 | =item * | |
60 | ||
61 | Introducing (non-read-only) globals | |
62 | ||
63 | Do not introduce any modifiable globals, truly global or file static. | |
64 | They are bad form and complicate multithreading and other forms of | |
9b22382a | 65 | concurrency. The right way is to introduce them as new interpreter |
04c692a8 DR |
66 | variables, see F<intrpvar.h> (at the very end for binary |
67 | compatibility). | |
68 | ||
69 | Introducing read-only (const) globals is okay, as long as you verify | |
70 | with e.g. C<nm libperl.a|egrep -v ' [TURtr] '> (if your C<nm> has | |
9b22382a | 71 | BSD-style output) that the data you added really is read-only. (If it |
04c692a8 DR |
72 | is, it shouldn't show up in the output of that command.) |
73 | ||
74 | If you want to have static strings, make them constant: | |
75 | ||
76 | static const char etc[] = "..."; | |
77 | ||
78 | If you want to have arrays of constant strings, note carefully the | |
79 | right combination of C<const>s: | |
80 | ||
81 | static const char * const yippee[] = | |
a63ef199 | 82 | {"hi", "ho", "silver"}; |
04c692a8 | 83 | |
04c692a8 DR |
84 | =item * |
85 | ||
86 | Not exporting your new function | |
87 | ||
88 | Some platforms (Win32, AIX, VMS, OS/2, to name a few) require any | |
89 | function that is part of the public API (the shared Perl library) to be | |
9b22382a | 90 | explicitly marked as exported. See the discussion about F<embed.pl> in |
04c692a8 DR |
91 | L<perlguts>. |
92 | ||
93 | =item * | |
94 | ||
95 | Exporting your new function | |
96 | ||
97 | The new shiny result of either genuine new functionality or your | |
9b22382a | 98 | arduous refactoring is now ready and correctly exported. So what could |
04c692a8 DR |
99 | possibly go wrong? |
100 | ||
101 | Maybe simply that your function did not need to be exported in the | |
9b22382a | 102 | first place. Perl has a long and not so glorious history of exporting |
04c692a8 DR |
103 | functions that it should not have. |
104 | ||
105 | If the function is used only inside one source code file, make it | |
9b22382a | 106 | static. See the discussion about F<embed.pl> in L<perlguts>. |
04c692a8 DR |
107 | |
108 | If the function is used across several files, but intended only for | |
109 | Perl's internal use (and this should be the common case), do not export | |
9b22382a | 110 | it to the public API. See the discussion about F<embed.pl> in |
04c692a8 DR |
111 | L<perlguts>. |
112 | ||
113 | =back | |
114 | ||
a66ca998 NC |
115 | =head2 C99 |
116 | ||
117 | Starting from 5.35.5 we now permit some C99 features in the core C source. | |
118 | However, code in dual life extensions still needs to be C89 only, because it | |
119 | needs to compile against earlier version of Perl running on older platforms. | |
120 | Also note that our headers need to also be valid as C++, because XS extensions | |
121 | written in C++ need to include them, hence I<member structure initialisers> | |
122 | can't be used in headers. | |
123 | ||
124 | C99 support is still far from complete on all platforms we currently support. | |
125 | As a baseline we can only assume C89 semantics with the specific C99 features | |
126 | described below, which we've verified work everywhere. It's fine to probe for | |
127 | additional C99 features and use them where available, providing there is also a | |
128 | fallback for compilers that don't support the feature. For example, we use C11 | |
129 | thread local storage when available, but fall back to POSIX thread specific | |
130 | APIs otherwise, and we use C<char> for booleans if C<< <stdbool.h> >> isn't | |
131 | available. | |
132 | ||
133 | Code can use (and rely on) the following C99 features being present | |
134 | ||
135 | =over | |
136 | ||
137 | =item * | |
138 | ||
139 | mixed declarations and code | |
140 | ||
141 | =item * | |
142 | ||
143 | 64 bit integer types | |
144 | ||
145 | For consistency with the existing source code, use the typedefs C<I64> and | |
146 | C<U64>, instead of using C<long long> and C<unsigned long long> directly. | |
147 | ||
148 | =item * | |
149 | ||
150 | variadic macros | |
151 | ||
152 | void greet(char *file, unsigned int line, char *format, ...); | |
153 | #define logged_greet(...) greet(__FILE__, __LINE__, __VA_ARGS__); | |
154 | ||
155 | Note that C<__VA_OPT__> is a gcc extension not yet in any published standard. | |
156 | ||
157 | =item * | |
158 | ||
159 | declarations in for loops | |
160 | ||
161 | for (const char *p = message; *p; ++p) { | |
162 | putchar(*p); | |
163 | } | |
164 | ||
165 | =item * | |
166 | ||
167 | member structure initialisers | |
168 | ||
169 | But not in headers, as support was only added to C++ relatively recently. | |
170 | ||
171 | Hence this is fine in C and XS code, but not headers: | |
172 | ||
173 | struct message { | |
174 | char *action; | |
175 | char *target; | |
176 | }; | |
d087776a | 177 | |
a66ca998 NC |
178 | struct message mcguffin = { |
179 | .target = "member structure initialisers", | |
180 | .action = "Built" | |
181 | }; | |
182 | ||
183 | =item * | |
184 | ||
185 | flexible array members | |
186 | ||
187 | This is standards conformant: | |
188 | ||
189 | struct greeting { | |
190 | unsigned int len; | |
191 | char message[]; | |
192 | }; | |
193 | ||
194 | However, the source code already uses the "unwarranted chumminess with the | |
195 | compiler" hack in many places: | |
196 | ||
197 | struct greeting { | |
198 | unsigned int len; | |
199 | char message[1]; | |
200 | }; | |
201 | ||
202 | Strictly it B<is> undefined behaviour accessing beyond C<message[0]>, but this | |
203 | has been a commonly used hack since K&R times, and using it hasn't been a | |
204 | practical issue anywhere (in the perl source or any other common C code). | |
205 | Hence it's unclear what we would gain from actively changing to the C99 | |
206 | approach. | |
207 | ||
208 | =item * | |
209 | ||
210 | C<//> comments | |
211 | ||
212 | All compilers we tested support their use. Not all humans we tested support | |
213 | their use. | |
214 | ||
215 | =back | |
216 | ||
217 | Code explicitly should not use any other C99 features. For example | |
218 | ||
219 | =over 4 | |
220 | ||
221 | =item * | |
222 | ||
223 | variable length arrays | |
224 | ||
225 | Not supported by B<any> MSVC, and this is not going to change. | |
226 | ||
227 | Even "variable" length arrays where the variable is a constant expression | |
228 | are syntax errors under MSVC. | |
229 | ||
230 | =item * | |
231 | ||
232 | C99 types in C<< <stdint.h> >> | |
233 | ||
234 | Use C<PERL_INT_FAST8_T> etc as defined in F<handy.h> | |
235 | ||
236 | =item * | |
237 | ||
a916f72f | 238 | C99 format strings in C<< <inttypes.h> >> |
a66ca998 NC |
239 | |
240 | C<snprintf> in the VMS libc only added support for C<PRIdN> etc very recently, | |
241 | meaning that there are live supported installations without this, or formats | |
242 | such as C<%zu>. | |
243 | ||
244 | (perl's C<sv_catpvf> etc use parser code code in C<sv.c>, which supports the | |
245 | C<z> modifier, along with perl-specific formats such as C<SVf>.) | |
246 | ||
247 | =back | |
248 | ||
135ed903 NC |
249 | If you want to use a C99 feature not listed above then you need to do one of |
250 | ||
251 | =over 4 | |
252 | ||
253 | =item * | |
254 | ||
255 | Probe for it in F<Configure>, set a variable in F<config.sh>, and add fallback logic in the headers for platforms which don't have it. | |
256 | ||
257 | =item * | |
258 | ||
259 | Write test code and verify that it works on platforms we need to support, before relying on it unconditionally. | |
260 | ||
261 | =back | |
262 | ||
263 | Likely you want to repeat the same plan as we used to get the current C99 | |
264 | feature set. See the message at https://markmail.org/thread/odr4fjrn72u2fkpz | |
265 | for the C99 probes we used before. Note that the two most "fussy" compilers | |
266 | appear to be MSVC and the vendor compiler on VMS. To date all the *nix | |
267 | compilers have been far more flexible in what they support. | |
268 | ||
a66ca998 NC |
269 | On *nix platforms, F<Configure> attempts to set compiler flags appropriately. |
270 | All vendor compilers that we tested defaulted to C99 (or C11) support. | |
271 | However, older versions of gcc default to C89, or permit I<most> C99 (with | |
272 | warnings), but forbid I<declarations in for loops> unless C<-std=gnu99> is | |
273 | added. The alternative C<-std=c99> B<might> seem better, but using it on some | |
274 | platforms can prevent C<< <unistd.h> >> declaring some prototypes being | |
275 | declared, which breaks the build. gcc's C<-ansi> flag implies C<-std=c89> so we | |
276 | can no longer set that, hence the Configure option C<-gccansipedantic> now only | |
277 | adds C<-pedantic>. | |
278 | ||
279 | The Perl core source code files (the ones at the top level of the source code | |
280 | distribution) are automatically compiled with as many as possible of the | |
281 | C<-std=gnu99>, C<-pedantic>, and a selection of C<-W> flags (see | |
282 | cflags.SH). Files in F<ext/> F<dist/> F<cpan/> etc are compiled with the same | |
283 | flags as the installed perl would use to compile XS extensions. | |
284 | ||
285 | Basically, it's safe to assume that F<Configure> and F<cflags.SH> have | |
286 | picked the best combination of flags for the version of gcc on the platform, | |
287 | and attempting to add more flags related to enforcing a C dialect will | |
288 | cause problems either locally, or on other systems that the code is shipped | |
289 | to. | |
290 | ||
291 | We believe that the C99 support in gcc 3.1 is good enough for us, but we don't | |
292 | have a 19 year old gcc handy to check this :-) | |
293 | If you have ancient vendor compilers that don't default to C99, the flags | |
294 | you might want to try are | |
295 | ||
296 | =over 4 | |
297 | ||
298 | =item AIX | |
299 | ||
300 | C<-qlanglvl=stdc99> | |
301 | ||
302 | =item HP/UX | |
303 | ||
304 | C<-AC99> | |
305 | ||
306 | =item Solaris | |
307 | ||
308 | C<-xc99> | |
309 | ||
310 | =back | |
311 | ||
04c692a8 DR |
312 | =head2 Portability problems |
313 | ||
314 | The following are common causes of compilation and/or execution | |
9b22382a FC |
315 | failures, not common to Perl as such. The C FAQ is good bedtime |
316 | reading. Please test your changes with as many C compilers and | |
04c692a8 DR |
317 | platforms as possible; we will, anyway, and it's nice to save oneself |
318 | from public embarrassment. | |
319 | ||
04c692a8 | 320 | Also study L<perlport> carefully to avoid any bad assumptions about the |
eb9df707 | 321 | operating system, filesystems, character set, and so forth. |
04c692a8 | 322 | |
04c692a8 DR |
323 | Do not assume an operating system indicates a certain compiler. |
324 | ||
325 | =over 4 | |
326 | ||
327 | =item * | |
328 | ||
329 | Casting pointers to integers or casting integers to pointers | |
330 | ||
331 | void castaway(U8* p) | |
332 | { | |
333 | IV i = p; | |
334 | ||
335 | or | |
336 | ||
337 | void castaway(U8* p) | |
338 | { | |
339 | IV i = (IV)p; | |
340 | ||
9b22382a FC |
341 | Both are bad, and broken, and unportable. Use the PTR2IV() macro that |
342 | does it right. (Likewise, there are PTR2UV(), PTR2NV(), INT2PTR(), and | |
04c692a8 DR |
343 | NUM2PTR().) |
344 | ||
345 | =item * | |
346 | ||
28ffebaf | 347 | Casting between function pointers and data pointers |
04c692a8 DR |
348 | |
349 | Technically speaking casting between function pointers and data | |
350 | pointers is unportable and undefined, but practically speaking it seems | |
351 | to work, but you should use the FPTR2DPTR() and DPTR2FPTR() macros. | |
352 | Sometimes you can also play games with unions. | |
353 | ||
354 | =item * | |
355 | ||
356 | Assuming sizeof(int) == sizeof(long) | |
357 | ||
358 | There are platforms where longs are 64 bits, and platforms where ints | |
359 | are 64 bits, and while we are out to shock you, even platforms where | |
9b22382a | 360 | shorts are 64 bits. This is all legal according to the C standard. (In |
04c692a8 DR |
361 | other words, "long long" is not a portable way to specify 64 bits, and |
362 | "long long" is not even guaranteed to be any wider than "long".) | |
363 | ||
364 | Instead, use the definitions IV, UV, IVSIZE, I32SIZE, and so forth. | |
365 | Avoid things like I32 because they are B<not> guaranteed to be | |
366 | I<exactly> 32 bits, they are I<at least> 32 bits, nor are they | |
a66ca998 NC |
367 | guaranteed to be B<int> or B<long>. If you explicitly need |
368 | 64-bit variables, use I64 and U64. | |
04c692a8 DR |
369 | |
370 | =item * | |
371 | ||
372 | Assuming one can dereference any type of pointer for any type of data | |
373 | ||
374 | char *p = ...; | |
56bb4b7b | 375 | long pony = *(long *)p; /* BAD */ |
04c692a8 DR |
376 | |
377 | Many platforms, quite rightly so, will give you a core dump instead of | |
768312ab | 378 | a pony if the p happens not to be correctly aligned. |
04c692a8 DR |
379 | |
380 | =item * | |
381 | ||
382 | Lvalue casts | |
383 | ||
384 | (int)*p = ...; /* BAD */ | |
385 | ||
9b22382a | 386 | Simply not portable. Get your lvalue to be of the right type, or maybe |
04c692a8 DR |
387 | use temporary variables, or dirty tricks with unions. |
388 | ||
389 | =item * | |
390 | ||
391 | Assume B<anything> about structs (especially the ones you don't | |
392 | control, like the ones coming from the system headers) | |
393 | ||
394 | =over 8 | |
395 | ||
396 | =item * | |
397 | ||
398 | That a certain field exists in a struct | |
399 | ||
400 | =item * | |
401 | ||
402 | That no other fields exist besides the ones you know of | |
403 | ||
404 | =item * | |
405 | ||
406 | That a field is of certain signedness, sizeof, or type | |
407 | ||
408 | =item * | |
409 | ||
410 | That the fields are in a certain order | |
411 | ||
412 | =over 8 | |
413 | ||
414 | =item * | |
415 | ||
416 | While C guarantees the ordering specified in the struct definition, | |
417 | between different platforms the definitions might differ | |
418 | ||
419 | =back | |
420 | ||
421 | =item * | |
422 | ||
423 | That the sizeof(struct) or the alignments are the same everywhere | |
424 | ||
425 | =over 8 | |
426 | ||
427 | =item * | |
428 | ||
429 | There might be padding bytes between the fields to align the fields - | |
430 | the bytes can be anything | |
431 | ||
432 | =item * | |
433 | ||
434 | Structs are required to be aligned to the maximum alignment required by | |
435 | the fields - which for native types is for usually equivalent to | |
436 | sizeof() of the field | |
437 | ||
438 | =back | |
439 | ||
440 | =back | |
441 | ||
442 | =item * | |
443 | ||
444 | Assuming the character set is ASCIIish | |
445 | ||
9b22382a | 446 | Perl can compile and run under EBCDIC platforms. See L<perlebcdic>. |
04c692a8 DR |
447 | This is transparent for the most part, but because the character sets |
448 | differ, you shouldn't use numeric (decimal, octal, nor hex) constants | |
eb9df707 KW |
449 | to refer to characters. You can safely say C<'A'>, but not C<0x41>. |
450 | You can safely say C<'\n'>, but not C<\012>. However, you can use | |
451 | macros defined in F<utf8.h> to specify any code point portably. | |
452 | C<LATIN1_TO_NATIVE(0xDF)> is going to be the code point that means | |
453 | LATIN SMALL LETTER SHARP S on whatever platform you are running on (on | |
454 | ASCII platforms it compiles without adding any extra code, so there is | |
455 | zero performance hit on those). The acceptable inputs to | |
456 | C<LATIN1_TO_NATIVE> are from C<0x00> through C<0xFF>. If your input | |
457 | isn't guaranteed to be in that range, use C<UNICODE_TO_NATIVE> instead. | |
458 | C<NATIVE_TO_LATIN1> and C<NATIVE_TO_UNICODE> translate the opposite | |
459 | direction. | |
460 | ||
461 | If you need the string representation of a character that doesn't have a | |
462 | mnemonic name in C, you should add it to the list in | |
c22aa07d | 463 | F<regen/unicode_constants.pl>, and have Perl create C<#define>'s for you, |
eb6d698b | 464 | based on the current platform. |
04c692a8 | 465 | |
eb9df707 KW |
466 | Note that the C<isI<FOO>> and C<toI<FOO>> macros in F<handy.h> work |
467 | properly on native code points and strings. | |
468 | ||
04c692a8 | 469 | Also, the range 'A' - 'Z' in ASCII is an unbroken sequence of 26 upper |
9b22382a FC |
470 | case alphabetic characters. That is not true in EBCDIC. Nor for 'a' to |
471 | 'z'. But '0' - '9' is an unbroken range in both systems. Don't assume | |
c22aa07d | 472 | anything about other ranges. (Note that special handling of ranges in |
f4240379 | 473 | regular expression patterns and transliterations makes it appear to Perl |
c22aa07d | 474 | code that the aforementioned ranges are all unbroken.) |
04c692a8 DR |
475 | |
476 | Many of the comments in the existing code ignore the possibility of | |
9b22382a | 477 | EBCDIC, and may be wrong therefore, even if the code works. This is |
04c692a8 DR |
478 | actually a tribute to the successful transparent insertion of being |
479 | able to handle EBCDIC without having to change pre-existing code. | |
480 | ||
481 | UTF-8 and UTF-EBCDIC are two different encodings used to represent | |
9b22382a | 482 | Unicode code points as sequences of bytes. Macros with the same names |
eb9df707 | 483 | (but different definitions) in F<utf8.h> and F<utfebcdic.h> are used to |
04c692a8 DR |
484 | allow the calling code to think that there is only one such encoding. |
485 | This is almost always referred to as C<utf8>, but it means the EBCDIC | |
9b22382a | 486 | version as well. Again, comments in the code may well be wrong even if |
eb9df707 | 487 | the code itself is right. For example, the concept of UTF-8 C<invariant |
9b22382a FC |
488 | characters> differs between ASCII and EBCDIC. On ASCII platforms, only |
489 | characters that do not have the high-order bit set (i.e. whose ordinals | |
04c692a8 DR |
490 | are strict ASCII, 0 - 127) are invariant, and the documentation and |
491 | comments in the code may assume that, often referring to something | |
9b22382a | 492 | like, say, C<hibit>. The situation differs and is not so simple on |
04c692a8 DR |
493 | EBCDIC machines, but as long as the code itself uses the |
494 | C<NATIVE_IS_INVARIANT()> macro appropriately, it works, even if the | |
495 | comments are wrong. | |
496 | ||
257844b9 KW |
497 | As noted in L<perlhack/TESTING>, when writing test scripts, the file |
498 | F<t/charset_tools.pl> contains some helpful functions for writing tests | |
499 | valid on both ASCII and EBCDIC platforms. Sometimes, though, a test | |
500 | can't use a function and it's inconvenient to have different test | |
501 | versions depending on the platform. There are 20 code points that are | |
502 | the same in all 4 character sets currently recognized by Perl (the 3 | |
503 | EBCDIC code pages plus ISO 8859-1 (ASCII/Latin1)). These can be used in | |
504 | such tests, though there is a small possibility that Perl will become | |
505 | available in yet another character set, breaking your test. All but one | |
506 | of these code points are C0 control characters. The most significant | |
507 | controls that are the same are C<\0>, C<\r>, and C<\N{VT}> (also | |
508 | specifiable as C<\cK>, C<\x0B>, C<\N{U+0B}>, or C<\013>). The single | |
509 | non-control is U+00B6 PILCROW SIGN. The controls that are the same have | |
510 | the same bit pattern in all 4 character sets, regardless of the UTF8ness | |
511 | of the string containing them. The bit pattern for U+B6 is the same in | |
512 | all 4 for non-UTF8 strings, but differs in each when its containing | |
513 | string is UTF-8 encoded. The only other code points that have some sort | |
514 | of sameness across all 4 character sets are the pair 0xDC and 0xFC. | |
515 | Together these represent upper- and lowercase LATIN LETTER U WITH | |
516 | DIAERESIS, but which is upper and which is lower may be reversed: 0xDC | |
517 | is the capital in Latin1 and 0xFC is the small letter, while 0xFC is the | |
518 | capital in EBCDIC and 0xDC is the small one. This factoid may be | |
519 | exploited in writing case insensitive tests that are the same across all | |
520 | 4 character sets. | |
521 | ||
04c692a8 DR |
522 | =item * |
523 | ||
524 | Assuming the character set is just ASCII | |
525 | ||
9b22382a | 526 | ASCII is a 7 bit encoding, but bytes have 8 bits in them. The 128 extra |
04c692a8 DR |
527 | characters have different meanings depending on the locale. Absent a |
528 | locale, currently these extra characters are generally considered to be | |
eb9df707 KW |
529 | unassigned, and this has presented some problems. This has being |
530 | changed starting in 5.12 so that these characters can be considered to | |
531 | be Latin-1 (ISO-8859-1). | |
04c692a8 DR |
532 | |
533 | =item * | |
534 | ||
535 | Mixing #define and #ifdef | |
536 | ||
537 | #define BURGLE(x) ... \ | |
538 | #ifdef BURGLE_OLD_STYLE /* BAD */ | |
539 | ... do it the old way ... \ | |
540 | #else | |
541 | ... do it the new way ... \ | |
542 | #endif | |
543 | ||
9b22382a | 544 | You cannot portably "stack" cpp directives. For example in the above |
04c692a8 DR |
545 | you need two separate BURGLE() #defines, one for each #ifdef branch. |
546 | ||
547 | =item * | |
548 | ||
549 | Adding non-comment stuff after #endif or #else | |
550 | ||
551 | #ifdef SNOSH | |
552 | ... | |
553 | #else !SNOSH /* BAD */ | |
554 | ... | |
555 | #endif SNOSH /* BAD */ | |
556 | ||
557 | The #endif and #else cannot portably have anything non-comment after | |
9b22382a | 558 | them. If you want to document what is going (which is a good idea |
04c692a8 DR |
559 | especially if the branches are long), use (C) comments: |
560 | ||
561 | #ifdef SNOSH | |
562 | ... | |
563 | #else /* !SNOSH */ | |
564 | ... | |
565 | #endif /* SNOSH */ | |
566 | ||
567 | The gcc option C<-Wendif-labels> warns about the bad variant (by | |
568 | default on starting from Perl 5.9.4). | |
569 | ||
570 | =item * | |
571 | ||
572 | Having a comma after the last element of an enum list | |
573 | ||
574 | enum color { | |
575 | CERULEAN, | |
576 | CHARTREUSE, | |
577 | CINNABAR, /* BAD */ | |
578 | }; | |
579 | ||
9b22382a | 580 | is not portable. Leave out the last comma. |
04c692a8 DR |
581 | |
582 | Also note that whether enums are implicitly morphable to ints varies | |
583 | between compilers, you might need to (int). | |
584 | ||
585 | =item * | |
586 | ||
04c692a8 DR |
587 | Mixing signed char pointers with unsigned char pointers |
588 | ||
589 | int foo(char *s) { ... } | |
590 | ... | |
591 | unsigned char *t = ...; /* Or U8* t = ... */ | |
592 | foo(t); /* BAD */ | |
593 | ||
594 | While this is legal practice, it is certainly dubious, and downright | |
595 | fatal in at least one platform: for example VMS cc considers this a | |
9b22382a | 596 | fatal error. One cause for people often making this mistake is that a |
04c692a8 DR |
597 | "naked char" and therefore dereferencing a "naked char pointer" have an |
598 | undefined signedness: it depends on the compiler and the flags of the | |
599 | compiler and the underlying platform whether the result is signed or | |
9b22382a | 600 | unsigned. For this very same reason using a 'char' as an array index is |
04c692a8 DR |
601 | bad. |
602 | ||
603 | =item * | |
604 | ||
605 | Macros that have string constants and their arguments as substrings of | |
606 | the string constants | |
607 | ||
608 | #define FOO(n) printf("number = %d\n", n) /* BAD */ | |
609 | FOO(10); | |
610 | ||
611 | Pre-ANSI semantics for that was equivalent to | |
612 | ||
613 | printf("10umber = %d\10"); | |
614 | ||
9b22382a | 615 | which is probably not what you were expecting. Unfortunately at least |
04c692a8 DR |
616 | one reasonably common and modern C compiler does "real backward |
617 | compatibility" here, in AIX that is what still happens even though the | |
618 | rest of the AIX compiler is very happily C89. | |
619 | ||
620 | =item * | |
621 | ||
622 | Using printf formats for non-basic C types | |
623 | ||
624 | IV i = ...; | |
625 | printf("i = %d\n", i); /* BAD */ | |
626 | ||
627 | While this might by accident work in some platform (where IV happens to | |
9b22382a | 628 | be an C<int>), in general it cannot. IV might be something larger. Even |
04c692a8 DR |
629 | worse the situation is with more specific types (defined by Perl's |
630 | configuration step in F<config.h>): | |
631 | ||
632 | Uid_t who = ...; | |
633 | printf("who = %d\n", who); /* BAD */ | |
634 | ||
635 | The problem here is that Uid_t might be not only not C<int>-wide but it | |
636 | might also be unsigned, in which case large uids would be printed as | |
637 | negative values. | |
638 | ||
639 | There is no simple solution to this because of printf()'s limited | |
640 | intelligence, but for many types the right format is available as with | |
641 | either 'f' or '_f' suffix, for example: | |
642 | ||
643 | IVdf /* IV in decimal */ | |
644 | UVxf /* UV is hexadecimal */ | |
645 | ||
646 | printf("i = %"IVdf"\n", i); /* The IVdf is a string constant. */ | |
647 | ||
648 | Uid_t_f /* Uid_t in decimal */ | |
649 | ||
650 | printf("who = %"Uid_t_f"\n", who); | |
651 | ||
652 | Or you can try casting to a "wide enough" type: | |
653 | ||
654 | printf("i = %"IVdf"\n", (IV)something_very_small_and_signed); | |
655 | ||
9dd1a77d KW |
656 | See L<perlguts/Formatted Printing of Size_t and SSize_t> for how to |
657 | print those. | |
658 | ||
04c692a8 DR |
659 | Also remember that the C<%p> format really does require a void pointer: |
660 | ||
661 | U8* p = ...; | |
662 | printf("p = %p\n", (void*)p); | |
663 | ||
664 | The gcc option C<-Wformat> scans for such problems. | |
665 | ||
666 | =item * | |
667 | ||
04c692a8 DR |
668 | Blindly passing va_list |
669 | ||
670 | Not all platforms support passing va_list to further varargs (stdarg) | |
9b22382a | 671 | functions. The right thing to do is to copy the va_list using the |
04c692a8 DR |
672 | Perl_va_copy() if the NEED_VA_COPY is defined. |
673 | ||
674 | =item * | |
675 | ||
676 | Using gcc statement expressions | |
677 | ||
678 | val = ({...;...;...}); /* BAD */ | |
679 | ||
91c7e172 | 680 | While a nice extension, it's not portable. Historically, Perl used |
a66ca998 NC |
681 | them in macros if available to gain some extra speed (essentially |
682 | as a funky form of inlining), but we now support (or emulate) C99 | |
683 | C<static inline> functions, so use them instead. Declare functions as | |
684 | C<PERL_STATIC_INLINE> to transparently fall back to emulation where needed. | |
04c692a8 DR |
685 | |
686 | =item * | |
687 | ||
688 | Binding together several statements in a macro | |
689 | ||
690 | Use the macros STMT_START and STMT_END. | |
691 | ||
692 | STMT_START { | |
693 | ... | |
694 | } STMT_END | |
695 | ||
696 | =item * | |
697 | ||
698 | Testing for operating systems or versions when should be testing for | |
699 | features | |
700 | ||
701 | #ifdef __FOONIX__ /* BAD */ | |
702 | foo = quux(); | |
703 | #endif | |
704 | ||
705 | Unless you know with 100% certainty that quux() is only ever available | |
706 | for the "Foonix" operating system B<and> that is available B<and> | |
707 | correctly working for B<all> past, present, B<and> future versions of | |
9b22382a | 708 | "Foonix", the above is very wrong. This is more correct (though still |
04c692a8 DR |
709 | not perfect, because the below is a compile-time check): |
710 | ||
711 | #ifdef HAS_QUUX | |
712 | foo = quux(); | |
713 | #endif | |
714 | ||
715 | How does the HAS_QUUX become defined where it needs to be? Well, if | |
716 | Foonix happens to be Unixy enough to be able to run the Configure | |
717 | script, and Configure has been taught about detecting and testing | |
9b22382a | 718 | quux(), the HAS_QUUX will be correctly defined. In other platforms, the |
04c692a8 DR |
719 | corresponding configuration step will hopefully do the same. |
720 | ||
721 | In a pinch, if you cannot wait for Configure to be educated, or if you | |
722 | have a good hunch of where quux() might be available, you can | |
723 | temporarily try the following: | |
724 | ||
725 | #if (defined(__FOONIX__) || defined(__BARNIX__)) | |
726 | # define HAS_QUUX | |
727 | #endif | |
728 | ||
729 | ... | |
730 | ||
731 | #ifdef HAS_QUUX | |
732 | foo = quux(); | |
733 | #endif | |
734 | ||
735 | But in any case, try to keep the features and operating systems | |
736 | separate. | |
737 | ||
b39b5b0c JH |
738 | A good resource on the predefined macros for various operating |
739 | systems, compilers, and so forth is | |
740 | L<http://sourceforge.net/p/predef/wiki/Home/> | |
741 | ||
38f18a30 KW |
742 | =item * |
743 | ||
744 | Assuming the contents of static memory pointed to by the return values | |
745 | of Perl wrappers for C library functions doesn't change. Many C library | |
746 | functions return pointers to static storage that can be overwritten by | |
747 | subsequent calls to the same or related functions. Perl has | |
748 | light-weight wrappers for some of these functions, and which don't make | |
749 | copies of the static memory. A good example is the interface to the | |
750 | environment variables that are in effect for the program. Perl has | |
751 | C<PerlEnv_getenv> to get values from the environment. But the return is | |
752 | a pointer to static memory in the C library. If you are using the value | |
753 | to immediately test for something, that's fine, but if you save the | |
754 | value and expect it to be unchanged by later processing, you would be | |
755 | wrong, but perhaps you wouldn't know it because different C library | |
756 | implementations behave differently, and the one on the platform you're | |
757 | testing on might work for your situation. But on some platforms, a | |
758 | subsequent call to C<PerlEnv_getenv> or related function WILL overwrite | |
759 | the memory that your first call points to. This has led to some | |
760 | hard-to-debug problems. Do a L<perlapi/savepv> to make a copy, thus | |
761 | avoiding these problems. You will have to free the copy when you're | |
762 | done to avoid memory leaks. If you don't have control over when it gets | |
763 | freed, you'll need to make the copy in a mortal scalar, like so: | |
764 | ||
765 | if ((s = PerlEnv_getenv("foo") == NULL) { | |
766 | ... /* handle NULL case */ | |
767 | } | |
768 | else { | |
769 | s = SvPVX(sv_2mortal(newSVpv(s, 0))); | |
770 | } | |
771 | ||
772 | The above example works only if C<"s"> is C<NUL>-terminated; otherwise | |
773 | you have to pass its length to C<newSVpv>. | |
774 | ||
04c692a8 DR |
775 | =back |
776 | ||
777 | =head2 Problematic System Interfaces | |
778 | ||
779 | =over 4 | |
780 | ||
781 | =item * | |
782 | ||
4aada8b9 KW |
783 | Perl strings are NOT the same as C strings: They may contain C<NUL> |
784 | characters, whereas a C string is terminated by the first C<NUL>. | |
785 | That is why Perl API functions that deal with strings generally take a | |
786 | pointer to the first byte and either a length or a pointer to the byte | |
787 | just beyond the final one. | |
788 | ||
789 | And this is the reason that many of the C library string handling | |
790 | functions should not be used. They don't cope with the full generality | |
791 | of Perl strings. It may be that your test cases don't have embedded | |
792 | C<NUL>s, and so the tests pass, whereas there may well eventually arise | |
793 | real-world cases where they fail. A lesson here is to include C<NUL>s | |
794 | in your tests. Now it's fairly rare in most real world cases to get | |
795 | C<NUL>s, so your code may seem to work, until one day a C<NUL> comes | |
796 | along. | |
797 | ||
798 | Here's an example. It used to be a common paradigm, for decades, in the | |
799 | perl core to use S<C<strchr("list", c)>> to see if the character C<c> is | |
800 | any of the ones given in C<"list">, a double-quote-enclosed string of | |
801 | the set of characters that we are seeing if C<c> is one of. As long as | |
802 | C<c> isn't a C<NUL>, it works. But when C<c> is a C<NUL>, C<strchr> | |
803 | returns a pointer to the terminating C<NUL> in C<"list">. This likely | |
804 | will result in a segfault or a security issue when the caller uses that | |
805 | end pointer as the starting point to read from. | |
806 | ||
807 | A solution to this and many similar issues is to use the C<mem>I<-foo> C | |
808 | library functions instead. In this case C<memchr> can be used to see if | |
809 | C<c> is in C<"list"> and works even if C<c> is C<NUL>. These functions | |
810 | need an additional parameter to give the string length. | |
811 | In the case of literal string parameters, perl has defined macros that | |
51b56f5c | 812 | calculate the length for you. See L<perlapi/String Handling>. |
4aada8b9 KW |
813 | |
814 | =item * | |
815 | ||
9b22382a FC |
816 | malloc(0), realloc(0), calloc(0, 0) are non-portable. To be portable |
817 | allocate at least one byte. (In general you should rarely need to work | |
04c692a8 DR |
818 | at this low level, but instead use the various malloc wrappers.) |
819 | ||
4059ba87 AC |
820 | =item * |
821 | ||
822 | snprintf() - the return type is unportable. Use my_snprintf() instead. | |
823 | ||
04c692a8 DR |
824 | =back |
825 | ||
826 | =head2 Security problems | |
827 | ||
828 | Last but not least, here are various tips for safer coding. | |
bbc89b61 | 829 | See also L<perlclib> for libc/stdio replacements one should use. |
04c692a8 DR |
830 | |
831 | =over 4 | |
832 | ||
833 | =item * | |
834 | ||
835 | Do not use gets() | |
836 | ||
9b22382a | 837 | Or we will publicly ridicule you. Seriously. |
04c692a8 DR |
838 | |
839 | =item * | |
840 | ||
bbc89b61 JH |
841 | Do not use tmpfile() |
842 | ||
843 | Use mkstemp() instead. | |
844 | ||
845 | =item * | |
846 | ||
04c692a8 DR |
847 | Do not use strcpy() or strcat() or strncpy() or strncat() |
848 | ||
849 | Use my_strlcpy() and my_strlcat() instead: they either use the native | |
850 | implementation, or Perl's own implementation (borrowed from the public | |
851 | domain implementation of INN). | |
852 | ||
853 | =item * | |
854 | ||
855 | Do not use sprintf() or vsprintf() | |
856 | ||
4059ba87 AC |
857 | If you really want just plain byte strings, use my_snprintf() and |
858 | my_vsnprintf() instead, which will try to use snprintf() and | |
859 | vsnprintf() if those safer APIs are available. If you want something | |
6bfe0388 KW |
860 | fancier than a plain byte string, use |
861 | L<C<Perl_form>()|perlapi/form> or SVs and | |
862 | L<C<Perl_sv_catpvf()>|perlapi/sv_catpvf>. | |
863 | ||
4059ba87 AC |
864 | Note that glibc C<printf()>, C<sprintf()>, etc. are buggy before glibc |
865 | version 2.17. They won't allow a C<%.s> format with a precision to | |
866 | create a string that isn't valid UTF-8 if the current underlying locale | |
867 | of the program is UTF-8. What happens is that the C<%s> and its operand are | |
868 | simply skipped without any notice. | |
869 | L<https://sourceware.org/bugzilla/show_bug.cgi?id=6530>. | |
870 | ||
c98823ff JH |
871 | =item * |
872 | ||
873 | Do not use atoi() | |
874 | ||
22ff3130 | 875 | Use grok_atoUV() instead. atoi() has ill-defined behavior on overflows, |
c98823ff | 876 | and cannot be used for incremental parsing. It is also affected by locale, |
338aa8b0 JH |
877 | which is bad. |
878 | ||
879 | =item * | |
880 | ||
881 | Do not use strtol() or strtoul() | |
882 | ||
22ff3130 | 883 | Use grok_atoUV() instead. strtol() or strtoul() (or their IV/UV-friendly |
338aa8b0 JH |
884 | macro disguises, Strtol() and Strtoul(), or Atol() and Atoul() are |
885 | affected by locale, which is bad. | |
c98823ff | 886 | |
04c692a8 DR |
887 | =back |
888 | ||
889 | =head1 DEBUGGING | |
890 | ||
891 | You can compile a special debugging version of Perl, which allows you | |
892 | to use the C<-D> option of Perl to tell more about what Perl is doing. | |
893 | But sometimes there is no alternative than to dive in with a debugger, | |
894 | either to see the stack trace of a core dump (very useful in a bug | |
895 | report), or trying to figure out what went wrong before the core dump | |
896 | happened, or how did we end up having wrong or unexpected results. | |
897 | ||
898 | =head2 Poking at Perl | |
899 | ||
900 | To really poke around with Perl, you'll probably want to build Perl for | |
901 | debugging, like this: | |
902 | ||
f075db89 | 903 | ./Configure -d -DDEBUGGING |
04c692a8 DR |
904 | make |
905 | ||
f075db89 DM |
906 | C<-DDEBUGGING> turns on the C compiler's C<-g> flag to have it produce |
907 | debugging information which will allow us to step through a running | |
908 | program, and to see in which C function we are at (without the debugging | |
909 | information we might see only the numerical addresses of the functions, | |
910 | which is not very helpful). It will also turn on the C<DEBUGGING> | |
911 | compilation symbol which enables all the internal debugging code in Perl. | |
028611fa DB |
912 | There are a whole bunch of things you can debug with this: |
913 | L<perlrun|perlrun/-Dletters> lists them all, and the best way to find out | |
914 | about them is to play about with them. The most useful options are | |
915 | probably | |
04c692a8 DR |
916 | |
917 | l Context (loop) stack processing | |
f075db89 | 918 | s Stack snapshots (with v, displays all stacks) |
04c692a8 DR |
919 | t Trace execution |
920 | o Method and overloading resolution | |
921 | c String/numeric conversions | |
922 | ||
f075db89 DM |
923 | For example |
924 | ||
925 | $ perl -Dst -e '$a + 1' | |
926 | .... | |
927 | (-e:1) gvsv(main::a) | |
928 | => UNDEF | |
929 | (-e:1) const(IV(1)) | |
930 | => UNDEF IV(1) | |
931 | (-e:1) add | |
932 | => NV(1) | |
933 | ||
934 | ||
935 | Some of the functionality of the debugging code can be achieved with a | |
936 | non-debugging perl by using XS modules: | |
04c692a8 DR |
937 | |
938 | -Dr => use re 'debug' | |
939 | -Dx => use O 'Debug' | |
940 | ||
941 | =head2 Using a source-level debugger | |
942 | ||
943 | If the debugging output of C<-D> doesn't help you, it's time to step | |
944 | through perl's execution with a source-level debugger. | |
945 | ||
946 | =over 3 | |
947 | ||
948 | =item * | |
949 | ||
950 | We'll use C<gdb> for our examples here; the principles will apply to | |
951 | any debugger (many vendors call their debugger C<dbx>), but check the | |
952 | manual of the one you're using. | |
953 | ||
954 | =back | |
955 | ||
956 | To fire up the debugger, type | |
957 | ||
958 | gdb ./perl | |
959 | ||
960 | Or if you have a core dump: | |
961 | ||
962 | gdb ./perl core | |
963 | ||
964 | You'll want to do that in your Perl source tree so the debugger can | |
9b22382a | 965 | read the source code. You should see the copyright message, followed by |
04c692a8 DR |
966 | the prompt. |
967 | ||
968 | (gdb) | |
969 | ||
970 | C<help> will get you into the documentation, but here are the most | |
971 | useful commands: | |
972 | ||
973 | =over 3 | |
974 | ||
975 | =item * run [args] | |
976 | ||
977 | Run the program with the given arguments. | |
978 | ||
979 | =item * break function_name | |
980 | ||
981 | =item * break source.c:xxx | |
982 | ||
983 | Tells the debugger that we'll want to pause execution when we reach | |
984 | either the named function (but see L<perlguts/Internal Functions>!) or | |
985 | the given line in the named source file. | |
986 | ||
987 | =item * step | |
988 | ||
989 | Steps through the program a line at a time. | |
990 | ||
991 | =item * next | |
992 | ||
993 | Steps through the program a line at a time, without descending into | |
994 | functions. | |
995 | ||
996 | =item * continue | |
997 | ||
998 | Run until the next breakpoint. | |
999 | ||
1000 | =item * finish | |
1001 | ||
1002 | Run until the end of the current function, then stop again. | |
1003 | ||
1004 | =item * 'enter' | |
1005 | ||
1006 | Just pressing Enter will do the most recent operation again - it's a | |
1007 | blessing when stepping through miles of source code. | |
1008 | ||
8b029fdf MH |
1009 | =item * ptype |
1010 | ||
1011 | Prints the C definition of the argument given. | |
1012 | ||
1013 | (gdb) ptype PL_op | |
1014 | type = struct op { | |
1015 | OP *op_next; | |
86cd3a13 | 1016 | OP *op_sibparent; |
8b029fdf MH |
1017 | OP *(*op_ppaddr)(void); |
1018 | PADOFFSET op_targ; | |
1019 | unsigned int op_type : 9; | |
1020 | unsigned int op_opt : 1; | |
1021 | unsigned int op_slabbed : 1; | |
1022 | unsigned int op_savefree : 1; | |
1023 | unsigned int op_static : 1; | |
1024 | unsigned int op_folded : 1; | |
1025 | unsigned int op_spare : 2; | |
1026 | U8 op_flags; | |
1027 | U8 op_private; | |
1028 | } * | |
1029 | ||
04c692a8 DR |
1030 | =item * print |
1031 | ||
9b22382a | 1032 | Execute the given C code and print its results. B<WARNING>: Perl makes |
04c692a8 | 1033 | heavy use of macros, and F<gdb> does not necessarily support macros |
9b22382a | 1034 | (see later L</"gdb macro support">). You'll have to substitute them |
04c692a8 DR |
1035 | yourself, or to invoke cpp on the source code files (see L</"The .i |
1036 | Targets">) So, for instance, you can't say | |
1037 | ||
1038 | print SvPV_nolen(sv) | |
1039 | ||
1040 | but you have to say | |
1041 | ||
1042 | print Perl_sv_2pv_nolen(sv) | |
1043 | ||
1044 | =back | |
1045 | ||
1046 | You may find it helpful to have a "macro dictionary", which you can | |
9b22382a | 1047 | produce by saying C<cpp -dM perl.c | sort>. Even then, F<cpp> won't |
04c692a8 DR |
1048 | recursively apply those macros for you. |
1049 | ||
1050 | =head2 gdb macro support | |
1051 | ||
1052 | Recent versions of F<gdb> have fairly good macro support, but in order | |
1053 | to use it you'll need to compile perl with macro definitions included | |
9b22382a FC |
1054 | in the debugging information. Using F<gcc> version 3.1, this means |
1055 | configuring with C<-Doptimize=-g3>. Other compilers might use a | |
04c692a8 DR |
1056 | different switch (if they support debugging macros at all). |
1057 | ||
1058 | =head2 Dumping Perl Data Structures | |
1059 | ||
1060 | One way to get around this macro hell is to use the dumping functions | |
1061 | in F<dump.c>; these work a little like an internal | |
1062 | L<Devel::Peek|Devel::Peek>, but they also cover OPs and other | |
9b22382a | 1063 | structures that you can't get at from Perl. Let's take an example. |
04c692a8 | 1064 | We'll use the C<$a = $b + $c> we used before, but give it a bit of |
9b22382a | 1065 | context: C<$b = "6XXXX"; $c = 2.3;>. Where's a good place to stop and |
04c692a8 DR |
1066 | poke around? |
1067 | ||
1068 | What about C<pp_add>, the function we examined earlier to implement the | |
1069 | C<+> operator: | |
1070 | ||
1071 | (gdb) break Perl_pp_add | |
1072 | Breakpoint 1 at 0x46249f: file pp_hot.c, line 309. | |
1073 | ||
1074 | Notice we use C<Perl_pp_add> and not C<pp_add> - see | |
9b22382a | 1075 | L<perlguts/Internal Functions>. With the breakpoint in place, we can |
04c692a8 DR |
1076 | run our program: |
1077 | ||
1078 | (gdb) run -e '$b = "6XXXX"; $c = 2.3; $a = $b + $c' | |
1079 | ||
1080 | Lots of junk will go past as gdb reads in the relevant source files and | |
1081 | libraries, and then: | |
1082 | ||
1083 | Breakpoint 1, Perl_pp_add () at pp_hot.c:309 | |
72876cce | 1084 | 1396 dSP; dATARGET; bool useleft; SV *svl, *svr; |
04c692a8 DR |
1085 | (gdb) step |
1086 | 311 dPOPTOPnnrl_ul; | |
1087 | (gdb) | |
1088 | ||
1089 | We looked at this bit of code before, and we said that | |
1090 | C<dPOPTOPnnrl_ul> arranges for two C<NV>s to be placed into C<left> and | |
1091 | C<right> - let's slightly expand it: | |
1092 | ||
1093 | #define dPOPTOPnnrl_ul NV right = POPn; \ | |
1094 | SV *leftsv = TOPs; \ | |
1095 | NV left = USE_LEFT(leftsv) ? SvNV(leftsv) : 0.0 | |
1096 | ||
1097 | C<POPn> takes the SV from the top of the stack and obtains its NV | |
1098 | either directly (if C<SvNOK> is set) or by calling the C<sv_2nv> | |
9b22382a FC |
1099 | function. C<TOPs> takes the next SV from the top of the stack - yes, |
1100 | C<POPn> uses C<TOPs> - but doesn't remove it. We then use C<SvNV> to | |
04c692a8 DR |
1101 | get the NV from C<leftsv> in the same way as before - yes, C<POPn> uses |
1102 | C<SvNV>. | |
1103 | ||
1104 | Since we don't have an NV for C<$b>, we'll have to use C<sv_2nv> to | |
9b22382a | 1105 | convert it. If we step again, we'll find ourselves there: |
04c692a8 | 1106 | |
8b029fdf | 1107 | (gdb) step |
04c692a8 DR |
1108 | Perl_sv_2nv (sv=0xa0675d0) at sv.c:1669 |
1109 | 1669 if (!sv) | |
1110 | (gdb) | |
1111 | ||
1112 | We can now use C<Perl_sv_dump> to investigate the SV: | |
1113 | ||
8b029fdf | 1114 | (gdb) print Perl_sv_dump(sv) |
04c692a8 DR |
1115 | SV = PV(0xa057cc0) at 0xa0675d0 |
1116 | REFCNT = 1 | |
1117 | FLAGS = (POK,pPOK) | |
1118 | PV = 0xa06a510 "6XXXX"\0 | |
1119 | CUR = 5 | |
1120 | LEN = 6 | |
1121 | $1 = void | |
1122 | ||
1123 | We know we're going to get C<6> from this, so let's finish the | |
1124 | subroutine: | |
1125 | ||
1126 | (gdb) finish | |
1127 | Run till exit from #0 Perl_sv_2nv (sv=0xa0675d0) at sv.c:1671 | |
1128 | 0x462669 in Perl_pp_add () at pp_hot.c:311 | |
1129 | 311 dPOPTOPnnrl_ul; | |
1130 | ||
1131 | We can also dump out this op: the current op is always stored in | |
9b22382a | 1132 | C<PL_op>, and we can dump it with C<Perl_op_dump>. This'll give us |
903b1101 | 1133 | similar output to CPAN module B::Debug. |
04c692a8 | 1134 | |
8b029fdf | 1135 | (gdb) print Perl_op_dump(PL_op) |
04c692a8 DR |
1136 | { |
1137 | 13 TYPE = add ===> 14 | |
1138 | TARG = 1 | |
1139 | FLAGS = (SCALAR,KIDS) | |
1140 | { | |
1141 | TYPE = null ===> (12) | |
1142 | (was rv2sv) | |
1143 | FLAGS = (SCALAR,KIDS) | |
1144 | { | |
1145 | 11 TYPE = gvsv ===> 12 | |
1146 | FLAGS = (SCALAR) | |
1147 | GV = main::b | |
1148 | } | |
1149 | } | |
1150 | ||
1151 | # finish this later # | |
1152 | ||
8b029fdf MH |
1153 | =head2 Using gdb to look at specific parts of a program |
1154 | ||
73013070 SF |
1155 | With the example above, you knew to look for C<Perl_pp_add>, but what if |
1156 | there were multiple calls to it all over the place, or you didn't know what | |
8b029fdf MH |
1157 | the op was you were looking for? |
1158 | ||
73013070 | 1159 | One way to do this is to inject a rare call somewhere near what you're looking |
9b22382a | 1160 | for. For example, you could add C<study> before your method: |
8b029fdf MH |
1161 | |
1162 | study; | |
1163 | ||
1164 | And in gdb do: | |
1165 | ||
1166 | (gdb) break Perl_pp_study | |
1167 | ||
9b22382a | 1168 | And then step until you hit what you're |
73013070 | 1169 | looking for. This works well in a loop |
8b029fdf MH |
1170 | if you want to only break at certain iterations: |
1171 | ||
1172 | for my $c (1..100) { | |
1173 | study if $c == 50; | |
1174 | } | |
1175 | ||
1176 | =head2 Using gdb to look at what the parser/lexer are doing | |
1177 | ||
73013070 | 1178 | If you want to see what perl is doing when parsing/lexing your code, you can |
72b22e55 | 1179 | use C<BEGIN {}>: |
8b029fdf MH |
1180 | |
1181 | print "Before\n"; | |
1182 | BEGIN { study; } | |
1183 | print "After\n"; | |
1184 | ||
1185 | And in gdb: | |
1186 | ||
1187 | (gdb) break Perl_pp_study | |
1188 | ||
1189 | If you want to see what the parser/lexer is doing inside of C<if> blocks and | |
1190 | the like you need to be a little trickier: | |
1191 | ||
73013070 | 1192 | if ($a && $b && do { BEGIN { study } 1 } && $c) { ... } |
8b029fdf | 1193 | |
04c692a8 DR |
1194 | =head1 SOURCE CODE STATIC ANALYSIS |
1195 | ||
1196 | Various tools exist for analysing C source code B<statically>, as | |
9b22382a | 1197 | opposed to B<dynamically>, that is, without executing the code. It is |
04c692a8 DR |
1198 | possible to detect resource leaks, undefined behaviour, type |
1199 | mismatches, portability problems, code paths that would cause illegal | |
1200 | memory accesses, and other similar problems by just parsing the C code | |
1201 | and looking at the resulting graph, what does it tell about the | |
9b22382a | 1202 | execution and data flows. As a matter of fact, this is exactly how C |
04c692a8 DR |
1203 | compilers know to give warnings about dubious code. |
1204 | ||
c707756e | 1205 | =head2 lint |
04c692a8 DR |
1206 | |
1207 | The good old C code quality inspector, C<lint>, is available in several | |
1208 | platforms, but please be aware that there are several different | |
1209 | implementations of it by different vendors, which means that the flags | |
1210 | are not identical across different platforms. | |
1211 | ||
c707756e | 1212 | There is a C<lint> target in Makefile, but you may have to |
04c692a8 DR |
1213 | diddle with the flags (see above). |
1214 | ||
1215 | =head2 Coverity | |
1216 | ||
4b05bc8e | 1217 | Coverity (L<http://www.coverity.com/>) is a product similar to lint and as |
04c692a8 DR |
1218 | a testbed for their product they periodically check several open source |
1219 | projects, and they give out accounts to open source developers to the | |
1220 | defect databases. | |
1221 | ||
d3c1eddb JH |
1222 | There is Coverity setup for the perl5 project: |
1223 | L<https://scan.coverity.com/projects/perl5> | |
1224 | ||
a72f2680 | 1225 | =head2 HP-UX cadvise (Code Advisor) |
65c4791f JH |
1226 | |
1227 | HP has a C/C++ static analyzer product for HP-UX caller Code Advisor. | |
1228 | (Link not given here because the URL is horribly long and seems horribly | |
1229 | unstable; use the search engine of your choice to find it.) The use of | |
1230 | the C<cadvise_cc> recipe with C<Configure ... -Dcc=./cadvise_cc> | |
1231 | (see cadvise "User Guide") is recommended; as is the use of C<+wall>. | |
1232 | ||
04c692a8 DR |
1233 | =head2 cpd (cut-and-paste detector) |
1234 | ||
9b22382a | 1235 | The cpd tool detects cut-and-paste coding. If one instance of the |
04c692a8 | 1236 | cut-and-pasted code changes, all the other spots should probably be |
9b22382a | 1237 | changed, too. Therefore such code should probably be turned into a |
04c692a8 DR |
1238 | subroutine or a macro. |
1239 | ||
5632ec47 TD |
1240 | cpd (L<https://pmd.github.io/latest/pmd_userdocs_cpd.html>) is part of the pmd project |
1241 | (L<https://pmd.github.io/>). pmd was originally written for static | |
04c692a8 DR |
1242 | analysis of Java code, but later the cpd part of it was extended to |
1243 | parse also C and C++. | |
1244 | ||
1245 | Download the pmd-bin-X.Y.zip () from the SourceForge site, extract the | |
1246 | pmd-X.Y.jar from it, and then run that on source code thusly: | |
1247 | ||
0cbf2b31 FC |
1248 | java -cp pmd-X.Y.jar net.sourceforge.pmd.cpd.CPD \ |
1249 | --minimum-tokens 100 --files /some/where/src --language c > cpd.txt | |
04c692a8 DR |
1250 | |
1251 | You may run into memory limits, in which case you should use the -Xmx | |
1252 | option: | |
1253 | ||
1254 | java -Xmx512M ... | |
1255 | ||
1256 | =head2 gcc warnings | |
1257 | ||
1258 | Though much can be written about the inconsistency and coverage | |
1259 | problems of gcc warnings (like C<-Wall> not meaning "all the warnings", | |
1260 | or some common portability problems not being covered by C<-Wall>, or | |
1261 | C<-ansi> and C<-pedantic> both being a poorly defined collection of | |
1262 | warnings, and so forth), gcc is still a useful tool in keeping our | |
1263 | coding nose clean. | |
1264 | ||
1265 | The C<-Wall> is by default on. | |
1266 | ||
a66ca998 NC |
1267 | It would be nice for C<-pedantic>) to be on always, but unfortunately it is not |
1268 | safe on all platforms - for example fatal conflicts with the system headers | |
1269 | (Solaris being a prime example). If Configure C<-Dgccansipedantic> is used, | |
1270 | the C<cflags> frontend selects C<-pedantic> for the platforms where it is known | |
1271 | to be safe. | |
04c692a8 | 1272 | |
2884c977 | 1273 | The following extra flags are added: |
04c692a8 DR |
1274 | |
1275 | =over 4 | |
1276 | ||
1277 | =item * | |
1278 | ||
1279 | C<-Wendif-labels> | |
1280 | ||
1281 | =item * | |
1282 | ||
1283 | C<-Wextra> | |
1284 | ||
1285 | =item * | |
1286 | ||
2884c977 DIM |
1287 | C<-Wc++-compat> |
1288 | ||
1289 | =item * | |
1290 | ||
1291 | C<-Wwrite-strings> | |
1292 | ||
1293 | =item * | |
1294 | ||
a66ca998 | 1295 | C<-Werror=pointer-arith> |
04c692a8 | 1296 | |
5997475b DIM |
1297 | =item * |
1298 | ||
a66ca998 | 1299 | C<-Werror=vla> |
5997475b | 1300 | |
04c692a8 DR |
1301 | =back |
1302 | ||
1303 | The following flags would be nice to have but they would first need | |
1304 | their own Augean stablemaster: | |
1305 | ||
1306 | =over 4 | |
1307 | ||
1308 | =item * | |
1309 | ||
04c692a8 DR |
1310 | C<-Wshadow> |
1311 | ||
1312 | =item * | |
1313 | ||
1314 | C<-Wstrict-prototypes> | |
1315 | ||
1316 | =back | |
1317 | ||
1318 | The C<-Wtraditional> is another example of the annoying tendency of gcc | |
1319 | to bundle a lot of warnings under one switch (it would be impossible to | |
1320 | deploy in practice because it would complain a lot) but it does contain | |
1321 | some warnings that would be beneficial to have available on their own, | |
1322 | such as the warning about string constants inside macros containing the | |
1323 | macro arguments: this behaved differently pre-ANSI than it does in | |
1324 | ANSI, and some C compilers are still in transition, AIX being an | |
1325 | example. | |
1326 | ||
1327 | =head2 Warnings of other C compilers | |
1328 | ||
1329 | Other C compilers (yes, there B<are> other C compilers than gcc) often | |
1330 | have their "strict ANSI" or "strict ANSI with some portability | |
1331 | extensions" modes on, like for example the Sun Workshop has its C<-Xa> | |
1332 | mode on (though implicitly), or the DEC (these days, HP...) has its | |
1333 | C<-std1> mode on. | |
1334 | ||
1335 | =head1 MEMORY DEBUGGERS | |
1336 | ||
d1fd4856 VP |
1337 | B<NOTE 1>: Running under older memory debuggers such as Purify, |
1338 | valgrind or Third Degree greatly slows down the execution: seconds | |
9b22382a | 1339 | become minutes, minutes become hours. For example as of Perl 5.8.1, the |
04c692a8 | 1340 | ext/Encode/t/Unicode.t takes extraordinarily long to complete under |
9b22382a FC |
1341 | e.g. Purify, Third Degree, and valgrind. Under valgrind it takes more |
1342 | than six hours, even on a snappy computer. The said test must be doing | |
1343 | something that is quite unfriendly for memory debuggers. If you don't | |
04c692a8 | 1344 | feel like waiting, that you can simply kill away the perl process. |
d1fd4856 VP |
1345 | Roughly valgrind slows down execution by factor 10, AddressSanitizer by |
1346 | factor 2. | |
04c692a8 DR |
1347 | |
1348 | B<NOTE 2>: To minimize the number of memory leak false alarms (see | |
1349 | L</PERL_DESTRUCT_LEVEL> for more information), you have to set the | |
9b22382a | 1350 | environment variable PERL_DESTRUCT_LEVEL to 2. For example, like this: |
04c692a8 DR |
1351 | |
1352 | env PERL_DESTRUCT_LEVEL=2 valgrind ./perl -Ilib ... | |
1353 | ||
1354 | B<NOTE 3>: There are known memory leaks when there are compile-time | |
1355 | errors within eval or require, seeing C<S_doeval> in the call stack is | |
9b22382a | 1356 | a good sign of these. Fixing these leaks is non-trivial, unfortunately, |
04c692a8 DR |
1357 | but they must be fixed eventually. |
1358 | ||
1359 | B<NOTE 4>: L<DynaLoader> will not clean up after itself completely | |
1360 | unless Perl is built with the Configure option | |
1361 | C<-Accflags=-DDL_UNLOAD_ALL_AT_EXIT>. | |
1362 | ||
04c692a8 DR |
1363 | =head2 valgrind |
1364 | ||
d1fd4856 | 1365 | The valgrind tool can be used to find out both memory leaks and illegal |
9b22382a | 1366 | heap memory accesses. As of version 3.3.0, Valgrind only supports Linux |
0263e49a | 1367 | on x86, x86-64 and PowerPC and Darwin (OS X) on x86 and x86-64. The |
d1fd4856 | 1368 | special "test.valgrind" target can be used to run the tests under |
9b22382a | 1369 | valgrind. Found errors and memory leaks are logged in files named |
037ab3f1 MH |
1370 | F<testfile.valgrind> and by default output is displayed inline. |
1371 | ||
1372 | Example usage: | |
1373 | ||
1374 | make test.valgrind | |
1375 | ||
1376 | Since valgrind adds significant overhead, tests will take much longer to | |
1377 | run. The valgrind tests support being run in parallel to help with this: | |
1378 | ||
1379 | TEST_JOBS=9 make test.valgrind | |
1380 | ||
1381 | Note that the above two invocations will be very verbose as reachable | |
1382 | memory and leak-checking is enabled by default. If you want to just see | |
1383 | pure errors, try: | |
73013070 | 1384 | |
037ab3f1 MH |
1385 | VG_OPTS='-q --leak-check=no --show-reachable=no' TEST_JOBS=9 \ |
1386 | make test.valgrind | |
04c692a8 DR |
1387 | |
1388 | Valgrind also provides a cachegrind tool, invoked on perl as: | |
1389 | ||
1390 | VG_OPTS=--tool=cachegrind make test.valgrind | |
1391 | ||
1392 | As system libraries (most notably glibc) are also triggering errors, | |
9b22382a | 1393 | valgrind allows to suppress such errors using suppression files. The |
04c692a8 | 1394 | default suppression file that comes with valgrind already catches a lot |
9b22382a | 1395 | of them. Some additional suppressions are defined in F<t/perl.supp>. |
04c692a8 DR |
1396 | |
1397 | To get valgrind and for more information see | |
1398 | ||
0061d4fa | 1399 | http://valgrind.org/ |
04c692a8 | 1400 | |
81c3bbe7 RU |
1401 | =head2 AddressSanitizer |
1402 | ||
6babf542 | 1403 | AddressSanitizer ("ASan") consists of a compiler instrumentation module |
53b3ccc9 RL |
1404 | and a run-time C<malloc> library. ASan is available for a variety of |
1405 | architectures, operating systems, and compilers (see project link below). | |
d087776a | 1406 | It checks for unsafe memory usage, such as use after free and buffer |
53b3ccc9 | 1407 | overflow conditions, and is fast enough that you can easily compile your |
6babf542 RL |
1408 | debugging or optimized perl with it. Modern versions of ASan check for |
1409 | memory leaks by default on most platforms, otherwise (e.g. x86_64 OS X) | |
1410 | this feature can be enabled via C<ASAN_OPTIONS=detect_leaks=1>. | |
1411 | ||
81c3bbe7 | 1412 | |
8a64fbaa VP |
1413 | To build perl with AddressSanitizer, your Configure invocation should |
1414 | look like: | |
81c3bbe7 | 1415 | |
e8596d90 | 1416 | sh Configure -des -Dcc=clang \ |
6babf542 RL |
1417 | -Accflags=-fsanitize=address -Aldflags=-fsanitize=address \ |
1418 | -Alddlflags=-shared\ -fsanitize=address \ | |
1419 | -fsanitize-blacklist=`pwd`/asan_ignore | |
81c3bbe7 RU |
1420 | |
1421 | where these arguments mean: | |
1422 | ||
1423 | =over 4 | |
1424 | ||
1425 | =item * -Dcc=clang | |
1426 | ||
8a64fbaa VP |
1427 | This should be replaced by the full path to your clang executable if it |
1428 | is not in your path. | |
81c3bbe7 | 1429 | |
6babf542 | 1430 | =item * -Accflags=-fsanitize=address |
81c3bbe7 | 1431 | |
8a64fbaa | 1432 | Compile perl and extensions sources with AddressSanitizer. |
81c3bbe7 | 1433 | |
6babf542 | 1434 | =item * -Aldflags=-fsanitize=address |
81c3bbe7 | 1435 | |
8a64fbaa | 1436 | Link the perl executable with AddressSanitizer. |
81c3bbe7 | 1437 | |
6babf542 | 1438 | =item * -Alddlflags=-shared\ -fsanitize=address |
81c3bbe7 | 1439 | |
9b22382a | 1440 | Link dynamic extensions with AddressSanitizer. You must manually |
e8596d90 VP |
1441 | specify C<-shared> because using C<-Alddlflags=-shared> will prevent |
1442 | Configure from setting a default value for C<lddlflags>, which usually | |
5dfc6e97 | 1443 | contains C<-shared> (at least on Linux). |
81c3bbe7 | 1444 | |
6babf542 RL |
1445 | =item * -fsanitize-blacklist=`pwd`/asan_ignore |
1446 | ||
1447 | AddressSanitizer will ignore functions listed in the C<asan_ignore> | |
1448 | file. (This file should contain a short explanation of why each of | |
1449 | the functions is listed.) | |
1450 | ||
81c3bbe7 RU |
1451 | =back |
1452 | ||
8a64fbaa | 1453 | See also |
a856e9cc | 1454 | L<https://github.com/google/sanitizers/wiki/AddressSanitizer>. |
81c3bbe7 RU |
1455 | |
1456 | ||
04c692a8 DR |
1457 | =head1 PROFILING |
1458 | ||
1459 | Depending on your platform there are various ways of profiling Perl. | |
1460 | ||
1461 | There are two commonly used techniques of profiling executables: | |
1462 | I<statistical time-sampling> and I<basic-block counting>. | |
1463 | ||
1464 | The first method takes periodically samples of the CPU program counter, | |
1465 | and since the program counter can be correlated with the code generated | |
1466 | for functions, we get a statistical view of in which functions the | |
9b22382a | 1467 | program is spending its time. The caveats are that very small/fast |
04c692a8 DR |
1468 | functions have lower probability of showing up in the profile, and that |
1469 | periodically interrupting the program (this is usually done rather | |
1470 | frequently, in the scale of milliseconds) imposes an additional | |
9b22382a | 1471 | overhead that may skew the results. The first problem can be alleviated |
04c692a8 DR |
1472 | by running the code for longer (in general this is a good idea for |
1473 | profiling), the second problem is usually kept in guard by the | |
1474 | profiling tools themselves. | |
1475 | ||
1476 | The second method divides up the generated code into I<basic blocks>. | |
1477 | Basic blocks are sections of code that are entered only in the | |
9b22382a FC |
1478 | beginning and exited only at the end. For example, a conditional jump |
1479 | starts a basic block. Basic block profiling usually works by | |
04c692a8 | 1480 | I<instrumenting> the code by adding I<enter basic block #nnnn> |
9b22382a FC |
1481 | book-keeping code to the generated code. During the execution of the |
1482 | code the basic block counters are then updated appropriately. The | |
04c692a8 DR |
1483 | caveat is that the added extra code can skew the results: again, the |
1484 | profiling tools usually try to factor their own effects out of the | |
1485 | results. | |
1486 | ||
1487 | =head2 Gprof Profiling | |
1488 | ||
e2aed43d | 1489 | I<gprof> is a profiling tool available in many Unix platforms which |
9b22382a FC |
1490 | uses I<statistical time-sampling>. You can build a profiled version of |
1491 | F<perl> by compiling using gcc with the flag C<-pg>. Either edit | |
1492 | F<config.sh> or re-run F<Configure>. Running the profiled version of | |
e2aed43d NC |
1493 | Perl will create an output file called F<gmon.out> which contains the |
1494 | profiling data collected during the execution. | |
04c692a8 | 1495 | |
e2aed43d NC |
1496 | quick hint: |
1497 | ||
1498 | $ sh Configure -des -Dusedevel -Accflags='-pg' \ | |
1499 | -Aldflags='-pg' -Alddlflags='-pg -shared' \ | |
1500 | && make perl | |
1501 | $ ./perl ... # creates gmon.out in current directory | |
1502 | $ gprof ./perl > out | |
1503 | $ less out | |
1504 | ||
1505 | (you probably need to add C<-shared> to the <-Alddlflags> line until RT | |
1506 | #118199 is resolved) | |
04c692a8 | 1507 | |
e2aed43d NC |
1508 | The F<gprof> tool can then display the collected data in various ways. |
1509 | Usually F<gprof> understands the following options: | |
04c692a8 DR |
1510 | |
1511 | =over 4 | |
1512 | ||
1513 | =item * -a | |
1514 | ||
1515 | Suppress statically defined functions from the profile. | |
1516 | ||
1517 | =item * -b | |
1518 | ||
1519 | Suppress the verbose descriptions in the profile. | |
1520 | ||
1521 | =item * -e routine | |
1522 | ||
1523 | Exclude the given routine and its descendants from the profile. | |
1524 | ||
1525 | =item * -f routine | |
1526 | ||
1527 | Display only the given routine and its descendants in the profile. | |
1528 | ||
1529 | =item * -s | |
1530 | ||
1531 | Generate a summary file called F<gmon.sum> which then may be given to | |
1532 | subsequent gprof runs to accumulate data over several runs. | |
1533 | ||
1534 | =item * -z | |
1535 | ||
1536 | Display routines that have zero usage. | |
1537 | ||
1538 | =back | |
1539 | ||
1540 | For more detailed explanation of the available commands and output | |
e2aed43d | 1541 | formats, see your own local documentation of F<gprof>. |
04c692a8 | 1542 | |
e2aed43d | 1543 | =head2 GCC gcov Profiling |
04c692a8 | 1544 | |
e2aed43d NC |
1545 | I<basic block profiling> is officially available in gcc 3.0 and later. |
1546 | You can build a profiled version of F<perl> by compiling using gcc with | |
9b22382a | 1547 | the flags C<-fprofile-arcs -ftest-coverage>. Either edit F<config.sh> |
e2aed43d | 1548 | or re-run F<Configure>. |
04c692a8 | 1549 | |
e2aed43d | 1550 | quick hint: |
04c692a8 | 1551 | |
e2aed43d NC |
1552 | $ sh Configure -des -Dusedevel -Doptimize='-g' \ |
1553 | -Accflags='-fprofile-arcs -ftest-coverage' \ | |
1554 | -Aldflags='-fprofile-arcs -ftest-coverage' \ | |
1555 | -Alddlflags='-fprofile-arcs -ftest-coverage -shared' \ | |
1556 | && make perl | |
1557 | $ rm -f regexec.c.gcov regexec.gcda | |
1558 | $ ./perl ... | |
1559 | $ gcov regexec.c | |
1560 | $ less regexec.c.gcov | |
04c692a8 | 1561 | |
e2aed43d NC |
1562 | (you probably need to add C<-shared> to the <-Alddlflags> line until RT |
1563 | #118199 is resolved) | |
04c692a8 DR |
1564 | |
1565 | Running the profiled version of Perl will cause profile output to be | |
9b22382a | 1566 | generated. For each source file an accompanying F<.gcda> file will be |
04c692a8 DR |
1567 | created. |
1568 | ||
e2aed43d | 1569 | To display the results you use the I<gcov> utility (which should be |
9b22382a | 1570 | installed if you have gcc 3.0 or newer installed). F<gcov> is run on |
04c692a8 DR |
1571 | source code files, like this |
1572 | ||
1573 | gcov sv.c | |
1574 | ||
9b22382a | 1575 | which will cause F<sv.c.gcov> to be created. The F<.gcov> files contain |
04c692a8 | 1576 | the source code annotated with relative frequencies of execution |
9b22382a | 1577 | indicated by "#" markers. If you want to generate F<.gcov> files for |
6f134219 NC |
1578 | all profiled object files, you can run something like this: |
1579 | ||
1580 | for file in `find . -name \*.gcno` | |
1581 | do sh -c "cd `dirname $file` && gcov `basename $file .gcno`" | |
1582 | done | |
04c692a8 DR |
1583 | |
1584 | Useful options of F<gcov> include C<-b> which will summarise the basic | |
1585 | block, branch, and function call coverage, and C<-c> which instead of | |
9b22382a | 1586 | relative frequencies will use the actual counts. For more information |
04c692a8 | 1587 | on the use of F<gcov> and basic block profiling with gcc, see the |
9b22382a | 1588 | latest GNU CC manual. As of gcc 4.8, this is at |
e2aed43d | 1589 | L<http://gcc.gnu.org/onlinedocs/gcc/Gcov-Intro.html#Gcov-Intro> |
04c692a8 | 1590 | |
696d6093 MH |
1591 | =head2 callgrind profiling |
1592 | ||
1593 | callgrind is a valgrind tool for profiling source code. Paired | |
1594 | with kcachegrind (a Qt based UI), it gives you an overview of | |
1595 | where code is taking up time, as well as the ability | |
1596 | to examine callers, call trees, and more. One of its benefits | |
1597 | is you can use it on perl and XS modules that have not been | |
1598 | compiled with debugging symbols. | |
1599 | ||
1600 | If perl is compiled with debugging symbols (C<-g>), you can view | |
1601 | the annotated source and click around, much like Devel::NYTProf's | |
1602 | HTML output. | |
1603 | ||
1604 | For basic usage: | |
1605 | ||
1606 | valgrind --tool=callgrind ./perl ... | |
1607 | ||
1608 | By default it will write output to F<callgrind.out.PID>, but you | |
1609 | can change that with C<--callgrind-out-file=...> | |
1610 | ||
1611 | To view the data, do: | |
1612 | ||
1613 | kcachegrind callgrind.out.PID | |
1614 | ||
1615 | If you'd prefer to view the data in a terminal, you can use | |
1616 | F<callgrind_annotate>. In it's basic form: | |
1617 | ||
1618 | callgrind_annotate callgrind.out.PID | less | |
1619 | ||
1620 | Some useful options are: | |
1621 | ||
1622 | =over 4 | |
1623 | ||
1624 | =item * --threshold | |
1625 | ||
1626 | Percentage of counts (of primary sort event) we are interested in. | |
1627 | The default is 99%, 100% might show things that seem to be missing. | |
1628 | ||
1629 | =item * --auto | |
1630 | ||
1631 | Annotate all source files containing functions that helped reach | |
1632 | the event count threshold. | |
1633 | ||
1634 | =back | |
1635 | ||
04c692a8 DR |
1636 | =head1 MISCELLANEOUS TRICKS |
1637 | ||
1638 | =head2 PERL_DESTRUCT_LEVEL | |
1639 | ||
1640 | If you want to run any of the tests yourself manually using e.g. | |
4dd56148 NC |
1641 | valgrind, please note that by default perl B<does not> explicitly |
1642 | cleanup all the memory it has allocated (such as global memory arenas) | |
1643 | but instead lets the exit() of the whole program "take care" of such | |
1644 | allocations, also known as "global destruction of objects". | |
04c692a8 DR |
1645 | |
1646 | There is a way to tell perl to do complete cleanup: set the environment | |
9b22382a | 1647 | variable PERL_DESTRUCT_LEVEL to a non-zero value. The t/TEST wrapper |
04c692a8 | 1648 | does set this to 2, and this is what you need to do too, if you don't |
f01ecde8 | 1649 | want to see the "global leaks": For example, for running under valgrind |
04c692a8 | 1650 | |
a63ef199 | 1651 | env PERL_DESTRUCT_LEVEL=2 valgrind ./perl -Ilib t/foo/bar.t |
04c692a8 DR |
1652 | |
1653 | (Note: the mod_perl apache module uses also this environment variable | |
9b22382a FC |
1654 | for its own purposes and extended its semantics. Refer to the mod_perl |
1655 | documentation for more information. Also, spawned threads do the | |
04c692a8 DR |
1656 | equivalent of setting this variable to the value 1.) |
1657 | ||
1658 | If, at the end of a run you get the message I<N scalars leaked>, you | |
b4986286 DM |
1659 | can recompile with C<-DDEBUG_LEAKING_SCALARS>, |
1660 | (C<Configure -Accflags=-DDEBUG_LEAKING_SCALARS>), which will cause the | |
04c692a8 | 1661 | addresses of all those leaked SVs to be dumped along with details as to |
9b22382a FC |
1662 | where each SV was originally allocated. This information is also |
1663 | displayed by Devel::Peek. Note that the extra details recorded with | |
04c692a8 | 1664 | each SV increases memory usage, so it shouldn't be used in production |
9b22382a | 1665 | environments. It also converts C<new_SV()> from a macro into a real |
04c692a8 DR |
1666 | function, so you can use your favourite debugger to discover where |
1667 | those pesky SVs were allocated. | |
1668 | ||
1669 | If you see that you're leaking memory at runtime, but neither valgrind | |
1670 | nor C<-DDEBUG_LEAKING_SCALARS> will find anything, you're probably | |
1671 | leaking SVs that are still reachable and will be properly cleaned up | |
9b22382a FC |
1672 | during destruction of the interpreter. In such cases, using the C<-Dm> |
1673 | switch can point you to the source of the leak. If the executable was | |
04c692a8 | 1674 | built with C<-DDEBUG_LEAKING_SCALARS>, C<-Dm> will output SV |
9b22382a | 1675 | allocations in addition to memory allocations. Each SV allocation has a |
04c692a8 | 1676 | distinct serial number that will be written on creation and destruction |
9b22382a | 1677 | of the SV. So if you're executing the leaking code in a loop, you need |
04c692a8 | 1678 | to look for SVs that are created, but never destroyed between each |
9b22382a | 1679 | cycle. If such an SV is found, set a conditional breakpoint within |
04c692a8 | 1680 | C<new_SV()> and make it break only when C<PL_sv_serial> is equal to the |
9b22382a | 1681 | serial number of the leaking SV. Then you will catch the interpreter in |
04c692a8 DR |
1682 | exactly the state where the leaking SV is allocated, which is |
1683 | sufficient in many cases to find the source of the leak. | |
1684 | ||
1685 | As C<-Dm> is using the PerlIO layer for output, it will by itself | |
9b22382a | 1686 | allocate quite a bunch of SVs, which are hidden to avoid recursion. You |
04c692a8 DR |
1687 | can bypass the PerlIO layer if you use the SV logging provided by |
1688 | C<-DPERL_MEM_LOG> instead. | |
1689 | ||
1690 | =head2 PERL_MEM_LOG | |
1691 | ||
6fb87544 MH |
1692 | If compiled with C<-DPERL_MEM_LOG> (C<-Accflags=-DPERL_MEM_LOG>), both |
1693 | memory and SV allocations go through logging functions, which is | |
1694 | handy for breakpoint setting. | |
04c692a8 | 1695 | |
6fb87544 MH |
1696 | Unless C<-DPERL_MEM_LOG_NOIMPL> (C<-Accflags=-DPERL_MEM_LOG_NOIMPL>) is |
1697 | also compiled, the logging functions read $ENV{PERL_MEM_LOG} to | |
1698 | determine whether to log the event, and if so how: | |
04c692a8 | 1699 | |
a63ef199 SF |
1700 | $ENV{PERL_MEM_LOG} =~ /m/ Log all memory ops |
1701 | $ENV{PERL_MEM_LOG} =~ /s/ Log all SV ops | |
1702 | $ENV{PERL_MEM_LOG} =~ /t/ include timestamp in Log | |
1703 | $ENV{PERL_MEM_LOG} =~ /^(\d+)/ write to FD given (default is 2) | |
04c692a8 DR |
1704 | |
1705 | Memory logging is somewhat similar to C<-Dm> but is independent of | |
1706 | C<-DDEBUGGING>, and at a higher level; all uses of Newx(), Renew(), and | |
1707 | Safefree() are logged with the caller's source code file and line | |
9b22382a FC |
1708 | number (and C function name, if supported by the C compiler). In |
1709 | contrast, C<-Dm> is directly at the point of C<malloc()>. SV logging is | |
04c692a8 DR |
1710 | similar. |
1711 | ||
1712 | Since the logging doesn't use PerlIO, all SV allocations are logged and | |
9b22382a | 1713 | no extra SV allocations are introduced by enabling the logging. If |
04c692a8 DR |
1714 | compiled with C<-DDEBUG_LEAKING_SCALARS>, the serial number for each SV |
1715 | allocation is also logged. | |
1716 | ||
1717 | =head2 DDD over gdb | |
1718 | ||
1719 | Those debugging perl with the DDD frontend over gdb may find the | |
1720 | following useful: | |
1721 | ||
1722 | You can extend the data conversion shortcuts menu, so for example you | |
1723 | can display an SV's IV value with one click, without doing any typing. | |
1724 | To do that simply edit ~/.ddd/init file and add after: | |
1725 | ||
1726 | ! Display shortcuts. | |
1727 | Ddd*gdbDisplayShortcuts: \ | |
1728 | /t () // Convert to Bin\n\ | |
1729 | /d () // Convert to Dec\n\ | |
1730 | /x () // Convert to Hex\n\ | |
1731 | /o () // Convert to Oct(\n\ | |
1732 | ||
1733 | the following two lines: | |
1734 | ||
1735 | ((XPV*) (())->sv_any )->xpv_pv // 2pvx\n\ | |
1736 | ((XPVIV*) (())->sv_any )->xiv_iv // 2ivx | |
1737 | ||
1738 | so now you can do ivx and pvx lookups or you can plug there the sv_peek | |
1739 | "conversion": | |
1740 | ||
1741 | Perl_sv_peek(my_perl, (SV*)()) // sv_peek | |
1742 | ||
9b22382a | 1743 | (The my_perl is for threaded builds.) Just remember that every line, |
04c692a8 DR |
1744 | but the last one, should end with \n\ |
1745 | ||
1746 | Alternatively edit the init file interactively via: 3rd mouse button -> | |
1747 | New Display -> Edit Menu | |
1748 | ||
1749 | Note: you can define up to 20 conversion shortcuts in the gdb section. | |
1750 | ||
470dd224 JH |
1751 | =head2 C backtrace |
1752 | ||
0762e42f JH |
1753 | On some platforms Perl supports retrieving the C level backtrace |
1754 | (similar to what symbolic debuggers like gdb do). | |
470dd224 JH |
1755 | |
1756 | The backtrace returns the stack trace of the C call frames, | |
1757 | with the symbol names (function names), the object names (like "perl"), | |
1758 | and if it can, also the source code locations (file:line). | |
1759 | ||
0762e42f JH |
1760 | The supported platforms are Linux, and OS X (some *BSD might |
1761 | work at least partly, but they have not yet been tested). | |
1762 | ||
1763 | This feature hasn't been tested with multiple threads, but it will | |
1764 | only show the backtrace of the thread doing the backtracing. | |
470dd224 JH |
1765 | |
1766 | The feature needs to be enabled with C<Configure -Dusecbacktrace>. | |
1767 | ||
0762e42f JH |
1768 | The C<-Dusecbacktrace> also enables keeping the debug information when |
1769 | compiling/linking (often: C<-g>). Many compilers/linkers do support | |
1770 | having both optimization and keeping the debug information. The debug | |
1771 | information is needed for the symbol names and the source locations. | |
1772 | ||
1773 | Static functions might not be visible for the backtrace. | |
470dd224 JH |
1774 | |
1775 | Source code locations, even if available, can often be missing or | |
0762e42f JH |
1776 | misleading if the compiler has e.g. inlined code. Optimizer can |
1777 | make matching the source code and the object code quite challenging. | |
470dd224 JH |
1778 | |
1779 | =over 4 | |
1780 | ||
1781 | =item Linux | |
1782 | ||
59b3baca | 1783 | You B<must> have the BFD (-lbfd) library installed, otherwise C<perl> will |
0762e42f | 1784 | fail to link. The BFD is usually distributed as part of the GNU binutils. |
470dd224 JH |
1785 | |
1786 | Summary: C<Configure ... -Dusecbacktrace> | |
1787 | and you need C<-lbfd>. | |
1788 | ||
1789 | =item OS X | |
1790 | ||
0762e42f JH |
1791 | The source code locations are supported B<only> if you have |
1792 | the Developer Tools installed. (BFD is B<not> needed.) | |
470dd224 JH |
1793 | |
1794 | Summary: C<Configure ... -Dusecbacktrace> | |
1795 | and installing the Developer Tools would be good. | |
1796 | ||
1797 | =back | |
1798 | ||
1799 | Optionally, for trying out the feature, you may want to enable | |
0762e42f JH |
1800 | automatic dumping of the backtrace just before a warning or croak (die) |
1801 | message is emitted, by adding C<-Accflags=-DUSE_C_BACKTRACE_ON_ERROR> | |
1802 | for Configure. | |
470dd224 JH |
1803 | |
1804 | Unless the above additional feature is enabled, nothing about the | |
1805 | backtrace functionality is visible, except for the Perl/XS level. | |
1806 | ||
1807 | Furthermore, even if you have enabled this feature to be compiled, | |
1808 | you need to enable it in runtime with an environment variable: | |
0762e42f JH |
1809 | C<PERL_C_BACKTRACE_ON_ERROR=10>. It must be an integer higher |
1810 | than zero, telling the desired frame count. | |
470dd224 JH |
1811 | |
1812 | Retrieving the backtrace from Perl level (using for example an XS | |
1813 | extension) would be much less exciting than one would hope: normally | |
1814 | you would see C<runops>, C<entersub>, and not much else. This API is | |
1815 | intended to be called B<from within> the Perl implementation, not from | |
1816 | Perl level execution. | |
1817 | ||
0762e42f | 1818 | The C API for the backtrace is as follows: |
470dd224 JH |
1819 | |
1820 | =over 4 | |
1821 | ||
1822 | =item get_c_backtrace | |
1823 | ||
1824 | =item free_c_backtrace | |
1825 | ||
1826 | =item get_c_backtrace_dump | |
1827 | ||
1828 | =item dump_c_backtrace | |
1829 | ||
1830 | =back | |
1831 | ||
04c692a8 DR |
1832 | =head2 Poison |
1833 | ||
1834 | If you see in a debugger a memory area mysteriously full of 0xABABABAB | |
1835 | or 0xEFEFEFEF, you may be seeing the effect of the Poison() macros, see | |
1836 | L<perlclib>. | |
1837 | ||
1838 | =head2 Read-only optrees | |
1839 | ||
9b22382a | 1840 | Under ithreads the optree is read only. If you want to enforce this, to |
04c692a8 | 1841 | check for write accesses from buggy code, compile with |
91fc0422 FC |
1842 | C<-Accflags=-DPERL_DEBUG_READONLY_OPS> |
1843 | to enable code that allocates op memory | |
4dd56148 NC |
1844 | via C<mmap>, and sets it read-only when it is attached to a subroutine. |
1845 | Any write access to an op results in a C<SIGBUS> and abort. | |
04c692a8 DR |
1846 | |
1847 | This code is intended for development only, and may not be portable | |
9b22382a FC |
1848 | even to all Unix variants. Also, it is an 80% solution, in that it |
1849 | isn't able to make all ops read only. Specifically it does not apply to | |
4dd56148 | 1850 | op slabs belonging to C<BEGIN> blocks. |
04c692a8 | 1851 | |
4dd56148 NC |
1852 | However, as an 80% solution it is still effective, as it has caught |
1853 | bugs in the past. | |
04c692a8 | 1854 | |
f789f6a4 FC |
1855 | =head2 When is a bool not a bool? |
1856 | ||
23805bfc KW |
1857 | There wasn't necessarily a standard C<bool> type on compilers prior to |
1858 | C99, and so some workarounds were created. The C<TRUE> and C<FALSE> | |
1859 | macros are still available as alternatives for C<true> and C<false>. | |
1860 | And the C<cBOOL> macro was created to correctly cast to a true/false | |
1861 | value in all circumstances, but should no longer be necessary. | |
1862 | Using S<C<(bool)> I<expr>>> should now always work. | |
f789f6a4 | 1863 | |
23805bfc KW |
1864 | There are no plans to remove any of C<TRUE>, C<FALSE>, nor C<cBOOL>. |
1865 | ||
1866 | =head2 Finding unsafe truncations | |
1867 | ||
1868 | You may wish to run C<Configure> with something like | |
50e4f4d4 | 1869 | |
cbc13c3d | 1870 | -Accflags='-Wconversion -Wno-sign-conversion -Wno-shorten-64-to-32' |
50e4f4d4 CB |
1871 | |
1872 | or your compiler's equivalent to make it easier to spot any unsafe truncations | |
1873 | that show up. | |
f789f6a4 | 1874 | |
04c692a8 DR |
1875 | =head2 The .i Targets |
1876 | ||
1877 | You can expand the macros in a F<foo.c> file by saying | |
1878 | ||
1879 | make foo.i | |
1880 | ||
d1fd4856 VP |
1881 | which will expand the macros using cpp. Don't be scared by the |
1882 | results. | |
04c692a8 DR |
1883 | |
1884 | =head1 AUTHOR | |
1885 | ||
1886 | This document was originally written by Nathan Torkington, and is | |
1887 | maintained by the perl5-porters mailing list. |