This is a live mirror of the Perl 5 development currently hosted at https://github.com/perl/perl5
Don't raise Wide char warning in UTF-8 locale
[perl5.git] / pod / perlhacktips.pod
CommitLineData
0061d4fa 1
04c692a8
DR
2=encoding utf8
3
4=for comment
5Consistent formatting of this file is achieved with:
6 perl ./Porting/podtidy pod/perlhacktips.pod
7
8=head1 NAME
9
10perlhacktips - Tips for Perl core C code hacking
11
12=head1 DESCRIPTION
13
14This document will help you learn the best way to go about hacking on
9b22382a 15the Perl core C code. It covers common problems, debugging, profiling,
04c692a8
DR
16and more.
17
18If you haven't read L<perlhack> and L<perlhacktut> yet, you might want
19to do that first.
20
21=head1 COMMON PROBLEMS
22
9b22382a 23Perl source plays by ANSI C89 rules: no C99 (or C++) extensions. In
04c692a8
DR
24some cases we have to take pre-ANSI requirements into consideration.
25You don't care about some particular platform having broken Perl? I
26hear there is still a strong demand for J2EE programmers.
27
28=head2 Perl environment problems
29
30=over 4
31
32=item *
33
34Not compiling with threading
35
36Compiling with threading (-Duseithreads) completely rewrites the
9b22382a 37function prototypes of Perl. You better try your changes with that.
04c692a8
DR
38Related to this is the difference between "Perl_-less" and "Perl_-ly"
39APIs, for example:
40
41 Perl_sv_setiv(aTHX_ ...);
42 sv_setiv(...);
43
44The first one explicitly passes in the context, which is needed for
9b22382a
FC
45e.g. threaded builds. The second one does that implicitly; do not get
46them mixed. If you are not passing in a aTHX_, you will need to do a
04c692a8
DR
47dTHX (or a dVAR) as the first thing in the function.
48
49See L<perlguts/"How multiple interpreters and concurrency are
50supported"> for further discussion about context.
51
52=item *
53
54Not compiling with -DDEBUGGING
55
56The DEBUGGING define exposes more code to the compiler, therefore more
9b22382a 57ways for things to go wrong. You should try it.
04c692a8
DR
58
59=item *
60
61Introducing (non-read-only) globals
62
63Do not introduce any modifiable globals, truly global or file static.
64They are bad form and complicate multithreading and other forms of
9b22382a 65concurrency. The right way is to introduce them as new interpreter
04c692a8
DR
66variables, see F<intrpvar.h> (at the very end for binary
67compatibility).
68
69Introducing read-only (const) globals is okay, as long as you verify
70with e.g. C<nm libperl.a|egrep -v ' [TURtr] '> (if your C<nm> has
9b22382a 71BSD-style output) that the data you added really is read-only. (If it
04c692a8
DR
72is, it shouldn't show up in the output of that command.)
73
74If you want to have static strings, make them constant:
75
76 static const char etc[] = "...";
77
78If you want to have arrays of constant strings, note carefully the
79right combination of C<const>s:
80
81 static const char * const yippee[] =
82 {"hi", "ho", "silver"};
83
84There is a way to completely hide any modifiable globals (they are all
85moved to heap), the compilation setting
9b22382a 86C<-DPERL_GLOBAL_STRUCT_PRIVATE>. It is not normally used, but can be
04c692a8
DR
87used for testing, read more about it in L<perlguts/"Background and
88PERL_IMPLICIT_CONTEXT">.
89
90=item *
91
92Not exporting your new function
93
94Some platforms (Win32, AIX, VMS, OS/2, to name a few) require any
95function that is part of the public API (the shared Perl library) to be
9b22382a 96explicitly marked as exported. See the discussion about F<embed.pl> in
04c692a8
DR
97L<perlguts>.
98
99=item *
100
101Exporting your new function
102
103The new shiny result of either genuine new functionality or your
9b22382a 104arduous refactoring is now ready and correctly exported. So what could
04c692a8
DR
105possibly go wrong?
106
107Maybe simply that your function did not need to be exported in the
9b22382a 108first place. Perl has a long and not so glorious history of exporting
04c692a8
DR
109functions that it should not have.
110
111If the function is used only inside one source code file, make it
9b22382a 112static. See the discussion about F<embed.pl> in L<perlguts>.
04c692a8
DR
113
114If the function is used across several files, but intended only for
115Perl's internal use (and this should be the common case), do not export
9b22382a 116it to the public API. See the discussion about F<embed.pl> in
04c692a8
DR
117L<perlguts>.
118
119=back
120
121=head2 Portability problems
122
123The following are common causes of compilation and/or execution
9b22382a
FC
124failures, not common to Perl as such. The C FAQ is good bedtime
125reading. Please test your changes with as many C compilers and
04c692a8
DR
126platforms as possible; we will, anyway, and it's nice to save oneself
127from public embarrassment.
128
129If using gcc, you can add the C<-std=c89> option which will hopefully
9b22382a 130catch most of these unportabilities. (However it might also catch
04c692a8
DR
131incompatibilities in your system's header files.)
132
133Use the Configure C<-Dgccansipedantic> flag to enable the gcc C<-ansi
134-pedantic> flags which enforce stricter ANSI rules.
135
136If using the C<gcc -Wall> note that not all the possible warnings (like
137C<-Wunitialized>) are given unless you also compile with C<-O>.
138
139Note that if using gcc, starting from Perl 5.9.5 the Perl core source
140code files (the ones at the top level of the source code distribution,
141but not e.g. the extensions under ext/) are automatically compiled with
142as many as possible of the C<-std=c89>, C<-ansi>, C<-pedantic>, and a
143selection of C<-W> flags (see cflags.SH).
144
145Also study L<perlport> carefully to avoid any bad assumptions about the
eb9df707 146operating system, filesystems, character set, and so forth.
04c692a8
DR
147
148You may once in a while try a "make microperl" to see whether we can
9b22382a 149still compile Perl with just the bare minimum of interfaces. (See
04c692a8
DR
150README.micro.)
151
152Do not assume an operating system indicates a certain compiler.
153
154=over 4
155
156=item *
157
158Casting pointers to integers or casting integers to pointers
159
160 void castaway(U8* p)
161 {
162 IV i = p;
163
164or
165
166 void castaway(U8* p)
167 {
168 IV i = (IV)p;
169
9b22382a
FC
170Both are bad, and broken, and unportable. Use the PTR2IV() macro that
171does it right. (Likewise, there are PTR2UV(), PTR2NV(), INT2PTR(), and
04c692a8
DR
172NUM2PTR().)
173
174=item *
175
28ffebaf 176Casting between function pointers and data pointers
04c692a8
DR
177
178Technically speaking casting between function pointers and data
179pointers is unportable and undefined, but practically speaking it seems
180to work, but you should use the FPTR2DPTR() and DPTR2FPTR() macros.
181Sometimes you can also play games with unions.
182
183=item *
184
185Assuming sizeof(int) == sizeof(long)
186
187There are platforms where longs are 64 bits, and platforms where ints
188are 64 bits, and while we are out to shock you, even platforms where
9b22382a 189shorts are 64 bits. This is all legal according to the C standard. (In
04c692a8
DR
190other words, "long long" is not a portable way to specify 64 bits, and
191"long long" is not even guaranteed to be any wider than "long".)
192
193Instead, use the definitions IV, UV, IVSIZE, I32SIZE, and so forth.
194Avoid things like I32 because they are B<not> guaranteed to be
195I<exactly> 32 bits, they are I<at least> 32 bits, nor are they
9b22382a 196guaranteed to be B<int> or B<long>. If you really explicitly need
04c692a8
DR
19764-bit variables, use I64 and U64, but only if guarded by HAS_QUAD.
198
199=item *
200
201Assuming one can dereference any type of pointer for any type of data
202
203 char *p = ...;
204 long pony = *p; /* BAD */
205
206Many platforms, quite rightly so, will give you a core dump instead of
768312ab 207a pony if the p happens not to be correctly aligned.
04c692a8
DR
208
209=item *
210
211Lvalue casts
212
213 (int)*p = ...; /* BAD */
214
9b22382a 215Simply not portable. Get your lvalue to be of the right type, or maybe
04c692a8
DR
216use temporary variables, or dirty tricks with unions.
217
218=item *
219
220Assume B<anything> about structs (especially the ones you don't
221control, like the ones coming from the system headers)
222
223=over 8
224
225=item *
226
227That a certain field exists in a struct
228
229=item *
230
231That no other fields exist besides the ones you know of
232
233=item *
234
235That a field is of certain signedness, sizeof, or type
236
237=item *
238
239That the fields are in a certain order
240
241=over 8
242
243=item *
244
245While C guarantees the ordering specified in the struct definition,
246between different platforms the definitions might differ
247
248=back
249
250=item *
251
252That the sizeof(struct) or the alignments are the same everywhere
253
254=over 8
255
256=item *
257
258There might be padding bytes between the fields to align the fields -
259the bytes can be anything
260
261=item *
262
263Structs are required to be aligned to the maximum alignment required by
264the fields - which for native types is for usually equivalent to
265sizeof() of the field
266
267=back
268
269=back
270
271=item *
272
273Assuming the character set is ASCIIish
274
9b22382a 275Perl can compile and run under EBCDIC platforms. See L<perlebcdic>.
04c692a8
DR
276This is transparent for the most part, but because the character sets
277differ, you shouldn't use numeric (decimal, octal, nor hex) constants
eb9df707
KW
278to refer to characters. You can safely say C<'A'>, but not C<0x41>.
279You can safely say C<'\n'>, but not C<\012>. However, you can use
280macros defined in F<utf8.h> to specify any code point portably.
281C<LATIN1_TO_NATIVE(0xDF)> is going to be the code point that means
282LATIN SMALL LETTER SHARP S on whatever platform you are running on (on
283ASCII platforms it compiles without adding any extra code, so there is
284zero performance hit on those). The acceptable inputs to
285C<LATIN1_TO_NATIVE> are from C<0x00> through C<0xFF>. If your input
286isn't guaranteed to be in that range, use C<UNICODE_TO_NATIVE> instead.
287C<NATIVE_TO_LATIN1> and C<NATIVE_TO_UNICODE> translate the opposite
288direction.
289
290If you need the string representation of a character that doesn't have a
291mnemonic name in C, you should add it to the list in
292F<regen/unicode_constants.pl>, and have Perl create C<#define>s for you,
eb6d698b 293based on the current platform.
04c692a8 294
eb9df707
KW
295Note that the C<isI<FOO>> and C<toI<FOO>> macros in F<handy.h> work
296properly on native code points and strings.
297
04c692a8 298Also, the range 'A' - 'Z' in ASCII is an unbroken sequence of 26 upper
9b22382a
FC
299case alphabetic characters. That is not true in EBCDIC. Nor for 'a' to
300'z'. But '0' - '9' is an unbroken range in both systems. Don't assume
04c692a8
DR
301anything about other ranges.
302
303Many of the comments in the existing code ignore the possibility of
9b22382a 304EBCDIC, and may be wrong therefore, even if the code works. This is
04c692a8
DR
305actually a tribute to the successful transparent insertion of being
306able to handle EBCDIC without having to change pre-existing code.
307
308UTF-8 and UTF-EBCDIC are two different encodings used to represent
9b22382a 309Unicode code points as sequences of bytes. Macros with the same names
eb9df707 310(but different definitions) in F<utf8.h> and F<utfebcdic.h> are used to
04c692a8
DR
311allow the calling code to think that there is only one such encoding.
312This is almost always referred to as C<utf8>, but it means the EBCDIC
9b22382a 313version as well. Again, comments in the code may well be wrong even if
eb9df707 314the code itself is right. For example, the concept of UTF-8 C<invariant
9b22382a
FC
315characters> differs between ASCII and EBCDIC. On ASCII platforms, only
316characters that do not have the high-order bit set (i.e. whose ordinals
04c692a8
DR
317are strict ASCII, 0 - 127) are invariant, and the documentation and
318comments in the code may assume that, often referring to something
9b22382a 319like, say, C<hibit>. The situation differs and is not so simple on
04c692a8
DR
320EBCDIC machines, but as long as the code itself uses the
321C<NATIVE_IS_INVARIANT()> macro appropriately, it works, even if the
322comments are wrong.
323
324=item *
325
326Assuming the character set is just ASCII
327
9b22382a 328ASCII is a 7 bit encoding, but bytes have 8 bits in them. The 128 extra
04c692a8
DR
329characters have different meanings depending on the locale. Absent a
330locale, currently these extra characters are generally considered to be
eb9df707
KW
331unassigned, and this has presented some problems. This has being
332changed starting in 5.12 so that these characters can be considered to
333be Latin-1 (ISO-8859-1).
04c692a8
DR
334
335=item *
336
337Mixing #define and #ifdef
338
339 #define BURGLE(x) ... \
340 #ifdef BURGLE_OLD_STYLE /* BAD */
341 ... do it the old way ... \
342 #else
343 ... do it the new way ... \
344 #endif
345
9b22382a 346You cannot portably "stack" cpp directives. For example in the above
04c692a8
DR
347you need two separate BURGLE() #defines, one for each #ifdef branch.
348
349=item *
350
351Adding non-comment stuff after #endif or #else
352
353 #ifdef SNOSH
354 ...
355 #else !SNOSH /* BAD */
356 ...
357 #endif SNOSH /* BAD */
358
359The #endif and #else cannot portably have anything non-comment after
9b22382a 360them. If you want to document what is going (which is a good idea
04c692a8
DR
361especially if the branches are long), use (C) comments:
362
363 #ifdef SNOSH
364 ...
365 #else /* !SNOSH */
366 ...
367 #endif /* SNOSH */
368
369The gcc option C<-Wendif-labels> warns about the bad variant (by
370default on starting from Perl 5.9.4).
371
372=item *
373
374Having a comma after the last element of an enum list
375
376 enum color {
377 CERULEAN,
378 CHARTREUSE,
379 CINNABAR, /* BAD */
380 };
381
9b22382a 382is not portable. Leave out the last comma.
04c692a8
DR
383
384Also note that whether enums are implicitly morphable to ints varies
385between compilers, you might need to (int).
386
387=item *
388
389Using //-comments
390
391 // This function bamfoodles the zorklator. /* BAD */
392
9b22382a 393That is C99 or C++. Perl is C89. Using the //-comments is silently
04c692a8
DR
394allowed by many C compilers but cranking up the ANSI C89 strictness
395(which we like to do) causes the compilation to fail.
396
397=item *
398
399Mixing declarations and code
400
401 void zorklator()
402 {
403 int n = 3;
404 set_zorkmids(n); /* BAD */
405 int q = 4;
406
9b22382a 407That is C99 or C++. Some C compilers allow that, but you shouldn't.
04c692a8
DR
408
409The gcc option C<-Wdeclaration-after-statements> scans for such
410problems (by default on starting from Perl 5.9.4).
411
412=item *
413
414Introducing variables inside for()
415
416 for(int i = ...; ...; ...) { /* BAD */
417
9b22382a 418That is C99 or C++. While it would indeed be awfully nice to have that
04c692a8
DR
419also in C89, to limit the scope of the loop variable, alas, we cannot.
420
421=item *
422
423Mixing signed char pointers with unsigned char pointers
424
425 int foo(char *s) { ... }
426 ...
427 unsigned char *t = ...; /* Or U8* t = ... */
428 foo(t); /* BAD */
429
430While this is legal practice, it is certainly dubious, and downright
431fatal in at least one platform: for example VMS cc considers this a
9b22382a 432fatal error. One cause for people often making this mistake is that a
04c692a8
DR
433"naked char" and therefore dereferencing a "naked char pointer" have an
434undefined signedness: it depends on the compiler and the flags of the
435compiler and the underlying platform whether the result is signed or
9b22382a 436unsigned. For this very same reason using a 'char' as an array index is
04c692a8
DR
437bad.
438
439=item *
440
441Macros that have string constants and their arguments as substrings of
442the string constants
443
444 #define FOO(n) printf("number = %d\n", n) /* BAD */
445 FOO(10);
446
447Pre-ANSI semantics for that was equivalent to
448
449 printf("10umber = %d\10");
450
9b22382a 451which is probably not what you were expecting. Unfortunately at least
04c692a8
DR
452one reasonably common and modern C compiler does "real backward
453compatibility" here, in AIX that is what still happens even though the
454rest of the AIX compiler is very happily C89.
455
456=item *
457
458Using printf formats for non-basic C types
459
460 IV i = ...;
461 printf("i = %d\n", i); /* BAD */
462
463While this might by accident work in some platform (where IV happens to
9b22382a 464be an C<int>), in general it cannot. IV might be something larger. Even
04c692a8
DR
465worse the situation is with more specific types (defined by Perl's
466configuration step in F<config.h>):
467
468 Uid_t who = ...;
469 printf("who = %d\n", who); /* BAD */
470
471The problem here is that Uid_t might be not only not C<int>-wide but it
472might also be unsigned, in which case large uids would be printed as
473negative values.
474
475There is no simple solution to this because of printf()'s limited
476intelligence, but for many types the right format is available as with
477either 'f' or '_f' suffix, for example:
478
479 IVdf /* IV in decimal */
480 UVxf /* UV is hexadecimal */
481
482 printf("i = %"IVdf"\n", i); /* The IVdf is a string constant. */
483
484 Uid_t_f /* Uid_t in decimal */
485
486 printf("who = %"Uid_t_f"\n", who);
487
488Or you can try casting to a "wide enough" type:
489
490 printf("i = %"IVdf"\n", (IV)something_very_small_and_signed);
491
492Also remember that the C<%p> format really does require a void pointer:
493
494 U8* p = ...;
495 printf("p = %p\n", (void*)p);
496
497The gcc option C<-Wformat> scans for such problems.
498
499=item *
500
501Blindly using variadic macros
502
503gcc has had them for a while with its own syntax, and C99 brought them
9b22382a 504with a standardized syntax. Don't use the former, and use the latter
04c692a8
DR
505only if the HAS_C99_VARIADIC_MACROS is defined.
506
507=item *
508
509Blindly passing va_list
510
511Not all platforms support passing va_list to further varargs (stdarg)
9b22382a 512functions. The right thing to do is to copy the va_list using the
04c692a8
DR
513Perl_va_copy() if the NEED_VA_COPY is defined.
514
515=item *
516
517Using gcc statement expressions
518
519 val = ({...;...;...}); /* BAD */
520
9b22382a 521While a nice extension, it's not portable. The Perl code does
04c692a8
DR
522admittedly use them if available to gain some extra speed (essentially
523as a funky form of inlining), but you shouldn't.
524
525=item *
526
527Binding together several statements in a macro
528
529Use the macros STMT_START and STMT_END.
530
531 STMT_START {
532 ...
533 } STMT_END
534
535=item *
536
537Testing for operating systems or versions when should be testing for
538features
539
540 #ifdef __FOONIX__ /* BAD */
541 foo = quux();
542 #endif
543
544Unless you know with 100% certainty that quux() is only ever available
545for the "Foonix" operating system B<and> that is available B<and>
546correctly working for B<all> past, present, B<and> future versions of
9b22382a 547"Foonix", the above is very wrong. This is more correct (though still
04c692a8
DR
548not perfect, because the below is a compile-time check):
549
550 #ifdef HAS_QUUX
551 foo = quux();
552 #endif
553
554How does the HAS_QUUX become defined where it needs to be? Well, if
555Foonix happens to be Unixy enough to be able to run the Configure
556script, and Configure has been taught about detecting and testing
9b22382a 557quux(), the HAS_QUUX will be correctly defined. In other platforms, the
04c692a8
DR
558corresponding configuration step will hopefully do the same.
559
560In a pinch, if you cannot wait for Configure to be educated, or if you
561have a good hunch of where quux() might be available, you can
562temporarily try the following:
563
564 #if (defined(__FOONIX__) || defined(__BARNIX__))
565 # define HAS_QUUX
566 #endif
567
568 ...
569
570 #ifdef HAS_QUUX
571 foo = quux();
572 #endif
573
574But in any case, try to keep the features and operating systems
575separate.
576
577=back
578
579=head2 Problematic System Interfaces
580
581=over 4
582
583=item *
584
9b22382a
FC
585malloc(0), realloc(0), calloc(0, 0) are non-portable. To be portable
586allocate at least one byte. (In general you should rarely need to work
04c692a8
DR
587at this low level, but instead use the various malloc wrappers.)
588
589=item *
590
9b22382a 591snprintf() - the return type is unportable. Use my_snprintf() instead.
04c692a8
DR
592
593=back
594
595=head2 Security problems
596
597Last but not least, here are various tips for safer coding.
bbc89b61 598See also L<perlclib> for libc/stdio replacements one should use.
04c692a8
DR
599
600=over 4
601
602=item *
603
604Do not use gets()
605
9b22382a 606Or we will publicly ridicule you. Seriously.
04c692a8
DR
607
608=item *
609
bbc89b61
JH
610Do not use tmpfile()
611
612Use mkstemp() instead.
613
614=item *
615
04c692a8
DR
616Do not use strcpy() or strcat() or strncpy() or strncat()
617
618Use my_strlcpy() and my_strlcat() instead: they either use the native
619implementation, or Perl's own implementation (borrowed from the public
620domain implementation of INN).
621
622=item *
623
624Do not use sprintf() or vsprintf()
625
626If you really want just plain byte strings, use my_snprintf() and
627my_vsnprintf() instead, which will try to use snprintf() and
9b22382a 628vsnprintf() if those safer APIs are available. If you want something
6bfe0388
KW
629fancier than a plain byte string, use
630L<C<Perl_form>()|perlapi/form> or SVs and
631L<C<Perl_sv_catpvf()>|perlapi/sv_catpvf>.
632
2e642750
KW
633Note that glibc C<printf()>, C<sprintf()>, etc. are buggy before glibc
634version 2.17. They won't allow a C<%.s> format with a precision to
635create a string that isn't valid UTF-8 if the current underlying locale
636of the program is UTF-8. What happens is that the C<%s> and its operand are
6bfe0388 637simply skipped without any notice.
2e642750 638L<https://sourceware.org/bugzilla/show_bug.cgi?id=6530>.
04c692a8 639
c98823ff
JH
640=item *
641
642Do not use atoi()
643
22ff3130 644Use grok_atoUV() instead. atoi() has ill-defined behavior on overflows,
c98823ff 645and cannot be used for incremental parsing. It is also affected by locale,
338aa8b0
JH
646which is bad.
647
648=item *
649
650Do not use strtol() or strtoul()
651
22ff3130 652Use grok_atoUV() instead. strtol() or strtoul() (or their IV/UV-friendly
338aa8b0
JH
653macro disguises, Strtol() and Strtoul(), or Atol() and Atoul() are
654affected by locale, which is bad.
c98823ff 655
04c692a8
DR
656=back
657
658=head1 DEBUGGING
659
660You can compile a special debugging version of Perl, which allows you
661to use the C<-D> option of Perl to tell more about what Perl is doing.
662But sometimes there is no alternative than to dive in with a debugger,
663either to see the stack trace of a core dump (very useful in a bug
664report), or trying to figure out what went wrong before the core dump
665happened, or how did we end up having wrong or unexpected results.
666
667=head2 Poking at Perl
668
669To really poke around with Perl, you'll probably want to build Perl for
670debugging, like this:
671
672 ./Configure -d -D optimize=-g
673 make
674
675C<-g> is a flag to the C compiler to have it produce debugging
676information which will allow us to step through a running program, and
677to see in which C function we are at (without the debugging information
678we might see only the numerical addresses of the functions, which is
679not very helpful).
680
681F<Configure> will also turn on the C<DEBUGGING> compilation symbol
9b22382a 682which enables all the internal debugging code in Perl. There are a
04c692a8
DR
683whole bunch of things you can debug with this: L<perlrun> lists them
684all, and the best way to find out about them is to play about with
9b22382a 685them. The most useful options are probably
04c692a8
DR
686
687 l Context (loop) stack processing
688 t Trace execution
689 o Method and overloading resolution
690 c String/numeric conversions
691
692Some of the functionality of the debugging code can be achieved using
693XS modules.
694
695 -Dr => use re 'debug'
696 -Dx => use O 'Debug'
697
698=head2 Using a source-level debugger
699
700If the debugging output of C<-D> doesn't help you, it's time to step
701through perl's execution with a source-level debugger.
702
703=over 3
704
705=item *
706
707We'll use C<gdb> for our examples here; the principles will apply to
708any debugger (many vendors call their debugger C<dbx>), but check the
709manual of the one you're using.
710
711=back
712
713To fire up the debugger, type
714
715 gdb ./perl
716
717Or if you have a core dump:
718
719 gdb ./perl core
720
721You'll want to do that in your Perl source tree so the debugger can
9b22382a 722read the source code. You should see the copyright message, followed by
04c692a8
DR
723the prompt.
724
725 (gdb)
726
727C<help> will get you into the documentation, but here are the most
728useful commands:
729
730=over 3
731
732=item * run [args]
733
734Run the program with the given arguments.
735
736=item * break function_name
737
738=item * break source.c:xxx
739
740Tells the debugger that we'll want to pause execution when we reach
741either the named function (but see L<perlguts/Internal Functions>!) or
742the given line in the named source file.
743
744=item * step
745
746Steps through the program a line at a time.
747
748=item * next
749
750Steps through the program a line at a time, without descending into
751functions.
752
753=item * continue
754
755Run until the next breakpoint.
756
757=item * finish
758
759Run until the end of the current function, then stop again.
760
761=item * 'enter'
762
763Just pressing Enter will do the most recent operation again - it's a
764blessing when stepping through miles of source code.
765
8b029fdf
MH
766=item * ptype
767
768Prints the C definition of the argument given.
769
770 (gdb) ptype PL_op
771 type = struct op {
772 OP *op_next;
773 OP *op_sibling;
774 OP *(*op_ppaddr)(void);
775 PADOFFSET op_targ;
776 unsigned int op_type : 9;
777 unsigned int op_opt : 1;
778 unsigned int op_slabbed : 1;
779 unsigned int op_savefree : 1;
780 unsigned int op_static : 1;
781 unsigned int op_folded : 1;
782 unsigned int op_spare : 2;
783 U8 op_flags;
784 U8 op_private;
785 } *
786
04c692a8
DR
787=item * print
788
9b22382a 789Execute the given C code and print its results. B<WARNING>: Perl makes
04c692a8 790heavy use of macros, and F<gdb> does not necessarily support macros
9b22382a 791(see later L</"gdb macro support">). You'll have to substitute them
04c692a8
DR
792yourself, or to invoke cpp on the source code files (see L</"The .i
793Targets">) So, for instance, you can't say
794
795 print SvPV_nolen(sv)
796
797but you have to say
798
799 print Perl_sv_2pv_nolen(sv)
800
801=back
802
803You may find it helpful to have a "macro dictionary", which you can
9b22382a 804produce by saying C<cpp -dM perl.c | sort>. Even then, F<cpp> won't
04c692a8
DR
805recursively apply those macros for you.
806
807=head2 gdb macro support
808
809Recent versions of F<gdb> have fairly good macro support, but in order
810to use it you'll need to compile perl with macro definitions included
9b22382a
FC
811in the debugging information. Using F<gcc> version 3.1, this means
812configuring with C<-Doptimize=-g3>. Other compilers might use a
04c692a8
DR
813different switch (if they support debugging macros at all).
814
815=head2 Dumping Perl Data Structures
816
817One way to get around this macro hell is to use the dumping functions
818in F<dump.c>; these work a little like an internal
819L<Devel::Peek|Devel::Peek>, but they also cover OPs and other
9b22382a 820structures that you can't get at from Perl. Let's take an example.
04c692a8 821We'll use the C<$a = $b + $c> we used before, but give it a bit of
9b22382a 822context: C<$b = "6XXXX"; $c = 2.3;>. Where's a good place to stop and
04c692a8
DR
823poke around?
824
825What about C<pp_add>, the function we examined earlier to implement the
826C<+> operator:
827
828 (gdb) break Perl_pp_add
829 Breakpoint 1 at 0x46249f: file pp_hot.c, line 309.
830
831Notice we use C<Perl_pp_add> and not C<pp_add> - see
9b22382a 832L<perlguts/Internal Functions>. With the breakpoint in place, we can
04c692a8
DR
833run our program:
834
835 (gdb) run -e '$b = "6XXXX"; $c = 2.3; $a = $b + $c'
836
837Lots of junk will go past as gdb reads in the relevant source files and
838libraries, and then:
839
840 Breakpoint 1, Perl_pp_add () at pp_hot.c:309
841 309 dSP; dATARGET; tryAMAGICbin(add,opASSIGN);
842 (gdb) step
843 311 dPOPTOPnnrl_ul;
844 (gdb)
845
846We looked at this bit of code before, and we said that
847C<dPOPTOPnnrl_ul> arranges for two C<NV>s to be placed into C<left> and
848C<right> - let's slightly expand it:
849
850 #define dPOPTOPnnrl_ul NV right = POPn; \
851 SV *leftsv = TOPs; \
852 NV left = USE_LEFT(leftsv) ? SvNV(leftsv) : 0.0
853
854C<POPn> takes the SV from the top of the stack and obtains its NV
855either directly (if C<SvNOK> is set) or by calling the C<sv_2nv>
9b22382a
FC
856function. C<TOPs> takes the next SV from the top of the stack - yes,
857C<POPn> uses C<TOPs> - but doesn't remove it. We then use C<SvNV> to
04c692a8
DR
858get the NV from C<leftsv> in the same way as before - yes, C<POPn> uses
859C<SvNV>.
860
861Since we don't have an NV for C<$b>, we'll have to use C<sv_2nv> to
9b22382a 862convert it. If we step again, we'll find ourselves there:
04c692a8 863
8b029fdf 864 (gdb) step
04c692a8
DR
865 Perl_sv_2nv (sv=0xa0675d0) at sv.c:1669
866 1669 if (!sv)
867 (gdb)
868
869We can now use C<Perl_sv_dump> to investigate the SV:
870
8b029fdf 871 (gdb) print Perl_sv_dump(sv)
04c692a8
DR
872 SV = PV(0xa057cc0) at 0xa0675d0
873 REFCNT = 1
874 FLAGS = (POK,pPOK)
875 PV = 0xa06a510 "6XXXX"\0
876 CUR = 5
877 LEN = 6
878 $1 = void
879
880We know we're going to get C<6> from this, so let's finish the
881subroutine:
882
883 (gdb) finish
884 Run till exit from #0 Perl_sv_2nv (sv=0xa0675d0) at sv.c:1671
885 0x462669 in Perl_pp_add () at pp_hot.c:311
886 311 dPOPTOPnnrl_ul;
887
888We can also dump out this op: the current op is always stored in
9b22382a 889C<PL_op>, and we can dump it with C<Perl_op_dump>. This'll give us
04c692a8
DR
890similar output to L<B::Debug|B::Debug>.
891
8b029fdf 892 (gdb) print Perl_op_dump(PL_op)
04c692a8
DR
893 {
894 13 TYPE = add ===> 14
895 TARG = 1
896 FLAGS = (SCALAR,KIDS)
897 {
898 TYPE = null ===> (12)
899 (was rv2sv)
900 FLAGS = (SCALAR,KIDS)
901 {
902 11 TYPE = gvsv ===> 12
903 FLAGS = (SCALAR)
904 GV = main::b
905 }
906 }
907
908# finish this later #
909
8b029fdf
MH
910=head2 Using gdb to look at specific parts of a program
911
912With the example above, you knew to look for C<Perl_pp_add>, but what if
913there were multiple calls to it all over the place, or you didn't know what
914the op was you were looking for?
915
916One way to do this is to inject a rare call somewhere near what you're looking
9b22382a 917for. For example, you could add C<study> before your method:
8b029fdf
MH
918
919 study;
920
921And in gdb do:
922
923 (gdb) break Perl_pp_study
924
9b22382a
FC
925And then step until you hit what you're
926looking for. This works well in a loop
8b029fdf
MH
927if you want to only break at certain iterations:
928
929 for my $c (1..100) {
930 study if $c == 50;
931 }
932
933=head2 Using gdb to look at what the parser/lexer are doing
934
935If you want to see what perl is doing when parsing/lexing your code, you can
72b22e55 936use C<BEGIN {}>:
8b029fdf
MH
937
938 print "Before\n";
939 BEGIN { study; }
940 print "After\n";
941
942And in gdb:
943
944 (gdb) break Perl_pp_study
945
946If you want to see what the parser/lexer is doing inside of C<if> blocks and
947the like you need to be a little trickier:
948
949 if ($a && $b && do { BEGIN { study } 1 } && $c) { ... }
950
04c692a8
DR
951=head1 SOURCE CODE STATIC ANALYSIS
952
953Various tools exist for analysing C source code B<statically>, as
9b22382a 954opposed to B<dynamically>, that is, without executing the code. It is
04c692a8
DR
955possible to detect resource leaks, undefined behaviour, type
956mismatches, portability problems, code paths that would cause illegal
957memory accesses, and other similar problems by just parsing the C code
958and looking at the resulting graph, what does it tell about the
9b22382a 959execution and data flows. As a matter of fact, this is exactly how C
04c692a8
DR
960compilers know to give warnings about dubious code.
961
962=head2 lint, splint
963
964The good old C code quality inspector, C<lint>, is available in several
965platforms, but please be aware that there are several different
966implementations of it by different vendors, which means that the flags
967are not identical across different platforms.
968
969There is a lint variant called C<splint> (Secure Programming Lint)
970available from http://www.splint.org/ that should compile on any
971Unix-like platform.
972
973There are C<lint> and <splint> targets in Makefile, but you may have to
974diddle with the flags (see above).
975
976=head2 Coverity
977
978Coverity (http://www.coverity.com/) is a product similar to lint and as
979a testbed for their product they periodically check several open source
980projects, and they give out accounts to open source developers to the
981defect databases.
982
983=head2 cpd (cut-and-paste detector)
984
9b22382a 985The cpd tool detects cut-and-paste coding. If one instance of the
04c692a8 986cut-and-pasted code changes, all the other spots should probably be
9b22382a 987changed, too. Therefore such code should probably be turned into a
04c692a8
DR
988subroutine or a macro.
989
990cpd (http://pmd.sourceforge.net/cpd.html) is part of the pmd project
9b22382a 991(http://pmd.sourceforge.net/). pmd was originally written for static
04c692a8
DR
992analysis of Java code, but later the cpd part of it was extended to
993parse also C and C++.
994
995Download the pmd-bin-X.Y.zip () from the SourceForge site, extract the
996pmd-X.Y.jar from it, and then run that on source code thusly:
997
0cbf2b31
FC
998 java -cp pmd-X.Y.jar net.sourceforge.pmd.cpd.CPD \
999 --minimum-tokens 100 --files /some/where/src --language c > cpd.txt
04c692a8
DR
1000
1001You may run into memory limits, in which case you should use the -Xmx
1002option:
1003
1004 java -Xmx512M ...
1005
1006=head2 gcc warnings
1007
1008Though much can be written about the inconsistency and coverage
1009problems of gcc warnings (like C<-Wall> not meaning "all the warnings",
1010or some common portability problems not being covered by C<-Wall>, or
1011C<-ansi> and C<-pedantic> both being a poorly defined collection of
1012warnings, and so forth), gcc is still a useful tool in keeping our
1013coding nose clean.
1014
1015The C<-Wall> is by default on.
1016
1017The C<-ansi> (and its sidekick, C<-pedantic>) would be nice to be on
1018always, but unfortunately they are not safe on all platforms, they can
1019for example cause fatal conflicts with the system headers (Solaris
9b22382a 1020being a prime example). If Configure C<-Dgccansipedantic> is used, the
04c692a8
DR
1021C<cflags> frontend selects C<-ansi -pedantic> for the platforms where
1022they are known to be safe.
1023
1024Starting from Perl 5.9.4 the following extra flags are added:
1025
1026=over 4
1027
1028=item *
1029
1030C<-Wendif-labels>
1031
1032=item *
1033
1034C<-Wextra>
1035
1036=item *
1037
1038C<-Wdeclaration-after-statement>
1039
1040=back
1041
1042The following flags would be nice to have but they would first need
1043their own Augean stablemaster:
1044
1045=over 4
1046
1047=item *
1048
1049C<-Wpointer-arith>
1050
1051=item *
1052
1053C<-Wshadow>
1054
1055=item *
1056
1057C<-Wstrict-prototypes>
1058
1059=back
1060
1061The C<-Wtraditional> is another example of the annoying tendency of gcc
1062to bundle a lot of warnings under one switch (it would be impossible to
1063deploy in practice because it would complain a lot) but it does contain
1064some warnings that would be beneficial to have available on their own,
1065such as the warning about string constants inside macros containing the
1066macro arguments: this behaved differently pre-ANSI than it does in
1067ANSI, and some C compilers are still in transition, AIX being an
1068example.
1069
1070=head2 Warnings of other C compilers
1071
1072Other C compilers (yes, there B<are> other C compilers than gcc) often
1073have their "strict ANSI" or "strict ANSI with some portability
1074extensions" modes on, like for example the Sun Workshop has its C<-Xa>
1075mode on (though implicitly), or the DEC (these days, HP...) has its
1076C<-std1> mode on.
1077
1078=head1 MEMORY DEBUGGERS
1079
d1fd4856
VP
1080B<NOTE 1>: Running under older memory debuggers such as Purify,
1081valgrind or Third Degree greatly slows down the execution: seconds
9b22382a 1082become minutes, minutes become hours. For example as of Perl 5.8.1, the
04c692a8 1083ext/Encode/t/Unicode.t takes extraordinarily long to complete under
9b22382a
FC
1084e.g. Purify, Third Degree, and valgrind. Under valgrind it takes more
1085than six hours, even on a snappy computer. The said test must be doing
1086something that is quite unfriendly for memory debuggers. If you don't
04c692a8 1087feel like waiting, that you can simply kill away the perl process.
d1fd4856
VP
1088Roughly valgrind slows down execution by factor 10, AddressSanitizer by
1089factor 2.
04c692a8
DR
1090
1091B<NOTE 2>: To minimize the number of memory leak false alarms (see
1092L</PERL_DESTRUCT_LEVEL> for more information), you have to set the
9b22382a 1093environment variable PERL_DESTRUCT_LEVEL to 2. For example, like this:
04c692a8
DR
1094
1095 env PERL_DESTRUCT_LEVEL=2 valgrind ./perl -Ilib ...
1096
1097B<NOTE 3>: There are known memory leaks when there are compile-time
1098errors within eval or require, seeing C<S_doeval> in the call stack is
9b22382a 1099a good sign of these. Fixing these leaks is non-trivial, unfortunately,
04c692a8
DR
1100but they must be fixed eventually.
1101
1102B<NOTE 4>: L<DynaLoader> will not clean up after itself completely
1103unless Perl is built with the Configure option
1104C<-Accflags=-DDL_UNLOAD_ALL_AT_EXIT>.
1105
04c692a8
DR
1106=head2 valgrind
1107
d1fd4856 1108The valgrind tool can be used to find out both memory leaks and illegal
9b22382a
FC
1109heap memory accesses. As of version 3.3.0, Valgrind only supports Linux
1110on x86, x86-64 and PowerPC and Darwin (OS X) on x86 and x86-64). The
d1fd4856 1111special "test.valgrind" target can be used to run the tests under
9b22382a 1112valgrind. Found errors and memory leaks are logged in files named
037ab3f1
MH
1113F<testfile.valgrind> and by default output is displayed inline.
1114
1115Example usage:
1116
1117 make test.valgrind
1118
1119Since valgrind adds significant overhead, tests will take much longer to
1120run. The valgrind tests support being run in parallel to help with this:
1121
1122 TEST_JOBS=9 make test.valgrind
1123
1124Note that the above two invocations will be very verbose as reachable
1125memory and leak-checking is enabled by default. If you want to just see
1126pure errors, try:
1127
1128 VG_OPTS='-q --leak-check=no --show-reachable=no' TEST_JOBS=9 \
1129 make test.valgrind
04c692a8
DR
1130
1131Valgrind also provides a cachegrind tool, invoked on perl as:
1132
1133 VG_OPTS=--tool=cachegrind make test.valgrind
1134
1135As system libraries (most notably glibc) are also triggering errors,
9b22382a 1136valgrind allows to suppress such errors using suppression files. The
04c692a8 1137default suppression file that comes with valgrind already catches a lot
9b22382a 1138of them. Some additional suppressions are defined in F<t/perl.supp>.
04c692a8
DR
1139
1140To get valgrind and for more information see
1141
0061d4fa 1142 http://valgrind.org/
04c692a8 1143
81c3bbe7
RU
1144=head2 AddressSanitizer
1145
4dd56148 1146AddressSanitizer is a clang and gcc extension, included in clang since
9b22382a 1147v3.1 and gcc since v4.8. It checks illegal heap pointers, global
4dd56148
NC
1148pointers, stack pointers and use after free errors, and is fast enough
1149that you can easily compile your debugging or optimized perl with it.
9b22382a 1150It does not check memory leaks though. AddressSanitizer is available
4dd56148 1151for Linux, Mac OS X and soon on Windows.
81c3bbe7 1152
8a64fbaa
VP
1153To build perl with AddressSanitizer, your Configure invocation should
1154look like:
81c3bbe7 1155
e8596d90
VP
1156 sh Configure -des -Dcc=clang \
1157 -Accflags=-faddress-sanitizer -Aldflags=-faddress-sanitizer \
1158 -Alddlflags=-shared\ -faddress-sanitizer
81c3bbe7
RU
1159
1160where these arguments mean:
1161
1162=over 4
1163
1164=item * -Dcc=clang
1165
8a64fbaa
VP
1166This should be replaced by the full path to your clang executable if it
1167is not in your path.
81c3bbe7
RU
1168
1169=item * -Accflags=-faddress-sanitizer
1170
8a64fbaa 1171Compile perl and extensions sources with AddressSanitizer.
81c3bbe7
RU
1172
1173=item * -Aldflags=-faddress-sanitizer
1174
8a64fbaa 1175Link the perl executable with AddressSanitizer.
81c3bbe7 1176
e8596d90 1177=item * -Alddlflags=-shared\ -faddress-sanitizer
81c3bbe7 1178
9b22382a 1179Link dynamic extensions with AddressSanitizer. You must manually
e8596d90
VP
1180specify C<-shared> because using C<-Alddlflags=-shared> will prevent
1181Configure from setting a default value for C<lddlflags>, which usually
5dfc6e97 1182contains C<-shared> (at least on Linux).
81c3bbe7
RU
1183
1184=back
1185
8a64fbaa
VP
1186See also
1187L<http://code.google.com/p/address-sanitizer/wiki/AddressSanitizer>.
81c3bbe7
RU
1188
1189
04c692a8
DR
1190=head1 PROFILING
1191
1192Depending on your platform there are various ways of profiling Perl.
1193
1194There are two commonly used techniques of profiling executables:
1195I<statistical time-sampling> and I<basic-block counting>.
1196
1197The first method takes periodically samples of the CPU program counter,
1198and since the program counter can be correlated with the code generated
1199for functions, we get a statistical view of in which functions the
9b22382a 1200program is spending its time. The caveats are that very small/fast
04c692a8
DR
1201functions have lower probability of showing up in the profile, and that
1202periodically interrupting the program (this is usually done rather
1203frequently, in the scale of milliseconds) imposes an additional
9b22382a 1204overhead that may skew the results. The first problem can be alleviated
04c692a8
DR
1205by running the code for longer (in general this is a good idea for
1206profiling), the second problem is usually kept in guard by the
1207profiling tools themselves.
1208
1209The second method divides up the generated code into I<basic blocks>.
1210Basic blocks are sections of code that are entered only in the
9b22382a
FC
1211beginning and exited only at the end. For example, a conditional jump
1212starts a basic block. Basic block profiling usually works by
04c692a8 1213I<instrumenting> the code by adding I<enter basic block #nnnn>
9b22382a
FC
1214book-keeping code to the generated code. During the execution of the
1215code the basic block counters are then updated appropriately. The
04c692a8
DR
1216caveat is that the added extra code can skew the results: again, the
1217profiling tools usually try to factor their own effects out of the
1218results.
1219
1220=head2 Gprof Profiling
1221
e2aed43d 1222I<gprof> is a profiling tool available in many Unix platforms which
9b22382a
FC
1223uses I<statistical time-sampling>. You can build a profiled version of
1224F<perl> by compiling using gcc with the flag C<-pg>. Either edit
1225F<config.sh> or re-run F<Configure>. Running the profiled version of
e2aed43d
NC
1226Perl will create an output file called F<gmon.out> which contains the
1227profiling data collected during the execution.
04c692a8 1228
e2aed43d
NC
1229quick hint:
1230
1231 $ sh Configure -des -Dusedevel -Accflags='-pg' \
1232 -Aldflags='-pg' -Alddlflags='-pg -shared' \
1233 && make perl
1234 $ ./perl ... # creates gmon.out in current directory
1235 $ gprof ./perl > out
1236 $ less out
1237
1238(you probably need to add C<-shared> to the <-Alddlflags> line until RT
1239#118199 is resolved)
04c692a8 1240
e2aed43d
NC
1241The F<gprof> tool can then display the collected data in various ways.
1242Usually F<gprof> understands the following options:
04c692a8
DR
1243
1244=over 4
1245
1246=item * -a
1247
1248Suppress statically defined functions from the profile.
1249
1250=item * -b
1251
1252Suppress the verbose descriptions in the profile.
1253
1254=item * -e routine
1255
1256Exclude the given routine and its descendants from the profile.
1257
1258=item * -f routine
1259
1260Display only the given routine and its descendants in the profile.
1261
1262=item * -s
1263
1264Generate a summary file called F<gmon.sum> which then may be given to
1265subsequent gprof runs to accumulate data over several runs.
1266
1267=item * -z
1268
1269Display routines that have zero usage.
1270
1271=back
1272
1273For more detailed explanation of the available commands and output
e2aed43d 1274formats, see your own local documentation of F<gprof>.
04c692a8 1275
e2aed43d 1276=head2 GCC gcov Profiling
04c692a8 1277
e2aed43d
NC
1278I<basic block profiling> is officially available in gcc 3.0 and later.
1279You can build a profiled version of F<perl> by compiling using gcc with
9b22382a 1280the flags C<-fprofile-arcs -ftest-coverage>. Either edit F<config.sh>
e2aed43d 1281or re-run F<Configure>.
04c692a8 1282
e2aed43d 1283quick hint:
04c692a8 1284
e2aed43d
NC
1285 $ sh Configure -des -Dusedevel -Doptimize='-g' \
1286 -Accflags='-fprofile-arcs -ftest-coverage' \
1287 -Aldflags='-fprofile-arcs -ftest-coverage' \
1288 -Alddlflags='-fprofile-arcs -ftest-coverage -shared' \
1289 && make perl
1290 $ rm -f regexec.c.gcov regexec.gcda
1291 $ ./perl ...
1292 $ gcov regexec.c
1293 $ less regexec.c.gcov
04c692a8 1294
e2aed43d
NC
1295(you probably need to add C<-shared> to the <-Alddlflags> line until RT
1296#118199 is resolved)
04c692a8
DR
1297
1298Running the profiled version of Perl will cause profile output to be
9b22382a 1299generated. For each source file an accompanying F<.gcda> file will be
04c692a8
DR
1300created.
1301
e2aed43d 1302To display the results you use the I<gcov> utility (which should be
9b22382a 1303installed if you have gcc 3.0 or newer installed). F<gcov> is run on
04c692a8
DR
1304source code files, like this
1305
1306 gcov sv.c
1307
9b22382a 1308which will cause F<sv.c.gcov> to be created. The F<.gcov> files contain
04c692a8 1309the source code annotated with relative frequencies of execution
9b22382a 1310indicated by "#" markers. If you want to generate F<.gcov> files for
6f134219
NC
1311all profiled object files, you can run something like this:
1312
1313 for file in `find . -name \*.gcno`
1314 do sh -c "cd `dirname $file` && gcov `basename $file .gcno`"
1315 done
04c692a8
DR
1316
1317Useful options of F<gcov> include C<-b> which will summarise the basic
1318block, branch, and function call coverage, and C<-c> which instead of
9b22382a 1319relative frequencies will use the actual counts. For more information
04c692a8 1320on the use of F<gcov> and basic block profiling with gcc, see the
9b22382a 1321latest GNU CC manual. As of gcc 4.8, this is at
e2aed43d 1322L<http://gcc.gnu.org/onlinedocs/gcc/Gcov-Intro.html#Gcov-Intro>
04c692a8
DR
1323
1324=head1 MISCELLANEOUS TRICKS
1325
1326=head2 PERL_DESTRUCT_LEVEL
1327
1328If you want to run any of the tests yourself manually using e.g.
4dd56148
NC
1329valgrind, please note that by default perl B<does not> explicitly
1330cleanup all the memory it has allocated (such as global memory arenas)
1331but instead lets the exit() of the whole program "take care" of such
1332allocations, also known as "global destruction of objects".
04c692a8
DR
1333
1334There is a way to tell perl to do complete cleanup: set the environment
9b22382a 1335variable PERL_DESTRUCT_LEVEL to a non-zero value. The t/TEST wrapper
04c692a8 1336does set this to 2, and this is what you need to do too, if you don't
f01ecde8 1337want to see the "global leaks": For example, for running under valgrind
04c692a8 1338
f01ecde8 1339 env PERL_DESTRUCT_LEVEL=2 valgrind ./perl -Ilib t/foo/bar.t
04c692a8
DR
1340
1341(Note: the mod_perl apache module uses also this environment variable
9b22382a
FC
1342for its own purposes and extended its semantics. Refer to the mod_perl
1343documentation for more information. Also, spawned threads do the
04c692a8
DR
1344equivalent of setting this variable to the value 1.)
1345
1346If, at the end of a run you get the message I<N scalars leaked>, you
1347can recompile with C<-DDEBUG_LEAKING_SCALARS>, which will cause the
1348addresses of all those leaked SVs to be dumped along with details as to
9b22382a
FC
1349where each SV was originally allocated. This information is also
1350displayed by Devel::Peek. Note that the extra details recorded with
04c692a8 1351each SV increases memory usage, so it shouldn't be used in production
9b22382a 1352environments. It also converts C<new_SV()> from a macro into a real
04c692a8
DR
1353function, so you can use your favourite debugger to discover where
1354those pesky SVs were allocated.
1355
1356If you see that you're leaking memory at runtime, but neither valgrind
1357nor C<-DDEBUG_LEAKING_SCALARS> will find anything, you're probably
1358leaking SVs that are still reachable and will be properly cleaned up
9b22382a
FC
1359during destruction of the interpreter. In such cases, using the C<-Dm>
1360switch can point you to the source of the leak. If the executable was
04c692a8 1361built with C<-DDEBUG_LEAKING_SCALARS>, C<-Dm> will output SV
9b22382a 1362allocations in addition to memory allocations. Each SV allocation has a
04c692a8 1363distinct serial number that will be written on creation and destruction
9b22382a 1364of the SV. So if you're executing the leaking code in a loop, you need
04c692a8 1365to look for SVs that are created, but never destroyed between each
9b22382a 1366cycle. If such an SV is found, set a conditional breakpoint within
04c692a8 1367C<new_SV()> and make it break only when C<PL_sv_serial> is equal to the
9b22382a 1368serial number of the leaking SV. Then you will catch the interpreter in
04c692a8
DR
1369exactly the state where the leaking SV is allocated, which is
1370sufficient in many cases to find the source of the leak.
1371
1372As C<-Dm> is using the PerlIO layer for output, it will by itself
9b22382a 1373allocate quite a bunch of SVs, which are hidden to avoid recursion. You
04c692a8
DR
1374can bypass the PerlIO layer if you use the SV logging provided by
1375C<-DPERL_MEM_LOG> instead.
1376
1377=head2 PERL_MEM_LOG
1378
1379If compiled with C<-DPERL_MEM_LOG>, both memory and SV allocations go
1380through logging functions, which is handy for breakpoint setting.
1381
1382Unless C<-DPERL_MEM_LOG_NOIMPL> is also compiled, the logging functions
1383read $ENV{PERL_MEM_LOG} to determine whether to log the event, and if
1384so how:
1385
1386 $ENV{PERL_MEM_LOG} =~ /m/ Log all memory ops
1387 $ENV{PERL_MEM_LOG} =~ /s/ Log all SV ops
1388 $ENV{PERL_MEM_LOG} =~ /t/ include timestamp in Log
1389 $ENV{PERL_MEM_LOG} =~ /^(\d+)/ write to FD given (default is 2)
1390
1391Memory logging is somewhat similar to C<-Dm> but is independent of
1392C<-DDEBUGGING>, and at a higher level; all uses of Newx(), Renew(), and
1393Safefree() are logged with the caller's source code file and line
9b22382a
FC
1394number (and C function name, if supported by the C compiler). In
1395contrast, C<-Dm> is directly at the point of C<malloc()>. SV logging is
04c692a8
DR
1396similar.
1397
1398Since the logging doesn't use PerlIO, all SV allocations are logged and
9b22382a 1399no extra SV allocations are introduced by enabling the logging. If
04c692a8
DR
1400compiled with C<-DDEBUG_LEAKING_SCALARS>, the serial number for each SV
1401allocation is also logged.
1402
1403=head2 DDD over gdb
1404
1405Those debugging perl with the DDD frontend over gdb may find the
1406following useful:
1407
1408You can extend the data conversion shortcuts menu, so for example you
1409can display an SV's IV value with one click, without doing any typing.
1410To do that simply edit ~/.ddd/init file and add after:
1411
1412 ! Display shortcuts.
1413 Ddd*gdbDisplayShortcuts: \
1414 /t () // Convert to Bin\n\
1415 /d () // Convert to Dec\n\
1416 /x () // Convert to Hex\n\
1417 /o () // Convert to Oct(\n\
1418
1419the following two lines:
1420
1421 ((XPV*) (())->sv_any )->xpv_pv // 2pvx\n\
1422 ((XPVIV*) (())->sv_any )->xiv_iv // 2ivx
1423
1424so now you can do ivx and pvx lookups or you can plug there the sv_peek
1425"conversion":
1426
1427 Perl_sv_peek(my_perl, (SV*)()) // sv_peek
1428
9b22382a 1429(The my_perl is for threaded builds.) Just remember that every line,
04c692a8
DR
1430but the last one, should end with \n\
1431
1432Alternatively edit the init file interactively via: 3rd mouse button ->
1433New Display -> Edit Menu
1434
1435Note: you can define up to 20 conversion shortcuts in the gdb section.
1436
470dd224
JH
1437=head2 C backtrace
1438
0762e42f
JH
1439On some platforms Perl supports retrieving the C level backtrace
1440(similar to what symbolic debuggers like gdb do).
470dd224
JH
1441
1442The backtrace returns the stack trace of the C call frames,
1443with the symbol names (function names), the object names (like "perl"),
1444and if it can, also the source code locations (file:line).
1445
0762e42f
JH
1446The supported platforms are Linux, and OS X (some *BSD might
1447work at least partly, but they have not yet been tested).
1448
1449This feature hasn't been tested with multiple threads, but it will
1450only show the backtrace of the thread doing the backtracing.
470dd224
JH
1451
1452The feature needs to be enabled with C<Configure -Dusecbacktrace>.
1453
0762e42f
JH
1454The C<-Dusecbacktrace> also enables keeping the debug information when
1455compiling/linking (often: C<-g>). Many compilers/linkers do support
1456having both optimization and keeping the debug information. The debug
1457information is needed for the symbol names and the source locations.
1458
1459Static functions might not be visible for the backtrace.
470dd224
JH
1460
1461Source code locations, even if available, can often be missing or
0762e42f
JH
1462misleading if the compiler has e.g. inlined code. Optimizer can
1463make matching the source code and the object code quite challenging.
470dd224
JH
1464
1465=over 4
1466
1467=item Linux
1468
59b3baca 1469You B<must> have the BFD (-lbfd) library installed, otherwise C<perl> will
0762e42f 1470fail to link. The BFD is usually distributed as part of the GNU binutils.
470dd224
JH
1471
1472Summary: C<Configure ... -Dusecbacktrace>
1473and you need C<-lbfd>.
1474
1475=item OS X
1476
0762e42f
JH
1477The source code locations are supported B<only> if you have
1478the Developer Tools installed. (BFD is B<not> needed.)
470dd224
JH
1479
1480Summary: C<Configure ... -Dusecbacktrace>
1481and installing the Developer Tools would be good.
1482
1483=back
1484
1485Optionally, for trying out the feature, you may want to enable
0762e42f
JH
1486automatic dumping of the backtrace just before a warning or croak (die)
1487message is emitted, by adding C<-Accflags=-DUSE_C_BACKTRACE_ON_ERROR>
1488for Configure.
470dd224
JH
1489
1490Unless the above additional feature is enabled, nothing about the
1491backtrace functionality is visible, except for the Perl/XS level.
1492
1493Furthermore, even if you have enabled this feature to be compiled,
1494you need to enable it in runtime with an environment variable:
0762e42f
JH
1495C<PERL_C_BACKTRACE_ON_ERROR=10>. It must be an integer higher
1496than zero, telling the desired frame count.
470dd224
JH
1497
1498Retrieving the backtrace from Perl level (using for example an XS
1499extension) would be much less exciting than one would hope: normally
1500you would see C<runops>, C<entersub>, and not much else. This API is
1501intended to be called B<from within> the Perl implementation, not from
1502Perl level execution.
1503
0762e42f 1504The C API for the backtrace is as follows:
470dd224
JH
1505
1506=over 4
1507
1508=item get_c_backtrace
1509
1510=item free_c_backtrace
1511
1512=item get_c_backtrace_dump
1513
1514=item dump_c_backtrace
1515
1516=back
1517
04c692a8
DR
1518=head2 Poison
1519
1520If you see in a debugger a memory area mysteriously full of 0xABABABAB
1521or 0xEFEFEFEF, you may be seeing the effect of the Poison() macros, see
1522L<perlclib>.
1523
1524=head2 Read-only optrees
1525
9b22382a 1526Under ithreads the optree is read only. If you want to enforce this, to
04c692a8 1527check for write accesses from buggy code, compile with
91fc0422
FC
1528C<-Accflags=-DPERL_DEBUG_READONLY_OPS>
1529to enable code that allocates op memory
4dd56148
NC
1530via C<mmap>, and sets it read-only when it is attached to a subroutine.
1531Any write access to an op results in a C<SIGBUS> and abort.
04c692a8
DR
1532
1533This code is intended for development only, and may not be portable
9b22382a
FC
1534even to all Unix variants. Also, it is an 80% solution, in that it
1535isn't able to make all ops read only. Specifically it does not apply to
4dd56148 1536op slabs belonging to C<BEGIN> blocks.
04c692a8 1537
4dd56148
NC
1538However, as an 80% solution it is still effective, as it has caught
1539bugs in the past.
04c692a8 1540
f789f6a4
FC
1541=head2 When is a bool not a bool?
1542
1543On pre-C99 compilers, C<bool> is defined as equivalent to C<char>.
1544Consequently assignment of any larger type to a C<bool> is unsafe and may
1545be truncated. The C<cBOOL> macro exists to cast it correctly.
1546
1547On those platforms and compilers where C<bool> really is a boolean (C++,
1548C99), it is easy to forget the cast. You can force C<bool> to be a C<char>
1549by compiling with C<-Accflags=-DPERL_BOOL_AS_CHAR>. You may also wish to
50e4f4d4
CB
1550run C<Configure> with something like
1551
cbc13c3d 1552 -Accflags='-Wconversion -Wno-sign-conversion -Wno-shorten-64-to-32'
50e4f4d4
CB
1553
1554or your compiler's equivalent to make it easier to spot any unsafe truncations
1555that show up.
f789f6a4 1556
04c692a8
DR
1557=head2 The .i Targets
1558
1559You can expand the macros in a F<foo.c> file by saying
1560
1561 make foo.i
1562
d1fd4856
VP
1563which will expand the macros using cpp. Don't be scared by the
1564results.
04c692a8
DR
1565
1566=head1 AUTHOR
1567
1568This document was originally written by Nathan Torkington, and is
1569maintained by the perl5-porters mailing list.