This is a live mirror of the Perl 5 development currently hosted at https://github.com/perl/perl5
PL_sv_objcount deprecation notice
[perl5.git] / pod / perldebguts.pod
CommitLineData
055fd3a9
GS
1=head1 NAME
2
3perldebguts - Guts of Perl debugging
4
5=head1 DESCRIPTION
6
ba555bf5 7This is not L<perldebug>, which tells you how to use
74410c12
JM
8the debugger. This manpage describes low-level details concerning
9the debugger's internals, which range from difficult to impossible
10to understand for anyone who isn't incredibly intimate with Perl's guts.
11Caveat lector.
055fd3a9
GS
12
13=head1 Debugger Internals
14
15Perl has special debugging hooks at compile-time and run-time used
16to create debugging environments. These hooks are not to be confused
4375e838
GS
17with the I<perl -Dxxx> command described in L<perlrun>, which is
18usable only if a special Perl is built per the instructions in the
055fd3a9
GS
19F<INSTALL> podpage in the Perl source tree.
20
21For example, whenever you call Perl's built-in C<caller> function
74410c12
JM
22from the package C<DB>, the arguments that the corresponding stack
23frame was called with are copied to the C<@DB::args> array. These
24mechanisms are enabled by calling Perl with the B<-d> switch.
25Specifically, the following additional features are enabled
26(cf. L<perlvar/$^P>):
055fd3a9 27
13a2d996 28=over 4
055fd3a9
GS
29
30=item *
31
32Perl inserts the contents of C<$ENV{PERL5DB}> (or C<BEGIN {require
33'perl5db.pl'}> if not present) before the first line of your program.
34
35=item *
36
aa0b556f 37Each array C<@{"_<$filename"}> holds the lines of $filename for a
74410c12
JM
38file compiled by Perl. The same is also true for C<eval>ed strings
39that contain subroutines, or which are currently being executed.
40The $filename for C<eval>ed strings looks like C<(eval 34)>.
8894c26d
MJD
41
42Values in this array are magical in numeric context: they compare
43equal to zero only if the line is not breakable.
055fd3a9
GS
44
45=item *
46
aa0b556f 47Each hash C<%{"_<$filename"}> contains breakpoints and actions keyed
055fd3a9
GS
48by line number. Individual entries (as opposed to the whole hash)
49are settable. Perl only cares about Boolean true here, although
50the values used by F<perl5db.pl> have the form
8894c26d 51C<"$break_condition\0$action">.
055fd3a9
GS
52
53The same holds for evaluated strings that contain subroutines, or
54which are currently being executed. The $filename for C<eval>ed strings
d24ca0c5 55looks like C<(eval 34)>.
055fd3a9
GS
56
57=item *
58
aa0b556f 59Each scalar C<${"_<$filename"}> contains C<"_<$filename">. This is
055fd3a9
GS
60also the case for evaluated strings that contain subroutines, or
61which are currently being executed. The $filename for C<eval>ed
d24ca0c5 62strings looks like C<(eval 34)>.
055fd3a9
GS
63
64=item *
65
66After each C<require>d file is compiled, but before it is executed,
67C<DB::postponed(*{"_<$filename"})> is called if the subroutine
68C<DB::postponed> exists. Here, the $filename is the expanded name of
69the C<require>d file, as found in the values of %INC.
70
71=item *
72
73After each subroutine C<subname> is compiled, the existence of
74C<$DB::postponed{subname}> is checked. If this key exists,
75C<DB::postponed(subname)> is called if the C<DB::postponed> subroutine
76also exists.
77
78=item *
79
80A hash C<%DB::sub> is maintained, whose keys are subroutine names
81and whose values have the form C<filename:startline-endline>.
82C<filename> has the form C<(eval 34)> for subroutines defined inside
d24ca0c5 83C<eval>s.
055fd3a9
GS
84
85=item *
86
87When the execution of your program reaches a point that can hold a
74410c12
JM
88breakpoint, the C<DB::DB()> subroutine is called if any of the variables
89C<$DB::trace>, C<$DB::single>, or C<$DB::signal> is true. These variables
055fd3a9
GS
90are not C<local>izable. This feature is disabled when executing
91inside C<DB::DB()>, including functions called from it
92unless C<< $^D & (1<<30) >> is true.
93
94=item *
95
96When execution of the program reaches a subroutine call, a call to
97C<&DB::sub>(I<args>) is made instead, with C<$DB::sub> holding the
74410c12 98name of the called subroutine. (This doesn't happen if the subroutine
055fd3a9
GS
99was compiled in the C<DB> package.)
100
101=back
102
103Note that if C<&DB::sub> needs external data for it to work, no
74410c12
JM
104subroutine call is possible without it. As an example, the standard
105debugger's C<&DB::sub> depends on the C<$DB::deep> variable
106(it defines how many levels of recursion deep into the debugger you can go
107before a mandatory break). If C<$DB::deep> is not defined, subroutine
108calls are not possible, even though C<&DB::sub> exists.
055fd3a9
GS
109
110=head2 Writing Your Own Debugger
111
74410c12 112=head3 Environment Variables
666f95b9 113
74410c12
JM
114The C<PERL5DB> environment variable can be used to define a debugger.
115For example, the minimal "working" debugger (it actually doesn't do anything)
116consists of one line:
666f95b9 117
055fd3a9
GS
118 sub DB::DB {}
119
74410c12 120It can easily be defined like this:
666f95b9 121
055fd3a9
GS
122 $ PERL5DB="sub DB::DB {}" perl -d your-script
123
74410c12 124Another brief debugger, slightly more useful, can be created
055fd3a9
GS
125with only the line:
126
127 sub DB::DB {print ++$i; scalar <STDIN>}
128
74410c12
JM
129This debugger prints a number which increments for each statement
130encountered and waits for you to hit a newline before continuing
131to the next statement.
666f95b9 132
74410c12 133The following debugger is actually useful:
666f95b9 134
055fd3a9
GS
135 {
136 package DB;
137 sub DB {}
138 sub sub {print ++$i, " $sub\n"; &$sub}
139 }
140
74410c12
JM
141It prints the sequence number of each subroutine call and the name of the
142called subroutine. Note that C<&DB::sub> is being compiled into the
143package C<DB> through the use of the C<package> directive.
055fd3a9 144
74410c12
JM
145When it starts, the debugger reads your rc file (F<./.perldb> or
146F<~/.perldb> under Unix), which can set important options.
147(A subroutine (C<&afterinit>) can be defined here as well; it is executed
148after the debugger completes its own initialization.)
055fd3a9
GS
149
150After the rc file is read, the debugger reads the PERLDB_OPTS
74410c12
JM
151environment variable and uses it to set debugger options. The
152contents of this variable are treated as if they were the argument
96090e4f 153of an C<o ...> debugger command (q.v. in L<perldebug/"Configurable Options">).
74410c12 154
7b406369 155=head3 Debugger Internal Variables
25cf7dea 156
74410c12
JM
157In addition to the file and subroutine-related variables mentioned above,
158the debugger also maintains various magical internal variables.
159
160=over 4
161
162=item *
055fd3a9 163
74410c12
JM
164C<@DB::dbline> is an alias for C<@{"::_<current_file"}>, which
165holds the lines of the currently-selected file (compiled by Perl), either
166explicitly chosen with the debugger's C<f> command, or implicitly by flow
167of execution.
168
169Values in this array are magical in numeric context: they compare
170equal to zero only if the line is not breakable.
171
172=item *
173
7b406369 174C<%DB::dbline> is an alias for C<%{"::_<current_file"}>, which
74410c12
JM
175contains breakpoints and actions keyed by line number in
176the currently-selected file, either explicitly chosen with the
055fd3a9
GS
177debugger's C<f> command, or implicitly by flow of execution.
178
74410c12
JM
179As previously noted, individual entries (as opposed to the whole hash)
180are settable. Perl only cares about Boolean true here, although
181the values used by F<perl5db.pl> have the form
182C<"$break_condition\0$action">.
183
184=back
185
7b406369 186=head3 Debugger Customization Functions
74410c12
JM
187
188Some functions are provided to simplify customization.
189
190=over 4
191
192=item *
193
71110851
RGS
194See L<perldebug/"Configurable Options"> for a description of options parsed by
195C<DB::parse_options(string)>.
74410c12
JM
196
197=item *
198
199C<DB::dump_trace(skip[,count])> skips the specified number of frames
200and returns a list containing information about the calling frames (all
201of them, if C<count> is missing). Each entry is reference to a hash
202with keys C<context> (either C<.>, C<$>, or C<@>), C<sub> (subroutine
055fd3a9
GS
203name, or info about C<eval>), C<args> (C<undef> or a reference to
204an array), C<file>, and C<line>.
205
74410c12
JM
206=item *
207
208C<DB::print_trace(FH, skip[, count[, short]])> prints
055fd3a9
GS
209formatted info about caller frames. The last two functions may be
210convenient as arguments to C<< < >>, C<< << >> commands.
211
74410c12
JM
212=back
213
055fd3a9
GS
214Note that any variables and functions that are not documented in
215this manpages (or in L<perldebug>) are considered for internal
216use only, and as such are subject to change without notice.
217
218=head1 Frame Listing Output Examples
219
220The C<frame> option can be used to control the output of frame
221information. For example, contrast this expression trace:
222
223 $ perl -de 42
224 Stack dump during die enabled outside of evals.
225
226 Loading DB routines from perl5db.pl patch level 0.94
227 Emacs support available.
228
ccf3535a 229 Enter h or 'h h' for help.
055fd3a9
GS
230
231 main::(-e:1): 0
232 DB<1> sub foo { 14 }
233
234 DB<2> sub bar { 3 }
235
236 DB<3> t print foo() * bar()
237 main::((eval 172):3): print foo() + bar();
238 main::foo((eval 168):2):
239 main::bar((eval 170):2):
240 42
241
492652be 242with this one, once the C<o>ption C<frame=2> has been set:
055fd3a9 243
492652be 244 DB<4> o f=2
055fd3a9
GS
245 frame = '2'
246 DB<5> t print foo() * bar()
247 3: foo() * bar()
248 entering main::foo
249 2: sub foo { 14 };
250 exited main::foo
251 entering main::bar
252 2: sub bar { 3 };
253 exited main::bar
254 42
255
256By way of demonstration, we present below a laborious listing
257resulting from setting your C<PERLDB_OPTS> environment variable to
258the value C<f=n N>, and running I<perl -d -V> from the command line.
7b406369
FC
259Examples using various values of C<n> are shown to give you a feel
260for the difference between settings. Long though it may be, this
055fd3a9
GS
261is not a complete listing, but only excerpts.
262
263=over 4
264
265=item 1
266
267 entering main::BEGIN
268 entering Config::BEGIN
269 Package lib/Exporter.pm.
270 Package lib/Carp.pm.
271 Package lib/Config.pm.
272 entering Config::TIEHASH
273 entering Exporter::import
274 entering Exporter::export
275 entering Config::myconfig
276 entering Config::FETCH
277 entering Config::FETCH
278 entering Config::FETCH
279 entering Config::FETCH
280
281=item 2
282
283 entering main::BEGIN
284 entering Config::BEGIN
285 Package lib/Exporter.pm.
286 Package lib/Carp.pm.
287 exited Config::BEGIN
288 Package lib/Config.pm.
289 entering Config::TIEHASH
290 exited Config::TIEHASH
291 entering Exporter::import
292 entering Exporter::export
293 exited Exporter::export
294 exited Exporter::import
295 exited main::BEGIN
296 entering Config::myconfig
297 entering Config::FETCH
298 exited Config::FETCH
299 entering Config::FETCH
300 exited Config::FETCH
301 entering Config::FETCH
302
d5e42f17 303=item 3
055fd3a9
GS
304
305 in $=main::BEGIN() from /dev/null:0
306 in $=Config::BEGIN() from lib/Config.pm:2
307 Package lib/Exporter.pm.
308 Package lib/Carp.pm.
309 Package lib/Config.pm.
310 in $=Config::TIEHASH('Config') from lib/Config.pm:644
311 in $=Exporter::import('Config', 'myconfig', 'config_vars') from /dev/null:0
312 in $=Exporter::export('Config', 'main', 'myconfig', 'config_vars') from li
313 in @=Config::myconfig() from /dev/null:0
314 in $=Config::FETCH(ref(Config), 'package') from lib/Config.pm:574
315 in $=Config::FETCH(ref(Config), 'baserev') from lib/Config.pm:574
316 in $=Config::FETCH(ref(Config), 'PERL_VERSION') from lib/Config.pm:574
317 in $=Config::FETCH(ref(Config), 'PERL_SUBVERSION') from lib/Config.pm:574
318 in $=Config::FETCH(ref(Config), 'osname') from lib/Config.pm:574
319 in $=Config::FETCH(ref(Config), 'osvers') from lib/Config.pm:574
320
d5e42f17 321=item 4
055fd3a9
GS
322
323 in $=main::BEGIN() from /dev/null:0
324 in $=Config::BEGIN() from lib/Config.pm:2
325 Package lib/Exporter.pm.
326 Package lib/Carp.pm.
327 out $=Config::BEGIN() from lib/Config.pm:0
328 Package lib/Config.pm.
329 in $=Config::TIEHASH('Config') from lib/Config.pm:644
330 out $=Config::TIEHASH('Config') from lib/Config.pm:644
331 in $=Exporter::import('Config', 'myconfig', 'config_vars') from /dev/null:0
332 in $=Exporter::export('Config', 'main', 'myconfig', 'config_vars') from lib/
333 out $=Exporter::export('Config', 'main', 'myconfig', 'config_vars') from lib/
334 out $=Exporter::import('Config', 'myconfig', 'config_vars') from /dev/null:0
335 out $=main::BEGIN() from /dev/null:0
336 in @=Config::myconfig() from /dev/null:0
337 in $=Config::FETCH(ref(Config), 'package') from lib/Config.pm:574
338 out $=Config::FETCH(ref(Config), 'package') from lib/Config.pm:574
339 in $=Config::FETCH(ref(Config), 'baserev') from lib/Config.pm:574
340 out $=Config::FETCH(ref(Config), 'baserev') from lib/Config.pm:574
341 in $=Config::FETCH(ref(Config), 'PERL_VERSION') from lib/Config.pm:574
342 out $=Config::FETCH(ref(Config), 'PERL_VERSION') from lib/Config.pm:574
343 in $=Config::FETCH(ref(Config), 'PERL_SUBVERSION') from lib/Config.pm:574
344
d5e42f17 345=item 5
055fd3a9
GS
346
347 in $=main::BEGIN() from /dev/null:0
348 in $=Config::BEGIN() from lib/Config.pm:2
349 Package lib/Exporter.pm.
350 Package lib/Carp.pm.
351 out $=Config::BEGIN() from lib/Config.pm:0
352 Package lib/Config.pm.
353 in $=Config::TIEHASH('Config') from lib/Config.pm:644
354 out $=Config::TIEHASH('Config') from lib/Config.pm:644
355 in $=Exporter::import('Config', 'myconfig', 'config_vars') from /dev/null:0
356 in $=Exporter::export('Config', 'main', 'myconfig', 'config_vars') from lib/E
357 out $=Exporter::export('Config', 'main', 'myconfig', 'config_vars') from lib/E
358 out $=Exporter::import('Config', 'myconfig', 'config_vars') from /dev/null:0
359 out $=main::BEGIN() from /dev/null:0
360 in @=Config::myconfig() from /dev/null:0
361 in $=Config::FETCH('Config=HASH(0x1aa444)', 'package') from lib/Config.pm:574
362 out $=Config::FETCH('Config=HASH(0x1aa444)', 'package') from lib/Config.pm:574
363 in $=Config::FETCH('Config=HASH(0x1aa444)', 'baserev') from lib/Config.pm:574
364 out $=Config::FETCH('Config=HASH(0x1aa444)', 'baserev') from lib/Config.pm:574
365
d5e42f17 366=item 6
055fd3a9
GS
367
368 in $=CODE(0x15eca4)() from /dev/null:0
369 in $=CODE(0x182528)() from lib/Config.pm:2
370 Package lib/Exporter.pm.
371 out $=CODE(0x182528)() from lib/Config.pm:0
372 scalar context return from CODE(0x182528): undef
373 Package lib/Config.pm.
374 in $=Config::TIEHASH('Config') from lib/Config.pm:628
375 out $=Config::TIEHASH('Config') from lib/Config.pm:628
376 scalar context return from Config::TIEHASH: empty hash
377 in $=Exporter::import('Config', 'myconfig', 'config_vars') from /dev/null:0
378 in $=Exporter::export('Config', 'main', 'myconfig', 'config_vars') from lib/Exporter.pm:171
379 out $=Exporter::export('Config', 'main', 'myconfig', 'config_vars') from lib/Exporter.pm:171
380 scalar context return from Exporter::export: ''
381 out $=Exporter::import('Config', 'myconfig', 'config_vars') from /dev/null:0
382 scalar context return from Exporter::import: ''
383
384=back
385
386In all cases shown above, the line indentation shows the call tree.
387If bit 2 of C<frame> is set, a line is printed on exit from a
388subroutine as well. If bit 4 is set, the arguments are printed
389along with the caller info. If bit 8 is set, the arguments are
390printed even if they are tied or references. If bit 16 is set, the
391return value is printed, too.
392
393When a package is compiled, a line like this
394
395 Package lib/Carp.pm.
396
397is printed with proper indentation.
398
7b406369 399=head1 Debugging Regular Expressions
055fd3a9
GS
400
401There are two ways to enable debugging output for regular expressions.
402
403If your perl is compiled with C<-DDEBUGGING>, you may use the
404B<-Dr> flag on the command line.
405
406Otherwise, one can C<use re 'debug'>, which has effects at
3d71525d
NJ
407compile time and run time. Since Perl 5.9.5, this pragma is lexically
408scoped.
055fd3a9 409
7b406369 410=head2 Compile-time Output
055fd3a9
GS
411
412The debugging output at compile time looks like this:
413
ccf3535a 414 Compiling REx '[bc]d(ef*g)+h[ij]k$'
1c102323
MJD
415 size 45 Got 364 bytes for offset annotations.
416 first at 1
417 rarest char g at 0
418 rarest char d at 0
419 1: ANYOF[bc](12)
420 12: EXACT <d>(14)
421 14: CURLYX[0] {1,32767}(28)
422 16: OPEN1(18)
423 18: EXACT <e>(20)
424 20: STAR(23)
425 21: EXACT <f>(0)
426 23: EXACT <g>(25)
427 25: CLOSE1(27)
428 27: WHILEM[1/1](0)
429 28: NOTHING(29)
430 29: EXACT <h>(31)
431 31: ANYOF[ij](42)
432 42: EXACT <k>(44)
433 44: EOL(45)
434 45: END(0)
ccf3535a
JK
435 anchored 'de' at 1 floating 'gh' at 3..2147483647 (checking floating)
436 stclass 'ANYOF[bc]' minlen 7
1c102323
MJD
437 Offsets: [45]
438 1[4] 0[0] 0[0] 0[0] 0[0] 0[0] 0[0] 0[0] 0[0] 0[0] 0[0] 5[1]
439 0[0] 12[1] 0[0] 6[1] 0[0] 7[1] 0[0] 9[1] 8[1] 0[0] 10[1] 0[0]
440 11[1] 0[0] 12[0] 12[0] 13[1] 0[0] 14[4] 0[0] 0[0] 0[0] 0[0]
441 0[0] 0[0] 0[0] 0[0] 0[0] 0[0] 18[1] 0[0] 19[1] 20[0]
442 Omitting $` $& $' support.
055fd3a9
GS
443
444The first line shows the pre-compiled form of the regex. The second
445shows the size of the compiled form (in arbitrary units, usually
1c102323
MJD
4464-byte words) and the total number of bytes allocated for the
447offset/length table, usually 4+C<size>*8. The next line shows the
448label I<id> of the first node that does a match.
055fd3a9 449
1c102323
MJD
450The
451
ccf3535a
JK
452 anchored 'de' at 1 floating 'gh' at 3..2147483647 (checking floating)
453 stclass 'ANYOF[bc]' minlen 7
1c102323
MJD
454
455line (split into two lines above) contains optimizer
055fd3a9
GS
456information. In the example shown, the optimizer found that the match
457should contain a substring C<de> at offset 1, plus substring C<gh>
458at some offset between 3 and infinity. Moreover, when checking for
459these substrings (to abandon impossible matches quickly), Perl will check
460for the substring C<gh> before checking for the substring C<de>. The
461optimizer may also use the knowledge that the match starts (at the
1c102323
MJD
462C<first> I<id>) with a character class, and no string
463shorter than 7 characters can possibly match.
055fd3a9 464
1c102323 465The fields of interest which may appear in this line are
055fd3a9 466
13a2d996 467=over 4
055fd3a9
GS
468
469=item C<anchored> I<STRING> C<at> I<POS>
470
471=item C<floating> I<STRING> C<at> I<POS1..POS2>
472
473See above.
474
475=item C<matching floating/anchored>
476
477Which substring to check first.
478
479=item C<minlen>
480
481The minimal length of the match.
482
483=item C<stclass> I<TYPE>
484
485Type of first matching node.
486
487=item C<noscan>
488
489Don't scan for the found substrings.
490
491=item C<isall>
492
1c102323 493Means that the optimizer information is all that the regular
055fd3a9
GS
494expression contains, and thus one does not need to enter the regex engine at
495all.
496
497=item C<GPOS>
498
499Set if the pattern contains C<\G>.
500
501=item C<plus>
502
503Set if the pattern starts with a repeated char (as in C<x+y>).
504
505=item C<implicit>
506
507Set if the pattern starts with C<.*>.
508
509=item C<with eval>
510
511Set if the pattern contain eval-groups, such as C<(?{ code })> and
512C<(??{ code })>.
513
514=item C<anchored(TYPE)>
515
7b406369 516If the pattern may match only at a handful of places, with C<TYPE>
055fd3a9
GS
517being C<BOL>, C<MBOL>, or C<GPOS>. See the table below.
518
519=back
520
521If a substring is known to match at end-of-line only, it may be
ccf3535a 522followed by C<$>, as in C<floating 'k'$>.
055fd3a9 523
1c102323
MJD
524The optimizer-specific information is used to avoid entering (a slow) regex
525engine on strings that will not definitely match. If the C<isall> flag
055fd3a9
GS
526is set, a call to the regex engine may be avoided even when the optimizer
527found an appropriate place for the match.
528
1c102323 529Above the optimizer section is the list of I<nodes> of the compiled
055fd3a9
GS
530form of the regex. Each line has format
531
532C< >I<id>: I<TYPE> I<OPTIONAL-INFO> (I<next-id>)
533
7b406369 534=head2 Types of Nodes
055fd3a9
GS
535
536Here are the possible types, with short descriptions:
537
65aa4ca7
FC
538=for comment
539This table is generated by regen/regcomp.pl. Any changes made here
540will be lost.
541
542=for regcomp.pl begin
543
5da6b59a
KW
544 # TYPE arg-description [num-args] [longjump-len] DESCRIPTION
545
546 # Exit points
65aa4ca7
FC
547
548 END no End of program.
549 SUCCEED no Return from a subroutine, basically.
5da6b59a
KW
550
551 # Anchors:
552
65aa4ca7
FC
553 BOL no Match "" at beginning of line.
554 MBOL no Same, assuming multiline.
555 SBOL no Same, assuming singleline.
556 EOS no Match "" at end of string.
557 EOL no Match "" at end of line.
558 MEOL no Same, assuming multiline.
559 SEOL no Same, assuming singleline.
560 BOUND no Match "" at any word boundary using
561 native charset semantics for non-utf8
562 BOUNDL no Match "" at any locale word boundary
563 BOUNDU no Match "" at any word boundary using
564 Unicode semantics
565 BOUNDA no Match "" at any word boundary using ASCII
566 semantics
567 NBOUND no Match "" at any word non-boundary using
568 native charset semantics for non-utf8
569 NBOUNDL no Match "" at any locale word non-boundary
570 NBOUNDU no Match "" at any word non-boundary using
571 Unicode semantics
572 NBOUNDA no Match "" at any word non-boundary using
573 ASCII semantics
574 GPOS no Matches where last m//g left off.
5da6b59a
KW
575
576 # [Special] alternatives:
577
65aa4ca7
FC
578 REG_ANY no Match any one character (except newline).
579 SANY no Match any one character.
580 CANY no Match any one byte.
581 ANYOF sv Match character in (or not in) this
582 class, single char match only
954a2af6
KW
583 ANYOF_WARN_SUPER sv Match character in (or not in) this
584 class, warn (if enabled) upon matching a
585 char above Unicode max;
9aa1e39f 586 ANYOF_SYNTHETIC sv Synthetic start class
65aa4ca7
FC
587
588 POSIXD none Some [[:class:]] under /d; the FLAGS
589 field gives which one
590 POSIXL none Some [[:class:]] under /l; the FLAGS
591 field gives which one
592 POSIXU none Some [[:class:]] under /u; the FLAGS
593 field gives which one
594 POSIXA none Some [[:class:]] under /a; the FLAGS
595 field gives which one
596 NPOSIXD none complement of POSIXD, [[:^class:]]
597 NPOSIXL none complement of POSIXL, [[:^class:]]
598 NPOSIXU none complement of POSIXU, [[:^class:]]
599 NPOSIXA none complement of POSIXA, [[:^class:]]
600
601 CLUMP no Match any extended grapheme cluster
602 sequence
5da6b59a
KW
603
604 # Alternation
605
65aa4ca7
FC
606 # BRANCH The set of branches constituting a single choice are
607 # hooked together with their "next" pointers, since
608 # precedence prevents anything being concatenated to
609 # any individual branch. The "next" pointer of the last
610 # BRANCH in a choice points to the thing following the
611 # whole choice. This is also where the final "next"
612 # pointer of each individual branch points; each branch
613 # starts with the operand node of a BRANCH node.
5da6b59a 614 #
65aa4ca7 615 BRANCH node Match this alternative, or the next...
5da6b59a
KW
616
617 # Back pointer
618
65aa4ca7
FC
619 # BACK Normal "next" pointers all implicitly point forward;
620 # BACK exists to make loop structures possible.
5da6b59a 621 # not used
65aa4ca7 622 BACK no Match "", "next" ptr points backward.
5da6b59a
KW
623
624 # Literals
625
65aa4ca7
FC
626 EXACT str Match this string (preceded by length).
627 EXACTF str Match this non-UTF-8 string (not
628 guaranteed to be folded) using /id rules
629 (w/len).
630 EXACTFL str Match this string (not guaranteed to be
631 folded) using /il rules (w/len).
632 EXACTFU str Match this string (folded iff in UTF-8,
633 length in folding doesn't change if not
634 in UTF-8) using /iu rules (w/len).
635 EXACTFA str Match this string (not guaranteed to be
636 folded) using /iaa rules (w/len).
637 EXACTFU_SS str Match this string (folded iff in UTF-8,
638 length in folding may change even if not
639 in UTF-8) using /iu rules (w/len).
640 EXACTFU_TRICKYFOLD str Match this folded UTF-8 string using /iu
641 rules
5da6b59a
KW
642
643 # Do nothing types
644
65aa4ca7 645 NOTHING no Match empty string.
5da6b59a 646 # A variant of above which delimits a group, thus stops optimizations
65aa4ca7
FC
647 TAIL no Match empty string. Can jump here from
648 outside.
5da6b59a
KW
649
650 # Loops
651
65aa4ca7
FC
652 # STAR,PLUS '?', and complex '*' and '+', are implemented as
653 # circular BRANCH structures using BACK. Simple cases
654 # (one character per match) are implemented with STAR
655 # and PLUS for speed and to minimize recursive plunges.
5da6b59a 656 #
65aa4ca7
FC
657 STAR node Match this (simple) thing 0 or more
658 times.
659 PLUS node Match this (simple) thing 1 or more
660 times.
5da6b59a 661
65aa4ca7
FC
662 CURLY sv 2 Match this simple thing {n,m} times.
663 CURLYN no 2 Capture next-after-this simple thing
664 CURLYM no 2 Capture this medium-complex thing {n,m}
665 times.
666 CURLYX sv 2 Match this complex thing {n,m} times.
5da6b59a
KW
667
668 # This terminator creates a loop structure for CURLYX
65aa4ca7
FC
669 WHILEM no Do curly processing and see if rest
670 matches.
5da6b59a
KW
671
672 # Buffer related
673
674 # OPEN,CLOSE,GROUPP ...are numbered at compile time.
65aa4ca7
FC
675 OPEN num 1 Mark this point in input as start of #n.
676 CLOSE num 1 Analogous to OPEN.
677
678 REF num 1 Match some already matched string
679 REFF num 1 Match already matched string, folded
680 using native charset semantics for non-
681 utf8
682 REFFL num 1 Match already matched string, folded in
683 loc.
684 REFFU num 1 Match already matched string, folded
685 using unicode semantics for non-utf8
686 REFFA num 1 Match already matched string, folded
687 using unicode semantics for non-utf8, no
688 mixing ASCII, non-ASCII
689
690 # Named references. Code in regcomp.c assumes that these all are after
691 # the numbered references
692 NREF no-sv 1 Match some already matched string
693 NREFF no-sv 1 Match already matched string, folded
694 using native charset semantics for non-
695 utf8
696 NREFFL no-sv 1 Match already matched string, folded in
697 loc.
698 NREFFU num 1 Match already matched string, folded
699 using unicode semantics for non-utf8
700 NREFFA num 1 Match already matched string, folded
701 using unicode semantics for non-utf8, no
702 mixing ASCII, non-ASCII
703
704 IFMATCH off 1 2 Succeeds if the following matches.
705 UNLESSM off 1 2 Fails if the following matches.
706 SUSPEND off 1 1 "Independent" sub-RE.
707 IFTHEN off 1 1 Switch, should be preceded by switcher.
708 GROUPP num 1 Whether the group matched.
5da6b59a
KW
709
710 # Support for long RE
711
65aa4ca7
FC
712 LONGJMP off 1 1 Jump far away.
713 BRANCHJ off 1 1 BRANCH with long offset.
5da6b59a
KW
714
715 # The heavy worker
716
65aa4ca7 717 EVAL evl 1 Execute some Perl code.
5da6b59a
KW
718
719 # Modifiers
720
65aa4ca7
FC
721 MINMOD no Next operator is not greedy.
722 LOGICAL no Next opcode should set the flag only.
5da6b59a
KW
723
724 # This is not used yet
65aa4ca7 725 RENUM off 1 1 Group with independently numbered parens.
5da6b59a
KW
726
727 # Trie Related
728
65aa4ca7
FC
729 # Behave the same as A|LIST|OF|WORDS would. The '..C' variants
730 # have inline charclass data (ascii only), the 'C' store it in the
731 # structure.
5da6b59a 732
65aa4ca7
FC
733 TRIE trie 1 Match many EXACT(F[ALU]?)? at once.
734 flags==type
735 TRIEC trie Same as TRIE, but with embedded charclass
736 charclass data
5da6b59a 737
65aa4ca7
FC
738 AHOCORASICK trie 1 Aho Corasick stclass. flags==type
739 AHOCORASICKC trie Same as AHOCORASICK, but with embedded
740 charclass charclass data
5da6b59a
KW
741
742 # Regex Subroutines
65aa4ca7
FC
743 GOSUB num/ofs 2L recurse to paren arg1 at (signed) ofs
744 arg2
745 GOSTART no recurse to start of pattern
5da6b59a
KW
746
747 # Special conditionals
65aa4ca7
FC
748 NGROUPP no-sv 1 Whether the group matched.
749 INSUBP num 1 Whether we are in a specific recurse.
750 DEFINEP none 1 Never execute directly.
5da6b59a
KW
751
752 # Backtracking Verbs
65aa4ca7
FC
753 ENDLIKE none Used only for the type field of verbs
754 OPFAIL none Same as (?!)
755 ACCEPT parno 1 Accepts the current matched string.
5da6b59a
KW
756
757 # Verbs With Arguments
65aa4ca7
FC
758 VERB no-sv 1 Used only for the type field of verbs
759 PRUNE no-sv 1 Pattern fails at this startpoint if no-
760 backtracking through this
761 MARKPOINT no-sv 1 Push the current location for rollback by
762 cut.
763 SKIP no-sv 1 On failure skip forward (to the mark)
764 before retrying
765 COMMIT no-sv 1 Pattern fails outright if backtracking
766 through this
767 CUTGROUP no-sv 1 On failure go to the next alternation in
768 the group
5da6b59a
KW
769
770 # Control what to keep in $&.
65aa4ca7 771 KEEPS no $& begins here.
5da6b59a
KW
772
773 # New charclass like patterns
65aa4ca7 774 LNBREAK none generic newline pattern
5da6b59a
KW
775
776 # SPECIAL REGOPS
777
65aa4ca7
FC
778 # This is not really a node, but an optimized away piece of a "long"
779 # node. To simplify debugging output, we mark it as if it were a node
780 OPTIMIZED off Placeholder for dump.
5da6b59a
KW
781
782 # Special opcode with the property that no opcode in a compiled program
783 # will ever be of this type. Thus it can be used as a flag value that
784 # no other opcode has been seen. END is used similarly, in that an END
65aa4ca7
FC
785 # node cant be optimized. So END implies "unoptimizable" and PSEUDO
786 # mean "not seen anything to optimize yet".
787 PSEUDO off Pseudo opcode for internal use.
788
789=for regcomp.pl end
055fd3a9 790
1c102323
MJD
791=for unprinted-credits
792Next section M-J. Dominus (mjd-perl-patch+@plover.com) 20010421
793
794Following the optimizer information is a dump of the offset/length
795table, here split across several lines:
796
797 Offsets: [45]
798 1[4] 0[0] 0[0] 0[0] 0[0] 0[0] 0[0] 0[0] 0[0] 0[0] 0[0] 5[1]
799 0[0] 12[1] 0[0] 6[1] 0[0] 7[1] 0[0] 9[1] 8[1] 0[0] 10[1] 0[0]
800 11[1] 0[0] 12[0] 12[0] 13[1] 0[0] 14[4] 0[0] 0[0] 0[0] 0[0]
801 0[0] 0[0] 0[0] 0[0] 0[0] 0[0] 18[1] 0[0] 19[1] 20[0]
802
803The first line here indicates that the offset/length table contains 45
804entries. Each entry is a pair of integers, denoted by C<offset[length]>.
17c338f3 805Entries are numbered starting with 1, so entry #1 here is C<1[4]> and
1c102323
MJD
806entry #12 is C<5[1]>. C<1[4]> indicates that the node labeled C<1:>
807(the C<1: ANYOF[bc]>) begins at character position 1 in the
808pre-compiled form of the regex, and has a length of 4 characters.
809C<5[1]> in position 12
810indicates that the node labeled C<12:>
811(the C<< 12: EXACT <d> >>) begins at character position 5 in the
812pre-compiled form of the regex, and has a length of 1 character.
813C<12[1]> in position 14
814indicates that the node labeled C<14:>
815(the C<< 14: CURLYX[0] {1,32767} >>) begins at character position 12 in the
816pre-compiled form of the regex, and has a length of 1 character---that
817is, it corresponds to the C<+> symbol in the precompiled regex.
818
819C<0[0]> items indicate that there is no corresponding node.
820
7b406369 821=head2 Run-time Output
055fd3a9
GS
822
823First of all, when doing a match, one may get no run-time output even
824if debugging is enabled. This means that the regex engine was never
825entered and that all of the job was therefore done by the optimizer.
826
827If the regex engine was entered, the output may look like this:
828
ccf3535a 829 Matching '[bc]d(ef*g)+h[ij]k$' against 'abcdefg__gh__'
055fd3a9
GS
830 Setting an EVAL scope, savestack=3
831 2 <ab> <cdefg__gh_> | 1: ANYOF
832 3 <abc> <defg__gh_> | 11: EXACT <d>
833 4 <abcd> <efg__gh_> | 13: CURLYX {1,32767}
834 4 <abcd> <efg__gh_> | 26: WHILEM
835 0 out of 1..32767 cc=effff31c
836 4 <abcd> <efg__gh_> | 15: OPEN1
837 4 <abcd> <efg__gh_> | 17: EXACT <e>
838 5 <abcde> <fg__gh_> | 19: STAR
839 EXACT <f> can match 1 times out of 32767...
840 Setting an EVAL scope, savestack=3
841 6 <bcdef> <g__gh__> | 22: EXACT <g>
842 7 <bcdefg> <__gh__> | 24: CLOSE1
843 7 <bcdefg> <__gh__> | 26: WHILEM
844 1 out of 1..32767 cc=effff31c
845 Setting an EVAL scope, savestack=12
846 7 <bcdefg> <__gh__> | 15: OPEN1
847 7 <bcdefg> <__gh__> | 17: EXACT <e>
848 restoring \1 to 4(4)..7
849 failed, try continuation...
850 7 <bcdefg> <__gh__> | 27: NOTHING
851 7 <bcdefg> <__gh__> | 28: EXACT <h>
852 failed...
853 failed...
854
855The most significant information in the output is about the particular I<node>
856of the compiled regex that is currently being tested against the target string.
857The format of these lines is
858
859C< >I<STRING-OFFSET> <I<PRE-STRING>> <I<POST-STRING>> |I<ID>: I<TYPE>
860
861The I<TYPE> info is indented with respect to the backtracking level.
862Other incidental information appears interspersed within.
863
7b406369 864=head1 Debugging Perl Memory Usage
055fd3a9
GS
865
866Perl is a profligate wastrel when it comes to memory use. There
867is a saying that to estimate memory usage of Perl, assume a reasonable
868algorithm for memory allocation, multiply that estimate by 10, and
869while you still may miss the mark, at least you won't be quite so
4375e838 870astonished. This is not absolutely true, but may provide a good
055fd3a9
GS
871grasp of what happens.
872
873Assume that an integer cannot take less than 20 bytes of memory, a
874float cannot take less than 24 bytes, a string cannot take less
875than 32 bytes (all these examples assume 32-bit architectures, the
876result are quite a bit worse on 64-bit architectures). If a variable
877is accessed in two of three different ways (which require an integer,
878a float, or a string), the memory footprint may increase yet another
b9449ee0 87920 bytes. A sloppy malloc(3) implementation can inflate these
055fd3a9
GS
880numbers dramatically.
881
882On the opposite end of the scale, a declaration like
883
884 sub foo;
885
886may take up to 500 bytes of memory, depending on which release of Perl
887you're running.
888
889Anecdotal estimates of source-to-compiled code bloat suggest an
890eightfold increase. This means that the compiled form of reasonable
891(normally commented, properly indented etc.) code will take
892about eight times more space in memory than the code took
893on disk.
894
b30f304a
JH
895The B<-DL> command-line switch is obsolete since circa Perl 5.6.0
896(it was available only if Perl was built with C<-DDEBUGGING>).
897The switch was used to track Perl's memory allocations and possible
898memory leaks. These days the use of malloc debugging tools like
5b6a3331 899F<Purify> or F<valgrind> is suggested instead. See also
7b406369 900L<perlhacktips/PERL_MEM_LOG>.
b30f304a
JH
901
902One way to find out how much memory is being used by Perl data
903structures is to install the Devel::Size module from CPAN: it gives
904you the minimum number of bytes required to store a particular data
905structure. Please be mindful of the difference between the size()
906and total_size().
907
908If Perl has been compiled using Perl's malloc you can analyze Perl
7b406369 909memory usage by setting $ENV{PERL_DEBUG_MSTATS}.
055fd3a9
GS
910
911=head2 Using C<$ENV{PERL_DEBUG_MSTATS}>
912
913If your perl is using Perl's malloc() and was compiled with the
914necessary switches (this is the default), then it will print memory
4375e838 915usage statistics after compiling your code when C<< $ENV{PERL_DEBUG_MSTATS}
055fd3a9
GS
916> 1 >>, and before termination of the program when C<<
917$ENV{PERL_DEBUG_MSTATS} >= 1 >>. The report format is similar to
918the following example:
919
920 $ PERL_DEBUG_MSTATS=2 perl -e "require Carp"
921 Memory allocation statistics after compilation: (buckets 4(4)..8188(8192)
922 14216 free: 130 117 28 7 9 0 2 2 1 0 0
923 437 61 36 0 5
924 60924 used: 125 137 161 55 7 8 6 16 2 0 1
925 74 109 304 84 20
926 Total sbrk(): 77824/21:119. Odd ends: pad+heads+chain+tail: 0+636+0+2048.
927 Memory allocation statistics after execution: (buckets 4(4)..8188(8192)
928 30888 free: 245 78 85 13 6 2 1 3 2 0 1
929 315 162 39 42 11
930 175816 used: 265 176 1112 111 26 22 11 27 2 1 1
931 196 178 1066 798 39
932 Total sbrk(): 215040/47:145. Odd ends: pad+heads+chain+tail: 0+2192+0+6144.
933
934It is possible to ask for such a statistic at arbitrary points in
b9449ee0 935your execution using the mstat() function out of the standard
055fd3a9
GS
936Devel::Peek module.
937
938Here is some explanation of that format:
939
13a2d996 940=over 4
055fd3a9
GS
941
942=item C<buckets SMALLEST(APPROX)..GREATEST(APPROX)>
943
944Perl's malloc() uses bucketed allocations. Every request is rounded
945up to the closest bucket size available, and a bucket is taken from
946the pool of buckets of that size.
947
948The line above describes the limits of buckets currently in use.
949Each bucket has two sizes: memory footprint and the maximal size
950of user data that can fit into this bucket. Suppose in the above
951example that the smallest bucket were size 4. The biggest bucket
952would have usable size 8188, and the memory footprint would be 8192.
953
954In a Perl built for debugging, some buckets may have negative usable
955size. This means that these buckets cannot (and will not) be used.
956For larger buckets, the memory footprint may be one page greater
7b406369 957than a power of 2. If so, the corresponding power of two is
055fd3a9
GS
958printed in the C<APPROX> field above.
959
960=item Free/Used
961
962The 1 or 2 rows of numbers following that correspond to the number
963of buckets of each size between C<SMALLEST> and C<GREATEST>. In
964the first row, the sizes (memory footprints) of buckets are powers
965of two--or possibly one page greater. In the second row, if present,
966the memory footprints of the buckets are between the memory footprints
967of two buckets "above".
968
4375e838 969For example, suppose under the previous example, the memory footprints
055fd3a9
GS
970were
971
972 free: 8 16 32 64 128 256 512 1024 2048 4096 8192
973 4 12 24 48 80
974
7b406369 975With a non-C<DEBUGGING> perl, the buckets starting from C<128> have
d1be9408 976a 4-byte overhead, and thus an 8192-long bucket may take up to
055fd3a9
GS
9778188-byte allocations.
978
979=item C<Total sbrk(): SBRKed/SBRKs:CONTINUOUS>
980
981The first two fields give the total amount of memory perl sbrk(2)ed
982(ess-broken? :-) and number of sbrk(2)s used. The third number is
983what perl thinks about continuity of returned chunks. So long as
984this number is positive, malloc() will assume that it is probable
985that sbrk(2) will provide continuous memory.
986
987Memory allocated by external libraries is not counted.
988
989=item C<pad: 0>
990
991The amount of sbrk(2)ed memory needed to keep buckets aligned.
992
993=item C<heads: 2192>
994
995Although memory overhead of bigger buckets is kept inside the bucket, for
996smaller buckets, it is kept in separate areas. This field gives the
997total size of these areas.
998
999=item C<chain: 0>
1000
1001malloc() may want to subdivide a bigger bucket into smaller buckets.
1002If only a part of the deceased bucket is left unsubdivided, the rest
1003is kept as an element of a linked list. This field gives the total
1004size of these chunks.
1005
1006=item C<tail: 6144>
1007
1008To minimize the number of sbrk(2)s, malloc() asks for more memory. This
1009field gives the size of the yet unused part, which is sbrk(2)ed, but
1010never touched.
1011
1012=back
1013
055fd3a9
GS
1014=head1 SEE ALSO
1015
1016L<perldebug>,
1017L<perlguts>,
1018L<perlrun>
1019L<re>,
1020and
fe854a6f 1021L<Devel::DProf>.