Commit | Line | Data |
---|---|---|
055fd3a9 GS |
1 | =head1 NAME |
2 | ||
3 | perldebguts - Guts of Perl debugging | |
4 | ||
5 | =head1 DESCRIPTION | |
6 | ||
7 | This is not the perldebug(1) manpage, which tells you how to use | |
74410c12 JM |
8 | the debugger. This manpage describes low-level details concerning |
9 | the debugger's internals, which range from difficult to impossible | |
10 | to understand for anyone who isn't incredibly intimate with Perl's guts. | |
11 | Caveat lector. | |
055fd3a9 GS |
12 | |
13 | =head1 Debugger Internals | |
14 | ||
15 | Perl has special debugging hooks at compile-time and run-time used | |
16 | to create debugging environments. These hooks are not to be confused | |
4375e838 GS |
17 | with the I<perl -Dxxx> command described in L<perlrun>, which is |
18 | usable only if a special Perl is built per the instructions in the | |
055fd3a9 GS |
19 | F<INSTALL> podpage in the Perl source tree. |
20 | ||
21 | For example, whenever you call Perl's built-in C<caller> function | |
74410c12 JM |
22 | from the package C<DB>, the arguments that the corresponding stack |
23 | frame was called with are copied to the C<@DB::args> array. These | |
24 | mechanisms are enabled by calling Perl with the B<-d> switch. | |
25 | Specifically, the following additional features are enabled | |
26 | (cf. L<perlvar/$^P>): | |
055fd3a9 | 27 | |
13a2d996 | 28 | =over 4 |
055fd3a9 GS |
29 | |
30 | =item * | |
31 | ||
32 | Perl inserts the contents of C<$ENV{PERL5DB}> (or C<BEGIN {require | |
33 | 'perl5db.pl'}> if not present) before the first line of your program. | |
34 | ||
35 | =item * | |
36 | ||
aa0b556f | 37 | Each array C<@{"_<$filename"}> holds the lines of $filename for a |
74410c12 JM |
38 | file compiled by Perl. The same is also true for C<eval>ed strings |
39 | that contain subroutines, or which are currently being executed. | |
40 | The $filename for C<eval>ed strings looks like C<(eval 34)>. | |
41 | Code assertions in regexes look like C<(re_eval 19)>. | |
8894c26d MJD |
42 | |
43 | Values in this array are magical in numeric context: they compare | |
44 | equal to zero only if the line is not breakable. | |
055fd3a9 GS |
45 | |
46 | =item * | |
47 | ||
aa0b556f | 48 | Each hash C<%{"_<$filename"}> contains breakpoints and actions keyed |
055fd3a9 GS |
49 | by line number. Individual entries (as opposed to the whole hash) |
50 | are settable. Perl only cares about Boolean true here, although | |
51 | the values used by F<perl5db.pl> have the form | |
8894c26d | 52 | C<"$break_condition\0$action">. |
055fd3a9 GS |
53 | |
54 | The same holds for evaluated strings that contain subroutines, or | |
55 | which are currently being executed. The $filename for C<eval>ed strings | |
56 | looks like C<(eval 34)> or C<(re_eval 19)>. | |
57 | ||
58 | =item * | |
59 | ||
aa0b556f | 60 | Each scalar C<${"_<$filename"}> contains C<"_<$filename">. This is |
055fd3a9 GS |
61 | also the case for evaluated strings that contain subroutines, or |
62 | which are currently being executed. The $filename for C<eval>ed | |
63 | strings looks like C<(eval 34)> or C<(re_eval 19)>. | |
64 | ||
65 | =item * | |
66 | ||
67 | After each C<require>d file is compiled, but before it is executed, | |
68 | C<DB::postponed(*{"_<$filename"})> is called if the subroutine | |
69 | C<DB::postponed> exists. Here, the $filename is the expanded name of | |
70 | the C<require>d file, as found in the values of %INC. | |
71 | ||
72 | =item * | |
73 | ||
74 | After each subroutine C<subname> is compiled, the existence of | |
75 | C<$DB::postponed{subname}> is checked. If this key exists, | |
76 | C<DB::postponed(subname)> is called if the C<DB::postponed> subroutine | |
77 | also exists. | |
78 | ||
79 | =item * | |
80 | ||
81 | A hash C<%DB::sub> is maintained, whose keys are subroutine names | |
82 | and whose values have the form C<filename:startline-endline>. | |
83 | C<filename> has the form C<(eval 34)> for subroutines defined inside | |
84 | C<eval>s, or C<(re_eval 19)> for those within regex code assertions. | |
85 | ||
86 | =item * | |
87 | ||
88 | When the execution of your program reaches a point that can hold a | |
74410c12 JM |
89 | breakpoint, the C<DB::DB()> subroutine is called if any of the variables |
90 | C<$DB::trace>, C<$DB::single>, or C<$DB::signal> is true. These variables | |
055fd3a9 GS |
91 | are not C<local>izable. This feature is disabled when executing |
92 | inside C<DB::DB()>, including functions called from it | |
93 | unless C<< $^D & (1<<30) >> is true. | |
94 | ||
95 | =item * | |
96 | ||
97 | When execution of the program reaches a subroutine call, a call to | |
98 | C<&DB::sub>(I<args>) is made instead, with C<$DB::sub> holding the | |
74410c12 | 99 | name of the called subroutine. (This doesn't happen if the subroutine |
055fd3a9 GS |
100 | was compiled in the C<DB> package.) |
101 | ||
102 | =back | |
103 | ||
104 | Note that if C<&DB::sub> needs external data for it to work, no | |
74410c12 JM |
105 | subroutine call is possible without it. As an example, the standard |
106 | debugger's C<&DB::sub> depends on the C<$DB::deep> variable | |
107 | (it defines how many levels of recursion deep into the debugger you can go | |
108 | before a mandatory break). If C<$DB::deep> is not defined, subroutine | |
109 | calls are not possible, even though C<&DB::sub> exists. | |
055fd3a9 GS |
110 | |
111 | =head2 Writing Your Own Debugger | |
112 | ||
74410c12 | 113 | =head3 Environment Variables |
666f95b9 | 114 | |
74410c12 JM |
115 | The C<PERL5DB> environment variable can be used to define a debugger. |
116 | For example, the minimal "working" debugger (it actually doesn't do anything) | |
117 | consists of one line: | |
666f95b9 | 118 | |
055fd3a9 GS |
119 | sub DB::DB {} |
120 | ||
74410c12 | 121 | It can easily be defined like this: |
666f95b9 | 122 | |
055fd3a9 GS |
123 | $ PERL5DB="sub DB::DB {}" perl -d your-script |
124 | ||
74410c12 | 125 | Another brief debugger, slightly more useful, can be created |
055fd3a9 GS |
126 | with only the line: |
127 | ||
128 | sub DB::DB {print ++$i; scalar <STDIN>} | |
129 | ||
74410c12 JM |
130 | This debugger prints a number which increments for each statement |
131 | encountered and waits for you to hit a newline before continuing | |
132 | to the next statement. | |
666f95b9 | 133 | |
74410c12 | 134 | The following debugger is actually useful: |
666f95b9 | 135 | |
055fd3a9 GS |
136 | { |
137 | package DB; | |
138 | sub DB {} | |
139 | sub sub {print ++$i, " $sub\n"; &$sub} | |
140 | } | |
141 | ||
74410c12 JM |
142 | It prints the sequence number of each subroutine call and the name of the |
143 | called subroutine. Note that C<&DB::sub> is being compiled into the | |
144 | package C<DB> through the use of the C<package> directive. | |
055fd3a9 | 145 | |
74410c12 JM |
146 | When it starts, the debugger reads your rc file (F<./.perldb> or |
147 | F<~/.perldb> under Unix), which can set important options. | |
148 | (A subroutine (C<&afterinit>) can be defined here as well; it is executed | |
149 | after the debugger completes its own initialization.) | |
055fd3a9 GS |
150 | |
151 | After the rc file is read, the debugger reads the PERLDB_OPTS | |
74410c12 JM |
152 | environment variable and uses it to set debugger options. The |
153 | contents of this variable are treated as if they were the argument | |
492652be | 154 | of an C<o ...> debugger command (q.v. in L<perldebug/Options>). |
74410c12 JM |
155 | |
156 | =head3 Debugger internal variables | |
157 | In addition to the file and subroutine-related variables mentioned above, | |
158 | the debugger also maintains various magical internal variables. | |
159 | ||
160 | =over 4 | |
161 | ||
162 | =item * | |
055fd3a9 | 163 | |
74410c12 JM |
164 | C<@DB::dbline> is an alias for C<@{"::_<current_file"}>, which |
165 | holds the lines of the currently-selected file (compiled by Perl), either | |
166 | explicitly chosen with the debugger's C<f> command, or implicitly by flow | |
167 | of execution. | |
168 | ||
169 | Values in this array are magical in numeric context: they compare | |
170 | equal to zero only if the line is not breakable. | |
171 | ||
172 | =item * | |
173 | ||
174 | C<%DB::dbline>, is an alias for C<%{"::_<current_file"}>, which | |
175 | contains breakpoints and actions keyed by line number in | |
176 | the currently-selected file, either explicitly chosen with the | |
055fd3a9 GS |
177 | debugger's C<f> command, or implicitly by flow of execution. |
178 | ||
74410c12 JM |
179 | As previously noted, individual entries (as opposed to the whole hash) |
180 | are settable. Perl only cares about Boolean true here, although | |
181 | the values used by F<perl5db.pl> have the form | |
182 | C<"$break_condition\0$action">. | |
183 | ||
184 | =back | |
185 | ||
7eabac42 | 186 | =head3 Debugger customization functions |
74410c12 JM |
187 | |
188 | Some functions are provided to simplify customization. | |
189 | ||
190 | =over 4 | |
191 | ||
192 | =item * | |
193 | ||
71110851 RGS |
194 | See L<perldebug/"Configurable Options"> for a description of options parsed by |
195 | C<DB::parse_options(string)>. | |
74410c12 JM |
196 | |
197 | =item * | |
198 | ||
199 | C<DB::dump_trace(skip[,count])> skips the specified number of frames | |
200 | and returns a list containing information about the calling frames (all | |
201 | of them, if C<count> is missing). Each entry is reference to a hash | |
202 | with keys C<context> (either C<.>, C<$>, or C<@>), C<sub> (subroutine | |
055fd3a9 GS |
203 | name, or info about C<eval>), C<args> (C<undef> or a reference to |
204 | an array), C<file>, and C<line>. | |
205 | ||
74410c12 JM |
206 | =item * |
207 | ||
208 | C<DB::print_trace(FH, skip[, count[, short]])> prints | |
055fd3a9 GS |
209 | formatted info about caller frames. The last two functions may be |
210 | convenient as arguments to C<< < >>, C<< << >> commands. | |
211 | ||
74410c12 JM |
212 | =back |
213 | ||
055fd3a9 GS |
214 | Note that any variables and functions that are not documented in |
215 | this manpages (or in L<perldebug>) are considered for internal | |
216 | use only, and as such are subject to change without notice. | |
217 | ||
218 | =head1 Frame Listing Output Examples | |
219 | ||
220 | The C<frame> option can be used to control the output of frame | |
221 | information. For example, contrast this expression trace: | |
222 | ||
223 | $ perl -de 42 | |
224 | Stack dump during die enabled outside of evals. | |
225 | ||
226 | Loading DB routines from perl5db.pl patch level 0.94 | |
227 | Emacs support available. | |
228 | ||
229 | Enter h or `h h' for help. | |
230 | ||
231 | main::(-e:1): 0 | |
232 | DB<1> sub foo { 14 } | |
233 | ||
234 | DB<2> sub bar { 3 } | |
235 | ||
236 | DB<3> t print foo() * bar() | |
237 | main::((eval 172):3): print foo() + bar(); | |
238 | main::foo((eval 168):2): | |
239 | main::bar((eval 170):2): | |
240 | 42 | |
241 | ||
492652be | 242 | with this one, once the C<o>ption C<frame=2> has been set: |
055fd3a9 | 243 | |
492652be | 244 | DB<4> o f=2 |
055fd3a9 GS |
245 | frame = '2' |
246 | DB<5> t print foo() * bar() | |
247 | 3: foo() * bar() | |
248 | entering main::foo | |
249 | 2: sub foo { 14 }; | |
250 | exited main::foo | |
251 | entering main::bar | |
252 | 2: sub bar { 3 }; | |
253 | exited main::bar | |
254 | 42 | |
255 | ||
256 | By way of demonstration, we present below a laborious listing | |
257 | resulting from setting your C<PERLDB_OPTS> environment variable to | |
258 | the value C<f=n N>, and running I<perl -d -V> from the command line. | |
259 | Examples use various values of C<n> are shown to give you a feel | |
260 | for the difference between settings. Long those it may be, this | |
261 | is not a complete listing, but only excerpts. | |
262 | ||
263 | =over 4 | |
264 | ||
265 | =item 1 | |
266 | ||
267 | entering main::BEGIN | |
268 | entering Config::BEGIN | |
269 | Package lib/Exporter.pm. | |
270 | Package lib/Carp.pm. | |
271 | Package lib/Config.pm. | |
272 | entering Config::TIEHASH | |
273 | entering Exporter::import | |
274 | entering Exporter::export | |
275 | entering Config::myconfig | |
276 | entering Config::FETCH | |
277 | entering Config::FETCH | |
278 | entering Config::FETCH | |
279 | entering Config::FETCH | |
280 | ||
281 | =item 2 | |
282 | ||
283 | entering main::BEGIN | |
284 | entering Config::BEGIN | |
285 | Package lib/Exporter.pm. | |
286 | Package lib/Carp.pm. | |
287 | exited Config::BEGIN | |
288 | Package lib/Config.pm. | |
289 | entering Config::TIEHASH | |
290 | exited Config::TIEHASH | |
291 | entering Exporter::import | |
292 | entering Exporter::export | |
293 | exited Exporter::export | |
294 | exited Exporter::import | |
295 | exited main::BEGIN | |
296 | entering Config::myconfig | |
297 | entering Config::FETCH | |
298 | exited Config::FETCH | |
299 | entering Config::FETCH | |
300 | exited Config::FETCH | |
301 | entering Config::FETCH | |
302 | ||
d5e42f17 | 303 | =item 3 |
055fd3a9 GS |
304 | |
305 | in $=main::BEGIN() from /dev/null:0 | |
306 | in $=Config::BEGIN() from lib/Config.pm:2 | |
307 | Package lib/Exporter.pm. | |
308 | Package lib/Carp.pm. | |
309 | Package lib/Config.pm. | |
310 | in $=Config::TIEHASH('Config') from lib/Config.pm:644 | |
311 | in $=Exporter::import('Config', 'myconfig', 'config_vars') from /dev/null:0 | |
312 | in $=Exporter::export('Config', 'main', 'myconfig', 'config_vars') from li | |
313 | in @=Config::myconfig() from /dev/null:0 | |
314 | in $=Config::FETCH(ref(Config), 'package') from lib/Config.pm:574 | |
315 | in $=Config::FETCH(ref(Config), 'baserev') from lib/Config.pm:574 | |
316 | in $=Config::FETCH(ref(Config), 'PERL_VERSION') from lib/Config.pm:574 | |
317 | in $=Config::FETCH(ref(Config), 'PERL_SUBVERSION') from lib/Config.pm:574 | |
318 | in $=Config::FETCH(ref(Config), 'osname') from lib/Config.pm:574 | |
319 | in $=Config::FETCH(ref(Config), 'osvers') from lib/Config.pm:574 | |
320 | ||
d5e42f17 | 321 | =item 4 |
055fd3a9 GS |
322 | |
323 | in $=main::BEGIN() from /dev/null:0 | |
324 | in $=Config::BEGIN() from lib/Config.pm:2 | |
325 | Package lib/Exporter.pm. | |
326 | Package lib/Carp.pm. | |
327 | out $=Config::BEGIN() from lib/Config.pm:0 | |
328 | Package lib/Config.pm. | |
329 | in $=Config::TIEHASH('Config') from lib/Config.pm:644 | |
330 | out $=Config::TIEHASH('Config') from lib/Config.pm:644 | |
331 | in $=Exporter::import('Config', 'myconfig', 'config_vars') from /dev/null:0 | |
332 | in $=Exporter::export('Config', 'main', 'myconfig', 'config_vars') from lib/ | |
333 | out $=Exporter::export('Config', 'main', 'myconfig', 'config_vars') from lib/ | |
334 | out $=Exporter::import('Config', 'myconfig', 'config_vars') from /dev/null:0 | |
335 | out $=main::BEGIN() from /dev/null:0 | |
336 | in @=Config::myconfig() from /dev/null:0 | |
337 | in $=Config::FETCH(ref(Config), 'package') from lib/Config.pm:574 | |
338 | out $=Config::FETCH(ref(Config), 'package') from lib/Config.pm:574 | |
339 | in $=Config::FETCH(ref(Config), 'baserev') from lib/Config.pm:574 | |
340 | out $=Config::FETCH(ref(Config), 'baserev') from lib/Config.pm:574 | |
341 | in $=Config::FETCH(ref(Config), 'PERL_VERSION') from lib/Config.pm:574 | |
342 | out $=Config::FETCH(ref(Config), 'PERL_VERSION') from lib/Config.pm:574 | |
343 | in $=Config::FETCH(ref(Config), 'PERL_SUBVERSION') from lib/Config.pm:574 | |
344 | ||
d5e42f17 | 345 | =item 5 |
055fd3a9 GS |
346 | |
347 | in $=main::BEGIN() from /dev/null:0 | |
348 | in $=Config::BEGIN() from lib/Config.pm:2 | |
349 | Package lib/Exporter.pm. | |
350 | Package lib/Carp.pm. | |
351 | out $=Config::BEGIN() from lib/Config.pm:0 | |
352 | Package lib/Config.pm. | |
353 | in $=Config::TIEHASH('Config') from lib/Config.pm:644 | |
354 | out $=Config::TIEHASH('Config') from lib/Config.pm:644 | |
355 | in $=Exporter::import('Config', 'myconfig', 'config_vars') from /dev/null:0 | |
356 | in $=Exporter::export('Config', 'main', 'myconfig', 'config_vars') from lib/E | |
357 | out $=Exporter::export('Config', 'main', 'myconfig', 'config_vars') from lib/E | |
358 | out $=Exporter::import('Config', 'myconfig', 'config_vars') from /dev/null:0 | |
359 | out $=main::BEGIN() from /dev/null:0 | |
360 | in @=Config::myconfig() from /dev/null:0 | |
361 | in $=Config::FETCH('Config=HASH(0x1aa444)', 'package') from lib/Config.pm:574 | |
362 | out $=Config::FETCH('Config=HASH(0x1aa444)', 'package') from lib/Config.pm:574 | |
363 | in $=Config::FETCH('Config=HASH(0x1aa444)', 'baserev') from lib/Config.pm:574 | |
364 | out $=Config::FETCH('Config=HASH(0x1aa444)', 'baserev') from lib/Config.pm:574 | |
365 | ||
d5e42f17 | 366 | =item 6 |
055fd3a9 GS |
367 | |
368 | in $=CODE(0x15eca4)() from /dev/null:0 | |
369 | in $=CODE(0x182528)() from lib/Config.pm:2 | |
370 | Package lib/Exporter.pm. | |
371 | out $=CODE(0x182528)() from lib/Config.pm:0 | |
372 | scalar context return from CODE(0x182528): undef | |
373 | Package lib/Config.pm. | |
374 | in $=Config::TIEHASH('Config') from lib/Config.pm:628 | |
375 | out $=Config::TIEHASH('Config') from lib/Config.pm:628 | |
376 | scalar context return from Config::TIEHASH: empty hash | |
377 | in $=Exporter::import('Config', 'myconfig', 'config_vars') from /dev/null:0 | |
378 | in $=Exporter::export('Config', 'main', 'myconfig', 'config_vars') from lib/Exporter.pm:171 | |
379 | out $=Exporter::export('Config', 'main', 'myconfig', 'config_vars') from lib/Exporter.pm:171 | |
380 | scalar context return from Exporter::export: '' | |
381 | out $=Exporter::import('Config', 'myconfig', 'config_vars') from /dev/null:0 | |
382 | scalar context return from Exporter::import: '' | |
383 | ||
384 | =back | |
385 | ||
386 | In all cases shown above, the line indentation shows the call tree. | |
387 | If bit 2 of C<frame> is set, a line is printed on exit from a | |
388 | subroutine as well. If bit 4 is set, the arguments are printed | |
389 | along with the caller info. If bit 8 is set, the arguments are | |
390 | printed even if they are tied or references. If bit 16 is set, the | |
391 | return value is printed, too. | |
392 | ||
393 | When a package is compiled, a line like this | |
394 | ||
395 | Package lib/Carp.pm. | |
396 | ||
397 | is printed with proper indentation. | |
398 | ||
399 | =head1 Debugging regular expressions | |
400 | ||
401 | There are two ways to enable debugging output for regular expressions. | |
402 | ||
403 | If your perl is compiled with C<-DDEBUGGING>, you may use the | |
404 | B<-Dr> flag on the command line. | |
405 | ||
406 | Otherwise, one can C<use re 'debug'>, which has effects at | |
407 | compile time and run time. It is not lexically scoped. | |
408 | ||
409 | =head2 Compile-time output | |
410 | ||
411 | The debugging output at compile time looks like this: | |
412 | ||
1c102323 MJD |
413 | Compiling REx `[bc]d(ef*g)+h[ij]k$' |
414 | size 45 Got 364 bytes for offset annotations. | |
415 | first at 1 | |
416 | rarest char g at 0 | |
417 | rarest char d at 0 | |
418 | 1: ANYOF[bc](12) | |
419 | 12: EXACT <d>(14) | |
420 | 14: CURLYX[0] {1,32767}(28) | |
421 | 16: OPEN1(18) | |
422 | 18: EXACT <e>(20) | |
423 | 20: STAR(23) | |
424 | 21: EXACT <f>(0) | |
425 | 23: EXACT <g>(25) | |
426 | 25: CLOSE1(27) | |
427 | 27: WHILEM[1/1](0) | |
428 | 28: NOTHING(29) | |
429 | 29: EXACT <h>(31) | |
430 | 31: ANYOF[ij](42) | |
431 | 42: EXACT <k>(44) | |
432 | 44: EOL(45) | |
433 | 45: END(0) | |
434 | anchored `de' at 1 floating `gh' at 3..2147483647 (checking floating) | |
435 | stclass `ANYOF[bc]' minlen 7 | |
436 | Offsets: [45] | |
437 | 1[4] 0[0] 0[0] 0[0] 0[0] 0[0] 0[0] 0[0] 0[0] 0[0] 0[0] 5[1] | |
438 | 0[0] 12[1] 0[0] 6[1] 0[0] 7[1] 0[0] 9[1] 8[1] 0[0] 10[1] 0[0] | |
439 | 11[1] 0[0] 12[0] 12[0] 13[1] 0[0] 14[4] 0[0] 0[0] 0[0] 0[0] | |
440 | 0[0] 0[0] 0[0] 0[0] 0[0] 0[0] 18[1] 0[0] 19[1] 20[0] | |
441 | Omitting $` $& $' support. | |
055fd3a9 GS |
442 | |
443 | The first line shows the pre-compiled form of the regex. The second | |
444 | shows the size of the compiled form (in arbitrary units, usually | |
1c102323 MJD |
445 | 4-byte words) and the total number of bytes allocated for the |
446 | offset/length table, usually 4+C<size>*8. The next line shows the | |
447 | label I<id> of the first node that does a match. | |
055fd3a9 | 448 | |
1c102323 MJD |
449 | The |
450 | ||
451 | anchored `de' at 1 floating `gh' at 3..2147483647 (checking floating) | |
452 | stclass `ANYOF[bc]' minlen 7 | |
453 | ||
454 | line (split into two lines above) contains optimizer | |
055fd3a9 GS |
455 | information. In the example shown, the optimizer found that the match |
456 | should contain a substring C<de> at offset 1, plus substring C<gh> | |
457 | at some offset between 3 and infinity. Moreover, when checking for | |
458 | these substrings (to abandon impossible matches quickly), Perl will check | |
459 | for the substring C<gh> before checking for the substring C<de>. The | |
460 | optimizer may also use the knowledge that the match starts (at the | |
1c102323 MJD |
461 | C<first> I<id>) with a character class, and no string |
462 | shorter than 7 characters can possibly match. | |
055fd3a9 | 463 | |
1c102323 | 464 | The fields of interest which may appear in this line are |
055fd3a9 | 465 | |
13a2d996 | 466 | =over 4 |
055fd3a9 GS |
467 | |
468 | =item C<anchored> I<STRING> C<at> I<POS> | |
469 | ||
470 | =item C<floating> I<STRING> C<at> I<POS1..POS2> | |
471 | ||
472 | See above. | |
473 | ||
474 | =item C<matching floating/anchored> | |
475 | ||
476 | Which substring to check first. | |
477 | ||
478 | =item C<minlen> | |
479 | ||
480 | The minimal length of the match. | |
481 | ||
482 | =item C<stclass> I<TYPE> | |
483 | ||
484 | Type of first matching node. | |
485 | ||
486 | =item C<noscan> | |
487 | ||
488 | Don't scan for the found substrings. | |
489 | ||
490 | =item C<isall> | |
491 | ||
1c102323 | 492 | Means that the optimizer information is all that the regular |
055fd3a9 GS |
493 | expression contains, and thus one does not need to enter the regex engine at |
494 | all. | |
495 | ||
496 | =item C<GPOS> | |
497 | ||
498 | Set if the pattern contains C<\G>. | |
499 | ||
500 | =item C<plus> | |
501 | ||
502 | Set if the pattern starts with a repeated char (as in C<x+y>). | |
503 | ||
504 | =item C<implicit> | |
505 | ||
506 | Set if the pattern starts with C<.*>. | |
507 | ||
508 | =item C<with eval> | |
509 | ||
510 | Set if the pattern contain eval-groups, such as C<(?{ code })> and | |
511 | C<(??{ code })>. | |
512 | ||
513 | =item C<anchored(TYPE)> | |
514 | ||
515 | If the pattern may match only at a handful of places, (with C<TYPE> | |
516 | being C<BOL>, C<MBOL>, or C<GPOS>. See the table below. | |
517 | ||
518 | =back | |
519 | ||
520 | If a substring is known to match at end-of-line only, it may be | |
521 | followed by C<$>, as in C<floating `k'$>. | |
522 | ||
1c102323 MJD |
523 | The optimizer-specific information is used to avoid entering (a slow) regex |
524 | engine on strings that will not definitely match. If the C<isall> flag | |
055fd3a9 GS |
525 | is set, a call to the regex engine may be avoided even when the optimizer |
526 | found an appropriate place for the match. | |
527 | ||
1c102323 | 528 | Above the optimizer section is the list of I<nodes> of the compiled |
055fd3a9 GS |
529 | form of the regex. Each line has format |
530 | ||
531 | C< >I<id>: I<TYPE> I<OPTIONAL-INFO> (I<next-id>) | |
532 | ||
533 | =head2 Types of nodes | |
534 | ||
535 | Here are the possible types, with short descriptions: | |
536 | ||
537 | # TYPE arg-description [num-args] [longjump-len] DESCRIPTION | |
538 | ||
539 | # Exit points | |
540 | END no End of program. | |
541 | SUCCEED no Return from a subroutine, basically. | |
542 | ||
543 | # Anchors: | |
544 | BOL no Match "" at beginning of line. | |
545 | MBOL no Same, assuming multiline. | |
546 | SBOL no Same, assuming singleline. | |
547 | EOS no Match "" at end of string. | |
548 | EOL no Match "" at end of line. | |
549 | MEOL no Same, assuming multiline. | |
550 | SEOL no Same, assuming singleline. | |
551 | BOUND no Match "" at any word boundary | |
552 | BOUNDL no Match "" at any word boundary | |
553 | NBOUND no Match "" at any word non-boundary | |
554 | NBOUNDL no Match "" at any word non-boundary | |
555 | GPOS no Matches where last m//g left off. | |
556 | ||
557 | # [Special] alternatives | |
558 | ANY no Match any one character (except newline). | |
559 | SANY no Match any one character. | |
560 | ANYOF sv Match character in (or not in) this class. | |
561 | ALNUM no Match any alphanumeric character | |
562 | ALNUML no Match any alphanumeric char in locale | |
563 | NALNUM no Match any non-alphanumeric character | |
564 | NALNUML no Match any non-alphanumeric char in locale | |
565 | SPACE no Match any whitespace character | |
566 | SPACEL no Match any whitespace char in locale | |
567 | NSPACE no Match any non-whitespace character | |
568 | NSPACEL no Match any non-whitespace char in locale | |
569 | DIGIT no Match any numeric character | |
570 | NDIGIT no Match any non-numeric character | |
571 | ||
572 | # BRANCH The set of branches constituting a single choice are hooked | |
573 | # together with their "next" pointers, since precedence prevents | |
574 | # anything being concatenated to any individual branch. The | |
575 | # "next" pointer of the last BRANCH in a choice points to the | |
576 | # thing following the whole choice. This is also where the | |
577 | # final "next" pointer of each individual branch points; each | |
578 | # branch starts with the operand node of a BRANCH node. | |
579 | # | |
580 | BRANCH node Match this alternative, or the next... | |
581 | ||
582 | # BACK Normal "next" pointers all implicitly point forward; BACK | |
583 | # exists to make loop structures possible. | |
584 | # not used | |
585 | BACK no Match "", "next" ptr points backward. | |
586 | ||
587 | # Literals | |
588 | EXACT sv Match this string (preceded by length). | |
589 | EXACTF sv Match this string, folded (prec. by length). | |
590 | EXACTFL sv Match this string, folded in locale (w/len). | |
591 | ||
592 | # Do nothing | |
593 | NOTHING no Match empty string. | |
594 | # A variant of above which delimits a group, thus stops optimizations | |
595 | TAIL no Match empty string. Can jump here from outside. | |
596 | ||
597 | # STAR,PLUS '?', and complex '*' and '+', are implemented as circular | |
598 | # BRANCH structures using BACK. Simple cases (one character | |
599 | # per match) are implemented with STAR and PLUS for speed | |
600 | # and to minimize recursive plunges. | |
601 | # | |
602 | STAR node Match this (simple) thing 0 or more times. | |
603 | PLUS node Match this (simple) thing 1 or more times. | |
604 | ||
605 | CURLY sv 2 Match this simple thing {n,m} times. | |
606 | CURLYN no 2 Match next-after-this simple thing | |
607 | # {n,m} times, set parens. | |
608 | CURLYM no 2 Match this medium-complex thing {n,m} times. | |
609 | CURLYX sv 2 Match this complex thing {n,m} times. | |
610 | ||
611 | # This terminator creates a loop structure for CURLYX | |
612 | WHILEM no Do curly processing and see if rest matches. | |
613 | ||
614 | # OPEN,CLOSE,GROUPP ...are numbered at compile time. | |
615 | OPEN num 1 Mark this point in input as start of #n. | |
616 | CLOSE num 1 Analogous to OPEN. | |
617 | ||
618 | REF num 1 Match some already matched string | |
619 | REFF num 1 Match already matched string, folded | |
620 | REFFL num 1 Match already matched string, folded in loc. | |
621 | ||
622 | # grouping assertions | |
623 | IFMATCH off 1 2 Succeeds if the following matches. | |
624 | UNLESSM off 1 2 Fails if the following matches. | |
625 | SUSPEND off 1 1 "Independent" sub-regex. | |
626 | IFTHEN off 1 1 Switch, should be preceded by switcher . | |
627 | GROUPP num 1 Whether the group matched. | |
628 | ||
629 | # Support for long regex | |
630 | LONGJMP off 1 1 Jump far away. | |
631 | BRANCHJ off 1 1 BRANCH with long offset. | |
632 | ||
633 | # The heavy worker | |
634 | EVAL evl 1 Execute some Perl code. | |
635 | ||
636 | # Modifiers | |
637 | MINMOD no Next operator is not greedy. | |
638 | LOGICAL no Next opcode should set the flag only. | |
639 | ||
640 | # This is not used yet | |
641 | RENUM off 1 1 Group with independently numbered parens. | |
642 | ||
643 | # This is not really a node, but an optimized away piece of a "long" node. | |
644 | # To simplify debugging output, we mark it as if it were a node | |
645 | OPTIMIZED off Placeholder for dump. | |
646 | ||
1c102323 MJD |
647 | =for unprinted-credits |
648 | Next section M-J. Dominus (mjd-perl-patch+@plover.com) 20010421 | |
649 | ||
650 | Following the optimizer information is a dump of the offset/length | |
651 | table, here split across several lines: | |
652 | ||
653 | Offsets: [45] | |
654 | 1[4] 0[0] 0[0] 0[0] 0[0] 0[0] 0[0] 0[0] 0[0] 0[0] 0[0] 5[1] | |
655 | 0[0] 12[1] 0[0] 6[1] 0[0] 7[1] 0[0] 9[1] 8[1] 0[0] 10[1] 0[0] | |
656 | 11[1] 0[0] 12[0] 12[0] 13[1] 0[0] 14[4] 0[0] 0[0] 0[0] 0[0] | |
657 | 0[0] 0[0] 0[0] 0[0] 0[0] 0[0] 18[1] 0[0] 19[1] 20[0] | |
658 | ||
659 | The first line here indicates that the offset/length table contains 45 | |
660 | entries. Each entry is a pair of integers, denoted by C<offset[length]>. | |
17c338f3 | 661 | Entries are numbered starting with 1, so entry #1 here is C<1[4]> and |
1c102323 MJD |
662 | entry #12 is C<5[1]>. C<1[4]> indicates that the node labeled C<1:> |
663 | (the C<1: ANYOF[bc]>) begins at character position 1 in the | |
664 | pre-compiled form of the regex, and has a length of 4 characters. | |
665 | C<5[1]> in position 12 | |
666 | indicates that the node labeled C<12:> | |
667 | (the C<< 12: EXACT <d> >>) begins at character position 5 in the | |
668 | pre-compiled form of the regex, and has a length of 1 character. | |
669 | C<12[1]> in position 14 | |
670 | indicates that the node labeled C<14:> | |
671 | (the C<< 14: CURLYX[0] {1,32767} >>) begins at character position 12 in the | |
672 | pre-compiled form of the regex, and has a length of 1 character---that | |
673 | is, it corresponds to the C<+> symbol in the precompiled regex. | |
674 | ||
675 | C<0[0]> items indicate that there is no corresponding node. | |
676 | ||
055fd3a9 GS |
677 | =head2 Run-time output |
678 | ||
679 | First of all, when doing a match, one may get no run-time output even | |
680 | if debugging is enabled. This means that the regex engine was never | |
681 | entered and that all of the job was therefore done by the optimizer. | |
682 | ||
683 | If the regex engine was entered, the output may look like this: | |
684 | ||
685 | Matching `[bc]d(ef*g)+h[ij]k$' against `abcdefg__gh__' | |
686 | Setting an EVAL scope, savestack=3 | |
687 | 2 <ab> <cdefg__gh_> | 1: ANYOF | |
688 | 3 <abc> <defg__gh_> | 11: EXACT <d> | |
689 | 4 <abcd> <efg__gh_> | 13: CURLYX {1,32767} | |
690 | 4 <abcd> <efg__gh_> | 26: WHILEM | |
691 | 0 out of 1..32767 cc=effff31c | |
692 | 4 <abcd> <efg__gh_> | 15: OPEN1 | |
693 | 4 <abcd> <efg__gh_> | 17: EXACT <e> | |
694 | 5 <abcde> <fg__gh_> | 19: STAR | |
695 | EXACT <f> can match 1 times out of 32767... | |
696 | Setting an EVAL scope, savestack=3 | |
697 | 6 <bcdef> <g__gh__> | 22: EXACT <g> | |
698 | 7 <bcdefg> <__gh__> | 24: CLOSE1 | |
699 | 7 <bcdefg> <__gh__> | 26: WHILEM | |
700 | 1 out of 1..32767 cc=effff31c | |
701 | Setting an EVAL scope, savestack=12 | |
702 | 7 <bcdefg> <__gh__> | 15: OPEN1 | |
703 | 7 <bcdefg> <__gh__> | 17: EXACT <e> | |
704 | restoring \1 to 4(4)..7 | |
705 | failed, try continuation... | |
706 | 7 <bcdefg> <__gh__> | 27: NOTHING | |
707 | 7 <bcdefg> <__gh__> | 28: EXACT <h> | |
708 | failed... | |
709 | failed... | |
710 | ||
711 | The most significant information in the output is about the particular I<node> | |
712 | of the compiled regex that is currently being tested against the target string. | |
713 | The format of these lines is | |
714 | ||
715 | C< >I<STRING-OFFSET> <I<PRE-STRING>> <I<POST-STRING>> |I<ID>: I<TYPE> | |
716 | ||
717 | The I<TYPE> info is indented with respect to the backtracking level. | |
718 | Other incidental information appears interspersed within. | |
719 | ||
720 | =head1 Debugging Perl memory usage | |
721 | ||
722 | Perl is a profligate wastrel when it comes to memory use. There | |
723 | is a saying that to estimate memory usage of Perl, assume a reasonable | |
724 | algorithm for memory allocation, multiply that estimate by 10, and | |
725 | while you still may miss the mark, at least you won't be quite so | |
4375e838 | 726 | astonished. This is not absolutely true, but may provide a good |
055fd3a9 GS |
727 | grasp of what happens. |
728 | ||
729 | Assume that an integer cannot take less than 20 bytes of memory, a | |
730 | float cannot take less than 24 bytes, a string cannot take less | |
731 | than 32 bytes (all these examples assume 32-bit architectures, the | |
732 | result are quite a bit worse on 64-bit architectures). If a variable | |
733 | is accessed in two of three different ways (which require an integer, | |
734 | a float, or a string), the memory footprint may increase yet another | |
b9449ee0 | 735 | 20 bytes. A sloppy malloc(3) implementation can inflate these |
055fd3a9 GS |
736 | numbers dramatically. |
737 | ||
738 | On the opposite end of the scale, a declaration like | |
739 | ||
740 | sub foo; | |
741 | ||
742 | may take up to 500 bytes of memory, depending on which release of Perl | |
743 | you're running. | |
744 | ||
745 | Anecdotal estimates of source-to-compiled code bloat suggest an | |
746 | eightfold increase. This means that the compiled form of reasonable | |
747 | (normally commented, properly indented etc.) code will take | |
748 | about eight times more space in memory than the code took | |
749 | on disk. | |
750 | ||
b30f304a JH |
751 | The B<-DL> command-line switch is obsolete since circa Perl 5.6.0 |
752 | (it was available only if Perl was built with C<-DDEBUGGING>). | |
753 | The switch was used to track Perl's memory allocations and possible | |
754 | memory leaks. These days the use of malloc debugging tools like | |
5b6a3331 SH |
755 | F<Purify> or F<valgrind> is suggested instead. See also |
756 | L<perlhack/PERL_MEM_LOG>. | |
b30f304a JH |
757 | |
758 | One way to find out how much memory is being used by Perl data | |
759 | structures is to install the Devel::Size module from CPAN: it gives | |
760 | you the minimum number of bytes required to store a particular data | |
761 | structure. Please be mindful of the difference between the size() | |
762 | and total_size(). | |
763 | ||
764 | If Perl has been compiled using Perl's malloc you can analyze Perl | |
765 | memory usage by setting the $ENV{PERL_DEBUG_MSTATS}. | |
055fd3a9 GS |
766 | |
767 | =head2 Using C<$ENV{PERL_DEBUG_MSTATS}> | |
768 | ||
769 | If your perl is using Perl's malloc() and was compiled with the | |
770 | necessary switches (this is the default), then it will print memory | |
4375e838 | 771 | usage statistics after compiling your code when C<< $ENV{PERL_DEBUG_MSTATS} |
055fd3a9 GS |
772 | > 1 >>, and before termination of the program when C<< |
773 | $ENV{PERL_DEBUG_MSTATS} >= 1 >>. The report format is similar to | |
774 | the following example: | |
775 | ||
776 | $ PERL_DEBUG_MSTATS=2 perl -e "require Carp" | |
777 | Memory allocation statistics after compilation: (buckets 4(4)..8188(8192) | |
778 | 14216 free: 130 117 28 7 9 0 2 2 1 0 0 | |
779 | 437 61 36 0 5 | |
780 | 60924 used: 125 137 161 55 7 8 6 16 2 0 1 | |
781 | 74 109 304 84 20 | |
782 | Total sbrk(): 77824/21:119. Odd ends: pad+heads+chain+tail: 0+636+0+2048. | |
783 | Memory allocation statistics after execution: (buckets 4(4)..8188(8192) | |
784 | 30888 free: 245 78 85 13 6 2 1 3 2 0 1 | |
785 | 315 162 39 42 11 | |
786 | 175816 used: 265 176 1112 111 26 22 11 27 2 1 1 | |
787 | 196 178 1066 798 39 | |
788 | Total sbrk(): 215040/47:145. Odd ends: pad+heads+chain+tail: 0+2192+0+6144. | |
789 | ||
790 | It is possible to ask for such a statistic at arbitrary points in | |
b9449ee0 | 791 | your execution using the mstat() function out of the standard |
055fd3a9 GS |
792 | Devel::Peek module. |
793 | ||
794 | Here is some explanation of that format: | |
795 | ||
13a2d996 | 796 | =over 4 |
055fd3a9 GS |
797 | |
798 | =item C<buckets SMALLEST(APPROX)..GREATEST(APPROX)> | |
799 | ||
800 | Perl's malloc() uses bucketed allocations. Every request is rounded | |
801 | up to the closest bucket size available, and a bucket is taken from | |
802 | the pool of buckets of that size. | |
803 | ||
804 | The line above describes the limits of buckets currently in use. | |
805 | Each bucket has two sizes: memory footprint and the maximal size | |
806 | of user data that can fit into this bucket. Suppose in the above | |
807 | example that the smallest bucket were size 4. The biggest bucket | |
808 | would have usable size 8188, and the memory footprint would be 8192. | |
809 | ||
810 | In a Perl built for debugging, some buckets may have negative usable | |
811 | size. This means that these buckets cannot (and will not) be used. | |
812 | For larger buckets, the memory footprint may be one page greater | |
813 | than a power of 2. If so, case the corresponding power of two is | |
814 | printed in the C<APPROX> field above. | |
815 | ||
816 | =item Free/Used | |
817 | ||
818 | The 1 or 2 rows of numbers following that correspond to the number | |
819 | of buckets of each size between C<SMALLEST> and C<GREATEST>. In | |
820 | the first row, the sizes (memory footprints) of buckets are powers | |
821 | of two--or possibly one page greater. In the second row, if present, | |
822 | the memory footprints of the buckets are between the memory footprints | |
823 | of two buckets "above". | |
824 | ||
4375e838 | 825 | For example, suppose under the previous example, the memory footprints |
055fd3a9 GS |
826 | were |
827 | ||
828 | free: 8 16 32 64 128 256 512 1024 2048 4096 8192 | |
829 | 4 12 24 48 80 | |
830 | ||
831 | With non-C<DEBUGGING> perl, the buckets starting from C<128> have | |
d1be9408 | 832 | a 4-byte overhead, and thus an 8192-long bucket may take up to |
055fd3a9 GS |
833 | 8188-byte allocations. |
834 | ||
835 | =item C<Total sbrk(): SBRKed/SBRKs:CONTINUOUS> | |
836 | ||
837 | The first two fields give the total amount of memory perl sbrk(2)ed | |
838 | (ess-broken? :-) and number of sbrk(2)s used. The third number is | |
839 | what perl thinks about continuity of returned chunks. So long as | |
840 | this number is positive, malloc() will assume that it is probable | |
841 | that sbrk(2) will provide continuous memory. | |
842 | ||
843 | Memory allocated by external libraries is not counted. | |
844 | ||
845 | =item C<pad: 0> | |
846 | ||
847 | The amount of sbrk(2)ed memory needed to keep buckets aligned. | |
848 | ||
849 | =item C<heads: 2192> | |
850 | ||
851 | Although memory overhead of bigger buckets is kept inside the bucket, for | |
852 | smaller buckets, it is kept in separate areas. This field gives the | |
853 | total size of these areas. | |
854 | ||
855 | =item C<chain: 0> | |
856 | ||
857 | malloc() may want to subdivide a bigger bucket into smaller buckets. | |
858 | If only a part of the deceased bucket is left unsubdivided, the rest | |
859 | is kept as an element of a linked list. This field gives the total | |
860 | size of these chunks. | |
861 | ||
862 | =item C<tail: 6144> | |
863 | ||
864 | To minimize the number of sbrk(2)s, malloc() asks for more memory. This | |
865 | field gives the size of the yet unused part, which is sbrk(2)ed, but | |
866 | never touched. | |
867 | ||
868 | =back | |
869 | ||
055fd3a9 GS |
870 | =head1 SEE ALSO |
871 | ||
872 | L<perldebug>, | |
873 | L<perlguts>, | |
874 | L<perlrun> | |
875 | L<re>, | |
876 | and | |
fe854a6f | 877 | L<Devel::DProf>. |