Commit | Line | Data |
---|---|---|
04c692a8 DR |
1 | =encoding utf8 |
2 | ||
3 | =for comment | |
4 | Consistent formatting of this file is achieved with: | |
5 | perl ./Porting/podtidy pod/perlinterp.pod | |
6 | ||
7 | =head1 NAME | |
8 | ||
9 | perlinterp - An overview of the Perl interpreter | |
10 | ||
11 | =head1 DESCRIPTION | |
12 | ||
13 | This document provides an overview of how the Perl interpreter works at | |
14 | the level of C code, along with pointers to the relevant C source code | |
15 | files. | |
16 | ||
17 | =head1 ELEMENTS OF THE INTERPRETER | |
18 | ||
19 | The work of the interpreter has two main stages: compiling the code | |
20 | into the internal representation, or bytecode, and then executing it. | |
21 | L<perlguts/Compiled code> explains exactly how the compilation stage | |
22 | happens. | |
23 | ||
24 | Here is a short breakdown of perl's operation: | |
25 | ||
26 | =head2 Startup | |
27 | ||
28 | The action begins in F<perlmain.c>. (or F<miniperlmain.c> for miniperl) | |
29 | This is very high-level code, enough to fit on a single screen, and it | |
30 | resembles the code found in L<perlembed>; most of the real action takes | |
31 | place in F<perl.c> | |
32 | ||
33 | F<perlmain.c> is generated by C<ExtUtils::Miniperl> from | |
34 | F<miniperlmain.c> at make time, so you should make perl to follow this | |
35 | along. | |
36 | ||
37 | First, F<perlmain.c> allocates some memory and constructs a Perl | |
38 | interpreter, along these lines: | |
39 | ||
40 | 1 PERL_SYS_INIT3(&argc,&argv,&env); | |
41 | 2 | |
42 | 3 if (!PL_do_undump) { | |
43 | 4 my_perl = perl_alloc(); | |
44 | 5 if (!my_perl) | |
45 | 6 exit(1); | |
46 | 7 perl_construct(my_perl); | |
47 | 8 PL_perl_destruct_level = 0; | |
48 | 9 } | |
49 | ||
50 | Line 1 is a macro, and its definition is dependent on your operating | |
51 | system. Line 3 references C<PL_do_undump>, a global variable - all | |
52 | global variables in Perl start with C<PL_>. This tells you whether the | |
53 | current running program was created with the C<-u> flag to perl and | |
54 | then F<undump>, which means it's going to be false in any sane context. | |
55 | ||
56 | Line 4 calls a function in F<perl.c> to allocate memory for a Perl | |
57 | interpreter. It's quite a simple function, and the guts of it looks | |
58 | like this: | |
59 | ||
60 | my_perl = (PerlInterpreter*)PerlMem_malloc(sizeof(PerlInterpreter)); | |
61 | ||
62 | Here you see an example of Perl's system abstraction, which we'll see | |
63 | later: C<PerlMem_malloc> is either your system's C<malloc>, or Perl's | |
64 | own C<malloc> as defined in F<malloc.c> if you selected that option at | |
65 | configure time. | |
66 | ||
67 | Next, in line 7, we construct the interpreter using perl_construct, | |
68 | also in F<perl.c>; this sets up all the special variables that Perl | |
69 | needs, the stacks, and so on. | |
70 | ||
71 | Now we pass Perl the command line options, and tell it to go: | |
72 | ||
fe2024f9 | 73 | if (!perl_parse(my_perl, xs_init, argc, argv, (char **)NULL)) |
04c692a8 DR |
74 | perl_run(my_perl); |
75 | ||
76 | exitstatus = perl_destruct(my_perl); | |
77 | ||
78 | perl_free(my_perl); | |
79 | ||
80 | C<perl_parse> is actually a wrapper around C<S_parse_body>, as defined | |
81 | in F<perl.c>, which processes the command line options, sets up any | |
82 | statically linked XS modules, opens the program and calls C<yyparse> to | |
83 | parse it. | |
84 | ||
85 | =head2 Parsing | |
86 | ||
87 | The aim of this stage is to take the Perl source, and turn it into an | |
88 | op tree. We'll see what one of those looks like later. Strictly | |
89 | speaking, there's three things going on here. | |
90 | ||
91 | C<yyparse>, the parser, lives in F<perly.c>, although you're better off | |
92 | reading the original YACC input in F<perly.y>. (Yes, Virginia, there | |
93 | B<is> a YACC grammar for Perl!) The job of the parser is to take your | |
94 | code and "understand" it, splitting it into sentences, deciding which | |
95 | operands go with which operators and so on. | |
96 | ||
97 | The parser is nobly assisted by the lexer, which chunks up your input | |
98 | into tokens, and decides what type of thing each token is: a variable | |
99 | name, an operator, a bareword, a subroutine, a core function, and so | |
100 | on. The main point of entry to the lexer is C<yylex>, and that and its | |
101 | associated routines can be found in F<toke.c>. Perl isn't much like | |
102 | other computer languages; it's highly context sensitive at times, it | |
103 | can be tricky to work out what sort of token something is, or where a | |
104 | token ends. As such, there's a lot of interplay between the tokeniser | |
105 | and the parser, which can get pretty frightening if you're not used to | |
106 | it. | |
107 | ||
108 | As the parser understands a Perl program, it builds up a tree of | |
109 | operations for the interpreter to perform during execution. The | |
110 | routines which construct and link together the various operations are | |
111 | to be found in F<op.c>, and will be examined later. | |
112 | ||
113 | =head2 Optimization | |
114 | ||
115 | Now the parsing stage is complete, and the finished tree represents the | |
116 | operations that the Perl interpreter needs to perform to execute our | |
117 | program. Next, Perl does a dry run over the tree looking for | |
118 | optimisations: constant expressions such as C<3 + 4> will be computed | |
119 | now, and the optimizer will also see if any multiple operations can be | |
120 | replaced with a single one. For instance, to fetch the variable | |
121 | C<$foo>, instead of grabbing the glob C<*foo> and looking at the scalar | |
122 | component, the optimizer fiddles the op tree to use a function which | |
123 | directly looks up the scalar in question. The main optimizer is C<peep> | |
124 | in F<op.c>, and many ops have their own optimizing functions. | |
125 | ||
126 | =head2 Running | |
127 | ||
128 | Now we're finally ready to go: we have compiled Perl byte code, and all | |
129 | that's left to do is run it. The actual execution is done by the | |
130 | C<runops_standard> function in F<run.c>; more specifically, it's done | |
131 | by these three innocent looking lines: | |
132 | ||
133 | while ((PL_op = PL_op->op_ppaddr(aTHX))) { | |
134 | PERL_ASYNC_CHECK(); | |
135 | } | |
136 | ||
137 | You may be more comfortable with the Perl version of that: | |
138 | ||
139 | PERL_ASYNC_CHECK() while $Perl::op = &{$Perl::op->{function}}; | |
140 | ||
141 | Well, maybe not. Anyway, each op contains a function pointer, which | |
142 | stipulates the function which will actually carry out the operation. | |
143 | This function will return the next op in the sequence - this allows for | |
144 | things like C<if> which choose the next op dynamically at run time. The | |
145 | C<PERL_ASYNC_CHECK> makes sure that things like signals interrupt | |
146 | execution if required. | |
147 | ||
148 | The actual functions called are known as PP code, and they're spread | |
149 | between four files: F<pp_hot.c> contains the "hot" code, which is most | |
150 | often used and highly optimized, F<pp_sys.c> contains all the | |
151 | system-specific functions, F<pp_ctl.c> contains the functions which | |
152 | implement control structures (C<if>, C<while> and the like) and F<pp.c> | |
153 | contains everything else. These are, if you like, the C code for Perl's | |
154 | built-in functions and operators. | |
155 | ||
156 | Note that each C<pp_> function is expected to return a pointer to the | |
157 | next op. Calls to perl subs (and eval blocks) are handled within the | |
158 | same runops loop, and do not consume extra space on the C stack. For | |
159 | example, C<pp_entersub> and C<pp_entertry> just push a C<CxSUB> or | |
160 | C<CxEVAL> block struct onto the context stack which contain the address | |
161 | of the op following the sub call or eval. They then return the first op | |
162 | of that sub or eval block, and so execution continues of that sub or | |
163 | block. Later, a C<pp_leavesub> or C<pp_leavetry> op pops the C<CxSUB> | |
164 | or C<CxEVAL>, retrieves the return op from it, and returns it. | |
165 | ||
166 | =head2 Exception handing | |
167 | ||
168 | Perl's exception handing (i.e. C<die> etc.) is built on top of the | |
169 | low-level C<setjmp()>/C<longjmp()> C-library functions. These basically | |
170 | provide a way to capture the current PC and SP registers and later | |
171 | restore them; i.e. a C<longjmp()> continues at the point in code where | |
172 | a previous C<setjmp()> was done, with anything further up on the C | |
173 | stack being lost. This is why code should always save values using | |
174 | C<SAVE_FOO> rather than in auto variables. | |
175 | ||
176 | The perl core wraps C<setjmp()> etc in the macros C<JMPENV_PUSH> and | |
177 | C<JMPENV_JUMP>. The basic rule of perl exceptions is that C<exit>, and | |
178 | C<die> (in the absence of C<eval>) perform a C<JMPENV_JUMP(2)>, while | |
179 | C<die> within C<eval> does a C<JMPENV_JUMP(3)>. | |
180 | ||
181 | At entry points to perl, such as C<perl_parse()>, C<perl_run()> and | |
182 | C<call_sv(cv, G_EVAL)> each does a C<JMPENV_PUSH>, then enter a runops | |
183 | loop or whatever, and handle possible exception returns. For a 2 | |
184 | return, final cleanup is performed, such as popping stacks and calling | |
185 | C<CHECK> or C<END> blocks. Amongst other things, this is how scope | |
186 | cleanup still occurs during an C<exit>. | |
187 | ||
188 | If a C<die> can find a C<CxEVAL> block on the context stack, then the | |
189 | stack is popped to that level and the return op in that block is | |
190 | assigned to C<PL_restartop>; then a C<JMPENV_JUMP(3)> is performed. | |
191 | This normally passes control back to the guard. In the case of | |
192 | C<perl_run> and C<call_sv>, a non-null C<PL_restartop> triggers | |
193 | re-entry to the runops loop. The is the normal way that C<die> or | |
194 | C<croak> is handled within an C<eval>. | |
195 | ||
196 | Sometimes ops are executed within an inner runops loop, such as tie, | |
197 | sort or overload code. In this case, something like | |
198 | ||
199 | sub FETCH { eval { die } } | |
200 | ||
201 | would cause a longjmp right back to the guard in C<perl_run>, popping | |
202 | both runops loops, which is clearly incorrect. One way to avoid this is | |
203 | for the tie code to do a C<JMPENV_PUSH> before executing C<FETCH> in | |
204 | the inner runops loop, but for efficiency reasons, perl in fact just | |
205 | sets a flag, using C<CATCH_SET(TRUE)>. The C<pp_require>, | |
206 | C<pp_entereval> and C<pp_entertry> ops check this flag, and if true, | |
207 | they call C<docatch>, which does a C<JMPENV_PUSH> and starts a new | |
208 | runops level to execute the code, rather than doing it on the current | |
209 | loop. | |
210 | ||
211 | As a further optimisation, on exit from the eval block in the C<FETCH>, | |
212 | execution of the code following the block is still carried on in the | |
213 | inner loop. When an exception is raised, C<docatch> compares the | |
214 | C<JMPENV> level of the C<CxEVAL> with C<PL_top_env> and if they differ, | |
215 | just re-throws the exception. In this way any inner loops get popped. | |
216 | ||
217 | Here's an example. | |
218 | ||
219 | 1: eval { tie @a, 'A' }; | |
220 | 2: sub A::TIEARRAY { | |
221 | 3: eval { die }; | |
222 | 4: die; | |
223 | 5: } | |
224 | ||
225 | To run this code, C<perl_run> is called, which does a C<JMPENV_PUSH> | |
226 | then enters a runops loop. This loop executes the eval and tie ops on | |
227 | line 1, with the eval pushing a C<CxEVAL> onto the context stack. | |
228 | ||
229 | The C<pp_tie> does a C<CATCH_SET(TRUE)>, then starts a second runops | |
230 | loop to execute the body of C<TIEARRAY>. When it executes the entertry | |
231 | op on line 3, C<CATCH_GET> is true, so C<pp_entertry> calls C<docatch> | |
232 | which does a C<JMPENV_PUSH> and starts a third runops loop, which then | |
233 | executes the die op. At this point the C call stack looks like this: | |
234 | ||
235 | Perl_pp_die | |
236 | Perl_runops # third loop | |
237 | S_docatch_body | |
238 | S_docatch | |
239 | Perl_pp_entertry | |
240 | Perl_runops # second loop | |
241 | S_call_body | |
242 | Perl_call_sv | |
243 | Perl_pp_tie | |
244 | Perl_runops # first loop | |
245 | S_run_body | |
246 | perl_run | |
247 | main | |
248 | ||
249 | and the context and data stacks, as shown by C<-Dstv>, look like: | |
250 | ||
251 | STACK 0: MAIN | |
252 | CX 0: BLOCK => | |
253 | CX 1: EVAL => AV() PV("A"\0) | |
254 | retop=leave | |
255 | STACK 1: MAGIC | |
256 | CX 0: SUB => | |
257 | retop=(null) | |
258 | CX 1: EVAL => * | |
259 | retop=nextstate | |
260 | ||
261 | The die pops the first C<CxEVAL> off the context stack, sets | |
262 | C<PL_restartop> from it, does a C<JMPENV_JUMP(3)>, and control returns | |
263 | to the top C<docatch>. This then starts another third-level runops | |
264 | level, which executes the nextstate, pushmark and die ops on line 4. At | |
265 | the point that the second C<pp_die> is called, the C call stack looks | |
266 | exactly like that above, even though we are no longer within an inner | |
267 | eval; this is because of the optimization mentioned earlier. However, | |
268 | the context stack now looks like this, ie with the top CxEVAL popped: | |
269 | ||
270 | STACK 0: MAIN | |
271 | CX 0: BLOCK => | |
272 | CX 1: EVAL => AV() PV("A"\0) | |
273 | retop=leave | |
274 | STACK 1: MAGIC | |
275 | CX 0: SUB => | |
276 | retop=(null) | |
277 | ||
278 | The die on line 4 pops the context stack back down to the CxEVAL, | |
279 | leaving it as: | |
280 | ||
281 | STACK 0: MAIN | |
282 | CX 0: BLOCK => | |
283 | ||
284 | As usual, C<PL_restartop> is extracted from the C<CxEVAL>, and a | |
285 | C<JMPENV_JUMP(3)> done, which pops the C stack back to the docatch: | |
286 | ||
287 | S_docatch | |
288 | Perl_pp_entertry | |
289 | Perl_runops # second loop | |
290 | S_call_body | |
291 | Perl_call_sv | |
292 | Perl_pp_tie | |
293 | Perl_runops # first loop | |
294 | S_run_body | |
295 | perl_run | |
296 | main | |
297 | ||
298 | In this case, because the C<JMPENV> level recorded in the C<CxEVAL> | |
299 | differs from the current one, C<docatch> just does a C<JMPENV_JUMP(3)> | |
300 | and the C stack unwinds to: | |
301 | ||
302 | perl_run | |
303 | main | |
304 | ||
305 | Because C<PL_restartop> is non-null, C<run_body> starts a new runops | |
306 | loop and execution continues. | |
307 | ||
308 | =head2 INTERNAL VARIABLE TYPES | |
309 | ||
310 | You should by now have had a look at L<perlguts>, which tells you about | |
311 | Perl's internal variable types: SVs, HVs, AVs and the rest. If not, do | |
312 | that now. | |
313 | ||
314 | These variables are used not only to represent Perl-space variables, | |
315 | but also any constants in the code, as well as some structures | |
316 | completely internal to Perl. The symbol table, for instance, is an | |
317 | ordinary Perl hash. Your code is represented by an SV as it's read into | |
318 | the parser; any program files you call are opened via ordinary Perl | |
319 | filehandles, and so on. | |
320 | ||
321 | The core L<Devel::Peek|Devel::Peek> module lets us examine SVs from a | |
322 | Perl program. Let's see, for instance, how Perl treats the constant | |
323 | C<"hello">. | |
324 | ||
325 | % perl -MDevel::Peek -e 'Dump("hello")' | |
326 | 1 SV = PV(0xa041450) at 0xa04ecbc | |
327 | 2 REFCNT = 1 | |
328 | 3 FLAGS = (POK,READONLY,pPOK) | |
329 | 4 PV = 0xa0484e0 "hello"\0 | |
330 | 5 CUR = 5 | |
331 | 6 LEN = 6 | |
332 | ||
333 | Reading C<Devel::Peek> output takes a bit of practise, so let's go | |
334 | through it line by line. | |
335 | ||
336 | Line 1 tells us we're looking at an SV which lives at C<0xa04ecbc> in | |
337 | memory. SVs themselves are very simple structures, but they contain a | |
338 | pointer to a more complex structure. In this case, it's a PV, a | |
339 | structure which holds a string value, at location C<0xa041450>. Line 2 | |
340 | is the reference count; there are no other references to this data, so | |
341 | it's 1. | |
342 | ||
343 | Line 3 are the flags for this SV - it's OK to use it as a PV, it's a | |
344 | read-only SV (because it's a constant) and the data is a PV internally. | |
345 | Next we've got the contents of the string, starting at location | |
346 | C<0xa0484e0>. | |
347 | ||
348 | Line 5 gives us the current length of the string - note that this does | |
349 | B<not> include the null terminator. Line 6 is not the length of the | |
350 | string, but the length of the currently allocated buffer; as the string | |
351 | grows, Perl automatically extends the available storage via a routine | |
352 | called C<SvGROW>. | |
353 | ||
354 | You can get at any of these quantities from C very easily; just add | |
355 | C<Sv> to the name of the field shown in the snippet, and you've got a | |
356 | macro which will return the value: C<SvCUR(sv)> returns the current | |
357 | length of the string, C<SvREFCOUNT(sv)> returns the reference count, | |
358 | C<SvPV(sv, len)> returns the string itself with its length, and so on. | |
359 | More macros to manipulate these properties can be found in L<perlguts>. | |
360 | ||
361 | Let's take an example of manipulating a PV, from C<sv_catpvn>, in | |
362 | F<sv.c> | |
363 | ||
364 | 1 void | |
5aaab254 | 365 | 2 Perl_sv_catpvn(pTHX_ SV *sv, const char *ptr, STRLEN len) |
04c692a8 DR |
366 | 3 { |
367 | 4 STRLEN tlen; | |
368 | 5 char *junk; | |
369 | ||
370 | 6 junk = SvPV_force(sv, tlen); | |
371 | 7 SvGROW(sv, tlen + len + 1); | |
372 | 8 if (ptr == junk) | |
373 | 9 ptr = SvPVX(sv); | |
374 | 10 Move(ptr,SvPVX(sv)+tlen,len,char); | |
375 | 11 SvCUR(sv) += len; | |
376 | 12 *SvEND(sv) = '\0'; | |
377 | 13 (void)SvPOK_only_UTF8(sv); /* validate pointer */ | |
378 | 14 SvTAINT(sv); | |
379 | 15 } | |
380 | ||
381 | This is a function which adds a string, C<ptr>, of length C<len> onto | |
382 | the end of the PV stored in C<sv>. The first thing we do in line 6 is | |
383 | make sure that the SV B<has> a valid PV, by calling the C<SvPV_force> | |
384 | macro to force a PV. As a side effect, C<tlen> gets set to the current | |
385 | value of the PV, and the PV itself is returned to C<junk>. | |
386 | ||
387 | In line 7, we make sure that the SV will have enough room to | |
388 | accommodate the old string, the new string and the null terminator. If | |
389 | C<LEN> isn't big enough, C<SvGROW> will reallocate space for us. | |
390 | ||
391 | Now, if C<junk> is the same as the string we're trying to add, we can | |
392 | grab the string directly from the SV; C<SvPVX> is the address of the PV | |
393 | in the SV. | |
394 | ||
395 | Line 10 does the actual catenation: the C<Move> macro moves a chunk of | |
396 | memory around: we move the string C<ptr> to the end of the PV - that's | |
397 | the start of the PV plus its current length. We're moving C<len> bytes | |
398 | of type C<char>. After doing so, we need to tell Perl we've extended | |
399 | the string, by altering C<CUR> to reflect the new length. C<SvEND> is a | |
400 | macro which gives us the end of the string, so that needs to be a | |
401 | C<"\0">. | |
402 | ||
403 | Line 13 manipulates the flags; since we've changed the PV, any IV or NV | |
404 | values will no longer be valid: if we have C<$a=10; $a.="6";> we don't | |
405 | want to use the old IV of 10. C<SvPOK_only_utf8> is a special | |
406 | UTF-8-aware version of C<SvPOK_only>, a macro which turns off the IOK | |
407 | and NOK flags and turns on POK. The final C<SvTAINT> is a macro which | |
408 | launders tainted data if taint mode is turned on. | |
409 | ||
410 | AVs and HVs are more complicated, but SVs are by far the most common | |
411 | variable type being thrown around. Having seen something of how we | |
412 | manipulate these, let's go on and look at how the op tree is | |
413 | constructed. | |
414 | ||
415 | =head1 OP TREES | |
416 | ||
417 | First, what is the op tree, anyway? The op tree is the parsed | |
418 | representation of your program, as we saw in our section on parsing, | |
419 | and it's the sequence of operations that Perl goes through to execute | |
420 | your program, as we saw in L</Running>. | |
421 | ||
422 | An op is a fundamental operation that Perl can perform: all the | |
423 | built-in functions and operators are ops, and there are a series of ops | |
424 | which deal with concepts the interpreter needs internally - entering | |
425 | and leaving a block, ending a statement, fetching a variable, and so | |
426 | on. | |
427 | ||
428 | The op tree is connected in two ways: you can imagine that there are | |
429 | two "routes" through it, two orders in which you can traverse the tree. | |
430 | First, parse order reflects how the parser understood the code, and | |
431 | secondly, execution order tells perl what order to perform the | |
432 | operations in. | |
433 | ||
434 | The easiest way to examine the op tree is to stop Perl after it has | |
435 | finished parsing, and get it to dump out the tree. This is exactly what | |
436 | the compiler backends L<B::Terse|B::Terse>, L<B::Concise|B::Concise> | |
903b1101 | 437 | and CPAN module <B::Debug do. |
04c692a8 DR |
438 | |
439 | Let's have a look at how Perl sees C<$a = $b + $c>: | |
440 | ||
441 | % perl -MO=Terse -e '$a=$b+$c' | |
442 | 1 LISTOP (0x8179888) leave | |
443 | 2 OP (0x81798b0) enter | |
444 | 3 COP (0x8179850) nextstate | |
445 | 4 BINOP (0x8179828) sassign | |
446 | 5 BINOP (0x8179800) add [1] | |
447 | 6 UNOP (0x81796e0) null [15] | |
448 | 7 SVOP (0x80fafe0) gvsv GV (0x80fa4cc) *b | |
449 | 8 UNOP (0x81797e0) null [15] | |
450 | 9 SVOP (0x8179700) gvsv GV (0x80efeb0) *c | |
451 | 10 UNOP (0x816b4f0) null [15] | |
452 | 11 SVOP (0x816dcf0) gvsv GV (0x80fa460) *a | |
453 | ||
454 | Let's start in the middle, at line 4. This is a BINOP, a binary | |
455 | operator, which is at location C<0x8179828>. The specific operator in | |
456 | question is C<sassign> - scalar assignment - and you can find the code | |
457 | which implements it in the function C<pp_sassign> in F<pp_hot.c>. As a | |
458 | binary operator, it has two children: the add operator, providing the | |
459 | result of C<$b+$c>, is uppermost on line 5, and the left hand side is | |
460 | on line 10. | |
461 | ||
462 | Line 10 is the null op: this does exactly nothing. What is that doing | |
463 | there? If you see the null op, it's a sign that something has been | |
464 | optimized away after parsing. As we mentioned in L</Optimization>, the | |
465 | optimization stage sometimes converts two operations into one, for | |
466 | example when fetching a scalar variable. When this happens, instead of | |
467 | rewriting the op tree and cleaning up the dangling pointers, it's | |
468 | easier just to replace the redundant operation with the null op. | |
469 | Originally, the tree would have looked like this: | |
470 | ||
471 | 10 SVOP (0x816b4f0) rv2sv [15] | |
472 | 11 SVOP (0x816dcf0) gv GV (0x80fa460) *a | |
473 | ||
474 | That is, fetch the C<a> entry from the main symbol table, and then look | |
f672247c | 475 | at the scalar component of it: C<gvsv> (C<pp_gvsv> in F<pp_hot.c>) |
04c692a8 DR |
476 | happens to do both these things. |
477 | ||
478 | The right hand side, starting at line 5 is similar to what we've just | |
f672247c | 479 | seen: we have the C<add> op (C<pp_add>, also in F<pp_hot.c>) add |
04c692a8 DR |
480 | together two C<gvsv>s. |
481 | ||
482 | Now, what's this about? | |
483 | ||
484 | 1 LISTOP (0x8179888) leave | |
485 | 2 OP (0x81798b0) enter | |
486 | 3 COP (0x8179850) nextstate | |
487 | ||
488 | C<enter> and C<leave> are scoping ops, and their job is to perform any | |
489 | housekeeping every time you enter and leave a block: lexical variables | |
490 | are tidied up, unreferenced variables are destroyed, and so on. Every | |
491 | program will have those first three lines: C<leave> is a list, and its | |
492 | children are all the statements in the block. Statements are delimited | |
493 | by C<nextstate>, so a block is a collection of C<nextstate> ops, with | |
494 | the ops to be performed for each statement being the children of | |
495 | C<nextstate>. C<enter> is a single op which functions as a marker. | |
496 | ||
497 | That's how Perl parsed the program, from top to bottom: | |
498 | ||
499 | Program | |
500 | | | |
501 | Statement | |
502 | | | |
503 | = | |
504 | / \ | |
505 | / \ | |
506 | $a + | |
507 | / \ | |
508 | $b $c | |
509 | ||
510 | However, it's impossible to B<perform> the operations in this order: | |
511 | you have to find the values of C<$b> and C<$c> before you add them | |
512 | together, for instance. So, the other thread that runs through the op | |
513 | tree is the execution order: each op has a field C<op_next> which | |
514 | points to the next op to be run, so following these pointers tells us | |
515 | how perl executes the code. We can traverse the tree in this order | |
516 | using the C<exec> option to C<B::Terse>: | |
517 | ||
518 | % perl -MO=Terse,exec -e '$a=$b+$c' | |
519 | 1 OP (0x8179928) enter | |
520 | 2 COP (0x81798c8) nextstate | |
521 | 3 SVOP (0x81796c8) gvsv GV (0x80fa4d4) *b | |
522 | 4 SVOP (0x8179798) gvsv GV (0x80efeb0) *c | |
523 | 5 BINOP (0x8179878) add [1] | |
524 | 6 SVOP (0x816dd38) gvsv GV (0x80fa468) *a | |
525 | 7 BINOP (0x81798a0) sassign | |
526 | 8 LISTOP (0x8179900) leave | |
527 | ||
528 | This probably makes more sense for a human: enter a block, start a | |
529 | statement. Get the values of C<$b> and C<$c>, and add them together. | |
530 | Find C<$a>, and assign one to the other. Then leave. | |
531 | ||
532 | The way Perl builds up these op trees in the parsing process can be | |
65169990 FC |
533 | unravelled by examining F<toke.c>, the lexer, and F<perly.y>, the YACC |
534 | grammar. Let's look at the code that constructs the tree for C<$a = $b + | |
535 | $c>. | |
536 | ||
537 | First, we'll look at the C<Perl_yylex> function in the lexer. We want to | |
538 | look for C<case 'x'>, where x is the first character of the operator. | |
539 | (Incidentally, when looking for the code that handles a keyword, you'll | |
540 | want to search for C<KEY_foo> where "foo" is the keyword.) Here is the code | |
541 | that handles assignment (there are quite a few operators beginning with | |
542 | C<=>, so most of it is omitted for brevity): | |
543 | ||
544 | 1 case '=': | |
545 | 2 s++; | |
546 | ... code that handles == => etc. and pod ... | |
547 | 3 pl_yylval.ival = 0; | |
548 | 4 OPERATOR(ASSIGNOP); | |
549 | ||
550 | We can see on line 4 that our token type is C<ASSIGNOP> (C<OPERATOR> is a | |
551 | macro, defined in F<toke.c>, that returns the token type, among other | |
552 | things). And C<+>: | |
553 | ||
554 | 1 case '+': | |
555 | 2 { | |
556 | 3 const char tmp = *s++; | |
557 | ... code for ++ ... | |
558 | 4 if (PL_expect == XOPERATOR) { | |
559 | ... | |
560 | 5 Aop(OP_ADD); | |
561 | 6 } | |
562 | ... | |
563 | 7 } | |
564 | ||
565 | Line 4 checks what type of token we are expecting. C<Aop> returns a token. | |
566 | If you search for C<Aop> elsewhere in F<toke.c>, you will see that it | |
567 | returns an C<ADDOP> token. | |
568 | ||
569 | Now that we know the two token types we want to look for in the parser, | |
570 | let's take the piece of F<perly.y> we need to construct the tree for | |
571 | C<$a = $b + $c> | |
04c692a8 DR |
572 | |
573 | 1 term : term ASSIGNOP term | |
574 | 2 { $$ = newASSIGNOP(OPf_STACKED, $1, $2, $3); } | |
575 | 3 | term ADDOP term | |
576 | 4 { $$ = newBINOP($2, 0, scalar($1), scalar($3)); } | |
577 | ||
578 | If you're not used to reading BNF grammars, this is how it works: | |
579 | You're fed certain things by the tokeniser, which generally end up in | |
65169990 FC |
580 | upper case. C<ADDOP> and C<ASSIGNOP> are examples of "terminal symbols", |
581 | because you can't get any simpler than | |
04c692a8 DR |
582 | them. |
583 | ||
584 | The grammar, lines one and three of the snippet above, tells you how to | |
585 | build up more complex forms. These complex forms, "non-terminal | |
586 | symbols" are generally placed in lower case. C<term> here is a | |
587 | non-terminal symbol, representing a single expression. | |
588 | ||
589 | The grammar gives you the following rule: you can make the thing on the | |
590 | left of the colon if you see all the things on the right in sequence. | |
591 | This is called a "reduction", and the aim of parsing is to completely | |
592 | reduce the input. There are several different ways you can perform a | |
593 | reduction, separated by vertical bars: so, C<term> followed by C<=> | |
594 | followed by C<term> makes a C<term>, and C<term> followed by C<+> | |
595 | followed by C<term> can also make a C<term>. | |
596 | ||
597 | So, if you see two terms with an C<=> or C<+>, between them, you can | |
598 | turn them into a single expression. When you do this, you execute the | |
599 | code in the block on the next line: if you see C<=>, you'll do the code | |
600 | in line 2. If you see C<+>, you'll do the code in line 4. It's this | |
601 | code which contributes to the op tree. | |
602 | ||
603 | | term ADDOP term | |
604 | { $$ = newBINOP($2, 0, scalar($1), scalar($3)); } | |
605 | ||
606 | What this does is creates a new binary op, and feeds it a number of | |
607 | variables. The variables refer to the tokens: C<$1> is the first token | |
608 | in the input, C<$2> the second, and so on - think regular expression | |
609 | backreferences. C<$$> is the op returned from this reduction. So, we | |
610 | call C<newBINOP> to create a new binary operator. The first parameter | |
611 | to C<newBINOP>, a function in F<op.c>, is the op type. It's an addition | |
612 | operator, so we want the type to be C<ADDOP>. We could specify this | |
613 | directly, but it's right there as the second token in the input, so we | |
614 | use C<$2>. The second parameter is the op's flags: 0 means "nothing | |
615 | special". Then the things to add: the left and right hand side of our | |
616 | expression, in scalar context. | |
617 | ||
65169990 FC |
618 | The functions that create ops, which have names like C<newUNOP> and |
619 | C<newBINOP>, call a "check" function associated with each op type, before | |
620 | returning the op. The check functions can mangle the op as they see fit, | |
621 | and even replace it with an entirely new one. These functions are defined | |
622 | in F<op.c>, and have a C<Perl_ck_> prefix. You can find out which | |
623 | check function is used for a particular op type by looking in | |
624 | F<regen/opcodes>. Take C<OP_ADD>, for example. (C<OP_ADD> is the token | |
625 | value from the C<Aop(OP_ADD)> in F<toke.c> which the parser passes to | |
626 | C<newBINOP> as its first argument.) Here is the relevant line: | |
627 | ||
628 | add addition (+) ck_null IfsT2 S S | |
629 | ||
630 | The check function in this case is C<Perl_ck_null>, which does nothing. | |
631 | Let's look at a more interesting case: | |
632 | ||
633 | readline <HANDLE> ck_readline t% F? | |
634 | ||
635 | And here is the function from F<op.c>: | |
636 | ||
637 | 1 OP * | |
638 | 2 Perl_ck_readline(pTHX_ OP *o) | |
639 | 3 { | |
640 | 4 PERL_ARGS_ASSERT_CK_READLINE; | |
641 | 5 | |
642 | 6 if (o->op_flags & OPf_KIDS) { | |
643 | 7 OP *kid = cLISTOPo->op_first; | |
644 | 8 if (kid->op_type == OP_RV2GV) | |
645 | 9 kid->op_private |= OPpALLOW_FAKE; | |
646 | 10 } | |
647 | 11 else { | |
648 | 12 OP * const newop | |
649 | 13 = newUNOP(OP_READLINE, 0, newGVOP(OP_GV, 0, | |
650 | 14 PL_argvgv)); | |
651 | 15 op_free(o); | |
652 | 16 return newop; | |
653 | 17 } | |
654 | 18 return o; | |
655 | 19 } | |
656 | ||
a1ac675e | 657 | One particularly interesting aspect is that if the op has no kids (i.e., |
65169990 FC |
658 | C<readline()> or C<< <> >>) the op is freed and replaced with an entirely |
659 | new one that references C<*ARGV> (lines 12-16). | |
660 | ||
04c692a8 DR |
661 | =head1 STACKS |
662 | ||
663 | When perl executes something like C<addop>, how does it pass on its | |
664 | results to the next op? The answer is, through the use of stacks. Perl | |
665 | has a number of stacks to store things it's currently working on, and | |
666 | we'll look at the three most important ones here. | |
667 | ||
668 | =head2 Argument stack | |
669 | ||
670 | Arguments are passed to PP code and returned from PP code using the | |
671 | argument stack, C<ST>. The typical way to handle arguments is to pop | |
672 | them off the stack, deal with them how you wish, and then push the | |
673 | result back onto the stack. This is how, for instance, the cosine | |
674 | operator works: | |
675 | ||
676 | NV value; | |
677 | value = POPn; | |
678 | value = Perl_cos(value); | |
679 | XPUSHn(value); | |
680 | ||
681 | We'll see a more tricky example of this when we consider Perl's macros | |
682 | below. C<POPn> gives you the NV (floating point value) of the top SV on | |
683 | the stack: the C<$x> in C<cos($x)>. Then we compute the cosine, and | |
684 | push the result back as an NV. The C<X> in C<XPUSHn> means that the | |
685 | stack should be extended if necessary - it can't be necessary here, | |
686 | because we know there's room for one more item on the stack, since | |
687 | we've just removed one! The C<XPUSH*> macros at least guarantee safety. | |
688 | ||
689 | Alternatively, you can fiddle with the stack directly: C<SP> gives you | |
690 | the first element in your portion of the stack, and C<TOP*> gives you | |
691 | the top SV/IV/NV/etc. on the stack. So, for instance, to do unary | |
692 | negation of an integer: | |
693 | ||
694 | SETi(-TOPi); | |
695 | ||
696 | Just set the integer value of the top stack entry to its negation. | |
697 | ||
698 | Argument stack manipulation in the core is exactly the same as it is in | |
699 | XSUBs - see L<perlxstut>, L<perlxs> and L<perlguts> for a longer | |
700 | description of the macros used in stack manipulation. | |
701 | ||
702 | =head2 Mark stack | |
703 | ||
704 | I say "your portion of the stack" above because PP code doesn't | |
705 | necessarily get the whole stack to itself: if your function calls | |
706 | another function, you'll only want to expose the arguments aimed for | |
707 | the called function, and not (necessarily) let it get at your own data. | |
708 | The way we do this is to have a "virtual" bottom-of-stack, exposed to | |
709 | each function. The mark stack keeps bookmarks to locations in the | |
710 | argument stack usable by each function. For instance, when dealing with | |
711 | a tied variable, (internally, something with "P" magic) Perl has to | |
712 | call methods for accesses to the tied variables. However, we need to | |
713 | separate the arguments exposed to the method to the argument exposed to | |
714 | the original function - the store or fetch or whatever it may be. | |
715 | Here's roughly how the tied C<push> is implemented; see C<av_push> in | |
716 | F<av.c>: | |
717 | ||
718 | 1 PUSHMARK(SP); | |
719 | 2 EXTEND(SP,2); | |
720 | 3 PUSHs(SvTIED_obj((SV*)av, mg)); | |
721 | 4 PUSHs(val); | |
722 | 5 PUTBACK; | |
723 | 6 ENTER; | |
724 | 7 call_method("PUSH", G_SCALAR|G_DISCARD); | |
725 | 8 LEAVE; | |
726 | ||
727 | Let's examine the whole implementation, for practice: | |
728 | ||
729 | 1 PUSHMARK(SP); | |
730 | ||
731 | Push the current state of the stack pointer onto the mark stack. This | |
732 | is so that when we've finished adding items to the argument stack, Perl | |
733 | knows how many things we've added recently. | |
734 | ||
735 | 2 EXTEND(SP,2); | |
736 | 3 PUSHs(SvTIED_obj((SV*)av, mg)); | |
737 | 4 PUSHs(val); | |
738 | ||
739 | We're going to add two more items onto the argument stack: when you | |
740 | have a tied array, the C<PUSH> subroutine receives the object and the | |
741 | value to be pushed, and that's exactly what we have here - the tied | |
742 | object, retrieved with C<SvTIED_obj>, and the value, the SV C<val>. | |
743 | ||
744 | 5 PUTBACK; | |
745 | ||
746 | Next we tell Perl to update the global stack pointer from our internal | |
747 | variable: C<dSP> only gave us a local copy, not a reference to the | |
748 | global. | |
749 | ||
750 | 6 ENTER; | |
751 | 7 call_method("PUSH", G_SCALAR|G_DISCARD); | |
752 | 8 LEAVE; | |
753 | ||
754 | C<ENTER> and C<LEAVE> localise a block of code - they make sure that | |
755 | all variables are tidied up, everything that has been localised gets | |
756 | its previous value returned, and so on. Think of them as the C<{> and | |
757 | C<}> of a Perl block. | |
758 | ||
759 | To actually do the magic method call, we have to call a subroutine in | |
760 | Perl space: C<call_method> takes care of that, and it's described in | |
761 | L<perlcall>. We call the C<PUSH> method in scalar context, and we're | |
762 | going to discard its return value. The call_method() function removes | |
763 | the top element of the mark stack, so there is nothing for the caller | |
764 | to clean up. | |
765 | ||
766 | =head2 Save stack | |
767 | ||
768 | C doesn't have a concept of local scope, so perl provides one. We've | |
769 | seen that C<ENTER> and C<LEAVE> are used as scoping braces; the save | |
770 | stack implements the C equivalent of, for example: | |
771 | ||
772 | { | |
773 | local $foo = 42; | |
774 | ... | |
775 | } | |
776 | ||
548d0ee5 | 777 | See L<perlguts/"Localizing changes"> for how to use the save stack. |
04c692a8 DR |
778 | |
779 | =head1 MILLIONS OF MACROS | |
780 | ||
781 | One thing you'll notice about the Perl source is that it's full of | |
782 | macros. Some have called the pervasive use of macros the hardest thing | |
783 | to understand, others find it adds to clarity. Let's take an example, | |
784 | the code which implements the addition operator: | |
785 | ||
786 | 1 PP(pp_add) | |
787 | 2 { | |
788 | 3 dSP; dATARGET; tryAMAGICbin(add,opASSIGN); | |
789 | 4 { | |
790 | 5 dPOPTOPnnrl_ul; | |
791 | 6 SETn( left + right ); | |
792 | 7 RETURN; | |
793 | 8 } | |
794 | 9 } | |
795 | ||
796 | Every line here (apart from the braces, of course) contains a macro. | |
797 | The first line sets up the function declaration as Perl expects for PP | |
798 | code; line 3 sets up variable declarations for the argument stack and | |
799 | the target, the return value of the operation. Finally, it tries to see | |
800 | if the addition operation is overloaded; if so, the appropriate | |
801 | subroutine is called. | |
802 | ||
803 | Line 5 is another variable declaration - all variable declarations | |
804 | start with C<d> - which pops from the top of the argument stack two NVs | |
805 | (hence C<nn>) and puts them into the variables C<right> and C<left>, | |
806 | hence the C<rl>. These are the two operands to the addition operator. | |
807 | Next, we call C<SETn> to set the NV of the return value to the result | |
808 | of adding the two values. This done, we return - the C<RETURN> macro | |
809 | makes sure that our return value is properly handled, and we pass the | |
810 | next operator to run back to the main run loop. | |
811 | ||
812 | Most of these macros are explained in L<perlapi>, and some of the more | |
813 | important ones are explained in L<perlxs> as well. Pay special | |
814 | attention to L<perlguts/Background and PERL_IMPLICIT_CONTEXT> for | |
815 | information on the C<[pad]THX_?> macros. | |
816 | ||
817 | =head1 FURTHER READING | |
818 | ||
819 | For more information on the Perl internals, please see the documents | |
820 | listed at L<perl/Internals and C Language Interface>. |