| 1 | Perl Compiler Kit, Version alpha4 |
| 2 | |
| 3 | Copyright (c) 1996, 1997, Malcolm Beattie |
| 4 | |
| 5 | This program is free software; you can redistribute it and/or modify |
| 6 | it under the terms of either: |
| 7 | |
| 8 | a) the GNU General Public License as published by the Free |
| 9 | Software Foundation; either version 1, or (at your option) any |
| 10 | later version, or |
| 11 | |
| 12 | b) the "Artistic License" which comes with this kit. |
| 13 | |
| 14 | This program is distributed in the hope that it will be useful, |
| 15 | but WITHOUT ANY WARRANTY; without even the implied warranty of |
| 16 | MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See either |
| 17 | the GNU General Public License or the Artistic License for more details. |
| 18 | |
| 19 | You should have received a copy of the Artistic License with this kit, |
| 20 | in the file named "Artistic". If not, you can get one from the Perl |
| 21 | distribution. You should also have received a copy of the GNU General |
| 22 | Public License, in the file named "Copying". If not, you can get one |
| 23 | from the Perl distribution or else write to the Free Software Foundation, |
| 24 | Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307, USA. |
| 25 | |
| 26 | CHANGES |
| 27 | |
| 28 | New since alpha3 |
| 29 | Anonymous subs work properly with C and CC. |
| 30 | Heuristics for forcing compilation of apparently unused subs/methods. |
| 31 | Subs which use the AutoLoader module are forcibly loaded at compile-time. |
| 32 | Slightly faster compilation. |
| 33 | Handles slightly more complex code within a BEGIN { }. |
| 34 | Minor bug fixes. |
| 35 | |
| 36 | New since alpha2 |
| 37 | CC backend now supports ".." and s//e. |
| 38 | Xref backend generates cross-reference reports |
| 39 | Cleanups to fix benign but irritating "-w" warnings |
| 40 | Minor cxstack fix |
| 41 | New since alpha1 |
| 42 | Working CC backend |
| 43 | Shared globs and pre-initialised hash support |
| 44 | Some XSUB support |
| 45 | Assorted bug fixes |
| 46 | |
| 47 | INSTALLATION |
| 48 | |
| 49 | (1) You need perl5.002 or later. |
| 50 | |
| 51 | (2) If you want to compile and run programs with the C or CC backends |
| 52 | which undefine (or redefine) subroutines, then you need to apply a |
| 53 | one-line patch to perl itself. One or two of the programs in perl's |
| 54 | own test suite do this. The patch is in file op.patch. It prevents |
| 55 | perl from calling free() on OPs with the magic sequence number (U16)-1. |
| 56 | The compiler declares all OPs as static structures and uses that magic |
| 57 | sequence number. |
| 58 | |
| 59 | (3) Type |
| 60 | perl Makefile.PL |
| 61 | to write a personalised Makefile for your system. If you want the |
| 62 | bytecode modules to support reading bytecode from strings (instead of |
| 63 | just from files) then add the option |
| 64 | -DINDIRECT_BGET_MACROS |
| 65 | into the middle of the definition of the CCCMD macro in the Makefile. |
| 66 | Your C compiler may need to be able to cope with Standard C for this. |
| 67 | I haven't tested this option yet with an old pre-Standard compiler. |
| 68 | |
| 69 | (4) If your platform supports dynamic loading then just type |
| 70 | make |
| 71 | and you can then use |
| 72 | perl -Iblib/arch -MO=foo bar |
| 73 | to use the compiler modules (see later for details). |
| 74 | If you need/want instead to make a statically linked perl which |
| 75 | contains the appropriate modules, then type |
| 76 | make perl |
| 77 | make byteperl |
| 78 | and you can then use |
| 79 | ./perl -MO=foo bar |
| 80 | to use the compiler modules. |
| 81 | In both cases, the byteperl executable is required for running standalone |
| 82 | bytecode programs. It is *not* a standard perl+XSUB perl executable. |
| 83 | |
| 84 | USAGE |
| 85 | |
| 86 | As of the alpha3 release, the Bytecode, C and CC backends are now all |
| 87 | functional enough to compile almost the whole of the main perl test |
| 88 | suite. In the case of the CC backend, any failures are all due to |
| 89 | differences and/or known bugs documented below. See the file TESTS. |
| 90 | In the following examples, you'll need to replace "perl" by |
| 91 | perl -Iblib/arch |
| 92 | if you have built the extensions for a dynamic loading platform but |
| 93 | haven't installed the extensions completely. You'll need to replace |
| 94 | "perl" by |
| 95 | ./perl |
| 96 | if you have built the extensions into a statically linked perl binary. |
| 97 | |
| 98 | (1) To compile perl program foo.pl with the C backend, do |
| 99 | perl -MO=C,-ofoo.c foo.pl |
| 100 | Then use the cc_harness perl program to compile the resulting C source: |
| 101 | perl cc_harness -O2 -o foo foo.c |
| 102 | |
| 103 | If you are using a non-ANSI pre-Standard C compiler that can't handle |
| 104 | pre-declaring static arrays, then add -DBROKEN_STATIC_REDECL to the |
| 105 | options you use: |
| 106 | perl cc_harness -O2 -o foo -DBROKEN_STATIC_REDECL foo.c |
| 107 | If you are using a non-ANSI pre-Standard C compiler that can't handle |
| 108 | static initialisation of structures with union members then add |
| 109 | -DBROKEN_UNION_INIT to the options you use. If you want command line |
| 110 | arguments passed to your executable to be interpreted by perl (e.g. -Dx) |
| 111 | then compile foo.c with -DALLOW_PERL_OPTIONS. Otherwise, all command line |
| 112 | arguments passed to foo will appear directly in @ARGV. The resulting |
| 113 | executable foo is the compiled version of foo.pl. See the file NOTES for |
| 114 | extra options you can pass to -MO=C. |
| 115 | |
| 116 | There are some constraints on the contents on foo.pl if you want to be |
| 117 | able to compile it successfully. Some problems can be fixed fairly easily |
| 118 | by altering foo.pl; some problems with the compiler are known to be |
| 119 | straightforward to solve and I'll do so soon. The file Todo lists a |
| 120 | number of known problems. See the XSUB section lower down for information |
| 121 | about compiling programs which use XSUBs. |
| 122 | |
| 123 | (2) To compile foo.pl with the CC backend (which generates actual |
| 124 | optimised C code for the execution path of your perl program), use |
| 125 | perl -MO=CC,-ofoo.c foo.pl |
| 126 | |
| 127 | and proceed just as with the C backend. You should almost certainly |
| 128 | use an option such as -O2 with the subsequent cc_harness invocation |
| 129 | so that your C compiler uses optimisation. The C code generated by |
| 130 | the Perl compiler's CC backend looks ugly to humans but is easily |
| 131 | optimised by C compilers. |
| 132 | |
| 133 | To make the most of this compiler backend, you need to tell the |
| 134 | compiler when you're using int or double variables so that it can |
| 135 | optimise appropriately (although this part of the compiler is the most |
| 136 | buggy). You currently do that by naming lexical variables ending in |
| 137 | "_i" for ints, "_d" for doubles, "_ir" for int "register" variables or |
| 138 | "_dr" for double "register" variables. Here "register" is a promise |
| 139 | that you won't pass a reference to the variable into a sub which then |
| 140 | modifies the variable. The compiler ought to catch attempts to use |
| 141 | "\$i" just as C compilers catch attempts to do "&i" for a register int |
| 142 | i but it doesn't at the moment. Bugs in the CC backend may make your |
| 143 | program fail in mysterious ways and give wrong answers rather than just |
| 144 | crash in boring ways. But, hey, this is an alpha release so you knew |
| 145 | that anyway. See the XSUB section lower down for information about |
| 146 | compiling programs which use XSUBs. |
| 147 | |
| 148 | If your program uses classes which define methods (or other subs which |
| 149 | are not exported and not apparently used until runtime) then you'll |
| 150 | need to use -u compile-time options (see the NOTES file) to force the |
| 151 | subs to be compiled. Future releases will probably default the other |
| 152 | way, do more auto-detection and provide more fine-grained control. |
| 153 | |
| 154 | Since compiled executables need linking with libperl, you may want |
| 155 | to turn libperl.a into a shared library if your platform supports |
| 156 | it. For example, with Digital UNIX, do something like |
| 157 | ld -shared -o libperl.so -all libperl.a -none -lc |
| 158 | and with Linux/ELF, rebuild the perl .c files with -fPIC (and I |
| 159 | also suggest -fomit-frame-pointer for Linux on Intel architetcures), |
| 160 | do "make libperl.a" and then do |
| 161 | gcc -shared -Wl,-soname,libperl.so.5 -o libperl.so.5.3 `ar t libperl.a` |
| 162 | and then |
| 163 | # cp libperl.so.5.3 /usr/lib |
| 164 | # cd /usr/lib |
| 165 | # ln -s libperl.so.5.3 libperl.so.5 |
| 166 | # ln -s libperl.so.5 libperl.so |
| 167 | # ldconfig |
| 168 | When you compile perl executables with cc_harness, append -L/usr/lib |
| 169 | otherwise the -L for the perl source directory will override it. For |
| 170 | example, |
| 171 | perl -Iblib/arch -MO=CC,-O2,-ofoo3.c foo3.bench |
| 172 | perl cc_harness -o foo3 -O2 foo3.c -L/usr/lib |
| 173 | ls -l foo3 |
| 174 | -rwxr-xr-x 1 mbeattie xzdg 11218 Jul 1 15:28 foo3 |
| 175 | You'll probably also want to link your main perl executable against |
| 176 | libperl.so; it's nice having an 11K perl executable. |
| 177 | |
| 178 | (3) To compile foo.pl into bytecode do |
| 179 | perl -MO=Bytecode,-ofoo foo.pl |
| 180 | To run the resulting bytecode file foo as a standalone program, you |
| 181 | use the program byteperl which should have been built along with the |
| 182 | extensions. |
| 183 | ./byteperl foo |
| 184 | Any extra arguments are passed in as @ARGV; they are not interpreted |
| 185 | as perl options. If you want to load chunks of bytecode into an already |
| 186 | running perl program then use the -m option and investigate the |
| 187 | byteload_fh and byteload_string functions exported by the B module. |
| 188 | See the NOTES file for details of these and other options (including |
| 189 | optimisation options and ways of getting at the intermediate "assembler" |
| 190 | code that the Bytecode backend uses). |
| 191 | |
| 192 | (3) There are little Bourne shell scripts and perl programs to aid with |
| 193 | some common operations: assemble, disassemble, run_bytecode_test, |
| 194 | run_test, cc_harness, test_harness, test_harness_bytecode. |
| 195 | |
| 196 | (4) Walk the op tree in execution order printing terse info about each op |
| 197 | perl -MO=Terse,exec foo.pl |
| 198 | |
| 199 | (5) Walk the op tree in syntax order printing lengthier debug info about |
| 200 | each op. You can also append ",exec" to walk in execution order, but the |
| 201 | formatting is designed to look nice with Terse rather than Debug. |
| 202 | perl -MO=Debug foo.pl |
| 203 | |
| 204 | (6) Produce a cross-reference report of the line numbers at which all |
| 205 | variables, subs and formats are defined and used. |
| 206 | perl -MO=Xref foo.pl |
| 207 | |
| 208 | XSUBS |
| 209 | |
| 210 | The C and CC backends can successfully compile some perl programs which |
| 211 | make use of XSUB extensions. [I'll add more detail to this section in a |
| 212 | later release.] As a prerequisite, such extensions must not need to do |
| 213 | anything in their BOOT: section which needs to be done at runtime rather |
| 214 | than compile time. Normally, the only code in the boot_Foo() function is |
| 215 | a list of newXS() calls which xsubpp puts there and the compiler handles |
| 216 | saving those XS subs itself. For each XSUB used, the C and CC compiler |
| 217 | will generate an initialiser in their C output which refers to the name |
| 218 | of the relevant C function (XS_Foo_somesub). What is not yet automated |
| 219 | is the necessary commands and cc command-line options (e.g. via |
| 220 | "perl cc_harness") which link against the extension libraries. For now, |
| 221 | you need the XSUB extension to have installed files in the right format |
| 222 | for using as C libraries (e.g. Foo.a or Foo.so). As the Foo.so files (or |
| 223 | your platform's version) aren't suitable for linking against, you will |
| 224 | have to reget the extension source and rebuild it as a static extension |
| 225 | to force the generation of a suitable Foo.a file. Then you need to make |
| 226 | a symlink (or copy or rename) of that file into a libFoo.a suitable for |
| 227 | cc linking. Then add the appropriate -L and -l options to your |
| 228 | "perl cc_harness" command line to find and link against those libraries. |
| 229 | You may also need to fix up some platform-dependent environment variable |
| 230 | to ensure that linked-against .so files are found at runtime too. |
| 231 | |
| 232 | DIFFERENCES |
| 233 | |
| 234 | The result of running a compiled Perl program can sometimes be different |
| 235 | from running the same program with standard perl. Think of the compiler |
| 236 | as having a slightly different implementation of the language Perl. |
| 237 | Unfortunately, since Perl has had a single implementation until now, |
| 238 | there are no formal standards or documents defining what behaviour is |
| 239 | guaranteed of Perl the language and what just "happens to work". |
| 240 | Some of the differences below are almost impossible to change because of |
| 241 | the way the compiler works. Others can be changed to produce "standard" |
| 242 | perl behaviour if it's deemed proper and the resulting performance hit |
| 243 | is accepted. I'll use "standard perl" to mean the result of running a |
| 244 | Perl program using the perl executable from the perl distribution. |
| 245 | I'll use "compiled Perl program" to mean running an executable produced |
| 246 | by this compiler kit ("the compiler") with the CC backend. |
| 247 | |
| 248 | Loops |
| 249 | Standard perl calculates the target of "next", "last", and "redo" |
| 250 | at run-time. The compiler calculates the targets at compile-time. |
| 251 | For example, the program |
| 252 | |
| 253 | sub skip_on_odd { next NUMBER if $_[0] % 2 } |
| 254 | NUMBER: for ($i = 0; $i < 5; $i++) { |
| 255 | skip_on_odd($i); |
| 256 | print $i; |
| 257 | } |
| 258 | |
| 259 | produces the output |
| 260 | 024 |
| 261 | with standard perl but gives a compile-time error with the compiler. |
| 262 | |
| 263 | Context of ".." |
| 264 | The context (scalar or array) of the ".." operator determines whether |
| 265 | it behaves as a range or a flip/flop. Standard perl delays until |
| 266 | runtime the decision of which context it is in but the compiler needs |
| 267 | to know the context at compile-time. For example, |
| 268 | @a = (4,6,1,0,0,1); |
| 269 | sub range { (shift @a)..(shift @a) } |
| 270 | print range(); |
| 271 | while (@a) { print scalar(range()) } |
| 272 | generates the output |
| 273 | 456123E0 |
| 274 | with standard Perl but gives a compile-time error with compiled Perl. |
| 275 | |
| 276 | Arithmetic |
| 277 | Compiled Perl programs use native C arithemtic much more frequently |
| 278 | than standard perl. Operations on large numbers or on boundary |
| 279 | cases may produce different behaviour. |
| 280 | |
| 281 | Deprecated features |
| 282 | Features of standard perl such as $[ which have been deprecated |
| 283 | in standard perl since version 5 was released have not been |
| 284 | implemented in the compiler. |
| 285 | |
| 286 | Others |
| 287 | I'll add to this list as I remember what they are. |
| 288 | |
| 289 | BUGS |
| 290 | |
| 291 | Here are some things which may cause the compiler problems. |
| 292 | |
| 293 | The following render the compiler useless (without serious hacking): |
| 294 | * Use of the DATA filehandle (via __END__ or __DATA__ tokens) |
| 295 | * Operator overloading with %OVERLOAD |
| 296 | * The (deprecated) magic array-offset variable $[ does not work |
| 297 | * The following operators are not yet implemented for CC |
| 298 | goto |
| 299 | sort with a non-default comparison (i.e. a named sub or inline block) |
| 300 | * You can't use "last" to exit from a non-loop block. |
| 301 | |
| 302 | The following may give significant problems: |
| 303 | * BEGIN blocks containing complex initialisation code |
| 304 | * Code which is only ever referred to at runtime (e.g. via eval "..." or |
| 305 | via method calls): see the -u option for the C and CC backends. |
| 306 | * Run-time lookups of lexical variables in "outside" closures |
| 307 | |
| 308 | The following may cause problems (not thoroughly tested): |
| 309 | * Dependencies on whether values of some "magic" Perl variables are |
| 310 | determined at compile-time or runtime. |
| 311 | * For the C and CC backends: compile-time strings which are longer than |
| 312 | your C compiler can cope with in a single line or definition. |
| 313 | * Reliance on intimate details of global destruction |
| 314 | * For the Bytecode backend: high -On optimisation numbers with code |
| 315 | that has complex flow of control. |
| 316 | * Any "-w" option in the first line of your perl program is seen and |
| 317 | acted on by perl itself before the compiler starts. The compiler |
| 318 | itself then runs with warnings turned on. This may cause perl to |
| 319 | print out warnings about the compiler itself since I haven't tested |
| 320 | it thoroughly with warnings turned on. |
| 321 | |
| 322 | There is a terser but more complete list in the Todo file. |
| 323 | |
| 324 | Malcolm Beattie |
| 325 | 2 September 1996 |