Current state of the patches here is with respect to perl5.002b1d ;-). ======================================================== The OS/2 patchkit was submitted by ilya@math.ohio-state.edu. I have applied some parts that I suspect won't cause any problems. Others do things that I haven't had time to fully consider. Still other patches included here should perhaps be integrated with the metaconfig package that generates Configure. Andy Dougherty ======================================================== Notes on the patch: ~~~~~~~~~~~~~~~~~~~ patches should be applied as patch -p0 <..... All the diff.* files and POSIX.mkfifo should be applied. Additional files are available on ftp://ftp.math.ohio-state.edu/pub/users/ilya/os2 including patched pdksh and gnumake, needed for build. Target: ~~~~~~~ This is not supposed to make a perfect Perl on OS/2. This patch is concerned only with perfect _build_ of Perl on OS/2. Some good features from Andreas Kaiser port missed this port. However, most of the features are available in different form. !!! Note that [gs]etpriority functions in this port are compatible !!! with *nix, not with ak's port!!! The priorities are absolute, go from 32 to -95, lower is quickier. 0 is default, Notes on build on OS/2: ~~~~~~~~~~~~~~~~~~~~~~~ The change of C code in this patch is based on the ak port of 5.001+. a) Make sure your sort is not the broken OS/2 one, and that you have /tmp on the build partition. b) when extracting perl5.*.tar.gz you need to extract perl5.*/Configure separately, since by default perl5.001m/configure may overwrite it; like this: tar vzxf perl5.004.tar.gz --case-sensitive perl5.004/Configure c) Necessary manual intervention when compiling on OS/2: Need to put perl.dll on LIBPATH after it is created. d) Compile summary: ~~~~~~~~~~~~~~~ !!! At the end of this README is independent description of the build !!! process by Rocco Caputo. # Look for hints/os2.sh and correct what is different on your system # I have rather spartan configuration. # Prefix means where to install: sh Configure -des -D prefix=f:/perl5.005 # Ignore the message about missing `ln', and about `c' option # to tr. make # Will probably die after build of miniperl (unless you have DLL # from previous compile). Need to move DLL where it belongs # # Somehow with 5.002b3 I needed to type another make after pod2man make # some warnings in POSIX.c make test # some tests fail, 9 or 10 on my system (see the list at end). # # before this you should create subdirs bin and lib in the # prefix directory (f:/perl5.005 above): # # To run finer tests, cd t && perl harness make install e) At the end of August GNU make and pdksh were too buggy for compile. Both maintainers have patches that make it possible to compile perl. The binaries are included in ftp://ftp.math.ohio-state.edu/pub/users/ilya/os2 patches are available too. Note that the pdksh5.2.4 broke builds with -Zexe option because of a changed order of executable extensions. A patch is sent to maintainer. The version 5.2.5alpha was OK for the build, !!!!!!!!!!!!!!!!! If you see that some '/' became '\' in pdksh 5.2.3, you did not apply my patches! Same with segfaults in Make 3.74. !!!!!!!!!!!!!!!!! Problems reported: a) one of the latest tr is broken, get an old one :-( 1.11 works. (On compuserver?) b) You need a link386. c) Get rid of invalid perl.dll on your LIBPATH. Note the EMX does not support en_us locale (most nobody does ;-). Some TCP/IP update could have installed it to your config.sys. You need to delete it until EMX is updated to support this newest discovery by IBM. Send comments to ilya@math.ohio-state.edu. ====================================================== Requires 0.9b (well, provision are made to make it build under 0.9a6, but they are not tested, please inform me on success). (earlier than 0.9b ttyname was not present, it is hard to maintain this difference automatically, though I try). ====================================================== Building with a.out style is supported by the `perl_' target of make. Dynamic extensions are not possible with perl_.exe, since boot code should return the retvalue on stack, the address of which is not known to the extension. The reason why compiling with a.out style executables leads to problems with dynamic extensions is: a) OS/2 does not export symbols from executables; b) Thus if extension needs to import symbols from an application the symbols for the application should reside in a .dll. c) You cannot export data from a .dll compiled with a.out style. On the other hand, aout-style compiled extension enjoys all the (dis)advantages of fork(). Check A.OUT compile with the following make targets: aout_test aout_install aout_clean ====================================================== Tests which fail with OMF compile: io/fs.t: 2-5, 7-11, 18 as they should. io/pipe: all, since open("|-") is not working (works with perl_.exe). lib/"all the dbm".t: 1 test should fail (file permission). op/fork all fail, as they should (except with perl_.exe) op/stat 3 20 35 as they should, 39 (-t on /dev/null) ???? Sometimes 4 - timing problem ???? Sometimes I have seen segfault in socket ????, only if run with Testing tools. A lot of `bad free'... in databases, bug in DB confirmed on other platforms. Fail: Total 30 subtests (if stat:4 fails) in 10 scripts (one of 10 is socket, which runs OK standalone). With newer configs I could not reproduce most the crashes. ======================================================= Changes to calls to external programs: ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Due to a popular demand the perl external program calling has been changed. _If_ perl needs to call an external program _via shell_, the X:/bin/sh.exe will be called. The name of the shell is not overridable, except the drive letter. Thus means that you need to pickup some copy of a sh.exe as well (I use one from pdksh). The drive X: above is set up automatically during the build, is settable in runtime from $ENV{PERL_SH_DRIVE}. Reasons: a consensus on perl5-porters was that perl should use one non-overridable shell per platform. The obvious choices for OS/2 are cmd.exe and sh.exe. Having perl build itself would be impossible with cmd.exe as a shell, thus I picked up sh.exe. Thus assures almost 100% compatibility with the scripts coming from *nix. Disadvantages: sh.exe calls external programs via fork/exec, and there is _no_ functioning exec on OS/2. exec is emulated by EMX by asyncroneous call while the caller waits for child completion (to pretend that pid did not change). This means that 1 _extra_ copy of sh.exe is made active via fork/exec, which may lead to some resources taken from the system. The long-term solution proposed on p5-p is to have a directive use OS2::Cmd; which will override system(), exec(), ``, and open(,' |'). With current perl you may override only system(), readpipe() - the explicit version of ``, and maybe exec(). The code will substitute a one-argument system by CORE::system('cmd.exe', '/c', shift). If you have some working code for OS2::Cmd.pm, please send it to me, I will include it into distribution. I have no need for such a module, so cannot test it. =================================================== OS/2 extensions ~~~~~~~~~~~~~~~ Since binaries cannot go into perl distribution, no extensions are included. They are available in .../os2/ilyaz directory of CPAN, as well as in my directory ftp://ftp.math.ohio-state.edu/pub/users/ilya/os2 I include 3 extensions by Andread Kaiser, OS2::REXX, OS2::UPM, and OS2::FTP, into my ftp directory, mirrored on CPAN. I made some minor changes needed to compile them by standard tools. I cannot test UPM and FTP, so I will appreciate your feedback. Other extensions there are OS2::ExtAttribs, OS2::PrfDB for tied access to EAs and .INI files - and maybe some other extensions at the time you read it. Note that OS2 perl defines 2 pseudo-extension functions OS2::Copy::copy and DynaLoader::mod2fname. The -R switch of older perl is deprecated. If you need to call a REXX code which needs access to variables, include the call into a REXX compartment created by REXX_call {...block...}; Two new functions are supported by REXX code, REXX_eval 'string'; REXX_eval_with 'string', REXX_function_name => \&perl_sub_reference; If you have some other extensions you want to share, send the code to me. Two jump to mind: tied access to EA's, and tied access to system databases. ================================================================== == == == User report [my comments in brackets, IZ] == == == ================================================================== Starting in x:/usr/src, using 4OS2/32 2.5 as the command interpreter on OS/2 2.30 with FixPak-17. DAX is installed, but this shouldn't be a factor. Drive X is a TVFS virtual drive pointing to several physical HPFS drives. >>> Make sure that no copies or perl are currently running. Miniperl may fail during the build because it will find an older version of perl.dll loaded in memory. Close any running perl scripts. Shut down anything that might run perl scripts, like cron. `emxload -l` to check for loaded versions of perl. `emxload -u perl.exe` to unload them. >>> Pre-load some common utilities: emxload -e sh.exe make.exe ls.exe tr.exe id.exe sed.exe SET GCCLOAD=30 (number of minutes to hold the compiler) [grep egrep fgrep cat rm uniq basename uniq sort - are not bad too.] The theory is that it's faster to demand-load the development tools from virtual memory than it is to re-load and re-link them all the time. This is definitely true with my system because swapfile.dat is on a faster drive than my development environment. ls, tr, and id represent the GNU file, text, and shell utilities. These may not be needed, but it makes sure that their respective DLLs are in memory. >>> Unpack the perl 5_002_01 archive onto an HPFS partition. tar vxzf perl5_002_01.tar-gz cd perl5.002_01 [Do not forget to extract Configure as described above.] >>> Read the README, keeping a copy open in another session for reference. start /c /fg less os2/README >>> Apply the OS/2 patches included with 5.002_01, as per the README. for %m in (os2\diff.*) patch -p0 < %m patch -p0 < os2\POSIX.mkfifo [The patch below is already applied.] >>> You may need to apply this patch if you plan to run a non-standard Configure (that is, if you defy the README). This patch will ensure that Makefile inherits the libraries specified during Configure. People running standard perl builds can probably ignore this patch. *** os2\Makefile.SHs Mon Mar 25 02:05:00 1996 --- os2\Makefile.SHs.new Fri May 24 10:37:10 1996 *************** *** 9,15 **** emximp -o perl.imp perl5.def perl.dll: $(obj) perl5.def perl$(OBJ_EXT) ! $(LD) $(LDDLFLAGS) -o $@ perl$(OBJ_EXT) $(obj) -lsocket perl5.def perl5.def: perl.linkexp echo "LIBRARY 'Perl' INITINSTANCE TERMINSTANCE" > $@ --- 9,15 ---- emximp -o perl.imp perl5.def perl.dll: $(obj) perl5.def perl$(OBJ_EXT) ! $(LD) $(LDDLFLAGS) -o $@ perl$(OBJ_EXT) $(obj) $(libs) perl5.def perl5.def: perl.linkexp echo "LIBRARY 'Perl' INITINSTANCE TERMINSTANCE" > $@ *************** *** 49,55 **** cat perl.exports perl.map | sort | uniq -d | sed -e 's/\w\+/ "\0"/' > perl.linkexp perl.map: $(obj) perl$(OBJ_EXT) miniperlmain$(OBJ_EXT) ! $(CC) $(LARGE) $(CLDFLAGS) $(CCDLFLAGS) -o dummy.exe miniperlmain$(OBJ_EXT) perl$(OBJ_EXT) $(obj) -lsocket -lm -Zmap -Zlinker /map awk '{if ($$3 == "") print $$2}' perl.map rm dummy.exe dummy.map --- 49,55 ---- cat perl.exports perl.map | sort | uniq -d | sed -e 's/\w\+/ "\0"/' > perl.linkexp perl.map: $(obj) perl$(OBJ_EXT) miniperlmain$(OBJ_EXT) ! $(CC) $(LARGE) $(CLDFLAGS) $(CCDLFLAGS) -o dummy.exe miniperlmain$(OBJ_EXT) perl$(OBJ_EXT) $(obj) $(libs) -Zmap -Zlinker /map awk '{if ($$3 == "") print $$2}' perl.map rm dummy.exe dummy.map >>> Apply the patches from Ilya's perl5.002_01 binary distribution: touch os2/dlfcn.h os2/dl_os2.c patch -p1 < f:\perllib\README.fix1 >>> Run Configure. Most people can run it by following the README: sh Configure -des -D prefix=f:/usr/local Advanced perl users (experienced C programmers, recommended) can run the interactive Configure and answer the questions. When in doubt about an answer, check the EMX headers and documentation. Pick the default answer if that doesn't help: sh Configure [Yet more advanced users just specify the answers on the command line of Configure, like I did with prefix.] Note: You may need to wrap an answer in quotes if it contains spaces. For example, "-lsocket -lm". Note: If you want to add some options to a long default, you can use $* to include the default in your answer: "$* -DDEBUGGING". Configure warnings and errors, and possible work-arounds: I don't know where 'ln' is.... (ignored; OS/2 doesn't have a ln command) nm didn't seem to work right. Trying emxomfar instead... (nothing to worry about) The recommended value for $d_shrplib on this machine was "define"! (kept the recommended value: y) Directory f:/usr/lib/perl5/os2/5.00201/CORE doesn't exist. (created the directory from another window with \usr\bin\mkdir -p f:/usr/lib/perl5/os2/5.00201/CORE and then answered: y. Your directory may look different.) [Ignore this as well, install script will create it for you.] The recommended value for $i_dlfcn on this machine was "define"! (kept the recommended value: y) The recommended value for $d_fork on this machine was "undef"! (kept the recommended value: y) Figuring out the flag used by open() for non-blocking I/O... Seems like we can use O_NONBLOCK. This seems to be used for informative purposes only. The errors that follow this (including a SIGPIPE) don't seem to affect perl at all. These were safely ignored. What pager is used on your system? [/usr/ucb/more] Had to answer "/usr/bin/less.exe" because Configure wants a leading / (unix full path). Need to edit config.sh later with the real full path to the pager, including the drive letter. [Apparently this setting is never used, so it is safe to ignore it.] Hmm... F:/USR/BIN/sed: Unterminated `s' command Perl built fine even with this error, so it seems safe to ignore. Things I did different from the defaults. Most (if not all) of these are optional changes. They're listed here to show how good Configure is at detecting the system setup. [I add the options to put it on command line of Configure, see below.] Selected 'none' for the man1 location. (I prefer the pod2html version.) [-D man1dir=none] Selected 'none' for the man3 location. (I prefer the pod2html version.) [-D man3dir=none] Changed the hostname and domain. (I wanted to override a dynamic PPP address. This only matters if other people will be using your perl build.) [-D myhostname=my_host_name -D mydomain=.foo.org] Fixed the e-mail address. (Put in a known working e-mail address. This only matters if other people will be using your perl build.) [-D cf_email=root@myhostname.uucp] Added some directories to the library search path. [-D "libpth=f:/emx/lib/st f:/emx/lib"] Added -g to the optimizer/debugger flags. [-D optimize=-g] Added "-lgdbm -ldb -lcrypt -lbsd" to the additional libraries. [ -D "libs=-lsocket -lcrypt -lgdbm" the rest of libraries will not be used] >>> Advanced users may want to edit config.sh when prompted by Configure. Most (all?) of these changes aren't really necessary: d_getprior='define' d_setprior='define' (getpriority and setpriority are included in os2.c, but Configure doesn't know to look there.) [fixed already] pager='f:/usr/bin/less.exe' (Correcting Configure's insistence on a leading slash.) bin_sh='f:/usr/bin/sh.exe' (If Configure detects sh.exe somewhere else first. Example: it saw sh.exe at /bin/sh.exe on my TVFS drive, but I want perl to look for it on the physical F drive.) aout_ccflags='... existing flags... -DDEBUGGING' aout_cppflags='... existing flags... -DDEBUGGING' (If you want to include DEBUGGING for the aout version.) [Do not do it, -D optimize=-g will automatically add these flags.] >>> Allow Configure to make the build scripts. >>> Allow Configure to run `make depend`. Ignore the following warning: perl.h:861: warning: `DEBUGGING_MSTATS' redefined [corrected now] >>> Rename any existing perl.dll, preventing anything from loading it and saving a known working copy in case something goes wrong: mv /usr/lib/perl.dll /usr/lib/ilya-perl.dll >>> Run `make`, and ignore the following warnings: perl.h:861: warning: `DEBUGGING_MSTATS' redefined [corrected now] invalid preprocessing directive name emxomf warning: Cycle detected by make_type LINK386 : warning L4071: application type not specified; assuming WINDOWCOMPAT Warning (will try anyway): No library found for -lposix Warning (will try anyway): No library found for -lcposix POSIX.c:203: warning: `mkfifo' redefined POSIX.c:4603: warning: assignment makes pointer from integer without a cast >>> If `make` dies while "Making DynaLoader (static)", you'll need to put miniperl in the OS/2 paths. This step is only necessary if `make` can't find miniperl: [I would be interested if somebody confirmes this.] cp perl.dll /usr/lib (where /usr/lib is in your LIBPATH) cp miniperl.exe /usr/bin (where /usr/bin is in your PATH) make (ignore the errors in the previous step) This should run to completion. >>> Test the build: make test These tests fail: io/fs..........FAILED on test 2 "OS/2 is not unix". Test 2 checks the link() command, which is not supported by OS/2. io/pipe........f:/usr/bin/sh.exe: -c requires an argument f:/usr/bin/sh.exe: -c requires an argument The Unsupported function fork function is unimplemented at io/pipe.t line 26. FAILED on test 1 More "OS/2 is not unix" errors. Read ahead to find out why fork() fails. op/exec........FAILED on test 4 if (system "true") {print "not ok 4\n";} else \ {print "ok 4\n";} This fails for me, but changing it to read like this works: if (system '\usr\bin\true.cmd') {print "not ok 4\n";} \ else {print "ok 4\n";} So you can count this as another "OS/2 is not unix". op/fork........The Unsupported function fork function is \ unimplemented at op/fork.t line 8. FAILED on test 1 The dynamically-loaded version of perl currently doesn't support fork(). This is a known behavior of EMX. op/magic....... Process terminated by SIGINT ok The test passed even with the SIGINT message. I don't know why, but I won't argue. op/stat........ls: /dev: No such file or directory f:/usr/bin/sh.exe: ln: not found ls: perl: No such file or directory FAILED on test 3 "OS/2 is not unix". We don't have the ln command. lib/anydbm.....Bad free() ignored at lib/anydbm.t line 51. Bad free() ignored at lib/anydbm.t line 51. Bad free() ignored at lib/anydbm.t line 51. Bad free() ignored during global destruction. Bad free() ignored during global destruction. Bad free() ignored during global destruction. FAILED on test 2 Test 2 looks at the file permissions for a database. "OS/2 is not unix" so the permissions aren't exactly what this test expects. lib/db-btree...Bad free() ignored at lib/db-btree.t line 109. Bad free() ignored at lib/db-btree.t line 221. Bad free() ignored at lib/db-btree.t line 337. Bad free() ignored at lib/db-btree.t line 349. Bad free() ignored at lib/db-btree.t line 349. Bad free() ignored at lib/db-btree.t line 399. Bad free() ignored at lib/db-btree.t line 400. Bad free() ignored at lib/db-btree.t line 401. FAILED on test 20 Another file permissions test fails. lib/db-hash....Bad free() ignored at lib/db-hash.t line 101. Bad free() ignored at lib/db-hash.t line 101. Bad free() ignored at lib/db-hash.t line 101. Bad free() ignored at lib/db-hash.t line 239. Bad free() ignored at lib/db-hash.t line 239. Bad free() ignored at lib/db-hash.t line 239. Bad free() ignored at lib/db-hash.t line 253. Bad free() ignored at lib/db-hash.t line 253. Bad free() ignored at lib/db-hash.t line 253. FAILED on test 16 Another file permissions test fails. lib/db-recno...Bad free() ignored at lib/db-recno.t line 138. Bad free() ignored at lib/db-recno.t line 138. FAILED on test 18 Another file permissions test fails. lib/gdbm.......FAILED on test 2 Another file permissions test fails. lib/sdbm.......FAILED on test 2 Another file permissions test fails. Failed 11/94 tests, 88.30% okay. All of which are known differences with unix or documented behaviors in EMX. I re-run the test with Ilya's version, and the same tests fail. This new build is a success. [Note that bad free() mentioned above are bugs in the Berkeley DB. They just are more visible under OS/2 with perl free(), because of "rigid" function name resolution. To get finer tests, cd to ./t and run perl harness ] (Actually, Ilya's perl release fails an extra test because I don't have sed in f:\emx.add. This shows how important it is to configure and build perl yourself instead of grabbing pre-built binaries.) [Hmm, should not happen... There is no mentions of full_sed under ./t directory...] >>> Cross your fingers and install it: make install Warnings encountered and workarounds presented.: WARNING: You've never run 'make test'!!! (Installing anyway.) (Lies! All lies! At least it still installs.) WARNING: Can't find libperl*.dll* to install into \ f:/usr/lib/perl5/os2/5.00201/CORE. (Installing other things anyway.) (Safe to ignore. The important one, libperl.lib, gets copied.) Couldn't copy f:/usr/bin/perl5.00201.exe to f:/usr/bin/perl.exe: \ No such file or directory cp /usr/bin/perl5.00201.exe /usr/bin/perl.exe Couldn't copy f:/usr/bin/perl.exe to /usr/bin/perl.exe: No such \ file or directory (I think this one is safe to ignore since the two directories point to the same place.) >>> Laugh maniacally because you just built and installed your own copy of perl, with all the paths set "just so" and with whatever little psychotic modifications you've always wanted but were afraid to add. ----------------------------------------------------------------------------- Development tools and versions: EMX 0.9b with emxfix04 applied. `ls --version` reports: 'GNU file utilities 3.12' `tr --version` reports: 'tr - GNU textutils 1.14' `id --version` reports: 'id - GNU sh-utils 1.12' `sed --version` reports: 'GNU sed version 2.05' `awk --version` reports: 'Gnu Awk (gawk) 2.15, patchlevel 6' `grep --version` reports an illegal option and: 'GNU grep version 2.0' (this includes egrep) `sort --version` reports: 'sort - GNU textutils 1.14' `uniq --version` reports: 'uniq - GNU textutils 1.14' `find --version` reports: 'GNU find version 4.1' KSH_VERSION='@(#)PD KSH v5.2.4 96/01/17' (Ilya's patched version.) `make --version` reports: 'GNU Make version 3.74' (Ilya's patched version.) `emxrev` reports: EMX : revision = 42 EMXIO : revision = 40 EMXLIBC : revision = 40 EMXLIBCM : revision = 43 EMXLIBCS : revision = 43 EMXWRAP : revision = 40 ----------------------------------------------------------------------------- Rocco