perl 5.003_01: README.os2

[perl5.git] / README.os2

Current state of the patches here is with respect to perl5.002b1d ;-).

========================================================

The OS/2 patchkit was submitted by ilya@math.ohio-state.edu.  I have
applied some parts that I suspect won't cause any problems.
Others do things that I haven't had time to fully consider.

Still other patches included here should perhaps be integrated with the
metaconfig package that generates Configure.

	Andy Dougherty		<doughera@lafcol.lafayette.edu>

========================================================

Notes on the patch:
~~~~~~~~~~~~~~~~~~~
patches should be applied as
	patch -p0 <.....
All the diff.* files and POSIX.mkfifo should be applied.

Additional files are available on
	ftp://ftp.math.ohio-state.edu/pub/users/ilya/os2
including patched pdksh and gnumake, needed for build.


Target:
~~~~~~~

This is not supposed to make a perfect Perl on OS/2. This patch is
concerned only with perfect _build_ of Perl on OS/2. Some good
features from Andreas Kaiser port missed this port. However, most of
the features are available in different form. 

!!! Note that [gs]etpriority functions in this port are compatible
!!! with *nix, not with ak's port!!! 

The priorities are absolute, go from 32 to -95, lower is quickier. 0
is default,

Notes on build on OS/2:
~~~~~~~~~~~~~~~~~~~~~~~
The change of C code in this patch is based on the ak port of 5.001+.

a) Make sure your sort is not the broken OS/2 one, and that you have /tmp
on the build partition.

b) when extracting perl5.*.tar.gz you need to extract perl5.*/Configure
separately, since by default perl5.001m/configure may overwrite it;
	like this:
		tar vzxf perl5.004.tar.gz --case-sensitive perl5.004/Configure

c) Necessary manual intervention when compiling on OS/2:

	Need to put perl.dll on LIBPATH after it is created.

d) Compile summary:
   ~~~~~~~~~~~~~~~
!!! At the end of this README is independent description of the build
!!! process by Rocco Caputo.

# Look for hints/os2.sh and correct what is different on your system
# I have rather spartan configuration.

	# Prefix means where to install:
sh Configure -des -D prefix=f:/perl5.005
	# Ignore the message about missing `ln', and about `c' option
	# to tr.
make
	# Will probably die after build of miniperl (unless you have DLL
	# from previous compile). Need to move DLL where it belongs
	#
	# Somehow with 5.002b3 I needed to type another make after pod2man
make
	# some warnings in POSIX.c
make test
	# some tests fail, 9 or 10 on my system (see the list at end).
	#
	# before this you should create subdirs bin and lib in the 
	# prefix directory (f:/perl5.005 above):
	#
	# To run finer tests, cd t && perl harness
make install

e) At the end of August GNU make and pdksh were too buggy for compile.
Both maintainers have patches that make it possible to compile perl.
The binaries are included in
	ftp://ftp.math.ohio-state.edu/pub/users/ilya/os2
patches are available too.
Note that the pdksh5.2.4 broke builds with -Zexe option because of a
changed order of executable extensions. A patch is sent to
maintainer. The version 5.2.5alpha was OK for the build,

!!!!!!!!!!!!!!!!!
If you see that some '/' became '\' in pdksh 5.2.3, you did not apply
my patches!
Same with segfaults in Make 3.74.
!!!!!!!!!!!!!!!!!

Problems reported: 

a) one of the latest tr is broken, get an old one :-(
	1.11 works. (On compuserver?)
b) You need a link386.
c) Get rid of invalid perl.dll on your LIBPATH.

Note the EMX does not support en_us locale (most nobody does ;-). Some
TCP/IP update could have installed it to your config.sys. You need to
delete it until EMX is updated to support this newest discovery by IBM.


Send comments to ilya@math.ohio-state.edu.

======================================================
Requires 0.9b (well, provision are made to make it build under 0.9a6,
but they are not tested, please inform me on success).
(earlier than 0.9b ttyname was not present, it is hard to maintain this
difference automatically, though I try).
======================================================

Building with a.out style is supported by the `perl_' target of make.
Dynamic extensions are not possible with perl_.exe, since boot code
should return the retvalue on stack, the address of which is not known
to the extension.

The reason why compiling with a.out style executables leads to problems
with dynamic extensions is:
	a) OS/2 does not export symbols from executables;
	b) Thus if extension needs to import symbols from an application
		the symbols for the application should reside in a .dll.
	c) You cannot export data from a .dll compiled with a.out style.
On the other hand, aout-style compiled extension enjoys all the 
(dis)advantages of fork().

Check A.OUT compile with the following make targets:

	aout_test 
	aout_install
	aout_clean

======================================================
Tests which fail with OMF compile:

io/fs.t: 2-5, 7-11, 18  as they should.
io/pipe: all, since open("|-") is not working (works with perl_.exe).
lib/"all the dbm".t: 1 test should fail (file permission).
op/fork all fail, as they should (except with perl_.exe)
op/stat 3 20 35 as they should, 39 (-t on /dev/null) ???? Sometimes 4
- timing problem ????

Sometimes I have seen segfault in socket ????, only if run with Testing tools.

A lot of `bad free'... in databases, bug in DB confirmed on other
platforms.

Fail: Total 30 subtests (if stat:4 fails) in 10 scripts (one of 10
is socket, which runs OK standalone). With newer configs I could not
reproduce most the crashes.

=======================================================

Changes to calls to external programs:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Due to a popular demand the perl
external program calling has been changed.  _If_ perl needs to call an
external program _via shell_, the X:/bin/sh.exe will be called. The
name of the shell is not overridable, except the drive letter.

Thus means that you need to pickup some copy of a sh.exe as well (I use one
from pdksh). The drive X: above is set up automatically during the
build, is settable in runtime from $ENV{PERL_SH_DRIVE}.

Reasons: a consensus on perl5-porters was that perl should use one 
non-overridable shell per platform. The obvious choices for OS/2 are cmd.exe
and sh.exe. Having perl build itself would be impossible with cmd.exe as
a shell, thus I picked up sh.exe. Thus assures almost 100% compatibility
with the scripts coming from *nix.

Disadvantages: sh.exe calls external programs via fork/exec, and there is
_no_ functioning exec on OS/2. exec is emulated by EMX by asyncroneous call
while the caller waits for child completion (to pretend that pid did
not change). This means that 1 _extra_ copy of sh.exe is made active via
fork/exec, which may lead to some resources taken from the system.

The long-term solution proposed on p5-p is to have a directive
	use OS2::Cmd;
which will override system(), exec(), ``, and open(,'   |'). With current
perl you may override only system(), readpipe() - the explicit version
of ``, and maybe exec(). The code will substitute a one-argument system
by CORE::system('cmd.exe', '/c', shift).

If you have some working code for OS2::Cmd.pm, please send it to me,
I will include it into distribution. I have no need for such a module, so
cannot test it.

===================================================

OS/2 extensions
~~~~~~~~~~~~~~~
Since binaries cannot go into perl distribution, no extensions are
included. They are available in .../os2/ilyaz directory of CPAN, as
well as in my directory
	ftp://ftp.math.ohio-state.edu/pub/users/ilya/os2

I include 3 extensions by Andread Kaiser, OS2::REXX, OS2::UPM, and OS2::FTP, 
into my ftp directory, mirrored on CPAN. I made
some minor changes needed to compile them by standard tools. I cannot 
test UPM and FTP, so I will appreciate your feedback. Other extensions
there are OS2::ExtAttribs, OS2::PrfDB for tied access to EAs and .INI
files - and maybe some other extensions at the time you read it.

Note that OS2 perl defines 2 pseudo-extension functions
OS2::Copy::copy and DynaLoader::mod2fname.

The -R switch of older perl is deprecated. If you need to call a REXX code
which needs access to variables, include the call into a REXX compartment
created by 
	REXX_call {...block...};

Two new functions are supported by REXX code, 
	REXX_eval 'string';
	REXX_eval_with 'string', REXX_function_name => \&perl_sub_reference;

If you have some other extensions you want to share, send the code to me.
Two jump to mind: tied access to EA's, and tied access to system databases.

==================================================================
==								==
==		User report [my comments in brackets, IZ]	==
==								==
==================================================================

Starting in x:/usr/src, using 4OS2/32 2.5 as the command interpreter on
OS/2 2.30 with FixPak-17.  DAX is installed, but this shouldn't be a
factor.  Drive X is a TVFS virtual drive pointing to several physical
HPFS drives.

>>> Make sure that no copies or perl are currently running.  Miniperl
    may fail during the build because it will find an older version
    of perl.dll loaded in memory.

        Close any running perl scripts.
        Shut down anything that might run perl scripts, like cron.
        `emxload -l` to check for loaded versions of perl.
        `emxload -u perl.exe` to unload them.

>>> Pre-load some common utilities:

        emxload -e sh.exe make.exe ls.exe tr.exe id.exe sed.exe
        SET GCCLOAD=30   (number of minutes to hold the compiler)
[grep egrep fgrep cat rm uniq basename uniq sort - are not bad too.]
    The theory is that it's faster to demand-load the development tools
    from virtual memory than it is to re-load and re-link them all the
    time.  This is definitely true with my system because swapfile.dat
    is on a faster drive than my development environment.

    ls, tr, and id represent the GNU file, text, and shell utilities.
    These may not be needed, but it makes sure that their respective
    DLLs are in memory.

>>> Unpack the perl 5_002_01 archive onto an HPFS partition.

        tar vxzf perl5_002_01.tar-gz
        cd perl5.002_01

[Do not forget to extract Configure as described above.]

>>> Read the README, keeping a copy open in another session for reference.

        start /c /fg less os2/README

>>> Apply the OS/2 patches included with 5.002_01, as per the README.

        for %m in (os2\diff.*) patch -p0 < %m
        patch -p0 < os2\POSIX.mkfifo

[The patch below is already applied.]

>>> You may need to apply this patch if you plan to run a non-standard
    Configure (that is, if you defy the README).  This patch will ensure
    that Makefile inherits the libraries specified during Configure.
    People running standard perl builds can probably ignore this patch.

*** os2\Makefile.SHs	Mon Mar 25 02:05:00 1996
--- os2\Makefile.SHs.new	Fri May 24 10:37:10 1996
***************
*** 9,15 ****
  	emximp -o perl.imp perl5.def
  
  perl.dll: $(obj) perl5.def perl$(OBJ_EXT)
! 	$(LD) $(LDDLFLAGS) -o $@ perl$(OBJ_EXT) $(obj) -lsocket perl5.def
  
  perl5.def: perl.linkexp
  	echo "LIBRARY 'Perl' INITINSTANCE TERMINSTANCE"	> $@
--- 9,15 ----
  	emximp -o perl.imp perl5.def
  
  perl.dll: $(obj) perl5.def perl$(OBJ_EXT)
! 	$(LD) $(LDDLFLAGS) -o $@ perl$(OBJ_EXT) $(obj) $(libs) perl5.def
  
  perl5.def: perl.linkexp
  	echo "LIBRARY 'Perl' INITINSTANCE TERMINSTANCE"	> $@
***************
*** 49,55 ****
  	cat perl.exports perl.map | sort | uniq -d | sed -e 's/\w\+/  "\0"/' > perl.linkexp
  
  perl.map: $(obj) perl$(OBJ_EXT) miniperlmain$(OBJ_EXT)
! 	$(CC) $(LARGE) $(CLDFLAGS) $(CCDLFLAGS) -o dummy.exe miniperlmain$(OBJ_EXT) perl$(OBJ_EXT) $(obj) -lsocket -lm -Zmap -Zlinker /map
  	awk '{if ($$3 == "") print $$2}' <dummy.map | sort | uniq > perl.map
  	rm dummy.exe dummy.map
  
--- 49,55 ----
  	cat perl.exports perl.map | sort | uniq -d | sed -e 's/\w\+/  "\0"/' > perl.linkexp
  
  perl.map: $(obj) perl$(OBJ_EXT) miniperlmain$(OBJ_EXT)
! 	$(CC) $(LARGE) $(CLDFLAGS) $(CCDLFLAGS) -o dummy.exe miniperlmain$(OBJ_EXT) perl$(OBJ_EXT) $(obj) $(libs) -Zmap -Zlinker /map
  	awk '{if ($$3 == "") print $$2}' <dummy.map | sort | uniq > perl.map
  	rm dummy.exe dummy.map

>>> Apply the patches from Ilya's perl5.002_01 binary distribution:

        touch os2/dlfcn.h os2/dl_os2.c
        patch -p1 < f:\perllib\README.fix1

>>> Run Configure.  Most people can run it by following the README:

        sh Configure -des -D prefix=f:/usr/local

    Advanced perl users (experienced C programmers, recommended) can run
    the interactive Configure and answer the questions.  When in doubt
    about an answer, check the EMX headers and documentation.  Pick the
    default answer if that doesn't help:

        sh Configure

[Yet more advanced users just specify the answers on the command line
of Configure, like I did with prefix.]

    Note:  You may need to wrap an answer in quotes if it contains
    spaces.  For example, "-lsocket -lm".

    Note:  If you want to add some options to a long default, you can
    use $* to include the default in your answer:  "$* -DDEBUGGING".

    Configure warnings and errors, and possible work-arounds:

        I don't know where 'ln' is....
            (ignored; OS/2 doesn't have a ln command)

        nm didn't seem to work right. Trying emxomfar instead...
            (nothing to worry about)

        The recommended value for $d_shrplib on this machine was "define"!
            (kept the recommended value: y)

        Directory f:/usr/lib/perl5/os2/5.00201/CORE doesn't exist.
            (created the directory from another window with
            \usr\bin\mkdir -p f:/usr/lib/perl5/os2/5.00201/CORE
            and then answered: y.  Your directory may look different.)

[Ignore this as well, install script will create it for you.]

        The recommended value for $i_dlfcn on this machine was "define"!
            (kept the recommended value: y)

        The recommended value for $d_fork on this machine was "undef"!
            (kept the recommended value: y)

        Figuring out the flag used by open() for non-blocking I/O...
        Seems like we can use O_NONBLOCK.
            This seems to be used for informative purposes only.
            The errors that follow this (including a SIGPIPE) don't seem
            to affect perl at all.  These were safely ignored.

        What pager is used on your system? [/usr/ucb/more]
            Had to answer "/usr/bin/less.exe" because Configure wants a
            leading / (unix full path).  Need to edit config.sh later with
            the real full path to the pager, including the drive letter.

[Apparently this setting is never used, so it is safe to ignore it.]

        Hmm... F:/USR/BIN/sed: Unterminated `s' command
            Perl built fine even with this error, so it seems safe to
            ignore.

    Things I did different from the defaults.  Most (if not all) of these
    are optional changes.  They're listed here to show how good Configure
    is at detecting the system setup.

[I add the options to put it on command line of Configure, see below.]

        Selected 'none' for the man1 location.
            (I prefer the pod2html version.)
[-D man1dir=none]
        Selected 'none' for the man3 location.
            (I prefer the pod2html version.)
[-D man3dir=none]
        Changed the hostname and domain.
            (I wanted to override a dynamic PPP address.  This only
            matters if other people will be using your perl build.)
[-D myhostname=my_host_name -D mydomain=.foo.org]
        Fixed the e-mail address.
            (Put in a known working e-mail address.  This only matters
            if other people will be using your perl build.)
[-D cf_email=root@myhostname.uucp]
        Added some directories to the library search path.
[-D "libpth=f:/emx/lib/st f:/emx/lib"]
        Added -g to the optimizer/debugger flags.
[-D optimize=-g]
        Added "-lgdbm -ldb -lcrypt -lbsd" to the additional libraries.
[  -D "libs=-lsocket -lcrypt -lgdbm"
  the rest of libraries will not be used]

>>> Advanced users may want to edit config.sh when prompted by Configure.
    Most (all?) of these changes aren't really necessary:

        d_getprior='define'
        d_setprior='define'
            (getpriority and setpriority are included in os2.c, but
            Configure doesn't know to look there.)
[fixed already]
        pager='f:/usr/bin/less.exe'
            (Correcting Configure's insistence on a leading slash.)
        bin_sh='f:/usr/bin/sh.exe'
            (If Configure detects sh.exe somewhere else first.  Example:
            it saw sh.exe at /bin/sh.exe on my TVFS drive, but I want
            perl to look for it on the physical F drive.)
        aout_ccflags='... existing flags... -DDEBUGGING'
        aout_cppflags='... existing flags... -DDEBUGGING'
            (If you want to include DEBUGGING for the aout version.)
[Do not do it, -D optimize=-g will automatically add these flags.]

>>> Allow Configure to make the build scripts.

>>> Allow Configure to run `make depend`.  Ignore the following warning:

        perl.h:861: warning: `DEBUGGING_MSTATS' redefined
[corrected now]

>>> Rename any existing perl.dll, preventing anything from loading it and
    saving a known working copy in case something goes wrong:

        mv /usr/lib/perl.dll /usr/lib/ilya-perl.dll

>>> Run `make`, and ignore the following warnings:

        perl.h:861: warning: `DEBUGGING_MSTATS' redefined
[corrected now]
        invalid preprocessing directive name
        emxomf warning: Cycle detected by make_type
        LINK386 :  warning L4071: application type not specified; assuming WINDOWCOMPAT
        Warning (will try anyway): No library found for -lposix
        Warning (will try anyway): No library found for -lcposix
        POSIX.c:203: warning: `mkfifo' redefined
        POSIX.c:4603: warning: assignment makes pointer from integer without a cast

>>> If `make` dies while "Making DynaLoader (static)", you'll need to
    put miniperl in the OS/2 paths.  This step is only necessary if `make`
    can't find miniperl:
[I would be interested if somebody confirmes this.]

        cp perl.dll /usr/lib        (where /usr/lib is in your LIBPATH)
        cp miniperl.exe /usr/bin    (where /usr/bin is in your PATH)
        make                        (ignore the errors in the previous step)

        This should run to completion.

>>> Test the build:

        make test

    These tests fail:

        io/fs..........FAILED on test 2

            "OS/2 is not unix".  Test 2 checks the link() command, which
            is not supported by OS/2.

        io/pipe........f:/usr/bin/sh.exe: -c requires an argument
        f:/usr/bin/sh.exe: -c requires an argument
        The Unsupported function fork function is unimplemented at
            io/pipe.t line 26.
        FAILED on test 1

            More "OS/2 is not unix" errors.  Read ahead to find out
            why fork() fails.

        op/exec........FAILED on test 4

                if (system "true") {print "not ok 4\n";} else \
                {print "ok 4\n";}

            This fails for me, but changing it to read like this works:

                if (system '\usr\bin\true.cmd') {print "not ok 4\n";} \
                else {print "ok 4\n";}

            So you can count this as another "OS/2 is not unix".

        op/fork........The Unsupported function fork function is \
            unimplemented at op/fork.t line 8.
        FAILED on test 1

            The dynamically-loaded version of perl currently doesn't
            support fork().  This is a known behavior of EMX.

        op/magic.......
        Process terminated by SIGINT
        ok

            The test passed even with the SIGINT message.  I don't
            know why, but I won't argue.

        op/stat........ls: /dev: No such file or directory
        f:/usr/bin/sh.exe: ln: not found
        ls: perl: No such file or directory
        FAILED on test 3

            "OS/2 is not unix".  We don't have the ln command.

        lib/anydbm.....Bad free() ignored at lib/anydbm.t line 51.
        Bad free() ignored at lib/anydbm.t line 51.
        Bad free() ignored at lib/anydbm.t line 51.
        Bad free() ignored during global destruction.
        Bad free() ignored during global destruction.
        Bad free() ignored during global destruction.
        FAILED on test 2

            Test 2 looks at the file permissions for a database.  "OS/2
            is not unix" so the permissions aren't exactly what this test
            expects.

        lib/db-btree...Bad free() ignored at lib/db-btree.t line 109.
        Bad free() ignored at lib/db-btree.t line 221.
        Bad free() ignored at lib/db-btree.t line 337.
        Bad free() ignored at lib/db-btree.t line 349.
        Bad free() ignored at lib/db-btree.t line 349.
        Bad free() ignored at lib/db-btree.t line 399.
        Bad free() ignored at lib/db-btree.t line 400.
        Bad free() ignored at lib/db-btree.t line 401.
        FAILED on test 20

            Another file permissions test fails.

        lib/db-hash....Bad free() ignored at lib/db-hash.t line 101.
        Bad free() ignored at lib/db-hash.t line 101.
        Bad free() ignored at lib/db-hash.t line 101.
        Bad free() ignored at lib/db-hash.t line 239.
        Bad free() ignored at lib/db-hash.t line 239.
        Bad free() ignored at lib/db-hash.t line 239.
        Bad free() ignored at lib/db-hash.t line 253.
        Bad free() ignored at lib/db-hash.t line 253.
        Bad free() ignored at lib/db-hash.t line 253.
        FAILED on test 16

            Another file permissions test fails.

        lib/db-recno...Bad free() ignored at lib/db-recno.t line 138.
        Bad free() ignored at lib/db-recno.t line 138.
        FAILED on test 18

            Another file permissions test fails.

        lib/gdbm.......FAILED on test 2

            Another file permissions test fails.

        lib/sdbm.......FAILED on test 2

            Another file permissions test fails.

        Failed 11/94 tests, 88.30% okay.

            All of which are known differences with unix or documented
            behaviors in EMX.  I re-run the test with Ilya's version,
            and the same tests fail.  This new build is a success.
[Note that bad free() mentioned above are bugs in the Berkeley
DB. They just are more visible under OS/2 with perl free(), because of
"rigid" function name resolution.
	To get finer tests, cd to ./t and run 
		perl harness
]

    (Actually, Ilya's perl release fails an extra test because I don't
    have sed in f:\emx.add.  This shows how important it is to configure
    and build perl yourself instead of grabbing pre-built binaries.)
[Hmm, should not happen... There is no mentions of full_sed under ./t
directory...]

>>> Cross your fingers and install it:

        make install

    Warnings encountered and workarounds presented.:

        WARNING: You've never run 'make test'!!!  (Installing anyway.)
            (Lies!  All lies!  At least it still installs.)

        WARNING: Can't find libperl*.dll* to install into \
        f:/usr/lib/perl5/os2/5.00201/CORE.  (Installing other things anyway.)
            (Safe to ignore.  The important one, libperl.lib, gets copied.)

        Couldn't copy f:/usr/bin/perl5.00201.exe to f:/usr/bin/perl.exe: \
        No such file or directory
            cp /usr/bin/perl5.00201.exe /usr/bin/perl.exe

        Couldn't copy f:/usr/bin/perl.exe to /usr/bin/perl.exe: No such \
        file or directory
            (I think this one is safe to ignore since the two directories
            point to the same place.)

>>> Laugh maniacally because you just built and installed your own copy
    of perl, with all the paths set "just so" and with whatever little
    psychotic modifications you've always wanted but were afraid to add.

-----------------------------------------------------------------------------

Development tools and versions:

        EMX 0.9b with emxfix04 applied.

        `ls --version` reports: 'GNU file utilities 3.12'
        `tr --version` reports: 'tr - GNU textutils 1.14'
        `id --version` reports: 'id - GNU sh-utils 1.12'

        `sed --version` reports: 'GNU sed version 2.05'
        `awk --version` reports: 'Gnu Awk (gawk) 2.15, patchlevel 6'
        `grep --version` reports an illegal option and: 'GNU grep version 2.0'
                (this includes egrep)

        `sort --version` reports: 'sort - GNU textutils 1.14'
        `uniq --version` reports: 'uniq - GNU textutils 1.14'
        `find --version` reports: 'GNU find version 4.1'

        KSH_VERSION='@(#)PD KSH v5.2.4 96/01/17'
                (Ilya's patched version.)

        `make --version` reports: 'GNU Make version 3.74'
                (Ilya's patched version.)

        `emxrev` reports:
                EMX : revision = 42
                EMXIO : revision = 40
                EMXLIBC : revision = 40
                EMXLIBCM : revision = 43
                EMXLIBCS : revision = 43
                EMXWRAP : revision = 40

-----------------------------------------------------------------------------

Rocco
<troc@shadow.net>