Cleanup & OS/390 "fix"

[perl5.git] / README.os2

If you read this file _as_is_, just ignore the funny characters you
see. It is written in the POD format (see perlpod manpage) which is
specially designed to be readable as is.

=head1 NAME

perlos2 - Perl under OS/2, DOS, Win0.3*, Win0.95 and WinNT.

=head1 SYNOPSIS

One can read this document in the following formats:

        man perlos2
        view perl perlos2
        explorer perlos2.html
        info perlos2

to list some (not all may be available simultaneously), or it may
be read I<as is>: either as F<README.os2>, or F<pod/perlos2.pod>.

To read the F<.INF> version of documentation (B<very> recommended)
outside of OS/2, one needs an IBM's reader (may be available on IBM
ftp sites (?)  (URL anyone?)) or shipped with PC DOS 7.0 and IBM's
Visual Age C++ 3.5.

A copy of a Win* viewer is contained in the "Just add OS/2 Warp" package

  ftp://ftp.software.ibm.com/ps/products/os2/tools/jaow/jaow.zip

in F<?:\JUST_ADD\view.exe>. This gives one an access to EMX's 
F<.INF> docs as well (text form is available in F</emx/doc> in 
EMX's distribution).

Note that if you have F<lynx.exe> installed, you can follow WWW links
from this document in F<.INF> format. If you have EMX docs installed 
correctly, you can follow library links (you need to have C<view emxbook>
working by setting C<EMXBOOK> environment variable as it is described
in EMX docs).

=cut

Contents
 
 perlos2 - Perl under OS/2, DOS, Win0.3*, Win0.95 and WinNT. 

      NAME 
      SYNOPSIS 
      DESCRIPTION 
         -  Target 
         -  Other OSes 
         -  Prerequisites 
         -  Starting Perl programs under OS/2 (and DOS and...)
         -  Starting OS/2 (and DOS) programs under Perl 
      Frequently asked questions 
         -  I cannot run external programs 
         -  I cannot embed perl into my program, or use perl.dll from my program. 
         -  `` and pipe-open do not work under DOS. 
         -  Cannot start find.exe "pattern" file
      INSTALLATION 
         -  Automatic binary installation 
         -  Manual binary installation 
         -  Warning 
      Accessing documentation 
         -  OS/2 .INF file 
         -  Plain text 
         -  Manpages 
         -  HTML 
         -  GNU info files 
         -  .PDF files 
         -  LaTeX docs 
      BUILD 
         -  Prerequisites 
         -  Getting perl source 
         -  Application of the patches 
         -  Hand-editing 
         -  Making 
         -  Testing 
         -  Installing the built perl 
         -  a.out-style build 
      Build FAQ 
         -  Some / became \ in pdksh. 
         -  'errno' - unresolved external 
         -  Problems with tr 
         -  Some problem (forget which ;-) 
         -  Library ... not found 
         -  Segfault in make 
      Specific (mis)features of EMX port 
         -  setpriority, getpriority 
         -  system() 
         -  extproc on the first line
         -  Additional modules: 
         -  Prebuilt methods: 
         -  Misfeatures 
         -  Modifications 
      Perl flavors 
         -  perl.exe 
         -  perl_.exe 
         -  perl__.exe 
         -  perl___.exe 
         -  Why strange names? 
         -  Why dynamic linking? 
         -  Why chimera build? 
      ENVIRONMENT 
         -  PERLLIB_PREFIX 
         -  PERL_BADLANG 
         -  PERL_BADFREE 
         -  PERL_SH_DIR 
         -  TMP or TEMP 
      Evolution 
         -  Priorities 
         -  DLL name mangling 
         -  Threading 
         -  Calls to external programs 
         -  Memory allocation 
         -  Threads
      AUTHOR 
      SEE ALSO 

=head1 DESCRIPTION

=head2 Target

The target is to make OS/2 the best supported platform for
using/building/developing Perl and I<Perl applications>, as well as
make Perl the best language to use under OS/2. The secondary target is
to try to make this work under DOS and Win* as well (but not B<too> hard).

The current state is quite close to this target. Known limitations:

=over 5

=item *

Some *nix programs use fork() a lot; with the mostly useful flavors of perl
for OS/2 (there are several built simultaneously) this is supported;
some flavors do not.  Using fork() after I<use>ing dynamically loading
extensions would not work with very old versions of EMX.

=item *

You need a separate perl executable F<perl__.exe> (see L<perl__.exe>)
if you want to use PM code in your application (as Perl/Tk or OpenGL
Perl modules do) without having a text-mode window present.

While using the standard F<perl.exe> from a text-mode window is possible
too, I have seen cases when this causes degradation of the system stability.
Using F<perl__.exe> avoids such a degradation.

=item *

There is no simple way to access WPS objects. The only way I know
is via C<OS2::REXX> extension (see L<OS2::REXX>), and we do not have access to
convenience methods of Object-REXX. (Is it possible at all? I know
of no Object-REXX API.)  The C<SOM> extension (currently in alpha-text)
may eventually remove this shortcoming.

=back

Please keep this list up-to-date by informing me about other items.

=head2 Other OSes

Since OS/2 port of perl uses a remarkable EMX environment, it can
run (and build extensions, and - possibly - be built itself) under any
environment which can run EMX. The current list is DOS,
DOS-inside-OS/2, Win0.3*, Win0.95 and WinNT. Out of many perl flavors,
only one works, see L<"perl_.exe">.

Note that not all features of Perl are available under these
environments. This depends on the features the I<extender> - most
probably RSX - decided to implement.

Cf. L<Prerequisites>.

=head2 Prerequisites

=over 6

=item EMX

EMX runtime is required (may be substituted by RSX). Note that
it is possible to make F<perl_.exe> to run under DOS without any
external support by binding F<emx.exe>/F<rsx.exe> to it, see L<emxbind>. Note
that under DOS for best results one should use RSX runtime, which
has much more functions working (like C<fork>, C<popen> and so on). In
fact RSX is required if there is no VCPI present. Note the
RSX requires DPMI.

Only the latest runtime is supported, currently C<0.9d fix 03>. Perl may run
under earlier versions of EMX, but this is not tested.

One can get different parts of EMX from, say

  http://www.leo.org/pub/comp/os/os2/leo/gnu/emx+gcc/
  http://powerusersbbs.com/pub/os2/dev/   [EMX+GCC Development]
  http://hobbes.nmsu.edu/pub/os2/dev/emx/v0.9d/

The runtime component should have the name F<emxrt.zip>.

B<NOTE>. It is enough to have F<emx.exe>/F<rsx.exe> on your path. One
does not need to specify them explicitly (though this

  emx perl_.exe -de 0

will work as well.)

=item RSX

To run Perl on DPMI platforms one needs RSX runtime. This is
needed under DOS-inside-OS/2, Win0.3*, Win0.95 and WinNT (see 
L<"Other OSes">). RSX would not work with VCPI
only, as EMX would, it requires DMPI.

Having RSX and the latest F<sh.exe> one gets a fully functional
B<*nix>-ish environment under DOS, say, C<fork>, C<``> and
pipe-C<open> work. In fact, MakeMaker works (for static build), so one
can have Perl development environment under DOS. 

One can get RSX from, say

  ftp://ftp.cdrom.com/pub/os2/emx09c/contrib
  ftp://ftp.uni-bielefeld.de/pub/systems/msdos/misc
  ftp://ftp.leo.org/pub/comp/os/os2/leo/devtools/emx+gcc/contrib

Contact the author on C<rainer@mathematik.uni-bielefeld.de>.

The latest F<sh.exe> with DOS hooks is available in

  ftp://ftp.math.ohio-state.edu/pub/users/ilya/os2/

as F<sh_dos.zip> or under similar names starting with C<sh>, C<pdksh> etc.

=item HPFS

Perl does not care about file systems, but to install the whole perl
library intact one needs a file system which supports long file names.

Note that if you do not plan to build the perl itself, it may be
possible to fool EMX to truncate file names. This is not supported,
read EMX docs to see how to do it.

=item pdksh

To start external programs with complicated command lines (like with
pipes in between, and/or quoting of arguments), Perl uses an external
shell. With EMX port such shell should be named F<sh.exe>, and located
either in the wired-in-during-compile locations (usually F<F:/bin>),
or in configurable location (see L<"PERL_SH_DIR">).

For best results use EMX pdksh. The standard binary (5.2.14 or later) runs
under DOS (with L<RSX>) as well, see

  ftp://ftp.math.ohio-state.edu/pub/users/ilya/os2/

=back

=head2 Starting Perl programs under OS/2 (and DOS and...)

Start your Perl program F<foo.pl> with arguments C<arg1 arg2 arg3> the
same way as on any other platform, by

        perl foo.pl arg1 arg2 arg3

If you want to specify perl options C<-my_opts> to the perl itself (as
opposed to to your program), use

        perl -my_opts foo.pl arg1 arg2 arg3

Alternately, if you use OS/2-ish shell, like CMD or 4os2, put
the following at the start of your perl script:

        extproc perl -S -my_opts

rename your program to F<foo.cmd>, and start it by typing

        foo arg1 arg2 arg3

Note that because of stupid OS/2 limitations the full path of the perl
script is not available when you use C<extproc>, thus you are forced to
use C<-S> perl switch, and your script should be on the C<PATH>. As a plus
side, if you know a full path to your script, you may still start it
with 

        perl ../../blah/foo.cmd arg1 arg2 arg3

(note that the argument C<-my_opts> is taken care of by the C<extproc> line
in your script, see L<C<extproc> on the first line>).

To understand what the above I<magic> does, read perl docs about C<-S>
switch - see L<perlrun>, and cmdref about C<extproc>:

        view perl perlrun
        man perlrun
        view cmdref extproc
        help extproc

or whatever method you prefer.

There are also endless possibilities to use I<executable extensions> of
4os2, I<associations> of WPS and so on... However, if you use
*nixish shell (like F<sh.exe> supplied in the binary distribution),
you need to follow the syntax specified in L<perlrun/"Switches">.

Note that B<-S> switch enables a search with additional extensions 
F<.cmd>, F<.btm>, F<.bat>, F<.pl> as well.

=head2 Starting OS/2 (and DOS) programs under Perl

This is what system() (see L<perlfunc/system>), C<``> (see
L<perlop/"I/O Operators">), and I<open pipe> (see L<perlfunc/open>)
are for. (Avoid exec() (see L<perlfunc/exec>) unless you know what you
do).

Note however that to use some of these operators you need to have a
sh-syntax shell installed (see L<"Pdksh">, 
L<"Frequently asked questions">), and perl should be able to find it
(see L<"PERL_SH_DIR">).

The cases when the shell is used are:

=over

=item 1

One-argument system() (see L<perlfunc/system>), exec() (see L<perlfunc/exec>)
with redirection or shell meta-characters;

=item 2

Pipe-open (see L<perlfunc/open>) with the command which contains redirection 
or shell meta-characters;

=item 3

Backticks C<``> (see L<perlop/"I/O Operators">) with the command which contains
redirection or shell meta-characters;

=item 4

If the executable called by system()/exec()/pipe-open()/C<``> is a script
with the "magic" C<#!> line or C<extproc> line which specifies shell;

=item 5

If the executable called by system()/exec()/pipe-open()/C<``> is a script
without "magic" line, and C<$ENV{EXECSHELL}> is set to shell;

=item 6

If the executable called by system()/exec()/pipe-open()/C<``> is not
found;

=item 7

For globbing (see L<perlfunc/glob>, L<perlop/"I/O Operators">).

=back

For the sake of speed for a common case, in the above algorithms 
backslashes in the command name are not considered as shell metacharacters.

Perl starts scripts which begin with cookies
C<extproc> or C<#!> directly, without an intervention of shell.  Perl uses the
same algorithm to find the executable as F<pdksh>: if the path
on C<#!> line does not work, and contains C</>, then the executable
is searched in F<.> and on C<PATH>.  To find arguments for these scripts
Perl uses a different algorithm than F<pdksh>: up to 3 arguments are 
recognized, and trailing whitespace is stripped.

If a script
does not contain such a cooky, then to avoid calling F<sh.exe>, Perl uses
the same algorithm as F<pdksh>: if C<$ENV{EXECSHELL}> is set, the
script is given as the first argument to this command, if not set, then
C<$ENV{COMSPEC} /c> is used (or a hardwired guess if C<$ENV{COMSPEC}> is
not set).

If starting scripts directly, Perl will use exactly the same algorithm as for 
the search of script given by B<-S> command-line option: it will look in
the current directory, then on components of C<$ENV{PATH}> using the 
following order of appended extensions: no extension, F<.cmd>, F<.btm>, 
F<.bat>, F<.pl>.

Note that Perl will start to look for scripts only if OS/2 cannot start the
specified application, thus C<system 'blah'> will not look for a script if 
there is an executable file F<blah.exe> I<anywhere> on C<PATH>.  

Note also that executable files on OS/2 can have an arbitrary extension, 
but F<.exe> will be automatically appended if no dot is present in the name.  
The workaround as as simple as that:  since F<blah.> and F<blah> denote the 
same file, to start an executable residing in file F<n:/bin/blah> (no 
extension) give an argument C<n:/bin/blah.> (dot appended) to system().

Perl will correctly start PM programs from VIO (=text-mode) Perl process;
the opposite is not true: when you start a non-PM program from a PM
Perl process, it would not run it in a separate session.  If a separate
session is desired, either ensure
that shell will be used, as in C<system 'cmd /c myprog'>, or start it using
optional arguments to system() documented in C<OS2::Process> module.  This
is considered to be a feature.

=head1 Frequently asked questions

=head2 "It does not work"

Perl binary distributions come with a F<testperl.cmd> script which tries
to detect common problems with misconfigured installations.  There is a
pretty large chance it will discover which step of the installation you
managed to goof.  C<;-)>

=head2 I cannot run external programs

=over 4

=item *

Did you run your programs with C<-w> switch? See 
L<Starting OS/2 (and DOS) programs under Perl>.

=item *

Do you try to run I<internal> shell commands, like C<`copy a b`>
(internal for F<cmd.exe>), or C<`glob a*b`> (internal for ksh)? You
need to specify your shell explicitly, like C<`cmd /c copy a b`>,
since Perl cannot deduce which commands are internal to your shell.

=back

=head2 I cannot embed perl into my program, or use F<perl.dll> from my
program. 

=over 4

=item Is your program EMX-compiled with C<-Zmt -Zcrtdll>?

If not, you need to build a stand-alone DLL for perl. Contact me, I
did it once. Sockets would not work, as a lot of other stuff.

=item Did you use L<ExtUtils::Embed>?

I had reports it does not work. Somebody would need to fix it.

=back

=head2 C<``> and pipe-C<open> do not work under DOS.

This may a variant of just L<"I cannot run external programs">, or a
deeper problem. Basically: you I<need> RSX (see L<"Prerequisites">)
for these commands to work, and you may need a port of F<sh.exe> which
understands command arguments. One of such ports is listed in
L<"Prerequisites"> under RSX. Do not forget to set variable
C<L<"PERL_SH_DIR">> as well.

DPMI is required for RSX.

=head2 Cannot start C<find.exe "pattern" file>

Use one of

  system 'cmd', '/c', 'find "pattern" file';
  `cmd /c 'find "pattern" file'`

This would start F<find.exe> via F<cmd.exe> via C<sh.exe> via
C<perl.exe>, but this is a price to pay if you want to use
non-conforming program. In fact F<find.exe> cannot be started at all
using C library API only. Otherwise the following command-lines would be
equivalent:

  find "pattern" file
  find pattern file

=head1 INSTALLATION

=head2 Automatic binary installation

The most convenient way of installing a binary distribution of perl is via perl installer
F<install.exe>. Just follow the instructions, and 99% of the
installation blues would go away. 

Note however, that you need to have F<unzip.exe> on your path, and
EMX environment I<running>. The latter means that if you just
installed EMX, and made all the needed changes to F<Config.sys>,
you may need to reboot in between. Check EMX runtime by running

        emxrev

A folder is created on your desktop which contains some useful
objects.

B<Things not taken care of by automatic binary installation:>

=over 15

=item C<PERL_BADLANG>

may be needed if you change your codepage I<after> perl installation,
and the new value is not supported by EMX. See L<"PERL_BADLANG">.

=item C<PERL_BADFREE>

see L<"PERL_BADFREE">.

=item F<Config.pm>

This file resides somewhere deep in the location you installed your
perl library, find it out by 

  perl -MConfig -le "print $INC{'Config.pm'}"

While most important values in this file I<are> updated by the binary
installer, some of them may need to be hand-edited. I know no such
data, please keep me informed if you find one.

=back

B<NOTE>. Because of a typo the binary installer of 5.00305
would install a variable C<PERL_SHPATH> into F<Config.sys>. Please
remove this variable and put C<L<PERL_SH_DIR>> instead.

=head2 Manual binary installation

As of version 5.00305, OS/2 perl binary distribution comes split
into 11 components. Unfortunately, to enable configurable binary
installation, the file paths in the zip files are not absolute, but
relative to some directory.

Note that the extraction with the stored paths is still necessary
(default with unzip, specify C<-d> to pkunzip). However, you
need to know where to extract the files. You need also to manually
change entries in F<Config.sys> to reflect where did you put the
files. Note that if you have some primitive unzipper (like
pkunzip), you may get a lot of warnings/errors during
unzipping. Upgrade to C<(w)unzip>.

Below is the sample of what to do to reproduce the configuration on my
machine:

=over 3

=item Perl VIO and PM executables (dynamically linked)

  unzip perl_exc.zip *.exe *.ico -d f:/emx.add/bin
  unzip perl_exc.zip *.dll -d f:/emx.add/dll

(have the directories with C<*.exe> on PATH, and C<*.dll> on
LIBPATH);

=item Perl_ VIO executable (statically linked)

  unzip perl_aou.zip -d f:/emx.add/bin

(have the directory on PATH);

=item Executables for Perl utilities

  unzip perl_utl.zip -d f:/emx.add/bin

(have the directory on PATH);

=item Main Perl library

  unzip perl_mlb.zip -d f:/perllib/lib

If this directory is exactly the same as the prefix which was compiled
into F<perl.exe>, you do not need to change
anything. However, for perl to find the library if you use a different
path, you need to
C<set PERLLIB_PREFIX> in F<Config.sys>, see L<"PERLLIB_PREFIX">.

=item Additional Perl modules

  unzip perl_ste.zip -d f:/perllib/lib/site_perl/5.8.3/

Same remark as above applies.  Additionally, if this directory is not
one of directories on @INC (and @INC is influenced by C<PERLLIB_PREFIX>), you
need to put this
directory and subdirectory F<./os2> in C<PERLLIB> or C<PERL5LIB>
variable. Do not use C<PERL5LIB> unless you have it set already. See
L<perl/"ENVIRONMENT">.

=item Tools to compile Perl modules

  unzip perl_blb.zip -d f:/perllib/lib

Same remark as for F<perl_ste.zip>.

=item Manpages for Perl and utilities

  unzip perl_man.zip -d f:/perllib/man

This directory should better be on C<MANPATH>. You need to have a
working man to access these files.

=item Manpages for Perl modules

  unzip perl_mam.zip -d f:/perllib/man

This directory should better be on C<MANPATH>. You need to have a
working man to access these files.

=item Source for Perl documentation

  unzip perl_pod.zip -d f:/perllib/lib

This is used by the C<perldoc> program (see L<perldoc>), and may be used to
generate HTML documentation usable by WWW browsers, and
documentation in zillions of other formats: C<info>, C<LaTeX>,
C<Acrobat>, C<FrameMaker> and so on.

=item Perl manual in F<.INF> format

  unzip perl_inf.zip -d d:/os2/book

This directory should better be on C<BOOKSHELF>.

=item Pdksh

  unzip perl_sh.zip -d f:/bin

This is used by perl to run external commands which explicitly
require shell, like the commands using I<redirection> and I<shell
metacharacters>. It is also used instead of explicit F</bin/sh>.

Set C<PERL_SH_DIR> (see L<"PERL_SH_DIR">) if you move F<sh.exe> from
the above location.

B<Note.> It may be possible to use some other sh-compatible shell
(file globbing - if done via shell - may break).

=back

After you installed the components you needed and updated the
F<Config.sys> correspondingly, you need to hand-edit
F<Config.pm>. This file resides somewhere deep in the location you
installed your perl library, find it out by

  perl -MConfig -le "print $INC{'Config.pm'}"

You need to correct all the entries which look like file paths (they
currently start with C<f:/>).

=head2 B<Warning>

The automatic and manual perl installation leave precompiled paths
inside perl executables. While these paths are overwriteable (see
L<"PERLLIB_PREFIX">, L<"PERL_SH_DIR">), one may get better results by
binary editing of paths inside the executables/DLLs.

=head1 Accessing documentation

Depending on how you built/installed perl you may have (otherwise
identical) Perl documentation in the following formats:

=head2 OS/2 F<.INF> file

Most probably the most convenient form. Under OS/2 view it as

  view perl
  view perl perlfunc
  view perl less
  view perl ExtUtils::MakeMaker

(currently the last two may hit a wrong location, but this may improve
soon). Under Win* see L<"SYNOPSIS">.

If you want to build the docs yourself, and have I<OS/2 toolkit>, run

        pod2ipf > perl.ipf

in F</perllib/lib/pod> directory, then

        ipfc /inf perl.ipf

(Expect a lot of errors during the both steps.) Now move it on your
BOOKSHELF path.

=head2 Plain text

If you have perl documentation in the source form, perl utilities
installed, and GNU groff installed, you may use 

        perldoc perlfunc
        perldoc less
        perldoc ExtUtils::MakeMaker

to access the perl documentation in the text form (note that you may get
better results using perl manpages).

Alternately, try running pod2text on F<.pod> files.

=head2 Manpages

If you have man installed on your system, and you installed perl
manpages, use something like this:

        man perlfunc
        man 3 less
        man ExtUtils.MakeMaker

to access documentation for different components of Perl. Start with

        man perl

Note that dot (F<.>) is used as a package separator for documentation
for packages, and as usual, sometimes you need to give the section - C<3>
above - to avoid shadowing by the I<less(1) manpage>.

Make sure that the directory B<above> the directory with manpages is
on our C<MANPATH>, like this

  set MANPATH=c:/man;f:/perllib/man

for Perl manpages in C<f:/perllib/man/man1/> etc.

=head2 HTML

If you have some WWW browser available, installed the Perl
documentation in the source form, and Perl utilities, you can build
HTML docs. Cd to directory with F<.pod> files, and do like this

        cd f:/perllib/lib/pod
        pod2html

After this you can direct your browser the file F<perl.html> in this
directory, and go ahead with reading docs, like this:

        explore file:///f:/perllib/lib/pod/perl.html

Alternatively you may be able to get these docs prebuilt from CPAN.

=head2 GNU C<info> files

Users of Emacs would appreciate it very much, especially with
C<CPerl> mode loaded. You need to get latest C<pod2info> from C<CPAN>,
or, alternately, prebuilt info pages.

=head2 F<.PDF> files

for C<Acrobat> are available on CPAN (for slightly old version of
perl).

=head2 C<LaTeX> docs

can be constructed using C<pod2latex>.

=head1 BUILD

Here we discuss how to build Perl under OS/2. There is an alternative
(but maybe older) view on http://www.shadow.net/~troc/os2perl.html

=head2 The short story

Assume that you are a seasoned porter, so are sure that all the necessary
tools are already present on your system, and you know how to get the Perl
source distribution.  Untar it, change to the extract directory, and

  gnupatch -p0 < os2\diff.configure
  sh Configure -des -D prefix=f:/perllib
  make
  make test
  make install
  make aout_test
  make aout_install

This puts the executables in f:/perllib/bin.  Manually move them to the
C<PATH>, manually move the built F<perl*.dll> to C<LIBPATH> (here F<*> is
a not-very-meaningful hex checksum), and run

  make installcmd INSTALLCMDDIR=d:/ir/on/path

What follows is a detailed guide through these steps.

=head2 Prerequisites

You need to have the latest EMX development environment, the full
GNU tool suite (gawk renamed to awk, and GNU F<find.exe>
earlier on path than the OS/2 F<find.exe>, same with F<sort.exe>, to
check use

  find --version
  sort --version

). You need the latest version of F<pdksh> installed as F<sh.exe>.

Check that you have B<BSD> libraries and headers installed, and - 
optionally - Berkeley DB headers and libraries, and crypt.

Possible locations to get this from are

  ftp://hobbes.nmsu.edu/os2/unix/
  ftp://ftp.cdrom.com/pub/os2/unix/
  ftp://ftp.cdrom.com/pub/os2/dev32/
  ftp://ftp.cdrom.com/pub/os2/emx09c/

It is reported that the following archives contain enough utils to
build perl: F<gnufutil.zip>, F<gnusutil.zip>, F<gnututil.zip>, F<gnused.zip>,
F<gnupatch.zip>, F<gnuawk.zip>, F<gnumake.zip>, F<bsddev.zip> and
F<ksh527rt.zip> (or a later version).  Note that all these utilities are
known to be available from LEO:

  ftp://ftp.leo.org/pub/comp/os/os2/leo/gnu

If you have I<exactly the same version of Perl> installed already,
make sure that no copies or perl are currently running.  Later steps
of the build may fail since an older version of F<perl.dll> loaded into
memory may be found. 

Also make sure that you have F</tmp> directory on the current drive,
and F<.> directory in your C<LIBPATH>. One may try to correct the
latter condition by

  set BEGINLIBPATH .

if you use something like F<CMD.EXE> or latest versions of F<4os2.exe>.

Make sure your gcc is good for C<-Zomf> linking: run C<omflibs>
script in F</emx/lib> directory.

Check that you have link386 installed. It comes standard with OS/2,
but may be not installed due to customization. If typing

  link386

shows you do not have it, do I<Selective install>, and choose C<Link
object modules> in I<Optional system utilities/More>. If you get into
link386 prompts, press C<Ctrl-C> to exit.

=head2 Getting perl source

You need to fetch the latest perl source (including developers
releases). With some probability it is located in 

  http://www.perl.com/CPAN/src/5.0
  http://www.perl.com/CPAN/src/5.0/unsupported

If not, you may need to dig in the indices to find it in the directory
of the current maintainer.

Quick cycle of developers release may break the OS/2 build time to
time, looking into 

  http://www.perl.com/CPAN/ports/os2/ilyaz/

may indicate the latest release which was publicly released by the
maintainer. Note that the release may include some additional patches
to apply to the current source of perl.

Extract it like this

  tar vzxf perl5.00409.tar.gz

You may see a message about errors while extracting F<Configure>. This is
because there is a conflict with a similarly-named file F<configure>.

Change to the directory of extraction.

=head2 Application of the patches

You need to apply the patches in F<./os2/diff.*> like this:

  gnupatch -p0 < os2\diff.configure

You may also need to apply the patches supplied with the binary
distribution of perl.

Note also that the F<db.lib> and F<db.a> from the EMX distribution
are not suitable for multi-threaded compile (even single-threaded
flavor of Perl uses multi-threaded C RTL, for
compatibility with XFree86-OS/2). Get a corrected one from

  ftp://ftp.math.ohio-state.edu/pub/users/ilya/os2/db_mt.zip

=head2 Hand-editing

You may look into the file F<./hints/os2.sh> and correct anything
wrong you find there. I do not expect it is needed anywhere.

=head2 Making

  sh Configure -des -D prefix=f:/perllib

C<prefix> means: where to install the resulting perl library. Giving
correct prefix you may avoid the need to specify C<PERLLIB_PREFIX>,
see L<"PERLLIB_PREFIX">.

I<Ignore the message about missing C<ln>, and about C<-c> option to
tr>. The latter is most probably already fixed, if you see it and can trace
where the latter spurious warning comes from, please inform me.

Now

  make

At some moment the built may die, reporting a I<version mismatch> or
I<unable to run F<perl>>.  This means that you do not have F<.> in
your LIBPATH, so F<perl.exe> cannot find the needed F<perl67B2.dll> (treat
these hex digits as line noise).  After this is fixed the build
should finish without a lot of fuss.

=head2 Testing

Now run

  make test

All tests should succeed (with some of them skipped).

Some tests may generate extra messages similar to

=over 4

=item A lot of C<bad free>

in database tests related to Berkeley DB. I<This should be fixed already.>
If it persists, you may disable this warnings, see L<"PERL_BADFREE">.

=item Process terminated by SIGTERM/SIGINT

This is a standard message issued by OS/2 applications. *nix
applications die in silence. It is considered to be a feature. One can
easily disable this by appropriate sighandlers. 

However the test engine bleeds these message to screen in unexpected
moments. Two messages of this kind I<should> be present during
testing.

=back

To get finer test reports, call

  perl t/harness

The report with F<io/pipe.t> failing may look like this:

  Failed Test  Status Wstat Total Fail  Failed  List of failed
  ------------------------------------------------------------
  io/pipe.t                    12    1   8.33%  9
  7 tests skipped, plus 56 subtests skipped.
  Failed 1/195 test scripts, 99.49% okay. 1/6542 subtests failed, 99.98% okay.

The reasons for most important skipped tests are:

=over 8

=item F<op/fs.t>

=over 4

=item 18

Checks C<atime> and C<mtime> of C<stat()> - unfortunately, HPFS
provides only 2sec time granularity (for compatibility with FAT?).

=item 25

Checks C<truncate()> on a filehandle just opened for write - I do not
know why this should or should not work.

=back

=item F<op/stat.t>

Checks C<stat()>. Tests:

=over 4

=item 4

Checks C<atime> and C<mtime> of C<stat()> - unfortunately, HPFS
provides only 2sec time granularity (for compatibility with FAT?).

=back

=back

=head2 Installing the built perl

If you haven't yet moved perl.dll onto LIBPATH, do it now.

Run

  make install

It would put the generated files into needed locations. Manually put
F<perl.exe>, F<perl__.exe> and F<perl___.exe> to a location on your
PATH, F<perl.dll> to a location on your LIBPATH.

Run

  make installcmd INSTALLCMDDIR=d:/ir/on/path

to convert perl utilities to F<.cmd> files and put them on
PATH. You need to put F<.EXE>-utilities on path manually. They are
installed in C<$prefix/bin>, here C<$prefix> is what you gave to
F<Configure>, see L<Making>.

=head2 C<a.out>-style build

Proceed as above, but make F<perl_.exe> (see L<"perl_.exe">) by

  make perl_

test and install by

  make aout_test
  make aout_install

Manually put F<perl_.exe> to a location on your PATH.

B<Note.> The build process for C<perl_> I<does not know> about all the
dependencies, so you should make sure that anything is up-to-date,
say, by doing

  make perl_dll

first.

=head1 Build FAQ

=head2 Some C</> became C<\> in pdksh.

You have a very old pdksh. See L<Prerequisites>.

=head2 C<'errno'> - unresolved external

You do not have MT-safe F<db.lib>. See L<Prerequisites>.

=head2 Problems with tr or sed

reported with very old version of tr and sed.

=head2 Some problem (forget which ;-)

You have an older version of F<perl.dll> on your LIBPATH, which
broke the build of extensions.

=head2 Library ... not found

You did not run C<omflibs>. See L<Prerequisites>.

=head2 Segfault in make

You use an old version of GNU make. See L<Prerequisites>.

=head2 op/sprintf test failure

This can result from a bug in emx sprintf which was fixed in 0.9d fix 03.

=head1 Specific (mis)features of OS/2 port

=head2 C<setpriority>, C<getpriority>

Note that these functions are compatible with *nix, not with the older
ports of '94 - 95. The priorities are absolute, go from 32 to -95,
lower is quicker. 0 is the default priority.

B<WARNING>.  Calling C<getpriority> on a non-existing process could lock
the system before Warp3 fixpak22.  Starting with Warp3, Perl will use
a workaround: it aborts getpriority() if the process is not present.
This is not possible on older versions C<2.*>, and has a race
condition anyway.

=head2 C<system()>

Multi-argument form of C<system()> allows an additional numeric
argument. The meaning of this argument is described in
L<OS2::Process>.

When finding a program to run, Perl first asks the OS to look for executables
on C<PATH> (OS/2 adds extension F<.exe> if no extension is present).
If not found, it looks for a script with possible extensions 
added in this order: no extension, F<.cmd>, F<.btm>, 
F<.bat>, F<.pl>.  If found, Perl checks the start of the file for magic
strings C<"#!"> and C<"extproc ">.  If found, Perl uses the rest of the
first line as the beginning of the command line to run this script.  The
only mangling done to the first line is extraction of arguments (currently
up to 3), and ignoring of the path-part of the "interpreter" name if it can't
be found using the full path.

E.g., C<system 'foo', 'bar', 'baz'> may lead Perl to finding
F<C:/emx/bin/foo.cmd> with the first line being

 extproc /bin/bash    -x   -c

If F</bin/bash.exe> is not found, then Perl looks for an executable F<bash.exe> on
C<PATH>.  If found in F<C:/emx.add/bin/bash.exe>, then the above system() is
translated to

  system qw(C:/emx.add/bin/bash.exe -x -c C:/emx/bin/foo.cmd bar baz)

One additional translation is performed: instead of F</bin/sh> Perl uses
the hardwired-or-customized shell (see C<L<"PERL_SH_DIR">>).

The above search for "interpreter" is recursive: if F<bash> executable is not
found, but F<bash.btm> is found, Perl will investigate its first line etc.
The only hardwired limit on the recursion depth is implicit: there is a limit
4 on the number of additional arguments inserted before the actual arguments
given to system().  In particular, if no additional arguments are specified
on the "magic" first lines, then the limit on the depth is 4.

If Perl finds that the found executable is of different type than the
current session, it will start the new process in a separate session of
necessary type.  Call via C<OS2::Process> to disable this magic.

B<WARNING>.  Due to the described logic, you need to explicitly
specify F<.com> extension if needed.  Moreover, if the executable
F<perl5.6.1> is requested, Perl will not look for F<perl5.6.1.exe>.
[This may change in the future.]

=head2 C<extproc> on the first line

If the first chars of a Perl script are C<"extproc ">, this line is treated
as C<#!>-line, thus all the switches on this line are processed (twice
if script was started via cmd.exe).  See L<perlrun/DESCRIPTION>.

=head2 Additional modules:

L<OS2::Process>, L<OS2::DLL>, L<OS2::REXX>, L<OS2::PrfDB>, L<OS2::ExtAttr>. These
modules provide access to additional numeric argument for C<system>
and to the information about the running process,
to DLLs having functions with REXX signature and to the REXX runtime, to
OS/2 databases in the F<.INI> format, and to Extended Attributes.

Two additional extensions by Andreas Kaiser, C<OS2::UPM>, and
C<OS2::FTP>, are included into C<ILYAZ> directory, mirrored on CPAN.

=head2 Prebuilt methods:

=over 4

=item C<File::Copy::syscopy>

used by C<File::Copy::copy>, see L<File::Copy>.

=item C<DynaLoader::mod2fname>

used by C<DynaLoader> for DLL name mangling.

=item  C<Cwd::current_drive()>

Self explanatory.

=item  C<Cwd::sys_chdir(name)>

leaves drive as it is.

=item  C<Cwd::change_drive(name)>

chanes the "current" drive.

=item  C<Cwd::sys_is_absolute(name)>

means has drive letter and is_rooted.

=item  C<Cwd::sys_is_rooted(name)>

means has leading C<[/\\]> (maybe after a drive-letter:).

=item  C<Cwd::sys_is_relative(name)>

means changes with current dir.

=item  C<Cwd::sys_cwd(name)>

Interface to cwd from EMX. Used by C<Cwd::cwd>.

=item  C<Cwd::sys_abspath(name, dir)>

Really really odious function to implement. Returns absolute name of
file which would have C<name> if CWD were C<dir>.  C<Dir> defaults to the
current dir.

=item  C<Cwd::extLibpath([type])>

Get current value of extended library search path. If C<type> is
present and I<true>, works with END_LIBPATH, otherwise with
C<BEGIN_LIBPATH>. 

=item  C<Cwd::extLibpath_set( path [, type ] )>

Set current value of extended library search path. If C<type> is
present and I<true>, works with END_LIBPATH, otherwise with
C<BEGIN_LIBPATH>. 

=item C<OS2::Error(do_harderror,do_exception)>

Returns C<undef> if it was not called yet, otherwise bit 1 is
set if on the previous call do_harderror was enabled, bit
2 is set if if on previous call do_exception was enabled.

This function enables/disables error popups associated with 
hardware errors (Disk not ready etc.) and software exceptions.

I know of no way to find out the state of popups I<before> the first call
to this function.

=item C<OS2::Errors2Drive(drive)>

Returns C<undef> if it was not called yet, otherwise return false if errors
were not requested to be written to a hard drive, or the drive letter if
this was requested.

This function may redirect error popups associated with hardware errors
(Disk not ready etc.) and software exceptions to the file POPUPLOG.OS2 at
the root directory of the specified drive.  Overrides OS2::Error() specified
by individual programs.  Given argument undef will disable redirection.

Has global effect, persists after the application exits.

I know of no way to find out the state of redirection of popups to the disk
I<before> the first call to this function.

=item OS2::SysInfo()

Returns a hash with system information. The keys of the hash are

        MAX_PATH_LENGTH, MAX_TEXT_SESSIONS, MAX_PM_SESSIONS,
        MAX_VDM_SESSIONS, BOOT_DRIVE, DYN_PRI_VARIATION,
        MAX_WAIT, MIN_SLICE, MAX_SLICE, PAGE_SIZE,
        VERSION_MAJOR, VERSION_MINOR, VERSION_REVISION,
        MS_COUNT, TIME_LOW, TIME_HIGH, TOTPHYSMEM, TOTRESMEM,
        TOTAVAILMEM, MAXPRMEM, MAXSHMEM, TIMER_INTERVAL,
        MAX_COMP_LENGTH, FOREGROUND_FS_SESSION,
        FOREGROUND_PROCESS

=item OS2::BootDrive()

Returns a letter without colon.

=item C<OS2::MorphPM(serve)>, C<OS2::UnMorphPM(serve)>

Transforms the current application into a PM application and back.
The argument true means that a real message loop is going to be served.
OS2::MorphPM() returns the PM message queue handle as an integer.

See L<"Centralized management of resources"> for additional details.

=item C<OS2::Serve_Messages(force)>

Fake on-demand retrieval of outstanding PM messages.  If C<force> is false,
will not dispatch messages if a real message loop is known to
be present.  Returns number of messages retrieved.

Dies with "QUITing..." if WM_QUIT message is obtained.

=item C<OS2::Process_Messages(force [, cnt])>

Retrieval of PM messages until window creation/destruction.  
If C<force> is false, will not dispatch messages if a real message loop
is known to be present.

Returns change in number of windows.  If C<cnt> is given,
it is incremented by the number of messages retrieved.

Dies with "QUITing..." if WM_QUIT message is obtained.

=item C<OS2::_control87(new,mask)>

the same as L<_control87(3)> of EMX.  Takes integers as arguments, returns
the previous coprocessor control word as an integer.  Only bits in C<new> which
are present in C<mask> are changed in the control word.

=item OS2::get_control87()

gets the coprocessor control word as an integer.

=item C<OS2::set_control87_em(new=MCW_EM,mask=MCW_EM)>

The variant of OS2::_control87() with default values good for
handling exception mask: if no C<mask>, uses exception mask part of C<new>
only.  If no C<new>, disables all the floating point exceptions.

See L<"Misfeatures"> for details.

=back

(Note that some of these may be moved to different libraries -
eventually).


=head2 Prebuilt variables:

=over 4

=item $OS2::emx_rev

same as _emx_rev of EMX, a string similar to C<0.9c>.

=item $OS2::emx_env

same as _emx_env of EMX, a number similar to 0x8001.

=item $OS2::os_ver

a number C<OS_MAJOR + 0.001 * OS_MINOR>.

=back

=head2 Misfeatures

=over 4

=item *

Since L<flock(3)> is present in EMX, but is not functional, it is 
emulated by perl.  To disable the emulations, set environment variable
C<USE_PERL_FLOCK=0>.

=item *

Here is the list of things which may be "broken" on
EMX (from EMX docs):

=over 4

=item *

The functions L<recvmsg(3)>, L<sendmsg(3)>, and L<socketpair(3)> are not
implemented.

=item *

L<sock_init(3)> is not required and not implemented.

=item *

L<flock(3)> is not yet implemented (dummy function).  (Perl has a workaround.)

=item *

L<kill(3)>:  Special treatment of PID=0, PID=1 and PID=-1 is not implemented.

=item *

L<waitpid(3)>:

      WUNTRACED
              Not implemented.
      waitpid() is not implemented for negative values of PID.

=back

Note that C<kill -9> does not work with the current version of EMX.

=item *

Since F<sh.exe> is used for globing (see L<perlfunc/glob>), the bugs
of F<sh.exe> plague perl as well. 

In particular, uppercase letters do not work in C<[...]>-patterns with
the current pdksh.

=item *

Unix-domain sockets on OS/2 live in a pseudo-file-system C</sockets/...>.
To avoid a failure to create a socket with a name of a different form,
C<"/socket/"> is prepended to the socket name (unless it starts with this
already).

This may lead to problems later in case the socket is accessed via the
"usual" file-system calls using the "initial" name.

=item *

Apparently, IBM used a compiler (for some period of time around '95?) which
changes FP mask right and left.  This is not I<that> bad for IBM's
programs, but the same compiler was used for DLLs which are used with
general-purpose applications.  When these DLLs are used, the state of
floating-point flags in the application is not predictable.

What is much worse, some DLLs change the floating point flags when in
_DLLInitTerm() (e.g., F<TCP32IP>).  This means that even if you do not I<call>
any function in the DLL, just the act of loading this DLL will reset your
flags.  What is worse, the same compiler was used to compile some HOOK DLLs.
Given that HOOK dlls are executed in the context of I<all> the applications
in the system, this means a complete unpredictablity of floating point
flags on systems using such HOOK DLLs.  E.g., F<GAMESRVR.DLL> of B<DIVE>
origin changes the floating point flags on each write to the TTY of a VIO
(windowed text-mode) applications.

Some other (not completely debugged) situations when FP flags change include
some video drivers (?), and some operations related to creation of the windows.
People who code B<OpenGL> may have more experience on this.

Perl is generally used in the situation when all the floating-point
exceptions are ignored, as is the default under EMX.  If they are not ignored,
some benign Perl programs would get a C<SIGFPE> and would die a horrible death.

To circumvent this, Perl uses two hacks.  They help against I<one> type of
damage only: FP flags changed when loading a DLL.

One of the hacks is to disable floating point exceptions on startup (as
is the default with EMX).  This helps only with compile-time-linked DLLs
changing the flags before main() had a chance to be called.

The other hack is to restore FP flags after a call to dlopen().  This helps
against similar damage done by DLLs _DLLInitTerm() at runtime.  Currently
no way to switch these hacks off is provided.

=back

=head2 Modifications

Perl modifies some standard C library calls in the following ways:

=over 9

=item C<popen>

C<my_popen> uses F<sh.exe> if shell is required, cf. L<"PERL_SH_DIR">.

=item C<tmpnam>

is created using C<TMP> or C<TEMP> environment variable, via
C<tempnam>.

=item C<tmpfile>

If the current directory is not writable, file is created using modified
C<tmpnam>, so there may be a race condition.

=item C<ctermid>

a dummy implementation.

=item C<stat>

C<os2_stat> special-cases F</dev/tty> and F</dev/con>.

=item C<mkdir>, C<rmdir>

these EMX functions do not work if the path contains a trailing C</>.
Perl contains a workaround for this.

=item C<flock>

Since L<flock(3)> is present in EMX, but is not functional, it is 
emulated by perl.  To disable the emulations, set environment variable
C<USE_PERL_FLOCK=0>.

=back

=head2 Identifying DLLs

All the DLLs built with the current versions of Perl have ID strings
identifying the name of the extension, its version, and the version
of Perl required for this DLL.  Run C<bldlevel DLL-name> to find this
info.

=head2 Centralized management of resources

Since to call certain OS/2 API one needs to have a correctly initialized
C<Win> subsystem, OS/2-specific extensions may require getting C<HAB>s and
C<HMQ>s.  If an extension would do it on its own, another extension could
fail to initialize.

Perl provides a centralized management of these resources:

=over

=item C<HAB>

To get the HAB, the extension should call C<hab = perl_hab_GET()> in C.  After
this call is performed, C<hab> may be accessed as C<Perl_hab>.  There is
no need to release the HAB after it is used.

If by some reasons F<perl.h> cannot be included, use

  extern int Perl_hab_GET(void);

instead.

=item C<HMQ>

There are two cases:

=over

=item *

the extension needs an C<HMQ> only because some API will not work otherwise.
Use C<serve = 0> below.

=item *

the extension needs an C<HMQ> since it wants to engage in a PM event loop.
Use C<serve = 1> below.

=back

To get an C<HMQ>, the extension should call C<hmq = perl_hmq_GET(serve)> in C.
After this call is performed, C<hmq> may be accessed as C<Perl_hmq>.

To signal to Perl that HMQ is not needed any more, call
C<perl_hmq_UNSET(serve)>.  Perl process will automatically morph/unmorph itself
into/from a PM process if HMQ is needed/not-needed.  Perl will automatically
enable/disable C<WM_QUIT> message during shutdown if the message queue is
served/not-served.

B<NOTE>.  If during a shutdown there is a message queue which did not disable
WM_QUIT, and which did not process the received WM_QUIT message, the
shutdown will be automatically cancelled.  Do not call C<perl_hmq_GET(1)>
unless you are going to process messages on an orderly basis.

=back

=head1 Perl flavors

Because of idiosyncrasies of OS/2 one cannot have all the eggs in the
same basket (though EMX environment tries hard to overcome this
limitations, so the situation may somehow improve). There are 4
executables for Perl provided by the distribution:

=head2 F<perl.exe>

The main workhorse. This is a chimera executable: it is compiled as an
C<a.out>-style executable, but is linked with C<omf>-style dynamic
library F<perl.dll>, and with dynamic CRT DLL. This executable is a
VIO application.

It can load perl dynamic extensions, and it can fork().

B<Note.> Keep in mind that fork() is needed to open a pipe to yourself.

=head2 F<perl_.exe>

This is a statically linked C<a.out>-style executable. It cannot
load dynamic Perl extensions. The executable supplied in binary
distributions has a lot of extensions prebuilt, thus the above restriction is 
important only if you use custom-built extensions. This executable is a VIO
application.

I<This is the only executable with does not require OS/2.> The
friends locked into C<M$> world would appreciate the fact that this
executable runs under DOS, Win0.3*, Win0.95 and WinNT with an
appropriate extender. See L<"Other OSes">.

=head2 F<perl__.exe>

This is the same executable as F<perl___.exe>, but it is a PM
application. 

B<Note.> Usually (unless explicitly redirected during the startup)
STDIN, STDERR, and STDOUT of a PM
application are redirected to F<nul>. However, it is possible to I<see>
them if you start C<perl__.exe> from a PM program which emulates a
console window, like I<Shell mode> of Emacs or EPM. Thus it I<is
possible> to use Perl debugger (see L<perldebug>) to debug your PM
application (but beware of the message loop lockups - this will not
work if you have a message queue to serve, unless you hook the serving
into the getc() function of the debugger).

Another way to see the output of a PM program is to run it as

  pm_prog args 2>&1 | cat -

with a shell I<different> from F<cmd.exe>, so that it does not create
a link between a VIO session and the session of C<pm_porg>.  (Such a link
closes the VIO window.)  E.g., this works with F<sh.exe> - or with Perl!

  open P, 'pm_prog args 2>&1 |' or die;
  print while <P>;

The flavor F<perl__.exe> is required if you want to start your program without
a VIO window present, but not C<detach>ed (run C<help detach> for more info).
Very useful for extensions which use PM, like C<Perl/Tk> or C<OpenGL>.

=head2 F<perl___.exe>

This is an C<omf>-style executable which is dynamically linked to
F<perl.dll> and CRT DLL. I know no advantages of this executable
over C<perl.exe>, but it cannot fork() at all. Well, one advantage is
that the build process is not so convoluted as with C<perl.exe>.

It is a VIO application.

=head2 Why strange names?

Since Perl processes the C<#!>-line (cf. 
L<perlrun/DESCRIPTION>, L<perlrun/Switches>,
L<perldiag/"Not a perl script">, 
L<perldiag/"No Perl script found in input">), it should know when a
program I<is a Perl>. There is some naming convention which allows
Perl to distinguish correct lines from wrong ones. The above names are
almost the only names allowed by this convention which do not contain
digits (which have absolutely different semantics).

=head2 Why dynamic linking?

Well, having several executables dynamically linked to the same huge
library has its advantages, but this would not substantiate the
additional work to make it compile. The reason is the complicated-to-developers
but very quick and convenient-to-users "hard" dynamic linking used by OS/2.

There are two distinctive features of the dyna-linking model of OS/2:
all the references to external functions are resolved at the compile time;
there is no runtime fixup of the DLLs after they are loaded into memory.
The first feature is an enormous advantage over other models: it avoids
conflicts when several DLLs used by an application export entries with
the same name.  In such cases "other" models of dyna-linking just choose
between these two entry points using some random criterion - with predictable
disasters as results.  But it is the second feature which requires the build
of F<perl.dll>.

The address tables of DLLs are patched only once, when they are
loaded. The addresses of the entry points into DLLs are guaranteed to be
the same for all the programs which use the same DLL.  This removes the
runtime fixup - once DLL is loaded, its code is read-only.

While this allows some (significant?) performance advantages, this makes life
much harder for developers, since the above scheme makes it impossible
for a DLL to be "linked" to a symbol in the F<.EXE> file.  Indeed, this
would need a DLL to have different relocations tables for the
(different) executables which use this DLL.

However, a dynamically loaded Perl extension is forced to use some symbols
from the perl
executable, e.g., to know how to find the arguments to the functions:
the arguments live on the perl
internal evaluation stack. The solution is to put the main code of
the interpreter into a DLL, and make the F<.EXE> file which just loads
this DLL into memory and supplies command-arguments.  The extension DLL
cannot link to symbols in F<.EXE>, but it has no problem linking
to symbols in the F<.DLL>.

This I<greatly> increases the load time for the application (as well as
complexity of the compilation). Since interpreter is in a DLL,
the C RTL is basically forced to reside in a DLL as well (otherwise
extensions would not be able to use CRT).  There are some advantages if
you use different flavors of perl, such as running F<perl.exe> and
F<perl__.exe> simultaneously: they share the memory of F<perl.dll>.

B<NOTE>.  There is one additional effect which makes DLLs more wasteful:
DLLs are loaded in the shared memory region, which is a scarse resource
given the 512M barrier of the "standard" OS/2 virtual memory.  The code of
F<.EXE> files is also shared by all the processes which use the particular
F<.EXE>, but they are "shared in the private address space of the process";
this is possible because the address at which different sections
of the F<.EXE> file are loaded is decided at compile-time, thus all the
processes have these sections loaded at same addresses, and no fixup
of internal links inside the F<.EXE> is needed.

Since DLLs may be loaded at run time, to have the same mechanism for for DLLs
one needs to have the address range of I<any of the loaded> DLLs in the
system to be available I<in all the processes> which did not load a particular
DLL yet.  This is why the DLLs are mapped to the shared memory region.

=head2 Why chimera build?

Current EMX environment does not allow DLLs compiled using Unixish
C<a.out> format to export symbols for data (or at least some types of
data). This forces C<omf>-style compile of F<perl.dll>.

Current EMX environment does not allow F<.EXE> files compiled in
C<omf> format to fork(). fork() is needed for exactly three Perl
operations:

=over 4

=item *

explicit fork() in the script, 

=item *

C<open FH, "|-">

=item *

C<open FH, "-|">, in other words, opening pipes to itself.

=back

While these operations are not questions of life and death, they are
needed for a lot of
useful scripts. This forces C<a.out>-style compile of
F<perl.exe>.


=head1 ENVIRONMENT

Here we list environment variables with are either OS/2- and DOS- and
Win*-specific, or are more important under OS/2 than under other OSes.

=head2 C<PERLLIB_PREFIX>

Specific for EMX port. Should have the form

  path1;path2

or

  path1 path2

If the beginning of some prebuilt path matches F<path1>, it is
substituted with F<path2>.

Should be used if the perl library is moved from the default
location in preference to C<PERL(5)LIB>, since this would not leave wrong
entries in @INC.  For example, if the compiled version of perl looks for @INC
in F<f:/perllib/lib>, and you want to install the library in
F<h:/opt/gnu>, do

  set PERLLIB_PREFIX=f:/perllib/lib;h:/opt/gnu

This will cause Perl with the prebuilt @INC of

  f:/perllib/lib/5.00553/os2
  f:/perllib/lib/5.00553
  f:/perllib/lib/site_perl/5.00553/os2
  f:/perllib/lib/site_perl/5.00553
  .

to use the following @INC:

  h:/opt/gnu/5.00553/os2
  h:/opt/gnu/5.00553
  h:/opt/gnu/site_perl/5.00553/os2
  h:/opt/gnu/site_perl/5.00553
  .

=head2 C<PERL_BADLANG>

If 0, perl ignores setlocale() failing. May be useful with some
strange I<locale>s.

=head2 C<PERL_BADFREE>

If 0, perl would not warn of in case of unwarranted free(). With older
perls this might be
useful in conjunction with the module DB_File, which was buggy when
dynamically linked and OMF-built.

Should not be set with newer Perls, since this may hide some I<real> problems.

=head2 C<PERL_SH_DIR>

Specific for EMX port. Gives the directory part of the location for
F<sh.exe>.

=head2 C<USE_PERL_FLOCK>

Specific for EMX port. Since L<flock(3)> is present in EMX, but is not 
functional, it is emulated by perl.  To disable the emulations, set 
environment variable C<USE_PERL_FLOCK=0>.

=head2 C<TMP> or C<TEMP>

Specific for EMX port. Used as storage place for temporary files.

=head1 Evolution

Here we list major changes which could make you by surprise.

=head2 Priorities

C<setpriority> and C<getpriority> are not compatible with earlier
ports by Andreas Kaiser. See C<"setpriority, getpriority">.

=head2 DLL name mangling: pre 5.6.2

With the release 5.003_01 the dynamically loadable libraries
should be rebuilt when a different version of Perl is compiled. In particular,
DLLs (including F<perl.dll>) are now created with the names
which contain a checksum, thus allowing workaround for OS/2 scheme of
caching DLLs.

It may be possible to code a simple workaround which would 

=over

=item *

find the old DLLs looking through the old @INC;

=item *

mangle the names according to the scheme of new perl and copy the DLLs to
these names;

=item *

edit the internal C<LX> tables of DLL to reflect the change of the name
(probably not needed for Perl extension DLLs, since the internally coded names
are not used for "specific" DLLs, they used only for "global" DLLs).

=item *

edit the internal C<IMPORT> tables and change the name of the "old"
F<perl????.dll> to the "new" F<perl????.dll>.

=back

=head2 DLL name mangling: 5.6.2 and beyond

In fact mangling of I<extension> DLLs was done due to misunderstanding
of the OS/2 dynaloading model.  OS/2 (effectively) maintains two
different tables of loaded DLL:

=over

=item Global DLLs

those loaded by the base name from C<LIBPATH>; including those
associated at link time;

=item specific DLLs

loaded by the full name.

=back

When resolving a request for a global DLL, the table of already-loaded
specific DLLs is (effectively) ignored; moreover, specific DLLs are
I<always> loaded from the prescribed path.

There is/was a minor twist which makes this scheme fragile: what to do
with DLLs loaded from

=over

=item C<BEGINLIBPATH> and C<ENDLIBPATH>

(which depend on the process)

=item F<.> from C<LIBPATH>

which I<effectively> depends on the process (although C<LIBPATH> is the
same for all the processes).

=back

Unless C<LIBPATHSTRICT> is set to C<T> (and the kernel is after
2000/09/01), such DLLs are considered to be global.  When loading a
global DLL it is first looked in the table of already-loaded global
DLLs.  Because of this the fact that one executable loaded a DLL from
C<BEGINLIBPATH> and C<ENDLIBPATH>, or F<.> from C<LIBPATH> may affect
I<which> DLL is loaded when I<another> executable requests a DLL with
the same name.  I<This> is the reason for version-specific mangling of
the DLL name for perl DLL.

Since the Perl extension DLLs are always loaded with the full path,
there is no need to mangle their names in a version-specific ways:
their directory already reflects the corresponding version of perl,
and @INC takes into account binary compatibility with older version.
Starting from C<5.6.2> the name mangling scheme is fixed to be the
same as for Perl 5.005_53 (same as in a popular binary release).  Thus
new Perls will be able to I<resolve the names> of old extension DLLs
if @INC allows finding their directories.

However, this still does not guarantie that these DLL may be loaded.
The reason is the mangling of the name of the I<Perl DLL>.  And since
the extension DLLs link with the Perl DLL, extension DLLs for older
versions would load an older Perl DLL, and would most probably
segfault (since the data in this DLL is not properly initialized).

There is a partial workaround (which can be made complete with newer
OS/2 kernels): create a forwarder DLL with the same name as the DLL of
the older version of Perl, which forwards the entry points to the
newer Perl's DLL.  Make this DLL accessible on (say) the C<BEGINLIBPATH> of
the new Perl executable.  When the new executable accesses old Perl's
extension DLLs, they would request the old Perl's DLL by name, get the
forwarder instead, so effectively will link with the currently running
(new) Perl DLL.

This may break in two ways:

=over

=item *

Old perl executable is started when a new executable is running has
loaded an extension compiled for the old executable (ouph!).  In this
case the old executable will get a forwarder DLL instead of the old
perl DLL, so would link with the new perl DLL.  While not directly
fatal, it will behave the same as new excutable.  This beats the whole
purpose of explicitly starting an old executable.

=item *

A new executable loads an extension compiled for the old executable
when an old perl executable is running.  In this case the extension
will not pick up the forwarder - with fatal results.

=back

With support for C<LIBPATHSTRICT> this may be circumvented - unless
one of DLLs is started from F<.> from C<LIBPATH> (I do not know
whether C<LIBPATHSTRICT> affects this case).

B<REMARK>.  Unless newer kernels allow F<.> in C<BEGINLIBPATH> (older
do not), this mess cannot be completely cleaned.


B<REMARK>.  C<LIBPATHSTRICT>, C<BEGINLIBPATH> and C<ENDLIBPATH> are
not environment variables, although F<cmd.exe> emulates them on C<SET
...> lines.  From Perl they may be accessed by L<Cwd::extLibpath> and
L<Cwd::extLibpath_set>.

=head2 DLL forwarder generation

Assume that the old DLL is named F<perlE0AC.dll> (as is one for
5.005_53), and the new version is 5.6.1.  Create a file
F<perl5shim.def-leader> with

  LIBRARY 'perlE0AC' INITINSTANCE TERMINSTANCE
  DESCRIPTION '@#perl5-porters@perl.org:5.006001#@ Perl module for 5.00553 -> Perl 5.6.1 forwarder'
  CODE LOADONCALL
  DATA LOADONCALL NONSHARED MULTIPLE
  EXPORTS

modifying the versions/names as needed.  Run

 perl -wnle "next if 0../EXPORTS/; print qq(  \"$1\") if /\"(\w+)\"/" perl5.def >lst

in the Perl build directory (to make the DLL smaller replace perl5.def
with the definition file for the older version of Perl if present).

 cat perl5shim.def-leader lst >perl5shim.def
 gcc -Zomf -Zdll -o perlE0AC.dll perl5shim.def -s -llibperl

(ignore multiple C<warning L4085>).

=head2 Threading

As of release 5.003_01 perl is linked to multithreaded C RTL
DLL.  If perl itself is not compiled multithread-enabled, so will not be perl's
malloc(). However, extensions may use multiple thread on their own
risk. 

This was needed to compile C<Perl/Tk> for XFree86-OS/2 out-of-the-box, and
link with DLLs for other useful libraries, which typically are compiled
with C<-Zmt -Zcrtdll>.

=head2 Calls to external programs

Due to a popular demand the perl external program calling has been
changed wrt Andreas Kaiser's port.  I<If> perl needs to call an
external program I<via shell>, the F<f:/bin/sh.exe> will be called, or
whatever is the override, see L<"PERL_SH_DIR">.

Thus means that you need to get some copy of a F<sh.exe> as well (I
use one from pdksh). The path F<F:/bin> above is set up automatically during
the build to a correct value on the builder machine, but is
overridable at runtime,

B<Reasons:> a consensus on C<perl5-porters> was that perl should use
one non-overridable shell per platform. The obvious choices for OS/2
are F<cmd.exe> and F<sh.exe>. Having perl build itself would be impossible
with F<cmd.exe> as a shell, thus I picked up C<sh.exe>. This assures almost
100% compatibility with the scripts coming from *nix. As an added benefit 
this works as well under DOS if you use DOS-enabled port of pdksh 
(see L<"Prerequisites">).

B<Disadvantages:> currently F<sh.exe> of pdksh calls external programs
via fork()/exec(), and there is I<no> functioning exec() on
OS/2. exec() is emulated by EMX by an asynchronous call while the caller
waits for child completion (to pretend that the C<pid> did not change). This
means that 1 I<extra> copy of F<sh.exe> is made active via fork()/exec(),
which may lead to some resources taken from the system (even if we do
not count extra work needed for fork()ing).

Note that this a lesser issue now when we do not spawn F<sh.exe>
unless needed (metachars found).

One can always start F<cmd.exe> explicitly via

  system 'cmd', '/c', 'mycmd', 'arg1', 'arg2', ...

If you need to use F<cmd.exe>, and do not want to hand-edit thousands of your
scripts, the long-term solution proposed on p5-p is to have a directive

  use OS2::Cmd;

which will override system(), exec(), C<``>, and
C<open(,'...|')>. With current perl you may override only system(),
readpipe() - the explicit version of C<``>, and maybe exec(). The code
will substitute the one-argument call to system() by
C<CORE::system('cmd.exe', '/c', shift)>.

If you have some working code for C<OS2::Cmd>, please send it to me,
I will include it into distribution. I have no need for such a module, so
cannot test it.

For the details of the current situation with calling external programs,
see L<Starting OS/2 (and DOS) programs under Perl>.  Set us mention a couple
of features:

=over 4

=item *

External scripts may be called by their basename.  Perl will try the same
extensions as when processing B<-S> command-line switch.

=item *

External scripts starting with C<#!> or C<extproc > will be executed directly,
without calling the shell, by calling the program specified on the rest of
the first line.

=back

=head2 Memory allocation

Perl uses its own malloc() under OS/2 - interpreters are usually malloc-bound
for speed, but perl is not, since its malloc is lightning-fast.
Perl-memory-usage-tuned benchmarks show that Perl's malloc is 5 times quicker
than EMX one.  I do not have convincing data about memory footprint, but
a (pretty random) benchmark showed that Perl's one is 5% better.

Combination of perl's malloc() and rigid DLL name resolution creates
a special problem with library functions which expect their return value to
be free()d by system's free(). To facilitate extensions which need to call 
such functions, system memory-allocation functions are still available with
the prefix C<emx_> added. (Currently only DLL perl has this, it should 
propagate to F<perl_.exe> shortly.)

=head2 Threads

One can build perl with thread support enabled by providing C<-D usethreads>
option to F<Configure>.  Currently OS/2 support of threads is very 
preliminary.

Most notable problems: 

=over 4

=item C<COND_WAIT> 

may have a race condition.  Needs a reimplementation (in terms of chaining
waiting threads, with the linked list stored in per-thread structure?).

=item F<os2.c>

has a couple of static variables used in OS/2-specific functions.  (Need to be
moved to per-thread structure, or serialized?)

=back

Note that these problems should not discourage experimenting, since they
have a low probability of affecting small programs.

=head1 BUGS

This description was not updated since 5.6.1, see F<os2/Changes> for
more info.

=cut

OS/2 extensions
~~~~~~~~~~~~~~~
I include 3 extensions by Andreas Kaiser, OS2::REXX, OS2::UPM, and OS2::FTP, 
into my ftp directory, mirrored on CPAN. I made
some minor changes needed to compile them by standard tools. I cannot 
test UPM and FTP, so I will appreciate your feedback. Other extensions
there are OS2::ExtAttr, OS2::PrfDB for tied access to EAs and .INI
files - and maybe some other extensions at the time you read it.

Note that OS2 perl defines 2 pseudo-extension functions
OS2::Copy::copy and DynaLoader::mod2fname (many more now, see
L<Prebuilt methods>).

The -R switch of older perl is deprecated. If you need to call a REXX code
which needs access to variables, include the call into a REXX compartment
created by 
        REXX_call {...block...};

Two new functions are supported by REXX code, 
        REXX_eval 'string';
        REXX_eval_with 'string', REXX_function_name => \&perl_sub_reference;

If you have some other extensions you want to share, send the code to
me.  At least two are available: tied access to EA's, and tied access
to system databases.

=head1 AUTHOR

Ilya Zakharevich, ilya@math.ohio-state.edu

=head1 SEE ALSO

perl(1).

=cut