applied suggested patch (mailed to perl-unicode@perl.org) with minor tweaks

[perl5.git] / pod / perltodo.pod

=head1 Perl TO-DO List

This is a list of wishes for Perl.  It is maintained by Nathan
Torkington for the Perl porters.  Send updates to
I<perl5-porters@perl.org>.  If you want to work on any of these
projects, be sure to check the perl5-porters archives for past ideas,
flames, and propaganda.  This will save you time and also prevent you
from implementing something that Larry has already vetoed.  One set
of archives may be found at:

    http://www.xray.mpe.mpg.de/mailing-lists/perl5-porters/


=head1 Infrastructure

=head2 Mailing list archives

Chaim suggests contacting egroup and asking them to archive the other
perl.org mailing lists.  Probably not advocacy, but definitely
perl6-porters, etc.

=head2 Bug tracking system

Richard Foley I<richard@perl.org> is writing one.  We looked at
several, like gnats and the Debian system, but at the time we
investigated them, none met our needs.  Since then, Jitterbug has
matured, and may be worth reinvestigation.

The system we've developed will eventually be recipient of perlbug
mail.  New bugs are entered into a mysql database, and sent on to
perl5-porters with the subject line rewritten to include a "ticket
number" (unique ID for the new bug).  If the incoming message already
had a ticket number in the subject line, then the message is logged
against that bug.  There is a separate email interface (not forwarding
to p5p) that permits porters to claim, categorize, and close tickets.

The next desire is a web interface.  It is hoped that code can be
reused between the mail and the web interfaces.

The current delay in implementation is caused by perl.org lockups.
One suspect is the mail handling system, possibly going into loops.

We're probably going to need a bugmaster, someone who will look at
every new "bug" and kill those that we already know about, those
that are not bugs at all, etc.

=head2 Regression Tests

The test suite for Perl serves two needs: ensuring features work, and
ensuring old bugs have not been reintroduced.  Both need work.

Brent LaVelle (lavelle@metronet.com) has stepped forward to work on
performance tests and improving the size of the test suite.

=over 4

=item Coverage

Do the tests that come with Perl exercise every line (or every block,
or ...)  of the Perl interpreter, and if not then how can we make them
do so?

=item Regression

No bug fixes should be made without a corresponding testsuite addition.
This needs a dedicated enforcer, as the current pumpking is either too
lazy or too stupid or both and lets enforcement wander all over the
map.  :-)

=item __DIE__

Tests that fail need to be of a form that can be readily mailed
to perlbug and diagnosed with minimal back-and-forth's to determine
which test failed, due to what cause, etc.

=item suidperl

We need regression/sanity tests for suidperl

=item The 25% slowdown from perl4 to perl5

This value may or may not be accurate, but it certainly is
eye-catching.  For some things perl5 is faster than perl4, but often
the reliability and extensability have come at a cost of speed.  The
benchmark suite that Gisle released earlier has been hailed as both a
fantastic solution and as a source of entirely meaningless figures.
Do we need to test "real applications"?  Can you do so?  Anyone have
machines to dedicate to the task?  Identify the things that have grown
slower, and see if there's a way to make them faster.

=back

=head1 Configure

Andy Dougherty maintain(ed|s) a list of "todo" items for the configure
that comes with Perl.  See Porting/pumpkin.pod in the latest
source release.

=head2 Install HTML

Have "make install" give you the option to install HTML as well.  This
would be part of Configure.  Andy Wardley (certified Perl studmuffin)
will look into the current problems of HTML installation--is
'installhtml' preventing this from happening cleanly, or is pod2html
the problem?  If the latter, Brad Appleton's pod work may fix the
problem for free.

=head1 Perl Language

=head2 our ($var)

Declare global variables (lexically or otherwise).

=head2 64-bit Perl

Verify complete 64 bit support so that the value of sysseek, or C<-s>, or
stat(), or tell can fit into a perl number without losing precision.
Work with the perl-64bit mailing list on perl.org.

=head2 Prototypes

=over 4

=item Named prototypes

Add proper named prototypes that actually work usefully.

=item Indirect objects

Fix prototype bug that forgets indirect objects.

=item Method calls

Prototypes for method calls.

=item Context

Return context prototype declarations.

=item Scoped subs

lexically-scoped subs, e.g. my sub

=back

=head2 Built-in globbing

Currently the C<E<lt>*.cE<gt>> syntax calls the c shell.  This causes
problems on sites without csh, systems where fork() is expensive, and
setuid environments.  Decide between Glob::BSD and File::KGlob, move
it into the core, and make Perl use it for globbing.  Ben Holzman and
Tye McQueen have claimed the pumpkin for this.

=head1 Perl Internals

=head2 magic_setisa

C<magic_setisa> should be made to update %FIELDS [???]

=head2 Garbage Collection

There was talk of a mark-and-sweep garbage collector at TPC2, but the
(to users) unpredictable nature of its behaviour put some off.
Sarathy, I believe, did the work.  Here's what he has to say:

Yeah, I hope to implement it someday too.  The points that were
raised in TPC2 were all to do with calling DESTROY() methods, but
I think we can accomodate that by extending bless() to stash
extra information for objects so we track their lifetime accurately
for those that want their DESTROY() to be predictable (this will be
a speed hit, naturally, and will therefore be optional, naturally. :)
    
[N.B. Don't even ask me about this now!  When I have the time to
write a cogent summary, I'll post it.]

=head2 Reliable signals

Sarathy and Dan Sugalski are working on this.  Chip posted a patch
earlier, but it was not accepted into 5.005.  The issue is tricky,
because it has the potential to greatly slow down the core.

There are at least three things to consider:

=over 4

=item Alternate runops() for signal despatch

Sarathy and Dan are discussed this on perl5-porters.

=item Figure out how to die() in delayed sighandler

=item Add tests for Thread::Signal

=item Automatic tests against CPAN

Is there some way to automatically build all/most of CPAN with
the new Perl and check that the modules there pass all the tests?

=back

=head2 Interpolated regex performance bugs

  while (<>) {
    $found = 0;
    foreach $pat (@patterns) {
      $found++ if /$pat/o;
    }
    print if $found;
  }

The qr// syntax added in 5.005 has solved this problem, but
it needs more thorough documentation.

=head2 Memory leaks from failed eval/regcomp

The only known memory leaks in Perl are in failed code or regexp
compilation.  Fix this.  Hugo Van Der Sanden will attempt this but
won't have tuits until January 1999.

=head2 Make XS easier to use

There was interest in SWIG from porters, but nothing has happened
lately.

=head2 Make embedded Perl easier to use

This is probably difficult for the same reasons that "XS For Dummies"
will be difficult.

=head2 Namespace cleanup

    CPP-space:    restrict CPP symbols exported from headers
    header-space: move into CORE/perl/
    API-space:    begin list of things that constitute public api
    env-space:    Configure should use PERL_CONFIG instead of CONFIG etc.

=head2 MULTIPLICITY

Complete work on safe recursive interpreters C<Perl-E<gt>new()>.
Sarathy says that a reference implementation exists.

=head2 MacPerl

Chris Nandor and Matthias Neeracher are working on better integrating
MacPerl into the Perl distribution.

=head1 Documentation

There's a lot of documentation that comes with Perl.  The quantity of
documentation makes it difficult for users to know which section of
which manpage to read in order to solve their problem.  Tom
Christiansen has done much of the documentation work in the past.

=head2 A clear division into tutorial and reference

Some manpages (e.g., perltoot and perlreftut) clearly set out to
educate the reader about a subject.  Other manpages (e.g., perlsub)
are references for which there is no tutorial, or are references with
a slight tutorial bent.  If things are either tutorial or reference,
then the reader knows which manpage to read to learn about a subject,
and which manpage to read to learn all about an aspect of that
subject.  Part of the solution to this is:

=head2 Remove the artificial distinction between operators and functions

History shows us that users, and often porters, aren't clear on the
operator-function distinction.  The present split in reference
material between perlfunc and perlop hinders user navigation.  Given
that perlfunc is by far the larger of the two, move operator reference
into perlfunc.

=head2 More tutorials

More documents of a tutorial nature could help.  Here are some
candidates:

=over 4

=item Regular expressions

Robin Berjon (r.berjon@ltconsulting.net) has volunteered.

=item I/O

Mark-Jason Dominus (mjd@plover.com) has an outline for perliotut.

=item pack/unpack

This is badly needed.  There has been some discussion on the
subject on perl5-porters.

=item Debugging

Ronald Kimball (rjk@linguist.dartmouth.edu) has volunteered.

=head2 Include a search tool

perldoc should be able to 'grep' fulltext indices of installed POD
files.  This would let people say:

  perldoc -find printing numbers with commas

and get back the perlfaq entry on 'commify'.

This solution, however, requires documentation to contain the keywords
the user is searching for.  Even when the users know what they're
looking for, often they can't spell it.

=head2 Include a locate tool

perldoc should be able to help people find the manpages on a
particular high-level subject:

  perldoc -find web

would tell them manpages, web pages, and books with material on web
programming.  Similarly C<perldoc -find databases>, C<perldoc -find
references> and so on.

We need something in the vicinity of:

  % perl -help random stuff
  No documentation for perl function `random stuff' found
  The following entry in perlfunc.pod matches /random/a:
    =item rand EXPR
     
    =item rand
     
    Returns a random fractional number greater than or equal to C<0> and less
    than the value of EXPR.  (EXPR should be positive.)  If EXPR is
    omitted, the value C<1> is used.  Automatically calls C<srand()> unless
    C<srand()> has already been called.  See also C<srand()>.
     
    (Note: If your rand function consistently returns numbers that are too
    large or too small, then your version of Perl was probably compiled
    with the wrong number of RANDBITS.)
  The following pod pages seem to have /stuff/a:
    perlfunc.pod	(7 hits)
    perlfaq7.pod	(6 hits)
    perlmod.pod		(4 hits)
    perlsyn.pod		(3 hits)
    perlfaq8.pod	(2 hits)
    perlipc.pod		(2 hits)
    perl5004delta.pod	(1 hit)
    perl5005delta.pod	(1 hit)
    perlcall.pod	(1 hit)
    perldelta.pod	(1 hit)
    perlfaq3.pod	(1 hit)
    perlfaq5.pod	(1 hit)
    perlhist.pod	(1 hit)
    perlref.pod		(1 hit)
    perltoc.pod		(1 hit)
    perltrap.pod	(1 hit)
  Proceed to open perlfunc.pod? [y] n
  Do you want to speak perl interactively? [y] n
  Should I dial 911? [y] n
  Do you need psychiatric help? [y] y
  <PELIZA> Hi, what bothers you today?
           A Python programmer in the next cubby is driving me nuts!
  <PELIZA> Hmm, thats fixable.  Just [rest censored]

=head2 Separate function manpages by default

Perl should install 'manpages' for every function/operator into the
3pl or 3p manual section.  By default.  The splitman program in the
Perl source distribution does the work of turning big perlfunc into
little 3p pages.

=head2 Users can't find the manpages

Make C<perldoc> tell users what they need to add to their .login or
.cshrc to set their MANPATH correctly.

=head2 Install ALL Documentation

Make the standard documentation kit include the VMS, OS/2, Win32,
Threads, etc information.  installperl and pod/Makefile should know
enough to copy README.foo to perlfoo.pod before building everything,
when appropriate.

=head2 Outstanding issues to be documented

Tom has a list of 5.005_5* features or changes that require
documentation.

Create one document that coherently explains the delta between the
last camel release and the current release.  perldelta was supposed
to be that, but no longer.  The things in perldelta never seemed to
get placed in the right places in the real manpages, either.  This
needs work.

=head2 Adapt www.linuxhq.com for Perl

This should help glorify documentation and get more people involved in
perl development.

=head2 Replace man with a perl program

Can we reimplement man in Perl?  Tom has a start.  I believe some of
the Linux systems distribute a manalike.  Alternatively, build on
perldoc to remove the unfeatures like "is slow" and "has no apropos".

=head2 Unicode tutorial

We could use more work on helping people understand Perl's new
Unicode support that Larry has created.

=head1 Modules

=head2 Update the POSIX extension to conform with the POSIX 1003.1 Edition 2

The current state of the POSIX extension is as of Edition 1, 1991,
whereas the Edition 2 came out in 1996.  ISO/IEC 9945:1-1996(E),
ANSI/IEEE Std 1003.1, 1996 Edition. ISBN 1-55937-573-6.  The updates
were legion: threads, IPC, and real time extensions.

=head2 Module versions

Automate the checking of versions in the standard distribution so
it's easy for a pumpking to check whether CPAN has a newer version
that we should be including?

=head2 New modules

Which modules should be added to the standard distribution?  This ties
in with the SDK discussed on the perl-sdk list at perl.org.

=head2 Profiler

Make the profiler (Devel::DProf) part of the standard release, and
document it well.

=head2 Tie Modules

=over 4

=item VecArray

Implement array using vec().  Nathan Torkington has working code to
do this.

=item SubstrArray

Implement array using substr()

=item VirtualArray

Implement array using a file

=item ShiftSplice

Defines shift et al in terms of splice method

=back

=head2 Procedural options

Support procedural interfaces for the common cases of Perl's
gratuitously OOO modules.  Tom objects to "use IO::File" reading many
thousands of lines of code.

=head2 RPC

Write a module for transparent, portable remote procedure calls.  (Not
core).  This touches on the CORBA and ILU work.

=head2 y2k localtime/gmtime

Write a module, Y2k::Catch, which overloads localtime and gmtime's
returned year value and catches "bad" attempts to use it.

=head2 Export File::Find variables

Make File::Find export C<$name> etc manually, at least if asked to.

=head2 Ioctl

Finish a proper Ioctl module.

=head2 Debugger attach/detach

Permit a user to debug an already-running program.

=head2 Regular Expression debugger

Create a visual profiler/debugger tool that stepped you through the
execution of a regular expression point by point.  Ilya has a module
to color-code and display regular expression parses and executions.
There's something at http://tkworld.org/ that might be a good start,
it's a Tk/Tcl RE wizard, that builds regexen of many flavours.

=head2 Alternative RE Syntax

Make an alternative regular expression syntax that is accessed through
a module.  For instance,

  use RE;
  $re = start_of_line()
      ->literal("1998/10/08")
      ->optional( whitespace() )
      ->literal("[")
      ->remember( many( or( "-", digit() ) ) );

  if (/$re/) {
    print "time is $1\n";
  }

Newbies to regular expressions typically only use a subset of the full
language.  Perhaps you wouldn't have to implement the full feature set.

=head2 Bundled modules

Nicholas Clark (nick@flirble.org) had a patch for storing modules in
zipped format.  This needs exploring and concluding.

=head2 Expect

Adopt IO::Tty, make it as portable as Don Libes' "expect" (can we link
against expect code?), and perfect a Perl version of expect.  IO::Tty
and expect could then be distributed as part of the core distribution,
replacing Comm.pl and other hacks.

=head2 GUI::Native

A simple-to-use interface to native graphical abilities would
be welcomed.  Oh, Perl's access Tk is nice enough, and reasonably
portable, but it's not particularly as fast as one would like.
Simple access to the mouse's cut buffer or mouse-presses shouldn't
required loading a few terabytes of Tk code.

=head2 Update semibroken auxiliary tools; h2ph, a2p, etc.

Kurt Starsinic is working on h2ph.  mjd has fixed bugs in a2p in the
past.  a2p apparently doesn't work on nawk and gawk extensions.
Graham Barr has an Include module that does h2ph work at runtime.

=head2 POD Converters

Brad's PodParser code needs to become part of the core, and the Pod::*
and pod2* programs rewritten to use this standard parser.  Currently
the converters take different options, some behave in different
fashions, and some are more picky than others in terms of the POD
files they accept.

=head2 pod2html

A short-term fix: pod2html generates absolute HTML links.  Make it
generate relative links.

=head2 Podchecker

Something like lint for Pod would be good.  Something that catches
common errors as well as gross ones.  Brad Appleton is putting
together something as part of his PodParser work.

=head1 Tom's Wishes

=head2 Webperl

Design a webperl environment that's as tightly integrated and as
easy-to-use as Perl's current command-line environment.

=head2 Mobile agents

More work on a safe and secure execution environment for mobile
agents would be neat; the Safe.pm module is a start, but there's a
still a lot to be done in that area.  Adopt Penguin?

=head2 POSIX on non-POSIX

Standard programming constructs for non-POSIX systems would help a
lot of programmers stuck on primitive, legacy systems.  For example,
Microsoft still hasn't made a usable POSIX interface on their clunky
systems, which means that standard operations such as alarm() and
fork(), both critical for sophisticated client-server programming,
must both be kludged around.

I'm unsure whether Tom means to emulate alarm( )and fork(), or merely
to provide a document like perlport.pod to say which features are
portable and which are not.

=head2 Portable installations

Figure out a portable semi-gelled installation, that is, one without
full paths.  Larry has said that he's thinking about this.  Ilya
pointed out that perllib_mangle() is good for this.

=head1 Win32 Stuff

=head2 Get PERL_OBJECT building under gcc

B<Part done>, according to Sarathy.  It builds under egcs on win32,
but doesn't run for occult reasons.  If anyone knows the right
breed of chicken to sacrifice, please speak up.

=head2 Rename new headers to be consistent with the rest

=head2 Sort out the spawnvp() mess

=head2 Work out DLL versioning

=head2 Get PERL_OBJECT building on non-win32

=head2 Style-check

=head1 Would be nice to have

=over 4

=item C<pack "(stuff)*">

=item Contiguous bitfields in pack/unpack

=item lexperl

=item Bundled perl preprocessor

=item Use posix calls internally where possible

=item format BOTTOM

=item -i rename file only when successfully changed

=item All ARGV input should act like <>

=item report HANDLE [formats].

=item support in perlmain to rerun debugger

=item lvalue functions

Tuomas Lukka, on behalf of the PDL project, greatly desires this and
Ilya has a patch for it (probably against an older version of Perl).
Tuomas points out that what PDL really wants is lvalue I<methods>,
not just subs.

=back

=head1 Possible pragmas

=head2 'less'

(use less memory, CPU)

=head1 Optimizations

=head2 constant function cache

=head2 foreach(reverse...)

=head2 Cache eval tree

Unless lexical outer scope used (mark in &compiling?).

=head2 rcatmaybe

=head2 Shrink opcode tables

Via multiple implementations selected in peep.

=head2 Cache hash value

Not a win, according to Guido.

=head2 Optimize away @_ where possible

=head2 Optimize sort by { $a <=> $b }

Greg Bacon added several more sort optimizations.  These have
made it into 5.005_55, thanks to Hans Mulder.

=head2 Rewrite regexp parser for better integrated optimization

The regexp parser was rewritten for 5.005.  Ilya's the regexp guru.

=head1 Vague possibilities

=over 4

=item ref function in list context

This seems impossible to do without substantially breaking code.

=item make tr/// return histogram in list context?

=item Loop control on do{} et al

=item Explicit switch statements

Nobody has yet managed to come up with a switch syntax that would
allow for mixed hash, constant, regexp checks.  Submit implementation
with syntax, please.

=item compile to real threaded code

=item structured types

=item Modifiable $1 et al

The intent is for this to be a means of editing the matched portions of
the target string.

=back

=head1 To Do Or Not To Do

These are things that have been discussed in the past and roundly
criticized for being of questionable value.

=head2 Making my() work on "package" variables

Being able to say my($Foo::Bar), something that sounds ludicrous and
the 5.006 pumpking has mocked.

=head2 "or" testing defined not truth

We tell people that C<||> can be used to give a default value to a
variable:

    $children = shift || 5;		# default is 5 children

which is almost (but not):

    $children = shift;
    $children = 5 unless $children;

but if the first argument was given and is "0", then it will be
considered false by C<||> and C<5> used instead.  Really we want
an C<||>-like operator that behaves like:

    $children = shift;
    $children = 5 unless defined $children;

Namely, a C<||> that tests defined-ness rather than truth.  One was
discussed, and a patch submitted, but the objections were many.  While
there were objections, many still feel the need.  At least it was
decided that C<??> is the best name for the operator.

=head2 "dynamic" lexicals

  my $x;
  sub foo {
    local $x;
  }

Localizing, as Tim Bunce points out, is a separate concept from
whether the variable is global or lexical.  Chip Salzenberg had
an implementation once, but Larry thought it had potential to
confuse.

=head2 "class"-based, rather than package-based "lexicals"

This is like what the Alias module provides, but the variables would
be lexicals reserved by perl at compile-time, which really are indices
pointing into the pseudo-hash object visible inside every method so
declared.

=head1 Threading

=head2 Modules

Which of the standard modules are thread-safe?  Which CPAN modules?
How easy is it to fix those non-safe modules?

=head2 Testing

Threading is still experimental.  Every reproducible bug identifies
something else for us to fix.  Find and submit more of these problems.

=head2 $AUTOLOAD

=head2 exit/die

Consistent semantics for exit/die in threads.

=head2 External threads

Better support for externally created threads.

=head2 Thread::Pool

=head2 thread-safety

Spot-check globals like statcache and global GVs for thread-safety.
"B<Part done>", says Sarathy.

=head2 Per-thread GVs

According to Sarathy, this would make @_ be the same in threaded
and non-threaded, as well as helping solve problems like filehandles
(the same filehandle currently cannot be used in two threads).

=head1 Compiler

=head2 Optimization

The compiler's back-end code-generators for creating bytecode or
compilable C code could use optimization work.

=head2 Byteperl

Figure out how and where byteperl will be built for the various
platforms.

=head2 Precompiled modules

Save byte-compiled modules on disk.

=head2 Executables

Auto-produce executable.

=head2 Typed lexicals

Typed lexicals should affect B::CC::load_pad.

=head2 Win32

Workarounds to help Win32 dynamic loading.

=head2 END blocks

END blocks need saving in compiled output.

=head2 _AUTOLOAD

_AUTOLOAD prodding.

=head2 comppadlist

Fix comppadlist (names in comppad_name can have fake SvCUR
from where newASSIGNOP steals the field).

=head2 Cached compilation

Can we install modules as bytecode?

=head1 Recently Finished Tasks

=head2 Figure a way out of $^(capital letter)

Figure out a clean way to extend $^(capital letter) beyond
the 26 alphabets.  (${^WORD} maybe?)

Mark-Jason Dominus sent a patch which went into 5.005_56.

=head2 Filenames

Make filenames in the distribution and in the standard module set
be 8.3 friendly where feasible.  Good luck changing the standard
modules, though.  B<Done>.

=head2 Proper tied array support

This was B<done> in 5.005 by Nick Ing-Simmons.

=head2 Foreign lines

Perl should be more generous in accepting foreign line terminations.
Mostly B<done> in 5.005.

=head2 Namespace cleanup

    symbol-space: "pl_" prefix for all global vars
                  "Perl_" prefix for all functions

    CPP-space:	  stop malloc()/free() pollution unless asked

=head2 Explain tool

Given a piece of Perl code, say what it does.  B::Deparse is doing
this.  B<Done>.

=head2 ISA.pm

Rename and alter ISA.pm.  B<Done>.  It is now base.pm.

=head2 Automate maintenance of most PERL_OBJECT code

B<Done>, says Sarathy.

=head2 -iprefix.

Added in 5.004_70.  B<Done>

=head2 gettimeofday

See Time::HiRes.

=head2 reference to compiled regexp

B<done>  This is the qr// support in 5.005.

=head2 eval qw() at compile time

qw() is presently compiled as a call to split.  This means the split
happens at runtime.  Change this so qw() is compiled as a real list
assignment.  This also avoids surprises like:

    $a = () = qw(What will $a hold?);

B<Done>.  Tom Hughes submitted a patch that went into 5.005_55.

=head2 autocroak?

B<Done>.  This is the Fatal.pm module, so any builtin that that does
not return success automatically die()s.  If you're feeling brave, tie
this in with the unified exceptions scheme.

=head2 Status variable

$^C to track compiler/checker status.  B<Done> in 5.005_54.

=cut