=head1 DESCRIPTION
-This is a list of wishes for Perl. Send updates to
-I<perl5-porters@perl.org>. If you want to work on any of these
-projects, be sure to check the perl5-porters archives for past ideas,
-flames, and propaganda. This will save you time and also prevent you
-from implementing something that Larry has already vetoed. One set
-of archives may be found at:
+This is a list of wishes for Perl. The tasks we think are smaller or easier
+are listed first. Anyone is welcome to work on any of these, but it's a good
+idea to first contact I<perl5-porters@perl.org> to avoid duplication of
+effort. By all means contact a pumpking privately first if you prefer.
- http://www.xray.mpe.mpg.de/mailing-lists/perl5-porters/
-
-=head1 To do during 5.6.x
-
-=head2 Support for I/O disciplines
-
-C<perlio> provides this, but the interface could be a lot more
-straightforward.
-
-=head2 Autoload bytes.pm
+Whilst patches to make the list shorter are most welcome, ideas to add to
+the list are also encouraged. Check the perl5-porters archives for past
+ideas, and any discussion about them. One set of archives may be found at:
-When the lexer sees, for instance, C<bytes::length>, it should
-automatically load the C<bytes> pragma.
+ http://www.xray.mpe.mpg.de/mailing-lists/perl5-porters/
-=head2 Make "\u{XXXX}" et al work
+What can we offer you in return? Fame, fortune, and everlasting glory? Maybe
+not, but if your patch is incorporated, then we'll add your name to the
+F<AUTHORS> file, which ships in the official distribution. How many other
+programming languages offer you 1 line of immortality?
-Danger, Will Robinson! Discussing the semantics of C<"\x{F00}">,
-C<"\xF00"> and C<"\U{F00}"> on P5P I<will> lead to a long and boring
-flamewar.
+=head1 The roadmap to 5.10
-=head2 Create a char *sv_pvprintify(sv, STRLEN *lenp, UV flags)
+The roadmap to 5.10 envisages feature based releases, as various items in this
+TODO are completed.
-For displaying PVs with control characters, embedded nulls, and Unicode.
-This would be useful for printing warnings, or data and regex dumping,
-not_a_number(), and so on.
+=head2 Needed for a 5.9.3 release
-Requirements: should handle both byte and UTF8 strings. isPRINT()
-characters printed as-is, character less than 256 as \xHH, Unicode
-characters as \x{HHH}. Don't assume ASCII-like, either, get somebody
-on EBCDIC to test the output.
+=over
-Possible options, controlled by the flags:
-- whitespace (other than ' ' of isPRINT()) printed as-is
-- use isPRINT_LC() instead of isPRINT()
-- print control characters like this: "\cA"
-- print control characters like this: "^A"
-- non-PRINTables printed as '.' instead of \xHH
-- use \OOO instead of \xHH
-- use the C/Perl-metacharacters like \n, \t
-- have a maximum length for the produced string (read it from *lenp)
-- append a "..." to the produced string if the maximum length is exceeded
-- really fancy: print unicode characters as \N{...}
+=item *
+Implement L</lexical pragmas>
-=head2 Overloadable regex assertions
+=back
-This may or may not be possible with the current regular expression
-engine. The idea is that, for instance, C<\b> needs to be
-algorithmically computed if you're dealing with Thai text. Hence, the
-B<\b> assertion wants to be overloaded by a function.
+=head2 Needed for a 5.9.4 release
-=head2 Unicode
-
-=over 4
+=over
=item *
+Review assertions. Review syntax to combine assertions. Can assertions take
+advantage of the lexical pragams work? L</What hooks would assertions need?>
-Allow for long form of the General Category Properties, e.g
-C<\p{IsOpenPunctuation}>, not just the abbreviated form, e.g.
-C<\p{IsPs}>.
-
-=item *
+=back
-Allow for the metaproperties: C<XID Start>, C<XID Continue>,
-C<NF*_NO>, C<NF*_MAYBE> (require the DerivedCoreProperties and
-DerviceNormalizationProperties files).
+=head2 Needed for a 5.9.5 release
-There are also enumerated properties: C<Decomposition Type>,
-C<Numeric Type>, C<East Asian Width>, C<Line Break>. These
-properties have multiple values: for uniqueness the property
-value should be appended. For example, C<\p{IsAlphabetic}>
-wouldbe the binary property, while C<\p{AlphabeticLineBreak}>
-would mean the enumerated property.
+=over
=item *
-
- Case Mappings? http://www.unicode.org/unicode/reports/tr21/
-
-lc(), uc(), lcfirst(), and ucfirst() work only for some of the
-simplest cases, where the mapping goes from a single Unicode character
-to another single Unicode character. See lib/unicore/SpecCase.txt
-(and CaseFold.txt).
+Implement L</_ prototype character>
=item *
-
-They have some tricks Perl doesn't yet implement like character
-class subtraction.
-
- http://www.unicode.org/unicode/reports/tr18/
+Implement L</state variables>
=back
-See L<perlunicode/UNICODE REGULAR EXPRESSION SUPPORT LEVEL> for what's
-there and what's missing. Almost all of Levels 2 and 3 is missing,
-and as of 5.8.0 not even all of Level 1 is there.
+=head2 Needed for a 5.9.6 release
-=head2 use Thread for iThreads
+Stabilisation. If all goes well, this will be the equivalent of a 5.10-beta.
-Artur Bergman's C<iThreads> module is a start on this, but needs to
-be more mature.
+=head1 Tasks that only need Perl knowledge
-=head2 make perl_clone optionally clone ops
+=head2 common test code for timed bail out
-So that pseudoforking, mod_perl, iThreads and nvi will work properly
-(but not as efficiently) until the regex engine is fixed to be threadsafe.
+Write portable self destruct code for tests to stop them burning CPU in
+infinite loops. This needs to avoid using alarm, as some of the tests are
+testing alarm/sleep or timers.
-=head2 Work out exit/die semantics for threads
+=head2 POD -> HTML conversion in the core still sucks
-=head2 Typed lexicals for compiler
+Which is crazy given just how simple POD purports to be, and how simple HTML
+can be. It's not actually I<as> simple as it sounds, particularly with the
+flexibility POD allows for C<=item>, but it would be good to improve the
+visual appeal of the HTML generated, and to avoid it having any validation
+errors. See also L</make HTML install work>, as the layout of installation tree
+is needed to improve the cross-linking.
-=head2 Compiler workarounds for Win32
+=head2 Parallel testing
-=head2 AUTOLOADing in the compiler
+The core regression test suite is getting ever more comprehensive, which has
+the side effect that it takes longer to run. This isn't so good. Investigate
+whether it would be feasible to give the harness script the B<option> of
+running sets of tests in parallel. This would be useful for tests in
+F<t/op/*.t> and F<t/uni/*.t> and maybe some sets of tests in F<lib/>.
-=head2 Fixing comppadlist when compiling
+Questions to answer
-=head2 Cleaning up exported namespace
-
-=head2 Complete signal handling
-
-Add C<PERL_ASYNC_CHECK> to opcodes which loop; replace C<sigsetjmp> with
-C<sigjmp>; check C<wait> for signal safety.
-
-=head2 Out-of-source builds
-
-This was done for 5.6.0, but needs reworking for 5.7.x
-
-=head2 POSIX realtime support
-
-POSIX 1003.1 1996 Edition support--realtime stuff: POSIX semaphores,
-message queues, shared memory, realtime clocks, timers, signals (the
-metaconfig units mostly already exist for these)
-
-=head2 UNIX98 support
-
-Reader-writer locks, realtime/asynchronous IO
-
-=head2 IPv6 Support
-
-There are non-core modules, such as C<Net::IPv6>, but these will need
-integrating when IPv6 actually starts to really happen. See RFC 2292
-and RFC 2553.
-
-=head2 Long double conversion
-
-Floating point formatting is still causing some weird test failures.
-
-=head2 Locales
-
-Locales and Unicode interact with each other in unpleasant ways.
-One possible solution would be to adopt/support ICU:
+=over 4
- http://oss.software.ibm.com/developerworks/opensource/icu/project/
+=item 1
-=head2 Thread-safe regexes
+How does screen layout work when you're running more than one test?
-The regular expression engine is currently non-threadsafe.
+=item 2
-=head2 Arithmetic on non-Arabic numerals
+How does the caller of test specify how many tests to run in parallel?
-C<[1234567890]> aren't the only numerals any more.
+=item 3
-=head2 POSIX Unicode character classes
+How do setup/teardown tests identify themselves?
-([=a=] for equivalance classes, [.ch.] for collation.)
-These are dependent on Unicode normalization and collation.
+=back
-=head2 Factoring out common suffices/prefices in regexps (trie optimization)
+Pugs already does parallel testing - can their approach be re-used?
-Currently, the user has to optimize C<foo|far> and C<foo|goo> into
-C<f(?:oo|ar)> and C<[fg]oo> by hand; this could be done automatically.
+=head2 Make Schwern poorer
-=head2 Security audit shipped utilities
+We should have for everything. When all the core's modules are tested,
+Schwern has promised to donate to $500 to TPF. We may need volunteers to
+hold him upside down and shake vigorously in order to actually extract the
+cash.
-All the code we ship with Perl needs to be sensible about temporary file
-handling, locking, input validation, and so on.
+See F<t/lib/1_compile.t> for the 3 remaining modules that need tests.
-=head2 Custom opcodes
+=head2 Improve the coverage of the core tests
-Have a way to introduce user-defined opcodes without the subroutine call
-overhead of an XSUB; the user should be able to create PP code. Simon
-Cozens has some ideas on this.
+Use Devel::Cover to ascertain the core's test coverage, then add tests that
+are currently missing.
-=head2 spawnvp() on Win32
+=head2 test B
-Win32 has problems spawning processes, particularly when the arguments
-to the child process contain spaces, quotes or tab characters.
+A full test suite for the B module would be nice.
-=head2 DLL Versioning
+=head2 A decent benchmark
-Windows needs a way to know what version of a XS or C<libperl> DLL it's
-loading.
+C<perlbench> seems impervious to any recent changes made to the perl core. It
+would be useful to have a reasonable general benchmarking suite that roughly
+represented what current perl programs do, and measurably reported whether
+tweaks to the core improve, degrade or don't really affect performance, to
+guide people attempting to optimise the guts of perl. Gisle would welcome
+new tests for perlbench.
-=head2 Introduce @( and @)
+=head2 fix tainting bugs
-C<$(> may return "foo bar baz". Unfortunately, since groups can
-theoretically have spaces in their names, this could be one, two or
-three groups.
+Fix the bugs revealed by running the test suite with the C<-t> switch (via
+C<make test.taintwarn>).
-=head2 Floating point handling
+=head2 Dual life everything
-C<NaN> and C<inf> support is particularly troublesome.
-(fp_classify(), fp_class(), fp_class_d(), class(), isinf(),
-isfinite(), finite(), isnormal(), unordered(), <ieeefp.h>,
-<fp_class.h> (there are metaconfig units for all these) (I think),
-fp_setmask(), fp_getmask(), fp_setround(), fp_getround()
-(no metaconfig units yet for these). Don't forget finitel(), fp_classl(),
-fp_class_l(), (yes, both do, unfortunately, exist), and unorderedl().)
+As part of the "dists" plan, anything that doesn't belong in the smallest perl
+distribution needs to be dual lifed. Anything else can be too. Figure out what
+changes would be needed to package that module and its tests up for CPAN, and
+do so. Test it with older perl releases, and fix the problems you find.
-As of Perl 5.6.1 is a Perl macro, Perl_isnan().
+=head2 Improving C<threads::shared>
-=head2 IV/UV preservation
+Investigate whether C<threads::shared> could share aggregates properly with
+only Perl level changes to shared.pm
-Nicholas Clark has done a lot of work on this, but work is continuing.
-C<+>, C<-> and C<*> work, but guards need to be in place for C<%>, C</>,
-C<&>, C<oct>, C<hex> and C<pack>.
+=head2 POSIX memory footprint
-=head2 Replace pod2html with something using Pod::Parser
+Ilya observed that use POSIX; eats memory like there's no tomorrow, and at
+various times worked to cut it down. There is probably still fat to cut out -
+for example POSIX passes Exporter some very memory hungry data structures.
-The CPAN module C<Malik::Pod::Html> may be a more suitable basis for a
-C<pod2html> convertor; the current one duplicates the functionality
-abstracted in C<Pod::Parser>, which makes updating the POD language
-difficult.
-=head2 Automate module testing on CPAN
-When a new Perl is being beta tested, porters have to manually grab
-their favourite CPAN modules and test them - this should be done
-automatically.
-=head2 sendmsg and recvmsg
-We have all the other BSD socket functions but these. There are
-metaconfig units for these functions which can be added. To avoid these
-being new opcodes, a solution similar to the way C<sockatmark> was added
-would be preferable. (Autoload the C<IO::whatever> module.)
-=head2 Rewrite perlre documentation
-The new-style patterns need full documentation, and the whole document
-needs to be a lot clearer.
+=head1 Tasks that need a little sysadmin-type knowledge
-=head2 Convert example code to IO::Handle filehandles
+Or if you prefer, tasks that you would learn from, and broaden your skills
+base...
-=head2 Document Win32 choices
+=head2 Relocatable perl
-=head2 Check new modules
+The C level patches needed to create a relocatable perl binary are done, as
+is the work on F<Config.pm>. All that's left to do is the C<Configure> tweaking
+to let people specify how they want to do the install.
-=head2 Make roffitall find pods and libs itself
+=head2 make HTML install work
-Simon Cozens has done some work on this but it needs a rethink.
+There is an C<installhtml> target in the Makefile. It's marked as
+"experimental". It would be good to get this tested, make it work reliably, and
+remove the "experimental" tag. This would include
-=head1 To do at some point
+=over 4
-These are ideas that have been regularly tossed around, that most
-people believe should be done maybe during 5.8.x
+=item 1
-=head2 Remove regular expression recursion
+Checking that cross linking between various parts of the documentation works.
+In particular that links work between the modules (files with POD in F<lib/>)
+and the core documentation (files in F<pod/>)
-Because the regular expression engine is recursive, badly designed
-expressions can lead to lots of recursion filling up the stack. Ilya
-claims that it is easy to convert the engine to being iterative, but
-this has still not yet been done. There may be a regular expression
-engine hit squad meeting at TPC5.
+=item 2
-=head2 Memory leaks after failed eval
+Work out how to split C<perlfunc> into chunks, preferably one per function
+group, preferably with general case code that could be used elsewhere.
+Challenges here are correctly identifying the groups of functions that go
+together, and making the right named external cross-links point to the right
+page. Things to be aware of are C<-X>, groups such as C<getpwnam> to
+C<endservent>, two or more C<=items> giving the different parameter lists, such
+as
-Perl will leak memory if you C<eval "hlagh hlagh hlagh hlagh">. This is
-partially because it attempts to build up an op tree for that code and
-doesn't properly free it. The same goes for non-syntactically-correct
-regular expressions. Hugo looked into this, but decided it needed a
-mark-and-sweep GC implementation.
+ =item substr EXPR,OFFSET,LENGTH,REPLACEMENT
+
+ =item substr EXPR,OFFSET,LENGTH
+
+ =item substr EXPR,OFFSET
-Alan notes that: The basic idea was to extend the parser token stack
-(C<YYSTYPE>) to include a type field so we knew what sort of thing each
-element of the stack was. The F<<perly.c> code would then have to be
-postprocessed to record the type of each entry on the stack as it was
-created, and the parser patched so that it could unroll the stack
-properly on error.
+and different parameter lists having different meanings. (eg C<select>)
-This is possible to do, but would be pretty messy to implement, as it
-would rely on even more sed hackery in F<perly.fixer>.
+=back
-=head2 pack "(stuff)*"
+=head2 compressed man pages
-That's to say, C<pack "(sI)40"> would be the same as C<pack "sI"x40>
+Be able to install them. This would probably need a configure test to see how
+the system does compressed man pages (same directory/different directory?
+same filename/different filename), as well as tweaking the F<installman> script
+to compress as necessary.
-=head2 bitfields in pack
+=head2 Add a code coverage target to the Makefile
-=head2 Cross compilation
+Make it easy for anyone to run Devel::Cover on the core's tests. The steps
+to do this manually are roughly
-Make Perl buildable with a cross-compiler. This will play havoc with
-Configure, which needs to how how the target system will respond to
-its tests; maybe C<microperl> will be a good starting point here.
-(Indeed, Bart Schuller reports that he compiled up C<microperl> for
-the Agenda PDA and it works fine.) A really big spanner in the works
-is the bootstrapping build process of Perl: if the filesystem the
-target systems sees is not the same what the build host sees, various
-input, output, and (Perl) library files need to be copied back and forth.
+=over 4
-As of 5.8.0 Configure mostly works for cross-compilation
-(used successfully for iPAQ Linux), miniperl gets built,
-but then building DynaLoader (and other extensions) fails
-since MakeMaker knows nothing of cross-compilation.
-(See INSTALL/Cross-compilation for the state of things.)
+=item *
-=head2 Perl preprocessor / macros
+do a normal C<Configure>, but include Devel::Cover as a module to install
+(see F<INSTALL> for how to do this)
-Source filters help with this, but do not get us all the way. For
-instance, it should be possible to implement the C<??> operator somehow;
-source filters don't (quite) cut it.
+=item *
-=head2 Perl lexer in Perl
+ make perl
-Damian Conway is planning to work on this, but it hasn't happened yet.
+=item *
-=head2 Using POSIX calls internally
+ cd t; HARNESS_PERL_SWITCHES=-MDevel::Cover ./perl -I../lib harness
-When faced with a BSD vs. SySV -style interface to some library or
-system function, perl's roots show in that it typically prefers the BSD
-interface (but falls back to the SysV one). One example is getpgrp().
-Other examples include C<memcpy> vs. C<bcopy>. There are others, mostly in
-F<<pp_sys.c>.
+=item *
-Mostly, this item is a suggestion for which way to start a journey into
-an C<#ifdef> forest. It is not primarily a suggestion to eliminate any of
-the C<#ifdef> forests.
+Process the resulting Devel::Cover database
-POSIX calls are perhaps more likely to be portable to unexpected
-architectures. They are also perhaps more likely to be actively
-maintained by a current vendor. They are also perhaps more likely to be
-available in thread-safe versions, if appropriate.
+=back
-=head2 -i rename file when changed
+This just give you the coverage of the F<.pm>s. To also get the C level
+coverage you need to
-It's only necessary to rename a file when inplace editing when the file
-has changed. Detecting a change is perhaps the difficult bit.
+=over 4
-=head2 All ARGV input should act like E<lt>E<gt>
+=item *
-eg C<read(ARGV, ...)> doesn't currently read across multiple files.
+Additionally tell C<Configure> to use the appropriate C compiler flags for
+C<gcov>
-=head2 Support for rerunning debugger
+=item *
-There should be a way of restarting the debugger on demand.
+ make perl.gcov
-=head2 Test Suite for the Debugger
+(instead of C<make perl>)
-The debugger is a complex piece of software and fixing something
-here may inadvertently break something else over there. To tame
-this chaotic behaviour, a test suite is necessary.
+=item *
-=head2 my sub foo { }
+After running the tests run C<gcov> to generate all the F<.gcov> files.
+(Including down in the subdirectories of F<ext/>
-The basic principle is sound, but there are problems with the semantics
-of self-referential and mutually referential lexical subs: how to
-declare the subs?
+=item *
-=head2 One-pass global destruction
+(From the top level perl directory) run C<gcov2perl> on all the C<.gcov> files
+to get their stats into the cover_db directory.
-Sweeping away all the allocated memory in one go is a laudable goal, but
-it's difficult and in most cases, it's easier to let the memory get
-freed by exiting.
+=item *
-=head2 Rewrite regexp parser
+Then process the Devel::Cover database
-There has been talk recently of rewriting the regular expression parser
-to produce an optree instead of a chain of opcodes; it's unclear whether
-or not this would be a win.
+=back
-=head2 Cache recently used regexps
+It would be good to add a single switch to C<Configure> to specify that you
+wanted to perform perl level coverage, and another to specify C level
+coverage, and have C<Configure> and the F<Makefile> do all the right things
+automatically.
-This is to speed up
+=head2 Make Config.pm cope with differences between build and installed perl
- for my $re (@regexps) {
- $matched++ if /$re/
- }
+Quite often vendors ship a perl binary compiled with their (pay-for)
+compilers. People install a free compiler, such as gcc. To work out how to
+build extensions, Perl interrogates C<%Config>, so in this situation
+C<%Config> describes compilers that aren't there, and extension building
+fails. This forces people into choosing between re-compiling perl themselves
+using the compiler they have, or only using modules that the vendor ships.
-C<qr//> already gives us a way of saving compiled regexps, but it should
-be done automatically.
+It would be good to find a way teach C<Config.pm> about the installation setup,
+possibly involving probing at install time or later, so that the C<%Config> in
+a binary distribution better describes the installed machine, when the
+installed machine differs from the build machine in some significant way.
-=head2 Re-entrant functions
+=head2 make parallel builds work
-Add configure probes for C<_r> forms of system calls and fit them to the
-core. Unfortunately, calling conventions for these functions and not
-standardised.
+Currently parallel builds (such as C<make -j3>) don't work reliably. We believe
+that this is due to incomplete dependency specification in the F<Makefile>.
+It would be good if someone were able to track down the causes of these
+problems, so that parallel builds worked properly.
-=head2 Cross-compilation support
+=head2 linker specification files
+
+Some platforms mandate that you provide a list of a shared library's external
+symbols to the linker, so the core already has the infrastructure in place to
+do this for generating shared perl libraries. My understanding is that the
+GNU toolchain can accept an optional linker specification file, and restrict
+visibility just to symbols declared in that file. It would be good to extend
+F<makedef.pl> to support this format, and to provide a means within
+C<Configure> to enable it. This would allow Unix users to test that the
+export list is correct, and to build a perl that does not pollute the global
+namespace with private symbols.
+
+
+
+
+=head1 Tasks that need a little C knowledge
+
+These tasks would need a little C knowledge, but don't need any specific
+background or experience with XS, or how the Perl interpreter works
-Bart Schuller reports that using C<microperl> and a cross-compiler, he
-got Perl working on the Agenda PDA. However, one cannot build a full
-Perl because Configure needs to get the results for the target platform,
-for the host.
+=head2 Make it clear from -v if this is the exact official release
+
+Currently perl from C<p4>/C<rsync> ships with a F<patchlevel.h> file that
+usually defines one local patch, of the form "MAINT12345" or "RC1". The output
+of perl -v doesn't report that a perl isn't an official release, and this
+information can get lost in bugs reports. Because of this, the minor version
+isn't bumped up until RC time, to minimise the possibility of versions of perl
+escaping that believe themselves to be newer than they actually are.
+
+It would be useful to find an elegant way to have the "this is an interim
+maintenance release" or "this is a release candidate" in the terse -v output,
+and have it so that it's easy for the pumpking to remove this just as the
+release tarball is rolled up. This way the version pulled out of rsync would
+always say "I'm a development release" and it would be safe to bump the
+reported minor version as soon as a release ships, which would aid perl
+developers.
-=head2 Bit-shifting bitvectors
+This task is really about thinking of an elegant way to arrange the C source
+such that it's trivial for the Pumpking to flag "this is an official release"
+when making a tarball, yet leave the default source saying "I'm not the
+official release".
-Given:
+=head2 Tidy up global variables
- vec($v, 1000, 1) = 1;
+There's a note in F<intrpvar.h>
-One should be able to do
+ /* These two variables are needed to preserve 5.8.x bincompat because
+ we can't change function prototypes of two exported functions.
+ Probably should be taken out of blead soon, and relevant prototypes
+ changed. */
- $v <<= 1;
+So doing this, and removing any of the unused variables still present would
+be good.
-and have the 999'th bit set.
+=head2 Ordering of "global" variables.
-Currently if you try with shift bitvectors you shift the NV/UV, instead
-of the bits in the PV. Not very logical.
+F<thrdvar.h> and F<intrpvarh> define the "global" variables that need to be
+per-thread under ithreads, where the variables are actually elements in a
+structure. As C dictates, the variables must be laid out in order of
+declaration. There is a comment
+C</* Important ones in the first cache line (if alignment is done right) */>
+which implies that at some point in the past the ordering was carefully chosen
+(at least in part). However, it's clear that the ordering is less than perfect,
+as currently there are things such as 7 C<bool>s in a row, then something
+typically requiring 4 byte alignment, and then an odd C<bool> later on.
+(C<bool>s are typically defined as C<char>s). So it would be good for someone
+to review the ordering of the variables, to see how much alignment padding can
+be removed.
-=head2 debugger pragma
+=head2 bincompat functions
-The debugger is implemented in Perl in F<perl5db.pl>; turning it into a
-pragma should be easy, but making it work lexically might be more
-difficult. Fiddling with C<$^P> would be necessary.
+There are lots of functions which are retained for binary compatibility.
+Clean these up. Move them to mathom.c, and don't compile for blead?
-=head2 use less pragma
+=head2 am I hot or not?
-Identify areas where speed/memory tradeoffs can be made and have a hint
-to switch between them.
+The idea of F<pp_hot.c> is that it contains the I<hot> ops, the ops that are
+most commonly used. The idea is that by grouping them, their object code will
+be adjacent in the executable, so they have a greater chance of already being
+in the CPU cache (or swapped in) due to being near another op already in use.
-=head2 switch structures
+Except that it's not clear if these really are the most commonly used ops. So
+anyone feeling like exercising their skill with coverage and profiling tools
+might want to determine what ops I<really> are the most commonly used. And in
+turn suggest evictions and promotions to achieve a better F<pp_hot.c>.
-Although we have C<Switch.pm> in core, Larry points to the dormant
-C<nswitch> and C<cswitch> ops in F<pp.c>; using these opcodes would be
-much faster.
+=head2 emulate the per-thread memory pool on Unix
-=head2 Cache eval tree
+For Windows, ithreads allocates memory for each thread from a separate pool,
+which it discards at thread exit. It also checks that memory is free()d to
+the correct pool. Neither check is done on Unix, so code developed there won't
+be subject to such strictures, so can harbour bugs that only show up when the
+code reaches Windows.
-=head2 rcatmaybe
+It would be good to be able to optionally emulate the Window pool system on
+Unix, to let developers who only have access to Unix, or want to use
+Unix-specific debugging tools, check for these problems. To do this would
+involve figuring out how the C<PerlMem_*> macros wrap C<malloc()> access, and
+providing a layer that records/checks the identity of the thread making the
+call, and recording all the memory allocated by each thread via this API so
+that it can be summarily free()d at thread exit. One implementation idea
+would be to increase the size of allocation, and store the C<my_perl> pointer
+(to identify the thread) at the start, along with pointers to make a linked
+list of blocks for this thread. To avoid alignment problems it would be
+necessary to do something like
-=head2 Shrink opcode tables
+ union memory_header_padded {
+ struct memory_header {
+ void *thread_id; /* For my_perl */
+ void *next; /* Pointer to next block for this thread */
+ } data;
+ long double padding; /* whatever type has maximal alignment constraint */
+ };
-=head2 Optimize away @_
-Look at the "reification" code in C<av.c>
+although C<long double> might not be the only type to add to the padding
+union.
-=head2 Prototypes versus indirect objects
+=head2 reduce duplication in sv_setsv_flags
-Currently, indirect object syntax bypasses prototype checks.
+C<Perl_sv_setsv_flags> has a comment
+C</* There's a lot of redundancy below but we're going for speed here */>
-=head2 Install HMTL
+Whilst this was true 10 years ago, the growing disparity between RAM and CPU
+speeds mean that the trade offs have changed. In addition, the duplicate code
+adds to the maintenance burden. It would be good to see how much of the
+redundancy can be pruned, particular in the less common paths. (Profiling
+tools at the ready...). For example, why does the test for
+"Can't redefine active sort subroutine" need to occur in two places?
-HTML versions of the documentation need to be installed by default; a
-call to C<installhtml> from C<installperl> may be all that's necessary.
-=head2 Prototype method calls
-=head2 Return context prototype declarations
-=head2 magic_setisa
+=head1 Tasks that need a knowledge of XS
-=head2 Garbage collection
+These tasks would need C knowledge, and roughly the level of knowledge of
+the perl API that comes from writing modules that use XS to interface to
+C.
-There have been persistent mumblings about putting a mark-and-sweep
-garbage detector into Perl; Alan Burlison has some ideas about this.
+=head2 IPv6
-=head2 IO tutorial
+Clean this up. Check everything in core works
-Mark-Jason Dominus has the beginnings of one of these.
+=head2 shrink C<GV>s, C<CV>s
-=head2 pack/unpack tutorial
+By removing unused elements and careful re-ordering, the structures for C<AV>s
+and C<HV>s have recently been shrunk considerably. It's probable that the same
+approach would find savings in C<GV>s and C<CV>s, if not all the other
+larger-than-C<PVMG> types.
-Simon Cozens has the beginnings of one of these.
+=head2 merge Perl_sv_2[inpu]v
-=head2 Rewrite perldoc
+There's a lot of code shared between C<Perl_sv_2iv_flags>,
+C<Perl_sv_2uv_flags>, C<Perl_sv_2nv>, and C<Perl_sv_2pv_flags>. It would be
+interesting to see if some of it can be merged into common shared static
+functions. In particular, C<Perl_sv_2uv_flags> started out as a cut&paste
+from C<Perl_sv_2iv_flags> around 5.005_50 time, and it may be possible to
+replace both with a single function that returns a value or union which is
+split out by the macros in F<sv.h>
-There are a few suggestions for what to do with C<perldoc>: maybe a
-full-text search, an index function, locating pages on a particular
-high-level subject, and so on.
+=head2 UTF8 caching code
-=head2 Install .3p manpages
+The string position/offset cache is not optional. It should be.
-This is a bone of contention; we can create C<.3p> manpages for each
-built-in function, but should we install them by default? Tcl does this,
-and it clutters up C<apropos>.
+=head2 Implicit Latin 1 => Unicode translation
-=head2 Unicode tutorial
+Conversions from byte strings to UTF-8 currently map high bit characters
+to Unicode without translation (or, depending on how you look at it, by
+implicitly assuming that the byte strings are in Latin-1). As perl assumes
+the C locale by default, upgrading a string to UTF-8 may change the
+meaning of its contents regarding character classes, case mapping, etc.
+This should probably emit a warning (at least).
-Simon Cozens promises to do this before he gets old.
+This task is incremental - even a little bit of work on it will help.
-=head2 Update POSIX.pm for 1003.1-2
+=head2 autovivification
-=head2 Retargetable installation
+Make all autovivification consistent w.r.t LVALUE/RVALUE and strict/no strict;
-Allow C<@INC> to be changed after Perl is built.
+This task is incremental - even a little bit of work on it will help.
-=head2 POSIX emulation on non-POSIX systems
+=head2 Unicode in Filenames
-Make C<POSIX.pm> behave as POSIXly as possible everywhere, meaning we
-have to implement POSIX equivalents for some functions if necessary.
+chdir, chmod, chown, chroot, exec, glob, link, lstat, mkdir, open,
+opendir, qx, readdir, readlink, rename, rmdir, stat, symlink, sysopen,
+system, truncate, unlink, utime, -X. All these could potentially accept
+Unicode filenames either as input or output (and in the case of system
+and qx Unicode in general, as input or output to/from the shell).
+Whether a filesystem - an operating system pair understands Unicode in
+filenames varies.
-=head2 Rename Win32 headers
+Known combinations that have some level of understanding include
+Microsoft NTFS, Apple HFS+ (In Mac OS 9 and X) and Apple UFS (in Mac
+OS X), NFS v4 is rumored to be Unicode, and of course Plan 9. How to
+create Unicode filenames, what forms of Unicode are accepted and used
+(UCS-2, UTF-16, UTF-8), what (if any) is the normalization form used,
+and so on, varies. Finding the right level of interfacing to Perl
+requires some thought. Remember that an OS does not implicate a
+filesystem.
-=head2 Finish off lvalue functions
+(The Windows -C command flag "wide API support" has been at least
+temporarily retired in 5.8.1, and the -C has been repurposed, see
+L<perlrun>.)
-They don't work in the debugger, and they don't work for list or hash
-slices.
+=head2 Unicode in %ENV
-=head2 Update sprintf documentation
+Currently the %ENV entries are always byte strings.
-Hugo van der Sanden plans to look at this.
+=head2 use less 'memory'
-=head2 Use fchown/fchmod internally
+Investigate trade offs to switch out perl's choices on memory usage.
+Particularly perl should be able to give memory back.
-This has been done in places, but needs a thorough code review.
-Also fchdir is available in some platforms.
+This task is incremental - even a little bit of work on it will help.
-=head1 Vague ideas
+=head2 Re-implement C<:unique> in a way that is actually thread-safe
-Ideas which have been discussed, and which may or may not happen.
+The old implementation made bad assumptions on several levels. A good 90%
+solution might be just to make C<:unique> work to share the string buffer
+of SvPVs. That way large constant strings can be shared between ithreads,
+such as the configuration information in F<Config>.
-=head2 ref() in list context
+=head2 Make tainting consistent
-It's unclear what this should do or how to do it without breaking old
-code.
+Tainting would be easier to use if it didn't take documented shortcuts and
+allow taint to "leak" everywhere within an expression.
-=head2 Make tr/// return histogram of characters in list context
+=head2 readpipe(LIST)
-There is a patch for this, but it may require Unicodification.
+system() accepts a LIST syntax (and a PROGRAM LIST syntax) to avoid
+running a shell. readpipe() (the function behind qx//) could be similarly
+extended.
-=head2 Compile to real threaded code
-=head2 Structured types
-=head2 Modifiable $1 et al.
- ($x = "elephant") =~ /e(ph)/;
- $1 = "g"; # $x = "elegant"
-What happens if there are multiple (nested?) brackets? What if the
-string changes between the match and the assignment?
+=head1 Tasks that need a knowledge of the interpreter
-=head2 Procedural interfaces for IO::*, etc.
+These tasks would need C knowledge, and knowledge of how the interpreter works,
+or a willingness to learn.
-Some core modules have been accused of being overly-OO. Adding
-procedural interfaces could demystify them.
+=head2 lexical pragmas
-=head2 RPC modules
+Reimplement the mechanism of lexical pragmas to be more extensible. Fix
+current pragmas that don't work well (or at all) with lexical scopes or in
+run-time eval(STRING) (C<sort>, C<re>, C<encoding> for example). MJD has a
+preliminary patch that implements this.
=head2 Attach/detach debugger from running program
-With C<gdb>, you can attach the debugger to a running program if you
-pass the process ID. It would be good to do this with the Perl debugger
-on a running Perl program, although I'm not sure how it would be done.
-
-=head2 Alternative RE syntax module
-
- use Regex::Newbie;
- $re = Regex::Newbie->new
- ->start
- ->match("foo")
- ->repeat(Regex::Newbie->class("char"),3)
- ->end;
- /$re/;
-
-=head2 GUI::Native
-
-A non-core module that would use "native" GUI to create graphical
-applications.
-
-=head2 foreach(reverse ...)
-
-Currently
-
- foreach (reverse @_) { ... }
-
-puts C<@_> on the stack, reverses it putting the reversed version on the
-stack, then iterates forwards. Instead, it could be special-cased to put
-C<@_> on the stack then iterate backwards.
-
-=head2 Constant function cache
-
-=head2 Approximate regular expression matching
-
-=head1 Ongoing
-
-These items B<always> need doing:
-
-=head2 Update guts documentation
-
-Simon Cozens tries to do this when possible, and contributions to the
-C<perlapi> documentation is welcome.
-
-=head2 Add more tests
+The old perltodo notes "With C<gdb>, you can attach the debugger to a running
+program if you pass the process ID. It would be good to do this with the Perl
+debugger on a running Perl program, although I'm not sure how it would be
+done." ssh and screen do this with named pipes in /tmp. Maybe we can too.
-Michael Schwern will donate $500 to Yet Another Society when all core
-modules have tests.
+=head2 inlining autoloaded constants
-=head2 Update auxiliary tools
+Currently the optimiser can inline constants when expressed as subroutines
+with prototype ($) that return a constant. Likewise, many packages wrapping
+C libraries export lots of constants as subroutines which are AUTOLOADed on
+demand. However, these have no prototypes, so can't be seen as constants by
+the optimiser. Some way of cheaply (low syntax, low memory overhead) to the
+perl compiler that a name is a constant would be great, so that it knows to
+call the AUTOLOAD routine at compile time, and then inline the constant.
-The code we ship with Perl should look like good Perl 5.
+=head2 Constant folding
-=head1 Recently done things
+The peephole optimiser should trap errors during constant folding, and give
+up on the folding, rather than bailing out at compile time. It is quite
+possible that the unfoldable constant is in unreachable code, eg something
+akin to C<$a = 0/0 if 0;>
-These are things which have been on the todo lists in previous releases
-but have recently been completed.
+=head2 LVALUE functions for lists
-=head2 Safe signal handling
+The old perltodo notes that lvalue functions don't work for list or hash
+slices. This would be good to fix.
-A new signal model went into 5.7.1 without much fanfare. Operations and
-C<malloc>s are no longer interrupted by signals, which are handled
-between opcodes. This means that C<PERL_ASYNC_CHECK> now actually does
-something. However, there are still a few things that need to be done.
+=head2 LVALUE functions in the debugger
-=head2 Tie Modules
+The old perltodo notes that lvalue functions don't work in the debugger. This
+would be good to fix.
-Modules which implement arrays in terms of strings, substrings or files
-can be found on the CPAN.
+=head2 _ prototype character
-=head2 gettimeofday
+Study the possibility of adding a new prototype character, C<_>, meaning
+"this argument defaults to $_".
-C<Time::Hires> has been integrated into the core.
+=head2 state variables
-=head2 setitimer and getimiter
+C<my $foo if 0;> is deprecated, and should be replaced with
+C<state $x = "initial value\n";> the syntax from Perl 6.
-Adding C<Time::Hires> got us this too.
+=head2 @INC source filter to Filter::Simple
-=head2 Testing __DIE__ hook
+The second return value from a sub in @INC can be a source filter. This isn't
+documented. It should be changed to use Filter::Simple, tested and documented.
-Tests have been added.
+=head2 regexp optimiser optional
-=head2 CPP equivalent in Perl
+The regexp optimiser is not optional. It should configurable to be, to allow
+its performance to be measured, and its bugs to be easily demonstrated.
-A C Yardley will probably have done this by the time you can read this.
-This allows for a generalization of the C constant detection used in
-building C<Errno.pm>.
+=head2 UNITCHECK
-=head2 Explicit switch statements
+Introduce a new special block, UNITCHECK, which is run at the end of a
+compilation unit (module, file, eval(STRING) block). This will correspond to
+the Perl 6 CHECK. Perl 5's CHECK cannot be changed or removed because the
+O.pm/B.pm backend framework depends on it.
-C<Switch.pm> has been integrated into the core to give you all manner of
-C<switch...case> semantics.
+=head2 optional optimizer
-=head2 autocroak
+Make the peephole optimizer optional. Currently it performs two tasks as
+it walks the optree - genuine peephole optimisations, and necessary fixups of
+ops. It would be good to find an efficient way to switch out the
+optimisations whilst keeping the fixups.
-This is C<Fatal.pm>.
+=head2 You WANT *how* many
-=head2 UTF/EBCDIC
+Currently contexts are void, scalar and list. split has a special mechanism in
+place to pass in the number of return values wanted. It would be useful to
+have a general mechanism for this, backwards compatible and little speed hit.
+This would allow proposals such as short circuiting sort to be implemented
+as a module on CPAN.
-Nick Ing-Simmons has made UTF-EBCDIC (UTR13) work with Perl.
+=head2 lexical aliases
- EBCDIC? http://www.unicode.org/unicode/reports/tr16/
+Allow lexical aliases (maybe via the syntax C<my \$alias = \$foo>.
-=head2 UTF Regexes
+=head2 entersub XS vs Perl
-Although there are probably some small bugs to be rooted out, Jarkko
-Hietaniemi has made regular expressions polymorphic between bytes and
-characters.
+At the moment pp_entersub is huge, and has code to deal with entering both
+perl and XS subroutines. Subroutine implementations rarely change between
+perl and XS at run time, so investigate using 2 ops to enter subs (one for
+XS, one for perl) and swap between if a sub is redefined.
-=head2 perlcc to produce executable
+=head2 Self ties
-C<perlcc> was recently rewritten, and can now produce standalone
-executables.
+self ties are currently illegal because they caused too many segfaults. Maybe
+the causes of these could be tracked down and self-ties on all types re-
+instated.
-=head2 END blocks saved in compiled output
-
-=head2 Secure temporary file module
-
-Tim Jenness' C<File::Temp> is now in core.
-
-=head2 Integrate Time::HiRes
-
-This module is now part of core.
-
-=head2 Turn Cwd into XS
-
-Benjamin Sugars has done this.
-
-=head2 Mmap for input
-
-Nick Ing-Simmons' C<perlio> supports an C<mmap> IO method.
-
-=head2 Byte to/from UTF8 and UTF8 to/from local conversion
-
-C<Encode> provides this.
-
-=head2 Add sockatmark support
-
-Added in 5.7.1
-
-=head2 Mailing list archives
-
-http://lists.perl.org/, http://archive.develooper.com/
-
-=head2 Bug tracking
-
-Richard Foley has written the bug tracking system at http://bugs.perl.org/
-
-=head2 Integrate MacPerl
-
-Chris Nandor and Matthias Neeracher have integrated the MacPerl changes
-into 5.6.0.
-
-=head2 Web "nerve center" for Perl
-
-http://use.perl.org/ is what you're looking for.
-
-=head2 Regular expression tutorial
-
-C<perlretut>, provided by Mark Kvale.
-
-=head2 Debugging Tutorial
-
-C<perldebtut>, written by Richard Foley.
-
-=head2 Integrate new modules
-
-Jarkko has been integrating madly into 5.7.x
-
-=head2 Integrate profiler
-
-C<Devel::DProf> is now a core module.
-
-=head2 Y2K error detection
-
-There's a configure option to detect unsafe concatenation with "19", and
-a CPAN module. (C<D'oh::Year>)
-
-=head2 Regular expression debugger
-
-While not part of core, Mark-Jason Dominus has written C<Rx> and has
-also come up with a generalised strategy for regular expression
-debugging.
-
-=head2 POD checker
-
-That's, uh, F<podchecker>
-
-=head2 "Dynamic" lexicals
-
-=head2 Cache precompiled modules
-
-=head1 Deprecated Wishes
-
-These are items which used to be in the todo file, but have been
-deprecated for some reason.
-
-=head2 Loop control on do{}
-
-This would break old code; use C<do{{ }}> instead.
-
-=head2 Lexically scoped typeglobs
-
-Not needed now we have lexical IO handles.
-
-=head2 format BOTTOM
-
-=head2 report HANDLE
-
-Damian Conway's text formatting modules seem to be the Way To Go.
-
-=head2 Generalised want()/caller())
-
-=head2 Named prototypes
-
-These both seem to be delayed until Perl 6.
-
-=head2 Built-in globbing
-
-The C<File::Glob> module has been used to replace the C<glob> function.
-
-=head2 Regression tests for suidperl
-
-C<suidperl> is deprecated in favour of common sense.
-
-=head2 Cached hash values
-
-We have shared hash keys, which perform the same job.
-
-=head2 Add compression modules
-
-The compression modules are a little heavy; meanwhile, Nick Clark is
-working on experimental pragmata to do transparent decompression on
-input.
-
-=head2 Reorganise documentation into tutorials/references
-
-Could not get consensus on P5P about this.
-
-=head2 Remove distinction between functions and operators
-
-Caution: highly flammable.
-
-=head2 Make XS easier to use
-
-Use C<Inline> instead, or SWIG.
-
-=head2 Make embedding easier to use
-
-Use C<Inline::CPR>.
+=head2 Optimize away @_
-=head2 man for perl
+The old perltodo notes "Look at the "reification" code in C<av.c>".
-See the Perl Power Tools. (http://language.perl.com/ppt/)
+=head2 switch ops
-=head2 my $Package::variable
+The old perltodo notes "Although we have C<Switch.pm> in core, Larry points to
+the dormant C<nswitch> and C<cswitch> ops in F<pp.c>; using these opcodes would
+be much faster."
-Use C<our> instead.
+=head2 What hooks would assertions need?
-=head2 "or" tests defined, not truth
+Assertions are in the core, and work. However, assertions needed to be added
+as a core patch, rather than an XS module in ext, or a CPAN module, because
+the core has no hooks in the necessary places. It would be useful to
+investigate what hooks would need to be added to make it possible to provide
+the full assertion support from a CPAN module, so that we aren't constraining
+the imagination of future CPAN authors.
-Suggesting this on P5P B<will> cause a boring and interminable flamewar.
-=head2 "class"-based lexicals
-Use flyweight objects, secure hashes or, dare I say it, pseudo-hashes instead.
-(Or whatever will replace pseudohashes in 5.10.)
-=head2 byteperl
-C<ByteLoader> covers this.
-=head2 Lazy evaluation / tail recursion removal
-C<List::Util> gives first() (a short-circuiting grep); tail recursion
-removal is done manually, with C<goto &whoami;>. (However, MJD has
-found that C<goto &whoami> introduces a performance penalty, so maybe
-there should be a way to do this after all: C<sub foo {START: ... goto
-START;> is better.)
+=head1 Big projects
-=head2 Make "use utf8" the default
+Tasks that will get your name mentioned in the description of the "Highlights
+of 5.10"
-Because of backward compatibility this is difficult: scripts could not
-contain B<any legacy eight-bit data> (like Latin-1) anymore, even in
-string literals or pod. Also would introduce a measurable slowdown of
-at least few percentages since all regular expression operations would
-be done in full UTF-8. But if you want to try this, add
--DUSE_UTF8_SCRIPTS to your compilation flags.
+=head2 make ithreads more robust
-=head2 Unicode collation and normalization
+Generally make ithreads more robust. See also L</iCOW>
-The Unicode::Collate and Unicode::Normalize modules
-by SADAHIRO Tomoyuki have been included since 5.8.0.
+This task is incremental - even a little bit of work on it will help, and
+will be greatly appreciated.
- Collation? http://www.unicode.org/unicode/reports/tr10/
- Normalization? http://www.unicode.org/unicode/reports/tr15/
+=head2 iCOW
-=head2 Create debugging macros
+Sarathy and Arthur have a proposal for an improved Copy On Write which
+specifically will be able to COW new ithreads. If this can be implemented
+it would be a good thing.
-Debugging macros (like printsv, dump) can make debugging perl inside a
-C debugger much easier. A good set for gdb comes with mod_perl.
-Something similar should be distributed with perl.
+=head2 (?{...}) closures in regexps
-The proper way to do this is to use and extend Devel::DebugInit.
-Devel::DebugInit also needs to be extended to support threads.
+Fix (or rewrite) the implementation of the C</(?{...})/> closures.
-See p5p archives for late May/early June 2001 for a recent discussion
-on this topic.
+=head2 A re-entrant regexp engine
-=cut
+This will allow the use of a regex from inside (?{ }), (??{ }) and
+(?(?{ })|) constructs.
https://perl5.git.perl.org / perl5.git / blobdiff