This is a live mirror of the Perl 5 development currently hosted at https://github.com/perl/perl5
A perltodone! ExtUtils::ParseXS uses strict
[perl5.git] / pod / perltodo.pod
CommitLineData
7711098a
GS
1=head1 NAME
2
3perltodo - Perl TO-DO List
4
5=head1 DESCRIPTION
e50bb9a1 6
049aabcb 7This is a list of wishes for Perl. The most up to date version of this file
b4af8972 8is at L<http://perl5.git.perl.org/perl.git/blob_plain/HEAD:/pod/perltodo.pod>
049aabcb
NC
9
10The tasks we think are smaller or easier are listed first. Anyone is welcome
11to work on any of these, but it's a good idea to first contact
12I<perl5-porters@perl.org> to avoid duplication of effort, and to learn from
13any previous attempts. By all means contact a pumpking privately first if you
14prefer.
e50bb9a1 15
0bdfc961
NC
16Whilst patches to make the list shorter are most welcome, ideas to add to
17the list are also encouraged. Check the perl5-porters archives for past
b4af8972
RB
18ideas, and any discussion about them. One set of archives may be found at
19L<http://www.xray.mpe.mpg.de/mailing-lists/perl5-porters/>
938c8732 20
617eabfa
NC
21What can we offer you in return? Fame, fortune, and everlasting glory? Maybe
22not, but if your patch is incorporated, then we'll add your name to the
23F<AUTHORS> file, which ships in the official distribution. How many other
24programming languages offer you 1 line of immortality?
938c8732 25
0bdfc961 26=head1 Tasks that only need Perl knowledge
e50bb9a1 27
de2b17d8
NC
28=head2 Migrate t/ from custom TAP generation
29
30Many tests below F<t/> still generate TAP by "hand", rather than using library
96090e4f 31functions. As explained in L<perlhack/TESTING>, tests in F<t/> are
de2b17d8
NC
32written in a particular way to test that more complex constructions actually
33work before using them routinely. Hence they don't use C<Test::More>, but
34instead there is an intentionally simpler library, F<t/test.pl>. However,
35quite a few tests in F<t/> have not been refactored to use it. Refactoring
36any of these tests, one at a time, is a useful thing TODO.
37
0d8e5a42
RGS
38The subdirectories F<base>, F<cmd> and F<comp>, that contain the most
39basic tests, should be excluded from this task.
40
0be987a2
NC
41=head2 Automate perldelta generation
42
43The perldelta file accompanying each release summaries the major changes.
44It's mostly manually generated currently, but some of that could be
45automated with a bit of perl, specifically the generation of
46
47=over
48
49=item Modules and Pragmata
50
51=item New Documentation
52
53=item New Tests
54
55=back
56
57See F<Porting/how_to_write_a_perldelta.pod> for details.
58
5a176cbc
NC
59=head2 Remove duplication of test setup.
60
61Schwern notes, that there's duplication of code - lots and lots of tests have
62some variation on the big block of C<$Is_Foo> checks. We can safely put this
63into a file, change it to build an C<%Is> hash and require it. Maybe just put
64it into F<test.pl>. Throw in the handy tainting subroutines.
65
87a942b1 66=head2 POD -E<gt> HTML conversion in the core still sucks
e50bb9a1 67
938c8732 68Which is crazy given just how simple POD purports to be, and how simple HTML
adebf063
NC
69can be. It's not actually I<as> simple as it sounds, particularly with the
70flexibility POD allows for C<=item>, but it would be good to improve the
71visual appeal of the HTML generated, and to avoid it having any validation
72errors. See also L</make HTML install work>, as the layout of installation tree
73is needed to improve the cross-linking.
938c8732 74
dc0fb092
SP
75The addition of C<Pod::Simple> and its related modules may make this task
76easier to complete.
77
0bdfc961 78=head2 Make Schwern poorer
e50bb9a1 79
613bd4f7 80We should have tests for everything. When all the core's modules are tested,
0bdfc961
NC
81Schwern has promised to donate to $500 to TPF. We may need volunteers to
82hold him upside down and shake vigorously in order to actually extract the
83cash.
3958b146 84
0bdfc961 85=head2 Improve the coverage of the core tests
e50bb9a1 86
e1020413 87Use Devel::Cover to ascertain the core modules' test coverage, then add
02f21748 88tests that are currently missing.
30222c0f 89
0bdfc961 90=head2 test B
e50bb9a1 91
0bdfc961 92A full test suite for the B module would be nice.
e50bb9a1 93
0bdfc961 94=head2 A decent benchmark
e50bb9a1 95
617eabfa 96C<perlbench> seems impervious to any recent changes made to the perl core. It
0bdfc961
NC
97would be useful to have a reasonable general benchmarking suite that roughly
98represented what current perl programs do, and measurably reported whether
99tweaks to the core improve, degrade or don't really affect performance, to
100guide people attempting to optimise the guts of perl. Gisle would welcome
101new tests for perlbench.
6168cf99 102
0bdfc961 103=head2 fix tainting bugs
6168cf99 104
0bdfc961
NC
105Fix the bugs revealed by running the test suite with the C<-t> switch (via
106C<make test.taintwarn>).
e50bb9a1 107
0bdfc961 108=head2 Dual life everything
e50bb9a1 109
0bdfc961
NC
110As part of the "dists" plan, anything that doesn't belong in the smallest perl
111distribution needs to be dual lifed. Anything else can be too. Figure out what
112changes would be needed to package that module and its tests up for CPAN, and
113do so. Test it with older perl releases, and fix the problems you find.
e50bb9a1 114
a393eb28
RGS
115To make a minimal perl distribution, it's useful to look at
116F<t/lib/commonsense.t>.
117
0bdfc961 118=head2 POSIX memory footprint
e50bb9a1 119
0bdfc961
NC
120Ilya observed that use POSIX; eats memory like there's no tomorrow, and at
121various times worked to cut it down. There is probably still fat to cut out -
122for example POSIX passes Exporter some very memory hungry data structures.
e50bb9a1 123
eed36644
NC
124=head2 embed.pl/makedef.pl
125
126There is a script F<embed.pl> that generates several header files to prefix
127all of Perl's symbols in a consistent way, to provide some semblance of
128namespace support in C<C>. Functions are declared in F<embed.fnc>, variables
907b3e23 129in F<interpvar.h>. Quite a few of the functions and variables
eed36644
NC
130are conditionally declared there, using C<#ifdef>. However, F<embed.pl>
131doesn't understand the C macros, so the rules about which symbols are present
132when is duplicated in F<makedef.pl>. Writing things twice is bad, m'kay.
133It would be good to teach C<embed.pl> to understand the conditional
134compilation, and hence remove the duplication, and the mistakes it has caused.
e50bb9a1 135
801de10e
NC
136=head2 use strict; and AutoLoad
137
138Currently if you write
139
140 package Whack;
141 use AutoLoader 'AUTOLOAD';
142 use strict;
143 1;
144 __END__
145 sub bloop {
146 print join (' ', No, strict, here), "!\n";
147 }
148
149then C<use strict;> isn't in force within the autoloaded subroutines. It would
150be more consistent (and less surprising) to arrange for all lexical pragmas
151in force at the __END__ block to be in force within each autoloaded subroutine.
152
773b3597
RGS
153There's a similar problem with SelfLoader.
154
91d0cbf6
NC
155=head2 profile installman
156
157The F<installman> script is slow. All it is doing text processing, which we're
158told is something Perl is good at. So it would be nice to know what it is doing
159that is taking so much CPU, and where possible address it.
160
c69ca1d4 161=head2 enable lexical enabling/disabling of individual warnings
a9ed9b74
JV
162
163Currently, warnings can only be enabled or disabled by category. There
164are times when it would be useful to quash a single warning, not a
165whole category.
91d0cbf6 166
85234543
KW
167=head2 document diagnostics
168
169Many diagnostic messages are not currently documented. The list is at the end
170of t/porting/diag.t.
171
0bdfc961 172=head1 Tasks that need a little sysadmin-type knowledge
e50bb9a1 173
0bdfc961
NC
174Or if you prefer, tasks that you would learn from, and broaden your skills
175base...
e50bb9a1 176
cd793d32 177=head2 make HTML install work
e50bb9a1 178
adebf063
NC
179There is an C<installhtml> target in the Makefile. It's marked as
180"experimental". It would be good to get this tested, make it work reliably, and
181remove the "experimental" tag. This would include
182
183=over 4
184
185=item 1
186
187Checking that cross linking between various parts of the documentation works.
188In particular that links work between the modules (files with POD in F<lib/>)
189and the core documentation (files in F<pod/>)
190
191=item 2
192
617eabfa
NC
193Work out how to split C<perlfunc> into chunks, preferably one per function
194group, preferably with general case code that could be used elsewhere.
195Challenges here are correctly identifying the groups of functions that go
196together, and making the right named external cross-links point to the right
197page. Things to be aware of are C<-X>, groups such as C<getpwnam> to
198C<endservent>, two or more C<=items> giving the different parameter lists, such
199as
adebf063
NC
200
201 =item substr EXPR,OFFSET,LENGTH,REPLACEMENT
adebf063 202 =item substr EXPR,OFFSET,LENGTH
adebf063
NC
203 =item substr EXPR,OFFSET
204
205and different parameter lists having different meanings. (eg C<select>)
206
207=back
3a89a73c 208
0bdfc961
NC
209=head2 compressed man pages
210
211Be able to install them. This would probably need a configure test to see how
212the system does compressed man pages (same directory/different directory?
213same filename/different filename), as well as tweaking the F<installman> script
214to compress as necessary.
215
30222c0f
NC
216=head2 Add a code coverage target to the Makefile
217
218Make it easy for anyone to run Devel::Cover on the core's tests. The steps
219to do this manually are roughly
220
221=over 4
222
223=item *
224
225do a normal C<Configure>, but include Devel::Cover as a module to install
f11a3063 226(see L<INSTALL> for how to do this)
30222c0f
NC
227
228=item *
229
230 make perl
231
232=item *
233
234 cd t; HARNESS_PERL_SWITCHES=-MDevel::Cover ./perl -I../lib harness
235
236=item *
237
238Process the resulting Devel::Cover database
239
240=back
241
242This just give you the coverage of the F<.pm>s. To also get the C level
243coverage you need to
244
245=over 4
246
247=item *
248
249Additionally tell C<Configure> to use the appropriate C compiler flags for
250C<gcov>
251
252=item *
253
254 make perl.gcov
255
256(instead of C<make perl>)
257
258=item *
259
260After running the tests run C<gcov> to generate all the F<.gcov> files.
261(Including down in the subdirectories of F<ext/>
262
263=item *
264
265(From the top level perl directory) run C<gcov2perl> on all the C<.gcov> files
266to get their stats into the cover_db directory.
267
268=item *
269
270Then process the Devel::Cover database
271
272=back
273
274It would be good to add a single switch to C<Configure> to specify that you
275wanted to perform perl level coverage, and another to specify C level
276coverage, and have C<Configure> and the F<Makefile> do all the right things
277automatically.
278
02f21748 279=head2 Make Config.pm cope with differences between built and installed perl
0bdfc961
NC
280
281Quite often vendors ship a perl binary compiled with their (pay-for)
282compilers. People install a free compiler, such as gcc. To work out how to
283build extensions, Perl interrogates C<%Config>, so in this situation
284C<%Config> describes compilers that aren't there, and extension building
285fails. This forces people into choosing between re-compiling perl themselves
286using the compiler they have, or only using modules that the vendor ships.
287
288It would be good to find a way teach C<Config.pm> about the installation setup,
289possibly involving probing at install time or later, so that the C<%Config> in
290a binary distribution better describes the installed machine, when the
291installed machine differs from the build machine in some significant way.
292
728f4ecd
NC
293=head2 linker specification files
294
295Some platforms mandate that you provide a list of a shared library's external
296symbols to the linker, so the core already has the infrastructure in place to
297do this for generating shared perl libraries. My understanding is that the
298GNU toolchain can accept an optional linker specification file, and restrict
299visibility just to symbols declared in that file. It would be good to extend
300F<makedef.pl> to support this format, and to provide a means within
301C<Configure> to enable it. This would allow Unix users to test that the
302export list is correct, and to build a perl that does not pollute the global
32d539f5
RU
303namespace with private symbols, and will fail in the same way as msvc or mingw
304builds or when using PERL_DL_NONLAZY=1.
728f4ecd 305
a229ae3b
RGS
306=head2 Cross-compile support
307
308Currently C<Configure> understands C<-Dusecrosscompile> option. This option
309arranges for building C<miniperl> for TARGET machine, so this C<miniperl> is
310assumed then to be copied to TARGET machine and used as a replacement of full
311C<perl> executable.
312
d1307786 313This could be done little differently. Namely C<miniperl> should be built for
a229ae3b 314HOST and then full C<perl> with extensions should be compiled for TARGET.
d1307786 315This, however, might require extra trickery for %Config: we have one config
87a942b1
JH
316first for HOST and then another for TARGET. Tools like MakeMaker will be
317mightily confused. Having around two different types of executables and
318libraries (HOST and TARGET) makes life interesting for Makefiles and
319shell (and Perl) scripts. There is $Config{run}, normally empty, which
320can be used as an execution wrapper. Also note that in some
321cross-compilation/execution environments the HOST and the TARGET do
322not see the same filesystem(s), the $Config{run} may need to do some
323file/directory copying back and forth.
0bdfc961 324
8537f021
RGS
325=head2 roffitall
326
327Make F<pod/roffitall> be updated by F<pod/buildtoc>.
328
98fca0e8
NC
329=head2 Split "linker" from "compiler"
330
331Right now, Configure probes for two commands, and sets two variables:
332
333=over 4
334
b91dd380 335=item * C<cc> (in F<cc.U>)
98fca0e8
NC
336
337This variable holds the name of a command to execute a C compiler which
338can resolve multiple global references that happen to have the same
339name. Usual values are F<cc> and F<gcc>.
340Fervent ANSI compilers may be called F<c89>. AIX has F<xlc>.
341
b91dd380 342=item * C<ld> (in F<dlsrc.U>)
98fca0e8
NC
343
344This variable indicates the program to be used to link
345libraries for dynamic loading. On some systems, it is F<ld>.
346On ELF systems, it should be C<$cc>. Mostly, we'll try to respect
347the hint file setting.
348
349=back
350
8d159ec1
NC
351There is an implicit historical assumption from around Perl5.000alpha
352something, that C<$cc> is also the correct command for linking object files
353together to make an executable. This may be true on Unix, but it's not true
354on other platforms, and there are a maze of work arounds in other places (such
355as F<Makefile.SH>) to cope with this.
98fca0e8
NC
356
357Ideally, we should create a new variable to hold the name of the executable
358linker program, probe for it in F<Configure>, and centralise all the special
359case logic there or in hints files.
360
361A small bikeshed issue remains - what to call it, given that C<$ld> is already
8d159ec1
NC
362taken (arguably for the wrong thing now, but on SunOS 4.1 it is the command
363for creating dynamically-loadable modules) and C<$link> could be confused with
364the Unix command line executable of the same name, which does something
365completely different. Andy Dougherty makes the counter argument "In parrot, I
366tried to call the command used to link object files and libraries into an
367executable F<link>, since that's what my vaguely-remembered DOS and VMS
368experience suggested. I don't think any real confusion has ensued, so it's
369probably a reasonable name for perl5 to use."
98fca0e8
NC
370
371"Alas, I've always worried that introducing it would make things worse,
372since now the module building utilities would have to look for
373C<$Config{link}> and institute a fall-back plan if it weren't found."
8d159ec1
NC
374Although I can see that as confusing, given that C<$Config{d_link}> is true
375when (hard) links are available.
98fca0e8 376
75585ce3
SP
377=head2 Configure Windows using PowerShell
378
379Currently, Windows uses hard-coded config files based to build the
380config.h for compiling Perl. Makefiles are also hard-coded and need to be
381hand edited prior to building Perl. While this makes it easy to create a perl.exe
382that works across multiple Windows versions, being able to accurately
383configure a perl.exe for a specific Windows versions and VS C++ would be
384a nice enhancement. With PowerShell available on Windows XP and up, this
385may now be possible. Step 1 might be to investigate whether this is possible
386and use this to clean up our current makefile situation. Step 2 would be to
387see if there would be a way to use our existing metaconfig units to configure a
388Windows Perl or whether we go in a separate direction and make it so. Of
389course, we all know what step 3 is.
390
0bdfc961
NC
391=head1 Tasks that need a little C knowledge
392
393These tasks would need a little C knowledge, but don't need any specific
394background or experience with XS, or how the Perl interpreter works
395
3d826b29
NC
396=head2 Weed out needless PERL_UNUSED_ARG
397
398The C code uses the macro C<PERL_UNUSED_ARG> to stop compilers warning about
399unused arguments. Often the arguments can't be removed, as there is an
400external constraint that determines the prototype of the function, so this
401approach is valid. However, there are some cases where C<PERL_UNUSED_ARG>
402could be removed. Specifically
403
404=over 4
405
406=item *
407
408The prototypes of (nearly all) static functions can be changed
409
410=item *
411
412Unused arguments generated by short cut macros are wasteful - the short cut
413macro used can be changed.
414
415=back
416
fbf638cb
RGS
417=head2 Modernize the order of directories in @INC
418
419The way @INC is laid out by default, one cannot upgrade core (dual-life)
420modules without overwriting files. This causes problems for binary
3d14fd97
AD
421package builders. One possible proposal is laid out in this
422message:
b4af8972 423L<http://www.xray.mpe.mpg.de/mailing-lists/perl5-porters/2002-04/msg02380.html>
fbf638cb 424
bcbaa2d5
RGS
425=head2 -Duse32bit*
426
427Natively 64-bit systems need neither -Duse64bitint nor -Duse64bitall.
428On these systems, it might be the default compilation mode, and there
429is currently no guarantee that passing no use64bitall option to the
430Configure process will build a 32bit perl. Implementing -Duse32bit*
fd2dadea 431options would be nice for perl 5.14.
bcbaa2d5 432
fee0a0f7 433=head2 Profile Perl - am I hot or not?
62403a3c 434
fee0a0f7
NC
435The Perl source code is stable enough that it makes sense to profile it,
436identify and optimise the hotspots. It would be good to measure the
437performance of the Perl interpreter using free tools such as cachegrind,
438gprof, and dtrace, and work to reduce the bottlenecks they reveal.
439
440As part of this, the idea of F<pp_hot.c> is that it contains the I<hot> ops,
441the ops that are most commonly used. The idea is that by grouping them, their
442object code will be adjacent in the executable, so they have a greater chance
443of already being in the CPU cache (or swapped in) due to being near another op
444already in use.
62403a3c
NC
445
446Except that it's not clear if these really are the most commonly used ops. So
fee0a0f7
NC
447as part of exercising your skills with coverage and profiling tools you might
448want to determine what ops I<really> are the most commonly used. And in turn
449suggest evictions and promotions to achieve a better F<pp_hot.c>.
62403a3c 450
91d0cbf6
NC
451One piece of Perl code that might make a good testbed is F<installman>.
452
98fed0ad
NC
453=head2 Allocate OPs from arenas
454
455Currently all new OP structures are individually malloc()ed and free()d.
456All C<malloc> implementations have space overheads, and are now as fast as
457custom allocates so it would both use less memory and less CPU to allocate
458the various OP structures from arenas. The SV arena code can probably be
459re-used for this.
460
539f2c54
JC
461Note that Configuring perl with C<-Accflags=-DPL_OP_SLAB_ALLOC> will use
462Perl_Slab_alloc() to pack optrees into a contiguous block, which is
463probably superior to the use of OP arenas, esp. from a cache locality
464standpoint. See L<Profile Perl - am I hot or not?>.
465
a229ae3b 466=head2 Improve win32/wince.c
0bdfc961 467
a229ae3b 468Currently, numerous functions look virtually, if not completely,
c23989d1 469identical in both F<win32/wince.c> and F<win32/win32.c> files, which can't
6d71adcd
NC
470be good.
471
c5b31784
SH
472=head2 Use secure CRT functions when building with VC8 on Win32
473
474Visual C++ 2005 (VC++ 8.x) deprecated a number of CRT functions on the basis
475that they were "unsafe" and introduced differently named secure versions of
476them as replacements, e.g. instead of writing
477
478 FILE* f = fopen(__FILE__, "r");
479
480one should now write
481
482 FILE* f;
483 errno_t err = fopen_s(&f, __FILE__, "r");
484
485Currently, the warnings about these deprecations have been disabled by adding
486-D_CRT_SECURE_NO_DEPRECATE to the CFLAGS. It would be nice to remove that
487warning suppressant and actually make use of the new secure CRT functions.
488
489There is also a similar issue with POSIX CRT function names like fileno having
490been deprecated in favour of ISO C++ conformant names like _fileno. These
26a6faa8 491warnings are also currently suppressed by adding -D_CRT_NONSTDC_NO_DEPRECATE. It
c5b31784
SH
492might be nice to do as Microsoft suggest here too, although, unlike the secure
493functions issue, there is presumably little or no benefit in this case.
494
038ae9a4
SH
495=head2 Fix POSIX::access() and chdir() on Win32
496
497These functions currently take no account of DACLs and therefore do not behave
498correctly in situations where access is restricted by DACLs (as opposed to the
499read-only attribute).
500
501Furthermore, POSIX::access() behaves differently for directories having the
502read-only attribute set depending on what CRT library is being used. For
503example, the _access() function in the VC6 and VC7 CRTs (wrongly) claim that
504such directories are not writable, whereas in fact all directories are writable
505unless access is denied by DACLs. (In the case of directories, the read-only
506attribute actually only means that the directory cannot be deleted.) This CRT
507bug is fixed in the VC8 and VC9 CRTs (but, of course, the directory may still
508not actually be writable if access is indeed denied by DACLs).
509
510For the chdir() issue, see ActiveState bug #74552:
b4af8972 511L<http://bugs.activestate.com/show_bug.cgi?id=74552>
038ae9a4
SH
512
513Therefore, DACLs should be checked both for consistency across CRTs and for
514the correct answer.
515
516(Note that perl's -w operator should not be modified to check DACLs. It has
517been written so that it reflects the state of the read-only attribute, even
518for directories (whatever CRT is being used), for symmetry with chmod().)
519
16815324
NC
520=head2 strcat(), strcpy(), strncat(), strncpy(), sprintf(), vsprintf()
521
522Maybe create a utility that checks after each libperl.a creation that
523none of the above (nor sprintf(), vsprintf(), or *SHUDDER* gets())
524ever creep back to libperl.a.
525
526 nm libperl.a | ./miniperl -alne '$o = $F[0] if /:$/; print "$o $F[1]" if $F[0] eq "U" && $F[1] =~ /^(?:strn?c(?:at|py)|v?sprintf|gets)$/'
527
528Note, of course, that this will only tell whether B<your> platform
529is using those naughty interfaces.
530
de96509d
JH
531=head2 -D_FORTIFY_SOURCE=2, -fstack-protector
532
533Recent glibcs support C<-D_FORTIFY_SOURCE=2> and recent gcc
534(4.1 onwards?) supports C<-fstack-protector>, both of which give
535protection against various kinds of buffer overflow problems.
536These should probably be used for compiling Perl whenever available,
537Configure and/or hints files should be adjusted to probe for the
538availability of these features and enable them as appropriate.
16815324 539
8964cfe0
NC
540=head2 Arenas for GPs? For MAGIC?
541
542C<struct gp> and C<struct magic> are both currently allocated by C<malloc>.
543It might be a speed or memory saving to change to using arenas. Or it might
544not. It would need some suitable benchmarking first. In particular, C<GP>s
545can probably be changed with minimal compatibility impact (probably nothing
546outside of the core, or even outside of F<gv.c> allocates them), but they
547probably aren't allocated/deallocated often enough for a speed saving. Whereas
548C<MAGIC> is allocated/deallocated more often, but in turn, is also something
549more externally visible, so changing the rules here may bite external code.
550
3880c8ec
NC
551=head2 Shared arenas
552
553Several SV body structs are now the same size, notably PVMG and PVGV, PVAV and
554PVHV, and PVCV and PVFM. It should be possible to allocate and return same
555sized bodies from the same actual arena, rather than maintaining one arena for
556each. This could save 4-6K per thread, of memory no longer tied up in the
557not-yet-allocated part of an arena.
558
8964cfe0 559
6d71adcd
NC
560=head1 Tasks that need a knowledge of XS
561
562These tasks would need C knowledge, and roughly the level of knowledge of
563the perl API that comes from writing modules that use XS to interface to
564C.
565
e851c105
DG
566=head2 Write an XS cookbook
567
568Create pod/perlxscookbook.pod with short, task-focused 'recipes' in XS that
569demonstrate common tasks and good practices. (Some of these might be
570extracted from perlguts.) The target audience should be XS novices, who need
571more examples than perlguts but something less overwhelming than perlapi.
572Recipes should provide "one pretty good way to do it" instead of TIMTOWTDI.
573
5b7d14ff
DG
574Rather than focusing on interfacing Perl to C libraries, such a cookbook
575should probably focus on how to optimize Perl routines by re-writing them
576in XS. This will likely be more motivating to those who mostly work in
577Perl but are looking to take the next step into XS.
578
579Deconstructing and explaining some simpler XS modules could be one way to
580bootstrap a cookbook. (List::Util? Class::XSAccessor? Tree::Ternary_XS?)
581Another option could be deconstructing the implementation of some simpler
582functions in op.c.
583
05fb4e20
NC
584=head2 Allow XSUBs to inline themselves as OPs
585
586For a simple XSUB, often the subroutine dispatch takes more time than the
587XSUB itself. The tokeniser already has the ability to inline constant
588subroutines - it would be good to provide a way to inline other subroutines.
589
590Specifically, simplest approach looks to be to allow an XSUB to provide an
591alternative implementation of itself as a custom OP. A new flag bit in
592C<CvFLAGS()> would signal to the peephole optimiser to take an optree
593such as this:
594
595 b <@> leave[1 ref] vKP/REFC ->(end)
596 1 <0> enter ->2
597 2 <;> nextstate(main 1 -e:1) v:{ ->3
598 a <2> sassign vKS/2 ->b
599 8 <1> entersub[t2] sKS/TARG,1 ->9
600 - <1> ex-list sK ->8
601 3 <0> pushmark s ->4
602 4 <$> const(IV 1) sM ->5
603 6 <1> rv2av[t1] lKM/1 ->7
604 5 <$> gv(*a) s ->6
605 - <1> ex-rv2cv sK ->-
606 7 <$> gv(*x) s/EARLYCV ->8
607 - <1> ex-rv2sv sKRM*/1 ->a
608 9 <$> gvsv(*b) s ->a
609
610perform the symbol table lookup of C<rv2cv> and C<gv(*x)>, locate the
611pointer to the custom OP that provides the direct implementation, and re-
612write the optree something like:
613
614 b <@> leave[1 ref] vKP/REFC ->(end)
615 1 <0> enter ->2
616 2 <;> nextstate(main 1 -e:1) v:{ ->3
617 a <2> sassign vKS/2 ->b
618 7 <1> custom_x -> 8
619 - <1> ex-list sK ->7
620 3 <0> pushmark s ->4
621 4 <$> const(IV 1) sM ->5
622 6 <1> rv2av[t1] lKM/1 ->7
623 5 <$> gv(*a) s ->6
624 - <1> ex-rv2cv sK ->-
625 - <$> ex-gv(*x) s/EARLYCV ->7
626 - <1> ex-rv2sv sKRM*/1 ->a
627 8 <$> gvsv(*b) s ->a
628
629I<i.e.> the C<gv(*)> OP has been nulled and spliced out of the execution
630path, and the C<entersub> OP has been replaced by the custom op.
631
632This approach should provide a measurable speed up to simple XSUBs inside
633tight loops. Initially one would have to write the OP alternative
634implementation by hand, but it's likely that this should be reasonably
635straightforward for the type of XSUB that would benefit the most. Longer
636term, once the run-time implementation is proven, it should be possible to
637progressively update ExtUtils::ParseXS to generate OP implementations for
638some XSUBs.
639
318bf708
NC
640=head2 Remove the use of SVs as temporaries in dump.c
641
642F<dump.c> contains debugging routines to dump out the contains of perl data
643structures, such as C<SV>s, C<AV>s and C<HV>s. Currently, the dumping code
644B<uses> C<SV>s for its temporary buffers, which was a logical initial
645implementation choice, as they provide ready made memory handling.
646
647However, they also lead to a lot of confusion when it happens that what you're
648trying to debug is seen by the code in F<dump.c>, correctly or incorrectly, as
649a temporary scalar it can use for a temporary buffer. It's also not possible
650to dump scalars before the interpreter is properly set up, such as during
651ithreads cloning. It would be good to progressively replace the use of scalars
652as string accumulation buffers with something much simpler, directly allocated
653by C<malloc>. The F<dump.c> code is (or should be) only producing 7 bit
654US-ASCII, so output character sets are not an issue.
655
656Producing and proving an internal simple buffer allocation would make it easier
657to re-write the internals of the PerlIO subsystem to avoid using C<SV>s for
658B<its> buffers, use of which can cause problems similar to those of F<dump.c>,
659at similar times.
660
5d96f598
NC
661=head2 safely supporting POSIX SA_SIGINFO
662
663Some years ago Jarkko supplied patches to provide support for the POSIX
664SA_SIGINFO feature in Perl, passing the extra data to the Perl signal handler.
665
666Unfortunately, it only works with "unsafe" signals, because under safe
667signals, by the time Perl gets to run the signal handler, the extra
668information has been lost. Moreover, it's not easy to store it somewhere,
669as you can't call mutexs, or do anything else fancy, from inside a signal
670handler.
671
672So it strikes me that we could provide safe SA_SIGINFO support
673
674=over 4
675
676=item 1
677
678Provide global variables for two file descriptors
679
680=item 2
681
682When the first request is made via C<sigaction> for C<SA_SIGINFO>, create a
683pipe, store the reader in one, the writer in the other
684
685=item 3
686
687In the "safe" signal handler (C<Perl_csighandler()>/C<S_raise_signal()>), if
688the C<siginfo_t> pointer non-C<NULL>, and the writer file handle is open,
689
690=over 8
691
692=item 1
693
694serialise signal number, C<struct siginfo_t> (or at least the parts we care
695about) into a small auto char buff
696
697=item 2
698
699C<write()> that (non-blocking) to the writer fd
700
701=over 12
702
703=item 1
704
705if it writes 100%, flag the signal in a counter of "signals on the pipe" akin
706to the current per-signal-number counts
707
708=item 2
709
710if it writes 0%, assume the pipe is full. Flag the data as lost?
711
712=item 3
713
714if it writes partially, croak a panic, as your OS is broken.
715
716=back
717
718=back
719
720=item 4
721
722in the regular C<PERL_ASYNC_CHECK()> processing, if there are "signals on
723the pipe", read the data out, deserialise, build the Perl structures on
724the stack (code in C<Perl_sighandler()>, the "unsafe" handler), and call as
725usual.
726
727=back
728
729I think that this gets us decent C<SA_SIGINFO> support, without the current risk
730of running Perl code inside the signal handler context. (With all the dangers
731of things like C<malloc> corruption that that currently offers us)
732
733For more information see the thread starting with this message:
b4af8972 734L<http://www.xray.mpe.mpg.de/mailing-lists/perl5-porters/2008-03/msg00305.html>
5d96f598 735
6d71adcd
NC
736=head2 autovivification
737
738Make all autovivification consistent w.r.t LVALUE/RVALUE and strict/no strict;
739
740This task is incremental - even a little bit of work on it will help.
741
742=head2 Unicode in Filenames
743
744chdir, chmod, chown, chroot, exec, glob, link, lstat, mkdir, open,
745opendir, qx, readdir, readlink, rename, rmdir, stat, symlink, sysopen,
746system, truncate, unlink, utime, -X. All these could potentially accept
747Unicode filenames either as input or output (and in the case of system
748and qx Unicode in general, as input or output to/from the shell).
749Whether a filesystem - an operating system pair understands Unicode in
750filenames varies.
751
752Known combinations that have some level of understanding include
753Microsoft NTFS, Apple HFS+ (In Mac OS 9 and X) and Apple UFS (in Mac
754OS X), NFS v4 is rumored to be Unicode, and of course Plan 9. How to
755create Unicode filenames, what forms of Unicode are accepted and used
756(UCS-2, UTF-16, UTF-8), what (if any) is the normalization form used,
757and so on, varies. Finding the right level of interfacing to Perl
758requires some thought. Remember that an OS does not implicate a
759filesystem.
760
761(The Windows -C command flag "wide API support" has been at least
762temporarily retired in 5.8.1, and the -C has been repurposed, see
763L<perlrun>.)
764
87a942b1
JH
765Most probably the right way to do this would be this:
766L</"Virtualize operating system access">.
767
6d71adcd
NC
768=head2 Unicode in %ENV
769
770Currently the %ENV entries are always byte strings.
87a942b1 771See L</"Virtualize operating system access">.
6d71adcd 772
1f2e7916
JD
773=head2 Unicode and glob()
774
775Currently glob patterns and filenames returned from File::Glob::glob()
87a942b1 776are always byte strings. See L</"Virtualize operating system access">.
1f2e7916 777
6d71adcd
NC
778=head2 use less 'memory'
779
780Investigate trade offs to switch out perl's choices on memory usage.
781Particularly perl should be able to give memory back.
782
783This task is incremental - even a little bit of work on it will help.
784
785=head2 Re-implement C<:unique> in a way that is actually thread-safe
786
787The old implementation made bad assumptions on several levels. A good 90%
788solution might be just to make C<:unique> work to share the string buffer
789of SvPVs. That way large constant strings can be shared between ithreads,
790such as the configuration information in F<Config>.
791
792=head2 Make tainting consistent
793
794Tainting would be easier to use if it didn't take documented shortcuts and
795allow taint to "leak" everywhere within an expression.
796
797=head2 readpipe(LIST)
798
799system() accepts a LIST syntax (and a PROGRAM LIST syntax) to avoid
800running a shell. readpipe() (the function behind qx//) could be similarly
801extended.
802
6d71adcd
NC
803=head2 Audit the code for destruction ordering assumptions
804
805Change 25773 notes
806
807 /* Need to check SvMAGICAL, as during global destruction it may be that
808 AvARYLEN(av) has been freed before av, and hence the SvANY() pointer
809 is now part of the linked list of SV heads, rather than pointing to
810 the original body. */
811 /* FIXME - audit the code for other bugs like this one. */
812
813adding the C<SvMAGICAL> check to
814
815 if (AvARYLEN(av) && SvMAGICAL(AvARYLEN(av))) {
816 MAGIC *mg = mg_find (AvARYLEN(av), PERL_MAGIC_arylen);
817
818Go through the core and look for similar assumptions that SVs have particular
819types, as all bets are off during global destruction.
820
749904bf
JH
821=head2 Extend PerlIO and PerlIO::Scalar
822
823PerlIO::Scalar doesn't know how to truncate(). Implementing this
824would require extending the PerlIO vtable.
825
826Similarly the PerlIO vtable doesn't know about formats (write()), or
827about stat(), or chmod()/chown(), utime(), or flock().
828
829(For PerlIO::Scalar it's hard to see what e.g. mode bits or ownership
830would mean.)
831
832PerlIO doesn't do directories or symlinks, either: mkdir(), rmdir(),
833opendir(), closedir(), seekdir(), rewinddir(), glob(); symlink(),
834readlink().
835
94da6c29
JH
836See also L</"Virtualize operating system access">.
837
3236f110
NC
838=head2 -C on the #! line
839
840It should be possible to make -C work correctly if found on the #! line,
841given that all perl command line options are strict ASCII, and -C changes
842only the interpretation of non-ASCII characters, and not for the script file
843handle. To make it work needs some investigation of the ordering of function
844calls during startup, and (by implication) a bit of tweaking of that order.
845
d6c1e11f
JH
846=head2 Organize error messages
847
848Perl's diagnostics (error messages, see L<perldiag>) could use
a8d0aeb9 849reorganizing and formalizing so that each error message has its
d6c1e11f
JH
850stable-for-all-eternity unique id, categorized by severity, type, and
851subsystem. (The error messages would be listed in a datafile outside
c4bd451b
CB
852of the Perl source code, and the source code would only refer to the
853messages by the id.) This clean-up and regularizing should apply
d6c1e11f
JH
854for all croak() messages.
855
856This would enable all sorts of things: easier translation/localization
857of the messages (though please do keep in mind the caveats of
858L<Locale::Maketext> about too straightforward approaches to
859translation), filtering by severity, and instead of grepping for a
860particular error message one could look for a stable error id. (Of
861course, changing the error messages by default would break all the
862existing software depending on some particular error message...)
863
864This kind of functionality is known as I<message catalogs>. Look for
865inspiration for example in the catgets() system, possibly even use it
866if available-- but B<only> if available, all platforms will B<not>
de96509d 867have catgets().
d6c1e11f
JH
868
869For the really pure at heart, consider extending this item to cover
870also the warning messages (see L<perllexwarn>, C<warnings.pl>).
3236f110 871
0bdfc961 872=head1 Tasks that need a knowledge of the interpreter
3298bd4d 873
0bdfc961
NC
874These tasks would need C knowledge, and knowledge of how the interpreter works,
875or a willingness to learn.
3298bd4d 876
10517af5
JD
877=head2 forbid labels with keyword names
878
879Currently C<goto keyword> "computes" the label value:
880
881 $ perl -e 'goto print'
882 Can't find label 1 at -e line 1.
883
343c8006
JD
884It is controversial if the right way to avoid the confusion is to forbid
885labels with keyword names, or if it would be better to always treat
886bareword expressions after a "goto" as a label and never as a keyword.
10517af5 887
de6375e3
RGS
888=head2 truncate() prototype
889
890The prototype of truncate() is currently C<$$>. It should probably
891be C<*$> instead. (This is changed in F<opcode.pl>)
892
2d0587d8
RGS
893=head2 decapsulation of smart match argument
894
895Currently C<$foo ~~ $object> will die with the message "Smart matching a
896non-overloaded object breaks encapsulation". It would be nice to allow
c69ca1d4 897to bypass this by using explicitly the syntax C<$foo ~~ %$object> or
2d0587d8
RGS
898C<$foo ~~ @$object>.
899
565590b5
NC
900=head2 error reporting of [$a ; $b]
901
902Using C<;> inside brackets is a syntax error, and we don't propose to change
903that by giving it any meaning. However, it's not reported very helpfully:
904
905 $ perl -e '$a = [$b; $c];'
906 syntax error at -e line 1, near "$b;"
907 syntax error at -e line 1, near "$c]"
908 Execution of -e aborted due to compilation errors.
909
910It should be possible to hook into the tokeniser or the lexer, so that when a
911C<;> is parsed where it is not legal as a statement terminator (ie inside
912C<{}> used as a hashref, C<[]> or C<()>) it issues an error something like
913I<';' isn't legal inside an expression - if you need multiple statements use a
914do {...} block>. See the thread starting at
b4af8972 915L<http://www.xray.mpe.mpg.de/mailing-lists/perl5-porters/2008-09/msg00573.html>
565590b5 916
718140ec
NC
917=head2 lexicals used only once
918
919This warns:
920
921 $ perl -we '$pie = 42'
922 Name "main::pie" used only once: possible typo at -e line 1.
923
924This does not:
925
926 $ perl -we 'my $pie = 42'
927
928Logically all lexicals used only once should warn, if the user asks for
d6f4ea2e
SP
929warnings. An unworked RT ticket (#5087) has been open for almost seven
930years for this discrepancy.
718140ec 931
a3d15f9a
RGS
932=head2 UTF-8 revamp
933
85c006b6
KW
934The handling of Unicode is unclean in many places. In the regex engine
935there are especially many problems. The swash data structure could be
936replaced my something better. Inversion lists and maps are likely
937candidates. The whole Unicode database could be placed in-core for a
938huge speed-up. Only minimal work was done on the optimizer when utf8
939was added, with the result that the synthetic start class often will
940fail to narrow down the possible choices when given non-Latin1 input.
a3d15f9a
RGS
941
942=head2 Properly Unicode safe tokeniser and pads.
943
944The tokeniser isn't actually very UTF-8 clean. C<use utf8;> is a hack -
945variable names are stored in stashes as raw bytes, without the utf-8 flag
946set. The pad API only takes a C<char *> pointer, so that's all bytes too. The
947tokeniser ignores the UTF-8-ness of C<PL_rsfp>, or any SVs returned from
948source filters. All this could be fixed.
949
636e63cb
NC
950=head2 state variable initialization in list context
951
952Currently this is illegal:
953
954 state ($a, $b) = foo();
955
a2874905 956In Perl 6, C<state ($a) = foo();> and C<(state $a) = foo();> have different
a8d0aeb9 957semantics, which is tricky to implement in Perl 5 as currently they produce
a2874905 958the same opcode trees. The Perl 6 design is firm, so it would be good to
a8d0aeb9 959implement the necessary code in Perl 5. There are comments in
a2874905
NC
960C<Perl_newASSIGNOP()> that show the code paths taken by various assignment
961constructions involving state variables.
636e63cb 962
4fedb12c
RGS
963=head2 Implement $value ~~ 0 .. $range
964
965It would be nice to extend the syntax of the C<~~> operator to also
966understand numeric (and maybe alphanumeric) ranges.
a393eb28
RGS
967
968=head2 A does() built-in
969
970Like ref(), only useful. It would call the C<DOES> method on objects; it
971would also tell whether something can be dereferenced as an
972array/hash/etc., or used as a regexp, etc.
973L<http://www.xray.mpe.mpg.de/mailing-lists/perl5-porters/2007-03/msg00481.html>
974
975=head2 Tied filehandles and write() don't mix
976
977There is no method on tied filehandles to allow them to be called back by
978formats.
4fedb12c 979
53967bb9
RGS
980=head2 Propagate compilation hints to the debugger
981
982Currently a debugger started with -dE on the command-line doesn't see the
983features enabled by -E. More generally hints (C<$^H> and C<%^H>) aren't
984propagated to the debugger. Probably it would be a good thing to propagate
985hints from the innermost non-C<DB::> scope: this would make code eval'ed
986in the debugger see the features (and strictures, etc.) currently in
987scope.
988
d10fc472 989=head2 Attach/detach debugger from running program
1626a787 990
cd793d32
NC
991The old perltodo notes "With C<gdb>, you can attach the debugger to a running
992program if you pass the process ID. It would be good to do this with the Perl
0bdfc961
NC
993debugger on a running Perl program, although I'm not sure how it would be
994done." ssh and screen do this with named pipes in /tmp. Maybe we can too.
1626a787 995
0bdfc961
NC
996=head2 LVALUE functions for lists
997
998The old perltodo notes that lvalue functions don't work for list or hash
999slices. This would be good to fix.
1000
0bdfc961
NC
1001=head2 regexp optimiser optional
1002
1003The regexp optimiser is not optional. It should configurable to be, to allow
1004its performance to be measured, and its bugs to be easily demonstrated.
1005
ef36c6a7
RGS
1006=head2 C</w> regex modifier
1007
1008That flag would enable to match whole words, and also to interpolate
1009arrays as alternations. With it, C</P/w> would be roughly equivalent to:
1010
1011 do { local $"='|'; /\b(?:P)\b/ }
1012
b4af8972
RB
1013See
1014L<http://www.xray.mpe.mpg.de/mailing-lists/perl5-porters/2007-01/msg00400.html>
ef36c6a7
RGS
1015for the discussion.
1016
0bdfc961
NC
1017=head2 optional optimizer
1018
1019Make the peephole optimizer optional. Currently it performs two tasks as
1020it walks the optree - genuine peephole optimisations, and necessary fixups of
1021ops. It would be good to find an efficient way to switch out the
1022optimisations whilst keeping the fixups.
1023
1024=head2 You WANT *how* many
1025
1026Currently contexts are void, scalar and list. split has a special mechanism in
1027place to pass in the number of return values wanted. It would be useful to
1028have a general mechanism for this, backwards compatible and little speed hit.
1029This would allow proposals such as short circuiting sort to be implemented
1030as a module on CPAN.
1031
1032=head2 lexical aliases
1033
1034Allow lexical aliases (maybe via the syntax C<my \$alias = \$foo>.
1035
1036=head2 entersub XS vs Perl
1037
1038At the moment pp_entersub is huge, and has code to deal with entering both
1039perl and XS subroutines. Subroutine implementations rarely change between
1040perl and XS at run time, so investigate using 2 ops to enter subs (one for
1041XS, one for perl) and swap between if a sub is redefined.
2810d901 1042
de535794 1043=head2 Self-ties
2810d901 1044
de535794 1045Self-ties are currently illegal because they caused too many segfaults. Maybe
a8d0aeb9 1046the causes of these could be tracked down and self-ties on all types
de535794 1047reinstated.
0bdfc961
NC
1048
1049=head2 Optimize away @_
1050
1051The old perltodo notes "Look at the "reification" code in C<av.c>".
1052
87a942b1
JH
1053=head2 Virtualize operating system access
1054
1055Implement a set of "vtables" that virtualizes operating system access
1056(open(), mkdir(), unlink(), readdir(), getenv(), etc.) At the very
1057least these interfaces should take SVs as "name" arguments instead of
1058bare char pointers; probably the most flexible and extensible way
e1a3d5d1
JH
1059would be for the Perl-facing interfaces to accept HVs. The system
1060needs to be per-operating-system and per-file-system
1061hookable/filterable, preferably both from XS and Perl level
87a942b1
JH
1062(L<perlport/"Files and Filesystems"> is good reading at this point,
1063in fact, all of L<perlport> is.)
1064
e1a3d5d1
JH
1065This has actually already been implemented (but only for Win32),
1066take a look at F<iperlsys.h> and F<win32/perlhost.h>. While all Win32
1067variants go through a set of "vtables" for operating system access,
e1020413 1068non-Win32 systems currently go straight for the POSIX/Unix-style
e1a3d5d1
JH
1069system/library call. Similar system as for Win32 should be
1070implemented for all platforms. The existing Win32 implementation
1071probably does not need to survive alongside this proposed new
1072implementation, the approaches could be merged.
87a942b1
JH
1073
1074What would this give us? One often-asked-for feature this would
94da6c29
JH
1075enable is using Unicode for filenames, and other "names" like %ENV,
1076usernames, hostnames, and so forth.
1077(See L<perlunicode/"When Unicode Does Not Happen">.)
1078
1079But this kind of virtualization would also allow for things like
1080virtual filesystems, virtual networks, and "sandboxes" (though as long
1081as dynamic loading of random object code is allowed, not very safe
1082sandboxes since external code of course know not of Perl's vtables).
1083An example of a smaller "sandbox" is that this feature can be used to
1084implement per-thread working directories: Win32 already does this.
1085
1086See also L</"Extend PerlIO and PerlIO::Scalar">.
87a942b1 1087
ac6197af
NC
1088=head2 Investigate PADTMP hash pessimisation
1089
9a2f2e6b 1090The peephole optimiser converts constants used for hash key lookups to shared
057163d7 1091hash key scalars. Under ithreads, something is undoing this work.
b4af8972
RB
1092See
1093L<http://www.xray.mpe.mpg.de/mailing-lists/perl5-porters/2007-09/msg00793.html>
ac6197af 1094
057163d7
NC
1095=head2 Store the current pad in the OP slab allocator
1096
1097=for clarification
1098I hope that I got that "current pad" part correct
1099
1100Currently we leak ops in various cases of parse failure. I suggested that we
1101could solve this by always using the op slab allocator, and walking it to
1102free ops. Dave comments that as some ops are already freed during optree
1103creation one would have to mark which ops are freed, and not double free them
1104when walking the slab. He notes that one problem with this is that for some ops
1105you have to know which pad was current at the time of allocation, which does
1106change. I suggested storing a pointer to the current pad in the memory allocated
1107for the slab, and swapping to a new slab each time the pad changes. Dave thinks
1108that this would work.
1109
52960e22
JC
1110=head2 repack the optree
1111
1112Repacking the optree after execution order is determined could allow
057163d7
NC
1113removal of NULL ops, and optimal ordering of OPs with respect to cache-line
1114filling. The slab allocator could be reused for this purpose. I think that
1115the best way to do this is to make it an optional step just before the
1116completed optree is attached to anything else, and to use the slab allocator
1117unchanged, so that freeing ops is identical whether or not this step runs.
1118Note that the slab allocator allocates ops downwards in memory, so one would
1119have to actually "allocate" the ops in reverse-execution order to get them
1120contiguous in memory in execution order.
1121
b4af8972
RB
1122See
1123L<http://www.nntp.perl.org/group/perl.perl5.porters/2007/12/msg131975.html>
057163d7
NC
1124
1125Note that running this copy, and then freeing all the old location ops would
1126cause their slabs to be freed, which would eliminate possible memory wastage if
1127the previous suggestion is implemented, and we swap slabs more frequently.
52960e22 1128
12e06b6f
NC
1129=head2 eliminate incorrect line numbers in warnings
1130
1131This code
1132
1133 use warnings;
1134 my $undef;
1135
1136 if ($undef == 3) {
1137 } elsif ($undef == 0) {
1138 }
1139
18a16cc5 1140used to produce this output:
12e06b6f
NC
1141
1142 Use of uninitialized value in numeric eq (==) at wrong.pl line 4.
1143 Use of uninitialized value in numeric eq (==) at wrong.pl line 4.
1144
18a16cc5
NC
1145where the line of the second warning was misreported - it should be line 5.
1146Rafael fixed this - the problem arose because there was no nextstate OP
1147between the execution of the C<if> and the C<elsif>, hence C<PL_curcop> still
1148reports that the currently executing line is line 4. The solution was to inject
1149a nextstate OPs for each C<elsif>, although it turned out that the nextstate
1150OP needed to be a nulled OP, rather than a live nextstate OP, else other line
1151numbers became misreported. (Jenga!)
12e06b6f
NC
1152
1153The problem is more general than C<elsif> (although the C<elsif> case is the
1154most common and the most confusing). Ideally this code
1155
1156 use warnings;
1157 my $undef;
1158
1159 my $a = $undef + 1;
1160 my $b
1161 = $undef
1162 + 1;
1163
1164would produce this output
1165
1166 Use of uninitialized value $undef in addition (+) at wrong.pl line 4.
1167 Use of uninitialized value $undef in addition (+) at wrong.pl line 7.
1168
1169(rather than lines 4 and 5), but this would seem to require every OP to carry
1170(at least) line number information.
1171
1172What might work is to have an optional line number in memory just before the
1173BASEOP structure, with a flag bit in the op to say whether it's present.
1174Initially during compile every OP would carry its line number. Then add a late
1175pass to the optimiser (potentially combined with L</repack the optree>) which
1176looks at the two ops on every edge of the graph of the execution path. If
1177the line number changes, flags the destination OP with this information.
1178Once all paths are traced, replace every op with the flag with a
1179nextstate-light op (that just updates C<PL_curcop>), which in turn then passes
1180control on to the true op. All ops would then be replaced by variants that
1181do not store the line number. (Which, logically, why it would work best in
1182conjunction with L</repack the optree>, as that is already copying/reallocating
1183all the OPs)
1184
18a16cc5
NC
1185(Although I should note that we're not certain that doing this for the general
1186case is worth it)
1187
52960e22
JC
1188=head2 optimize tail-calls
1189
1190Tail-calls present an opportunity for broadly applicable optimization;
1191anywhere that C<< return foo(...) >> is called, the outer return can
1192be replaced by a goto, and foo will return directly to the outer
1193caller, saving (conservatively) 25% of perl's call&return cost, which
1194is relatively higher than in C. The scheme language is known to do
1195this heavily. B::Concise provides good insight into where this
1196optimization is possible, ie anywhere entersub,leavesub op-sequence
1197occurs.
1198
1199 perl -MO=Concise,-exec,a,b,-main -e 'sub a{ 1 }; sub b {a()}; b(2)'
1200
1201Bottom line on this is probably a new pp_tailcall function which
1202combines the code in pp_entersub, pp_leavesub. This should probably
1203be done 1st in XS, and using B::Generate to patch the new OP into the
1204optrees.
1205
0c397127
KW
1206=head2 Add C<00dddd>
1207
1208It has been proposed that octal constants be specifiable through the syntax
1209C<0oddddd>, parallel to the existing construct to specify hex constants
1210C<0xddddd>
1211
0bdfc961
NC
1212=head1 Big projects
1213
1214Tasks that will get your name mentioned in the description of the "Highlights
fd2dadea 1215of 5.14"
0bdfc961
NC
1216
1217=head2 make ithreads more robust
1218
4e577f8b 1219Generally make ithreads more robust. See also L</iCOW>
0bdfc961
NC
1220
1221This task is incremental - even a little bit of work on it will help, and
1222will be greatly appreciated.
1223
07577ec1
FC
1224One bit would be to determine how to clone directory handles on systems
1225without a C<fchdir> function (in sv.c:Perl_dirp_dup).
6c047da7 1226
59c7f7d5
RGS
1227Fix Perl_sv_dup, et al so that threads can return objects.
1228
0bdfc961
NC
1229=head2 iCOW
1230
1231Sarathy and Arthur have a proposal for an improved Copy On Write which
1232specifically will be able to COW new ithreads. If this can be implemented
1233it would be a good thing.
1234
1235=head2 (?{...}) closures in regexps
1236
1237Fix (or rewrite) the implementation of the C</(?{...})/> closures.
1238
6bda09f9
YO
1239=head2 Add class set operations to regexp engine
1240
1241Apparently these are quite useful. Anyway, Jeffery Friedl wants them.
1242
1243demerphq has this on his todo list, but right at the bottom.
44a7a252
JV
1244
1245
1246=head1 Tasks for microperl
1247
1248
1249[ Each and every one of these may be obsolete, but they were listed
1250 in the old Todo.micro file]
1251
1252
1253=head2 make creating uconfig.sh automatic
1254
1255=head2 make creating Makefile.micro automatic
1256
1257=head2 do away with fork/exec/wait?
1258
1259(system, popen should be enough?)
1260
1261=head2 some of the uconfig.sh really needs to be probed (using cc) in buildtime:
1262
1263(uConfigure? :-) native datatype widths and endianness come to mind
1264