This is a live mirror of the Perl 5 development currently hosted at https://github.com/perl/perl5
Retract #21273, #21264, and #21263 as requested by Schwern.
[perl5.git] / pod / perliol.pod
CommitLineData
50b80e25
NIS
1=head1 NAME
2
3perliol - C API for Perl's implementation of IO in Layers.
4
5=head1 SYNOPSIS
6
7 /* Defining a layer ... */
8 #include <perliol.h>
9
50b80e25
NIS
10=head1 DESCRIPTION
11
9d799145
NIS
12This document describes the behavior and implementation of the PerlIO
13abstraction described in L<perlapio> when C<USE_PERLIO> is defined (and
14C<USE_SFIO> is not).
50b80e25
NIS
15
16=head2 History and Background
17
9d799145
NIS
18The PerlIO abstraction was introduced in perl5.003_02 but languished as
19just an abstraction until perl5.7.0. However during that time a number
d1be9408 20of perl extensions switched to using it, so the API is mostly fixed to
9d799145 21maintain (source) compatibility.
50b80e25 22
9d799145
NIS
23The aim of the implementation is to provide the PerlIO API in a flexible
24and platform neutral manner. It is also a trial of an "Object Oriented
25C, with vtables" approach which may be applied to perl6.
50b80e25
NIS
26
27=head2 Layers vs Disciplines
28
9d799145
NIS
29Initial discussion of the ability to modify IO streams behaviour used
30the term "discipline" for the entities which were added. This came (I
31believe) from the use of the term in "sfio", which in turn borrowed it
32from "line disciplines" on Unix terminals. However, this document (and
33the C code) uses the term "layer".
34
1d11c889
JH
35This is, I hope, a natural term given the implementation, and should
36avoid connotations that are inherent in earlier uses of "discipline"
37for things which are rather different.
50b80e25
NIS
38
39=head2 Data Structures
40
41The basic data structure is a PerlIOl:
42
43 typedef struct _PerlIO PerlIOl;
44 typedef struct _PerlIO_funcs PerlIO_funcs;
45 typedef PerlIOl *PerlIO;
46
47 struct _PerlIO
48 {
49 PerlIOl * next; /* Lower layer */
50 PerlIO_funcs * tab; /* Functions for this layer */
51 IV flags; /* Various flags for state */
52 };
53
1d11c889
JH
54A C<PerlIOl *> is a pointer to the struct, and the I<application>
55level C<PerlIO *> is a pointer to a C<PerlIOl *> - i.e. a pointer
56to a pointer to the struct. This allows the application level C<PerlIO *>
57to remain constant while the actual C<PerlIOl *> underneath
58changes. (Compare perl's C<SV *> which remains constant while its
59C<sv_any> field changes as the scalar's type changes.) An IO stream is
60then in general represented as a pointer to this linked-list of
61"layers".
50b80e25 62
9d799145 63It should be noted that because of the double indirection in a C<PerlIO *>,
d4165bde 64a C<< &(perlio->next) >> "is" a C<PerlIO *>, and so to some degree
11e1c8f2 65at least one layer can use the "standard" API on the next layer down.
50b80e25
NIS
66
67A "layer" is composed of two parts:
68
69=over 4
70
210b36aa 71=item 1.
50b80e25 72
210b36aa
AMS
73The functions and attributes of the "layer class".
74
75=item 2.
76
77The per-instance data for a particular handle.
50b80e25
NIS
78
79=back
80
81=head2 Functions and Attributes
82
9d799145
NIS
83The functions and attributes are accessed via the "tab" (for table)
84member of C<PerlIOl>. The functions (methods of the layer "class") are
85fixed, and are defined by the C<PerlIO_funcs> type. They are broadly the
86same as the public C<PerlIO_xxxxx> functions:
50b80e25 87
b76cc8ba
NIS
88 struct _PerlIO_funcs
89 {
2dc2558e 90 Size_t fsize;
b76cc8ba
NIS
91 char * name;
92 Size_t size;
93 IV kind;
2dc2558e 94 IV (*Pushed)(pTHX_ PerlIO *f,const char *mode,SV *arg, PerlIO_funcs *tab);
d4165bde 95 IV (*Popped)(pTHX_ PerlIO *f);
b76cc8ba
NIS
96 PerlIO * (*Open)(pTHX_ PerlIO_funcs *tab,
97 AV *layers, IV n,
98 const char *mode,
99 int fd, int imode, int perm,
100 PerlIO *old,
101 int narg, SV **args);
86e05cf2 102 IV (*Binmode)(pTHX_ PerlIO *f);
d4165bde
SB
103 SV * (*Getarg)(pTHX_ PerlIO *f, CLONE_PARAMS *param, int flags)
104 IV (*Fileno)(pTHX_ PerlIO *f);
105 PerlIO * (*Dup)(pTHX_ PerlIO *f, PerlIO *o, CLONE_PARAMS *param, int flags)
b76cc8ba 106 /* Unix-like functions - cf sfio line disciplines */
d4165bde
SB
107 SSize_t (*Read)(pTHX_ PerlIO *f, void *vbuf, Size_t count);
108 SSize_t (*Unread)(pTHX_ PerlIO *f, const void *vbuf, Size_t count);
109 SSize_t (*Write)(pTHX_ PerlIO *f, const void *vbuf, Size_t count);
110 IV (*Seek)(pTHX_ PerlIO *f, Off_t offset, int whence);
111 Off_t (*Tell)(pTHX_ PerlIO *f);
112 IV (*Close)(pTHX_ PerlIO *f);
b76cc8ba 113 /* Stdio-like buffered IO functions */
d4165bde
SB
114 IV (*Flush)(pTHX_ PerlIO *f);
115 IV (*Fill)(pTHX_ PerlIO *f);
116 IV (*Eof)(pTHX_ PerlIO *f);
117 IV (*Error)(pTHX_ PerlIO *f);
118 void (*Clearerr)(pTHX_ PerlIO *f);
119 void (*Setlinebuf)(pTHX_ PerlIO *f);
b76cc8ba 120 /* Perl's snooping functions */
d4165bde
SB
121 STDCHAR * (*Get_base)(pTHX_ PerlIO *f);
122 Size_t (*Get_bufsiz)(pTHX_ PerlIO *f);
123 STDCHAR * (*Get_ptr)(pTHX_ PerlIO *f);
124 SSize_t (*Get_cnt)(pTHX_ PerlIO *f);
125 void (*Set_ptrcnt)(pTHX_ PerlIO *f,STDCHAR *ptr,SSize_t cnt);
b76cc8ba
NIS
126 };
127
2dc2558e
NIS
128The first few members of the struct give a function table size for
129compatibility check "name" for the layer, the size to C<malloc> for the per-instance data,
130and some flags which are attributes of the class as whole (such as whether it is a buffering
9d799145 131layer), then follow the functions which fall into four basic groups:
50b80e25
NIS
132
133=over 4
134
aa500c9e 135=item 1.
50b80e25 136
aa500c9e 137Opening and setup functions
50b80e25 138
aa500c9e 139=item 2.
50b80e25 140
aa500c9e
RB
141Basic IO operations
142
143=item 3.
144
145Stdio class buffering options.
146
147=item 4.
148
149Functions to support Perl's traditional "fast" access to the buffer.
50b80e25
NIS
150
151=back
152
1d11c889
JH
153A layer does not have to implement all the functions, but the whole
154table has to be present. Unimplemented slots can be NULL (which will
155result in an error when called) or can be filled in with stubs to
156"inherit" behaviour from a "base class". This "inheritance" is fixed
157for all instances of the layer, but as the layer chooses which stubs
158to populate the table, limited "multiple inheritance" is possible.
50b80e25
NIS
159
160=head2 Per-instance Data
161
1d11c889
JH
162The per-instance data are held in memory beyond the basic PerlIOl
163struct, by making a PerlIOl the first member of the layer's struct
164thus:
50b80e25
NIS
165
166 typedef struct
167 {
168 struct _PerlIO base; /* Base "class" info */
169 STDCHAR * buf; /* Start of buffer */
170 STDCHAR * end; /* End of valid part of buffer */
171 STDCHAR * ptr; /* Current position in buffer */
172 Off_t posn; /* Offset of buf into the file */
173 Size_t bufsiz; /* Real size of buffer */
174 IV oneword; /* Emergency buffer */
175 } PerlIOBuf;
176
1d11c889
JH
177In this way (as for perl's scalars) a pointer to a PerlIOBuf can be
178treated as a pointer to a PerlIOl.
50b80e25
NIS
179
180=head2 Layers in action.
181
182 table perlio unix
183 | |
184 +-----------+ +----------+ +--------+
185 PerlIO ->| |--->| next |--->| NULL |
186 +-----------+ +----------+ +--------+
187 | | | buffer | | fd |
188 +-----------+ | | +--------+
189 | | +----------+
190
191
192The above attempts to show how the layer scheme works in a simple case.
9d799145
NIS
193The application's C<PerlIO *> points to an entry in the table(s)
194representing open (allocated) handles. For example the first three slots
195in the table correspond to C<stdin>,C<stdout> and C<stderr>. The table
196in turn points to the current "top" layer for the handle - in this case
197an instance of the generic buffering layer "perlio". That layer in turn
198points to the next layer down - in this case the lowlevel "unix" layer.
50b80e25 199
9d799145
NIS
200The above is roughly equivalent to a "stdio" buffered stream, but with
201much more flexibility:
50b80e25
NIS
202
203=over 4
204
205=item *
206
9d799145
NIS
207If Unix level C<read>/C<write>/C<lseek> is not appropriate for (say)
208sockets then the "unix" layer can be replaced (at open time or even
209dynamically) with a "socket" layer.
50b80e25
NIS
210
211=item *
212
1d11c889
JH
213Different handles can have different buffering schemes. The "top"
214layer could be the "mmap" layer if reading disk files was quicker
215using C<mmap> than C<read>. An "unbuffered" stream can be implemented
216simply by not having a buffer layer.
50b80e25
NIS
217
218=item *
219
220Extra layers can be inserted to process the data as it flows through.
9d799145 221This was the driving need for including the scheme in perl 5.7.0+ - we
d1be9408 222needed a mechanism to allow data to be translated between perl's
9d799145
NIS
223internal encoding (conceptually at least Unicode as UTF-8), and the
224"native" format used by the system. This is provided by the
225":encoding(xxxx)" layer which typically sits above the buffering layer.
50b80e25
NIS
226
227=item *
228
1d11c889
JH
229A layer can be added that does "\n" to CRLF translation. This layer
230can be used on any platform, not just those that normally do such
231things.
50b80e25
NIS
232
233=back
234
235=head2 Per-instance flag bits
236
1d11c889
JH
237The generic flag bits are a hybrid of C<O_XXXXX> style flags deduced
238from the mode string passed to C<PerlIO_open()>, and state bits for
239typical buffer layers.
50b80e25 240
9d799145 241=over 4
50b80e25
NIS
242
243=item PERLIO_F_EOF
244
245End of file.
246
247=item PERLIO_F_CANWRITE
248
3039a93d 249Writes are permitted, i.e. opened as "w" or "r+" or "a", etc.
50b80e25
NIS
250
251=item PERLIO_F_CANREAD
252
3039a93d 253Reads are permitted i.e. opened "r" or "w+" (or even "a+" - ick).
50b80e25
NIS
254
255=item PERLIO_F_ERROR
256
d4165bde 257An error has occurred (for C<PerlIO_error()>).
50b80e25
NIS
258
259=item PERLIO_F_TRUNCATE
260
261Truncate file suggested by open mode.
262
263=item PERLIO_F_APPEND
264
265All writes should be appends.
266
267=item PERLIO_F_CRLF
268
11e1c8f2
PP
269Layer is performing Win32-like "\n" mapped to CR,LF for output and CR,LF
270mapped to "\n" for input. Normally the provided "crlf" layer is the only
271layer that need bother about this. C<PerlIO_binmode()> will mess with this
9d799145
NIS
272flag rather than add/remove layers if the C<PERLIO_K_CANCRLF> bit is set
273for the layers class.
50b80e25
NIS
274
275=item PERLIO_F_UTF8
276
3039a93d 277Data written to this layer should be UTF-8 encoded; data provided
50b80e25
NIS
278by this layer should be considered UTF-8 encoded. Can be set on any layer
279by ":utf8" dummy layer. Also set on ":encoding" layer.
280
281=item PERLIO_F_UNBUF
282
283Layer is unbuffered - i.e. write to next layer down should occur for
284each write to this layer.
285
286=item PERLIO_F_WRBUF
287
288The buffer for this layer currently holds data written to it but not sent
289to next layer.
290
291=item PERLIO_F_RDBUF
292
293The buffer for this layer currently holds unconsumed data read from
294layer below.
295
296=item PERLIO_F_LINEBUF
297
9d799145
NIS
298Layer is line buffered. Write data should be passed to next layer down
299whenever a "\n" is seen. Any data beyond the "\n" should then be
300processed.
50b80e25
NIS
301
302=item PERLIO_F_TEMP
303
9d799145 304File has been C<unlink()>ed, or should be deleted on C<close()>.
50b80e25
NIS
305
306=item PERLIO_F_OPEN
307
308Handle is open.
309
310=item PERLIO_F_FASTGETS
311
9d799145
NIS
312This instance of this layer supports the "fast C<gets>" interface.
313Normally set based on C<PERLIO_K_FASTGETS> for the class and by the
d1be9408 314existence of the function(s) in the table. However a class that
50b80e25
NIS
315normally provides that interface may need to avoid it on a
316particular instance. The "pending" layer needs to do this when
d1be9408 317it is pushed above a layer which does not support the interface.
9d799145 318(Perl's C<sv_gets()> does not expect the streams fast C<gets> behaviour
50b80e25
NIS
319to change during one "get".)
320
321=back
322
323=head2 Methods in Detail
324
325=over 4
326
e2d9456f 327=item fsize
2dc2558e
NIS
328
329 Size_t fsize;
330
a489db4d
JH
331Size of the function table. This is compared against the value PerlIO
332code "knows" as a compatibility check. Future versions I<may> be able
333to tolerate layers compiled against an old version of the headers.
2dc2558e 334
5cb3728c
RB
335=item name
336
337 char * name;
d4165bde
SB
338
339The name of the layer whose open() method Perl should invoke on
340open(). For example if the layer is called APR, you will call:
341
342 open $fh, ">:APR", ...
343
344and Perl knows that it has to invoke the PerlIOAPR_open() method
345implemented by the APR layer.
346
5cb3728c
RB
347=item size
348
349 Size_t size;
d4165bde
SB
350
351The size of the per-instance data structure, e.g.:
352
353 sizeof(PerlIOAPR)
354
a489db4d
JH
355If this field is zero then C<PerlIO_pushed> does not malloc anything
356and assumes layer's Pushed function will do any required layer stack
357manipulation - used to avoid malloc/free overhead for dummy layers.
2dc2558e
NIS
358If the field is non-zero it must be at least the size of C<PerlIOl>,
359C<PerlIO_pushed> will allocate memory for the layer's data structures
360and link new layer onto the stream's stack. (If the layer's Pushed
361method returns an error indication the layer is popped again.)
362
5cb3728c
RB
363=item kind
364
365 IV kind;
d4165bde 366
d4165bde
SB
367=over 4
368
369=item * PERLIO_K_BUFFERED
370
86e05cf2
NIS
371The layer is buffered.
372
373=item * PERLIO_K_RAW
374
375The layer is acceptable to have in a binmode(FH) stack - i.e. it does not
376(or will configure itself not to) transform bytes passing through it.
377
d4165bde
SB
378=item * PERLIO_K_CANCRLF
379
86e05cf2
NIS
380Layer can translate between "\n" and CRLF line ends.
381
d4165bde
SB
382=item * PERLIO_K_FASTGETS
383
86e05cf2
NIS
384Layer allows buffer snooping.
385
d4165bde
SB
386=item * PERLIO_K_MULTIARG
387
388Used when the layer's open() accepts more arguments than usual. The
389extra arguments should come not before the C<MODE> argument. When this
390flag is used it's up to the layer to validate the args.
391
d4165bde
SB
392=back
393
5cb3728c
RB
394=item Pushed
395
396 IV (*Pushed)(pTHX_ PerlIO *f,const char *mode, SV *arg);
50b80e25 397
1d11c889
JH
398The only absolutely mandatory method. Called when the layer is pushed
399onto the stack. The C<mode> argument may be NULL if this occurs
400post-open. The C<arg> will be non-C<NULL> if an argument string was
401passed. In most cases this should call C<PerlIOBase_pushed()> to
402convert C<mode> into the appropriate C<PERLIO_F_XXXXX> flags in
403addition to any actions the layer itself takes. If a layer is not
404expecting an argument it need neither save the one passed to it, nor
405provide C<Getarg()> (it could perhaps C<Perl_warn> that the argument
406was un-expected).
50b80e25 407
d4165bde
SB
408Returns 0 on success. On failure returns -1 and should set errno.
409
5cb3728c
RB
410=item Popped
411
412 IV (*Popped)(pTHX_ PerlIO *f);
50b80e25 413
1d11c889
JH
414Called when the layer is popped from the stack. A layer will normally
415be popped after C<Close()> is called. But a layer can be popped
416without being closed if the program is dynamically managing layers on
417the stream. In such cases C<Popped()> should free any resources
418(buffers, translation tables, ...) not held directly in the layer's
419struct. It should also C<Unread()> any unconsumed data that has been
420read and buffered from the layer below back to that layer, so that it
421can be re-provided to what ever is now above.
b76cc8ba 422
3077d0b1
JH
423Returns 0 on success and failure. If C<Popped()> returns I<true> then
424I<perlio.c> assumes that either the layer has popped itself, or the
425layer is super special and needs to be retained for other reasons.
426In most cases it should return I<false>.
d4165bde 427
5cb3728c
RB
428=item Open
429
430 PerlIO * (*Open)(...);
b76cc8ba 431
1d11c889
JH
432The C<Open()> method has lots of arguments because it combines the
433functions of perl's C<open>, C<PerlIO_open>, perl's C<sysopen>,
434C<PerlIO_fdopen> and C<PerlIO_reopen>. The full prototype is as
435follows:
b76cc8ba
NIS
436
437 PerlIO * (*Open)(pTHX_ PerlIO_funcs *tab,
438 AV *layers, IV n,
439 const char *mode,
440 int fd, int imode, int perm,
441 PerlIO *old,
442 int narg, SV **args);
443
1d11c889
JH
444Open should (perhaps indirectly) call C<PerlIO_allocate()> to allocate
445a slot in the table and associate it with the layers information for
446the opened file, by calling C<PerlIO_push>. The I<layers> AV is an
447array of all the layers destined for the C<PerlIO *>, and any
448arguments passed to them, I<n> is the index into that array of the
449layer being called. The macro C<PerlIOArg> will return a (possibly
450C<NULL>) SV * for the argument passed to the layer.
451
452The I<mode> string is an "C<fopen()>-like" string which would match
453the regular expression C</^[I#]?[rwa]\+?[bt]?$/>.
454
455The C<'I'> prefix is used during creation of C<stdin>..C<stderr> via
456special C<PerlIO_fdopen> calls; the C<'#'> prefix means that this is
457C<sysopen> and that I<imode> and I<perm> should be passed to
458C<PerlLIO_open3>; C<'r'> means B<r>ead, C<'w'> means B<w>rite and
459C<'a'> means B<a>ppend. The C<'+'> suffix means that both reading and
a489db4d
JH
460writing/appending are permitted. The C<'b'> suffix means file should
461be binary, and C<'t'> means it is text. (Almost all layers should do
462the IO in binary mode, and ignore the b/t bits. The C<:crlf> layer
463should be pushed to handle the distinction.)
1d11c889
JH
464
465If I<old> is not C<NULL> then this is a C<PerlIO_reopen>. Perl itself
466does not use this (yet?) and semantics are a little vague.
467
468If I<fd> not negative then it is the numeric file descriptor I<fd>,
469which will be open in a manner compatible with the supplied mode
470string, the call is thus equivalent to C<PerlIO_fdopen>. In this case
471I<nargs> will be zero.
472
473If I<nargs> is greater than zero then it gives the number of arguments
474passed to C<open>, otherwise it will be 1 if for example
475C<PerlIO_open> was called. In simple cases SvPV_nolen(*args) is the
476pathname to open.
477
478Having said all that translation-only layers do not need to provide
479C<Open()> at all, but rather leave the opening to a lower level layer
480and wait to be "pushed". If a layer does provide C<Open()> it should
481normally call the C<Open()> method of next layer down (if any) and
482then push itself on top if that succeeds.
b76cc8ba 483
3077d0b1
JH
484If C<PerlIO_push> was performed and open has failed, it must
485C<PerlIO_pop> itself, since if it's not, the layer won't be removed
486and may cause bad problems.
487
d4165bde
SB
488Returns C<NULL> on failure.
489
86e05cf2
NIS
490=item Binmode
491
492 IV (*Binmode)(pTHX_ PerlIO *f);
493
494Optional. Used when C<:raw> layer is pushed (explicitly or as a result
495of binmode(FH)). If not present layer will be popped. If present
496should configure layer as binary (or pop itself) and return 0.
497If it returns -1 for error C<binmode> will fail with layer
498still on the stack.
499
5cb3728c
RB
500=item Getarg
501
502 SV * (*Getarg)(pTHX_ PerlIO *f,
503 CLONE_PARAMS *param, int flags);
b76cc8ba 504
d4165bde
SB
505Optional. If present should return an SV * representing the string
506argument passed to the layer when it was
507pushed. e.g. ":encoding(ascii)" would return an SvPV with value
508"ascii". (I<param> and I<flags> arguments can be ignored in most
509cases)
b76cc8ba 510
5cb3728c
RB
511=item Fileno
512
513 IV (*Fileno)(pTHX_ PerlIO *f);
b76cc8ba 514
d1be9408 515Returns the Unix/Posix numeric file descriptor for the handle. Normally
b76cc8ba
NIS
516C<PerlIOBase_fileno()> (which just asks next layer down) will suffice
517for this.
50b80e25 518
a489db4d
JH
519Returns -1 on error, which is considered to include the case where the
520layer cannot provide such a file descriptor.
d4165bde 521
5cb3728c
RB
522=item Dup
523
524 PerlIO * (*Dup)(pTHX_ PerlIO *f, PerlIO *o,
525 CLONE_PARAMS *param, int flags);
d4165bde 526
2dc2558e
NIS
527XXX: Needs more docs.
528
a489db4d
JH
529Used as part of the "clone" process when a thread is spawned (in which
530case param will be non-NULL) and when a stream is being duplicated via
531'&' in the C<open>.
d4165bde
SB
532
533Similar to C<Open>, returns PerlIO* on success, C<NULL> on failure.
534
5cb3728c
RB
535=item Read
536
537 SSize_t (*Read)(pTHX_ PerlIO *f, void *vbuf, Size_t count);
d4165bde
SB
538
539Basic read operation.
50b80e25 540
d4165bde
SB
541Typically will call C<Fill> and manipulate pointers (possibly via the
542API). C<PerlIOBuf_read()> may be suitable for derived classes which
543provide "fast gets" methods.
50b80e25 544
d4165bde
SB
545Returns actual bytes read, or -1 on an error.
546
5cb3728c
RB
547=item Unread
548
549 SSize_t (*Unread)(pTHX_ PerlIO *f,
550 const void *vbuf, Size_t count);
50b80e25 551
9d799145
NIS
552A superset of stdio's C<ungetc()>. Should arrange for future reads to
553see the bytes in C<vbuf>. If there is no obviously better implementation
554then C<PerlIOBase_unread()> provides the function by pushing a "fake"
555"pending" layer above the calling layer.
50b80e25 556
d4165bde
SB
557Returns the number of unread chars.
558
5cb3728c
RB
559=item Write
560
561 SSize_t (*Write)(PerlIO *f, const void *vbuf, Size_t count);
50b80e25 562
d4165bde 563Basic write operation.
50b80e25 564
d4165bde
SB
565Returns bytes written or -1 on an error.
566
5cb3728c
RB
567=item Seek
568
569 IV (*Seek)(pTHX_ PerlIO *f, Off_t offset, int whence);
50b80e25 570
1d11c889
JH
571Position the file pointer. Should normally call its own C<Flush>
572method and then the C<Seek> method of next layer down.
50b80e25 573
d4165bde
SB
574Returns 0 on success, -1 on failure.
575
5cb3728c
RB
576=item Tell
577
578 Off_t (*Tell)(pTHX_ PerlIO *f);
50b80e25 579
9d799145
NIS
580Return the file pointer. May be based on layers cached concept of
581position to avoid overhead.
50b80e25 582
d4165bde
SB
583Returns -1 on failure to get the file pointer.
584
5cb3728c
RB
585=item Close
586
587 IV (*Close)(pTHX_ PerlIO *f);
50b80e25 588
9d799145
NIS
589Close the stream. Should normally call C<PerlIOBase_close()> to flush
590itself and close layers below, and then deallocate any data structures
591(buffers, translation tables, ...) not held directly in the data
592structure.
50b80e25 593
d4165bde
SB
594Returns 0 on success, -1 on failure.
595
5cb3728c
RB
596=item Flush
597
598 IV (*Flush)(pTHX_ PerlIO *f);
50b80e25 599
9d799145
NIS
600Should make stream's state consistent with layers below. That is, any
601buffered write data should be written, and file position of lower layers
d1be9408 602adjusted for data read from below but not actually consumed.
b76cc8ba 603(Should perhaps C<Unread()> such data to the lower layer.)
50b80e25 604
d4165bde
SB
605Returns 0 on success, -1 on failure.
606
5cb3728c
RB
607=item Fill
608
609 IV (*Fill)(pTHX_ PerlIO *f);
d4165bde
SB
610
611The buffer for this layer should be filled (for read) from layer
612below. When you "subclass" PerlIOBuf layer, you want to use its
613I<_read> method and to supply your own fill method, which fills the
614PerlIOBuf's buffer.
50b80e25 615
d4165bde 616Returns 0 on success, -1 on failure.
50b80e25 617
5cb3728c
RB
618=item Eof
619
620 IV (*Eof)(pTHX_ PerlIO *f);
50b80e25 621
9d799145 622Return end-of-file indicator. C<PerlIOBase_eof()> is normally sufficient.
50b80e25 623
d4165bde
SB
624Returns 0 on end-of-file, 1 if not end-of-file, -1 on error.
625
5cb3728c
RB
626=item Error
627
628 IV (*Error)(pTHX_ PerlIO *f);
50b80e25 629
9d799145 630Return error indicator. C<PerlIOBase_error()> is normally sufficient.
50b80e25 631
d4165bde
SB
632Returns 1 if there is an error (usually when C<PERLIO_F_ERROR> is set,
6330 otherwise.
634
5cb3728c
RB
635=item Clearerr
636
637 void (*Clearerr)(pTHX_ PerlIO *f);
50b80e25 638
9d799145
NIS
639Clear end-of-file and error indicators. Should call C<PerlIOBase_clearerr()>
640to set the C<PERLIO_F_XXXXX> flags, which may suffice.
50b80e25 641
5cb3728c
RB
642=item Setlinebuf
643
644 void (*Setlinebuf)(pTHX_ PerlIO *f);
50b80e25 645
b76cc8ba
NIS
646Mark the stream as line buffered. C<PerlIOBase_setlinebuf()> sets the
647PERLIO_F_LINEBUF flag and is normally sufficient.
50b80e25 648
5cb3728c
RB
649=item Get_base
650
651 STDCHAR * (*Get_base)(pTHX_ PerlIO *f);
50b80e25
NIS
652
653Allocate (if not already done so) the read buffer for this layer and
d4165bde 654return pointer to it. Return NULL on failure.
50b80e25 655
5cb3728c
RB
656=item Get_bufsiz
657
658 Size_t (*Get_bufsiz)(pTHX_ PerlIO *f);
50b80e25 659
9d799145 660Return the number of bytes that last C<Fill()> put in the buffer.
50b80e25 661
5cb3728c
RB
662=item Get_ptr
663
664 STDCHAR * (*Get_ptr)(pTHX_ PerlIO *f);
50b80e25 665
3039a93d 666Return the current read pointer relative to this layer's buffer.
50b80e25 667
5cb3728c
RB
668=item Get_cnt
669
670 SSize_t (*Get_cnt)(pTHX_ PerlIO *f);
50b80e25
NIS
671
672Return the number of bytes left to be read in the current buffer.
673
5cb3728c
RB
674=item Set_ptrcnt
675
676 void (*Set_ptrcnt)(pTHX_ PerlIO *f,
677 STDCHAR *ptr, SSize_t cnt);
50b80e25
NIS
678
679Adjust the read pointer and count of bytes to match C<ptr> and/or C<cnt>.
680The application (or layer above) must ensure they are consistent.
681(Checking is allowed by the paranoid.)
682
683=back
684
210e727c
JH
685=head2 Implementing PerlIO Layers
686
2535a4f7
JH
687If you find the implementation document unclear or not sufficient,
688look at the existing perlio layer implementations, which include:
689
690=over
691
692=item * C implementations
693
694PerlIO::encoding, PerlIO::scalar, PerlIO::via in the Perl core.
695
696PerlIO::gzip and APR::PerlIO (mod_perl 2.0) on CPAN.
697
698=item * Perl implementations
699
700PerlIO::via::QuotedPrint in the Perl core and PerlIO::via::* on CPAN.
701
702=back
703
210e727c
JH
704If you are creating a PerlIO layer, you may want to be lazy, in other
705words, implement only the methods that interest you. The other methods
706you can either replace with the "blank" methods
707
708 PerlIOBase_noop_ok
709 PerlIOBase_noop_fail
710
711(which do nothing, and return zero and -1, respectively) or for
712certain methods you may assume a default behaviour by using a NULL
61bdadae
JH
713method. The Open method looks for help in the 'parent' layer.
714The following table summarizes the behaviour:
210e727c
JH
715
716 method behaviour with NULL
717
718 Clearerr PerlIOBase_clearerr
719 Close PerlIOBase_close
61bdadae 720 Dup PerlIOBase_dup
210e727c
JH
721 Eof PerlIOBase_eof
722 Error PerlIOBase_error
723 Fileno PerlIOBase_fileno
724 Fill FAILURE
725 Flush SUCCESS
61bdadae 726 Getarg SUCCESS
210e727c
JH
727 Get_base FAILURE
728 Get_bufsiz FAILURE
729 Get_cnt FAILURE
730 Get_ptr FAILURE
61bdadae
JH
731 Open INHERITED
732 Popped SUCCESS
733 Pushed SUCCESS
210e727c
JH
734 Read PerlIOBase_read
735 Seek FAILURE
736 Set_cnt FAILURE
737 Set_ptrcnt FAILURE
738 Setlinebuf PerlIOBase_setlinebuf
739 Tell FAILURE
740 Unread PerlIOBase_unread
741 Write FAILURE
50b80e25 742
61bdadae
JH
743 FAILURE Set errno (to EINVAL in UNIXish, to LIB$_INVARG in VMS) and
744 return -1 (for numeric return values) or NULL (for pointers)
745 INHERITED Inherited from the layer below
746 SUCCESS Return 0 (for numeric return values) or a pointer
747
50b80e25
NIS
748=head2 Core Layers
749
750The file C<perlio.c> provides the following layers:
751
752=over 4
753
754=item "unix"
755
9d799145
NIS
756A basic non-buffered layer which calls Unix/POSIX C<read()>, C<write()>,
757C<lseek()>, C<close()>. No buffering. Even on platforms that distinguish
758between O_TEXT and O_BINARY this layer is always O_BINARY.
50b80e25
NIS
759
760=item "perlio"
761
9d799145
NIS
762A very complete generic buffering layer which provides the whole of
763PerlIO API. It is also intended to be used as a "base class" for other
1d11c889
JH
764layers. (For example its C<Read()> method is implemented in terms of
765the C<Get_cnt()>/C<Get_ptr()>/C<Set_ptrcnt()> methods).
50b80e25 766
9d799145
NIS
767"perlio" over "unix" provides a complete replacement for stdio as seen
768via PerlIO API. This is the default for USE_PERLIO when system's stdio
1d11c889
JH
769does not permit perl's "fast gets" access, and which do not
770distinguish between C<O_TEXT> and C<O_BINARY>.
50b80e25
NIS
771
772=item "stdio"
773
9d799145
NIS
774A layer which provides the PerlIO API via the layer scheme, but
775implements it by calling system's stdio. This is (currently) the default
776if system's stdio provides sufficient access to allow perl's "fast gets"
777access and which do not distinguish between C<O_TEXT> and C<O_BINARY>.
50b80e25
NIS
778
779=item "crlf"
780
9d799145
NIS
781A layer derived using "perlio" as a base class. It provides Win32-like
782"\n" to CR,LF translation. Can either be applied above "perlio" or serve
783as the buffer layer itself. "crlf" over "unix" is the default if system
784distinguishes between C<O_TEXT> and C<O_BINARY> opens. (At some point
785"unix" will be replaced by a "native" Win32 IO layer on that platform,
786as Win32's read/write layer has various drawbacks.) The "crlf" layer is
787a reasonable model for a layer which transforms data in some way.
50b80e25
NIS
788
789=item "mmap"
790
9d799145
NIS
791If Configure detects C<mmap()> functions this layer is provided (with
792"perlio" as a "base") which does "read" operations by mmap()ing the
793file. Performance improvement is marginal on modern systems, so it is
794mainly there as a proof of concept. It is likely to be unbundled from
795the core at some point. The "mmap" layer is a reasonable model for a
796minimalist "derived" layer.
50b80e25
NIS
797
798=item "pending"
799
9d799145 800An "internal" derivative of "perlio" which can be used to provide
1d11c889
JH
801Unread() function for layers which have no buffer or cannot be
802bothered. (Basically this layer's C<Fill()> pops itself off the stack
803and so resumes reading from layer below.)
50b80e25
NIS
804
805=item "raw"
806
9d799145 807A dummy layer which never exists on the layer stack. Instead when
86e05cf2
NIS
808"pushed" it actually pops the stack removing itself, it then calls
809Binmode function table entry on all the layers in the stack - normally
810this (via PerlIOBase_binmode) removes any layers which do not have
811C<PERLIO_K_RAW> bit set. Layers can modify that behaviour by defining
812their own Binmode entry.
50b80e25
NIS
813
814=item "utf8"
815
9d799145 816Another dummy layer. When pushed it pops itself and sets the
1d11c889
JH
817C<PERLIO_F_UTF8> flag on the layer which was (and now is once more)
818the top of the stack.
50b80e25
NIS
819
820=back
821
9d799145
NIS
822In addition F<perlio.c> also provides a number of C<PerlIOBase_xxxx()>
823functions which are intended to be used in the table slots of classes
824which do not need to do anything special for a particular method.
50b80e25
NIS
825
826=head2 Extension Layers
827
1d11c889
JH
828Layers can made available by extension modules. When an unknown layer
829is encountered the PerlIO code will perform the equivalent of :
b76cc8ba
NIS
830
831 use PerlIO 'layer';
832
1d11c889 833Where I<layer> is the unknown layer. F<PerlIO.pm> will then attempt to:
b76cc8ba
NIS
834
835 require PerlIO::layer;
836
1d11c889
JH
837If after that process the layer is still not defined then the C<open>
838will fail.
b76cc8ba
NIS
839
840The following extension layers are bundled with perl:
50b80e25
NIS
841
842=over 4
843
b76cc8ba 844=item ":encoding"
50b80e25
NIS
845
846 use Encoding;
847
1d11c889
JH
848makes this layer available, although F<PerlIO.pm> "knows" where to
849find it. It is an example of a layer which takes an argument as it is
850called thus:
50b80e25 851
b31b80f9 852 open( $fh, "<:encoding(iso-8859-7)", $pathname );
50b80e25 853
385e1f9f 854=item ":scalar"
b76cc8ba 855
b31b80f9 856Provides support for reading data from and writing data to a scalar.
b76cc8ba 857
385e1f9f 858 open( $fh, "+<:scalar", \$scalar );
50b80e25 859
1d11c889
JH
860When a handle is so opened, then reads get bytes from the string value
861of I<$scalar>, and writes change the value. In both cases the position
862in I<$scalar> starts as zero but can be altered via C<seek>, and
863determined via C<tell>.
b76cc8ba 864
385e1f9f
EM
865Please note that this layer is implied when calling open() thus:
866
867 open( $fh, "+<", \$scalar );
868
869=item ":via"
b76cc8ba 870
4f7853f4
GA
871Provided to allow layers to be implemented as Perl code. For instance:
872
e934609f 873 use PerlIO::via::StripHTML;
385e1f9f 874 open( my $fh, "<:via(StripHTML)", "index.html" );
4f7853f4 875
e934609f 876See L<PerlIO::via> for details.
b76cc8ba
NIS
877
878=back
50b80e25 879
d4165bde
SB
880=head1 TODO
881
882Things that need to be done to improve this document.
883
884=over
885
886=item *
887
888Explain how to make a valid fh without going through open()(i.e. apply
889a layer). For example if the file is not opened through perl, but we
890want to get back a fh, like it was opened by Perl.
891
892How PerlIO_apply_layera fits in, where its docs, was it made public?
893
894Currently the example could be something like this:
895
896 PerlIO *foo_to_PerlIO(pTHX_ char *mode, ...)
897 {
898 char *mode; /* "w", "r", etc */
899 const char *layers = ":APR"; /* the layer name */
900 PerlIO *f = PerlIO_allocate(aTHX);
901 if (!f) {
902 return NULL;
903 }
904
905 PerlIO_apply_layers(aTHX_ f, mode, layers);
906
907 if (f) {
908 PerlIOAPR *st = PerlIOSelf(f, PerlIOAPR);
909 /* fill in the st struct, as in _open() */
910 st->file = file;
911 PerlIOBase(f)->flags |= PERLIO_F_OPEN;
912
913 return f;
914 }
915 return NULL;
916 }
917
918=item *
919
920fix/add the documentation in places marked as XXX.
921
922=item *
923
924The handling of errors by the layer is not specified. e.g. when $!
925should be set explicitly, when the error handling should be just
926delegated to the top layer.
927
928Probably give some hints on using SETERRNO() or pointers to where they
929can be found.
930
931=item *
932
933I think it would help to give some concrete examples to make it easier
934to understand the API. Of course I agree that the API has to be
935concise, but since there is no second document that is more of a
936guide, I think that it'd make it easier to start with the doc which is
937an API, but has examples in it in places where things are unclear, to
938a person who is not a PerlIO guru (yet).
939
940=back
941
50b80e25 942=cut