This is a live mirror of the Perl 5 development currently hosted at https://github.com/perl/perl5
UseNicerNames instead of SHOUTING_LIKE_THIS.
[perl5.git] / pod / perliol.pod
CommitLineData
50b80e25
NIS
1=head1 NAME
2
3perliol - C API for Perl's implementation of IO in Layers.
4
5=head1 SYNOPSIS
6
7 /* Defining a layer ... */
8 #include <perliol.h>
9
50b80e25
NIS
10=head1 DESCRIPTION
11
9d799145
NIS
12This document describes the behavior and implementation of the PerlIO
13abstraction described in L<perlapio> when C<USE_PERLIO> is defined (and
14C<USE_SFIO> is not).
50b80e25
NIS
15
16=head2 History and Background
17
9d799145
NIS
18The PerlIO abstraction was introduced in perl5.003_02 but languished as
19just an abstraction until perl5.7.0. However during that time a number
d1be9408 20of perl extensions switched to using it, so the API is mostly fixed to
9d799145 21maintain (source) compatibility.
50b80e25 22
9d799145
NIS
23The aim of the implementation is to provide the PerlIO API in a flexible
24and platform neutral manner. It is also a trial of an "Object Oriented
25C, with vtables" approach which may be applied to perl6.
50b80e25
NIS
26
27=head2 Layers vs Disciplines
28
9d799145
NIS
29Initial discussion of the ability to modify IO streams behaviour used
30the term "discipline" for the entities which were added. This came (I
31believe) from the use of the term in "sfio", which in turn borrowed it
32from "line disciplines" on Unix terminals. However, this document (and
33the C code) uses the term "layer".
34
1d11c889
JH
35This is, I hope, a natural term given the implementation, and should
36avoid connotations that are inherent in earlier uses of "discipline"
37for things which are rather different.
50b80e25
NIS
38
39=head2 Data Structures
40
41The basic data structure is a PerlIOl:
42
43 typedef struct _PerlIO PerlIOl;
44 typedef struct _PerlIO_funcs PerlIO_funcs;
45 typedef PerlIOl *PerlIO;
46
47 struct _PerlIO
48 {
49 PerlIOl * next; /* Lower layer */
50 PerlIO_funcs * tab; /* Functions for this layer */
51 IV flags; /* Various flags for state */
52 };
53
1d11c889
JH
54A C<PerlIOl *> is a pointer to the struct, and the I<application>
55level C<PerlIO *> is a pointer to a C<PerlIOl *> - i.e. a pointer
56to a pointer to the struct. This allows the application level C<PerlIO *>
57to remain constant while the actual C<PerlIOl *> underneath
58changes. (Compare perl's C<SV *> which remains constant while its
59C<sv_any> field changes as the scalar's type changes.) An IO stream is
60then in general represented as a pointer to this linked-list of
61"layers".
50b80e25 62
9d799145 63It should be noted that because of the double indirection in a C<PerlIO *>,
d4165bde 64a C<< &(perlio->next) >> "is" a C<PerlIO *>, and so to some degree
11e1c8f2 65at least one layer can use the "standard" API on the next layer down.
50b80e25
NIS
66
67A "layer" is composed of two parts:
68
69=over 4
70
210b36aa 71=item 1.
50b80e25 72
210b36aa
AMS
73The functions and attributes of the "layer class".
74
75=item 2.
76
77The per-instance data for a particular handle.
50b80e25
NIS
78
79=back
80
81=head2 Functions and Attributes
82
9d799145
NIS
83The functions and attributes are accessed via the "tab" (for table)
84member of C<PerlIOl>. The functions (methods of the layer "class") are
85fixed, and are defined by the C<PerlIO_funcs> type. They are broadly the
86same as the public C<PerlIO_xxxxx> functions:
50b80e25 87
b76cc8ba
NIS
88 struct _PerlIO_funcs
89 {
2dc2558e 90 Size_t fsize;
b76cc8ba
NIS
91 char * name;
92 Size_t size;
93 IV kind;
2dc2558e 94 IV (*Pushed)(pTHX_ PerlIO *f,const char *mode,SV *arg, PerlIO_funcs *tab);
d4165bde 95 IV (*Popped)(pTHX_ PerlIO *f);
b76cc8ba
NIS
96 PerlIO * (*Open)(pTHX_ PerlIO_funcs *tab,
97 AV *layers, IV n,
98 const char *mode,
99 int fd, int imode, int perm,
100 PerlIO *old,
101 int narg, SV **args);
86e05cf2 102 IV (*Binmode)(pTHX_ PerlIO *f);
d4165bde
SB
103 SV * (*Getarg)(pTHX_ PerlIO *f, CLONE_PARAMS *param, int flags)
104 IV (*Fileno)(pTHX_ PerlIO *f);
105 PerlIO * (*Dup)(pTHX_ PerlIO *f, PerlIO *o, CLONE_PARAMS *param, int flags)
b76cc8ba 106 /* Unix-like functions - cf sfio line disciplines */
d4165bde
SB
107 SSize_t (*Read)(pTHX_ PerlIO *f, void *vbuf, Size_t count);
108 SSize_t (*Unread)(pTHX_ PerlIO *f, const void *vbuf, Size_t count);
109 SSize_t (*Write)(pTHX_ PerlIO *f, const void *vbuf, Size_t count);
110 IV (*Seek)(pTHX_ PerlIO *f, Off_t offset, int whence);
111 Off_t (*Tell)(pTHX_ PerlIO *f);
112 IV (*Close)(pTHX_ PerlIO *f);
b76cc8ba 113 /* Stdio-like buffered IO functions */
d4165bde
SB
114 IV (*Flush)(pTHX_ PerlIO *f);
115 IV (*Fill)(pTHX_ PerlIO *f);
116 IV (*Eof)(pTHX_ PerlIO *f);
117 IV (*Error)(pTHX_ PerlIO *f);
118 void (*Clearerr)(pTHX_ PerlIO *f);
119 void (*Setlinebuf)(pTHX_ PerlIO *f);
b76cc8ba 120 /* Perl's snooping functions */
d4165bde
SB
121 STDCHAR * (*Get_base)(pTHX_ PerlIO *f);
122 Size_t (*Get_bufsiz)(pTHX_ PerlIO *f);
123 STDCHAR * (*Get_ptr)(pTHX_ PerlIO *f);
124 SSize_t (*Get_cnt)(pTHX_ PerlIO *f);
125 void (*Set_ptrcnt)(pTHX_ PerlIO *f,STDCHAR *ptr,SSize_t cnt);
b76cc8ba
NIS
126 };
127
2dc2558e
NIS
128The first few members of the struct give a function table size for
129compatibility check "name" for the layer, the size to C<malloc> for the per-instance data,
130and some flags which are attributes of the class as whole (such as whether it is a buffering
9d799145 131layer), then follow the functions which fall into four basic groups:
50b80e25
NIS
132
133=over 4
134
aa500c9e 135=item 1.
50b80e25 136
aa500c9e 137Opening and setup functions
50b80e25 138
aa500c9e 139=item 2.
50b80e25 140
aa500c9e
RB
141Basic IO operations
142
143=item 3.
144
145Stdio class buffering options.
146
147=item 4.
148
149Functions to support Perl's traditional "fast" access to the buffer.
50b80e25
NIS
150
151=back
152
1d11c889
JH
153A layer does not have to implement all the functions, but the whole
154table has to be present. Unimplemented slots can be NULL (which will
155result in an error when called) or can be filled in with stubs to
156"inherit" behaviour from a "base class". This "inheritance" is fixed
157for all instances of the layer, but as the layer chooses which stubs
158to populate the table, limited "multiple inheritance" is possible.
50b80e25
NIS
159
160=head2 Per-instance Data
161
1d11c889
JH
162The per-instance data are held in memory beyond the basic PerlIOl
163struct, by making a PerlIOl the first member of the layer's struct
164thus:
50b80e25
NIS
165
166 typedef struct
167 {
168 struct _PerlIO base; /* Base "class" info */
169 STDCHAR * buf; /* Start of buffer */
170 STDCHAR * end; /* End of valid part of buffer */
171 STDCHAR * ptr; /* Current position in buffer */
172 Off_t posn; /* Offset of buf into the file */
173 Size_t bufsiz; /* Real size of buffer */
174 IV oneword; /* Emergency buffer */
175 } PerlIOBuf;
176
1d11c889
JH
177In this way (as for perl's scalars) a pointer to a PerlIOBuf can be
178treated as a pointer to a PerlIOl.
50b80e25
NIS
179
180=head2 Layers in action.
181
182 table perlio unix
183 | |
184 +-----------+ +----------+ +--------+
185 PerlIO ->| |--->| next |--->| NULL |
186 +-----------+ +----------+ +--------+
187 | | | buffer | | fd |
188 +-----------+ | | +--------+
189 | | +----------+
190
191
192The above attempts to show how the layer scheme works in a simple case.
9d799145
NIS
193The application's C<PerlIO *> points to an entry in the table(s)
194representing open (allocated) handles. For example the first three slots
195in the table correspond to C<stdin>,C<stdout> and C<stderr>. The table
196in turn points to the current "top" layer for the handle - in this case
197an instance of the generic buffering layer "perlio". That layer in turn
198points to the next layer down - in this case the lowlevel "unix" layer.
50b80e25 199
9d799145
NIS
200The above is roughly equivalent to a "stdio" buffered stream, but with
201much more flexibility:
50b80e25
NIS
202
203=over 4
204
205=item *
206
9d799145
NIS
207If Unix level C<read>/C<write>/C<lseek> is not appropriate for (say)
208sockets then the "unix" layer can be replaced (at open time or even
209dynamically) with a "socket" layer.
50b80e25
NIS
210
211=item *
212
1d11c889
JH
213Different handles can have different buffering schemes. The "top"
214layer could be the "mmap" layer if reading disk files was quicker
215using C<mmap> than C<read>. An "unbuffered" stream can be implemented
216simply by not having a buffer layer.
50b80e25
NIS
217
218=item *
219
220Extra layers can be inserted to process the data as it flows through.
9d799145 221This was the driving need for including the scheme in perl 5.7.0+ - we
d1be9408 222needed a mechanism to allow data to be translated between perl's
9d799145
NIS
223internal encoding (conceptually at least Unicode as UTF-8), and the
224"native" format used by the system. This is provided by the
225":encoding(xxxx)" layer which typically sits above the buffering layer.
50b80e25
NIS
226
227=item *
228
1d11c889
JH
229A layer can be added that does "\n" to CRLF translation. This layer
230can be used on any platform, not just those that normally do such
231things.
50b80e25
NIS
232
233=back
234
235=head2 Per-instance flag bits
236
1d11c889
JH
237The generic flag bits are a hybrid of C<O_XXXXX> style flags deduced
238from the mode string passed to C<PerlIO_open()>, and state bits for
239typical buffer layers.
50b80e25 240
9d799145 241=over 4
50b80e25
NIS
242
243=item PERLIO_F_EOF
244
245End of file.
246
247=item PERLIO_F_CANWRITE
248
3039a93d 249Writes are permitted, i.e. opened as "w" or "r+" or "a", etc.
50b80e25
NIS
250
251=item PERLIO_F_CANREAD
252
3039a93d 253Reads are permitted i.e. opened "r" or "w+" (or even "a+" - ick).
50b80e25
NIS
254
255=item PERLIO_F_ERROR
256
d4165bde 257An error has occurred (for C<PerlIO_error()>).
50b80e25
NIS
258
259=item PERLIO_F_TRUNCATE
260
261Truncate file suggested by open mode.
262
263=item PERLIO_F_APPEND
264
265All writes should be appends.
266
267=item PERLIO_F_CRLF
268
11e1c8f2
PP
269Layer is performing Win32-like "\n" mapped to CR,LF for output and CR,LF
270mapped to "\n" for input. Normally the provided "crlf" layer is the only
271layer that need bother about this. C<PerlIO_binmode()> will mess with this
9d799145
NIS
272flag rather than add/remove layers if the C<PERLIO_K_CANCRLF> bit is set
273for the layers class.
50b80e25
NIS
274
275=item PERLIO_F_UTF8
276
3039a93d 277Data written to this layer should be UTF-8 encoded; data provided
50b80e25
NIS
278by this layer should be considered UTF-8 encoded. Can be set on any layer
279by ":utf8" dummy layer. Also set on ":encoding" layer.
280
281=item PERLIO_F_UNBUF
282
283Layer is unbuffered - i.e. write to next layer down should occur for
284each write to this layer.
285
286=item PERLIO_F_WRBUF
287
288The buffer for this layer currently holds data written to it but not sent
289to next layer.
290
291=item PERLIO_F_RDBUF
292
293The buffer for this layer currently holds unconsumed data read from
294layer below.
295
296=item PERLIO_F_LINEBUF
297
9d799145
NIS
298Layer is line buffered. Write data should be passed to next layer down
299whenever a "\n" is seen. Any data beyond the "\n" should then be
300processed.
50b80e25
NIS
301
302=item PERLIO_F_TEMP
303
9d799145 304File has been C<unlink()>ed, or should be deleted on C<close()>.
50b80e25
NIS
305
306=item PERLIO_F_OPEN
307
308Handle is open.
309
310=item PERLIO_F_FASTGETS
311
9d799145
NIS
312This instance of this layer supports the "fast C<gets>" interface.
313Normally set based on C<PERLIO_K_FASTGETS> for the class and by the
d1be9408 314existence of the function(s) in the table. However a class that
50b80e25
NIS
315normally provides that interface may need to avoid it on a
316particular instance. The "pending" layer needs to do this when
d1be9408 317it is pushed above a layer which does not support the interface.
9d799145 318(Perl's C<sv_gets()> does not expect the streams fast C<gets> behaviour
50b80e25
NIS
319to change during one "get".)
320
321=back
322
323=head2 Methods in Detail
324
325=over 4
326
e2d9456f 327=item fsize
2dc2558e
NIS
328
329 Size_t fsize;
330
a489db4d
JH
331Size of the function table. This is compared against the value PerlIO
332code "knows" as a compatibility check. Future versions I<may> be able
333to tolerate layers compiled against an old version of the headers.
2dc2558e 334
5cb3728c
RB
335=item name
336
337 char * name;
d4165bde
SB
338
339The name of the layer whose open() method Perl should invoke on
340open(). For example if the layer is called APR, you will call:
341
342 open $fh, ">:APR", ...
343
344and Perl knows that it has to invoke the PerlIOAPR_open() method
345implemented by the APR layer.
346
5cb3728c
RB
347=item size
348
349 Size_t size;
d4165bde
SB
350
351The size of the per-instance data structure, e.g.:
352
353 sizeof(PerlIOAPR)
354
a489db4d
JH
355If this field is zero then C<PerlIO_pushed> does not malloc anything
356and assumes layer's Pushed function will do any required layer stack
357manipulation - used to avoid malloc/free overhead for dummy layers.
2dc2558e
NIS
358If the field is non-zero it must be at least the size of C<PerlIOl>,
359C<PerlIO_pushed> will allocate memory for the layer's data structures
360and link new layer onto the stream's stack. (If the layer's Pushed
361method returns an error indication the layer is popped again.)
362
5cb3728c
RB
363=item kind
364
365 IV kind;
d4165bde 366
d4165bde
SB
367=over 4
368
369=item * PERLIO_K_BUFFERED
370
86e05cf2
NIS
371The layer is buffered.
372
373=item * PERLIO_K_RAW
374
375The layer is acceptable to have in a binmode(FH) stack - i.e. it does not
376(or will configure itself not to) transform bytes passing through it.
377
d4165bde
SB
378=item * PERLIO_K_CANCRLF
379
86e05cf2
NIS
380Layer can translate between "\n" and CRLF line ends.
381
d4165bde
SB
382=item * PERLIO_K_FASTGETS
383
86e05cf2
NIS
384Layer allows buffer snooping.
385
d4165bde
SB
386=item * PERLIO_K_MULTIARG
387
388Used when the layer's open() accepts more arguments than usual. The
389extra arguments should come not before the C<MODE> argument. When this
390flag is used it's up to the layer to validate the args.
391
d4165bde
SB
392=back
393
5cb3728c
RB
394=item Pushed
395
396 IV (*Pushed)(pTHX_ PerlIO *f,const char *mode, SV *arg);
50b80e25 397
1d11c889
JH
398The only absolutely mandatory method. Called when the layer is pushed
399onto the stack. The C<mode> argument may be NULL if this occurs
400post-open. The C<arg> will be non-C<NULL> if an argument string was
401passed. In most cases this should call C<PerlIOBase_pushed()> to
402convert C<mode> into the appropriate C<PERLIO_F_XXXXX> flags in
403addition to any actions the layer itself takes. If a layer is not
404expecting an argument it need neither save the one passed to it, nor
405provide C<Getarg()> (it could perhaps C<Perl_warn> that the argument
406was un-expected).
50b80e25 407
d4165bde
SB
408Returns 0 on success. On failure returns -1 and should set errno.
409
5cb3728c
RB
410=item Popped
411
412 IV (*Popped)(pTHX_ PerlIO *f);
50b80e25 413
1d11c889
JH
414Called when the layer is popped from the stack. A layer will normally
415be popped after C<Close()> is called. But a layer can be popped
416without being closed if the program is dynamically managing layers on
417the stream. In such cases C<Popped()> should free any resources
418(buffers, translation tables, ...) not held directly in the layer's
419struct. It should also C<Unread()> any unconsumed data that has been
420read and buffered from the layer below back to that layer, so that it
421can be re-provided to what ever is now above.
b76cc8ba 422
d4165bde
SB
423Returns 0 on success and failure.
424
5cb3728c
RB
425=item Open
426
427 PerlIO * (*Open)(...);
b76cc8ba 428
1d11c889
JH
429The C<Open()> method has lots of arguments because it combines the
430functions of perl's C<open>, C<PerlIO_open>, perl's C<sysopen>,
431C<PerlIO_fdopen> and C<PerlIO_reopen>. The full prototype is as
432follows:
b76cc8ba
NIS
433
434 PerlIO * (*Open)(pTHX_ PerlIO_funcs *tab,
435 AV *layers, IV n,
436 const char *mode,
437 int fd, int imode, int perm,
438 PerlIO *old,
439 int narg, SV **args);
440
1d11c889
JH
441Open should (perhaps indirectly) call C<PerlIO_allocate()> to allocate
442a slot in the table and associate it with the layers information for
443the opened file, by calling C<PerlIO_push>. The I<layers> AV is an
444array of all the layers destined for the C<PerlIO *>, and any
445arguments passed to them, I<n> is the index into that array of the
446layer being called. The macro C<PerlIOArg> will return a (possibly
447C<NULL>) SV * for the argument passed to the layer.
448
449The I<mode> string is an "C<fopen()>-like" string which would match
450the regular expression C</^[I#]?[rwa]\+?[bt]?$/>.
451
452The C<'I'> prefix is used during creation of C<stdin>..C<stderr> via
453special C<PerlIO_fdopen> calls; the C<'#'> prefix means that this is
454C<sysopen> and that I<imode> and I<perm> should be passed to
455C<PerlLIO_open3>; C<'r'> means B<r>ead, C<'w'> means B<w>rite and
456C<'a'> means B<a>ppend. The C<'+'> suffix means that both reading and
a489db4d
JH
457writing/appending are permitted. The C<'b'> suffix means file should
458be binary, and C<'t'> means it is text. (Almost all layers should do
459the IO in binary mode, and ignore the b/t bits. The C<:crlf> layer
460should be pushed to handle the distinction.)
1d11c889
JH
461
462If I<old> is not C<NULL> then this is a C<PerlIO_reopen>. Perl itself
463does not use this (yet?) and semantics are a little vague.
464
465If I<fd> not negative then it is the numeric file descriptor I<fd>,
466which will be open in a manner compatible with the supplied mode
467string, the call is thus equivalent to C<PerlIO_fdopen>. In this case
468I<nargs> will be zero.
469
470If I<nargs> is greater than zero then it gives the number of arguments
471passed to C<open>, otherwise it will be 1 if for example
472C<PerlIO_open> was called. In simple cases SvPV_nolen(*args) is the
473pathname to open.
474
475Having said all that translation-only layers do not need to provide
476C<Open()> at all, but rather leave the opening to a lower level layer
477and wait to be "pushed". If a layer does provide C<Open()> it should
478normally call the C<Open()> method of next layer down (if any) and
479then push itself on top if that succeeds.
b76cc8ba 480
d4165bde
SB
481Returns C<NULL> on failure.
482
86e05cf2
NIS
483=item Binmode
484
485 IV (*Binmode)(pTHX_ PerlIO *f);
486
487Optional. Used when C<:raw> layer is pushed (explicitly or as a result
488of binmode(FH)). If not present layer will be popped. If present
489should configure layer as binary (or pop itself) and return 0.
490If it returns -1 for error C<binmode> will fail with layer
491still on the stack.
492
5cb3728c
RB
493=item Getarg
494
495 SV * (*Getarg)(pTHX_ PerlIO *f,
496 CLONE_PARAMS *param, int flags);
b76cc8ba 497
d4165bde
SB
498Optional. If present should return an SV * representing the string
499argument passed to the layer when it was
500pushed. e.g. ":encoding(ascii)" would return an SvPV with value
501"ascii". (I<param> and I<flags> arguments can be ignored in most
502cases)
b76cc8ba 503
5cb3728c
RB
504=item Fileno
505
506 IV (*Fileno)(pTHX_ PerlIO *f);
b76cc8ba 507
d1be9408 508Returns the Unix/Posix numeric file descriptor for the handle. Normally
b76cc8ba
NIS
509C<PerlIOBase_fileno()> (which just asks next layer down) will suffice
510for this.
50b80e25 511
a489db4d
JH
512Returns -1 on error, which is considered to include the case where the
513layer cannot provide such a file descriptor.
d4165bde 514
5cb3728c
RB
515=item Dup
516
517 PerlIO * (*Dup)(pTHX_ PerlIO *f, PerlIO *o,
518 CLONE_PARAMS *param, int flags);
d4165bde 519
2dc2558e
NIS
520XXX: Needs more docs.
521
a489db4d
JH
522Used as part of the "clone" process when a thread is spawned (in which
523case param will be non-NULL) and when a stream is being duplicated via
524'&' in the C<open>.
d4165bde
SB
525
526Similar to C<Open>, returns PerlIO* on success, C<NULL> on failure.
527
5cb3728c
RB
528=item Read
529
530 SSize_t (*Read)(pTHX_ PerlIO *f, void *vbuf, Size_t count);
d4165bde
SB
531
532Basic read operation.
50b80e25 533
d4165bde
SB
534Typically will call C<Fill> and manipulate pointers (possibly via the
535API). C<PerlIOBuf_read()> may be suitable for derived classes which
536provide "fast gets" methods.
50b80e25 537
d4165bde
SB
538Returns actual bytes read, or -1 on an error.
539
5cb3728c
RB
540=item Unread
541
542 SSize_t (*Unread)(pTHX_ PerlIO *f,
543 const void *vbuf, Size_t count);
50b80e25 544
9d799145
NIS
545A superset of stdio's C<ungetc()>. Should arrange for future reads to
546see the bytes in C<vbuf>. If there is no obviously better implementation
547then C<PerlIOBase_unread()> provides the function by pushing a "fake"
548"pending" layer above the calling layer.
50b80e25 549
d4165bde
SB
550Returns the number of unread chars.
551
5cb3728c
RB
552=item Write
553
554 SSize_t (*Write)(PerlIO *f, const void *vbuf, Size_t count);
50b80e25 555
d4165bde 556Basic write operation.
50b80e25 557
d4165bde
SB
558Returns bytes written or -1 on an error.
559
5cb3728c
RB
560=item Seek
561
562 IV (*Seek)(pTHX_ PerlIO *f, Off_t offset, int whence);
50b80e25 563
1d11c889
JH
564Position the file pointer. Should normally call its own C<Flush>
565method and then the C<Seek> method of next layer down.
50b80e25 566
d4165bde
SB
567Returns 0 on success, -1 on failure.
568
5cb3728c
RB
569=item Tell
570
571 Off_t (*Tell)(pTHX_ PerlIO *f);
50b80e25 572
9d799145
NIS
573Return the file pointer. May be based on layers cached concept of
574position to avoid overhead.
50b80e25 575
d4165bde
SB
576Returns -1 on failure to get the file pointer.
577
5cb3728c
RB
578=item Close
579
580 IV (*Close)(pTHX_ PerlIO *f);
50b80e25 581
9d799145
NIS
582Close the stream. Should normally call C<PerlIOBase_close()> to flush
583itself and close layers below, and then deallocate any data structures
584(buffers, translation tables, ...) not held directly in the data
585structure.
50b80e25 586
d4165bde
SB
587Returns 0 on success, -1 on failure.
588
5cb3728c
RB
589=item Flush
590
591 IV (*Flush)(pTHX_ PerlIO *f);
50b80e25 592
9d799145
NIS
593Should make stream's state consistent with layers below. That is, any
594buffered write data should be written, and file position of lower layers
d1be9408 595adjusted for data read from below but not actually consumed.
b76cc8ba 596(Should perhaps C<Unread()> such data to the lower layer.)
50b80e25 597
d4165bde
SB
598Returns 0 on success, -1 on failure.
599
5cb3728c
RB
600=item Fill
601
602 IV (*Fill)(pTHX_ PerlIO *f);
d4165bde
SB
603
604The buffer for this layer should be filled (for read) from layer
605below. When you "subclass" PerlIOBuf layer, you want to use its
606I<_read> method and to supply your own fill method, which fills the
607PerlIOBuf's buffer.
50b80e25 608
d4165bde 609Returns 0 on success, -1 on failure.
50b80e25 610
5cb3728c
RB
611=item Eof
612
613 IV (*Eof)(pTHX_ PerlIO *f);
50b80e25 614
9d799145 615Return end-of-file indicator. C<PerlIOBase_eof()> is normally sufficient.
50b80e25 616
d4165bde
SB
617Returns 0 on end-of-file, 1 if not end-of-file, -1 on error.
618
5cb3728c
RB
619=item Error
620
621 IV (*Error)(pTHX_ PerlIO *f);
50b80e25 622
9d799145 623Return error indicator. C<PerlIOBase_error()> is normally sufficient.
50b80e25 624
d4165bde
SB
625Returns 1 if there is an error (usually when C<PERLIO_F_ERROR> is set,
6260 otherwise.
627
5cb3728c
RB
628=item Clearerr
629
630 void (*Clearerr)(pTHX_ PerlIO *f);
50b80e25 631
9d799145
NIS
632Clear end-of-file and error indicators. Should call C<PerlIOBase_clearerr()>
633to set the C<PERLIO_F_XXXXX> flags, which may suffice.
50b80e25 634
5cb3728c
RB
635=item Setlinebuf
636
637 void (*Setlinebuf)(pTHX_ PerlIO *f);
50b80e25 638
b76cc8ba
NIS
639Mark the stream as line buffered. C<PerlIOBase_setlinebuf()> sets the
640PERLIO_F_LINEBUF flag and is normally sufficient.
50b80e25 641
5cb3728c
RB
642=item Get_base
643
644 STDCHAR * (*Get_base)(pTHX_ PerlIO *f);
50b80e25
NIS
645
646Allocate (if not already done so) the read buffer for this layer and
d4165bde 647return pointer to it. Return NULL on failure.
50b80e25 648
5cb3728c
RB
649=item Get_bufsiz
650
651 Size_t (*Get_bufsiz)(pTHX_ PerlIO *f);
50b80e25 652
9d799145 653Return the number of bytes that last C<Fill()> put in the buffer.
50b80e25 654
5cb3728c
RB
655=item Get_ptr
656
657 STDCHAR * (*Get_ptr)(pTHX_ PerlIO *f);
50b80e25 658
3039a93d 659Return the current read pointer relative to this layer's buffer.
50b80e25 660
5cb3728c
RB
661=item Get_cnt
662
663 SSize_t (*Get_cnt)(pTHX_ PerlIO *f);
50b80e25
NIS
664
665Return the number of bytes left to be read in the current buffer.
666
5cb3728c
RB
667=item Set_ptrcnt
668
669 void (*Set_ptrcnt)(pTHX_ PerlIO *f,
670 STDCHAR *ptr, SSize_t cnt);
50b80e25
NIS
671
672Adjust the read pointer and count of bytes to match C<ptr> and/or C<cnt>.
673The application (or layer above) must ensure they are consistent.
674(Checking is allowed by the paranoid.)
675
676=back
677
678
679=head2 Core Layers
680
681The file C<perlio.c> provides the following layers:
682
683=over 4
684
685=item "unix"
686
9d799145
NIS
687A basic non-buffered layer which calls Unix/POSIX C<read()>, C<write()>,
688C<lseek()>, C<close()>. No buffering. Even on platforms that distinguish
689between O_TEXT and O_BINARY this layer is always O_BINARY.
50b80e25
NIS
690
691=item "perlio"
692
9d799145
NIS
693A very complete generic buffering layer which provides the whole of
694PerlIO API. It is also intended to be used as a "base class" for other
1d11c889
JH
695layers. (For example its C<Read()> method is implemented in terms of
696the C<Get_cnt()>/C<Get_ptr()>/C<Set_ptrcnt()> methods).
50b80e25 697
9d799145
NIS
698"perlio" over "unix" provides a complete replacement for stdio as seen
699via PerlIO API. This is the default for USE_PERLIO when system's stdio
1d11c889
JH
700does not permit perl's "fast gets" access, and which do not
701distinguish between C<O_TEXT> and C<O_BINARY>.
50b80e25
NIS
702
703=item "stdio"
704
9d799145
NIS
705A layer which provides the PerlIO API via the layer scheme, but
706implements it by calling system's stdio. This is (currently) the default
707if system's stdio provides sufficient access to allow perl's "fast gets"
708access and which do not distinguish between C<O_TEXT> and C<O_BINARY>.
50b80e25
NIS
709
710=item "crlf"
711
9d799145
NIS
712A layer derived using "perlio" as a base class. It provides Win32-like
713"\n" to CR,LF translation. Can either be applied above "perlio" or serve
714as the buffer layer itself. "crlf" over "unix" is the default if system
715distinguishes between C<O_TEXT> and C<O_BINARY> opens. (At some point
716"unix" will be replaced by a "native" Win32 IO layer on that platform,
717as Win32's read/write layer has various drawbacks.) The "crlf" layer is
718a reasonable model for a layer which transforms data in some way.
50b80e25
NIS
719
720=item "mmap"
721
9d799145
NIS
722If Configure detects C<mmap()> functions this layer is provided (with
723"perlio" as a "base") which does "read" operations by mmap()ing the
724file. Performance improvement is marginal on modern systems, so it is
725mainly there as a proof of concept. It is likely to be unbundled from
726the core at some point. The "mmap" layer is a reasonable model for a
727minimalist "derived" layer.
50b80e25
NIS
728
729=item "pending"
730
9d799145 731An "internal" derivative of "perlio" which can be used to provide
1d11c889
JH
732Unread() function for layers which have no buffer or cannot be
733bothered. (Basically this layer's C<Fill()> pops itself off the stack
734and so resumes reading from layer below.)
50b80e25
NIS
735
736=item "raw"
737
9d799145 738A dummy layer which never exists on the layer stack. Instead when
86e05cf2
NIS
739"pushed" it actually pops the stack removing itself, it then calls
740Binmode function table entry on all the layers in the stack - normally
741this (via PerlIOBase_binmode) removes any layers which do not have
742C<PERLIO_K_RAW> bit set. Layers can modify that behaviour by defining
743their own Binmode entry.
50b80e25
NIS
744
745=item "utf8"
746
9d799145 747Another dummy layer. When pushed it pops itself and sets the
1d11c889
JH
748C<PERLIO_F_UTF8> flag on the layer which was (and now is once more)
749the top of the stack.
50b80e25
NIS
750
751=back
752
9d799145
NIS
753In addition F<perlio.c> also provides a number of C<PerlIOBase_xxxx()>
754functions which are intended to be used in the table slots of classes
755which do not need to do anything special for a particular method.
50b80e25
NIS
756
757=head2 Extension Layers
758
1d11c889
JH
759Layers can made available by extension modules. When an unknown layer
760is encountered the PerlIO code will perform the equivalent of :
b76cc8ba
NIS
761
762 use PerlIO 'layer';
763
1d11c889 764Where I<layer> is the unknown layer. F<PerlIO.pm> will then attempt to:
b76cc8ba
NIS
765
766 require PerlIO::layer;
767
1d11c889
JH
768If after that process the layer is still not defined then the C<open>
769will fail.
b76cc8ba
NIS
770
771The following extension layers are bundled with perl:
50b80e25
NIS
772
773=over 4
774
b76cc8ba 775=item ":encoding"
50b80e25
NIS
776
777 use Encoding;
778
1d11c889
JH
779makes this layer available, although F<PerlIO.pm> "knows" where to
780find it. It is an example of a layer which takes an argument as it is
781called thus:
50b80e25 782
b31b80f9 783 open( $fh, "<:encoding(iso-8859-7)", $pathname );
50b80e25 784
385e1f9f 785=item ":scalar"
b76cc8ba 786
b31b80f9 787Provides support for reading data from and writing data to a scalar.
b76cc8ba 788
385e1f9f 789 open( $fh, "+<:scalar", \$scalar );
50b80e25 790
1d11c889
JH
791When a handle is so opened, then reads get bytes from the string value
792of I<$scalar>, and writes change the value. In both cases the position
793in I<$scalar> starts as zero but can be altered via C<seek>, and
794determined via C<tell>.
b76cc8ba 795
385e1f9f
EM
796Please note that this layer is implied when calling open() thus:
797
798 open( $fh, "+<", \$scalar );
799
800=item ":via"
b76cc8ba 801
4f7853f4
GA
802Provided to allow layers to be implemented as Perl code. For instance:
803
e934609f 804 use PerlIO::via::StripHTML;
385e1f9f 805 open( my $fh, "<:via(StripHTML)", "index.html" );
4f7853f4 806
e934609f 807See L<PerlIO::via> for details.
b76cc8ba
NIS
808
809=back
50b80e25 810
d4165bde
SB
811=head1 TODO
812
813Things that need to be done to improve this document.
814
815=over
816
817=item *
818
819Explain how to make a valid fh without going through open()(i.e. apply
820a layer). For example if the file is not opened through perl, but we
821want to get back a fh, like it was opened by Perl.
822
823How PerlIO_apply_layera fits in, where its docs, was it made public?
824
825Currently the example could be something like this:
826
827 PerlIO *foo_to_PerlIO(pTHX_ char *mode, ...)
828 {
829 char *mode; /* "w", "r", etc */
830 const char *layers = ":APR"; /* the layer name */
831 PerlIO *f = PerlIO_allocate(aTHX);
832 if (!f) {
833 return NULL;
834 }
835
836 PerlIO_apply_layers(aTHX_ f, mode, layers);
837
838 if (f) {
839 PerlIOAPR *st = PerlIOSelf(f, PerlIOAPR);
840 /* fill in the st struct, as in _open() */
841 st->file = file;
842 PerlIOBase(f)->flags |= PERLIO_F_OPEN;
843
844 return f;
845 }
846 return NULL;
847 }
848
849=item *
850
851fix/add the documentation in places marked as XXX.
852
853=item *
854
855The handling of errors by the layer is not specified. e.g. when $!
856should be set explicitly, when the error handling should be just
857delegated to the top layer.
858
859Probably give some hints on using SETERRNO() or pointers to where they
860can be found.
861
862=item *
863
864I think it would help to give some concrete examples to make it easier
865to understand the API. Of course I agree that the API has to be
866concise, but since there is no second document that is more of a
867guide, I think that it'd make it easier to start with the doc which is
868an API, but has examples in it in places where things are unclear, to
869a person who is not a PerlIO guru (yet).
870
871=back
872
50b80e25 873=cut