This is a live mirror of the Perl 5 development currently hosted at https://github.com/perl/perl5
Better advertising.
[perl5.git] / pod / perlfaq5.pod
CommitLineData
68dc0745
PP
1=head1 NAME
2
d2321c93 3perlfaq5 - Files and Formats ($Revision: 1.9 $, $Date: 2002/02/11 19:30:21 $)
68dc0745
PP
4
5=head1 DESCRIPTION
6
7This section deals with I/O and the "f" issues: filehandles, flushing,
8formats, and footers.
9
5a964f20 10=head2 How do I flush/unbuffer an output filehandle? Why must I do this?
68dc0745
PP
11
12The C standard I/O library (stdio) normally buffers characters sent to
a6dd486b 13devices. This is done for efficiency reasons so that there isn't a
68dc0745
PP
14system call for each byte. Any time you use print() or write() in
15Perl, you go though this buffering. syswrite() circumvents stdio and
16buffering.
17
5a964f20 18In most stdio implementations, the type of output buffering and the size of
68dc0745
PP
19the buffer varies according to the type of device. Disk files are block
20buffered, often with a buffer size of more than 2k. Pipes and sockets
21are often buffered with a buffer size between 1/2 and 2k. Serial devices
22(e.g. modems, terminals) are normally line-buffered, and stdio sends
23the entire line when it gets the newline.
24
25Perl does not support truly unbuffered output (except insofar as you can
26C<syswrite(OUT, $char, 1)>). What it does instead support is "command
27buffering", in which a physical write is performed after every output
28command. This isn't as hard on your system as unbuffering, but does
29get the output where you want it when you want it.
30
31If you expect characters to get to your device when you print them there,
5a964f20
TC
32you'll want to autoflush its handle.
33Use select() and the C<$|> variable to control autoflushing
34(see L<perlvar/$|> and L<perlfunc/select>):
35
36 $old_fh = select(OUTPUT_HANDLE);
37 $| = 1;
38 select($old_fh);
39
40Or using the traditional idiom:
41
42 select((select(OUTPUT_HANDLE), $| = 1)[0]);
43
44Or if don't mind slowly loading several thousand lines of module code
45just because you're afraid of the C<$|> variable:
68dc0745
PP
46
47 use FileHandle;
5a964f20 48 open(DEV, "+</dev/tty"); # ceci n'est pas une pipe
68dc0745
PP
49 DEV->autoflush(1);
50
51or the newer IO::* modules:
52
53 use IO::Handle;
54 open(DEV, ">/dev/printer"); # but is this?
55 DEV->autoflush(1);
56
57or even this:
58
59 use IO::Socket; # this one is kinda a pipe?
60 $sock = IO::Socket::INET->new(PeerAddr => 'www.perl.com',
61 PeerPort => 'http(80)',
62 Proto => 'tcp');
63 die "$!" unless $sock;
64
65 $sock->autoflush();
5a964f20
TC
66 print $sock "GET / HTTP/1.0" . "\015\012" x 2;
67 $document = join('', <$sock>);
68dc0745
PP
68 print "DOC IS: $document\n";
69
8305e449 70Note the bizarrely hard coded carriage return and newline in their octal
5a964f20 71equivalents. This is the ONLY way (currently) to assure a proper flush
d92eb7b0 72on all platforms, including Macintosh. That's the way things work in
5a964f20
TC
73network programming: you really should specify the exact bit pattern
74on the network line terminator. In practice, C<"\n\n"> often works,
75but this is not portable.
68dc0745 76
5a964f20 77See L<perlfaq9> for other examples of fetching URLs over the web.
68dc0745
PP
78
79=head2 How do I change one line in a file/delete a line in a file/insert a line in the middle of a file/append to the beginning of a file?
80
1f089b22
JH
81Use the Tie::File module, which is included in the standard
82distribution since Perl 5.8.0.
68dc0745
PP
83
84=head2 How do I count the number of lines in a file?
85
86One fairly efficient way is to count newlines in the file. The
87following program uses a feature of tr///, as documented in L<perlop>.
88If your text file doesn't end with a newline, then it's not really a
89proper text file, so this may report one fewer line than you expect.
90
91 $lines = 0;
92 open(FILE, $filename) or die "Can't open `$filename': $!";
93 while (sysread FILE, $buffer, 4096) {
94 $lines += ($buffer =~ tr/\n//);
95 }
96 close FILE;
97
5a964f20
TC
98This assumes no funny games with newline translations.
99
68dc0745
PP
100=head2 How do I make a temporary file name?
101
16394a69 102Use the File::Temp module, see L<File::Temp> for more information.
68dc0745 103
16394a69 104 use File::Temp qw/ tempfile tempdir /;
a6dd486b 105
16394a69
JH
106 $dir = tempdir( CLEANUP => 1 );
107 ($fh, $filename) = tempfile( DIR => $dir );
5a964f20 108
16394a69 109 # or if you don't need to know the filename
5a964f20 110
16394a69 111 $fh = tempfile( DIR => $dir );
5a964f20 112
16394a69
JH
113The File::Temp has been a standard module since Perl 5.6.1. If you
114don't have a modern enough Perl installed, use the C<new_tmpfile>
115class method from the IO::File module to get a filehandle opened for
116reading and writing. Use it if you don't need to know the file's name:
5a964f20 117
16394a69
JH
118 use IO::File;
119 $fh = IO::File->new_tmpfile()
120 or die "Unable to make new temporary file: $!";
5a964f20 121
a6dd486b
JB
122If you're committed to creating a temporary file by hand, use the
123process ID and/or the current time-value. If you need to have many
124temporary files in one process, use a counter:
5a964f20
TC
125
126 BEGIN {
68dc0745 127 use Fcntl;
16394a69 128 my $temp_dir = -d '/tmp' ? '/tmp' : $ENV{TMPDIR} || $ENV{TEMP};
68dc0745
PP
129 my $base_name = sprintf("%s/%d-%d-0000", $temp_dir, $$, time());
130 sub temp_file {
5a964f20 131 local *FH;
68dc0745 132 my $count = 0;
5a964f20 133 until (defined(fileno(FH)) || $count++ > 100) {
68dc0745 134 $base_name =~ s/-(\d+)$/"-" . (1 + $1)/e;
5a964f20 135 sysopen(FH, $base_name, O_WRONLY|O_EXCL|O_CREAT);
68dc0745 136 }
5a964f20
TC
137 if (defined(fileno(FH))
138 return (*FH, $base_name);
68dc0745
PP
139 } else {
140 return ();
141 }
142 }
143 }
144
68dc0745
PP
145=head2 How can I manipulate fixed-record-length files?
146
5a964f20 147The most efficient way is using pack() and unpack(). This is faster than
65acb1b1 148using substr() when taking many, many strings. It is slower for just a few.
5a964f20
TC
149
150Here is a sample chunk of code to break up and put back together again
151some fixed-format input lines, in this case from the output of a normal,
152Berkeley-style ps:
68dc0745
PP
153
154 # sample input line:
155 # 15158 p5 T 0:00 perl /home/tchrist/scripts/now-what
156 $PS_T = 'A6 A4 A7 A5 A*';
157 open(PS, "ps|");
5a964f20 158 print scalar <PS>;
68dc0745
PP
159 while (<PS>) {
160 ($pid, $tt, $stat, $time, $command) = unpack($PS_T, $_);
161 for $var (qw!pid tt stat time command!) {
162 print "$var: <$$var>\n";
163 }
164 print 'line=', pack($PS_T, $pid, $tt, $stat, $time, $command),
165 "\n";
166 }
167
5a964f20
TC
168We've used C<$$var> in a way that forbidden by C<use strict 'refs'>.
169That is, we've promoted a string to a scalar variable reference using
8305e449 170symbolic references. This is okay in small programs, but doesn't scale
5a964f20
TC
171well. It also only works on global variables, not lexicals.
172
68dc0745
PP
173=head2 How can I make a filehandle local to a subroutine? How do I pass filehandles between subroutines? How do I make an array of filehandles?
174
5a964f20
TC
175The fastest, simplest, and most direct way is to localize the typeglob
176of the filehandle in question:
68dc0745 177
5a964f20 178 local *TmpHandle;
68dc0745 179
5a964f20
TC
180Typeglobs are fast (especially compared with the alternatives) and
181reasonably easy to use, but they also have one subtle drawback. If you
182had, for example, a function named TmpHandle(), or a variable named
183%TmpHandle, you just hid it from yourself.
68dc0745 184
68dc0745 185 sub findme {
5a964f20
TC
186 local *HostFile;
187 open(HostFile, "</etc/hosts") or die "no /etc/hosts: $!";
188 local $_; # <- VERY IMPORTANT
189 while (<HostFile>) {
68dc0745
PP
190 print if /\b127\.(0\.0\.)?1\b/;
191 }
5a964f20
TC
192 # *HostFile automatically closes/disappears here
193 }
194
a6dd486b 195Here's how to use typeglobs in a loop to open and store a bunch of
5a964f20
TC
196filehandles. We'll use as values of the hash an ordered
197pair to make it easy to sort the hash in insertion order.
198
199 @names = qw(motd termcap passwd hosts);
200 my $i = 0;
201 foreach $filename (@names) {
202 local *FH;
203 open(FH, "/etc/$filename") || die "$filename: $!";
204 $file{$filename} = [ $i++, *FH ];
68dc0745
PP
205 }
206
5a964f20
TC
207 # Using the filehandles in the array
208 foreach $name (sort { $file{$a}[0] <=> $file{$b}[0] } keys %file) {
209 my $fh = $file{$name}[1];
210 my $line = <$fh>;
211 print "$name $. $line";
212 }
213
c8db1d39 214For passing filehandles to functions, the easiest way is to
13a2d996
SP
215preface them with a star, as in func(*STDIN).
216See L<perlfaq7/"Passing Filehandles"> for details.
c8db1d39 217
65acb1b1 218If you want to create many anonymous handles, you should check out the
5a964f20
TC
219Symbol, FileHandle, or IO::Handle (etc.) modules. Here's the equivalent
220code with Symbol::gensym, which is reasonably light-weight:
221
222 foreach $filename (@names) {
223 use Symbol;
224 my $fh = gensym();
225 open($fh, "/etc/$filename") || die "open /etc/$filename: $!";
226 $file{$filename} = [ $i++, $fh ];
227 }
68dc0745 228
a6dd486b 229Here's using the semi-object-oriented FileHandle module, which certainly
65acb1b1 230isn't light-weight:
46fc3d4c
PP
231
232 use FileHandle;
233
46fc3d4c 234 foreach $filename (@names) {
5a964f20
TC
235 my $fh = FileHandle->new("/etc/$filename") or die "$filename: $!";
236 $file{$filename} = [ $i++, $fh ];
46fc3d4c
PP
237 }
238
5a964f20 239Please understand that whether the filehandle happens to be a (probably
a6dd486b 240localized) typeglob or an anonymous handle from one of the modules
5a964f20
TC
241in no way affects the bizarre rules for managing indirect handles.
242See the next question.
243
244=head2 How can I use a filehandle indirectly?
245
246An indirect filehandle is using something other than a symbol
247in a place that a filehandle is expected. Here are ways
a6dd486b 248to get indirect filehandles:
5a964f20
TC
249
250 $fh = SOME_FH; # bareword is strict-subs hostile
251 $fh = "SOME_FH"; # strict-refs hostile; same package only
252 $fh = *SOME_FH; # typeglob
253 $fh = \*SOME_FH; # ref to typeglob (bless-able)
254 $fh = *SOME_FH{IO}; # blessed IO::Handle from *SOME_FH typeglob
255
a6dd486b 256Or, you can use the C<new> method from the FileHandle or IO modules to
5a964f20
TC
257create an anonymous filehandle, store that in a scalar variable,
258and use it as though it were a normal filehandle.
259
260 use FileHandle;
261 $fh = FileHandle->new();
262
263 use IO::Handle; # 5.004 or higher
264 $fh = IO::Handle->new();
265
266Then use any of those as you would a normal filehandle. Anywhere that
267Perl is expecting a filehandle, an indirect filehandle may be used
268instead. An indirect filehandle is just a scalar variable that contains
368c9434 269a filehandle. Functions like C<print>, C<open>, C<seek>, or
c47ff5f1 270the C<< <FH> >> diamond operator will accept either a read filehandle
5a964f20
TC
271or a scalar variable containing one:
272
273 ($ifh, $ofh, $efh) = (*STDIN, *STDOUT, *STDERR);
274 print $ofh "Type it: ";
275 $got = <$ifh>
276 print $efh "What was that: $got";
277
368c9434 278If you're passing a filehandle to a function, you can write
5a964f20
TC
279the function in two ways:
280
281 sub accept_fh {
282 my $fh = shift;
283 print $fh "Sending to indirect filehandle\n";
46fc3d4c
PP
284 }
285
5a964f20 286Or it can localize a typeglob and use the filehandle directly:
46fc3d4c 287
5a964f20
TC
288 sub accept_fh {
289 local *FH = shift;
290 print FH "Sending to localized filehandle\n";
46fc3d4c
PP
291 }
292
5a964f20
TC
293Both styles work with either objects or typeglobs of real filehandles.
294(They might also work with strings under some circumstances, but this
295is risky.)
296
297 accept_fh(*STDOUT);
298 accept_fh($handle);
299
300In the examples above, we assigned the filehandle to a scalar variable
a6dd486b
JB
301before using it. That is because only simple scalar variables, not
302expressions or subscripts of hashes or arrays, can be used with
303built-ins like C<print>, C<printf>, or the diamond operator. Using
8305e449 304something other than a simple scalar variable as a filehandle is
5a964f20
TC
305illegal and won't even compile:
306
307 @fd = (*STDIN, *STDOUT, *STDERR);
308 print $fd[1] "Type it: "; # WRONG
309 $got = <$fd[0]> # WRONG
310 print $fd[2] "What was that: $got"; # WRONG
311
312With C<print> and C<printf>, you get around this by using a block and
313an expression where you would place the filehandle:
314
315 print { $fd[1] } "funny stuff\n";
316 printf { $fd[1] } "Pity the poor %x.\n", 3_735_928_559;
317 # Pity the poor deadbeef.
318
319That block is a proper block like any other, so you can put more
320complicated code there. This sends the message out to one of two places:
321
322 $ok = -x "/bin/cat";
323 print { $ok ? $fd[1] : $fd[2] } "cat stat $ok\n";
324 print { $fd[ 1+ ($ok || 0) ] } "cat stat $ok\n";
325
326This approach of treating C<print> and C<printf> like object methods
327calls doesn't work for the diamond operator. That's because it's a
328real operator, not just a function with a comma-less argument. Assuming
329you've been storing typeglobs in your structure as we did above, you
330can use the built-in function named C<readline> to reads a record just
c47ff5f1 331as C<< <> >> does. Given the initialization shown above for @fd, this
5a964f20
TC
332would work, but only because readline() require a typeglob. It doesn't
333work with objects or strings, which might be a bug we haven't fixed yet.
334
335 $got = readline($fd[0]);
336
337Let it be noted that the flakiness of indirect filehandles is not
338related to whether they're strings, typeglobs, objects, or anything else.
339It's the syntax of the fundamental operators. Playing the object
340game doesn't help you at all here.
46fc3d4c 341
68dc0745
PP
342=head2 How can I set up a footer format to be used with write()?
343
54310121 344There's no builtin way to do this, but L<perlform> has a couple of
68dc0745
PP
345techniques to make it possible for the intrepid hacker.
346
347=head2 How can I write() into a string?
348
65acb1b1 349See L<perlform/"Accessing Formatting Internals"> for an swrite() function.
68dc0745
PP
350
351=head2 How can I output my numbers with commas added?
352
353This one will do it for you:
354
355 sub commify {
24f1ba9b
JH
356 my $number = shift;
357 1 while ($number =~ s/^([-+]?\d+)(\d{3})/$1,$2/);
358 return $number;
68dc0745
PP
359 }
360
361 $n = 23659019423.2331;
362 print "GOT: ", commify($n), "\n";
363
364 GOT: 23,659,019,423.2331
365
366You can't just:
367
65acb1b1 368 s/^([-+]?\d+)(\d{3})/$1,$2/g;
68dc0745
PP
369
370because you have to put the comma in and then recalculate your
371position.
372
a6dd486b 373Alternatively, this code commifies all numbers in a line regardless of
46fc3d4c
PP
374whether they have decimal portions, are preceded by + or -, or
375whatever:
376
377 # from Andrew Johnson <ajohnson@gpu.srv.ualberta.ca>
378 sub commify {
379 my $input = shift;
380 $input = reverse $input;
381 $input =~ s<(\d\d\d)(?=\d)(?!\d*\.)><$1,>g;
65acb1b1 382 return scalar reverse $input;
46fc3d4c
PP
383 }
384
68dc0745
PP
385=head2 How can I translate tildes (~) in a filename?
386
575cc754
JH
387Use the <> (glob()) operator, documented in L<perlfunc>. Older
388versions of Perl require that you have a shell installed that groks
389tildes. Recent perl versions have this feature built in. The
d6260402 390File::KGlob module (available from CPAN) gives more portable glob
575cc754 391functionality.
68dc0745
PP
392
393Within Perl, you may use this directly:
394
395 $filename =~ s{
396 ^ ~ # find a leading tilde
397 ( # save this in $1
398 [^/] # a non-slash character
399 * # repeated 0 or more times (0 means me)
400 )
401 }{
402 $1
403 ? (getpwnam($1))[7]
404 : ( $ENV{HOME} || $ENV{LOGDIR} )
405 }ex;
406
5a964f20 407=head2 How come when I open a file read-write it wipes it out?
68dc0745
PP
408
409Because you're using something like this, which truncates the file and
410I<then> gives you read-write access:
411
5a964f20 412 open(FH, "+> /path/name"); # WRONG (almost always)
68dc0745
PP
413
414Whoops. You should instead use this, which will fail if the file
d92eb7b0
GS
415doesn't exist.
416
417 open(FH, "+< /path/name"); # open for update
418
c47ff5f1 419Using ">" always clobbers or creates. Using "<" never does
d92eb7b0 420either. The "+" doesn't change this.
68dc0745 421
5a964f20
TC
422Here are examples of many kinds of file opens. Those using sysopen()
423all assume
68dc0745 424
5a964f20 425 use Fcntl;
68dc0745 426
5a964f20 427To open file for reading:
68dc0745 428
5a964f20
TC
429 open(FH, "< $path") || die $!;
430 sysopen(FH, $path, O_RDONLY) || die $!;
431
432To open file for writing, create new file if needed or else truncate old file:
433
434 open(FH, "> $path") || die $!;
435 sysopen(FH, $path, O_WRONLY|O_TRUNC|O_CREAT) || die $!;
436 sysopen(FH, $path, O_WRONLY|O_TRUNC|O_CREAT, 0666) || die $!;
437
438To open file for writing, create new file, file must not exist:
439
440 sysopen(FH, $path, O_WRONLY|O_EXCL|O_CREAT) || die $!;
441 sysopen(FH, $path, O_WRONLY|O_EXCL|O_CREAT, 0666) || die $!;
442
443To open file for appending, create if necessary:
444
445 open(FH, ">> $path") || die $!;
446 sysopen(FH, $path, O_WRONLY|O_APPEND|O_CREAT) || die $!;
447 sysopen(FH, $path, O_WRONLY|O_APPEND|O_CREAT, 0666) || die $!;
448
449To open file for appending, file must exist:
450
451 sysopen(FH, $path, O_WRONLY|O_APPEND) || die $!;
452
453To open file for update, file must exist:
454
455 open(FH, "+< $path") || die $!;
456 sysopen(FH, $path, O_RDWR) || die $!;
457
458To open file for update, create file if necessary:
459
460 sysopen(FH, $path, O_RDWR|O_CREAT) || die $!;
461 sysopen(FH, $path, O_RDWR|O_CREAT, 0666) || die $!;
462
463To open file for update, file must not exist:
464
465 sysopen(FH, $path, O_RDWR|O_EXCL|O_CREAT) || die $!;
466 sysopen(FH, $path, O_RDWR|O_EXCL|O_CREAT, 0666) || die $!;
467
468To open a file without blocking, creating if necessary:
469
470 sysopen(FH, "/tmp/somefile", O_WRONLY|O_NDELAY|O_CREAT)
471 or die "can't open /tmp/somefile: $!":
472
473Be warned that neither creation nor deletion of files is guaranteed to
474be an atomic operation over NFS. That is, two processes might both
a6dd486b
JB
475successfully create or unlink the same file! Therefore O_EXCL
476isn't as exclusive as you might wish.
68dc0745 477
87275199 478See also the new L<perlopentut> if you have it (new for 5.6).
65acb1b1 479
c47ff5f1 480=head2 Why do I sometimes get an "Argument list too long" when I use <*>?
68dc0745 481
c47ff5f1 482The C<< <> >> operator performs a globbing operation (see above).
3a4b19e4
GS
483In Perl versions earlier than v5.6.0, the internal glob() operator forks
484csh(1) to do the actual glob expansion, but
68dc0745
PP
485csh can't handle more than 127 items and so gives the error message
486C<Argument list too long>. People who installed tcsh as csh won't
487have this problem, but their users may be surprised by it.
488
3a4b19e4 489To get around this, either upgrade to Perl v5.6.0 or later, do the glob
d6260402 490yourself with readdir() and patterns, or use a module like File::KGlob,
3a4b19e4 491one that doesn't use the shell to do globbing.
68dc0745
PP
492
493=head2 Is there a leak/bug in glob()?
494
495Due to the current implementation on some operating systems, when you
496use the glob() function or its angle-bracket alias in a scalar
a6dd486b 497context, you may cause a memory leak and/or unpredictable behavior. It's
68dc0745
PP
498best therefore to use glob() only in list context.
499
c47ff5f1 500=head2 How can I open a file with a leading ">" or trailing blanks?
68dc0745
PP
501
502Normally perl ignores trailing blanks in filenames, and interprets
503certain leading characters (or a trailing "|") to mean something
a6dd486b
JB
504special. To avoid this, you might want to use a routine like the one below.
505It turns incomplete pathnames into explicit relative ones, and tacks a
68dc0745
PP
506trailing null byte on the name to make perl leave it alone:
507
508 sub safe_filename {
509 local $_ = shift;
65acb1b1
TC
510 s#^([^./])#./$1#;
511 $_ .= "\0";
512 return $_;
68dc0745
PP
513 }
514
65acb1b1
TC
515 $badpath = "<<<something really wicked ";
516 $fn = safe_filename($badpath");
517 open(FH, "> $fn") or "couldn't open $badpath: $!";
518
519This assumes that you are using POSIX (portable operating systems
520interface) paths. If you are on a closed, non-portable, proprietary
521system, you may have to adjust the C<"./"> above.
522
523It would be a lot clearer to use sysopen(), though:
524
525 use Fcntl;
526 $badpath = "<<<something really wicked ";
a6dd486b 527 sysopen (FH, $badpath, O_WRONLY | O_CREAT | O_TRUNC)
65acb1b1 528 or die "can't open $badpath: $!";
68dc0745 529
65acb1b1 530For more information, see also the new L<perlopentut> if you have it
87275199 531(new for 5.6).
68dc0745
PP
532
533=head2 How can I reliably rename a file?
534
d2321c93 535If your operating system supports a proper mv(1) utility or its functional
d92eb7b0 536equivalent, this works:
68dc0745
PP
537
538 rename($old, $new) or system("mv", $old, $new);
539
d2321c93
JH
540It may be more portable to use the File::Copy module instead.
541You just copy to the new file to the new name (checking return
542values), then delete the old one. This isn't really the same
543semantically as a rename(), which preserves meta-information like
68dc0745
PP
544permissions, timestamps, inode info, etc.
545
d2321c93 546Newer versions of File::Copy export a move() function.
5a964f20 547
68dc0745
PP
548=head2 How can I lock a file?
549
54310121 550Perl's builtin flock() function (see L<perlfunc> for details) will call
68dc0745
PP
551flock(2) if that exists, fcntl(2) if it doesn't (on perl version 5.004 and
552later), and lockf(3) if neither of the two previous system calls exists.
553On some systems, it may even use a different form of native locking.
554Here are some gotchas with Perl's flock():
555
556=over 4
557
558=item 1
559
560Produces a fatal error if none of the three system calls (or their
561close equivalent) exists.
562
563=item 2
564
565lockf(3) does not provide shared locking, and requires that the
566filehandle be open for writing (or appending, or read/writing).
567
568=item 3
569
d92eb7b0
GS
570Some versions of flock() can't lock files over a network (e.g. on NFS file
571systems), so you'd need to force the use of fcntl(2) when you build Perl.
a6dd486b 572But even this is dubious at best. See the flock entry of L<perlfunc>
d92eb7b0
GS
573and the F<INSTALL> file in the source distribution for information on
574building Perl to do this.
575
576Two potentially non-obvious but traditional flock semantics are that
a6dd486b 577it waits indefinitely until the lock is granted, and that its locks are
d92eb7b0
GS
578I<merely advisory>. Such discretionary locks are more flexible, but
579offer fewer guarantees. This means that files locked with flock() may
580be modified by programs that do not also use flock(). Cars that stop
581for red lights get on well with each other, but not with cars that don't
582stop for red lights. See the perlport manpage, your port's specific
583documentation, or your system-specific local manpages for details. It's
584best to assume traditional behavior if you're writing portable programs.
a6dd486b 585(If you're not, you should as always feel perfectly free to write
d92eb7b0
GS
586for your own system's idiosyncrasies (sometimes called "features").
587Slavish adherence to portability concerns shouldn't get in the way of
588your getting your job done.)
68dc0745 589
13a2d996
SP
590For more information on file locking, see also
591L<perlopentut/"File Locking"> if you have it (new for 5.6).
65acb1b1 592
68dc0745
PP
593=back
594
65acb1b1 595=head2 Why can't I just open(FH, ">file.lock")?
68dc0745
PP
596
597A common bit of code B<NOT TO USE> is this:
598
599 sleep(3) while -e "file.lock"; # PLEASE DO NOT USE
600 open(LCK, "> file.lock"); # THIS BROKEN CODE
601
602This is a classic race condition: you take two steps to do something
603which must be done in one. That's why computer hardware provides an
604atomic test-and-set instruction. In theory, this "ought" to work:
605
5a964f20 606 sysopen(FH, "file.lock", O_WRONLY|O_EXCL|O_CREAT)
68dc0745
PP
607 or die "can't open file.lock: $!":
608
609except that lamentably, file creation (and deletion) is not atomic
610over NFS, so this won't work (at least, not every time) over the net.
65acb1b1 611Various schemes involving link() have been suggested, but
46fc3d4c 612these tend to involve busy-wait, which is also subdesirable.
68dc0745 613
fc36a67e 614=head2 I still don't get locking. I just want to increment the number in the file. How can I do this?
68dc0745 615
46fc3d4c 616Didn't anyone ever tell you web-page hit counters were useless?
5a964f20 617They don't count number of hits, they're a waste of time, and they serve
a6dd486b
JB
618only to stroke the writer's vanity. It's better to pick a random number;
619they're more realistic.
68dc0745 620
5a964f20 621Anyway, this is what you can do if you can't help yourself.
68dc0745 622
e2c57c3e 623 use Fcntl qw(:DEFAULT :flock);
5a964f20 624 sysopen(FH, "numfile", O_RDWR|O_CREAT) or die "can't open numfile: $!";
65acb1b1 625 flock(FH, LOCK_EX) or die "can't flock numfile: $!";
68dc0745
PP
626 $num = <FH> || 0;
627 seek(FH, 0, 0) or die "can't rewind numfile: $!";
628 truncate(FH, 0) or die "can't truncate numfile: $!";
629 (print FH $num+1, "\n") or die "can't write numfile: $!";
68dc0745
PP
630 close FH or die "can't close numfile: $!";
631
46fc3d4c 632Here's a much better web-page hit counter:
68dc0745
PP
633
634 $hits = int( (time() - 850_000_000) / rand(1_000) );
635
636If the count doesn't impress your friends, then the code might. :-)
637
f52f3be2 638=head2 All I want to do is append a small amount of text to the end of a file. Do I still have to use locking?
05caf3a7
GJ
639
640If you are on a system that correctly implements flock() and you use the
641example appending code from "perldoc -f flock" everything will be OK
642even if the OS you are on doesn't implement append mode correctly (if
643such a system exists.) So if you are happy to restrict yourself to OSs
644that implement flock() (and that's not really much of a restriction)
645then that is what you should do.
646
647If you know you are only going to use a system that does correctly
648implement appending (i.e. not Win32) then you can omit the seek() from
649the above code.
650
651If you know you are only writing code to run on an OS and filesystem that
652does implement append mode correctly (a local filesystem on a modern
653Unix for example), and you keep the file in block-buffered mode and you
654write less than one buffer-full of output between each manual flushing
8305e449 655of the buffer then each bufferload is almost guaranteed to be written to
05caf3a7
GJ
656the end of the file in one chunk without getting intermingled with
657anyone else's output. You can also use the syswrite() function which is
658simply a wrapper around your systems write(2) system call.
659
660There is still a small theoretical chance that a signal will interrupt
661the system level write() operation before completion. There is also a
662possibility that some STDIO implementations may call multiple system
663level write()s even if the buffer was empty to start. There may be some
664systems where this probability is reduced to zero.
665
68dc0745
PP
666=head2 How do I randomly update a binary file?
667
668If you're just trying to patch a binary, in many cases something as
669simple as this works:
670
671 perl -i -pe 's{window manager}{window mangler}g' /usr/bin/emacs
672
673However, if you have fixed sized records, then you might do something more
674like this:
675
676 $RECSIZE = 220; # size of record, in bytes
677 $recno = 37; # which record to update
678 open(FH, "+<somewhere") || die "can't update somewhere: $!";
679 seek(FH, $recno * $RECSIZE, 0);
680 read(FH, $record, $RECSIZE) == $RECSIZE || die "can't read record $recno: $!";
681 # munge the record
65acb1b1 682 seek(FH, -$RECSIZE, 1);
68dc0745
PP
683 print FH $record;
684 close FH;
685
686Locking and error checking are left as an exercise for the reader.
a6dd486b 687Don't forget them or you'll be quite sorry.
68dc0745 688
68dc0745
PP
689=head2 How do I get a file's timestamp in perl?
690
691If you want to retrieve the time at which the file was last read,
46fc3d4c 692written, or had its meta-data (owner, etc) changed, you use the B<-M>,
8305e449 693B<-A>, or B<-C> file test operations as documented in L<perlfunc>. These
68dc0745
PP
694retrieve the age of the file (measured against the start-time of your
695program) in days as a floating point number. To retrieve the "raw"
696time in seconds since the epoch, you would call the stat function,
697then use localtime(), gmtime(), or POSIX::strftime() to convert this
698into human-readable form.
699
700Here's an example:
701
702 $write_secs = (stat($file))[9];
c8db1d39
TC
703 printf "file %s updated at %s\n", $file,
704 scalar localtime($write_secs);
68dc0745
PP
705
706If you prefer something more legible, use the File::stat module
707(part of the standard distribution in version 5.004 and later):
708
65acb1b1 709 # error checking left as an exercise for reader.
68dc0745
PP
710 use File::stat;
711 use Time::localtime;
712 $date_string = ctime(stat($file)->mtime);
713 print "file $file updated at $date_string\n";
714
65acb1b1
TC
715The POSIX::strftime() approach has the benefit of being,
716in theory, independent of the current locale. See L<perllocale>
717for details.
68dc0745
PP
718
719=head2 How do I set a file's timestamp in perl?
720
721You use the utime() function documented in L<perlfunc/utime>.
722By way of example, here's a little program that copies the
723read and write times from its first argument to all the rest
724of them.
725
726 if (@ARGV < 2) {
727 die "usage: cptimes timestamp_file other_files ...\n";
728 }
729 $timestamp = shift;
730 ($atime, $mtime) = (stat($timestamp))[8,9];
731 utime $atime, $mtime, @ARGV;
732
65acb1b1 733Error checking is, as usual, left as an exercise for the reader.
68dc0745
PP
734
735Note that utime() currently doesn't work correctly with Win95/NT
736ports. A bug has been reported. Check it carefully before using
a6dd486b 737utime() on those platforms.
68dc0745
PP
738
739=head2 How do I print to more than one file at once?
740
741If you only have to do this once, you can do this:
742
743 for $fh (FH1, FH2, FH3) { print $fh "whatever\n" }
744
745To connect up to one filehandle to several output filehandles, it's
746easiest to use the tee(1) program if you have it, and let it take care
747of the multiplexing:
748
749 open (FH, "| tee file1 file2 file3");
750
5a964f20
TC
751Or even:
752
753 # make STDOUT go to three files, plus original STDOUT
754 open (STDOUT, "| tee file1 file2 file3") or die "Teeing off: $!\n";
755 print "whatever\n" or die "Writing: $!\n";
756 close(STDOUT) or die "Closing: $!\n";
68dc0745 757
5a964f20 758Otherwise you'll have to write your own multiplexing print
a6dd486b 759function--or your own tee program--or use Tom Christiansen's,
a93751fa 760at http://www.cpan.org/authors/id/TOMC/scripts/tct.gz , which is
5a964f20
TC
761written in Perl and offers much greater functionality
762than the stock version.
68dc0745 763
d92eb7b0
GS
764=head2 How can I read in an entire file all at once?
765
766The customary Perl approach for processing all the lines in a file is to
767do so one line at a time:
768
769 open (INPUT, $file) || die "can't open $file: $!";
770 while (<INPUT>) {
771 chomp;
772 # do something with $_
773 }
774 close(INPUT) || die "can't close $file: $!";
775
776This is tremendously more efficient than reading the entire file into
777memory as an array of lines and then processing it one element at a time,
a6dd486b 778which is often--if not almost always--the wrong approach. Whenever
d92eb7b0
GS
779you see someone do this:
780
781 @lines = <INPUT>;
782
a6dd486b 783you should think long and hard about why you need everything loaded
d92eb7b0 784at once. It's just not a scalable solution. You might also find it
106325ad 785more fun to use the standard DB_File module's $DB_RECNO bindings,
d92eb7b0
GS
786which allow you to tie an array to a file so that accessing an element
787the array actually accesses the corresponding line in the file.
788
789On very rare occasion, you may have an algorithm that demands that
790the entire file be in memory at once as one scalar. The simplest solution
a6dd486b 791to that is
d92eb7b0
GS
792
793 $var = `cat $file`;
794
795Being in scalar context, you get the whole thing. In list context,
796you'd get a list of all the lines:
797
798 @lines = `cat $file`;
799
87275199
GS
800This tiny but expedient solution is neat, clean, and portable to
801all systems on which decent tools have been installed. For those
802who prefer not to use the toolbox, you can of course read the file
803manually, although this makes for more complicated code.
d92eb7b0
GS
804
805 {
806 local(*INPUT, $/);
807 open (INPUT, $file) || die "can't open $file: $!";
808 $var = <INPUT>;
809 }
810
811That temporarily undefs your record separator, and will automatically
812close the file at block exit. If the file is already open, just use this:
813
814 $var = do { local $/; <INPUT> };
815
68dc0745
PP
816=head2 How can I read in a file by paragraphs?
817
65acb1b1 818Use the C<$/> variable (see L<perlvar> for details). You can either
68dc0745
PP
819set it to C<""> to eliminate empty paragraphs (C<"abc\n\n\n\ndef">,
820for instance, gets treated as two paragraphs and not three), or
821C<"\n\n"> to accept empty paragraphs.
822
d05ac700 823Note that a blank line must have no blanks in it. Thus
c4db748a 824S<C<"fred\n \nstuff\n\n">> is one paragraph, but C<"fred\n\nstuff\n\n"> is two.
65acb1b1 825
68dc0745
PP
826=head2 How can I read a single character from a file? From the keyboard?
827
828You can use the builtin C<getc()> function for most filehandles, but
829it won't (easily) work on a terminal device. For STDIN, either use
a6dd486b 830the Term::ReadKey module from CPAN or use the sample code in
68dc0745
PP
831L<perlfunc/getc>.
832
65acb1b1
TC
833If your system supports the portable operating system programming
834interface (POSIX), you can use the following code, which you'll note
835turns off echo processing as well.
68dc0745
PP
836
837 #!/usr/bin/perl -w
838 use strict;
839 $| = 1;
840 for (1..4) {
841 my $got;
842 print "gimme: ";
843 $got = getone();
844 print "--> $got\n";
845 }
846 exit;
847
848 BEGIN {
849 use POSIX qw(:termios_h);
850
851 my ($term, $oterm, $echo, $noecho, $fd_stdin);
852
853 $fd_stdin = fileno(STDIN);
854
855 $term = POSIX::Termios->new();
856 $term->getattr($fd_stdin);
857 $oterm = $term->getlflag();
858
859 $echo = ECHO | ECHOK | ICANON;
860 $noecho = $oterm & ~$echo;
861
862 sub cbreak {
863 $term->setlflag($noecho);
864 $term->setcc(VTIME, 1);
865 $term->setattr($fd_stdin, TCSANOW);
866 }
867
868 sub cooked {
869 $term->setlflag($oterm);
870 $term->setcc(VTIME, 0);
871 $term->setattr($fd_stdin, TCSANOW);
872 }
873
874 sub getone {
875 my $key = '';
876 cbreak();
877 sysread(STDIN, $key, 1);
878 cooked();
879 return $key;
880 }
881
882 }
883
884 END { cooked() }
885
a6dd486b 886The Term::ReadKey module from CPAN may be easier to use. Recent versions
65acb1b1 887include also support for non-portable systems as well.
68dc0745
PP
888
889 use Term::ReadKey;
890 open(TTY, "</dev/tty");
891 print "Gimme a char: ";
892 ReadMode "raw";
893 $key = ReadKey 0, *TTY;
894 ReadMode "normal";
895 printf "\nYou said %s, char number %03d\n",
896 $key, ord $key;
897
65acb1b1 898=head2 How can I tell whether there's a character waiting on a filehandle?
68dc0745 899
5a964f20 900The very first thing you should do is look into getting the Term::ReadKey
65acb1b1
TC
901extension from CPAN. As we mentioned earlier, it now even has limited
902support for non-portable (read: not open systems, closed, proprietary,
903not POSIX, not Unix, etc) systems.
5a964f20
TC
904
905You should also check out the Frequently Asked Questions list in
68dc0745
PP
906comp.unix.* for things like this: the answer is essentially the same.
907It's very system dependent. Here's one solution that works on BSD
908systems:
909
910 sub key_ready {
911 my($rin, $nfd);
912 vec($rin, fileno(STDIN), 1) = 1;
913 return $nfd = select($rin,undef,undef,0);
914 }
915
65acb1b1
TC
916If you want to find out how many characters are waiting, there's
917also the FIONREAD ioctl call to be looked at. The I<h2ph> tool that
918comes with Perl tries to convert C include files to Perl code, which
919can be C<require>d. FIONREAD ends up defined as a function in the
920I<sys/ioctl.ph> file:
68dc0745 921
5a964f20 922 require 'sys/ioctl.ph';
68dc0745 923
5a964f20
TC
924 $size = pack("L", 0);
925 ioctl(FH, FIONREAD(), $size) or die "Couldn't call ioctl: $!\n";
926 $size = unpack("L", $size);
68dc0745 927
5a964f20
TC
928If I<h2ph> wasn't installed or doesn't work for you, you can
929I<grep> the include files by hand:
68dc0745 930
5a964f20
TC
931 % grep FIONREAD /usr/include/*/*
932 /usr/include/asm/ioctls.h:#define FIONREAD 0x541B
68dc0745 933
5a964f20 934Or write a small C program using the editor of champions:
68dc0745 935
5a964f20
TC
936 % cat > fionread.c
937 #include <sys/ioctl.h>
938 main() {
939 printf("%#08x\n", FIONREAD);
940 }
941 ^D
65acb1b1 942 % cc -o fionread fionread.c
5a964f20
TC
943 % ./fionread
944 0x4004667f
945
8305e449 946And then hard code it, leaving porting as an exercise to your successor.
5a964f20
TC
947
948 $FIONREAD = 0x4004667f; # XXX: opsys dependent
949
950 $size = pack("L", 0);
951 ioctl(FH, $FIONREAD, $size) or die "Couldn't call ioctl: $!\n";
952 $size = unpack("L", $size);
953
a6dd486b 954FIONREAD requires a filehandle connected to a stream, meaning that sockets,
5a964f20 955pipes, and tty devices work, but I<not> files.
68dc0745
PP
956
957=head2 How do I do a C<tail -f> in perl?
958
959First try
960
961 seek(GWFILE, 0, 1);
962
963The statement C<seek(GWFILE, 0, 1)> doesn't change the current position,
964but it does clear the end-of-file condition on the handle, so that the
965next <GWFILE> makes Perl try again to read something.
966
967If that doesn't work (it relies on features of your stdio implementation),
968then you need something more like this:
969
970 for (;;) {
971 for ($curpos = tell(GWFILE); <GWFILE>; $curpos = tell(GWFILE)) {
972 # search for some stuff and put it into files
973 }
974 # sleep for a while
975 seek(GWFILE, $curpos, 0); # seek to where we had been
976 }
977
978If this still doesn't work, look into the POSIX module. POSIX defines
979the clearerr() method, which can remove the end of file condition on a
980filehandle. The method: read until end of file, clearerr(), read some
981more. Lather, rinse, repeat.
982
65acb1b1
TC
983There's also a File::Tail module from CPAN.
984
68dc0745
PP
985=head2 How do I dup() a filehandle in Perl?
986
987If you check L<perlfunc/open>, you'll see that several of the ways
988to call open() should do the trick. For example:
989
990 open(LOG, ">>/tmp/logfile");
991 open(STDERR, ">&LOG");
992
993Or even with a literal numeric descriptor:
994
995 $fd = $ENV{MHCONTEXTFD};
996 open(MHCONTEXT, "<&=$fd"); # like fdopen(3S)
997
c47ff5f1 998Note that "<&STDIN" makes a copy, but "<&=STDIN" make
5a964f20
TC
999an alias. That means if you close an aliased handle, all
1000aliases become inaccessible. This is not true with
1001a copied one.
1002
1003Error checking, as always, has been left as an exercise for the reader.
68dc0745
PP
1004
1005=head2 How do I close a file descriptor by number?
1006
1007This should rarely be necessary, as the Perl close() function is to be
1008used for things that Perl opened itself, even if it was a dup of a
a6dd486b 1009numeric descriptor as with MHCONTEXT above. But if you really have
68dc0745
PP
1010to, you may be able to do this:
1011
1012 require 'sys/syscall.ph';
1013 $rc = syscall(&SYS_close, $fd + 0); # must force numeric
1014 die "can't sysclose $fd: $!" unless $rc == -1;
1015
a6dd486b 1016Or, just use the fdopen(3S) feature of open():
d92eb7b0
GS
1017
1018 {
1019 local *F;
1020 open F, "<&=$fd" or die "Cannot reopen fd=$fd: $!";
1021 close F;
1022 }
1023
46fc3d4c 1024=head2 Why can't I use "C:\temp\foo" in DOS paths? What doesn't `C:\temp\foo.exe` work?
68dc0745
PP
1025
1026Whoops! You just put a tab and a formfeed into that filename!
1027Remember that within double quoted strings ("like\this"), the
1028backslash is an escape character. The full list of these is in
1029L<perlop/Quote and Quote-like Operators>. Unsurprisingly, you don't
1030have a file called "c:(tab)emp(formfeed)oo" or
65acb1b1 1031"c:(tab)emp(formfeed)oo.exe" on your legacy DOS filesystem.
68dc0745
PP
1032
1033Either single-quote your strings, or (preferably) use forward slashes.
46fc3d4c 1034Since all DOS and Windows versions since something like MS-DOS 2.0 or so
68dc0745 1035have treated C</> and C<\> the same in a path, you might as well use the
a6dd486b 1036one that doesn't clash with Perl--or the POSIX shell, ANSI C and C++,
65acb1b1
TC
1037awk, Tcl, Java, or Python, just to mention a few. POSIX paths
1038are more portable, too.
68dc0745
PP
1039
1040=head2 Why doesn't glob("*.*") get all the files?
1041
1042Because even on non-Unix ports, Perl's glob function follows standard
46fc3d4c 1043Unix globbing semantics. You'll need C<glob("*")> to get all (non-hidden)
65acb1b1
TC
1044files. This makes glob() portable even to legacy systems. Your
1045port may include proprietary globbing functions as well. Check its
1046documentation for details.
68dc0745
PP
1047
1048=head2 Why does Perl let me delete read-only files? Why does C<-i> clobber protected files? Isn't this a bug in Perl?
1049
06a5f41f
JH
1050This is elaborately and painstakingly described in the
1051F<file-dir-perms> article in the "Far More Than You Ever Wanted To
1052Know" collection in http://www.cpan.org/olddoc/FMTEYEWTK.tgz .
68dc0745
PP
1053
1054The executive summary: learn how your filesystem works. The
1055permissions on a file say what can happen to the data in that file.
1056The permissions on a directory say what can happen to the list of
1057files in that directory. If you delete a file, you're removing its
1058name from the directory (so the operation depends on the permissions
1059of the directory, not of the file). If you try to write to the file,
1060the permissions of the file govern whether you're allowed to.
1061
1062=head2 How do I select a random line from a file?
1063
1064Here's an algorithm from the Camel Book:
1065
1066 srand;
1067 rand($.) < 1 && ($line = $_) while <>;
1068
1069This has a significant advantage in space over reading the whole
5a964f20 1070file in. A simple proof by induction is available upon
a6dd486b 1071request if you doubt the algorithm's correctness.
68dc0745 1072
65acb1b1
TC
1073=head2 Why do I get weird spaces when I print an array of lines?
1074
1075Saying
1076
1077 print "@lines\n";
1078
1079joins together the elements of C<@lines> with a space between them.
1080If C<@lines> were C<("little", "fluffy", "clouds")> then the above
a6dd486b 1081statement would print
65acb1b1
TC
1082
1083 little fluffy clouds
1084
1085but if each element of C<@lines> was a line of text, ending a newline
1086character C<("little\n", "fluffy\n", "clouds\n")> then it would print:
1087
1088 little
1089 fluffy
1090 clouds
1091
1092If your array contains lines, just print them:
1093
1094 print @lines;
1095
68dc0745
PP
1096=head1 AUTHOR AND COPYRIGHT
1097
0bc0ad85 1098Copyright (c) 1997-2002 Tom Christiansen and Nathan Torkington.
5a964f20
TC
1099All rights reserved.
1100
5a7beb56
JH
1101This documentation is free; you can redistribute it and/or modify it
1102under the same terms as Perl itself.
c8db1d39 1103
87275199 1104Irrespective of its distribution, all code examples here are in the public
c8db1d39
TC
1105domain. You are permitted and encouraged to use this code and any
1106derivatives thereof in your own programs for fun or for profit as you
1107see fit. A simple comment in the code giving credit to the FAQ would
1108be courteous but is not required.