3 perlfaq9 - Web, Email and Networking
7 This section deals with questions related to running web sites,
8 sending and receiving email as well as general networking.
10 =head2 Should I use a web framework?
12 Yes. If you are building a web site with any level of interactivity
13 (forms / users / databases), you
14 will want to use a framework to make handling requests
17 If there is no interactivity then you may still want
18 to look at using something like L<Template Toolkit|https://metacpan.org/module/Template>
19 or L<Plack::Middleware::TemplateToolkit>
20 so maintenance of your HTML files (and other assets) is easier.
22 =head2 Which web framework should I use?
23 X<framework> X<CGI.pm> X<CGI> X<Catalyst> X<Dancer>
25 There is no simple answer to this question. Perl frameworks can run everything
26 from basic file servers and small scale intranets to massive multinational
27 multilingual websites that are the core to international businesses.
29 Below is a list of a few frameworks with comments which might help you in
30 making a decision, depending on your specific requirements. Start by reading
31 the docs, then ask questions on the relevant mailing list or IRC channel.
37 Strongly object-oriented and fully-featured with a long development history and
38 a large community and addon ecosystem. It is excellent for large and complex
39 applications, where you have full control over the server.
43 Young and free of legacy weight, providing a lightweight and easy to learn API.
44 Has a growing addon ecosystem. It is best used for smaller projects and
45 very easy to learn for beginners.
49 Fairly young with a focus on HTML5 and real-time web technologies such as
54 Currently experimental, strongly object-oriented, built for speed and intended
55 as a toolkit for building micro web apps, custom frameworks or for tieing
56 together existing Plack-compatible web applications with one central dispatcher.
60 All of these interact with or use L<Plack> which is worth understanding
61 the basics of when building a website in Perl (there is a lot of useful
62 L<Plack::Middleware|https://metacpan.org/search?q=plack%3A%3Amiddleware>).
64 =head2 What is Plack and PSGI?
66 L<PSGI> is the Perl Web Server Gateway Interface Specification, it is
67 a standard that many Perl web frameworks use, you should not need to
68 understand it to build a web site, the part you might want to use is L<Plack>.
70 L<Plack> is a set of tools for using the PSGI stack. It contains
71 L<middleware|https://metacpan.org/search?q=plack%3A%3Amiddleware>
72 components, a reference server and utilities for Web application frameworks.
73 Plack is like Ruby's Rack or Python's Paste for WSGI.
75 You could build a web site using L<Plack> and your own code,
76 but for anything other than a very basic web site, using a web framework
77 (that uses L<Plack>) is a better option.
79 =head2 How do I remove HTML from a string?
81 Use L<HTML::Strip>, or L<HTML::FormatText> which not only removes HTML
82 but also attempts to do a little simple formatting of the resulting
85 =head2 How do I extract URLs?
87 L<HTML::SimpleLinkExtor> will extract URLs from HTML, it handles anchors,
88 images, objects, frames, and many other tags that can contain a URL.
89 If you need anything more complex, you can create your own subclass of
90 L<HTML::LinkExtor> or L<HTML::Parser>. You might even use
91 L<HTML::SimpleLinkExtor> as an example for something specifically
94 You can use L<URI::Find> to extract URLs from an arbitrary text document.
96 =head2 How do I fetch an HTML file?
98 (contributed by brian d foy)
100 Use the libwww-perl distribution. The L<LWP::Simple> module can fetch web
101 resources and give their content back to you as a string:
103 use LWP::Simple qw(get);
105 my $html = get( "http://www.example.com/index.html" );
107 It can also store the resource directly in a file:
109 use LWP::Simple qw(getstore);
111 getstore( "http://www.example.com/index.html", "foo.html" );
113 If you need to do something more complicated, you can use
114 L<LWP::UserAgent> module to create your own user-agent (e.g. browser)
115 to get the job done. If you want to simulate an interactive web
116 browser, you can use the L<WWW::Mechanize> module.
118 =head2 How do I automate an HTML form submission?
120 If you are doing something complex, such as moving through many pages
121 and forms or a web site, you can use L<WWW::Mechanize>. See its
122 documentation for all the details.
124 If you're submitting values using the GET method, create a URL and encode
125 the form using the C<query_form> method:
130 my $url = url('L<http://www.perl.com/cgi-bin/cpan_mod')>;
131 $url->query_form(module => 'DB_File', readme => 1);
132 $content = get($url);
134 If you're using the POST method, create your own user agent and encode
135 the content appropriately.
137 use HTTP::Request::Common qw(POST);
140 my $ua = LWP::UserAgent->new();
141 my $req = POST 'L<http://www.perl.com/cgi-bin/cpan_mod'>,
142 [ module => 'DB_File', readme => 1 ];
143 my $content = $ua->request($req)->as_string;
145 =head2 How do I decode or create those %-encodings on the web?
146 X<URI> X<URI::Escape> X<RFC 2396>
148 Most of the time you should not need to do this as
149 your web framework, or if you are making a request,
150 the L<LWP> or other module would handle it for you.
152 To encode a string yourself, use the L<URI::Escape> module. The C<uri_escape>
153 function returns the escaped string:
155 my $original = "Colon : Hash # Percent %";
157 my $escaped = uri_escape( $original );
159 print "$escaped\n"; # 'Colon%20%3A%20Hash%20%23%20Percent%20%25'
161 To decode the string, use the C<uri_unescape> function:
163 my $unescaped = uri_unescape( $escaped );
165 print $unescaped; # back to original
167 Remember not to encode a full URI, you need to escape each
168 component separately and then join them together.
170 =head2 How do I redirect to another page?
172 Most Perl Web Frameworks will have a mechanism for doing this,
173 using the L<Catalyst> framework it would be:
175 $c->res->redirect($url);
178 If you are using Plack (which most frameworks do), then
179 L<Plack::Middleware::Rewrite> is worth looking at if you
180 are migrating from Apache or have URL's you want to always
183 =head2 How do I put a password on my web pages?
185 See if the web framework you are using has an
186 authentication system and if that fits your needs.
188 Alternativly look at L<Plack::Middleware::Auth::Basic>,
189 or one of the other L<Plack authentication|https://metacpan.org/search?q=plack+auth>
192 =head2 How do I make sure users can't enter values into a form that causes my CGI script to do bad things?
194 (contributed by brian d foy)
196 You can't prevent people from sending your script bad data. Even if
197 you add some client-side checks, people may disable them or bypass
198 them completely. For instance, someone might use a module such as
199 L<LWP> to submit to your web site. If you want to prevent data that
200 try to use SQL injection or other sorts of attacks (and you should
201 want to), you have to not trust any data that enter your program.
203 The L<perlsec> documentation has general advice about data security.
204 If you are using the L<DBI> module, use placeholder to fill in data.
205 If you are running external programs with C<system> or C<exec>, use
206 the list forms. There are many other precautions that you should take,
207 too many to list here, and most of them fall under the category of not
208 using any data that you don't intend to use. Trust no one.
210 =head2 How do I parse a mail header?
212 Use the L<Email::MIME> module. It's well-tested and supports all the
213 craziness that you'll see in the real world (comment-folding whitespace,
214 encodings, comments, etc.).
218 my $message = Email::MIME->new($rfc2822);
219 my $subject = $message->header('Subject');
220 my $from = $message->header('From');
222 If you've already got some other kind of email object, consider passing
223 it to L<Email::Abstract> and then using its cast method to get an
224 L<Email::MIME> object:
226 my $mail_message_object = read_message();
227 my $abstract = Email::Abstract->new($mail_message_object);
228 my $email_mime_object = $abstract->cast('Email::MIME');
230 =head2 How do I check a valid mail address?
232 (partly contributed by Aaron Sherman)
234 This isn't as simple a question as it sounds. There are two parts:
236 a) How do I verify that an email address is correctly formatted?
238 b) How do I verify that an email address targets a valid recipient?
240 Without sending mail to the address and seeing whether there's a human
241 on the other end to answer you, you cannot fully answer part I<b>, but
242 the L<Email::Valid> module will do both part I<a> and part I<b> as far
243 as you can in real-time.
245 Our best advice for verifying a person's mail address is to have them
246 enter their address twice, just as you normally do to change a
247 password. This usually weeds out typos. If both versions match, send
248 mail to that address with a personal message. If you get the message
249 back and they've followed your directions, you can be reasonably
250 assured that it's real.
252 A related strategy that's less open to forgery is to give them a PIN
253 (personal ID number). Record the address and PIN (best that it be a
254 random one) for later processing. In the mail you send, include a link to
255 your site with the PIN included. If the mail bounces, you know it's not
256 valid. If they don't click on the link, either they forged the address or
257 (assuming they got the message) following through wasn't important so you
258 don't need to worry about it.
260 =head2 How do I decode a MIME/BASE64 string?
262 The L<MIME::Base64> package handles this as well as the MIME/QP encoding.
263 Decoding base 64 becomes as simple as:
266 my $decoded = decode_base64($encoded);
268 The L<Email::MIME> module can decode base 64-encoded email message parts
269 transparently so the developer doesn't need to worry about it.
271 =head2 How do I find the user's mail address?
273 Ask them for it. There are so many email providers available that it's
274 unlikely the local system has any idea how to determine a user's email address.
276 The exception is for organization-specific email (e.g. foo@yourcompany.com)
277 where policy can be codified in your program. In that case, you could look at
278 $ENV{USER}, $ENV{LOGNAME}, and getpwuid($<) in scalar context, like so:
280 my $user_name = getpwuid($<)
282 But you still cannot make assumptions about whether this is correct, unless
283 your policy says it is. You really are best off asking the user.
285 =head2 How do I send email?
287 Use the L<Email::MIME> and L<Email::Sender::Simple> modules, like so:
289 # first, create your message
290 my $message = Email::MIME->create(
292 From => 'you@example.com',
293 To => 'friend@example.com',
294 Subject => 'Happy birthday!',
297 encoding => 'quoted-printable',
300 body_str => "Happy birthday to you!\n",
303 use Email::Sender::Simple qw(sendmail);
306 By default, L<Email::Sender::Simple> will try `sendmail` first, if it exists
307 in your $PATH. This generally isn't the case. If there's a remote mail
308 server you use to send mail, consider investigating one of the Transport
309 classes. At time of writing, the available transports include:
313 =item L<Email::Sender::Transport::Sendmail>
315 This is the default. If you can use the L<mail(1)> or L<mailx(1)>
316 program to send mail from the machine where your code runs, you should
319 =item L<Email::Sender::Transport::SMTP>
321 This transport contacts a remote SMTP server over TCP. It optionally
322 uses SSL and can authenticate to the server via SASL.
324 =item L<Email::Sender::Transport::SMTP::TLS>
326 This is like the SMTP transport, but uses TLS security. You can
327 authenticate with this module as well, using any mechanisms your server
328 supports after STARTTLS.
332 Telling L<Email::Sender::Simple> to use your transport is straightforward.
337 transport => $email_sender_transport_object,
341 =head2 How do I use MIME to make an attachment to a mail message?
343 L<Email::MIME> directly supports multipart messages. L<Email::MIME>
344 objects themselves are parts and can be attached to other L<Email::MIME>
345 objects. Consult the L<Email::MIME> documentation for more information,
346 including all of the supported methods and examples of their use.
348 =head2 How do I read email?
350 Use the L<Email::Folder> module, like so:
354 my $folder = Email::Folder->new('/path/to/email/folder');
355 while(my $message = $folder->next_message) {
356 # next_message returns Email::Simple objects, but we want
357 # Email::MIME objects as they're more robust
358 my $mime = Email::MIME->new($message->as_string);
361 There are different classes in the L<Email::Folder> namespace for
362 supporting various mailbox types. Note that these modules are generally
363 rather limited and only support B<reading> rather than writing.
365 =head2 How do I find out my hostname, domainname, or IP address?
366 X<hostname, domainname, IP address, host, domain, hostfqdn, inet_ntoa,
367 gethostbyname, Socket, Net::Domain, Sys::Hostname>
369 (contributed by brian d foy)
371 The L<Net::Domain> module, which is part of the Standard Library starting
372 in Perl 5.7.3, can get you the fully qualified domain name (FQDN), the host
373 name, or the domain name.
375 use Net::Domain qw(hostname hostfqdn hostdomain);
377 my $host = hostfqdn();
379 The L<Sys::Hostname> module, part of the Standard Library, can also get the
387 The L<Sys::Hostname::Long> module takes a different approach and tries
388 harder to return the fully qualified hostname:
390 use Sys::Hostname::Long 'hostname_long';
392 my $hostname = hostname_long();
394 To get the IP address, you can use the C<gethostbyname> built-in function
395 to turn the name into a number. To turn that number into the dotted octet
396 form (a.b.c.d) that most people expect, use the C<inet_ntoa> function
397 from the L<Socket> module, which also comes with perl.
401 my $address = inet_ntoa(
402 scalar gethostbyname( $host || 'localhost' )
405 =head2 How do I fetch/put an (S)FTP file?
407 L<Net::FTP>, and L<Net::SFTP> allow you to interact with FTP and SFTP (Secure
410 =head2 How can I do RPC in Perl?
412 Use one of the RPC modules( L<https://metacpan.org/search?q=RPC> ).
414 =head1 AUTHOR AND COPYRIGHT
416 Copyright (c) 1997-2010 Tom Christiansen, Nathan Torkington, and
417 other authors as noted. All rights reserved.
419 This documentation is free; you can redistribute it and/or modify it
420 under the same terms as Perl itself.
422 Irrespective of its distribution, all code examples in this file
423 are hereby placed into the public domain. You are permitted and
424 encouraged to use this code in your own programs for fun
425 or for profit as you see fit. A simple comment in the code giving
426 credit would be courteous but is not required.