This is Info file pm.info, produced by Makeinfo version 1.68 from the
input file bigpm.texi.
File: pm.info, Node: MIME/Lite/HTML, Next: MIME/Parser, Prev: MIME/Lite, Up: Module List
Provide routine to transform a HTML page in a MIME-Lite mail
************************************************************
NAME
====
MIME::Lite::HTML - Provide routine to transform a HTML page in a
MIME-Lite mail
SYNOPSIS
========
use MIME::Lite;
use MIME::Lite::HTML;
my $mailHTML = new MIME::Lite::HTML
From => 'MIME-Lite@alianwebserver.com',
To => 'alian@jupiter',
Subject => 'Mail in HTML with images';
$MIMEmail = $mailHTML->parse('http://www.alianwebserver.com');
$MIMEmail->send; # or for win user : $mail->send_by_smtp('smtp.fai.com');
DESCRIPTION
===========
This module provide routine to transform a HTML page in MIME::Lite mail.
So you need this module to use MIME-Lite-HTML possibilities
What's happen ?
---------------
The job done is:
* Get the file (LWP) if needed
* Parse page to find include images
* Attach them to mail with adequat cid
* Include external CSS,Javascript file
* Replace relative url with absolute one
Usage
-----
It can be used by example in a HTML newsletter. You make a classic HTML
page, and give just url to MIME::Lite::HTML.
Construction
------------
MIME-Lite-HTML use a MIME-Lite object, build with
--> multipart/alternative
------> text/plain if present
------> multipart/related
-------------> text/html
-------------> each images
Documentation
-------------
Additionnal documentation can be found here:
* MIME-lite module
* RFC 822, RFC 1521, RFC 1522 and specially RFC 2257 (MIME Encapsulation
of Aggregate Documents, such as HTML)
Clients tested
--------------
HTML in mail is not a standart, so this module can't work with all mail
clients.
Clients who has been tested:
* Outlook Express : 100% ok
* Netscape Messager : 100% ok
* Eudora : Two problems : Two additional headers will be displayed in
HTML part and Eudora didn't recognize multipart/alternative part as
describe in RFC 2257 so text and HTML part will be displayed both,
text part in first. Version 1.0 of this module correct major problem
of headers with image include in HTML part. May be a solution is not
use mail text with MIME-Lite-HTML, and use MIME-Lite for text mail.
Ask your users what they want ...
VERSION
=======
$Revision: 1.1 $
METHODS
=======
new(%hash)
----------
Create a new instance of MIME::Lite::HTML.
%hash can have this key : [Proxy], [Debug], [HashTemplate] Others keys
are use with MIME::Lite constructor.
This MIME-Lite keys are: Bcc, Encrypted, Received, Sender, Cc, From,
References, Subject, Comments, Keywords, Reply-To To, Content-*,
Message-ID,Resent-*, X-*,Date,MIME-Version,Return-Path, Organization
$hash{'HashTemplate'} is a reference to a hash. If present,
MIME::Lite::HTML will substitute with
$hash{'HashTemplate'}{'name'} when parse url to send.
$hash{'HashTemplate'} can be used too for include data for subelement. Ex:
$hash{'HashTemplate'}{'http://www.alianwebserver.com/images/sommaire.gif'}=\@data;
or
$hash{'HashTemplate'}{'http://www.alianwebserver.com/script.js'}="alert("Hello
world");";
When module find the image
http://www.alianwebserver.com/images/sommaire.gif in buffer, it don't get
image with LWP but use data found in $hash{'HashTemplate'}.
parse($html, [$url_txt], [$url_base])
-------------------------------------
Subroutine used for created HTML mail with MIME-Lite
Parameters:
$html
Url of HTML file to send, can be a local file. If $url is not an url
(http or https or ftp or file or nntp), $url is used as a buffer.
Example : http://www.alianwebserver.com, file://c|/tmp/index.html or
'
'.
$url_txt
Url of text part to send for person who doesn't support HTML mail.
As $html, $url_txt can be a simple buffer.
$url_base
$url_base is used if $html is a buffer, for get element found in HTML
buffer.
Return the MIME::Lite part to send
Private methods
===============
include_css($gabarit,$root)
---------------------------
(private)
Search in HTML buffer ($gabarit) to remplace call to extern CSS file
with his content. $root is original absolute url where css file will be
found.
include_javascript($gabarit,$root)
----------------------------------
(private)
Search in HTML buffer ($gabarit) to remplace call to extern javascript
file with his content. $root is original absolute url where javascript
file will be found.
input_image($gabarit,$root)
---------------------------
(private)
Search in HTML buffer ($gabarit) to remplace input form image with his
cid
Return final buffer and list of MIME::Lite part
link_form($gabarit,$root)
-------------------------
(private)
Replace link to formulaire with absolute link
fill_template($masque,$vars)
----------------------------
$masque : Path of template
$vars : hash ref with keys/val to substitue
Give template with remplaced variables Ex: if $$vars{age}=12, and
$masque have
J'ai ans,
this function give:
J'ai 12 ans,
Error Handling
==============
The set_err routine is used privately. You can ask for an array of all
the errors which occured inside the parse routine by calling:
@errors = $mailHTML->errstr;
If no errors where found, it'll return undef.
AUTHOR
======
Alain BARBET alian@alianwebserver.com
File: pm.info, Node: MIME/Parser, Next: MIME/Parser/Filer, Prev: MIME/Lite/HTML, Up: Module List
experimental class for parsing MIME streams
*******************************************
NAME
====
MIME::Parser - experimental class for parsing MIME streams
SYNOPSIS
========
Before reading further, you should see *Note MIME/Tools: MIME/Tools, to
make sure that you understand where this module fits into the grand scheme
of things. Go on, do it now. I'll wait.
Ready? Ok...
Basic usage examples
--------------------
### Create a new parser object:
my $parser = new MIME::Parser;
### Tell it where to put things:
$parser->output_under("/tmp");
### Parse an input filehandle:
$entity = $parser->parse(\*STDIN);
### Congratulations: you now have a (possibly multipart) MIME entity!
$entity->dump_skeleton; # for debugging
Examples of input
-----------------
### Parse from filehandles:
$entity = $parser->parse(\*STDIN);
$entity = $parser->parse(IO::File->new("some command|");
### Parse from any object that supports getline() and read():
$entity = $parser->parse($myHandle);
### Parse an in-core MIME message:
$entity = $parser->parse_data($message);
### Parse an MIME message in a file:
$entity = $parser->parse_open("/some/file.msg");
### Parse an MIME message out of a pipeline:
$entity = $parser->parse_open("gunzip - < file.msg.gz |");
### Parse already-split input (as "deliver" would give it to you):
$entity = $parser->parse_two("msg.head", "msg.body");
Examples of output control
--------------------------
### Keep parsed message bodies in core (default outputs to disk):
$parser->output_to_core(1);
### Output each message body to a one-per-message directory:
$parser->output_under("/tmp");
### Output each message body to the same directory:
$parser->output_dir("/tmp");
### Change how nameless message-component files are named:
$parser->output_prefix("msg");
Examples of error recovery
--------------------------
### Normal mechanism:
eval { $entity = $parser->parse(\*STDIN) };
if ($@) {
$results = $parser->results;
$decapitated = $parser->last_head; ### get last top-level head
}
### Ultra-tolerant mechanism:
$parser->ignore_errors(1);
$entity = eval { $parser->parse(\*STDIN) };
$error = ($@ || $parser->last_error);
### Cleanup all files created by the parse:
eval { $entity = $parser->parse(\*STDIN) };
...
$parser->filer->purge;
Examples of parser options
--------------------------
### Automatically attempt to RFC-1522-decode the MIME headers?
$parser->decode_headers(1); ### default is false
### Parse contained "message/rfc822" objects as nested MIME streams?
$parser->extract_nested_messages(0); ### default is true
### Look for uuencode in "text" messages, and extract it?
$parser->extract_uuencode(1); ### default is false
### Should we forgive normally-fatal errors?
$parser->ignore_errors(0); ### default is true
Miscellaneous examples
----------------------
### Convert a Mail::Internet object to a MIME::Entity:
@lines = (@{$mail->header}, "\n", @{$mail->body});
$entity = $parser->parse_data(\@lines);
DESCRIPTION
===========
You can inherit from this class to create your own subclasses that
parse MIME streams into MIME::Entity objects.
PUBLIC INTERFACE
================
Construction
------------
new ARGS...
*Class method.* Create a new parser object. Once you do this, you
can then set up various parameters before doing the actual parsing.
For example:
my $parser = new MIME::Parser;
$parser->output_dir("/tmp");
$parser->output_prefix("msg1");
my $entity = $parser->parse(\*STDIN);
Any arguments are passed into `init()'. Don't override this in your
subclasses; override init() instead.
init ARGS...
*Instance method.* Initiallize a new MIME::Parser object. This is
automatically sent to a new object; you may want to override it. If
you override this, be sure to invoke the inherited method.
init_parse
*Instance method.* Invoked automatically whenever one of the
top-level parse() methods is called, to reset the parser to a "ready"
state.
Altering how messages are parsed
--------------------------------
decode_headers [YESNO]
*Instance method.* Controls whether the parser will attempt to decode
all the MIME headers (as per RFC-1522) the moment it sees them.
*This is not advisable for two very important reasons:*
* *It screws up the extraction of information from MIME fields.*
If you fully decode the headers into bytes, you can inadvertently
transform a parseable MIME header like this:
Content-type: text/plain; filename="=?ISO-8859-1?Q?Hi=22Ho?="
into unparseable gobbledygook; in this case:
Content-type: text/plain; filename="Hi"Ho"
* *It is information-lossy.* An encoded string which contains
both Latin-1 and Cyrillic characters will be turned into a binary
mishmosh which simply can't be rendered.
*History.* This method was once the only out-of-the-box way to deal
with attachments whose filenames had non-ASCII characters. However,
since MIME-tools 5.4xx this is no longer necessary.
*Parameters.* If YESNO is true, decoding is done. However, you will
get a warning unless you use one of the special "true" values:
"I_NEED_TO_FIX_THIS"
Just shut up and do it. Not recommended.
Provided only for those who need to keep old scripts functioning.
"I_KNOW_WHAT_I_AM_DOING"
Just shut up and do it. Not recommended.
Provided for those who REALLY know what they are doing.
If YESNO is false (the default), no attempt at decoding will be done.
With no argument, just returns the current setting. *Remember:* you
can always decode the headers after the parsing has completed (see
`MIME::Head::decode()|MIME::Head' in this node), or decode the words
on demand (see *Note MIME/Words: MIME/Words,).
extract_nested_messages OPTION
*Instance method.* Some MIME messages will contain a part of type
`message/rfc822': literally, the text of an embedded
mail/news/whatever message. This option controls whether (and how)
we parse that embedded message.
If the OPTION is false, we treat such a message just as if it were a
`text/plain' document, without attempting to decode its contents.
If the OPTION is true (the default), the body of the `message/rfc822'
part is parsed by this parser, creating an entity object. What
happens then is determined by the actual OPTION:
NEST or 1
The default setting. The contained message becomes the sole
"part" of the `message/rfc822' entity (as if the containing
message were a special kind of "multipart" message). You can
recover the sub-entity by invoking the `parts()|MIME::Entity' in
this node method on the `message/rfc822' entity.
REPLACE
The contained message replaces the `message/rfc822' entity, as
though the `message/rfc822' "container" never existed.
Warning: notice that, with this option, all the header
information in the `message/rfc822' header is lost. This might
seriously bother you if you're dealing with a top-level message,
and you've just lost the sender's address and the subject line.
`:-/'.
*Thanks to Andreas Koenig for suggesting this method.*
extract_uuencode [YESNO]
*Instance method.* If set true, then whenever we are confronted with
a message whose effective content-type is "text/plain" and whose
encoding is 7bit/8bit/binary, we scan the encoded body to see if it
contains uuencoded data (generally given away by a "begin XXX" line).
If it does, we explode the uuencoded message into a multipart, where
the text before the first "begin XXX" becomes the first part, and all
"begin...end" sections following become the subsequent parts. The
filename (if given) is accessible through the normal means.
ignore_errors [YESNO]
*Instance method.* Controls whether the parser will attempt to ignore
normally-fatal errors, treating them as warnings and continuing with
the parse.
If YESNO is true (the default), many syntax errors are tolerated. If
YESNO is false, fatal errors throw exceptions. With no argument,
just returns the current setting.
Parsing an input source
-----------------------
parse_data DATA
*Instance method.* Parse a MIME message that's already in core. You
may supply the DATA in any of a number of ways...
* *A scalar* which holds the message.
* *A ref to a scalar* which holds the message. This is an
efficiency hack.
* *A ref to an array of scalars.* They are treated as a stream
which (conceptually) consists of simply concatenating the
scalars.
Returns the parsed MIME::Entity on success. Throws exception on
failure.
parse INSTREAM
*Instance method.* Takes a MIME-stream and splits it into its
component entities.
The INSTREAM can be given as a readable FileHandle, an IO::File, a
globref filehandle (like `\*STDIN'), or as any blessed object
conforming to the IO:: interface (which minimally implements
getline() and read()).
Returns the parsed MIME::Entity on success. Throws exception on
failure.
parse_open EXPR
*Instance method.* Convenience front-end onto parse(). Simply give
this method any expression that may be sent as the second argument to
open() to open a filehandle for reading.
Returns the parsed MIME::Entity on success. Throws exception on
failure.
parse_two HEADFILE, BODYFILE
*Instance method.* Convenience front-end onto `parse_open()',
intended for programs running under mail-handlers like *deliver*,
which splits the incoming mail message into a header file and a body
file. Simply give this method the paths to the respective files.
Warning: it is assumed that, once the files are cat'ed together,
there will be a blank line separating the head part and the body part.
Warning: new implementation slurps files into line array for
portability, instead of using 'cat'. May be an issue if your
messages are large.
Returns the parsed MIME::Entity on success. Throws exception on
failure.
Specifying output destination
-----------------------------
Warning: in 5.212 and before, this was done by methods of MIME::Parser.
However, since many users have requested fine-tuned control over how this
is done, the logic has been split off from the parser into its own class,
MIME::Parser::Filer Every MIME::Parser maintains an instance of a
MIME::Parser::Filer subclass to manage disk output (see *Note
MIME/Parser/Filer: MIME/Parser/Filer, for details.)
The benefit to this is that the MIME::Parser code won't be confounded
with a lot of garbage related to disk output. The drawback is that the
way you override the default behavior will change.
For now, all the normal public-interface methods are still provided,
but many are only stubs which create or delegate to the underlying
MIME::Parser::Filer object.
filer [FILER]
*Instance method.* Get/set the FILER object used to manage the output
of files to disk. This will be some subclass of
`MIME::Parser::Filer|MIME::Parser::Filer' in this node.
output_dir DIRECTORY
*Instance method.* Causes messages to be filed directly into the
given DIRECTORY. It does this by setting the underlying `filer()|'
in this node to a new instance of MIME::Parser::FileInto, and passing
the arguments into that class' new() method.
Note: Since this method replaces the underlying filer, you must
invoke it before doing changing any attributes of the filer, like the
output prefix; otherwise those changes will be lost.
output_under BASEDIR, OPTS...
*Instance method.* Causes messages to be filed directly into
subdirectories of the given BASEDIR, one subdirectory per message.
It does this by setting the underlying `filer()|' in this node to a
new instance of MIME::Parser::FileUnder, and passing the arguments
into that class' new() method.
Note: Since this method replaces the underlying filer, you must
invoke it before doing changing any attributes of the filer, like the
output prefix; otherwise those changes will be lost.
output_path HEAD
*Instance method, DEPRECATED.* Given a MIME head for a file to be
extracted, come up with a good output pathname for the extracted file.
Identical to the preferred form:
$parser->filer->output_path(...args...);
We just delegate this to the underlying `filer()|' in this node
object.
output_prefix [PREFIX]
*Instance method, DEPRECATED.* Get/set the short string that all
filenames for extracted body-parts will begin with (assuming that
there is no better "recommended filename"). Identical to the
preferred form:
$parser->filer->output_prefix(...args...);
We just delegate this to the underlying `filer()|' in this node
object.
evil_filename NAME
*Instance method, DEPRECATED.* Identical to the preferred form:
$parser->filer->evil_filename(...args...);
We just delegate this to the underlying `filer()|' in this node
object.
output_to_core YESNO
*Instance method.* Normally, instances of this class output all their
decoded body data to disk files (via MIME::Body::File). However, you
can change this behaviour by invoking this method before parsing:
If YESNO is false (the default), then all body data goes to disk
files.
If YESNO is true, then all body data goes to in-core data structures
This is a little risky (what if someone emails you an MPEG or a tar
file, hmmm?) but people seem to want this bit of noose-shaped rope,
so I'm providing it. Note that setting this attribute true *does
not* mean that parser-internal temporary files are avoided! Use
`tmp_to_core()|' in this node for that.
With no argument, returns the current setting as a boolean.
tmp_recycling [YESNO]
*Instance method.* Normally, tmpfiles are created when needed during
parsing, and destroyed automatically when they go out of scope. But
for efficiency, you might prefer for your parser to attempt to rewind
and reuse the same file until the parser itself is destroyed.
If YESNO is true (the default), we allow recycling; tmpfiles persist
until the parser itself is destroyed. If YESNO is false, we do not
allow recycling; tmpfiles persist only as long as they are needed
during the parse. With no argument, just returns the current setting.
tmp_to_core [YESNO]
*Instance method.* Should `new_tmpfile()|' in this node create real
temp files, or use fake in-core ones? Normally we allow the creation
of temporary disk files, since this allows us to handle huge
attachments even when core is limited.
If YESNO is true, we implement new_tmpfile() via in-core handles. If
YESNO is false (the default), we use real tmpfiles. With no
argument, just returns the current setting.
use_inner_files [YESNO]
*Instance method.* If you are parsing from a handle which supports
seek() and tell(), then we can avoid tmpfiles completely by using
IO::InnerFile, if so desired: basically, we simulate a temporary file
via pointers to virtual start- and end-positions in the input stream.
If YESNO is false (the default), then we will not use IO::InnerFile.
If YESNO is true, we use IO::InnerFile if we can. With no argument,
just returns the current setting.
Note: inner files are slower than real tmpfiles, but possibly faster
than *in-core* tmpfiles... so your choice for this option will
probably depend on your choice for `tmp_to_core()|' in this node and
the kind of input streams you are parsing.
Specifying classes to be instantiated
-------------------------------------
interface ROLE,[VALUE]
*Instance method.* During parsing, the parser normally creates
instances of certain classes, like MIME::Entity. However, you may
want to create a parser subclass that uses your own experimental
head, entity, etc. classes (for example, your "head" class may
provide some additional MIME-field-oriented methods).
If so, then this is the method that your subclass should invoke during
init. Use it like this:
package MyParser;
@ISA = qw(MIME::Parser);
...
sub init {
my $self = shift;
$self->SUPER::init(@_); ### do my parent's init
$self->interface(ENTITY_CLASS => 'MIME::MyEntity');
$self->interface(HEAD_CLASS => 'MIME::MyHead');
$self; ### return
}
With no VALUE, returns the VALUE currently associated with that ROLE.
new_body_for HEAD
*Instance method.* Based on the HEAD of a part we are parsing, return
a new body object (any desirable subclass of MIME::Body) for
receiving that part's data.
If you set the output_to_core option to false before parsing (the
default), then we call `output_path()' and create a new
MIME::Body::File on that filename.
If you set the output_to_core option to true before parsing, then you
get a MIME::Body::InCore instead.
If you want the parser to do something else entirely, you can
override this method in a subclass.
new_tmpfile [RECYCLE]
*Instance method.* Return an IO handle to be used to hold temporary
data during a parse. The default uses the standard
IO::File->new_tmpfile() method unless `tmp_to_core()|' in this node
dictates otherwise, but you can override this. You shouldn't need to.
If you do override this, make certain that the object you return is
set for binmode(), and is able to handle the following methods:
read(BUF, NBYTES)
getline()
getlines()
print(@ARGS)
flush()
seek(0, 0)
Fatal exception if the stream could not be established.
If RECYCLE is given, it is an object returned by a previous invocation
of this method; to recycle it, this method must effectively rewind and
truncate it, and return the same object. If you don't want to support
recycling, just ignore it and always return a new object.
Parse results and error recovery
--------------------------------
last_error
*Instance method.* Return the error (if any) that we ignored in the
last parse.
last_head
*Instance method.* Return the top-level MIME header of the last
stream we attempted to parse. This is useful for replying to people
who sent us bad MIME messages.
### Parse an input stream:
eval { $entity = $parser->parse(\*STDIN) };
if (!$entity) { ### parse failed!
my $decapitated = $parser->last_head;
...
}
results
*Instance method.* Return an object containing lots of info from the
last entity parsed. This will be an instance of class
`MIME::Parser::Results|MIME::Parser::Results' in this node.
OPTIMIZING YOUR PARSER
======================
Maximizing speed
----------------
Optimum input mechanisms:
parse() YES (if you give it a globref or a
subclass of IO::File)
parse_open() YES
parse_data() NO (see below)
parse_two() NO (see below)
Optimum settings:
decode_headers() *** (no real difference; 0 is slightly faster)
extract_nested_messages() 0 (may be slightly faster, but in
general you want it set to 1)
output_to_core() 0 (will be MUCH faster)
tmp_recycling() 1? (probably, but should be investigated)
tmp_to_core() 0 (will be MUCH faster)
use_inner_files() 0 (if tmp_to_core() is 0;
use 1 otherwise)
*File I/O is much faster than in-core I/O.* Although it *seems* like
slurping a message into core and processing it in-core should be faster...
it isn't. Reason: Perl's filehandle-based I/O translates directly into
native operating-system calls, whereas the in-core I/O is implemented in
Perl.
*Inner files are slower than real tmpfiles, but faster than in-core
ones.* If speed is your concern, that's why you should set
use_inner_files(true) if you set tmp_to_core(true): so that we can bypass
the slow in-core tmpfiles if the input stream permits.
*Native I/O is much faster than object-oriented I/O.* It's much faster
to use <$foo> than $foo->getline. For backwards compatibilty, this module
must continue to use object-oriented I/O in most places, but if you use
`parse()|' in this node with a "real" filehandle (string, globref, or
subclass of IO::File) then MIME::Parser is able to perform some crucial
optimizations.
*The parse_two() call is very inefficient.* Currently this is just a
front-end onto parse_data(). If your OS supports it, you're *far* better
off doing something like:
$parser->parse_open("/bin/cat msg.head msg.body |");
Minimizing memory
-----------------
Optimum input mechanisms:
parse() YES
parse_open() YES
parse_data() NO (in-core I/O will burn core)
parse_two() NO (in-core I/O will burn core)
Optimum settings:
decode_headers() *** (no real difference)
extract_nested_messages() *** (no real difference)
output_to_core() 0 (will use MUCH less memory)
tmp_recycling() 0? (promotes faster GC if
tmp_to_core is 1)
tmp_to_core() 0 (will use MUCH less memory)
use_inner_files() *** (no real difference, but set it to 1
if you *must* have tmp_to_core set to 1,
so that you avoid in-core tmpfiles)
Maximizing tolerance of bad MIME
--------------------------------
Optimum input mechanisms:
parse() *** (doesn't matter)
parse_open() *** (doesn't matter)
parse_data() *** (doesn't matter)
parse_two() *** (doesn't matter)
Optimum settings:
decode_headers() 0 (sidesteps problem of bad hdr encodings)
extract_nested_messages() 0 (sidesteps problems of bad nested messages,
but often you want it set to 1 anyway).
output_to_core() *** (doesn't matter)
tmp_recycling() *** (doesn't matter)
tmp_to_core() *** (doesn't matter)
use_inner_files() *** (doesn't matter)
Avoiding disk-based temporary files
-----------------------------------
Optimum input mechanisms:
parse() YES (if you give it a seekable handle)
parse_open() YES (becomes a seekable handle)
parse_data() NO (unless you set tmp_to_core(1))
parse_two() NO (unless you set tmp_to_core(1))
Optimum settings:
decode_headers() *** (doesn't matter)
extract_nested_messages() *** (doesn't matter)
output_to_core() *** (doesn't matter)
tmp_recycling 1 (restricts created files to 1 per parser)
tmp_to_core() 1
use_inner_files() 1
*If we can use them, inner files avoid most tmpfiles.* If you parse
from a seekable-and-tellable filehandle, then the internal
process_to_bound() doesn't need to extract each part into a temporary
buffer; it can use IO::InnerFile (*warning:* this will slow down the
parsing of messages with large attachments).
*You can veto tmpfiles entirely.* If you might not be parsing from a
seekable-and-tellable filehandle, you can set `tmp_to_core()|' in this
node true: this will always use in-core I/O for the buffering (*warning:*
this will slow down the parsing of messages with large attachments).
*Final resort.* You can always override `new_tmpfile()|' in this node
in a subclass.
WARNINGS
========
Multipart messages are always read line-by-line
Multipart document parts are read line-by-line, so that the
encapsulation boundaries may easily be detected. However, bad MIME
composition agents (for example, naive CGI scripts) might return
multipart documents where the parts are, say, unencoded bitmap
files... and, consequently, where such "lines" might be veeeeeeeeery
long indeed.
A better solution for this case would be to set up some form of state
machine for input processing. This will be left for future versions.
Multipart parts read into temp files before decoding
In my original implementation, the MIME::Decoder classes had to be
aware of encapsulation boundaries in multipart MIME documents. While
this decode-while-parsing approach obviated the need for temporary
files, it resulted in inflexible and complex decoder implementations.
The revised implementation uses a temporary file (a la tmpfile())
during parsing to hold the *encoded* portion of the current MIME
document or part. This file is deleted automatically after the
current part is decoded and the data is written to the "body stream"
object; you'll never see it, and should never need to worry about it.
Some folks have asked for the ability to bypass this temp-file
mechanism, I suppose because they assume it would slow down their
application. I considered accomodating this wish, but the temp-file
approach solves a lot of thorny problems in parsing, and it also
protects against hidden bugs in user applications (what if you've
directed the encoded part into a scalar, and someone unexpectedly
sends you a 6 MB tar file?). Finally, I'm just not conviced that the
temp-file use adds significant overhead.
Fuzzing of CRLF and newline on input
RFC-1521 dictates that MIME streams have lines terminated by CRLF
(`"\r\n"'). However, it is extremely likely that folks will want to
parse MIME streams where each line ends in the local newline
character `"\n"' instead.
An attempt has been made to allow the parser to handle both CRLF and
newline-terminated input.
Fuzzing of CRLF and newline on output
The `"7bit"' and `"8bit"' decoders will decode both a `"\n"' and a
`"\r\n"' end-of-line sequence into a `"\n"'.
The `"binary"' decoder (default if no encoding specified) still
outputs stuff verbatim... so a MIME message with CRLFs and no
explicit encoding will be output as a text file that, on many
systems, will have an annoying ^M at the end of each line... *but
this is as it should be*.
Inability to handle multipart boundaries that contain newlines
First, let's get something straight: *this is an evil, EVIL practice,*
and is incompatible with RFC-1521... hence, it's not valid MIME.
If your mailer creates multipart boundary strings that contain
newlines *when they appear in the message body,* give it two weeks
notice and find another one. If your mail robot receives MIME mail
like this, regard it as syntactically incorrect MIME, which it is.
Why do I say that? Well, in RFC-1521, the syntax of a boundary is
given quite clearly:
boundary := 0*69 bcharsnospace
bchars := bcharsnospace / " "
bcharsnospace := DIGIT / ALPHA / "'" / "(" / ")" / "+" /"_"
/ "," / "-" / "." / "/" / ":" / "=" / "?"
All of which means that a valid boundary string *cannot* have
newlines in it, and any newlines in such a string in the message
header are expected to be solely the result of *folding* the string
(i.e., inserting to-be-removed newlines for readability and
line-shortening *only*).
Yet, there is at least one brain-damaged user agent out there that
composes mail like this:
MIME-Version: 1.0
Content-type: multipart/mixed; boundary="----ABC-
123----"
Subject: Hi... I'm a dork!
This is a multipart MIME message (yeah, right...)
----ABC-
123----
Hi there!
We have *got* to discourage practices like this (and the recent file
upload idiocy where binary files that are part of a multipart MIME
message aren't base64-encoded) if we want MIME to stay relatively
simple, and MIME parsers to be relatively robust.
*Thanks to Andreas Koenig for bringing a baaaaaaaaad user agent to my
attention.*
AUTHOR
======
Eryq (`eryq@zeegee.com'), ZeeGee Software Inc (`http://www.zeegee.com').
All rights reserved. This program is free software; you can
redistribute it and/or modify it under the same terms as Perl itself.
VERSION
=======
$Revision: 5.406 $ $Date: 2000/11/12 05:55:11 $
File: pm.info, Node: MIME/Parser/Filer, Next: MIME/Parser/Reader, Prev: MIME/Parser, Up: Module List
manage file-output of the parser
********************************
NAME
====
MIME::Parser::Filer - manage file-output of the parser
SYNOPSIS
========
Before reading further, you should see *Note MIME/Parser: MIME/Parser,
to make sure that you understand where this module fits into the grand
scheme of things. Go on, do it now. I'll wait.
Ready? Ok... now read `"DESCRIPTION"' in this node below, and
everything else should make sense.
Public interface
----------------
### Create a "filer" of the desired class:
my $filer = MIME::Parser::FileInto->new($dir);
my $filer = MIME::Parser::FileUnder->new($basedir);
...
### Want added security? Don't let outsiders name your files:
$filer->ignore_filename(1);
### Prepare for the parsing of a new top-level message:
$filer->init_parse;
### Return the path where this message's data should be placed:
$path = $filer->output_path($head);
Semi-public interface
---------------------
These methods might be overriden or ignored in some subclasses, so they
don't all make sense in all circumstances:
### Tweak the mapping from content-type to extension:
$emap = $filer->output_extension_map;
$emap->{"text/html"} = ".htm";
DESCRIPTION
===========
How this class is used when parsing
-----------------------------------
When a MIME::Parser decides that it wants to output a file to disk, it
uses its "Filer" object - an instance of a MIME::Parser::Filer subclass -
to determine where to put the file.
Every parser has a single Filer object, which it uses for all parsing.
You can get the Filer for a given $parser like this:
$filer = $parser->filer;
At the beginning of each parse(), the filer's internal state is reset
by the parser:
$parser->filer->init_parse;
The parser can then get a path for each entity in the message by
handing that entity's header (a MIME::Head) to the filer and having it do
the work, like this:
$new_file = $parser->filer->output_path($head);
Since it's nice to be able to clean up after a parse (especially a
failed parse), the parser tells the filer when it has actually used a path:
$parser->filer->purgeable($new_file);
Then, if you want to clean up the files which were created for a
particular parse (and also any directories that the Filer created), you
would do this:
$parser->filer->purge;
Writing your own subclasses
---------------------------
There are two standard "Filer" subclasses (see below):
MIME::Parser::FileInto, which throws all files from all parses into the
same directory, and MIME::Parser::FileUnder (preferred), which creates a
subdirectory for each message. Hopefully, these will be sufficient for
most uses, but just in case...
The only method you have to override is `output_path()|' in this node:
$filer->output_path($head);
This method is invoked by MIME::Parser when it wants to put a decoded
message body in an output file. The method should return a path to the
file to create. Failure is indicated by throwing an exception.
The path returned by `output_path()' should be "ready for open()": any
necessary parent directories need to exist at that point. These
directories can be created by the Filer, if course, and they should be
marked as *purgeable()* if a purge should delete them.
Actually, if your issue is more where the files go than what they're
named, you can use the default `output_path()|' in this node method and
just override one of its components:
$dir = $filer->output_dir($head);
$name = $filer->output_filename($head);
...
PUBLIC INTERFACE
================
MIME::Parser::Filer
-------------------
This is the abstract superclass of all "filer" objects.
new INITARGS...
*Class method, constructor.* Create a new outputter for the given
parser. Any subsequent arguments are given to init(), which
subclasses should override for their own use (the default init does
nothing).
results RESULTS
*Instance method.* Link this filer to a MIME::Parser::Results object
which will tally the messages. Notice that we avoid linking it to the
parser to avoid circular reference!
init_parse
*Instance method.* Prepare to start parsing a new message.
Subclasses should always be sure to invoke the inherited method.
evil_filename FILENAME
*Instance method.* Is this an evil filename; i.e., one which should
not be used in generating a disk file name? It is if any of these
are true:
* it is empty
* it is a string of dots: ".", "..", etc.
* it contains a known "path" character: '/' '\' ':' '[' ']'
* it is too long
If you just want to change this behavior, you should override this
method in the subclass of MIME::Parser::Filer that you use.
Warning: at the time this method is invoked, the FILENAME has already
been unmime'd into the local character set. If you're using any
character set other than ASCII, ISO-8859-*, or UTF-8, the
interpretation of the "path" characters might be very different, and
you will probably need to override this method. See `unmime', *Note
MIME/WordDecoder: MIME/WordDecoder, for more details.
Note: subclasses of MIME::Parser::Filer which override output_path()
might not consult this method; note, however, that the built-in
subclasses do consult it.
*Thanks to Andrew Pimlott for finding a real dumb bug in the original
version. Thanks to Nickolay Saukh for noting that evil is in the eye
of the beholder.*
exorcise_filename FILENAME
*Instance method.* If a given filename is evil (see `' in this node)
we try to rescue it by performing some basic operations: shortening
it, removing bad characters, etc., and checking each against
evil_filename().
Returns the exorcised filename (which is guaranteed to not be evil),
or undef if it could not be salvaged.
Warning: at the time this method is invoked, the FILENAME has already
been unmime'd into the local character set. If you're using anything
character set other than ASCII, ISO-8859-*, or UTF-8, the
interpretation of the "path" characters might be very very different,
and you will probably need to override this method. See `unmime',
*Note MIME/WordDecoder: MIME/WordDecoder, for more details.
find_unused_path DIR, FILENAME
*Instance method, subclasses only.* We have decided on an output
directory and tentative filename, but there is a chance that it might
already exist. Keep adding a numeric suffix "-1", "-2", etc. to the
filename until an unused path is found, and then return that path.
The suffix is actually added before the first "." in the filename is
there is one; for example:
picture.gif archive.tar.gz readme
picture-1.gif archive-1.tar.gz readme-1
picture-2.gif archive-2.tar.gz readme-2
... ... ...
picture-10.gif
...
This can be a costly operation, and risky if you don't want files
renamed, so it is in your best interest to minimize situations where
these kinds of collisions occur. Unfortunately, if a multipart
message gives all of its parts the same recommended filename, and you
are placing them all in the same directory, this method might be
unavoidable.
ignore_filename [YESNO]
*Instance method.* Return true if we should always ignore recommended
filenames in messages, choosing instead to always generate our own
filenames. With argument, sets this value.
Note: subclasses of MIME::Parser::Filer which override output_path()
might not honor this setting; note, however, that the built-in
subclasses honor it.
output_dir HEAD
*Instance method.* Return the output directory for the given header.
The default method returns ".".
output_filename HEAD
*Instance method, subclasses only.* A given recommended filename was
either not given, or it was judged to be evil. Return a fake name,
possibly using information in the message HEADer. Note that this is
just the filename, not the full path.
Used by `output_path()|' in this node. If you're using the default
`output_path()', you probably don't need to worry about avoiding
collisions with existing files; we take care of that in
`find_unused_path()|' in this node.
output_prefix [PREFIX]
*Instance method.* Get the short string that all filenames for
extracted body-parts will begin with (assuming that there is no
better "recommended filename"). The default is `"msg"'.
If PREFIX *is not* given, the current output prefix is returned. If
PREFIX *is* given, the output prefix is set to the new value, and the
previous value is returned.
Used by `output_filename()|' in this node.
Note: subclasses of MIME::Parser::Filer which override output_path()
or output_filename() might not honor this setting; note, however,
that the built-in subclasses honor it.
output_type_ext
*Instance method.* Return a reference to the hash used by the default
`output_filename()|' in this node for mapping from content-types to
extensions when there is no default extension to use.
$emap = $filer->output_typemap;
$emap->{'text/plain'} = '.txt';
$emap->{'text/html'} = '.html';
$emap->{'text/*'} = '.txt';
$emap->{'*/*'} = '.dat';
Note: subclasses of MIME::Parser::Filer which override output_path()
or output_filename() might not consult this hash; note, however, that
the built-in subclasses consult it.
output_path HEAD
*Instance method, subclasses only.* Given a MIME head for a file to
be extracted, come up with a good output pathname for the extracted
file. This is the only method you need to worry about if you are
building a custom filer.
The default implementation does a lot of work; subclass implementers
*really* should try to just override its components instead of the
whole thing. It works basically as follows:
$directory = $self->output_dir($head);
$filename = $head->recommended_filename();
if (!$filename or
$self->ignore_filename() or
$self->evil_filename($filename)) {
$filename = $self->output_filename($head);
}
return $self->find_unused_path($directory, $filename);
Note: There are many, many, many ways you might want to control the
naming of files, based on your application. If you don't like the
behavior of this function, you can easily define your own subclass of
MIME::Parser::Filer and override it there.
Note: Nickolay Saukh pointed out that, given the subjective nature of
what is "evil", this function really shouldn't warn about an evil
filename, but maybe just issue a debug message. I considered that,
but then I thought: if debugging were off, people wouldn't know why
(or even if) a given filename had been ignored. In mail robots that
depend on externally-provided filenames, this could cause
hard-to-diagnose problems. So, the message is still a warning.
*Thanks to Laurent Amon for pointing out problems with the original
implementation, and for making some good suggestions. Thanks also to
Achim Bohnet for pointing out that there should be a hookless, OO way
of overriding the output path.*
purge
*Instance method, final.* Purge all files/directories created by the
last parse. This method simply goes through the purgeable list in
reverse order (see `' in this node) and removes all existing
files/directories in it. You should not need to override this method.
purgeable [FILE]
*Instance method, final.* Add FILE to the list of "purgeable"
files/directories (those which will be removed if you do a purge()).
You should not need to override this method.
If FILE is not given, the "purgeable" list is returned. This may be
used for more-sophisticated purging.
As a special case, invoking this method with a FILE that is an
arrayref will replace the purgeable list with a copy of the array's
contents, so [] may be used to clear the list.
Note that the "purgeable" list is cleared when a parser begins a new
parse; therefore, if you want to use purge() to do cleanup, you must
do so before starting a new parse!
MIME::Parser::FileInto
----------------------
This concrete subclass of MIME::Parser::Filer supports filing into a
given directory.
init DIRECTORY
*Instance method, initiallizer.* Set the directory where all files
will go.
MIME::Parser::FileUnder
-----------------------
This concrete subclass of MIME::Parser::Filer supports filing under a
given directory, using one subdirectory per message, but with all message
parts in the same directory.
init BASEDIR, OPTSHASH...
*Instance method, initiallizer.* Set the base directory which will
contain the message directories. If used, then each parse of begins
by creating a new subdirectory of BASEDIR where the actual parts of
the message are placed. OPTSHASH can contain the following:
DirName
Explicitly set the name of the subdirectory which is created.
The default is to use the time, process id, and a sequence
number, but you might want a predictable directory.
Purge
Automatically purge the contents of the directory (including all
subdirectories) before each parse. This is really only needed if
using an explicit DirName, and is provided as a convenience only.
Currently we use the 1-arg form of File::Path::rmtree; you should
familiarize yourself with the caveats therein.
The output_dir() will return the path to this message-specific
directory until the next parse is begun, so you can do this:
use File::Path;
$parser->output_under("/tmp");
$ent = eval { $parser->parse_open($msg); }; ### parse
if (!$ent) { ### parse failed
rmtree($parser->output_dir);
die "parse failed: $@";
}
else { ### parse succeeded
...do stuff...
}
AUTHOR
======
Eryq (`eryq@zeegee.com'), ZeeGee Software Inc (`http://www.zeegee.com').
All rights reserved. This program is free software; you can
redistribute it and/or modify it under the same terms as Perl itself.
VERSION
=======
$Revision: 5.406 $