This is gnus, produced by makeinfo version 4.5 from gnus.texi. INFO-DIR-SECTION Emacs START-INFO-DIR-ENTRY * Gnus: (gnus). The newsreader Gnus. END-INFO-DIR-ENTRY This file documents Gnus, the GNU Emacs newsreader. Copyright (C) 1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003 Free Software Foundation, Inc. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.1 or any later version published by the Free Software Foundation; with the Invariant Sections being none, with the Front-Cover texts being "A GNU Manual", and with the Back-Cover Texts as in (a) below. A copy of the license is included in the section entitled "GNU Free Documentation License" in the Emacs manual. (a) The FSF's Back-Cover Text is: "You have freedom to copy and modify this GNU Manual, like GNU software. Copies published by the Free Software Foundation raise funds for GNU development." This document is part of a collection distributed under the GNU Free Documentation License. If you want to distribute this document separately from the collection, you can do so by adding a copy of the license to the document, as described in section 6 of the license.  File: gnus, Node: NoCeM, Next: Undo, Prev: Daemons, Up: Various NoCeM ===== "Spamming" is posting the same article lots and lots of times. Spamming is bad. Spamming is evil. Spamming is usually canceled within a day or so by various anti-spamming agencies. These agencies usually also send out "NoCeM" messages. NoCeM is pronounced "no see-'em", and means what the name implies--these are messages that make the offending articles, like, go away. What use are these NoCeM messages if the articles are canceled anyway? Some sites do not honor cancel messages and some sites just honor cancels from a select few people. Then you may wish to make use of the NoCeM messages, which are distributed in the `alt.nocem.misc' newsgroup. Gnus can read and parse the messages in this group automatically, and this will make spam disappear. There are some variables to customize, of course: `gnus-use-nocem' Set this variable to `t' to set the ball rolling. It is `nil' by default. `gnus-nocem-groups' Gnus will look for NoCeM messages in the groups in this list. The default is ("news.lists.filters" "news.admin.net-abuse.bulletins" "alt.nocem.misc" "news.admin.net-abuse.announce") `gnus-nocem-issuers' There are many people issuing NoCeM messages. This list says what people you want to listen to. The default is ("Automoose-1" "clewis@ferret.ocunix.on.ca" "cosmo.roadkill" "SpamHippo" "hweede@snafu.de") fine, upstanding citizens all of them. Known despammers that you can put in this list are listed at `http://www.xs4all.nl/~rosalind/nocemreg/nocemreg.html'. You do not have to heed NoCeM messages from all these people--just the ones you want to listen to. You also don't have to accept all NoCeM messages from the people you like. Each NoCeM message has a "type" header that gives the message a (more or less, usually less) rigorous definition. Common types are `spam', `spew', `mmf', `binary', and `troll'. To specify this, you have to use `(ISSUER CONDITIONS ...)' elements in the list. Each condition is either a string (which is a regexp that matches types you want to use) or a list on the form `(not STRING)', where STRING is a regexp that matches types you don't want to use. For instance, if you want all NoCeM messages from Chris Lewis except his `troll' messages, you'd say: ("clewis@ferret.ocunix.on.ca" ".*" (not "troll")) On the other hand, if you just want nothing but his `spam' and `spew' messages, you'd say: ("clewis@ferret.ocunix.on.ca" (not ".*") "spew" "spam") The specs are applied left-to-right. `gnus-nocem-verifyer' This should be a function for verifying that the NoCeM issuer is who she says she is. The default is `mc-verify', which is a Mailcrypt function. If this is too slow and you don't care for verification (which may be dangerous), you can set this variable to `nil'. If you want signed NoCeM messages to be verified and unsigned messages not to be verified (but used anyway), you could do something like: (setq gnus-nocem-verifyer 'my-gnus-mc-verify) (defun my-gnus-mc-verify () (not (eq 'forged (ignore-errors (if (mc-verify) t 'forged))))) This might be dangerous, though. `gnus-nocem-directory' This is where Gnus will store its NoCeM cache files. The default is `~/News/NoCeM/'. `gnus-nocem-expiry-wait' The number of days before removing old NoCeM entries from the cache. The default is 15. If you make it shorter Gnus will be faster, but you might then see old spam. `gnus-nocem-check-from' Non-`nil' means check for valid issuers in message bodies. Otherwise don't bother fetching articles unless their author matches a valid issuer; that is much faster if you are selective about the issuers. `gnus-nocem-check-article-limit' If non-`nil', the maximum number of articles to check in any NoCeM group. NoCeM groups can be huge and very slow to process. Using NoCeM could potentially be a memory hog. If you have many living (i. e., subscribed or unsubscribed groups), your Emacs process will grow big. If this is a problem, you should kill off all (or most) of your unsubscribed groups (*note Subscription Commands::).  File: gnus, Node: Undo, Next: Predicate Specifiers, Prev: NoCeM, Up: Various Undo ==== It is very useful to be able to undo actions one has done. In normal Emacs buffers, it's easy enough--you just push the `undo' button. In Gnus buffers, however, it isn't that simple. The things Gnus displays in its buffer is of no value whatsoever to Gnus--it's all just data designed to look nice to the user. Killing a group in the group buffer with `C-k' makes the line disappear, but that's just a side-effect of the real action--the removal of the group in question from the internal Gnus structures. Undoing something like that can't be done by the normal Emacs `undo' function. Gnus tries to remedy this somewhat by keeping track of what the user does and coming up with actions that would reverse the actions the user takes. When the user then presses the `undo' key, Gnus will run the code to reverse the previous action, or the previous actions. However, not all actions are easily reversible, so Gnus currently offers a few key functions to be undoable. These include killing groups, yanking groups, and changing the list of read articles of groups. That's it, really. More functions may be added in the future, but each added function means an increase in data to be stored, so Gnus will never be totally undoable. The undoability is provided by the `gnus-undo-mode' minor mode. It is used if `gnus-use-undo' is non-`nil', which is the default. The `C-M-_' key performs the `gnus-undo' command, which should feel kinda like the normal Emacs `undo' command.  File: gnus, Node: Predicate Specifiers, Next: Moderation, Prev: Undo, Up: Various Predicate Specifiers ==================== Some Gnus variables are "predicate specifiers". This is a special form that allows flexible specification of predicates without having to type all that much. These specifiers are lists consisting of functions, symbols and lists. Here's an example: (or gnus-article-unseen-p gnus-article-unread-p) The available symbols are `or', `and' and `not'. The functions all take one parameter. Internally, Gnus calls `gnus-make-predicate' on these specifiers to create a function that can be called. This input parameter to this function will be passed along to all the functions in the predicate specifier.  File: gnus, Node: Moderation, Next: Image Enhancements, Prev: Predicate Specifiers, Up: Various Moderation ========== If you are a moderator, you can use the `gnus-mdrtn.el' package. It is not included in the standard Gnus package. Write a mail to `larsi@gnus.org' and state what group you moderate, and you'll get a copy. The moderation package is implemented as a minor mode for summary buffers. Put (add-hook 'gnus-summary-mode-hook 'gnus-moderate) in your `~/.gnus.el' file. If you are the moderator of `rec.zoofle', this is how it's supposed to work: 1. You split your incoming mail by matching on `Newsgroups:.*rec.zoofle', which will put all the to-be-posted articles in some mail group--for instance, `nnml:rec.zoofle'. 2. You enter that group once in a while and post articles using the `e' (edit-and-post) or `s' (just send unedited) commands. 3. If, while reading the `rec.zoofle' newsgroup, you happen upon some articles that weren't approved by you, you can cancel them with the `c' command. To use moderation mode in these two groups, say: (setq gnus-moderated-list "^nnml:rec.zoofle$\\|^rec.zoofle$")  File: gnus, Node: Image Enhancements, Next: Fuzzy Matching, Prev: Moderation, Up: Various Image Enhancements ================== XEmacs, as well as Emacs 21(1), is able to display pictures and stuff, so Gnus has taken advantage of that. * Menu: * X-Face:: Display a funky, teensy black-and-white image. * Face:: Display a funkier, teensier colored image. * Smileys:: Show all those happy faces the way they were meant to be shown. * Picons:: How to display pictures of what you're reading. * XVarious:: Other XEmacsy Gnusey variables. ---------- Footnotes ---------- (1) Emacs 21 on MS Windows doesn't support images yet.  File: gnus, Node: X-Face, Next: Face, Up: Image Enhancements X-Face ------ `X-Face' headers describe a 48x48 pixel black-and-white (1 bit depth) image that's supposed to represent the author of the message. It seems to be supported by an ever-growing number of mail and news readers. Decoding an `X-Face' header either requires an Emacs that has `compface' support (which most XEmacs versions has), or that you have `compface' installed on your system. If either is true, Gnus will default to displaying `X-Face' headers. The variable that controls this is the `gnus-article-x-face-command' variable. If this variable is a string, this string will be executed in a sub-shell. If it is a function, this function will be called with the face as the argument. If the `gnus-article-x-face-too-ugly' (which is a regexp) matches the `From' header, the face will not be shown. The default action under Emacs 20 is to fork off the `display' program(1) to view the face. Under XEmacs or Emacs 21+ with suitable image support, the default action is to display the face before the `From' header. (It's nicer if XEmacs has been compiled with `X-Face' support--that will make display somewhat faster. If there's no native `X-Face' support, Gnus will try to convert the `X-Face' header using external programs from the `pbmplus' package and friends.(2)) (Note: `x-face' is used in the variable/function names, not `xface'). Gnus provides a few convenience functions and variables to allow easier insertion of X-Face headers in outgoing messages. `gnus-random-x-face' goes through all the `pbm' files in `gnus-x-face-directory' and picks one at random, and then converts it to the X-Face format by using the `gnus-convert-pbm-to-x-face-command' shell command. The `pbm' files should be 48x48 pixels big. It returns the X-Face header data as a string. `gnus-insert-random-x-face-header' calls `gnus-random-x-face' and inserts a `X-Face' header with the randomly generated data. `gnus-x-face-from-file' takes a GIF file as the parameter, and then converts the file to X-Face format by using the `gnus-convert-image-to-x-face-command' shell command. Here's how you would typically use the first function. Put something like the following in your `~/.gnus.el' file: (setq message-required-news-headers (nconc message-required-news-headers (list '(X-Face . gnus-random-x-face)))) Using the last function would be something like this: (setq message-required-news-headers (nconc message-required-news-headers (list '(X-Face . (lambda () (gnus-x-face-from-file "~/My-face.gif")))))) ---------- Footnotes ---------- (1) `display' is from the ImageMagick package. For the `uncompface' and `icontopbm' programs look for a package like `compface' or `faces-xface' on a GNU/Linux system. (2) On a GNU/Linux system look for packages with names like `netpbm', `libgr-progs' and `compface'.  File: gnus, Node: Face, Next: Smileys, Prev: X-Face, Up: Image Enhancements Face ---- `Face' headers are essentially a funkier version of `X-Face' ones. They describe a 48x48 pixel colored image that's supposed to represent the author of the message. The contents of a `Face' header must be a base64 encoded PNG image. See `http://quimby.gnus.org/circus/face/' for the precise specifications. Gnus provides a few convenience functions and variables to allow easier insertion of Face headers in outgoing messages. `gnus-convert-png-to-face' takes a 48x48 PNG image, no longer than 726 bytes long, and converts it to a face. `gnus-face-from-file' takes a JPEG file as the parameter, and then converts the file to Face format by using the `gnus-convert-image-to-face-command' shell command. Here's how you would typically use this function. Put something like the following in your `~/.gnus.el' file: (setq message-required-news-headers (nconc message-required-news-headers (list '(Face . (lambda () (gnus-face-from-file "~/face.jpg"))))))  File: gnus, Node: Smileys, Next: Picons, Prev: Face, Up: Image Enhancements Smileys ------- "Smiley" is a package separate from Gnus, but since Gnus is currently the only package that uses Smiley, it is documented here. In short--to use Smiley in Gnus, put the following in your `~/.gnus.el' file: (setq gnus-treat-display-smileys t) Smiley maps text smiley faces--`:-)', `8-)', `:-(' and the like--to pictures and displays those instead of the text smiley faces. The conversion is controlled by a list of regexps that matches text and maps that to file names. The alist used is specified by the `smiley-regexp-alist' variable. The first item in each element is the regexp to be matched; the second element is the regexp match group that is to be replaced by the picture; and the third element is the name of the file to be displayed. The following variables customize where Smiley will look for these files: `smiley-data-directory' Where Smiley will look for smiley faces files. `gnus-smiley-file-types' List of suffixes on smiley file names to try.  File: gnus, Node: Picons, Next: XVarious, Prev: Smileys, Up: Image Enhancements Picons ------ So... You want to slow down your news reader even more! This is a good way to do so. It's also a great way to impress people staring over your shoulder as you read news. What are Picons? To quote directly from the Picons Web site: "Picons" is short for "personal icons". They're small, constrained images used to represent users and domains on the net, organized into databases so that the appropriate image for a given e-mail address can be found. Besides users and domains, there are picon databases for Usenet newsgroups and weather forecasts. The picons are in either monochrome `XBM' format or color `XPM' and `GIF' formats. For instructions on obtaining and installing the picons databases, point your Web browser at `http://www.cs.indiana.edu/picons/ftp/index.html'. If you are using Debian GNU/Linux, saying `apt-get install picons.*' will install the picons where Gnus can find them. To enable displaying picons, simply make sure that `gnus-picon-databases' points to the directory containing the Picons databases. The following variables offer control over where things are located. `gnus-picon-databases' The location of the picons database. This is a list of directories containing the `news', `domains', `users' (and so on) subdirectories. Defaults to `("/usr/lib/picon" "/usr/local/faces")'. `gnus-picon-news-directories' List of subdirectories to search in `gnus-picon-databases' for newsgroups faces. `("news")' is the default. `gnus-picon-user-directories' List of subdirectories to search in `gnus-picon-databases' for user faces. `("users" "usenix" "local" "misc")' is the default. `gnus-picon-domain-directories' List of subdirectories to search in `gnus-picon-databases' for domain name faces. Defaults to `("domains")'. Some people may want to add `"unknown"' to this list. `gnus-picon-file-types' Ordered list of suffixes on picon file names to try. Defaults to `("xpm" "gif" "xbm")' minus those not built-in your Emacs.  File: gnus, Node: XVarious, Prev: Picons, Up: Image Enhancements Various XEmacs Variables ------------------------ `gnus-xmas-glyph-directory' This is where Gnus will look for pictures. Gnus will normally auto-detect this directory, but you may set it manually if you have an unusual directory structure. `gnus-xmas-logo-color-alist' This is an alist where the key is a type symbol and the values are the foreground and background color of the splash page glyph. `gnus-xmas-logo-color-style' This is the key used to look up the color in the alist described above. Valid values include `flame', `pine', `moss', `irish', `sky', `tin', `velvet', `grape', `labia', `berry', `neutral', and `september'. `gnus-xmas-modeline-glyph' A glyph displayed in all Gnus mode lines. It is a tiny gnu head by default. Toolbar ....... `gnus-use-toolbar' If `nil', don't display toolbars. If non-`nil', it should be one of `default-toolbar', `top-toolbar', `bottom-toolbar', `right-toolbar', or `left-toolbar'. `gnus-group-toolbar' The toolbar in the group buffer. `gnus-summary-toolbar' The toolbar in the summary buffer. `gnus-summary-mail-toolbar' The toolbar in the summary buffer of mail groups.  File: gnus, Node: Fuzzy Matching, Next: Thwarting Email Spam, Prev: Image Enhancements, Up: Various Fuzzy Matching ============== Gnus provides "fuzzy matching" of `Subject' lines when doing things like scoring, thread gathering and thread comparison. As opposed to regular expression matching, fuzzy matching is very fuzzy. It's so fuzzy that there's not even a definition of what "fuzziness" means, and the implementation has changed over time. Basically, it tries to remove all noise from lines before comparing. `Re: ', parenthetical remarks, white space, and so on, are filtered out of the strings before comparing the results. This often leads to adequate results--even when faced with strings generated by text manglers masquerading as newsreaders.  File: gnus, Node: Thwarting Email Spam, Next: Other modes, Prev: Fuzzy Matching, Up: Various Thwarting Email Spam ==================== In these last days of the Usenet, commercial vultures are hanging about and grepping through news like crazy to find email addresses they can foist off their scams and products to. As a reaction to this, many people have started putting nonsense addresses into their `From' lines. I think this is counterproductive--it makes it difficult for people to send you legitimate mail in response to things you write, as well as making it difficult to see who wrote what. This rewriting may perhaps be a bigger menace than the unsolicited commercial email itself in the end. The biggest problem I have with email spam is that it comes in under false pretenses. I press `g' and Gnus merrily informs me that I have 10 new emails. I say "Golly gee! Happy is me!" and select the mail group, only to find two pyramid schemes, seven advertisements ("New! Miracle tonic for growing full, lustrous hair on your toes!") and one mail asking me to repent and find some god. This is annoying. Here's what you can do about it. * Menu: * The problem of spam:: Some background, and some solutions * Anti-Spam Basics:: Simple steps to reduce the amount of spam. * SpamAssassin:: How to use external anti-spam tools. * Hashcash:: Reduce spam by burning CPU time. * Filtering Spam Using The Spam ELisp Package:: * Filtering Spam Using Statistics with spam-stat::  File: gnus, Node: The problem of spam, Next: Anti-Spam Basics, Up: Thwarting Email Spam The problem of spam ------------------- First, some background on spam. If you have access to e-mail, you are familiar with spam (technically termed UCE, Unsolicited Commercial E-mail). Simply put, it exists because e-mail delivery is very cheap compared to paper mail, so only a very small percentage of people need to respond to an UCE to make it worthwhile to the advertiser. Ironically, one of the most common spams is the one offering a database of e-mail addresses for further spamming. Senders of spam are usually called _spammers_, but terms like _vermin_, _scum_, _sociopaths_, and _morons_ are in common use as well. Spam comes from a wide variety of sources. It is simply impossible to dispose of all spam without discarding useful messages. A good example is the TMDA system, which requires senders unknown to you to confirm themselves as legitimate senders before their e-mail can reach you. Without getting into the technical side of TMDA, a downside is clearly that e-mail from legitimate sources may be discarded if those sources can't or won't confirm themselves through the TMDA system. Another problem with TMDA is that it requires its users to have a basic understanding of e-mail delivery and processing. The simplest approach to filtering spam is filtering, at the mail server or when you sort through incoming mail. If you get 200 spam messages per day from `random-address@vmadmin.com', you block `vmadmin.com'. If you get 200 messages about `VIAGRA', you discard all messages with `VIAGRA' in the message. If you get lots of spam from China, for example, you try to filter all mail from Chinese IPs. This, unfortunately, is a great way to discard legitimate e-mail. For instance, the very informative and useful RISKS digest has been blocked by overzealous mail filters because it *contained* words that were common in spam messages. The risks of blocking a whole country from contacting you should also be obvious, so don't do it if you have the choice. Nevertheless, in isolated cases, with great care, direct filtering of mail can be useful. Another approach to filtering e-mail is the distributed spam processing, for instance DCC implements such a system. In essence, N systems around the world agree that a machine X in Ghana, Estonia, or California is sending out spam e-mail, and these N systems enter X or the spam e-mail from X into a database. The criteria for spam detection vary--it may be the number of messages sent, the content of the messages, and so on. When a user of the distributed processing system wants to find out if a message is spam, he consults one of those N systems. Distributed spam processing works very well against spammers that send a large number of messages at once, but it requires the user to set up fairly complicated checks. There are commercial and free distributed spam processing systems. Distributed spam processing has its risks as well. For instance legitimate e-mail senders have been accused of sending spam, and their web sites and mailing lists have been shut down for some time because of the incident. The statistical approach to spam filtering is also popular. It is based on a statistical analysis of previous spam messages. Usually the analysis is a simple word frequency count, with perhaps pairs of words or 3-word combinations thrown into the mix. Statistical analysis of spam works very well in most of the cases, but it can classify legitimate e-mail as spam in some cases. It takes time to run the analysis, the full message must be analyzed, and the user has to store the database of spam analyses. Statistical analysis on the server is gaining popularity. This has the advantage of letting the user Just Read Mail, but has the disadvantage that it's harder to tell the server that it has misclassified mail. Fighting spam is not easy, no matter what anyone says. There is no magic switch that will distinguish Viagra ads from Mom's e-mails. Even people are having a hard time telling spam apart from non-spam, because spammers are actively looking to fool us into thinking they are Mom, essentially. Spamming is irritating, irresponsible, and idiotic behavior from a bunch of people who think the world owes them a favor. We hope the following sections will help you in fighting the spam plague.  File: gnus, Node: Anti-Spam Basics, Next: SpamAssassin, Prev: The problem of spam, Up: Thwarting Email Spam Anti-Spam Basics ---------------- One way of dealing with spam is having Gnus split out all spam into a `spam' mail group (*note Splitting Mail::). First, pick one (1) valid mail address that you can be reached at, and put it in your `From' header of all your news articles. (I've chosen `larsi@trym.ifi.uio.no', but for many addresses on the form `larsi+usenet@ifi.uio.no' will be a better choice. Ask your sysadmin whether your sendmail installation accepts keywords in the local part of the mail address.) (setq message-default-news-headers "From: Lars Magne Ingebrigtsen \n") Then put the following split rule in `nnmail-split-fancy' (*note Fancy Mail Splitting::): (... (to "larsi@trym.ifi.uio.no" (| ("subject" "re:.*" "misc") ("references" ".*@.*" "misc") "spam")) ...) This says that all mail to this address is suspect, but if it has a `Subject' that starts with a `Re:' or has a `References' header, it's probably ok. All the rest goes to the `spam' group. (This idea probably comes from Tim Pierce.) In addition, many mail spammers talk directly to your SMTP server and do not include your email address explicitly in the `To' header. Why they do this is unknown--perhaps it's to thwart this thwarting scheme? In any case, this is trivial to deal with--you just put anything not addressed to you in the `spam' group by ending your fancy split rule in this way: ( ... (to "larsi" "misc") "spam") In my experience, this will sort virtually everything into the right group. You still have to check the `spam' group from time to time to check for legitimate mail, though. If you feel like being a good net citizen, you can even send off complaints to the proper authorities on each unsolicited commercial email--at your leisure. This works for me. It allows people an easy way to contact me (they can just press `r' in the usual way), and I'm not bothered at all with spam. It's a win-win situation. Forging `From' headers to point to non-existent domains is yucky, in my opinion. Be careful with this approach. Spammers are wise to it.  File: gnus, Node: SpamAssassin, Next: Hashcash, Prev: Anti-Spam Basics, Up: Thwarting Email Spam SpamAssassin, Vipul's Razor, DCC, etc ------------------------------------- The days where the hints in the previous section were sufficient in avoiding spam are coming to an end. There are many tools out there that claim to reduce the amount of spam you get. This section could easily become outdated fast, as new products replace old, but fortunately most of these tools seem to have similar interfaces. Even though this section will use SpamAssassin as an example, it should be easy to adapt it to most other tools. Note that this section does not involve the `spam.el' package, which is discussed in the next section. If you don't care for all the features of `spam.el', you can make do with these simple recipes. If the tool you are using is not installed on the mail server, you need to invoke it yourself. Ideas on how to use the `:postscript' mail source parameter (*note Mail Source Specifiers::) follow. (setq mail-sources '((file :prescript "formail -bs spamassassin < /var/mail/%u") (pop :user "jrl" :server "pophost" :postscript "mv %t /tmp/foo; formail -bs spamc < /tmp/foo > %t"))) Once you manage to process your incoming spool somehow, thus making the mail contain e.g. a header indicating it is spam, you are ready to filter it out. Using normal split methods (*note Splitting Mail::): (setq nnmail-split-methods '(("spam" "^X-Spam-Flag: YES") ...)) Or using fancy split methods (*note Fancy Mail Splitting::): (setq nnmail-split-methods 'nnmail-split-fancy nnmail-split-fancy '(| ("X-Spam-Flag" "YES" "spam") ...)) Some people might not like the idea of piping the mail through various programs using a `:prescript' (if some program is buggy, you might lose all mail). If you are one of them, another solution is to call the external tools during splitting. Example fancy split method: (setq nnmail-split-fancy '(| (: kevin-spamassassin) ...)) (defun kevin-spamassassin () (save-excursion (save-restriction (widen) (if (eq 1 (call-process-region (point-min) (point-max) "spamc" nil nil nil "-c")) "spam")))) Note that with the nnimap backend, message bodies will not be downloaded by default. You need to set `nnimap-split-download-body' to t to do that (*note Splitting in IMAP::). That is about it. As some spam is likely to get through anyway, you might want to have a nifty function to call when you happen to read spam. And here is the nifty function: (defun my-gnus-raze-spam () "Submit SPAM to Vipul's Razor, then mark it as expirable." (interactive) (gnus-summary-show-raw-article) (gnus-summary-save-in-pipe "razor-report -f -d") (gnus-summary-mark-as-expirable 1))  File: gnus, Node: Hashcash, Next: Filtering Spam Using The Spam ELisp Package, Prev: SpamAssassin, Up: Thwarting Email Spam Hashcash -------- A novel technique to fight spam is to require senders to do something costly for each message they send. This has the obvious drawback that you cannot rely on everyone in the world using this technique, since it is not part of the Internet standards, but it may be useful in smaller communities. While the tools in the previous section work well in practice, they work only because the tools are constantly maintained and updated as new form of spam appears. This means that a small percentage of spam will always get through. It also means that somewhere, someone needs to read lots of spam to update these tools. Hashcash avoids that, but instead prefers that everyone you contact through e-mail supports the scheme. You can view the two approaches as pragmatic vs dogmatic. The approaches have their own advantages and disadvantages, but as often in the real world, a combination of them is stronger than either one of them separately. The "something costly" is to burn CPU time, more specifically to compute a hash collision up to a certain number of bits. The resulting hashcash cookie is inserted in a `X-Hashcash:' header. For more details, and for the external application `hashcash' you need to install to use this feature, see `http://www.cypherspace.org/~adam/hashcash/'. Even more information can be found at `http://www.camram.org/'. If you wish to call hashcash for each message you send, say something like: (require 'hashcash) (add-hook 'message-send-hook 'mail-add-payment) The `hashcash.el' library can be found in the Gnus development contrib directory or at `http://users.actrix.gen.nz/mycroft/hashcash.el'. You will need to set up some additional variables as well: `hashcash-default-payment' This variable indicates the default number of bits the hash collision should consist of. By default this is 0, meaning nothing will be done. Suggested useful values include 17 to 29. `hashcash-payment-alist' Some receivers may require you to spend burn more CPU time than the default. This variable contains a list of `(ADDR AMOUNT)' cells, where ADDR is the receiver (email address or newsgroup) and AMOUNT is the number of bits in the collision that is needed. It can also contain `(ADDR STRING AMOUNT)' cells, where the STRING is the string to use (normally the email address or newsgroup name is used). `hashcash' Where the `hashcash' binary is installed. Currently there is no built in functionality in Gnus to verify hashcash cookies, it is expected that this is performed by your hand customized mail filtering scripts. Improvements in this area would be a useful contribution, however.  File: gnus, Node: Filtering Spam Using The Spam ELisp Package, Next: Filtering Spam Using Statistics with spam-stat, Prev: Hashcash, Up: Thwarting Email Spam Filtering Spam Using The Spam ELisp Package ------------------------------------------- The idea behind `spam.el' is to have a control center for spam detection and filtering in Gnus. To that end, `spam.el' does two things: it filters new mail, and it analyzes mail known to be spam or ham. "Ham" is the name used throughout `spam.el' to indicate non-spam messages. First of all, you *must* run the function `spam-initialize' to autoload `spam.el' and to install the `spam.el' hooks. There is one exception: if you use the `spam-use-stat' (*note spam-stat spam filtering::) setting, you should turn it on before `spam-initialize': (setq spam-use-stat t) ;; if needed (spam-initialize) So, what happens when you load `spam.el'? First, some hooks will get installed by `spam-initialize'. There are some hooks for `spam-stat' so it can save its databases, and there are hooks so interesting things will happen when you enter and leave a group. More on the sequence of events later (*note Spam ELisp Package Sequence of Events::). You get the following keyboard commands: `M-d' `M s x' `S x' `gnus-summary-mark-as-spam'. Mark current article as spam, showing it with the `$' mark. Whenever you see a spam article, make sure to mark its summary line with `M-d' before leaving the group. This is done automatically for unread articles in _spam_ groups. `M s t' `S t' `spam-bogofilter-score'. You must have Bogofilter installed for that command to work properly. *Note Bogofilter::. Also, when you load `spam.el', you will be able to customize its variables. Try `customize-group' on the `spam' variable group. * Menu: * Spam ELisp Package Sequence of Events:: * Spam ELisp Package Filtering of Incoming Mail:: * Spam ELisp Package Global Variables:: * Spam ELisp Package Configuration Examples:: * Blacklists and Whitelists:: * BBDB Whitelists:: * Gmane Spam Reporting:: * Anti-spam Hashcash Payments:: * Blackholes:: * Regular Expressions Header Matching:: * Bogofilter:: * ifile spam filtering:: * spam-stat spam filtering:: * SpamOracle:: * Extending the Spam ELisp package::  File: gnus, Node: Spam ELisp Package Sequence of Events, Next: Spam ELisp Package Filtering of Incoming Mail, Up: Filtering Spam Using The Spam ELisp Package Spam ELisp Package Sequence of Events ..................................... You must read this section to understand how `spam.el' works. Do not skip, speed-read, or glance through this section. There are two _contact points_, if you will, between `spam.el' and the rest of Gnus: checking new mail for spam, and leaving a group. Getting new mail is done in one of two ways. You can either split your incoming mail or you can classify new articles as ham or spam when you enter the group. Splitting incoming mail is better suited to mail backends such as `nnml' or `nnimap' where new mail appears in a single file called a "Spool File". See *Note Spam ELisp Package Filtering of Incoming Mail::. For backends such as `nntp' there is no incoming mail spool, so an alternate mechanism must be used. This may also happen for backends where the server is in charge of splitting incoming mail, and Gnus does not do further splitting. The `spam-autodetect' and `spam-autodetect-methods' group parameters (accessible with `G c' and `G p' as usual), and the corresponding variables `gnus-spam-autodetect-methods' and `gnus-spam-autodetect-methods' (accessible with `M-x customize-variable' as usual). When `spam-autodetect' is used, it hooks into the process of entering a group. Thus, entering a group with unseen or unread articles becomes the substitute for checking incoming mail. Whether only unseen articles or all unread articles will be processed is determined by the `spam-autodetect-recheck-messages'. When set to t, unread messages will be rechecked. `spam-autodetect' grants the user at once more and less control of spam filtering. The user will have more control over each group's spam methods, so for instance the `ding' group may have `spam-use-BBDB' as the autodetection method, while the `suspect' group may have the `spam-use-blacklist' and `spam-use-bogofilter' methods enabled. Every article detected to be spam will be marked with the spam mark `$' and processed on exit from the group as normal spam. The user has less control over the _sequence_ of checks, as he might with `spam-split'. When the newly split mail goes into groups, or messages are autodetected to be ham or spam, those groups must be exited (after entering, if needed) for further spam processing to happen. It matters whether the group is considered a ham group, a spam group, or is unclassified, based on its `spam-content' parameter (*note Spam ELisp Package Global Variables::). Spam groups have the additional characteristic that, when entered, any unseen or unread articles (depending on the `spam-mark-only-unseen-as-spam' variable) will be marked as spam. Thus, mail split into a spam group gets automatically marked as spam when you enter the group. So, when you exit a group, the `spam-processors' are applied, if any are set, and the processed mail is moved to the `ham-process-destination' or the `spam-process-destination' depending on the article's classification. If the `ham-process-destination' or the `spam-process-destination', whichever is appropriate, are nil, the article is left in the current group. If a spam is found in any group (this can be changed to only non-spam groups with `spam-move-spam-nonspam-groups-only'), it is processed by the active `spam-processors' (*note Spam ELisp Package Global Variables::) when the group is exited. Furthermore, the spam is moved to the `spam-process-destination' (*note Spam ELisp Package Global Variables::) for further training or deletion. You have to load the `gnus-registry.el' package and enable the `spam-log-to-registry' variable if you want spam to be processed no more than once. Thus, spam is detected and processed everywhere, which is what most people want. If the `spam-process-destination' is nil, the spam is marked as expired, which is usually the right thing to do. If spam can not be moved - because of a read-only backend such as NNTP, for example, it will be copied. If a ham mail is found in a ham group, as determined by the `ham-marks' parameter, it is processed as ham by the active ham `spam-processor' when the group is exited. With the variables `spam-process-ham-in-spam-groups' and `spam-process-ham-in-nonham-groups' the behavior can be further altered so ham found anywhere can be processed. You have to load the `gnus-registry.el' package and enable the `spam-log-to-registry' variable if you want ham to be processed no more than once. Thus, ham is detected and processed only when necessary, which is what most people want. More on this in *Note Spam ELisp Package Configuration Examples::. If ham can not be moved - because of a read-only backend such as NNTP, for example, it will be copied. If all this seems confusing, don't worry. Soon it will be as natural as typing Lisp one-liners on a neural interface... err, sorry, that's 50 years in the future yet. Just trust us, it's not so bad.  File: gnus, Node: Spam ELisp Package Filtering of Incoming Mail, Next: Spam ELisp Package Global Variables, Prev: Spam ELisp Package Sequence of Events, Up: Filtering Spam Using The Spam ELisp Package Spam ELisp Package Filtering of Incoming Mail ............................................. To use the `spam.el' facilities for incoming mail filtering, you must add the following to your fancy split list `nnmail-split-fancy' or `nnimap-split-fancy': (: spam-split) Note that the fancy split may be called `nnmail-split-fancy' or `nnimap-split-fancy', depending on whether you use the nnmail or nnimap back ends to retrieve your mail. The `spam-split' function will process incoming mail and send the mail considered to be spam into the group name given by the variable `spam-split-group'. By default that group name is `spam', but you can customize `spam-split-group'. Make sure the contents of `spam-split-group' are an _unqualified_ group name, for instance in an `nnimap' server `your-server' the value `spam' will turn out to be `nnimap+your-server:spam'. The value `nnimap+server:spam', therefore, is wrong and will actually give you the group `nnimap+your-server:nnimap+server:spam' which may or may not work depending on your server's tolerance for strange group names. You can also give `spam-split' a parameter, e.g. `'spam-use-regex-headers' or `"maybe-spam"'. Why is this useful? Take these split rules (with `spam-use-regex-headers' and `spam-use-blackholes' set): nnimap-split-fancy '(| (any "ding" "ding") (: spam-split) ;; default mailbox "mail") Now, the problem is that you want all ding messages to make it to the ding folder. But that will let obvious spam (for example, spam detected by SpamAssassin, and `spam-use-regex-headers') through, when it's sent to the ding list. On the other hand, some messages to the ding list are from a mail server in the blackhole list, so the invocation of `spam-split' can't be before the ding rule. You can let SpamAssassin headers supersede ding rules, but all other `spam-split' rules (including a second invocation of the regex-headers check) will be after the ding rule: nnimap-split-fancy '(| ;;; all spam detected by spam-use-regex-headers goes to "regex-spam" (: spam-split "regex-spam" 'spam-use-regex-headers) (any "ding" "ding") ;;; all other spam detected by spam-split goes to spam-split-group (: spam-split) ;; default mailbox "mail") This lets you invoke specific `spam-split' checks depending on your particular needs, and to target the results of those checks to a particular spam group. You don't have to throw all mail into all the spam tests. Another reason why this is nice is that messages to mailing lists you have rules for don't have to have resource-intensive blackhole checks performed on them. You could also specify different spam checks for your nnmail split vs. your nnimap split. Go crazy. You should still have specific checks such as `spam-use-regex-headers' set to `t', even if you specifically invoke `spam-split' with the check. The reason is that when loading `spam.el', some conditional loading is done depending on what `spam-use-xyz' variables you have set. This is usually not critical, though. _Note for IMAP users_ The boolean variable `nnimap-split-download-body' needs to be set, if you want to split based on the whole message instead of just the headers. By default, the nnimap back end will only retrieve the message headers. If you use `spam-check-bogofilter', `spam-check-ifile', or `spam-check-stat' (the splitters that can benefit from the full message body), you should set this variable. It is not set by default because it will slow IMAP down, and that is not an appropriate decision to make on behalf of the user. *Note Splitting in IMAP::. _TODO: spam.el needs to provide a uniform way of training all the statistical databases. Some have that functionality built-in, others don't._