[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[leafnode-list] Re: applyfilter doesn't seem to work



clemens fischer <ino-news@xxxxxxxxxxxxxxxxxxxxxxxxx> writes:

> the newsgroup sci.crypt is currently under attack, as some might know.
> it is tiresome to find a pattern in the random noise posted, and it is
> a DDoS using different accounts on different servers.  thus i have many
> articles in the spool now where previous patterns no longer work.
>
> after finding some patterns[1], i wanted to use applyfilters, which
> doesn't work as advertised, so to speak:
>
> /src/leafnode-2.0.0.alpha20070108a
> 0  # applyfilter -n 'sci.crypt'
> Usage:
> applyfilter [options] object [...]
> options are:
>     -d spooldir    - use spooldir instead of compiled-in default
>     -D debug       - set debug flags (see config.example for documentation)
>     -e             - redirect logging to standard error
>     -F config      - read config file as configuration instead of default
>     -v [...]       - increase verbosity level (can be repeated)
>     -V             - just print version and exit, perform no other operation
>     -c             - check mode, print if filters match files on command line
>     -n             - dry run, do not actually delete anything
>
> the `-n' switch doesn't work, i always get the usage() message.  i have
> the feeling that applyfilter isn't used all that much and might have
> a few bugs left.

Yup. Turns out I broke it in January 2005 when moving the "-C" option
(cancel a certain article locally) from applyfilter to texpire:

Wed Jan  5 05:08:51 CET 2005  Matthias Andree <matthias.andree@xxxxxx>
  * Drop semi-broken applyfilter -C option. texpire -C works better.
...
@@ -233,7 +227,7 @@
     }

     if (optind + 1 > argc
-           || (cancel && dryrun)) {
+           || (dryrun)) {
        usage();
        exit(EXIT_FAILURE);
     }

> 1.  when strace'ing "strace applyfilter -e -D 96 'sci.crypt'", one can
>     see that many of the spool-directories for leafnode are
>     (re-)created, and an UDP(?) connection is attempted to the hostname
>     specified in the configuration file.  why does this happen?

All leafnode programs try to canonicalize the fqdn to make sure the
hostname used is consistent. This is important with remote and
distributed filesystems such as NFS to keep locking working even without
lock manager support.

> 2.  the code of routine "main()" contains the passage:
>
>       if (optind + 1 > argc
>           || (dryrun)) {
>         usage();
>         exit(EXIT_FAILURE);
>       }
>
>     this is the reason for bailing out after giving `-n' (for dryrun).
>     no indication what went wrong, because nothing was even tried.  the
>     code does contain dryrun checks, though.

That's the one. Sorry for breaking the code.

> pattern = ^Path:.+!newsfe[[:xdigit:]]{2}\.(lga|phx)\.POSTED![[:xdigit:]]+!not-for-mail
> action = -5000
> pattern = ^Path:.+!hw-filter.(phx|lga)!newsfe.+![[:xdigit:]]+!not-for-mail
> action = -5000
> pattern = ^Path:.+!news.highwinds-media.com!hw-filter.phx!.+\.POSTED![[:xdigit:]]+!not-for-mail
> action = -5000
> pattern = ^(References|Message-ID): .+@xxxxxxxxxxxx>
> action = -5000
> pattern = ^X-Trace: newsfe[[:xdigit:]]{2}\.(lga|phx)
> action = -5000
> pattern = ^Organization: (Optimum Online|Will you rule on top of the winter,)
> action = -5000
> pattern = ^X-Complaints-To: (abuse|admin)@((rr|mchsi)\.com|(suddenlink|cox|cv)\.net|shaw\.ca)
> action = -5000
> pattern = ^NNTP-Posting-Host: (64.59.144.75|68.8.120.74)
> action = -5000
>
> when i use "action = kill", articles get deleted, but i wanted to use
> a negative quantity in order to override any decisions with positive
> quantities when articles should be "whitelisted" with patterns later
> in the file. this doesn't work. i'd like to use scoring numbers in the
> actions which accumulate until values greater than zero keep the article
> and values less than zero remove it.  it seems fetchnews uses a slightly
> different scheme, as my logfiles tell me that it kills articles given
> negative numbers as actions.

Both programs use exactly the same killfilter() function (which returns
a boolean value), so there should not be any differences; if there are,
that's a bug.

> also, the code seems to indicate that patterns are kept in a linked
> list.  can i rewrite the rules like this?
>
> newsgroups = (?i)some\.group
> pattern = ...
> pattern = ...
> pattern = ...
> action = kill
>
> ie. a newsgroup line at the top, a single action line at the bottom and
> patterns in between?

No, but you can omit the intermediate newsgroups, see the second example
is in man 5 filterfile.

New leafnode-2 snapshot in a few moments.

-- 
Matthias Andree
-- 
_______________________________________________
leafnode-list mailing list
leafnode-list@xxxxxxxxxxxxxxxxxxxxxxxxxxxx
https://www.dt.e-technik.uni-dortmund.de/mailman/listinfo/leafnode-list
http://leafnode.sourceforge.net/