Re: [leafnode-list] Filtering revisited.

[Joerg Dietrich <uzsv7x@xxxxxxxxxxx>]

On Thu, Sep 30, 1999 at 04:55:40PM -0400, Lloyd Zusman wrote:
> Filtering is very useful for those of us who use leafnode to manage a
> small news spool behind a relatively slow connection to an upstream
> server.  However, the currently available filtering capabilities in
> leafnode don't allow me to take full advantage of the filtering that
> would be ideal for me.

Already here I disagree. Leafnode is a news-server. As this it provides
service to one or more clients (users). Leafnode is designed to be most
effective for small leaf sites. These sites are admittedly often dial-up
boxes used by single person. This is the only case in which filtering
by the news service  providing package can be useful. In all other cases
different users will have different criteria to unselect articles.
	IMHO filtering should be done with the newsreader and not with
any tool belonging to the server. The only exception maybe a the
aforementioned single-user dial-up box with a *slow* newsfeed.
	This brings me to the next point. The lack of speed of fetchnews
is almost *never* due to a slow internet connection (unless you have a
14.4 modem or worse, maybe), but is a
consequence of 

(1) the NNTP,
(2) the way the fetch process is implemented.

I believe, that if Leafnode was able to either open multiple connections
, and/or fetch asynchronously, or use UUCP, filtering would be
superflous because the filtering process would consume more time than
the fetching of the article itself. 
	Unfortunately we are now at point were filtering is implemented
and it has to be supported in future versions of Leafnode.

[description of LZ's patch and wishlist for filters erased]

> So ... I'd like to offer some suggestions about some possible methods
> for more sophisticated scoring within leafnode, and perhaps we could
> discuss the merits and drawbacks each as a possible leafnode
> enhancement in the future:
> 
> (1)  Score-based filtering rules, with the current filtering
>      capabilities being a subset of this so that existing
>      filter files will still work (I already wrote and
>      submitted a prototype of this a few months ago).

Again, scoring is implemented in any sensible newsreader and that is
were it belongs, if you ask me.

> (2)  Allow for the current regular-expression matching to be wrapped
>      by some sort of new filtering language that implements nesting,
>      boolean logic, etc.

Sounds very slow. Exactly the opposite of what you want if you filter
with fetchnews.

> (3)  Allow for pluggable, optional filtering modules that could
>      be written in C and dynamically linked with the leafnode
>      exectuables.  A library of convenience routines could be
>      supplied to aid the the writers of these optional modules
>      to make it easier to do things like locating and extracting
>      information about headers, article size, newsgroups, etc.  This
>      would allow the leafnode administrators to create arbitrarily
>      sophisticated filtering mechanisms that run relatively quickly.

This sounds interesting. It would leave the true server/fetchnews code
intact and would allow everybody to do what he wants to do. If any
filtering at all, than this is IMHO the method of choice. The existing
filtering could be used as a module and everybody would be happy :-)

> (4)  Allow for some sort of embedded scripting language to be
>      optionally built into leafnode ... python comes to mind, as does
>      perl.  This would allow for scripts to be written that serve the
>      same function as the pluggable, dynamically linked modules I
>      described above in (3).  This would run a bit slower than the
>      approach that uses the dynamically linked C modules, but it would
>      probably be easier to use.

Again, filtering is for slow connections. If the method of filtering is
slow using this filter is pointless.

> At any rate, these four things come to mind, but I'm sure that there
> also are other possibilities which could be just as useful.

To emphasize it again: If you ask me increasing the speed of fetchnews
is the top priority task in leafnode development. Most users simply
won't need filtering at all anymore, if fetchnews is as fast as UUCP or
suck with multiple connections. 
	At the time I proposed continuing the 1.9.x tree to get a stable
and bugfree[tm] version rather than releasing a fast 1.10 (or 2.0),
1.10b2 contained an rnews program. I wrote two patches to bring this
program into a stage where I consider it worth a beta test.
Unfortunatley it won't work with 1.9.4 because it relies on changes to
other parts of the Leafnode code (IIRC). My suggestion, therefore, would
be to have a new beta version ASAP that includes rnews and whatever
Cornelius has done since then.
	Cornelius, could you give a brief summary of what you've been
doing and what the current stage of leafnode development is? I hope I
don't ask for too much. I know that you have life besides leafnode and
RfC's and I guess you're buried alive in patches.

> What do all you folks think?  I'm optimistic that we could agree on
> something that would be quite useful and not all that hard to
> implement.

Ok, I wrote enough, suggested a lot, chose (3), which is probably the
most difficult one to implement, and didn't write line of useful code
since June or so. Feel free to flame me.

Regards,
	Jo:rg
 

-- 
leafnode-list@xxxxxxxxxxxxxxxxxxxxxxxxxxxx -- mailing list for leafnode
To unsubscribe, send mail with "unsubscribe" in the subject to the list