Re: Huge numbers of slow, spurious discards ...

[Greg Weeks <gweeks@xxxxxxxxxxxxx>]

Lloyd Zusman wrote:
> 
> I'm using leafnode-1.9.2, and now I'm getting huge quantities of slow,
> spurious discards when `fetch' is running.  Hundreds, if not thousands
> of lines like these are appearing in my syslog (I'm splitting each
> line at the word "Subject" and indenting so this will fit in a news
> message, but in actuality, each is one long line):
> 
>   Mar 15 18:19:23 home fetch[12131]: Discarding article 34298
>      Subject: Waiting for a process to finish? - older than 10 days
>   Mar 15 18:19:24 home fetch[12131]: Discarding article 34297
>      Subject: Re: XEmacs 20.3 and RedHat 5.2 - older than 10 days
>   Mar 15 18:19:27 home fetch[12131]: Discarding article 34295
>      Subject: Re: GTk+ XEmacs? - older than 10 days
>   ... etc. etc. etc. etc. ...
> 
> Each line takes maybe 1 or 2 seconds to print, so this is taking up
> huge amounts of time.
> 
> Note that the article numbers are decreasing.
> 
> It's also noteworthy that my local leafnode article numbers are not
> anywhere close to these numbers ... my local numbers are no more than
> several hundred, so these must be article numbers from the upstream
> servers (I am using three servers).
> 
> Also, note that I only installed leafnode for the first time maybe 5
> or 6 days ago and have been running it only since then, and yet each
> message states that the discarded articles are older than 10 days old.
> 
> I'm also enclosing a copy of my /etc/leafnode/config file (with comments
> removed and user-id/passwords changed).
> 
> Can anyone shed any light as to what's happening here, and how to
> prevent it?

I think I can explain what happened. When you start accessing a new
group fetch tries to download the entire group from your upstream
server. There is a default that it won't download anything over 10 days
old. It has to fetch the header for every message to find out how old it
is though. At least this looks like what's happening. Once it goes
through this for a given group it shouldn't need to do it again. If you
add a new server it goes through this for every group on the new server.
I'm not saying this is definitely what's going on. I haven't dug through
the code enough. I've seen behavior like this though. The article
numbers are for the upstream server it's fetching from by the way.

Greg Weeks
-- 
http://durendal.tzo.com/greg/

-- 
leafnode-list@xxxxxxxxxxxxxxxxxxxxxxxxxxxx -- mailing list for leafnode
To unsubscribe, send mail with "unsubscribe" in the subject to the list