[leafnode-list] Re: Fetchnews messages

["Matthias Andree" <matthias.andree@xxxxxx>]

Am 16.05.2009, 12:28 Uhr, schrieb Enrico Wiki  
<enrico.c.ml.address@xxxxxxxxx>:

> On Wed, May 13, 2009 at 10:45 PM, Matthias Andree  
> <matthias.andree@xxxxxx>wrote:
>
>
>> Hi Enrico,
>>
>> leafnode isn't made to find magic ways when underlying transports  
>> protocol or physical links fail. Leafnode relies on the operating  
>> system to handle TCP properly (most do this today for the common subset  
>> that leafnode uses); and when connections break, well, that's it.
>
>
> Of course so. But there are times connections are unstable, come and go,  
> and
> perhaps some retrievers try harder than others? I don't know, it's just a
> wild guess of the reason I could not complete a leafnode job, when the
> connection was rough, but could complete retrieval with some readers.

Possibly. But hey, you are using Unix or a Unix-like operating system, so  
other than with the typical graphical tool that has to integrate all the  
features including retries, you can add features yourself, see below.

> Having said that, I know the real culprit was my connection, like I said.
> Now the connection is fine and leafnode is doing very well.

The obvious workaround will be do tell fetchnews to retry, by for  
instance, along these lines as a cron entry:

17 * * * * while ! fetchnews [--options] ; do sleep 300 ; done

This will poll hourly at X:17 o'clock, and after failure retry after  
sleeping for 300 seconds. Watch your logs though...

>> Perhaps - but usually a sign that the upstream server's database is
>> corrupt/inconsistent, particularly the overview data doesn't match the
>> available articles: the XOVER command offers articles that aren't  
>> available any more.
>
> Ok. I tried both xover and xhdr, same result.

They would be unlikely to differ among each others, as both will access  
the overview database, which is different from the article database in  
many implementations - including leafnode (although leafnode can afford to  
fix inconsistencies on the assumption that the overall load is lower, so  
you'll usually see consistent data): there are the message.id and group  
directories, and there are the .overview files in the group directories...

> Well, no, it wasn't aborted.
> I tried again (starting from scratch) and had the same results.

Have some of the articles been crossposted to several groups? Then  
leafnode will have fetched it for one of the groups, and when it's listed  
in another, it will not download another copy, because it already has it.

HTH

-- 
Matthias Andree
-- 
_______________________________________________
leafnode-list mailing list
leafnode-list@xxxxxxxxxxxxxxxxxxxxxxxxxxxx
https://www.dt.e-technik.uni-dortmund.de/mailman/listinfo/leafnode-list
http://leafnode.sourceforge.net/