Re: [leafnode-list] fetchnews in parallel?

[Matthias Andree <ma@xxxxxxxxxxxxxxxxxxxxxxxxxxxx>]

Witold Wladyslaw Wojciech Wilk <maniack@xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx> writes:

> and I was wondering - will fetchnews someday will be able to go into
> parallel with this? almost al of them carry all the groups, so the fetch
> could go faster, way faster when going parallel instead of serial.
>
> is it planned, is it made in leafnode 2.x, or will it never be? :)

That would indeed be nice to have, but currently, servers must still be
queried sequentially, even in leafnode 2.x. There are two and a half
places in the leafnode spool that require this:

1) the groupinfo file updates. parallel fetch would currently corrupt
   data or require very inefficient locking.

2) the .overview file updates. parallel fetch would not currently update
   the .overview file properly.

2.5) the store into the newsgroup itself. It should cope with races, but
     inefficiently.

I'm not sure if multi-threading could help us here or bring us there
faster than multi-process can (fork()) -- only that I'm not really
acquainted with threads.

Therefore, I cannot promise leafnode 2.0.0.rel will have this feature as
it is released (talk about the future...), because I cannot defer
leafnode 2.0.0.rel ad infinitum. Sorry.

-- 
Matthias Andree

-- 
leafnode-list@xxxxxxxxxxxxxxxxxxxxxxxxxxxx -- mailing list for leafnode
To unsubscribe, send mail with "unsubscribe" in the subject to the list