[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [leafnode-list] fetchnews: slow and hung



--IJpNTDwzlM2Ie8A6
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

on Tue, Mar 20, 2001 at 07:58:59AM +0100, Cornelius Krasel (krasel@xxxxxxxx=
oxi.uni-wuerzburg.de) wrote:
> Karsten Self wrote:
>=20
> > Running Debian/Sid, with leafnode version: 1.9.18-3, ISP is Mindspring,
> > newserver news.earthlink.net.
> >
> > I'm finding that news downloads with fetchnews (about 76 groups, mostly
> > comp.*, active), are slow, and that fetchnews often seems to hang.
>=20
> fetchnews is very bad at detecting dropped connections, I am afraid.
> The 1.9.18 package is rather slow in general, partly because it uses
> XHDR (this is a problem if your upstream server uses INN) and partly
> because it needs two commands to fetch every single article (HEAD
> and BODY). The XHDR problem can be ameliorated by switching to
> Matthias Andree's patched version 1.9.18ma4 which is available from
> http://www-dt.e-technik.uni-dortmund.de/~ma/leafnode/testing/ , or
> you can try the latest leafnode-2.0b which is currently leafnode-2.0b8,
> available from ftp://wpxx02.toxi.uni-wuerzburg.de/pub/ . However,
> 2.0b8 still has serious problems with dropped connections.

Some thoughts on this.  I don't follow leafnode development closely,
though I check list topics periodically.  I've reviewed the FAQ and TODO
for 1.9.18-3 (Debian) prior to posting this.

For dropped connections, might a timeout be used as a proxy for this?  I
don't know what's typical for people here, but I tend to cron fetchnews
to run somewhere between every 20 minutes to 24 hours -- depending on
how well things are working at a point in time.  Some inactivity timeout
-- remote host not responding -- of five minutes or so, ought to be a
reasonably good proxy.  Better yet, make the functionality and length of
timeout user-configurable.

The other problem is fetching of groups when the connection is
reestablished.  Currently, fetchnews attempts to catch up with all
groups, starting with the first listed.  For a site with very unstable
connections, the result is that early-listed NGs are downloaded fairly
frequently, while late-listed NGs are skipped fairly often.  Saving
state in the form of most recently D/L'd groups might be a workaround
for these two problems.  Not sure how much overhead keeping track of
this would impose.

This is mostly of use to people with slow, unreliable network
connections.  Not sure if this covers most of the group or not, but it
describes me pretty accurately.

Thanks for your time.

--=20
Karsten M. Self <kmself@xxxxxxxxxxxxx>    http://kmself.home.netcom.com/
 What part of "Gestalt" don't you understand?       There is no K5 cabal
  http://gestalt-system.sourceforge.net/         http://www.kuro5hin.org

--IJpNTDwzlM2Ie8A6
Content-Type: application/pgp-signature
Content-Disposition: inline

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.4 (GNU/Linux)
Comment: For info see http://www.gnupg.org

iD8DBQE6uxaPOEeIn1XyubARAgVpAJ9H6yC8Hl1QGDOqybPYOMlyrd9+5QCdFPdM
9O2Tlr+qA+0auTPl0/1/jjE=
=iKWE
-----END PGP SIGNATURE-----

--IJpNTDwzlM2Ie8A6--

-- 
leafnode-list@xxxxxxxxxxxxxxxxxxxxxxxxxxxx -- mailing list for leafnode
To unsubscribe, send mail with "unsubscribe" in the subject to the list