Re: [leafnode-list] Bandwidth limiter for fetch downloads?

[dr-erc@xxxxxxxxxxx (Dieter Rohlfing)]

Cornelius Krasel wrote:
> 
> Dieter Rohlfing wrote:
> 
> > Instead of issueing for each article a HEADER command, you can send a
> > XOVER command to the news-server and you'll get the most important
> > headers (not all headers) for a specified range of articles, that means
> > 1 command for N articles. The bigger N, the bigger the performance gain.
> 
> You mean that you write only the XOVER headers to a file?

Yes, I do. I have no other headers.

> Personally, I think that filtering based only on the XOVER headers is
> unsatisfactory for me. ... Therefore, I am very reluctant to use XOVER
> information only for filtering.

What is sufficient and what not, depends on your needs and/or your
preferences. In your special case I totally agree with you. If connect
time doesn't matter, I wouldn't care about fetch's speed and only
emphasize the comfort of filtering.

In my case connect time is an important factor, I do pay for every
minute. 95% of my filtering rules depend on the subject, the remaining
on the newsgroup and the from line. And these information are delivered
by XOVER. Therefore, filtering the XOVER information is sufficient for
me.

My main interest is to reduce the connect time, and with the XOVER
feature built into fetch I get a better performance than with the
HEADER/BODY combo. Have a look at the numbers I got from my yesterday's
download (ISP is T-Online/Germany, modem is 33.6):

     seconds total:   359.0
      kbytes total:  1551.4
        kbytes/sec:     4.3

     seconds xover:   235.0
      kbytes xover:  1314.4
        kbytes/sec:     5.6
 articles in xover:  4140.0
      articles/sec:    17.6

      seconds body:    42.0
       kbytes body:    92.2
        kbytes/sec:     2.2
  fetched articles:    59.0
      articles/sec:     1.4

      seconds mark:    58.0
       kbytes mark:   144.9
        kbytes/sec:     2.5
  fetched articles:    95.0
      articles/sec:     1.6

      210 articles ignored
     3930 articles processed
       59 articles with headers & bodies
     3871 articles with headers only
     4140 articles totally

This is a summary of 45 newsgroups, it includes low traffic groups as
well as high traffic groups.

I think, leafnode should contain everything what's possible and
sensible. Let the user decide, what's the best fit for his needs.

I can only repeat my offer to you, to cooperate in building in the XOVER
feature. Two weeks ago I made some suggestions to you how to achieve
your goals as well as my goals. Up to now I haven't gotten any answer to
my proposals. Last, but not least I've added some other features, which
are specially very useful for a one-user system (e.g. watch feature,
suppress multiple postings etc), all configurable. So, depending on your
config file, you can get a leafnode that operates the same way as now or
operates with the new features.

It's up to you, Cornelius, to pick or drop the offer.

Dieter Rohlfing

-- 
leafnode-list@xxxxxxxxxxxxxxxxxxxxxxxxxxxx -- mailing list for leafnode
To unsubscribe, send mail with "unsubscribe" in the subject to the list