Re: [leafnode-list] FYI articles out of order

[Matthias Andree <matthias.andree@xxxxxx>]

Mike Vanecek schrieb am 2004-02-04:

>         There are at least two ways to check for new posts within the
> NNTP spec (NEWNEWS and high-water mark), but I get the sense that he's
> trying to make a point, not ask a question.

Indeed I am (send them my kind regards if you're taking this back).

The solution they are suggesting is expensive, and soon to become
obsolete.  Clive Feather's NNTP draft is nearing completion and going to
obsolete RFC-977 and prohibits out-of-order creation.

>         On the assumption that NNTP servers don't skip article numbers
> for no good reason, one way to compensate for this problem would be to
> keep track of which article numbers have not appeared within a range,
> and go back and look for those articles on subsequent retrievals.

Tracking which article numbers are missing requires orders of magnitude
more space than tracking the high-water mark which takes about 6 bytes.
I'd need to store all numbers of all articles missed until the low-water
mark rises above the first entries in my list, which I can then drop.

How often am I supposed to check for back-filled articles? As long as my
articles are between their low-water and high-water mark? That's going
to be REALLY expensive particularly for high-traffic groups or those
that see many cancels.  Some article numbers that will never appear, for
instance from superseded, canceled and expired articles.

On servers that hold articles for a long time, it can take months until
we can finally purge the "gap ranges" we're tracking in fetchnews. Not
useful.

If someone runs fetchnews every four hours, that's plenty of time for
automated spam cancels to arrive and punch holes in the otherwise linear
list.

>         By the way, we aren't the only provider who makes articles
> available "out of [article number] order."  Under normal circumstances,
> it doesn't matter that much, because the gaps are filled in quickly
> (within seconds or milliseconds).  Problems are arising now with us
> because the delay is longer (sometimes minutes).

Doing the wrong thing in a group doesn't make it right.

The NNTP draft #20 (not yet a standard) is pretty explicit on that:

  "The server MUST ensure that article numbers are issued in order of
  arrival timestamp; that is, articles arriving later MUST have higher
  numbers than those that arrive earlier. The server SHOULD allocate the
  next sequential unused number to each new article."
       -- Clive D. W. Feather,
       ftp.rfc-editor.org /in-notes/internet-drafts/draft-ietf-nntpext-base-20.txt
       section 6. Article posting and retrieval

I'm certainly not going to support out-of-order creation in leafnode-1
beyond what we have now in the form of "-x 1000" when the next NNTP
revision is going to prohibit out of order creation. I may or may not
support it in leafnode-2, I haven't yet decided on that.

-- 
Matthias Andree

Encrypt your mail: my GnuPG key ID is 0x052E7D95
-- 
_______________________________________________
leafnode-list mailing list
leafnode-list@xxxxxxxxxxxxxxxxxxxxxxxxxxxx
http://www.dt.e-technik.uni-dortmund.de/mailman/listinfo/leafnode-list
http://leafnode.sourceforge.net/