[leafnode-list] Re: texpire deleting more articles than it should

[Matthias Andree <matthias.andree@xxxxxx>]

Am 10.09.2011 01:08, schrieb John Wiegley:
>>>>>> Matthias Andree <matthias.andree@xxxxxx> writes:
> 
>> sorry for that.  Are there logs beyond what you've quoted?
> 
> I found most of my problem: duplicate message-ids being strangely handled.
> 
> I had two files, a and b, both with the same message-id.  a was linked into
> message.id, but b wasn't.
> 
> When I ran texpire -D 257 -vvv -f -e -n wg21.c++.all, it told me it was going
> to delete a, but that it was *also* going to delete b because it wasn't in
> message.id.
> 
> Then I ran texpire -vvv -f -r -n wg21.c++.all, which fixed up the hard links
> in message.id.  *Then*, when I ran the first command again, it reported it was
> only going to delete a now, and not b anymore.

John,

So something on your computer has broken the hard links.

Does Git handle and restore hard links?  I never bothered to check
because I never needed such a functionality.  If Git instead handles a
hard linked file as two separate files that happen to have identical
content, it would store it quite efficiently (because the delta is null
which compresses quite well too) but break leafnode's spool.

Have you ever copied or moved the spool?

Restored things from backup?

Or upgraded from an older leafnode version without running texpire -r?

If so, which tools/commands have you used to act upon the spool?


Typically fetchnews will NOT download duplicates (that can happen if you
upgrade from leafnode-1 or across particular bug fixes in ancient
leafnode-2 versions). There have been rare occasions when the
calculation of the number in the .../message.id/123 path names changed,
and then fetchnews may redownload articles it already has if the old
spool is used with the new software.  Texpire -r will repair the hard
links and re-sort the Message-ID links across the message.id/NNN
directories, however, it is not usually necessary to run it during an
upgrade (I'd add a note to NEWS if it is required).

It's safe to do routinely though, but costs some time.


Normally, for all versions, crossposted articles will be hard linked
into the $spooldir/group/name/ directories and to .../message.id.

This assumption wasn't fulfilled in your case causing the trouble you'd
seen originally.

> Otherwise, the deletions it proposed all seem like legitimate duplicates after
> analysis.  The only bug here is that it would have deleted both a and b until
> I repaired the message.id hard links for that group.  In essence this means
> that, to be safe, I have to always run this:
> 
>   texpire -vvv -f -r -n
>   texpire -vvv -f
> 
>> Are you running leafnode on a 64-bit computer?  Please show me the output of
>> "leafnode-version -v".
> 
> Yes:
> 
> --8<---------------cut here---------------start------------->8---
> version: leafnode-2.0.0.alpha20110815a.luascript
> current machine: Darwin vulcan 10.8.0 Darwin Kernel Version 10.8.0: Tue Jun  7 16:33:36 PDT 2011; root:xnu-1504.15.3~1/RELEASE_I386 i386

The I386 is a hint you're running the 32-bit version, a 64-bit version
would probably identify as x86_64, AMD64 or similar (dunno about Darwin).

Therefore I propose to use no larger numbers than 12000 days for
expiration periods for now.

Best,
Matthias
-- 
_______________________________________________
leafnode-list mailing list
leafnode-list@xxxxxxxxxxxxxxxxxxxxxxxxxxxx
https://www.dt.e-technik.uni-dortmund.de/mailman/listinfo/leafnode-list
http://leafnode.sourceforge.net/