[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [leafnode-list] Not picking up some new groups?



John Carlyle- Clarke schrieb am 2004-01-27:

> Looking in /var/spool/news/leaf.node I am puzzled by what is there
> (although I don't know if it is correct or not!).  There is an
> ":active.read" file dated Jan 22, and a "groupinfo" file dated today.
> There is a file for each server, also dated today, which seems to
> contain the interesting groups for each server, rather than a full
> list (although I notice a couple of groups seem to appear in all these
> files - even on servers that don't carry them).  There is also a
> "last:servername.0" file for each server, dated Jan 22, size 0.

The mtime of your newsgroups.borland.com, determines which date is sent
in NEWGROUPS. Was any action (restore from backup, manual edit) taken
that might have changed the mtime outside fetchnews?

I just checked if the server time of newsgroups.borland.com is exact, it
is. Is your system clock exact? (ntpdate or ntpd when going online; ntpd
or chrony when "always-on") If it is ahead, fetchnews may miss new
groups.

> If I telnet into the Borland server (newsgroups.borland.com) on and do
> a newgroups, I get this:
>
> 200 Borland NNTP service: Posting allowed,  5.7c2,, S0 newgroups
> 040126

While this is not the right syntax, giving the proper syntax as leafnode
will do yields the same result from the server.

> Although I see some new groups on some servers, I see nothing on the borland server at all.  Here is a sample output:
> 
> [jcc@pc69 news]$ gzip -dc news.all.1.gz | grep -E "found [1-9][0-9]* new newsgroups" | more
> Jan 26 08:01:37 pc69 fetchnews[19395]: news.gmane.org: found 1 new newsgroups
...
> Jan 27 05:01:46 pc69 fetchnews[22579]: news.gmane.org: found 1 new newsgroups
> 
> I am running version 2.0.0.alpha20040119a.  On Jan 22 I added an only_groups_pcre line to one of the servers in my config.  Here are my server definitions.

This is unrelated.

I am attaching a new README-leaf.node file that is to appear in the next
snapshot release, I hope you find it useful. If you (anyone) have (has)
further questions/suggestions as to the file, let me know.

-- 
Matthias Andree

Encrypt your mail: my GnuPG key ID is 0x052E7D95
README-leaf.node
================

This file describes the files in the leaf.node directory.
(C) Copyright 2004 Matthias Andree. GNU GPL v2

1. Persistent files
-------------------

":active.read"
	tracks the last time when the active file was completely
	refreshed, either through fetchnews -f or because
	"timeout_active" days have passed since the last refresh.

	fetchnews checks the mtime of this file, which, in regular
	operation, should never be older than "timeout_active" days.

	this file is always 0 bytes.

"groupinfo"
	contains the list of newsgroups that leafnode knows about
	and is both large and preciuos

	The syntax is, in Pseudo-C:
	groupname TAB mode TAB high TAB low TAB timestamp TAB description

	groupname and description are self-documenting, timestamp is the
	UNIX time when the group was added to leafnode, and mode can be
	n (no posting), m (moderated), y (posting allowed).

	low and high are the lowest and highest article numbers (natural
	numbers) in leafnode's group store. These need to be tracked
	explicitly because the "high" number must be monotonically
	increasing.

last:SERVER:PORT
	tracks when the last _full_ fetch of the active from SERVER:PORT
	was completed. A PORT of 0 means looking up "nntp/tcp" from
	/etc/services.

	These files are empty.

SERVER:PORT
SERVER
	these files store the "high number" of the upstream server that
	fetchnews has last seen, for the interesting groups. A missing
	:PORT means looking up "nntp/tcp" from /etc/services.

	These files contain lines with the following syntax:
	group.name SPACE high

	high is a natural number.

	The mtime of this file is used to build the NEWGROUPS request.

2. Intermittent files
---------------------

SERVER:PORT~
SERVER~
	these files correspond to SERVER:PORT and SERVER and exist while
	fetchnews fetches news from the server in question, these
	contain checkpoint information that is read back next time
	fetchnews runs, in order to not re-check XOVER information that
	has successfully been fetched before.

groupinfo.??????????
	temporary file that exists for the few seconds while the
	groupinfo is written back to disk. It is renamed to groupinfo
	only when successfully written.

lock.file
	is present while a leafnode process that can modify groupinfo is
	running (everything except leafnode-version, newsq).

	It contains the process ID (PID) of the process that set the
	lock file on the first line and the host name on the second
	line. The second line is there to avoid cleaning "stale" locks
	from other hosts, should the spooldir be shared by some hosts
	(via NFS, for instance).

	It can be removed manually when the process ID recorded in the
	file does not belong to a leafnode related process (i. e. after
	crash and reboot).

lock.file.??????????
	this file is used to work around NFS non-atomicity in the
	open(2) O_EXCL mode. It can be safely deleted at any time,
	although leafnode takes care not to leave one behind if it can
	help it.
-- 
_______________________________________________
leafnode-list mailing list
leafnode-list@xxxxxxxxxxxxxxxxxxxxxxxxxxxx
http://www.dt.e-technik.uni-dortmund.de/mailman/listinfo/leafnode-list
http://leafnode.sourceforge.net/