Re: [leafnode-list] Re: How to debug missing (not-fetched)

[Matthias Andree <ma@xxxxxxxxxxxxxxxxxxxxxxxxxxxx>]

Reiner Steib <4.uce.03.r.s@xxxxxxxxxxxxxx> writes:

>> What group would applyfilter choose? The first one listed? Try all?
>
> What does leafnode do if an article matches the filter only in group1,
> but not in others?  I suppose the article is stored only in group2
> (and group3, ...)?  So I think "Try all" (for all in
> $interesting_groups) would be the best.

Hm. OK. Given that this is a mode that needs to produce human-readable
output, we can just iterate over Newsgroups: so the user see into which
groups it would have gone.

>> Log at LOG_INFO priority rather than LOG_DEBUG? 
>
> I don't have strong preferences concerning this.  The main problem is
> that filter/store debugging gives too much output.

Well, yes. That's why it's called "debug" mode. We need to identify the
useful bits and promote them to info or notice so they are of general
use without drowning in the other debug stuff. grep or less is a
must-use currently.

> Yes.  My initial idea was to extend applyfilter, because it probably
> already does most of the task.  But a combination of "MID-Logging",
> "fetchnews -M" (this step should be eliminated) and "applyfilter -c"
> is okay.  But it should be a simple command.

We have shells to hide complexity :->

> - "MID-Logging": Confirms that it has been filtered
>
> - "fetchnews -M": Get article by MID.  It would be good to be able to
>   eliminated this step.

Can you think of a way to do that? You may have an obvious idea that I
don't see. There could be a) a global option that just prints filter
results without applying them (easy) or b) awk the logs an run fetchnews
-M.

>   Concerning `special' output level: -c should produce an output like
>   "article NNN <MID> rejected by filter (XOVER), line X in
>   /path/to/filter" or similar, without other `noise' (debugging
>   output).  More debugging output could be added when
>   additional/several -v are given on the command line.

OK.

> A stdin-option for "applyfilter -c" would be nifty in Gnus (or other
> newsreaders): Use `^' or `j' to get the article and pipe (`|') it to
> "applyfilter -c" and see why it has been filtered.

How do you retrieve the "hole"? Is there some Gnus function I missed?
gnus-try-to-fetch-parent-from-all-servers?

> One remark (maybe that's not obvious for others): Simlilar to what you
> mentioned about news.debug, I had to add "news.info
> -/var/log/news/news.info" to /etc/syslog.conf to see those messages in
> the logs (SuSE 8.2).

I've looked at a machine with mostly vanilly syslog.conf and can confirm
that. I've added a section to README{,.html} on syslog
configuration. (Will ship with the next snaphost.)

> | store: article <00005fe70a0b$00006df0$00004c95@xxxxxxxxxxx> rejected
> | by filter

> Why "store" and not "XOVER" here?  Does this mean, that the article
> header has been fetched?

Rather the whole article.

> | Xref: main.gmane.org gmane.comp.graphics.gnuplot.devel:18 gmane.spam.detected:113141
>
> I'm using leafnode-2.0.0.alpha20030731a plus your patch from
> yesterday.  Is this okay

The version is fine.

> or shouldn't it be filtered at "XOVER" level?

That's a violent (if you have little bandwidth, that is) bug. Counting
the newsgroups killed the Xref: line of the pseudo header passed to the
filter - therefore, Xref: based filters were practically unusable. (The
Xref: header was chopped after the first group name, and the gmane spam
detection group was appended, thus cut off and never seen before the
article had been downloaded).

Try this patch on top of what you have, it works for me.

I think I'll do another snapshot, 20030814a.

Index: fetchnews.c
===================================================================
RCS file: /var/CVS/leafnode-2/fetchnews.c,v
retrieving revision 1.122
diff -u -r1.122 fetchnews.c
--- fetchnews.c	12 Aug 2003 16:33:55 -0000	1.122
+++ fetchnews.c	13 Aug 2003 23:58:18 -0000
@@ -793,7 +793,7 @@
 	unsigned long first, unsigned long last,
 	/*@null@*/ struct filterlist *filtlst, char *groupname)
 {
-    char *l;
+    char *l, *xref_scratch;
     unsigned long count = 0, dupes = 0, seen = 0;
     long reply;
     struct stringlist *helpptr = NULL;
@@ -834,10 +834,11 @@
 	bytes = xover[6];
 	lines = xover[7];
 	xref = xover[8];
+	xref_scratch = critstrdup(xref, "fn_doxover");
 
 	/* is there an Xref: header present as well? */
 	if (xover[8] == NULL ||
-	    (num_groups = parsekill_xref_line(xover[8], &newsgroups_list, NULL, 0)) == -1) {
+	    (num_groups = parsekill_xref_line(xref_scratch, &newsgroups_list, NULL, 0)) == -1) {
 	    /* newsgroups filling by hand */
 	    num_groups = 1;
 	    newsgroups_list = critmalloc(num_groups * sizeof *newsgroups_list, "doxover");
@@ -850,6 +851,8 @@
 	    s = create_pseudo_header(subject, from, date, messageid,
 		    references, newsgroups_list, num_groups, bytes, lines,
 		    xref);
+
+	    free(xref_scratch);
 
 	    if (filtlst && killfilter(filtlst, mastr_str(s))) {
 		groupkilled++;

-- 
Matthias Andree

-- 
leafnode-list@xxxxxxxxxxxxxxxxxxxxxxxxxxxx -- mailing list for leafnode
To unsubscribe, send mail with "unsubscribe" in the subject to the list