[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [leafnode-list] How to debug missing (not-fetched) articles?



Reiner Steib <4.uce.03.r.s@xxxxxxxxxxxxxx> writes:

> - Would it be possible to add an option to applyfilter?
>
>   "applyfilter -c /path/to/some/article" would read
>   /path/to/some/article and let the processing act as with
>   "applyfilter -n -vvvvv some.group".  The usual "newsgroup" argument
>   (needed for choosing the right filters) could be extract the
>   "Newsgroups:" header.

You mean: applyfilter -c FILE should see if the article in FILE matches
filters? What group would applyfilter choose? The first one listed? Try
all?

> - Given a Message-Id, let fetchnews (say: "fetchnews -M MID") try to
>   get this article.  *time passes* ... Oops this feature already
>   exists! :-)
>
> So I did "fetchnews -M MID" and the article is there.  Fine.  But why
> wasn't it fetched in the normal runs?

fetchnews -M MID bypasses filters. See the getbymsgid function in
fetchnews.c, near line #539:

    putaline(nntpout, "ARTICLE %s", msgid);
    return (getarticle(NULL, &artno, delayflg) > 0) ? TRUE : FALSE;

refer to lines #965ff to see that "NULL" means "no filters".

    static int
    getarticle(/*@null@*/ struct filterlist *filtlst, unsigned long *artno,
               int delayflg)

> Okay, I changed "debugmode" in leafnode/config to include "Filtering"
> and "Store", run "applyfilter -vvvvv -D 2233 -n some.group" and grep
> the output for "regexp filter:.*matched".  Done, found the
> culprit. :-)
>
> Is the above really the best method or am I missing something obvious?

Well, filter logging happens at "debug" priority. Two conditions:

1. debugmode must be an odd number, "1" is sufficient (else, the ln_log
   library discards debug logging)

2. syslog must capture news.debug logging (I have yet to see any distro
   which does this by default), so modify /etc/syslog.conf

Optionally:

3. bitwise-OR DEBUG_FILTER (32) into debugmode to enhance filter-related
   logging. It tends to get very verbose though.

I agree that this may be a bit too strict.

What modifications to this scheme would you suggest? Log at LOG_INFO
priority rather than LOG_DEBUG? Print filter line numbers to ease
identification of the filter line? There should indeed be an easy way
for the user to enable a "what filter ate my news" option.

I'm attaching a gzipped update patch that bumps Message-ID logging to
INFO priority and makes the filter logging more uniform (you can grep
for "rejected by filter" or, when debugmode is enabled, egrep for
"rejected|matched"). Also see the new "LOGGING" section of man 5 filterfile.

To accomodate GMANE Message-ID obfuscation, I'm using base64 encoding as
emitted by GNU uuencode (GNU sharutils) 4.2c. To decode, use a recent
GNU uudecode or uudeview (my uudeview fails to extract the file
name). ChangeLog:

2003-08-12  Matthias Andree <matthias.andree@xxxxxx>

        * fetchnews.c, filterfile.5.in, filterutil.c, store.c: Enhance
        filter logging.  

2003-08-11  Matthias Andree <matthias.andree@xxxxxx>

        * Makefile.am, NEWS, grouplist.c, grouplist.h, texpire.c,
        xoverutil.c: Bugfix: Expire groups that are no longer in "active".  

begin-base64 600 leafnode-20030731a-to-current.patch.gz
H4sIAIQXOT8CA8U8aXfaSLaf4VdU6EkCZjFiMTZue+J2SMKMtwN4OjmTORxZKkBjIdFabPN68t/f
vbeqpBKLQ3dPe3zSxipV3br7VkX3PZs/ddn5zPSm/MKf5k/++E9+cD5kE8flXbb/YAb75/8Y7rvc
nHi+zauN/WSvykM+4FHg8AfHm7IAPkLH95hRM+qtvO1MJqwas2pAzxqG1Wo1fco1Dfa32GWNer3J
Go1u47DbOmTVOvzkBKByuaxNNxrsLJ6K6cZBt9nqGi0xPf/uHasalSYrGxWjyd69y5dxVrV+WIVF
7NKMopljhuzMswPO2Y9zOVAzaeDddP5Us/lpvpwv5/bYhEfWzOOPYc2qIC8iHiBHau2a46mBOHJc
fB1GfsBrVpf1PMDT4gBATGCuP50Cb2qMIdgEHeM3o3Np3nPa35xX2FXv52GFTQM/XrhOGCEK6cOs
wiL+tHAQowqsffIfFKZd9lM8nTigLj2aIFaFLJqZETPh2fMBY2B0wByPFUwrch54QSDPBPKdavO3
IM/yLHfCQs9chDM/Ihj1TtMw2Um+LxRXI+wFVFdn4zblbR1ue3GUUWpU04BGUaE1yKijqUo3u+3D
rlFPVBrAo0Zn5hsZna53uvWjdP6RVOwDVOwD1Gv2A6vuVdlcgsAHHHtn+d7EmcYBHzveIo7e5as/
sL/07QyTKw9EIm22X+/sA64Jinw+N1E12F/y5W0rj+TKw33AOkFWWwkiP7sdXV+e/b03vr4Z9a+v
hqACE7AQZ+oxG1S0evd/zqIBylZ1vDAyXdfxJj7APgAarBm37qtodwAI6e4cIeHw+5Aoz015BPN5
bca+iqdHEwxtbgb3YKlibB75kwk9lHOakaw8SwBkAdzlVqSWayNizmzMg8AP6L3UWrTAF1BXMvRt
eto42Paik9HTxoH46JCeIsjnfS7ARQWlic9qJkDUXC5pZlk46pCFDvjB1N67+XL19zkf5XoEGLYJ
fuMQ4LMUvrlYuEvpgAM+B/8XSofNbdgpciwXRwJ/DptzFi5832WAhGsGU17bBikGJXXtkM24afMg
ZHccdVpMQQGIaWGNDfjCDyJuK13RIskLqIwet7aG54ax9U0jG7gbhvxskPZo0HONQ02J6t1Wswu+
JAncsAVqkb5gNXSD22lrCxqkTYftJph6GT+OyOLBABn8OBNWRBa7YcTevGH3jusKjqvRCvjDMArG
8F8xLJVK7FdYKawZJ3O7XD7OV3M51xtDVC5eXF1cfxwP3/d+uv1YYeLp/Gww6p9f9Co4L1dIdMaP
I6U37HVYqOCD55eOwZ+sgOtffbheh1ZGaCkA/Bfwf4ODAdh3S6k5rPj5+h+9QUmBF8vmPAzNKXds
2A3IQS7Y/C6ezkHa7A0j9Mcf+hcjWIkTkFU7UQhzaXLhk9DnrqAsw8PjRIWz+c9LqPFKxrVNlbf5
wKwLFB5QOMAVyLkDTYuNbrvVbaRaLDzh6ooNenyQLJE+sdWgeA0fAA7VWABBLK04CLgXuUvmhAwy
Ne6F4OlA6p7NHkFTme/BO/IrzJygZqCXepz5Ls8najgz0QNxCKj+o+f6IEIbPFe+XBt+YiDoj/2r
j/Bwc5MvnymHlzjadd0LaTxFDNNW9JURK5DtYoAu5MsT03JcJ1ruLwLHD+CPGhvN+JJFkCQA1vDf
hHAFxzgPMXFV2F6xHy9Pn9F5nEtJdDchcPMCnPg4A6MEkE5Im6kFXjy/E7EDRyUtIsgQay/lgnz5
UthUtf8+QXglOmDUKX/xY2aZHjPd0GembTPpEBgZIKb1BfRKS5jmcVg7x3Bg88gERbFV5g9WfAfe
w4lwd8tfOH4cEjqQD8euDUEvAjmyOIQlsQdZNPdQlGwULFkQex6AAL4rJ8r8IBOTHp1oRgQUqu+b
TVblBeYvIrAAoNRnIWTmYWzNkioEYykqSO/z2eXNRW+IzhV1JGPlolR4MROXNdTWJOdoy4tmPZvk
HNEHjKY2LkDnoBZMLLxeh5jTbXXSHOZIM3G14Dn7boo6s9Vso4HjB+RIaOD48y2P/1CXTdAjkDfk
t5DSmgHby5ehRrJm6Fkd0Et6wBhVxnVAYhx4TIyyv7KC7QjVoIEC67IC/cXtAgSdbyjH/T3Ycg9W
mjbpk9QImZ3U6OVAgI2CmKOmKkuRqgdqJwIjlLOg4zQF9hTkHTSwhi63DjoVQziwHFBG8ac4rZ66
zhxU+uQEPF0JwzGOgSsLZNhFmgQtJ6y4sKAg4U/cUpOggL29uKiwmQ1/IjdKwBWXe0UYKInYS9xk
9Qr+E5PrJXZ6Ar8p6G58TxFSpQpbg2SC33cjJUAFXFj2pxDwKdAg2Q2K/TrcF5ETiAPmAo/5E+Ui
UpJKbijDrFRLlI4gslIZBOgxuBr0t8UpcYgIXkUCWbobCqhvArzgXLIdCq++fU/Bo2+SAUqWr+iP
Y1A+8LMYE01IwskJQWoM2mchw/f2UVkYlypVBBQsM+TWfFHMIgi8eQI/XChJXJRksApgpsJhAftO
HA/wk7pdLLyHkq9bUKiSvh4amDW2DtuqTswpnG/OB71xbzC4HoyvoCodnX86zgH+tu+9lZLA8GLz
iRm7kUD+T1ciQXgiQfwbwsNrF0xRWBb9KbwCBIbXNiaFGwApS5TC3qgru+0l0trK+vqVLTL6pJRk
d3Fj6R5uFrjL0JtIcR61SZxHSdmvNnoJ0RCSCcPE0/fEsw7Jrewqnh33QxGtA1jf5g+JyPF2FFG7
jrIpt43my4tIIvliItp5vxcRkfl0t4x2ElGDSul2o/O/sCJCMmGZePozrWjH/V5IRFbgh+HCD6Pt
YlJSanUqHZBS24CPREr/RemwTdxK8Es4hvWFbMmt8qywMQ/Sg9OOYqIoszuvIZWWpYnWxv0vVyY6
ZFFQXGXGsJzYt/nDvhe7bs5gf4OK0Djq1LGUoH/y8AmrCG3d851TgyRfr9TprKpFBUQZMvqyCLla
zxp2h5KSB+CDOMMxLFlN11XdU1nuUg+zJtef+4tl4ExnESuelxCD9dMvrG9p7mgGNuI6d4EZUD9i
gsc5oT+JHs2AH1NxizUw1Mawd+BALcuxmIWKY98PJAxQUWeyxOHYs2W7AlEOVXn98eqWXUDRDe8+
co8Hpstu4jvXsSSAC8fiHlgVoLfA8XAman5c+wERGkqE2AcftjCxwq1tJSHF1FbsmfkLLvofgCT1
WETNPYndioSCVfrP/dGn69sRO7v6wn4+GwzOrkZfjim/xQ4gf+ACmjMH6QBwwCgwvWgJZEogl73B
+SdYdfZT/6I/+oLl+of+6Ko3HLIP1wN2xm6o/3Z7cTZgN7eDm+thr8bYkHPFJsWQTcxK2DQBsFqz
IUxZgS0LWd3NzAfs9FjcecC+BvYelr9ZHuSrZJMhZfKxrBQr7DFwQB8if11SElIqrwrre1atwtpH
bMSBgZzduFA9VNgwRhjNJlRyP4E/wpmXZ1DbNQzDqBrNeoex2+EZwIPUHOj8wfEsNwafWNCOcgr6
OLUwsd2Bw9qLH2MPJtu12ak+GC7D/Wi54GF2vADjEZ9nQdNkrO1XYcj9VgZt4JcYhOoC7feKvDMz
9GkmikEAxOEJTGWj3nC0AsrxaYreUSCbt50AHO0PGJR07EVQENhzqJ0mosNGXQlqPyxDMGtYjFEs
tiImSCAHs7cHHlrfaBEA+k8VWhj5ILoH7pbyZdm7eN8fsD0b4hnYGEEs1LC0pXcSOIxiL27P5nIc
IUHJyXEZyH2CWgKT8OlYKbNgTNFVsOSzQKaUTKNAiSEW2wDUUMkwMTdZAGHRBOgE1xRALAO6sQZH
bAkxK/KDJXa/K9Qw8yCEMzrv++oV8Gg/oIci/PZ8KqIls1c6/1BhJmF3dH1D3fute81X9yoIwEJS
OdkOwjyuLIIhfTzO8LC3CHkBMAobP8htu1RirzTqc4lATfyFUZgMAsRIPMup1AXSFptXT+2xZ87B
DAs1vCfw4PDHAgGsy/D95g3ywJoFmdlva29XtsW5oDSR48UoZQjeYiulMArQ6rYu2HyZFEDLlDaD
w06AOJ+AXKXYqh8dUJKBu7hIbgbwmzACzkx9cE534CTVRKl1sA8iNBz3h4PeR+BHLYzGmGqV0u2V
ghoqkwHVQxWPfDSTIhhJqot6XohAwSg2A9VUPYGLiwQoxVLEbS8devu1/raEClV88B27JHjwYAHF
C4UEChDUVecAyUbtIXU2A8FaLBFAuqakY2TNSL3Sl5p0RBsp8SGufm60KFEzbt0SBf2r5phC+Qrp
4VdQggk10GtkflnclOVtOOVaNb9nAG8AK+2O4GZUhvQul7KjUAPntqJWZKFC/Os0r7uf2TpK6H1W
yH3W8+S+R/uWTebbSZduRnb7UJY2dyHxBHEebzBHpL37/HShaRbYNxd+Sr3QvRvxdx2A5vQ2gVEq
+kqYKBjLK2VXQj+fsVQNtESkLlvqkIC/wyT/3d4+2xAV8+Upj8ZJ0iHwSqLghgVTJow5E6qEZFTo
LpUUEnKimpYq75spyBPkZgivjMROs54XU3bCaEos1rZdp3WqaM2oKYZjKjZPRPgVBxSORycVZjC1
KjIT2IOHh+TUYgPNkBPu/QL/uRmiEQQ7RRJ+ZYp0wBKB/dP417HCEYviLI8zwnZBkxdxFBYLQqFl
fEa7UvPICDE9LkYAyz1m0TGTGoELo+qpQJYY9QtMgREMwseSjcWI3uDiX57TFWk2a8Xp7E8rTmcb
itPZ7yxOZ7+hOKXGEWqLh+rycXB9e3PRH47Gn9KsNjOYSULlWZ9MxJ83L7bBukiJsryWVy1f4HhS
7rT9mmBr24t29ppgS3y0SVoS6urlmWaz20wPJVvi0qua++x5JMBFcTXrLewiiQ9xAEIdF3mqHgXp
UT8gKU5SmAgQmVsf+EZ0YdRxn37TZn0mWld191sn5Q1TN9+ZWSFAuz2zfisAKZA3ZMhWLZFdsdV4
LnUouR77AlqUXsXdpkfNzrYXhxk9Qv0IaBT1KIGbuctn1LvtTred2jIAR01KZz9r+QCbdKmJR9vw
W2gSnYnl2Ra73lKPa9OtwInmopRmmypUbRAvjpi0ni7RNOuo00ZTuiEmC1j0DYokBDKOsT9/X9Rr
VpRItoqFiTILyGP/VIUVuRbnl9iPIsGtPpNgkbeDkCzWgT7uvw5lapXsQPfDeVKcUAKHMbt3dd27
GpU2WsHv3oNl8k/27P22LGBs3iLoZ8CKK7ht8i2d9kGiEcz2Bf9J6F3GtSudUmM0YeWrmemq55B0
nCFXqrDImfNxJCEBj3ZYoovXW4OA3BBk7O+xD/3Pl70ulNDUIoRYQ9fi8YoNix4di/9VvaIETWwB
LwX9B22i/6Cj6Icfka5ifXjPF5HoXKTbhffOgvH5IlqqRi2wpJqkMukmeHQg8v0PZxfDXqmkJTza
LC99z2R7QN/uzgwm6NeBLuvRhmzXv3P5nDYlAg6bRMBhWyMAluGxOoj+sUp3qOkaycyZzuQj3akm
UVLh5gQhknl7cX31cXx59lmqiGuGkvqqqG8h+XJogBGL8VrBP51/1ehOI2avDljZkwXpOuRm5bKT
ZJP6ynQOrtgAJ1kqYlRmBn8CHxSmhyTi5ihif7oBlLwnqshbn3Asr6EfEAuPDjUWbj6k6X2+6Q96
pZ1MMT2cKaCJSzN67cZoko8sFYQYQ+msDOqHZokuCTgVIbQKyUg/j/G2zUiokoj8mJxcJTqXuoVD
w8C7T+VDo7lyBwp/ohl2qCi3O2F3sePa43SoKISrdoz8yHTF2xBm0zs5G5L2ZFVJ6li8sIEFWBxJ
glX6n77wFGhx/xsbRpC/pKAUA3Sat0pyR7eayoFkqegh4rpCfpCJCL/RFaJLJJbKZve1nqY9Ogcr
2xmY+nXNh23mNEm4cVBpNkHCjcNK6yCRMFZKujAz69aFT46xKvVT2Rl9HqcukdzIKR0MoiqyX/GO
DmatQLblB9g6ZY4Xmh7XYgB9bwQda06uw6LPpKangjsV5d/6zrJv+Ee2FTJb2zr3LcNkUq1gCcFk
3SVkUt+NHkHeAU0OoQBSFYOWdiRboAMtIUtUDfLj95wvxIO8/rtZ2XZdquuajHwVEfiq6lnJnnpU
uxCaNRd9w3SLAryEF7iTRkAWBd2z7QzKWweSSOsVDZDVb8gytUY5YIRNHNLvXBqyRZeQrkUKBVc/
RScc0705HkYg1MR9yc7qf/6zPt3GjNiLVqaWshOFjtPdzJUdPLlAHsfJH8RMg+0lUMs51afeQPjF
2XB0cz0c9a8+poTnZCohdSg55CBNEgnQwqTTH/XK4SIZymlHGsGcvDn177JnAJQVHB/LtvPa2QYO
ajxPrjhqAOXSJLJpyfiod3kz+iInfDfnp0m5ggC9sbcqmYLz7sAB3h+rjb9lWsl1pJEOLKiUqmEt
VZAnFmqFXC+/8vFKJHZEEGI1vjkbfcIkTH3NZZd6Ql2sLUhgtVqt8toupTdwqN0sSdH3SK/c/o6l
Erkkf0A+VFeElrnyoSUYB/VKowXhp9PGMKTCD1QfiQlStVJj19GMB48OVECxF8Z3IZScd4CVTL3J
yXkc8/07nsCQ5l9TA1p+/kx6rrLVlWmePoG+lkiJZNFYibgYOsUbfIEXXKqZOuk7JU9JP8TNl7XZ
oexHY/Wq9Wa1xd40eyirwz1+tqG7sS8r4rxecSVh3sMW9FQGd2nhMAZex0vcGBbZMsmEwOn79/Fi
LJ6LXprVVXPZKtCbaolbNedN8TtdmQ7xr989mE1PgO8i0/HkN0SQLOE/ZKeXxJn2gJMaRess418Q
AE/pHDf1wZpnpbey5VzacIwylXebBXXa7AoT33+s4AV7nLmFV9mOdpZZWFOr16v5ruxqJ0cHySm7
pH40uO1JBRVfwzXqbcryy0a905Ffr5f1pweFygmpQlEeOUo90JER5CR9el1xFU50mUTMS8YQQTmk
IOQoRZsFsKv5aC4loxhk/mZSrMoxdRCyZnqrMBJ3oqAk3NGDqcKe8l8l4iTtIZLm4dSxkzG6Zk43
/gNwNmYorvm4vnXPuBm4Dt6V8vFs6gEDJH57O6benw/FPHo0QAfKFy8twoWRazEZYYlWVz39tp72
jf8X6HBqu23vlde3vTCyvfK6+DCox6lBpn656SVf4sGWefqN5ZY44dDnP/+VenHE0WpTW0R8UM9c
SNYJyCZsl0ITmrTSj5IMVXRWlAyiK9LOh9TfRur6E3UQPQ35VTUQM8idEiLwQ9mYJv+nEujn/eTL
bKAG/w/IZePZ70MAAA==
====


-- 
Matthias Andree

-- 
leafnode-list@xxxxxxxxxxxxxxxxxxxxxxxxxxxx -- mailing list for leafnode
To unsubscribe, send mail with "unsubscribe" in the subject to the list