[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [leafnode-list] Why was support for 8 bit characters in headers



"Peter N. M. Hansteen" <peter@xxxxxxxxx> writes:

> One morning last week, after upgrading (apt-get update; apt-get upgrade)
> my leafnode to 1.9.24, was a bit baffled to find that my followup to a
> no.it.tjenester.news.diverse thread with "Subject: Re: Mail etter
> postinger på no.test" was rejected by my newly refreshed leafnode with
> a "441 format error" message claiming illegal use of 8-bit characters
> in the header.
>
> I am aware that the formal standardization of 8 bit and wider characters
> in usenet messages has been slow-moving to put it mildly. However, in
> the meantime the regional hierarchies whose languages require 8 bit
> or wider characters have adopted agreed-upon practices. I've been using
> leafnode here since version 0.7 or thereabouts, and it has always
> handled the conventions of various regional hierarchies without any
> sort trouble.
>
> Until 1.9.24, that is. I'm hoping leafnode doesn't stay broken.

Leafnode is not broken, but pickier now. The current Usenet article
format is based on RFC-822, now superseded by RFC-2822, which still
declares 8-bit characters in headers illegal -- and rightly so, because
there is no agreed character set other than US-ASCII.

We don't want fetchnews to send garbage out, and you can use RFC-2047
encoded headers to send your å (aa) character out ("på" then looks like
this: "=?ISO-8859-15?Q?p=E5?=").

The very moment UTF-8 will be ratified as official RFC (it's in the
development process and an Internet Draft of the USEFOR group), we will
allow 8-bit characters again. Feel free to bug me should I miss the
future release of the RFC that supersedes RFC-1036.

Patches to validate if a character string consists only of valid UTF-8
characters will be accepted against leafnode-2, but beware of security
issues; with UTF-8, characters are not always what they seem to be at
first glance. There have been security discussions some weeks or months
ago: about Cyrillic characters that look the same as Latin characters,
but have a different internal presentation. I believe that was about
domain colliding attacks.

-- 
Matthias Andree

-- 
leafnode-list@xxxxxxxxxxxxxxxxxxxxxxxxxxxx -- mailing list for leafnode
To unsubscribe, send mail with "unsubscribe" in the subject to the list