Re: [leafnode-list] getaline() (once more)

[Matthias Andree <ma@xxxxxxxxxxxxxxxxxxxxxxxxxxxx>]

krasel@xxxxxxxxxxxxxxxxxxxxxxxxxxxx (Cornelius Krasel) writes:

> Matthias Andree wrote:
> > I did not see what was wrong with my getline/getaline implementation,
> 
> IIRC you suspected that your getaline implementation was very slow.

Better slow than broken ;)

> I must confess that I have not benchmarked it. I tried to use GNU
> getline() (from the glibc of SuSE 6.3 which identifies itself as
> "GNU C Library stable release version 2.1.2") and observed the symptoms
> mentioned in the ChangeLog when running the test suite included in
> 1.9.17ma3.

I'll forward port the test suite (might require perl though, since I'm
not intending to keep everything that's a fingerpress in automake
available to Solaris-/bin/sh-boxes.)

> The reason I changed getaline() again was that I had 2.0b5 exit with
> a huge memory allocation (i.e. proper exit code by critrealloc())
> when retrieving a certain XOVER record from upstream. 

I'd have liked to have that broken record. 

> It turned out that the References: in this XOVER record were truncated
> by the upstream DNEWS (or possibly on the server before) and contained
> a NUL byte. getaline() had some difficulties identifying this (or the
> "real") NUL byte and therefore continued to reallocate exponentially
> increasing amounts of memory.

getaline() does not identify anything like that.

> 1) I forced getaline() to always start with 256 bytes of RAM. This
>    is IMO meaningful because most lines will not have more than 256
>    chars in length. Therefore, we conserve memory.

Hum. This may be only useful if it had allocated more than 4096 bytes;
even while your malloc implementation may pool small bits, there's
nothing wrong with using an entire page.

> 2) I switched back to the 1.9.16 version of getaline(). I found out
>    that this version is very similar to the corresponding function
>    of the tin newsreader. NNTP traffic is characterized in that a
>    line always ends with \r\n (or, if reading from a textfile, with
>    \n).

You don't read NNTP from text files, since those tend to not interact ;-)

>    I tested the 2.0b6 version of getaline() against the getaline()
>    test suite of 1.9.17ma3 and found that it failed test 5 (IIRC),
>    namely it will not read a single NUL byte from a file. I think
>    this case is not relevant to either NNTP traffic or the reading
>    of textfiles.

Oh well, if it breaks, it needs fixing. The question is: is it GNU libc
2.1.2 (SuSE 6.3) that is broken, or is it get(a)?line.

> I agree that this version is still not satisfactory in that it cuts
> off chars after encountering a NUL byte. In the case mentioned above,
> this apparently did no harm (probably because tin could live without
> the missing Xref: information) but I don't want to rely on this kind
> of luck :-)

I don't want to rely on luck either. This is leafnode, not Poker ;-)

-- 
Matthias Andree

-- 
leafnode-list@xxxxxxxxxxxxxxxxxxxxxxxxxxxx -- mailing list for leafnode
To unsubscribe, send mail with "unsubscribe" in the subject to the list