Re: [leafnode-list] C++-ifying leafnode

[Timo Geusch <timo@xxxxxxxxxxxxxxxx>]

On Wed, Jul 23, 2003 at 10:45:43AM +0200, Matthias Andree wrote:
> Timo Geusch schrieb am 2003-07-23:
> 
> > Mathias mentioned some time back on this list that he was considering
> > to convert part of leafnode-2.0 to either C++ or Python or both.
> 
> Looks like it's time to refine and mention my plans.
> 
> I think I'll have leafnode-2.0.0 released as a purely C version.
> 
> It already offers some advantages over leafnode-1, local groups,
> filtering enhancements, much faster fetchnews, some more efficient
> internal handling through red-black trees (thanks do Damian Ivereigh's
> nice redblack module which is under LGPL), thread-based expire as has
> always been promised and never implemented, and I think for portability
> versions, I should release such a version.

Makes a lot of sense to me. I've been using the 2.0 alphas/betas for
quite some time and they seem to be rather stable. Any kind of
language change or so would only introduce unnecessary delays and
instabilities.

> > Well, it is my pleasure to announce that I've started converting the
> > nntp server (nntpd.c) to C++. I'll do the conversion in stages,
> > namely:
> > 
> > 1. Replace printfs with ostreams, C PODs with C++ data types where
> > applicable (essentially converting char * to std::string whenever
> > possible). The benefits of this are less chance of a printf-induced
> > buffer overflow, typesafe data conversions in the ostreams and some
> > reduction in the memory management complexity when using std::string.
> > 
> > 2. Start converting the support library to use C++ data types like
> > strings, streams and potentially STL containers. At this point, some
> > of the other programs like fetchnews and texpire can also start
> > benefiting from the C++ support.
> > 
> > 3. Change the structure of programs to take advantage of C++.
> 
> The C++ change isn't meant to just translate from C to C++. I haven't
> yet decided if I want to go with STL either, because it _easily_ bloats
> the binary code.

There's an initial hit but once you're over that the code tends to
grow by the similar factors to C code if you know what you're
doing. Not using the C++ standard library would be a mistake IMO
because it's well understood and decent implementations are available
on most platforms. And one can always download STLport if it turns out
that the compiler's standard library isn't up to scratch (like the one
in VC6 that has a lot of known bugs.

> I'll also be looking at PTypes.

I've had a quick look, looks quite interesting but my objection would
be that any potential contributor who knows C++ would also have to
learn PTypes. Not sure if that's worth it.

Also, I'm not too impressed by the container types either - looks like
one is trading the fully typesafe approach for the 'derived from
common base class' approach like you find it in Java. 

> The stages you mention aren't very much to my intentions.

Well, one has to start somewhere, plus if you want to make proper
use of the standard C++ datatypes like strings, you're better off
using the other parts of the C++ library as well, otherwise you'll be
forever converting C++ data into C data and back. Kind of defeats the
objective IMO.

> I'd think I'd identify how I can represent certain common data types as
> classes, say, the article store, the "active" store and the overview
> stuff. I really don't mind if _libraries_, trivial functions, are still
> in C imperative style. There just isn't motivation to just throw stdio
> out for the sake of it, because some time into the past, the whole stdio
> stuff was analyzed, and string handling often has been changed to the
> mastr() module that automatically extends memory as needed, so I'm not
> having sleepless nights over buffer handling.

You have missed the other argument, which was 'type-safe input/
output'. Plus, most of the standard C++ data types already know how to
write themselves to an ostream.

> With respect to the library: that's the thing I'd bother about
> converting last. If C++ allowed me to simplify code, I'd do it,

You mean, like turning the current getaline()/getline()/_getline()
into something like this:

bool getaline(FILE *stream, std::string &line) {
  std::ostringstream line_buffer;
  int c;
  char last_char = 0;

  while (((c = getc(stream)) != EOF) && (c != '\n')) {
    if (last_char != 0)
      line_buffer << last_char;
    last_char = static_cast<char>(c);
  }

  if (last_char != '\r')
    line_buffer << last_char;

  if (ferror(stream))
    return false;

  line = line_buffer.str();

  if (debugmode & DEBUG_IO)
    ln_log(LNLOG_SDEBUG, LNLOG_CTOP, "<%s", line.c_str());/* FIXME: CTOP? */

  return (c != EOF) ? true : false;
}

Same functionality (OK, the interface is slightly different because I
wanted to make sure that I don't hide the 'C' getaline function), no
explicit memory allocation, plus it appears to be at least as fast as
the "C" implementation on my FBSD server, a humble 400MHz
P2. Unfortunately, it *does* make use of the standard library.

> if the code doesn't lose complexity, I'd rather leave it alone.

Agreed.


> Going into more detail: I've considered storing articles in wire format
> (CRLF) for a long time now so that the server can use sendfile or just
> mmap and write large chunks,

sendfile sounds good to me, but from reading the man page here it
appears to assume that you're talking to a socket. Not sure how well
it would handle talking to stdout, even if stdout is connected to a
socket.

> > Currently I'm in phase 1 and part of the nntp server is converted.
> > I'll do the rest sometime this week and post the code on my website -
> > I'll send an announcement to the list once that's done. The server is
> > still stable :-) and although the C++ binary is considerably bigger on
> > FreeBSD compared to the C version I didn't notice a performance
> > degradation.
> 
> That's one of the STL misfeatures. You end up with a dozen
> implementations of the same algorithm linked into the code :-(

Not necessarily. 

> I'll clean that up when I merge the parts of the code that interest me.

TBH I think if I would continue to work on this then we should come to
some agreement *before* as to what is going to be done. I don't
exactly have ample spare time so I'm not going to work for the bit
bucket, sorry. I also have no interest in forking the project by
maintaining a separate "C++ version" so unless we can work out some
kind of C++ roadmap I don't think it's worth continuing on the
off-chance that you'd like to include a snippet here or there.

> As written above, I have no plans to just translate from C to C++, there
> will be no benefit from that.

I didn't suggest that. But if you want to make use of the existing,
wildly available C++ building blocks, you need to start somewhere. And
that somewhere usually means introducing the use of the standard
library to a certain extent, as a lot of other very useful "utility"
libraries like boost build on top of it or use similar idioms. Plus
you already mention below that there *will* be some benefit.

> The goal is to only use C++ where
> switching makes the code easier to read or maintain, for example when
> destructors save me the hassle of managing memory in each function.
> 
> Having said all that, I'll of course appreciate voluntary help with
> leafnode, and if my pondering about "C++ or Python" may have been
> premature, because it didn't mention plans when that would happen or
> how.

Well it would've been handy to know a bit more about your plans for
leafnode as at least my crystal ball was just in for a service...
 
-- 
Timo Geusch

UNIX/NT System programmer & JAVA wizzard-in-training

'I ask for so little ... and boy, do I get it' Dilbert

-- 
leafnode-list@xxxxxxxxxxxxxxxxxxxxxxxxxxxx -- mailing list for leafnode
To unsubscribe, send mail with "unsubscribe" in the subject to the list