Re: [ANN] nixio 0.3 - System, Networking and I/O library

[Steven Barth <steven@...>]

Am Montag 13 Juli 2009 10:20:57 schrieb Miles Bader:
> Any reason for the duplicating the functionality of luafilesystem,
> luasocket, etc ( which AFAICT seem well regarded)?

I understand your points and I really don't want to insult any developer of 
the mentioned Lua libraries.
Even before writing nixio I by myself relied mainly on the libraries you 
mentioned. They worked fine in most cases and really met my demands for over a 
year.
To clarify: I mainly work on embedded devices with 4 or 8 MB of flash so sizes 
of libraries really do matter. Everything is being cross-compiled and I have 
to take care of the additional Lua libraries for the project working well with 
the cross-compile toolchain. So each new library adds an additional 
administrative overhead over time.
And as the project evolved the problems I had with the "classic" Lua modules 
began to increase, mainly:

luaposix
* It does not cover all POSIX functionality that I need. A co-developer began 
writing patches to add functionality. I think at least one of them - the 
crypt() function - went back upstream.
* When it comes to third-party developers wanting to join the project - the 
lack of documentation was a relatively big problem.
* As we had a cross-plattform project in sight this could be problematic as 
well.

lfs
* I used this before switching to luaposix. Works well for what it does. Also 
the documentation is good but unfortunately it has a very limited scope. For 
the POSIX part I need chmod(), chown(), realpath() and so on.

bitlib
* Autotools for a single C-file?!
* Not possible to cross-compile without doing dirty hacks.

luasocket:
* A really nice piece of software, but:
* I need IPv6 support in the near future.
* What has always bugged me is that a receive() given an amount of bytes to 
read always blocked until the whole chunk was available. So when writing 
network software - like a JSON-RPC server or client - where the protocol is 
not line-based and you don't know how big the request-chunk will be, it was 
hard to get the request without fidddling with asynchronous I/O which would be 
clearly overkill in this case.
* I need zero-copy-support (sendfile and splice)  -at least on the server-side 
which is Linux - to efficiently transfer files from a USB disk attached to an 
embedded device over the network.

luasec:
* As I also need TLS support this was the next library I stumbled upon. 
Unfortunately it was - besides luaxyssl where xyssl was discontinued for a 
longer period - the only one that I found and as I said above if you are 
working on embedded devices you are thinking twice about putting the 1MB 
openssl monster - even in a compressed form - on your 4MB of flash memory.

So given the overall overhead of maintaining all these third-party libraries 
in the buildsystem that cover only 80% or 90% of my demands and then writing 
and maintaining patches. I decided to start a new library that has everything 
I need for my projects.


Am Montag 13 Juli 2009 10:24:52 schrieb steve donovan:
> I'd second that question.  What stuff does it bring to the table that
> isn't covered by existing modules like lfs, lposix, md5, etc?

To name a few things:
* IPv6 support
* sendfile() support
* Some other misc POSIX features like poll(), nice(), signal(), ...
* TLS and crypto support on top of CyaSSL or axTLS (or OpenSSL)
* A unified I/O interface (read, write, readall, writeall, linesource, 
blocksource, sink, copy, ...) for dealing with files, pipes, TCP-sockets and 
TLS-sockets the same way.
* Having only ONE library ;-)

Regards
Steven