[Date Prev][Date Next][Thread Prev][Thread Next]
[Date Index]
[Thread Index]
- Subject: [ANN] push-decoding of bencoded messages
- From: Jorge <xxopxe@...>
- Date: Thu, 16 Jan 2014 14:11:11 -0200
Bencode is a lightweight serialization protocol, with several
interesting properties (I found a nice writeup on bencode here:
http://jeelabs.org/2012/10/05/whats-the-deal-with-bencode/).
Usually, what you do to work with bencode is to go with
https://bitbucket.org/wilhelmy/lua-bencode/ .
But, suppose you have the following scenario:
* You are reading rather big objects in the relation to the total memory
you have.
* You are receiving rather long strings, but the reading is made in tiny
chunks (say, over a slow serial line).
* You can process said chunks as they arrive, say, compute a CRC or feed
it to load()
(Bear with my, this could happen)
In these cases a push-parser could be more adequate. There is a
coroutine based example at
http://jeelabs.org/2012/10/18/push-scannin-with-coroutines/, but it's
blocking.
So I wrote mine own, available here:
https://github.com/xopxe/lua-bencode-push
Internally, it's a tail-call based state machine. It attempts to fiddle
as little as possible with strings (no concatenations). It returns
string in fragments as they arrive, signalling the end with an empty
string. It also pushes the total string length prior to pushing the
fragments, if you're interested in such a thing.
It misses any kind of error checks, and no encoder is provided (yet).
Greetings,
Jorge