Re: Fleece: fast Lua to JSON

[Henning Diedrich <hd2010@...>]

On 3/20/11 2:48 AM, Matthew Wild wrote:

On 20 March 2011 00:49, Alexander Gladysh <agladysh@gmail.com> wrote:

On Sun, Mar 20, 2011 at 03:33, Henning Diedrich <hd2010@eonblast.com> wrote:

Fleece is a ~ 10 times faster Lua to JSON converter.

The first question that entered my mind was "10 times faster than
what?". 

I guess I blurred that sentence to make it clearer.*

It's ~ 10x faster than the fastest ones I found, LuaJSON and Lua-Yajl.

That search was cursory but not hasty. I tried LuaJIT but to my understanding it can't accelerate LuaJSON or Lua-Yajl, while JSON4 just started out too far behind for LuaJIT to make a difference. I may be off here as I lack experience with LuaJIT. From what I see, a LuaJIT version is in the wings.

Fleece is up to 100 times faster *sometimes* than JSON4. It's hard to start from there and my (brief) LuaJIT/JSON4 experiments did not convince me. If there's a better combo, please let me know.

At the upper end, Fleece is ~25x faster than LuaJSON, ~40x f. t. Lua-Yajl, ~60x f. t. JSON4, for integer arrays.

At the lower, ~3x f. t. LuaJSON, ~6x f. t. Lua-Yajl, ~20 f. t. JSON4, for associative tables of escaped strings with a high frequency of control characters. Which is as bad as it gets.

The preservation of speed in the face of more challenging types of data is 'dynamic': the loss vanishes proportionally, when less control characters are encountered, the fixed overhead is minimal. Not by coincidence, it's made so. Plus, there are a number of different options ('E0'-'E4') controlling how to tune that case and a compiler flag to use x86 inline assembler to re-gain around 10% of speed for properly escaped strings.  This is demonstrated in this benchmark sample, that also encompasses Yajl:

http://eonblast.github.com/fleece-lite/test/SAMPLE.html

(thanks to Vadim Peretokin).

If you can live without them, common sense would dictate to do the ctrl char checking on your strings only once, when retrieving them, way further up front in the pipeline than where Fleece kicks in. And then let Fleece be ignorant about them, do the next error checking in the browser where for all practical means it doesn't cost you.**

Anyway, the overall performance advantage of Fleece could be around one magnitude, which would be an enabler.

Henning

I think the motivation is fairly obvious :)

That's what I thought. I also wrote it, to be sure, 3 sentences into the README or so.

Henning

*) noise reduction measure: this sentence has multiple layers. The author is aware of the interferences between some of them and they are what appeals to his sense of humor. It is also true.
**) just don't bother please.