[Date Prev][Date Next][Thread Prev][Thread Next]
[Date Index]
[Thread Index]
- Subject: [ANN] lua-simdjson BETA
- From: Fourier Transformer <ftransformer@...>
- Date: Mon, 25 May 2020 23:31:28 +0000
Hello Lua Users!
Announcing another (slightly faster) JSON parser for Lua: lua-simdjson!
Feel free to try it out:
It does require a Lua build system with a modern c++ compiler, and will only run on a 64 bit system.
lua-simdjson, as the name implies, is a Lua binding around the incredibly fast simdjson library (
https://simdjson.org/). Having worked on a fast lua csv parser (ftcsv), simdjson caught me eye, in it's unique way of parsing. It uses various architecture instructions to do multi-byte parsing of JSON. I noticed there weren't any bindings to Lua, and figured I could learn more about lua/c++ bindings. I spent some time on it and got something going! As this is my first Lua binding, I was hoping someone could take a look and offer feedback on how I implemented it, before doing a general release.
## API (up for discussion)
For now, there are a few methods for parsing json:
* local simdjson = require("simdjson") -- to initialize.
* 'simdjson.parse' and 'simdjson.parseFile' will parse the entire JSON document (either in memory or as a file on disk respectively) and return a normal Lua table.
* 'simdjson.open' and 'simdjson.openFile' will parse the JSON document, but keep it in memory as simdjson's internal structure. Then, you have to use a JSON pointer to access the internal elements (ex: "statuses/0/id").
The open methods end up being a bit quicker if you know what elements you want from the JSON body, and don't need large chunks of it. It could also be used to lazily return elements, but I ran into some issues with nesting (ex: myTable["deeply"]["nested"]), any help here would be appreciated.
I plan to keep it up-to-date with new releases of simdjson, and don't want to add too much onto it. However, having said that, there currently isn't an encoder with simdjson, and I feel like people might want that in a JSON library, so I may work on one in the future. If someone wants to help out, that would be great!
## License
It's licensed under the Apache-2.0 license, inline with the original simdjson.
Let me know if there are any questions!
--FourierTransformer