Re: Ziplists

[Mason Bogue <scythe+lua@...>]

A ziplist is an array of concatenated short memory blocks (<128 B,
i.e. unsigned char) with the length of the block used as the
delimiter, i.e.:

[0x03, f, o, o, 0x03, 0x04, q, u, u, x, 0x04, ...]

This has some dramatic advantages over a linked list for Redis'
use-case, but it can already be implemented in Lua as it stands,
although you might have to write a bit of C code to copy the output of
multiple `string.byte()` calls into a single array efficiently. I
think it's too specific to become part of the language, though.
Read-only zip-lists can also be constructed by concatenating strings,
although I think this is slow in Lua.

On Mon, Sep 26, 2016 at 7:42 AM, Glenn edgar <glenn-edgar@onyxengr.com> wrote:
> Ziplist is not a tar.  It is a compact data structure where all the elements
> of a redis data structure are stored in a compact data structure.  The data
> structure is described in there documentation of saving memory.  Just google
> "redis ziplist".
>
> The motivation is that the memory consumption of pointers dominates small
> objects.  Redis is that the redis database has to fit in Ram.
>
> About 10 years ago I had to fit lua on set top boxes and analog devices
> dsp's.  I found that the average size of memory block was around 20 bytes.
> I had to develop my own private memory manager to reduce the heap overhead.
>
> The embedded lua project introduced the concept of rom tables to reduce
> memory consumption.  The redis ziplist concept is a way to implement rom
> tables.
>
>
> On Sep 26, 2016 5:24 AM, "kamicc olo [via Lua]" <[hidden email]> wrote:
>>
>> ...and the ziplist itself is? Did tar(gz) in memory container myself some
>> time ago. I should publish it later.
>>
>>
>> On Sep 25, 2016 2:23 PM, "Glenn edgar" <[hidden email]> wrote:
>>>
>>> Redis.io uses lua as a scripting language.  For small lists and hashes
>>> they
>>> store the entire object in a ziplist.
>>>
>>> A ziplist might be usefull in the small processor model and for readonly
>>> tables.  At run time a ziplist read only table would require no
>>> initialization.
>>>
>>>
>>>
>>> --
>>> View this message in context:
>>> http://lua.2524044.n2.nabble.com/Ziplists-tp7675065.html
>>> Sent from the Lua-l mailing list archive at Nabble.com.
>>>
>>
>>
>> ________________________________
>> If you reply to this email, your message will be added to the discussion
>> below:
>> http://lua.2524044.n2.nabble.com/Ziplists-tp7675065p7675071.html
>> To unsubscribe from Ziplists, click here.
>> NAML
>
>
> ________________________________
> View this message in context: Re: Ziplists
>
> Sent from the Lua-l mailing list archive at Nabble.com.