[Date Prev][Date Next][Thread Prev][Thread Next]
- Subject: Re: avoiding copy - Luajit FFI
- From: Fabio Kaminski <fabiokaminski@...>
- Date: Tue, 7 Jun 2011 14:00:37 -0300
On Tue, Jun 7, 2011 at 1:12 PM, Justin Cormack
> On Tue, Jun 7, 2011 at 4:58 PM, Fabio Kaminski <email@example.com>
>> On Tue, Jun 7, 2011 at 12:53 PM, Justin Cormack
>> <firstname.lastname@example.org> wrote:
>> > On Tue, Jun 7, 2011 at 3:54 PM, Fabio Kaminski <email@example.com>
>> > wrote:
>> >> Hi Folks,
>> >> Im trying to get less memory copy as possible with luajit FFI, and in
>> >> the api to get your hands in a (void *) or (char *) pointer
>> >> is copying it through "ffi.string()" wich in turns, will create a lua
>> >> string..
>> >> my first question: why cant ffi.string() reuse the given pointer.. vm
>> >> design? avoid concurrency races from the C side?
>> >> the c api "lua_pushlstring()" works the same way?
>> >> second:
>> >> When you simply need to push a string or even a file buffer thats
>> >> fine, but if whe are talking about things like byte buffers over a
>> >> network
>> >> those copy would make it loose a lot of performance in the long run..
>> >> even creating a bytebuffer on the C side and passing only the
>> >> requested byte string on the lua api side..
>> >> Anyone has found a way (writing c logic or not) to avoid those copies?
>> > Only call ffi.string when you absolutely need to, the rest of the time
>> > you
>> > can just pass around the raw buffers. You can just allocate them with
>> > ffi.new.
>> > Justin
>> Yeah, thats the way to go.. create more C abstraction (and black
>> boxes from lua app perspective) and try to switch C/Lua VM the least
>> as possible..
>> no skinny C interface for now..
> It is quite easy to accept either strings or buffers in your API, or have
> optional buffers, which will be used if they are provided. That way for
> performance critical parts the user can provide buffers, for other cases
> they just use strings. This also helps to avoid allocations in fast paths.
looking from this perspective the ctypes from ffi do an excelente job..
like buffering mechanics, without have to write C lua api code for that ..
something not possible with immutable string design.. thats a win for sure..
but memory pointers are the digital currency of kernel/userspace
switches, or even third party apis, and my point of view is more
in that way.. get some constantly feeded buffer from C userland..
but looking in the ffi api we have this nice len argument in the
str = ffi.string(ptr [,len])
So in theory you can break ptr in parts, and since you are allowed to
do pointer arithmetic in ptr
you can advance the cursor inside a lua context..
that's fast and can minimize the copies..