Re: C arrays as Lua tables

[Eric Wing <ewmailing@...>]

On 2/28/12, Dimitris Papavasiliou <dpapavas@gmail.com> wrote:
>> I strongly recommend you try LuaJIT's FFI interface, which IMO
>> implement C array in a Lua-elegant way.
>>
>> At least, you can see this document and get some idea:
>>
>> See this section: Motivating Example: Using C Data Structures
>> http://luajit.org/ext_ffi.html
>
> I have.  I did in no way mean to say that there's something wrong or
> unelegant about LuaFFI, it just takes a different route.  From what I
> understand the FFI library in LuaJIT depends on the JIT nature of the
> compilation to emit the correct C declarations and code upon
> compilation while the standalone LuaFFI module incorporates a C parser
> in order to parse your declarations.  As I've said this allows you to
> do a lot more than just expose multidimensional arrays to Lua but,
> depending on the circumstances, it might present additional problems
> too.
>
> In the first case you need to stick to LuaJIT which might be a problem
> or not, and with LuaFFI you have a lot of added complexity.  Besides
> the matter of dealing with it as a programmer there's also the matter
> of how you use Lua.  In my case, I need it to script an application
> into which it's embedded.  The biggest virtue of Lua in this respect
> (in my opinion of course) is its simplicity.  I can keep the C core
> which needs to be efficient separate and hidden from the end user and
> expose it via Lua in a simple and elegant way.  Now I know, I could
> probably try to hide the C declarations inside module code somewhere
> and present the user with a simple Lua-like interface for what I need
> to do.  That's a pretty valid approach but in my experience complexity
> has a way of crawling to the surface so unless I really need it, I
> prefer to do without it.  I repeat that this depends entirely on what
> you need to do.
>
> Dimitris
>

I am interested in something like the project you propose. I want some
kind of psuedo-standard C-array userdata type in Lua that can be
reused across different/independent Lua based libraries. I think it
should be small/streamlined so independent projects are more willing
to include it in their own projects. And I think it should have no
dependencies so it can go everywhere. (This is one challenge I have
with LuaFFI.)

To give you insight on why I want this and how I want to use it, here
are some scenarios that I have encountered.

- In LuaCocoa (a Obj-C/Lua bridge), there are some APIs that deal with
C-arrays. When crossing from Cocoa to Lua, I push lightuserdata
(pointers). The idea is that when going back from Lua to Cocoa, I can
send that pointer through, but in Lua, I cannot access that data at
the moment. Nor can I create new arrays in Lua to be passed to
C/Obj-C. I could invent my own userdata type, but if I want to
interoperate with other Lua 3rd party libraries, I'm kind of stuck.
I've been putting off implementing my own hoping I can find a solution
that allows more interoperability with other libraries.

For example, when dealing with OpenGL or OpenAL Lua bindings, those
APIs also want buffers. They have their own userdata systems which
would be incompatible with any I invent for LuaCocoa (or any other
project.) FYI, I have my own OpenAL Lua binding which is also waiting
for me to solve this interoperability problem.

- Another example is closed Lua environments like a commercial
shipping game or Corona SDK. In these environments, you can use any
Lua library that doesn't depend on anything native not already
provided in that environment. If there was a pseudo-standardized
C-array userdata type that enough projects used, then it people in
these environments may have more options on libraries they can use.
(FYI, I have some influence on what can be put into Corona so this is
another reason a C-array library interests me.)


Anyway, these are my initial thoughts about what I think might need to
be in such a library.

- A new userdata type for C-arrays.

- I think a matrix type should be a separate library

- metamethods to access elements

- I think indicies should should start counting at 1 for consistency
in Lua, though maybe a runtime property can be set to change this if
it is a real sticking point for some people

- I don't know if there is some kind of API compatibility interface
that can be created to work with LuaFFI. (But I don't know how
userdata checks would ever work.)

- Memory management I think is the most challenging part. There are
cases where you might want the memory to be managed through normal Lua
memory management (like a regular Lua table), but there are cases
where you are creating the memory to feed to a C-API (such as an
OpenGL texture or OpenAL buffer). These may need to outlive the Lua
state that created it. In addition to specifying the management mode,
it seems you might also want to specify the memory allocator, e.g.
malloc/free, new/delete, Lua allocator, some custom allocator.


Thanks,
Eric
-- 
Beginning iPhone Games Development
http://playcontrol.net/iphonegamebook/