C arrays as Lua tables

[Dimitris Papavasiliou <dpapavas@...>]

Hello all,

The attached archive contains a first version of a module that
essentially serves as a bridge between Lua and C arrays by exposing C
arrays (of all types) to Lua as tables with suitable index, newindex,
len and pairs/ipairs metamethods.  By itself this is probably of
little use in Lua but it should be quite useful as a base for other
modules or perhaps in the case that Lua is used as an extension
language for an application dealing with large matrix/vector data.

For instance image manipulation modules can use such arrays to
represent images.  No space is wasted as all data is tightly packed in
C char arrays, filtering operations and such can be implemented in C
and be efficient but the pixels are still available to Lua as a
(seemingly) ordinary table.  Other modules that could make use of this
would be linear algebra routines, audio processing modules and
anything dealing with matrix or tensor-type data.  I'm just mentioning
the ones I've thought of because I need (and intend to write) them in
the context of game programming.

Example annotated session follows:

> require "array"

Create an array out of packed data.

> a = array.doubles (2, 2, 5, "\000\000\000\000\000\000\240\063\000\000\000\000\000\000\000\064\000\000\000\000\000\000\008\064\000\000\000\000\000\000\016\064\000\000\000\000\000\000\020\064\000\000\000\000\000\000\024\064\000\000\000\000\000\000\028\064\000\000\000\000\000\000\032\064\000\000\000\000\000\000\034\064\000\000\000\000\000\000\036\064\000\000\000\000\000\000\038\064\000\000\000\000\000\000\040\064\000\000\000\000\000\000\042\064\000\000\000\000\000\000\044\064\000\000\000\000\000\000\046\064\000\000\000\000\000\000\048\064\000\000\000\000\000\000\049\064\000\000\000\000\000\000\050\064\000\000\000\000\000\000\051\064\000\000\000\000\000\000\052\064")
> print(array.pretty(a))
{
  {
    {1, 2, 3, 4, 5},
    {6, 7, 8, 9, 10}
  },
  {
    {11, 12, 13, 14, 15},
    {16, 17, 18, 19, 20}
  }
}

Indexing an array of higher dimensionality creates another array which
is a view of the first array (no values are copied):

> b = a[1]
> print(array.pretty(b))
{
  {1, 2, 3, 4, 5},
  {6, 7, 8, 9, 10}
}

Indexing an array with one dimension yields a number:

> print (b[1][2])
2

Changing an element in a subarray changes the initial array from which
it was created (as expected since the subarray references the same
values):

> b[1][2] = -2
> print (array.pretty(b))
{
  {1, -2, 3, 4, 5},
  {6, 7, 8, 9, 10}
}
> print (array.pretty(a))
{
  {
    {1, -2, 3, 4, 5},
    {6, 7, 8, 9, 10}
  },
  {
    {11, 12, 13, 14, 15},
    {16, 17, 18, 19, 20}
  }
}

Arrays can also be created from tables:

> c = array.doubles {
>>    {111, 112, 113, 114, 115},
>>    {211, 212, 213, 214, 215},
>> }
> print (c)
array[2, 5]

Whole subarrays can be set in one go.  Elements are copied via memcpy
in the underlying C array:

> a[2] = c
> print(array.pretty(a))
{
  {
    {1, 2, 3, 4, 5},
    {6, 7, 8, 9, 10}
  },
  {
    {111, 112, 113, 114, 115},
    {211, 212, 213, 214, 215}
  }
}
> a[1][1] = {311, 312, 313, 314, 315}
> print(array.pretty(a))
{
  {
    {311, 312, 313, 314, 315},
    {6, 7, 8, 9, 10}
  },
  {
    {111, 112, 113, 114, 115},
    {211, 212, 213, 214, 215}
  }
}

Existing arrays can be recast into arrays of congruent dimensionality.
Again both the initial and the cast array reference the same values:

> d = array.cast (2, 10, a)
> print(array.pretty(d))
{
  {311, 312, 313, 314, 315, 6, 7, 8, 9, 10},
  {111, 112, 113, 114, 115, 211, 212, 213, 214, 215}
}

I've written this for my own needs but seeing as its pretty generic
and perhaps of use to others I though I'd post it here too.  It works
with both Lua 5.1 and Lua 5.2 although with 5.1 you won't get any
pairs/ipairs metamethod functionality of course. If you think it's
useful I can distribute it as a separate module.  Any comments are
welcome of course.  I haven't written a (public) Lua module before so
there might be guidelines about coding or packaging that I'm not aware
of.

In particular I'd be interested to know if there's some accepted way
of exposing a C API from a module.  In addition to the Lua functions
to create array tables out of C arrays I'd like to expose a set of C
functions so that the host C application can use them.  Is there some
other module that does this?  I'd like to see if there's some sort of
convention for naming types and functions (e.g. larray_pushdoubles and
larray_Array or something like that).

Dimitris

Attachment: larray.tar.gz
Description: GNU Zip compressed data