Re: typed arrays

[Luis Carvalho <lexcarvalho@...>]

>    A simple API might have four functions (using Lua syntax to describe both
>    Lua and C APIs):
>    a = typed_vector(size,tcode) -- allocs 1D array of the given size and type
>    a:size() -- returns the size of the 1D typed array
>    a:type_code() -- returns the type of the 1D typed array
>    a[index] -- needed only in Lua, for simple element access
>    a:address_of() -- needed only in C, pointer access to elements
>    Type codes would exist for common standard C types ("char", "uint",
>    "float", ...), and common sized types ("uint8", "float32", "float64"),
>    ...).
>    Although such a simple API may mean that getting two libraries to talk to
>    each other may involve extra copying, even with that, it's a lot faster
>    and easier than having nothing at all. A more complex interface would be a
>    Lua adaptation of a subset of the Python buffer and memory view APIs
>    (http://docs.python.org/c-api/buffer.html); they allow data to be exposed
>    without copying, but at the cost of higher complexity.

I think that, as Javier said, the "translation" would need a semantic context,
and that thus adopting a Py_buffer would be overkill for general purposes; you
could simply look at a userdata as a byte array and apply the right context
when needed. For instance, a "memoryview" module can be put together quite
easily using LuaJIT (just to avoid a C interface) and Roberto's struct:

-- Python's memoryview userdata
-- This code is in public domain

local ffi = require "ffi"
local struct = require "struct"
local pack, unpack = struct.pack, struct.unpack

local bytearray = function (s) return ffi.cast("char *", s) end

local tobytes = function (mv) return ffi.string(mv.udata, mv.l * mv.n) end

local tolist = function (mv)
  local s, fmt = tobytes(mv), mv.format
  local l, p = {}, 1
  for i = 1, mv.n do l[i], p = unpack(fmt, s, p) end
  return l
end

local len = function (mv) return mv.n end

local mvmt = {
  __index = function (mv, k)
    if k > 0 and k <= mv.n then
      return (unpack(mv.format, ffi.string(mv.udata + (k - 1) * mv.l, mv.l)))
    end
  end,
  __newindex = function (mv, k, v)
    assert(not mv.readonly, "memview is readonly")
    if k > 0 and k <= mv.n then
      ffi.copy(ffi.cast("void *", mv.udata + (k - 1) * mv.l),
          pack(mv.format, v), mv.l)
    end
  end,
  __call = function (mv, f) -- in-place map of `f`
    local s, fmt, u, l, p = tobytes(mv), mv.format, mv.udata, mv.l, 1
    for i = 1, mv.n do
      local v, p1 = unpack(fmt, s, p)
      ffi.copy(ffi.cast("void *", u + p - 1), pack(fmt, f(v)), l)
      p = p1
    end
    return mv
  end
}

local newmemview = function (_, udata, format, n, readonly)
  local f = format or "c"
  local n = n or #udata
  local l = struct.size(f)
  return setmetatable({udata = bytearray(udata), l = l, format = f, n = n,
    readonly = readonly or false}, mvmt)
end

return setmetatable({tobytes = tobytes, tolist = tolist, len = len},
  {__call = newmemview})


This should pretty much cover the API you've required. A few examples,
including the creation of vectors (1D arrays):

> mv = require "memoryview"
> s = mv"gello"
> print(mv.tobytes(s))
gello
> s[1] = "h"
> print(mv.tobytes(s))
hello
> 
> rot13 = function(c) return math.mod(c+13-97,26)+97 end
> s = mv(mv.tobytes(s), "B")
> print(mv.tobytes(s(rot13)))
uryyb
> print(mv.tobytes(s(rot13)))
hello
> 
> struct = require "struct"
> vector = function (n) return mv(string.rep(struct.pack("d", 0), n), "d", n) end
> x = vector(4)
> for i = 1, mv.len(x) do x[i] = i end
> table.foreach(mv.tolist(x), print)
1	1
2	2
3	3
4	4
> x[1] = math.pi
> table.foreach(mv.tolist(x), print)
1	3.1415926535898
2	2
3	3
4	4

Cheers,
Luis

-- 
Computers are useless. They can only give you answers.
                -- Pablo Picasso

-- 
Luis Carvalho (Kozure)
lua -e 'print((("lexcarvalho@NO.gmail.SPAM.com"):gsub("(%u+%.)","")))'