Wrapping a Lua table around a C array

I need to specify arrays through Lua which later will be converted to normal C arrays. The trouble is that these arrays may potentially be pretty large and some of them may only need to hold small (byte or short sized) integers. Therefore, given that numbers in Lua are doubles which I don't want to change, using normal Lua tables to do the job results in a 8x overhead in terms of memory use. This is not that bad since the values are converted back to unsigned chars where needed when reading the data from C but these arrays are contained inside Lua scripts and these scripts therefore end up being eight times larger than they need to be.

The way I've handled this up to now is to create Lua functions which take an array as binary data (a Lua string) and convert it into a table on the fly. So instead of containing something like this:

This approach has the merit that it is completely transparent. You can still specify a normal Lua table (you actually do that, but frombytes() creates the table for you) but the script file isn't larger than needed. The problem is that you now have the overhead of converting binary data into a Lua table and back to binary data later inside the application. This is necessary because it allows you to process the data inside Lua before passing them on to the application but when that is not necessary you're still left with a (potentially big) slowdown when loading the data to perform the needless conversions.

So I thought I would only keep the C arrays and make frombytes() and other similar functions create empty tables or userdata with metamethods to allow you to get and set individual array elements. When read from the application the array can then be used without conversion where possible. These tables or userdata need to behave like normal tables in order to keep the transparency of the previous approach. Although getting and setting is easy enough, I'm having trouble with the table length. The __len metamethod doesn't work at all for tables so that #foo would always return 0 and although it works for the # operator with userdata it doesn't work when calling lua_objlen from inside C.

So the question is: apart from the obvious solutions of implementing my own lua_objlen, hacking the internals of Lua etc. can you think of a slick way to create a table or userdata object that will contain a reference to a string with binary data and provide set, get and length access to it so that it can pose as a normal Lua table?

PS: I know that most of these issues are addressed in Lua 5.2 but since we have no idea when that will be released I'd rather find a more immediate solution.