Re: Feature request: userdata slice

[Sean Conner <sean@...>]

It was thus said that the Great 云风 Cloud Wu once stated:
> 
> Let me try to explain one of real user case.
> 
> In my MMO game server, I create about 10K lua vm in one process. The
> average size of them is about  10M. (So it needs more than 100G altogether,
> I run it on a 128G RAM server) They works together like erlang's processes.
> 
> We have a large tree based document read from a large xml or json file.
> About 10M size and more than 100K records. It's immutable and shared among
> all of the lua vm.
> 
> We can't load this document in each lua vm because of size (And parsing
> time of the document is also a problem), so I must use a C object to store
> this document, and share the pointer.
> 
> My problem is how to traverse this C object as lua tables efficiently.

  I can think of several ways, but they all depend upon how the data is
stored internally (external to Lua).  Assuming a pointer to a node is
self-describing (or the data pointed to by the pointer is self-describing so
you know what you are pointing to, if you get my drift), then you might be
best served using lightuserdata (a direct pointer to the shared data) and a
custom metatable for lightuserdata (which would be shared across all
lightuserdata).  So, as long as you "know" where you are based on a pointer,
then you save the overhead of allocating memory from Lua.  

	static int lightusedata___index(lua_State *L)
	{
	  struct datanode *node = lua_touserdata(L,1);
	  const char      *idx  = lua_tostring(L,2);
	  struct datanode *new;

	  new = find(node,idx);
	  if (new)
	  {
	    if (new->type == NODE)
	      lua_pushlightuserdata(L,new);
	    else if (new->type == STRING)
	      lua_pushstring(L,new->val.string);
	    else if (new->type == NUMBER)
	      lua_pushnumber(L,new->val.number);
	    /* ... */
	  }
	  else
	    /* handle appropriately */
	  return 1;
	}

  -spc (That's about as lightweight as I can imagine it being)