Re: Serialized test data in table format?

[tobias@...]

Quoting Russell Haley <russ.haley@gmail.com>:

> Hi,
>
> I'm starting some testing on my lua-persist library that serializes
> tables into lmdb. I suck at creating test data and keep getting
> stalled out on the task. I thought I'd ask if anyone has some
> serialized tables they could offer for my testing?
>
> One problem so far is that this thing is blazing fast. I had one set
> of data that was around 6 MB and the system didn't even blink. If
> anyone can offer 50-100MB of data (or more!) I'd be REALLY grateful.
>
> The code is here: https://github.com/RussellHaley/lua-persist but it's
> pretty much a prototype. Check the doc folder for ldocs. I will work
> on a dependencies list.
>
> I want to do some testing on the current features and add the indexing
> feature. Then I want to open up the api discussion the same as Tobias
> did (it's his fault I'm even on this list so I'm allowed to copy him).
>
> Current features;
> - open env, create multiple databases(tables)
> - Get all, get item, search data content(via lua function)
> - Change tracking and commiting.
>
> Still planned
> - table data indexes
> - add unit tests
> - moses integration
> - expose cursor library
> - expose for loop iterator
>
> Thanks,
>
> Russ

save as rtvg.lua
start with:

> Rtvg=require'rtvg'
> x=Rtvg()
> t=x:getVals(10)
> t
table: 0x55f54bb50210
> for k,v in pairs(t) do print(k,v) end
1       function: 0x55f54bb500c0
2       1538788.5647199
3       false
4       function: 0x55f54bb50970
5       6357118
6       859483.47221478
7       table: 0x55f54bb50fe0
8       function: 0x55f54bb51130
9       table: 0x55f54bb514f0
10      true

you can modify from here to create different keys etc.




----------------------------------------------------------
local makeWord = function( )
	local wrd = {}
	for i=1,math.random(3,12) do
		table.insert( wrd, string.char( math.random( 32, 123 ) ) )
	end
	return table.concat( wrd, '' ) .. ' '
end

local m = {
	getKey = function( self )
		local key  = self.keys[ math.random( 1, #self.keys ) ]( self )
		while nil ~= self.aKeys[ key ] do
			key = self.keys[ math.random( 1, #self.keys ) ]( self )
		end
		self.aKeys[ key ] = true
		return key
	end,
	getVal = function( self )
		local val  = self.vals[ math.random( 1, #self.vals ) ]( self )
		while nil ~= self.aVals[ val ] do
			val = self.vals[ math.random( 1, #self.vals ) ]( self )
		end
		self.aVals[ val ] = true
		return val
	end,
	getKeys = function( self, n )
		local tbl = {}
		for k=1,n do
			table.insert( tbl, self:getKey( ) )
		end
		return tbl
	end,
	getVals = function( self, n )
		local tbl = {}
		for k=1,n do
			table.insert( tbl, self:getVal( ) )
		end
		return tbl
	end,
	getHash = function( self, n )
		local hsh = {}
		for k=1,n do
			hsh[ self:getKey( ) ] = self:getVal( )
		end
		return hsh
	end,
	getFunction = function( self )
		local f = load( "return function(x) return x end" )
		return f( )
	end,
	getCoroutine = function( self )
		local c = load( "return coroutine.create( function(x) return x end )" )
		return c( )
	end,
	getBoolean = function( self )
		return math.random( 1, self.max ) % 2 == 1
	end,
	getString = function( self, n )
		local str   = { }
		local count = 0
		n = n or math.random( 1000, 3000 )
		while count<n do
			local word = makeWord( )
			count = count + #word
			if count <= n then
				table.insert( str, word )
			else
				table.insert( str, word:sub( 1,  #word-(count-n) ) )
			end
		end
		return table.concat( str, '' )
	end,
	getWords = function( self, n )
		local str = {}
		for i=1,math.random(1, n or 12) do
			table.insert( str, makeWord( ) )
		end
		return table.concat( str, '' )
	end,
	getFloat = function( self )
		return math.random( ) * math.random( 1, self.max )
	end,
	getInteger = function( self )
		return math.random( 1, self.max )
	end,
	getTable = function( self )
		local tbl = { }
		for i=1,math.random(1, 12) do
			table.insert( tbl, math.random( 1, self.max ) )
		end
		return tbl
	end
}

return setmetatable(
	  { }
	, {
		__call  = function( self, max )
			local rtvg =  {
				max  = max or 10000000,
				keys = { m.getTable, m.getWords, m.getFunction, m.getBoolean,  
m.getCoroutine, m.getFloat },
				vals = { m.getTable, m.getWords, m.getFunction, m.getBoolean,  
m.getCoroutine, m.getFloat, m.getInteger }
			}
			rtvg.aKeys  = { [ rtvg.keys[ math.random( 1, #rtvg.keys ) ]( rtvg )  
] = true }
			rtvg.aVals  = { [ rtvg.vals[ math.random( 1, #rtvg.vals ) ]( rtvg )  
] = true }

			return setmetatable( rtvg, { __index = m } )
		end
	}
)