Re: [ANN] Numeric Lua 0.3

[Luis Carvalho <lexcarvalho@...>]

Hi Dirk,

> Ubuntu 11.4, kept up to date by `aptitude safe-upgrade` up to 
>   17 July 2011.
> 
> libatlas-dev   3.8.3-29  (two other atlas libraries at the same version)

So my guess was right. :)

<snip>

> > [1] Try building your own ATLAS lib  
> And do this every time I upgrade Ubuntu?  

No, only once if you install it to a place that is not touched by Ubuntu, like
the default /usr/local/atlas.
 
> > http://math-atlas.sourceforge.net/errata.html
> > 
> > "GCC's violation of the x86 ABI causes seg faults/bus errors when mixed with
> > other compilers", for more information.
> I followed the link from there.  Over my head, frankly, but nevertheless
> fascinating.  The guys discussing the topic were real experts who quietly 
> opted out the moment the public started chipping in.  
> 
> But surely Ubuntu's libatlas is compiled by gcc?  So no other compilers
> in my case? 

Yes, that's the point from what I gather: GCC uses 8-byte alignments while
ATLAS relies on 4-byte (ABI compliant, according to them) alignments. It's a
bit over my head as well, but I don't take it to be Numlua's problem, at least
not for now.
 
> > [2] The other solution is to specify LUAI_USER_ALIGNMENT_T 
> > to attain the 8-byte boundary in luaconf.h. Minimal change:
> > 
> > #define LUAI_USER_ALIGNMENT_T union { double u[3]; void *s; long l; }
> > 
> Would I need to rebuild my Lua or only the NumLua package?

You have to rebuild Lua. The way I see it, this is a price you have to pay
for more numerical performance; if you want to use vanilla Lua, you can either
build your own optimized BLAS kernel or use an unoptimized BLAS as I suggested
earlier. I don't see a "clean", portable way of addressing this issue without
access to Lua internals, that is, without knowing sizeof(Udata) to pad the
payload later, for example.
 
> > While you're at it, you might as well patch llex.c with your favorite
> > extension -- short lambda! -- or add token filters. :)
> > 
> Not I — I'm a Lua puritan. 

Short lambdas are great in the interpreter, and you can always resist the
temptation of using them in your Lua code. :)
 
> > Another straightforward solution -- and a quick way to know if your problem
> > is memory alignment -- is to use LuaJIT since it guarantees 8-byte aligned
> > userdata. 
> 
> This works.

Good to know.
 
> I have tried to narrow down the problem, and can also offer this: if 
> run the code several times with various matrices, I get quite a nice 
> spectrum of error messages:
> 
>   Sementation fault
> 
>   lua: malloc.c:4631:  _int_malloc: Assertion `(unsigned long)(size) >= (unsigned long)(nb)' failed.
> 
>   ** On entry to DGESVD parameter number 11 had an illegal value
> 
>   *** glibc detected *** lua5.2: malloc(): memory corruption: 0x09fb0a40 ***
> 
>   not enough memory
> 
> (the last of these does not crash the Lua interpreter)

These are most likely related to ATLAS, but could you please send me your
matrices and code so that I can test them here?
 
> Moreover, the following other routines also have this problem:
> 
>     cond qr svd rank
> 
> These are obviously related: cond and rank call svd at the matrix.lua
> level, and qr and svd, though calling different LAPACK routines, make
> heavy use of Householder reflections.

The problem seems to be on an even lower level: from your last output it came
from an optimized DGEMV.
 
> Luajit works on all of them.  Unfortunately for the quest towards scientific
> truth, that fact kills my enthusiasm for finding the cause in the case of
> standard Lua.

I wouldn't worry much: as I said, I don't think this is Numlua's -- or even
Lua's -- problem. Moreover, making Numlua run on LuaJIT's ffi is one of my
priorities (right after adding a plotting facility), and that would solve this
problem anyway.

Many thanks for the report and feedback!

Cheers,
Luis

-- 
Computers are useless. They can only give you answers.
                -- Pablo Picasso

-- 
Luis Carvalho (Kozure)
lua -e 'print((("lexcarvalho@NO.gmail.SPAM.com"):gsub("(%u+%.)","")))'