Re: crash in lua 5.2 garbage collection

[Andres Perera <andres.p@...>]

On Tue, Dec 11, 2012 at 2:15 PM, Sergey Zavadski
<sergeyzavadski@gmail.com> wrote:
> Thanks for feedback
>
> -  threads are created only on startup, and pthread_create is called once.
> New threads are not created as requests come in.
> I will add checks though to see if it produces any failures

honestly, i would read through the pthread manpages and add error
checking to every call

>
>  - clock_gettime is not portable, as it is available only on linux. Result
> of gettimeofday is not super important as it is used for calculating average
> response time. No application logic is affected by it

clock_gettime isn't exclusive to linux. it's a posix facility

you use pthread_, which is *another* posix facility

i'm sure windows, or whatever platform you subscribe to, has a
clock_gettime equivalent. gettimeofday is a very poor benchmark tool
unless your os doesn't do multitasking

>
>
>
> On Tue, Dec 11, 2012 at 10:29 AM, Andres Perera <andres.p@zoho.com> wrote:
>>
>> On Tue, Dec 11, 2012 at 12:48 PM, Sergey Zavadski
>> <sergeyzavadski@gmail.com> wrote:
>> > The fact that crash happens only under extremely high load generated by
>> > test
>> > (more than 900K requests per minute) also points me that I am hitting
>> > some
>> > other limitation (either OS or hardware)
>> >
>> >
>> > On Tue, Dec 11, 2012 at 7:55 AM, James Graves
>> > <james.c.graves.jr@gmail.com>
>> > wrote:
>> >>
>> >> Roberto Ierusalimschy <roberto@inf.puc-rio.br> wrote:
>> >>
>> >> > * Because the GC traverses all objects in the system, it is one of
>> >> > the
>> >> > main "candidates" to suffer from a memory corruption created
>> >> > somewhere
>> >> > else.
>> >>
>> >> There was a recent article linked via Hacker News about one of the
>> >> popular web cache applications (maybe it was Varnish?).  Anyway, this
>> >> program uses a lot of RAM, and they've been seen un-reproducable bugs
>> >> occasionally.  The authors suspect it is a hardware problem in many
>> >> cases, so they've incorporated a memory test as part of their crash
>> >> dump.  They were doing something clever with multiple XOR passes to
>> >> check the memory and still preserve the original data.
>> >>
>> >> At any rate, many hosting providers are not using ECC memory on their
>> >> servers, and as memory sizes keep getting larger and larger (and
>> >> process size for the memory itself smaller and smaller) the rate of
>> >> bit errors due to cosmic radiation keeps going up.
>> >>
>> >> The short version: also try to run a memory test on the server and see
>> >> if that turns up anything.
>> >>
>> >> James Graves
>> >>
>> >
>>
>> you never check the return of pthread_create()... in libpropeller you
>> don't check the return of any pthread_ function for that matter
>>
>> also, gettimeofday is only good for benchmarking if you also want to
>> measure whatever process/thread your scheduler decided to give time
>> to. look into clock_gettime
>>
>