Re: destructive iteration patterns

[Sean Conner <sean@...>]

It was thus said that the Great Josh Haberman once stated:
> Sean Conner <sean <at> conman.org> writes:
> > It was thus said that the Great Josh Haberman once stated:
> > > I ultimately want to avoid having to do a per-entry allocation.
> > > malloc() is *expensive* when performed this often.
> >
> >   In other words, don't worry about calling malloc() unless you have proof
> > it's a performance issue.
> 
> If I had a nickel for every time someone to convince me that my
> performance concerns are invalid...

  Perhaps if you had included some figures we could have avoided this back
and forth a bit.  There was no indication that you measured anything and to
me, it just came across as "I feel doing it this way is slow so I'm doing it
this horribly complicated way to make sure it's fast."

> I'll see your profile and raise you a benchmark.  Here are numbers for
> my protobuf parser parsing into a data structure.  I have optimizations
> that avoid malloc()/free() and memcpy() for the strings contained in the
> protobuf, but I can tweak my code a little bit to turn these optimizations
> off and benchmark the results:
> 
>   malloc/free   memcpy    MB/s
>   -----------   ------    ----
>                            712
>                   X        643
>        X                   465
>        X          X        436

  Sure, your C code can burn through the file at 712 MB/s, but how fast is
it when you add Lua to the mix?  The two variations I see are your Lua
bindings as is (fields passed back only when requested) vs. a full Lua table
of results for each entry.

  Or alternatively, just document that the Lua programmer is reponsible for
saving any data from the results.  

  -spc