Re: LUA: add values in nested table

[Francisco Olarte <folarte@...>]

Wilmar:

On Fri, Jun 12, 2020 at 6:14 PM Wilmar Pérez <darioperezb@gmail.com> wrote:
> I have a performance issue in my application. I would like to gather some ideas on what I can do to improve it. The application is very easy: I need to add values inside a nested table to get the total an user wants to pay out of all the pending payments. The user chooses a number of payments and I calculate how much it is they will pay.
...
> It works. At the end I get what I need. However, I notice that it takes ages to do the calculation. Each JSON has only twelve pending payments (one per month). It is taking between two to three seconds to come up with a result!. I tried on different machines and LUA 5.1, 5.2., 5.3. and the result is the same.

As people in SO told you, this is not lua code, try posting some real example.

I very much doubt the table manipulation is taking that much time, I
do much heavier ops in microseconds in the code I'm working right now.
I suspect you are not timing what you are giving us.You may be getting
a HUGE value for limit, although I suspect this will end in error.

I've made some timings with this little test:
$ cat tstspeed.lua
local base=os.time()
local function P(...)
   local t = os.time()
   print(t, t-base, ...)
   base=t
end

local limit = 100000

P("Starting, filling table with "..limit.." rows.");

local p = {}
for i=1,limit do
   p[i]={ month = "month"..i, amount=i }
end
local t = { name ="John",surname="Doe",pending_payments=p }

-- straightforward:
local repeats = 10000
P("Starting straightforward loop.","repeats ="..repeats)
for r=1,repeats do
   local total = 0;
   for i=1, limit, 1 do -- last 1 is not needed anyway.
      total = total + t.pending_payments[i].amount;
   end
end
P("Starting local payments loop.","repeats ="..repeats)
for r=1,repeats do
   local total = 0;
   local pending_payments = t.pending_payments
   for i=1, limit, 1 do -- last 1 is not needed anyway.
      total = total + pending_payments[i].amount;
   end
end

P("Terminated.")

With the origial code I get this times ( note I do a 100k table ad 10k repeats )

folarte@7of9:~/tmp$ lua tstspeed.lua
1591980496    0    Starting, filling table with 100000 rows.
1591980496    0    Starting straightforward loop.    repeats =10000
1591980515    19    Starting local payments loop.    repeats =10000
1591980532    17    Terminated.

So this is aprox. 1.9 ms for a 100k table with 10k repeats, caching
payments saves a little, as expected.

If I turn to a hundred element table and a million iterations...

1591980752    0    Starting, filling table with 100 rows.
1591980752    0    Starting straightforward loop.    repeats =1000000
1591980754    2    Starting local payments loop.    repeats =1000000
1591980755    1    Terminated.

It seems os.time lacks precision for 'only' a million iterations. So
let's try a thousand elements.

1591980773    0    Starting, filling table with 1000 rows.
1591980773    0    Starting straightforward loop.    repeats =1000000
1591980791    18    Starting local payments loop.    repeats =1000000
1591980804    13    Terminated.

so 1.8/1.3 microseconds per loop for 1k elements. Seems pretty fast.
So either your problem is in lunajson or in something you are not
showing or my machine is much faster, which I doubt.

> Can anyone please suggest how I can implement this better?

Measure what is slowing you down really, make a small reproducible
test case if you have it. Post it here.

btw, Debian 10, lua 5.3.3, AMD Ryzen 5 3600X not overclocked, not too fast.

Francisco Olarte.