Re: Memory usage and speed differences with lua 5.2 vs 5.3

[Dane Springmeyer <dane@...>]

> On Sep 28, 2016, at 1:14 AM, Dirk Laurie <dirk.laurie@gmail.com> wrote:
> 
> 2016-09-28 8:19 GMT+02:00 Dane Springmeyer <dane@mapbox.com>:
> 
>> The program I'm seeing this with is osrm-backend, a tool used to process the
>> 35 GB http://wiki.openstreetmap.org/wiki/Planet.osm into a graph structure
>> for routing.
>> 
>> Testing is being done on r3.8xlarge amazon instances running ubuntu trusty
>> with 32 vCPU and 244 GB ram.
> 
> This is a seriously large amount of data and a seriously large machine.
> Factors O(log(N)) in algorithm times can no longer be pooh-poohed.

Yes, and the process runs for 4 hours on this hardware.
 
> The only place I can think of is garbage collection.

Okay, thanks for the suggestion. I've not looked in that direction.

> There were a few
> threads on that in the past six months or so.

Okay, will research as I have time.

> While you wait for the
> New World's day to start, when the real experts can chip in, maybe you
> could experiment with the settings available in collectgarbage().

Thank you.