[Date Prev][Date Next][Thread Prev][Thread Next]
- Subject: Re: Memory usage and speed differences with lua 5.2 vs 5.3
- From: Dane Springmeyer <dane@...>
- Date: Wed, 28 Sep 2016 16:58:39 -0700
> On Sep 28, 2016, at 1:14 AM, Dirk Laurie <firstname.lastname@example.org> wrote:
> 2016-09-28 8:19 GMT+02:00 Dane Springmeyer <email@example.com>:
>> The program I'm seeing this with is osrm-backend, a tool used to process the
>> 35 GB http://wiki.openstreetmap.org/wiki/Planet.osm into a graph structure
>> for routing.
>> Testing is being done on r3.8xlarge amazon instances running ubuntu trusty
>> with 32 vCPU and 244 GB ram.
> This is a seriously large amount of data and a seriously large machine.
> Factors O(log(N)) in algorithm times can no longer be pooh-poohed.
Yes, and the process runs for 4 hours on this hardware.
> The only place I can think of is garbage collection.
Okay, thanks for the suggestion. I've not looked in that direction.
> There were a few
> threads on that in the past six months or so.
Okay, will research as I have time.
> While you wait for the
> New World's day to start, when the real experts can chip in, maybe you
> could experiment with the settings available in collectgarbage().