[Date Prev][Date Next][Thread Prev][Thread Next]
- Subject: Re: Openstreetmap XML
- From: Geoff Leyland <geoff_leyland@...>
- Date: Wed, 27 Nov 2013 21:04:27 +1300
On 27/11/2013, at 8:54 pm, steve donovan <firstname.lastname@example.org> wrote:
> On Wed, Nov 27, 2013 at 9:49 AM, Geoff Leyland
> <email@example.com> wrote:
>> The files are large enough that I suspect it’s not practical to read the whole file into a string and then generate a table
> How big are the files? It's amazing what you can get away with these
> days with modern computers!
To be honest, I don’t actually know, since I haven’t downloaded a whole country yet. I just know that the data I have for a small area is as big as data for the whole country in more compact formats (admittedly I live in a small country). I would expect several GB. I tend to use LuaJIT, so there’s a hard limit on how big a table can be, as well as a practical limit imposed by how much you want the GC to manage.
> If it's too big, then a stream-based LuaExpat way would be the
> (standard) way to go.
I was just going to fiddle with Roberto’s parser to make it a bit more streamy, but I’ve just read there’s an alternative format for OSM data anyway.