Process does not always exit on completion of data ingestion about openstreetmap HOT 4 CLOSED

+1

Seen same issue, stops pelias chef recipe completing.

Seen on imports of west-yorkshire and south-yorkshire osm.pbf GBR imports.

from openstreetmap.

Any update or workaround to this?

from openstreetmap.

@cesozgen , yes, a quick and very dirty hack: change the module.exports = stats; in /stream/stats.js to module.exports = through.obj;. This will prevent the OSM parser from spitting out ingestion stats (though you'll still see elasticsearch indexer stats), but the process will exit. Here's the reason why:

tl;dr, the OSM ingestion statistics-logging setInterval() isn't getting clearInterval()'d.

The stats module exports a stream that tracks the number of records that pass through, and stores the total number of such streams instantiated globally. The counter is incremented on pipe and decremented on unpipe events respectively: the interval gets set when the first pipe's created and torn down when the last one's closed, which is inferenced from the value of the counter. It turns out that the number of pipes and unpipes detected is !=, meaning that the counter doesn't drop to 0 like it should.

from openstreetmap.

resolved by #27

from openstreetmap.