node-osm-stream
Overview
A fast and flexible NodeJS-based streaming parser for OpenStreetMap (.osm) files.
- Both incoming and outgoing streams available for piping from/to other streams
- Able to deal with large .osm files easily without loading the whole file onto memory
- Provided with the flexibility to work with and modify OSM objects (nodes, ways, relations) easily as it streams to you
Powered by node-expat for blazing fast parsing.
Table of Content
Quick Start
Install
npm install node-osm-stream
No fuss, no lints
Example: Reading from a local .osm file
var fs = require('fs');
var OSMStream = require('node-osm-stream');
var parser = OSMStream();
// open a local .osm filestream
fs.createReadStream('./path/to/file.osm')
.pipe(parser);
parser.on('node', function(node, callback){
// Modify current node object as you wish
// and pass it back to the callback.
// Or pass 'null' or 'false' to prevent the object being
// written to outgoing stream
console.log(node);
callback(node);
});
parser.on('way', function(way, callback){ callback(way); });
parser.on('relation', function(way, callback){ callback(relation); });
Easy-peasy, lemon-squeezy.
More examples
More advanced examples are available in the ./examples folder
Stream and format outgoing data to console (process.stdout)
Source: /examples/stream-to-stdout.js
To run example:
node ./examples/stream-to-stdout.js
Writing to a JSON file using Writeable steam (fs.createWriteStream)
Source: /examples/write-to-json.js
To run example:
node ./examples/write-to-json.js
API
Class: OSMStream
Class inherited from stream.Transform
Methods
All methods are inherited from stream.Transform
Events
Events: 'node', 'way', 'relation'
When an object (node/way/relation) from the .osm file has been parsed fully with its attributes and children (if any) ready, it will emit a 'node' or 'way' or 'relation' event, depending on the object type.
You can modify the outgoing data and passing it back to the callback. Or you can prevent the data from being passed downstream by passing back a null or false
It's important to note that since this is a streaming parser, any other objects (ways/relations) that may have referenced a skipped node may still hold its reference. It is up to the implementation to remove its references.
To see an example of a possible implementation, take a look at /examples/write-to-json.js
Note: If this event was registered, the callback must be passed back.
parser.on('node', function(node, callback) {
// modify the node object as necessary and pass back to callback
// or send a null or false to prevent it from going downstream
callback(node);
});
parser.on('way', function(way, callback) {
...
callback(way);
});
parser.on('relation', function(relation, callback) {
...
callback(relation);
});
Event: 'writeable'
When a chunk of data is ready to be written to the outgoing stream, it will emit a 'writeable' event.
You can modify the outgoing data and passing it back to the callback. Or you can prevent the data from being passed downstream by passing back a null or false
Note: If this event was registered, the callback must be passed back.
parser.on('writeable', function(data, callback) {
// there is some data to be passed to outgoing stream
// modify 'data' as needed
callback(data);
});
Event: 'flush'
After all the written data has been consumed through the outgoing stream, it will emit a 'flush' event. This will happened before the 'end' event is sent to signal the end of the readable side.
You can choose to pass in as much data as needed by passing it through the callback. Any data passed back will be written to the outgoing stream before the 'end' event is emitted.
Note: If this event was registered, the callback must be passed back.
parser.on('flush', function(callback) {
var last_data = 'This is the last string sent to the outgoing stream';
callback(last_data);
});
Events inherited from stream.Transform
In addition to the events above, the following are events inherited from stream.Transform class. Please refer to the offical documentation for more info: NodeJS API Documentation: stream.Transform
- Event: 'readable'
- Event: 'data'
- Event: 'end'
- Event: 'close'
- Event: 'error'
- Event: 'drain'
- Event: 'finish'
- Event: 'pipe'
- Event: 'unpipe'
Test
npm test
Known Issues
Credits
Links
License
Copyright (c) 2014 Hafiz Ismail. This software is licensed under the MIT License.
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent