Code Monkey home page Code Monkey logo

node-osm-pbf-stream's Introduction

osm-pbf-stream

npm npm license npm downloads build status

OpenStreetMap Protocol Buffer Stream

Install via npm

$ npm install --save osm-pbf-stream

Index

What

As OpenStreetMap's .osm.pbf files aren't pure Protocol Buffer messages, but are in a chunked, size delimited (and optionally, compressed) custom crafted format – this parses out the Protobuf message chunks.

So, osm.pbf goes in, pbf comes out.

Usage

var OsmPbf = require( 'osm-pbf-stream' )
var pbfStream = fs.createReadStream( 'berlin-latest.osm.pbf' )
  .pipe( new OsmPbf.BlobParser() )
  .on( 'header', ... )
  .on( 'blob', ... )
// Now you can process the Protocol Buffer stream
// with a streaming Protobuf parser, like `pbs`
var pbs = require( 'pbs' )
var osmSchema = fs.readFileSync( 'osmformat.proto', 'utf8' )
var messages = pbs( osmSchema )

var decoder = messages.PrimitiveBlock.decode()

decoder.primitivegroup( function( group, next ) {
  console.log( 'PrimitiveGroup.nodes', group.dense || group.nodes )
  next()
})

decoder.once( 'finish', function() {
  console.log( 'EOD' )
})

// Pipe the PBF stream to the decoder
pbfStream.pipe( decoder )

Speed

With the ~49 MB test file, it processes about 50 MB/s on a MacBook Pro.

BlobParser
  ✓ 1MB chunk size (986ms)
  ✓ Default (16KB) chunk size (1009ms)

node-osm-pbf-stream's People

Contributors

jhermsmeier avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

Forkers

profcab

node-osm-pbf-stream's Issues

Node tags & PBF parsing

Hi @jhermsmeier ,

nice project! I was wondering how do I get tags for nodes with this parser? I seem to be getting only this info from a primitive group:

selection_163

Btw, could you recommend some module (if such exist), which could convert the PBF representation that your project returns to a JS objects (so that objects are "undensed")? Preferably, with the possibility to define which attributes/tags have to be extracted.

Best regards,
Alex

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.