Code Monkey home page Code Monkey logo

osrm's Introduction

OSRM cookbook

This cookbook can do the following things for you

  • Install and configure the OSRM route daemon
  • Download, extract, contract map data and keep them up to date

To use the following recipes and providers, add the following to your metadata.rb

depends 'osrm'

Recipes

default

Runs install_git

install_git

Clone and install the latest HEAD from Project-OSRM to node['osrm']['target'] (default /opt/osrm).

Providers

osrm_map

osrm_map is a shortcut that calls the other map providers in the following order

  • osrm_map_download
  • osrm_map_extract
  • osrm_contract

This example downloads, extracts and contracts the osrm map for Germany:

osrm_map 'germany'

The following additional attributes are allowed:

osrm_map 'europe'
  map_dir         '/srv/my_map_data'     # Where to put the maps (defaults to '/opt/osrm-data')

  # Path to the map (osm.pbf or osm.bz2) to use.
  # This is filled in automatically if the file is downloaded.
  # Only use it in case you are using a custom map.
  map             '/path/to/map.osm.bpf'

  profile         'car'                  # Profile to use (defaults to 'car')
  profile_dir     '/srv/my_profiles'     # Where to look for profiles (.lua files)
  extract_command 'osrm-extract'         # Path to osrm-extract binary
  contract_command 'osrm-contract'         # Path to osrm-contract binary
  user            'my_osrm_user'         # User to run commands as
  cwd             '/srv/my_osrm'         # Set working directory for osrm-extract
  threads         5                      # How many threads to use (defaults to number of cpu cores)
  cleanup         false                  # Do not cleanup .osrm and .osrm.restrictions after preparing
  timeout         3600                   # Timeout in seconds for osrm-extract/osrm-contract. Defaults to 24h
  stxxl_size      150000                 # Size (in MB) of stxxl temporary file. Dynamically allocated by default
  stxxl_file      '/tmp/stxxl'           # Location of stxxl temporary file. Defaults to '/var/tmp/stxxl'

  # The files checksum can also be checked (defaults to true)
  # When set to true, it will use the default checksum set in the attributes (if existent)
  # When set to false, it doesn't check the checksum
  # When set to an ftp:// or http:// URL, retrieve the file and use the checksum in its content
  # When set to a string, use it as the checksum
  checksum        'http://url.to/checksum.md5'
end

osrm_map_download

Downloads map data for the specified region.

osrm_map_download 'planet'
osrm_map_download 'europe'
osrm_map_download 'north-america'
osrm_map_download 'germany'
osrm_map_download 'us-west'

For a full list of supported regions, take a look at attributes/regions.rb. If necessary, it can be easily extended to support even more, feel free to file a pull request!

By default, the provider automatically re-downloads the file if it was changed on the server. You can prevent this behaviour when using the :download_if_missing action:

osrm_map_download 'europe' do
  action :download_is_missing
end

Furthermore, you can specify the following attributes:

osrm_map_download 'us-west' do
  user     'my_osrm_user'
  map_dir  '/srv/my_map_data' # Where to put the downloaded files (defaults to '/opt/osrm-data')

  # Manually specify url to download map from.
  # Defaults to the URL in the attributes (available for most regions)
  url      'http://my.geoserver.com/map.osm.bpf'

  # The files checksum can also be checked (defaults to false)
  # When set to true, it will use the default checksum set in the attributes
  # When set to false, it doesn't check the checksum
  # When set to an ftp:// or http:// URL, use its contents as a checksum
  # When set to a string, use it as the checksum
  checksum 'http://url.to/checksum.md5'
end

osrm_map_extract

Extracts downloaded map data, using osrm-extract.

Example:

osrm_map_extract 'europe'

The following attributes are supported:

osrm_map_extract 'europe' do
  map_dir     '/srv/my_map_data'  # Use the same directory you used in osrm_map_download

  # Path to the map (osm.pbf or osm.bz2) to use.
  # This is set automatically (via attributes),
  # if you use a map downloaded by osrm_map_download.
  # Only use it in case you are using a custom map.
  map        '/path/to/map.osm.bpf'

  profile     'car'               # Profile to use (defaults to 'car')
  profile_dir '/srv/my_profiles'  # Where to look for profiles (.lua files)
  command     'osrm-extract'      # Binary to use
  user        'my_osrm_user'
  cwd         '/srv/my_osrm'      # Set working directory for osrm-extract
  threads     5                   # How many threads to use (defaults to number of cpu cores)
  timeout     3600                # Timeout in seconds for osrm-extract. Defaults to 24h
  stxxl_size  150000              # Size (in MB) of stxxl temporary file. Dynamically allocated by default
  stxxl_file  '/tmp/stxxl'        # Location of stxxl temporary file. Defaults to '/var/tmp/stxxl'
end

osrm_map_contract

contracts extracted map data, using osrm-contract.

Example:

osrm_map_contract 'europe'

The following attribtues are supported:

osrm_map_contract 'europe' do
  map_dir     '/srv/my_map_data'  # Use the same directory you used in osrm_map_download

  # Path to the map (osm.pbf or osm.bz2) to use.
  # This is set automatically (via attributes),
  # if you use a map downloaded by osrm_map_download.
  # Only use it in case you are using a custom map.
  map        '/path/to/map.osm.bpf'

  profile     'car'               # Profile to use (defaults to 'car')
  command     'osrm-contract'      # Binary to use
  user        'my_osrm_user'
  cwd         '/srv/my_osrm'      # Set working directory for osrm-contract
  threads     5                   # How many threads to use (defaults to number of cpu cores)
  cleanup     false               # Do not cleanup .osrm and .osrm.restrictions after preparing
  timeout     3600                # Timeout in seconds for osrm-extract. Defaults to 24h
end

osrm_node

Sets up and starts the node-osrm service.

osrm_node 'europe' do
  profile  'car'               # Profile for which to start the daemon
  user     'osrm-node'         # User to run the daemon as
  port     5000                # TCP port to bind to
  listen   '127.0.0.1'         # TCP address to listen on
  map_base '/path/to/map_base' # Base path of the (contracted) map
                               # e.g. '/opt/osrm-data/europe/car/europe-lastest'
                               # (skip the file extention, like .edges or .osm.bpf)
  shared_memory false          # Use a shared-memory segment (created by osrm-datastore, see next provider)
end

osrm_routed

Sets up and starts osrm-routed (using upstart) for the specified region. NOTE: osrm_routed is not recommended for production usage. Use the osrm_node provider instead to deploy node-osrm.

Example:

osrm_routed 'europe'

The following attributes are supported:

osrm_routed 'europe' do
  service_name 'osrm-routed-%s'    # %s will be replaced with the selected region and profile
  profile      'car'               # Profile for which to start the daemon
  user         'osrm-routed'       # User to run the daemon as

  daemon       '/path/to/osrm-routed'

  map_dir      '/srv/my_map_data'  # Use the same directory you used in osrm_map_download)
  map_base     '/path/to/map_base' # Base path of the (contracted) map
                                   # e.g. '/opt/osrm-data/europe/car/europe-lastest'
                                   # (skip the file extention, like .edges or .osm.bpf)

  threads      16                  # How many threads to use (defaults to number of cpu cores)

  port         5000                # TCP port to bind to
  listen       '127.0.0.1'         # TCP address to listen on

  shared_memory false              # Use a shared-memory segment (created by osrm-datastore, see next provider)
end

osrm_datastore

Loads a specified map into memory

Example:

osrm_datastore 'europe'

The following attributes are supported:

osrm_datastore 'europe' do
  profile      'car'               # Profile for which to start the daemon
  shmmax       5_000_000_000       # Memory limit in bytes. Defaults to a value that should be enough for world
  user         'osrm-routed'       # User to run the daemon as (will be created if not existent)

  command      '/path/to/osrm-datastore'

  map_dir      '/srv/my_map_data'  # Use the same directory you used in osrm_map_download)
  map_base     '/path/to/map_base' # Base path of the (contractd) map
                                   # e.g. '/opt/osrm-data/europe/car/europe-lastest'
                                   # (skip the file extention, like .edges or .osm.bpf)
end

Attributes

You can set the following attributes if you need settings that differ form the defaults

default

node['osrm']['repository'] = 'https://github.com/Project-OSRM/osrm-backend'
node['osrm']['branch'] = 'master' # use e.g. 'v0.3.5' for a stable version

node['osrm']['target'] = '/opt/osrm'
node['osrm']['map_dir'] = '/opt/osrm-data'

node['osrm']['threads'] = node['cpu']['total']

routed

node['osrm']['routed']['user'] = 'osrm-routed'
node['osrm']['routed']['service_name'] = 'osrm-routed-%s'

regions

You can add custom regions like this

node['osrm']['map_data']['your-region']['profiles'] = %w(car)
node['osrm']['map_data']['your-region']['url'] = "http://download.geofabrik.de/your-region-latest.osm.pbf"
node['osrm']['map_data']['your-region']['checksum'] = "#{node['osrm']['map_data']['your-region']['url']}.md5"

Contributing

Pull requests are very welcome!

  1. Fork the repository on Github
  2. Create a named feature branch (like add_component_x)
  3. Write you change
  4. Write tests for your change (if applicable)
  5. Run the tests, ensuring they all pass
  6. Submit a Pull Request using Github

osrm's People

Contributors

chr4 avatar danyeaw avatar softmodal avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

osrm's Issues

osrm-datastore

Do you plan to support the osrm-datastore with sharedmemory support ?

osrm_datastore does not work if osrm_map action is :create

Here is my recipe:

include_recipe 'apt'
include_recipe 'osrm'

region = 'delaware'

group 'osrm-routed'
user 'osrm-routed' do
  gid 'osrm-routed'
  shell '/bin/false'
end

osrm_map region do
  action :create
end

osrm_datastore region

osrm_routed region do
  listen '0.0.0.0'
  port   5000
  shared_memory true
end

And when I run kitchen converge:

================================================================================
           Error executing action `create` on resource 'osrm_datastore[delaware]'
           ================================================================================

           Mixlib::ShellOut::ShellCommandFailed
           ------------------------------------
           execute[/opt/osrm/build/osrm-datastore /opt/osrm-data/delaware/car/delaware-latest.osrm] (/tmp/kitchen/cache/cookbooks/osrm/providers/datastore.rb line 59) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '1'
           ---- Begin output of /opt/osrm/build/osrm-datastore /opt/osrm-data/delaware/car/delaware-latest.osrm ----
           STDOUT:
           STDERR: [warn] Invalid file path given!

If I delete /opt/osrm-data/delaware/car and change the osrm_map action to :create_if_missing, kitchen converge works fine.

It looks like some files are deleted inadvertently from osrm_contract causing osrm_datastore to fail.

Fresh installation failed on downloading osm.pbf stage.

Hello

Tried to install server with map for north-america region.

Environment:
EC2 west 2 instance, Ubuntu 14.04.

Used Chef Client, version 11.4.4, chef-solo and roundsman.

recipes/default.rb

include_recipe 'osrm::install_git'

osrm_map 'north-america' do
  action :create_if_missing
  user   'ubuntu'
end


osrm_routed 'north-america' do
  user   'ubuntu'
  listen '0.0.0.0'
  port   5000
end

Got an error on stage of downloading .osm.pbf file.

** [out :: ] Recipe: osrm::default
** [out :: ] * osrm_map[north-america] action create_if_missing
** [out :: ] (up to date)
** [out :: ] Recipe: <Dynamically Defined Resource>
** [out :: ] * osrm_map_download[north-america] action download_if_missing
** [out :: ] (skipped due to only_if)
** [out :: ] * osrm_map_extract[north-america] action extract_if_missing
** [out :: ] (up to date)
** [out :: ] * template[/opt/osrm/build/extractor.ini] action create
** [out :: ] (up to date)
** [out :: ] * file[/opt/osrm/build/.stxxl] action create
** [out :: ] (skipped due to only_if)
** [out :: ] * directory[/opt/osrm-data/north-america/car] action create
** [out :: ] (up to date)
** [out :: ] * link[/opt/osrm-data/north-america/car/north-america-latest.osm.pbf] action create
** [out :: ] (up to date)
** [out :: ] * execute[rm -f /opt/osrm-data/north-america/car/north-america-latest.osrm] action run
** [out :: ] (skipped due to not_if)
** [out :: ] * execute[rm -f /opt/osrm-data/north-america/car/north-america-latest.osrm.names] action run
** [out :: ] (skipped due to not_if)
** [out :: ] * execute[rm -f /opt/osrm-data/north-america/car/north-america-latest.osrm.restrictions] action run
** [out :: ] (skipped due to not_if)
** [out :: ] * execute[osrm-north-america-car-extract] action run
** [out :: ] 
** [out :: ] ================================================================================
** [out :: ] Error executing action `run` on resource 'execute[osrm-north-america-car-extract]'
** [out :: ] ================================================================================
** [out :: ] 
** [out :: ] 
** [out :: ] Mixlib::ShellOut::ShellCommandFailed
** [out :: ] ------------------------------------
** [out :: ] Expected process to exit with [0], but received '1'
** [out :: ] ---- Begin output of /opt/osrm/build/osrm-extract /opt/osrm-data/north-america/car/north-america-latest.osm.pbf -p /opt/osrm/profiles/car.lua ----
** [out :: ] STDOUT: [info] Reading options from: extractor.ini
** [out :: ] STDERR: [warn] Input file /opt/osrm-data/north-america/car/north-america-latest.osm.pbf not found!
** [out :: ] ---- End output of /opt/osrm/build/osrm-extract /opt/osrm-data/north-america/car/north-america-latest.osm.pbf -p /opt/osrm/profiles/car.lua ----
** [out :: ] Ran /opt/osrm/build/osrm-extract /opt/osrm-data/north-america/car/north-america-latest.osm.pbf -p /opt/osrm/profiles/car.lua returned 1
** [out :: ] 
** [out :: ] 
** [out :: ] Resource Declaration:
** [out :: ] ---------------------
** [out :: ] # In /tmp/roundsman/config/cookbooks/osrm/providers/map_extract.rb
** [out :: ] 
** [out :: ] 72:   execute "osrm-#{new_resource.region}-#{new_resource.profile}-extract" do
** [out :: ] 
** [out :: ] 73:     user    new_resource.user if new_resource.user
** [out :: ] 74:     cwd     cwd
** [out :: ] 75:     timeout new_resource.timeout
** [out :: ] 
** [out :: ] 76:     command "#{command} #{linked_map} -p #{profile_dir}/#{new_resource.profile}.lua"
** [out :: ] 77:     not_if  { ::File.exists?("#{map_stripped_path}.osrm.names") }
** [out :: ] 78:   end
** [out :: ] 79:
** [out :: ] 
** [out :: ] 
** [out :: ] 
** [out :: ] Compiled Resource:
** [out :: ] ------------------
** [out :: ] # Declared in /tmp/roundsman/config/cookbooks/osrm/providers/map_extract.rb:72:in `extract'
** [out :: ] 
** [out :: ] execute("osrm-north-america-car-extract") do
** [out :: ] action "run"
** [out :: ] retries 0
** [out :: ] retry_delay 2
** [out :: ] command "/opt/osrm/build/osrm-extract /opt/osrm-data/north-america/car/north-america-latest.osm.pbf -p /opt/osrm/profiles/car.lua"
** [out :: ] backup 5
** [out :: ] cwd "/opt/osrm/build"
** [out :: ] returns 0
** [out :: ] 
** [out :: ] timeout 86400
** [out :: ] user "ubuntu"
** [out :: ] cookbook_name :osrm
** [out :: ] not_if { #code block }
** [out :: ] end
** [out :: ] 
** [out :: ] 
** [out :: ] 
** [out :: ] 
** [out :: ] [2014-06-03T09:55:13+00:00] ERROR: Running exception handlers
** [out :: ] [2014-06-03T09:55:13+00:00] ERROR: Exception handlers complete
** [out :: ] Chef Client failed. 0 resources updated
** [out :: ] [2014-06-03T09:55:13+00:00] FATAL: Stacktrace dumped to /tmp/roundsman/cache/chef-stacktrace.out
** [out :: ] [2014-06-03T09:55:13+00:00] FATAL: Mixlib::ShellOut::ShellCommandFailed: execute[osrm-north-america-car-extract] (/tmp/roundsman/config/cookbooks/osrm/providers/map_extract.rb line 72) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '1'
** [out :: ] ---- Begin output of /opt/osrm/build/osrm-extract /opt/osrm-data/north-america/car/north-america-latest.osm.pbf -p /opt/osrm/profiles/car.lua ----
** [out :: ] STDOUT: [info] Reading options from: extractor.ini
** [out :: ] STDERR: [warn] Input file /opt/osrm-data/north-america/car/north-america-latest.osm.pbf not found!
** [out :: ] ---- End output of /opt/osrm/build/osrm-extract /opt/osrm-data/north-america/car/north-america-latest.osm.pbf -p /opt/osrm/profiles/car.lua ----
** [out :: ] Ran /opt/osrm/build/osrm-extract /opt/osrm-data/north-america/car/north-america-latest.osm.pbf -p /opt/osrm/profiles/car.lua returned 1
command finished in 3602ms
failed: "sh -c 'sudo -p '\\''sudo password: '\\'' chef-solo -c /tmp/roundsman/solo.rb -j /tmp/roundsman/solo.json'" on 

After brief research, found this line https://github.com/chr4-cookbooks/osrm/blob/master/providers/map.rb#L31 where new_resource.map is blank. I was unable to find the place where map is filled with URL from attributes.

Temporally comment this line and rerun chef installation - successfully started downloading file.

Any ideas?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.