Comments (7)
You can check out my solution here:
https://github.com/codeformuenster/verkehrsunfaelle/blob/python-importer/docker-compose.yml
Also check out the files in the overpass directory in the same branch / repo
This assumes you want a container for single use.
from overpass-api.
I guess that you're missing OVERPASS_PLANET_SEQUENCE_ID
environment variable. You need this, to reflect, how the dump that you're loaded is positioned against minute diff repository. Each diff repository has its own sequence so you need to check the one that you choose for correct number.
Please bear in mind, that though you've loaded only one Land, your diffs will be updating your database with whole planet so you will have partial (but growing) data from other parts of the world.
from overpass-api.
Thanks for your answer, how to find the sequence id though?
As per your suggestion I switched to http://download.openstreetmap.fr/replication/europe/germany/nordrhein_westfalen/munster/minute/
for the diff_url and 3397105
(which is the sequenceNumber
from this file) for the sequence id.
Or is there a way to disable the updates altogether? I only need the overpass container for bulk queries. I tried to comment out the portions [program:fetch_diff]
, [program:apply_diff]
, [program:dispatcher_areas]
and [program:areas_rules]
in supervisord.conf, then mounting the edited file into the container with -v
, but sadly this resulted in almost the same error.
from overpass-api.
I didn't thought about running instance that's not updating itself, so this is something that's missing.
As for sequenceNumber
I usually do this way, that I check the timestamp of planet
file, then take sequence number that's a bit before this date. To find the sequence number, I navigate the directories in minute diffs, to find file that was modified at the data that I'm looking for.
So for example, If I have a dump file from 2019-02-23 04:00:00 am, I'd look for sequence file from midnight that day (using modified date), and it is this file. In the contents you'll have the sequence number.
It's good to have a sequence number not later than your dump so you'll not loose any objects that were created/updated between your planet dump file and since you're starting synchronization. Overlapping is not a problem, as the data will easily converge to current state.
Did it work properly when you set the sequence id?
from overpass-api.
Sorry for the late reply. I found out some stuff
It seems the --meta
stuff does not work as expected. When setting it to no
, the script dispatcher_start.sh
does not work anymore. The empty string is being parsed as parameter which is then unknown to the actual dispatcher
binary.
Also omitting the meta environment variable completely should have disabled the meta stuff but .meta
files are generated nevertheless ?!
I have now done the following:
my supervisord.conf
:
[supervisord]
nodaemon=true
[program:overpass_dispatch]
command=/bin/bash -c "find /db/db -type s -print0 | xargs -0 --no-run-if-empty rm && /app/bin/dispatcher --osm-base --db-dir=/db/db"
user=overpass
redirect_stderr=true
priority=1
[program:nginx]
command=nginx
priority=2
[program:fcgiwrap]
command=/bin/bash -c "find /nginx -type s -print0 | xargs -0 --no-run-if-empty rm && fcgiwrap -s unix:/nginx/fcgiwrap.socket"
redirect_stderr=true
user=nginx
priority=3
My docker-entrypoint.sh
:
#!/bin/bash
set -eo pipefail
shopt -s nullglob
OVERPASS_COMPRESSION=${OVERPASS_COMPRESSION:-gz}
if [ ! -d /db/db ] ; then
lftp -c "get -c \"$OVERPASS_PLANET_URL\" -o /db/planet; exit" \
&& /app/bin/init_osm3s.sh /db/planet /db/db /app "--compression-method=$OVERPASS_COMPRESSION" \
&& rm /db/planet \
&& cp -r /app/etc/rules /db/db \
&& chown -R overpass:overpass /db \
&& echo "Overpass ready, you can start your container with docker start"
exit
fi
exec /usr/bin/supervisord -c /etc/supervisor/conf.d/supervisord.conf
from overpass-api.
I am struggling with the same issue, this is my docker-compose.yml
version: '3'
services:
api:
image: wiktorn/overpass-api:latest
container_name: overpass-api
restart: always
ports:
- '12345:80'
environment:
- OVERPASS_META=yes
- OVERPASS_MODE=init
- OVERPASS_PLANET_URL=http://download.geofabrik.de/europe/spain-latest.osm.bz2
- OVERPASS_DIFF_URL=http://download.openstreetmap.fr/replication/europe/spain/minute/
- OVERPASS_PLANET_SEQUENCE_ID=3370000
volumes:
- /Workspace/docker/overpass_db/:/db
stdin_open: true
tty: true
I followed the steps to get the PLANET_SEQUENCE_ID and it didn't work, also tried with different META config and nothing, any idea to start investigating from?
from overpass-api.
It worked 🎉
A single use container is ok, thanks a lot @ubergesundheit !
from overpass-api.
Related Issues (20)
- A complete offline overpass API without update and diff checking HOT 6
- Push new build? HOT 2
- Change default of OVERPASS_CLONE_SOURCE to https HOT 2
- File_Blocks_Index: Unsupported index file format version HOT 2
- Clone overpass docker container with data to another VM issue HOT 1
- Update docker image to contain Overpass API version v0.7.61 HOT 1
- Handle (or not) non-minute diffs HOT 1
- Failed to process planet file HOT 5
- Error when running the same query twice HOT 5
- Docker status showing "health: starting" continuously HOT 1
- Docker status showing "health: starting" continuously HOT 3
- Multiple countries + updates HOT 6
- localhost didn’t send any data. ERR_EMPTY_RESPONSE. HOT 1
- Overpass API default output format HOT 1
- "allow-duplicate-queries" seems to work partially HOT 8
- [Question] How to update env vars after the container is created? HOT 1
- How to improve query speed? HOT 1
- [Question] How to import offline data? HOT 1
- Error during diff download. Bailing out. HOT 5
- Error starting from osm.pbf file HOT 6
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from overpass-api.