Comments (8)
giantamoeba
commented on July 19, 2024
1
I had exactly the same problem with germany-latest, it turned out to be a download error. The original http://download.geofabrik.de/europe/germany-latest.osm.pbf had moved to http://ftp5.gwdg.de/pub/misc/openstreetmap/download.geofabrik.de/germany-latest.osm.pbf
Using the new correct file works now but I am facing the next problem described in #10
from nominatim-docker.
thomasnordquist
commented on July 19, 2024
1
This should fix your issue helvalius/nominatim-docker#18
@giantamoeba also got it right
Curl does not follow redirect by default
If you try to open http://download.geofabrik.de/europe/germany-latest.osm.pbf you will be redirected (via HTTP 302) to http://ftp5.gwdg.de/pub/misc/openstreetmap/download.geofabrik.de/germany-latest.osm.pbf
Avoiding this redirect will solve the issue.
from nominatim-docker.
ImreSamu
commented on July 19, 2024
But it seems that it is more of a postegres params issue. Any idea?
Osm2pgsql failed due to ERROR: PBF error: invalid BlobHeader size (> max_blob_header_size)
imho : probably a download error ( and problem with reading the input osm file )
Maybe an md5sum check will be helpful detecting this type of errors. The md5sum info can be downloaded from http://download.geofabrik.de/europe.html
for example:
from nominatim-docker.
maxze
commented on July 19, 2024
I ran into the same issue. It's not the pbf-file itself as I've tried using it in helvalius' docker container for nominatim and it seems to work just fine. Must be something else.
from nominatim-docker.
Bakaniko
commented on July 19, 2024
Hi,
Actually, I had the same problem today with Europe dataset.
I checked past issues so I found this specific one and change the url from https://download.geofabrik.de/europe-latest.osm.pbf to https://ftp5.gwdg.de/pub/misc/openstreetmap/download.geofabrik.de/europe-latest.osm.pbf
I think it is worth it to document that for large datasets, there is a redirection and that curl don't accept redirections.
Especially with latest version of nominatim-docker who specifically download the data (maybe we can have use a local uri, but I'm not sure).
from nominatim-docker.
leonardehrenfried
commented on July 19, 2024
curl does follow redirects if you pass the -L flag which we do: https://github.com/mediagis/nominatim-docker/blob/master/3.6/init.sh#L28
Can you double-check your issue?
from nominatim-docker.
philipkozeny
commented on July 19, 2024
I also ran the 3.6 image with the European dataset at least 10 times now without a problem. Do you have a more specific error log?
from nominatim-docker.
Bakaniko
commented on July 19, 2024
Hi,
I tried to reproduce this morning and I didn't get the error this time.
But Friday I had this one:
Fri Apr 23 11:30:25 UTC 2021
Error response from daemon: No such container: nominatim
0b98f53a84a1f798bff77d7759cf1e792126e9e6
+ trap stopServices TERM
+ /app/config.sh
+ id nominatim
+ useradd -m -p very_secure_password nominatim
+ IMPORT_FINISHED=/var/lib/postgresql/12/main/import-finished
+ '[' '!' -f /var/lib/postgresql/12/main/import-finished ']'
+ /app/init.sh
+ DATA_DIR=/app/src/data
+ OSMFILE=/app/src/data/data.osm.pbf
+ '[' false = true ']'
+ echo 'Skipping optional Wikipedia importance import'
Skipping optional Wikipedia importance import
+ '[' true = true ']'
+ curl https://www.nominatim.org/data/gb_postcode_data.sql.gz -o /app/src/data/gb_postcode_data.sql.gz
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 35.0M 100 35.0M 0 0 55.9M 0 --:--:-- --:--:-- --:--:-- 55.8M
+ '[' '' = true ']'
+ echo 'Skipping optional US postcode import'
Skipping optional US postcode import
+ echo Downloading OSM extract from https://download.geofabrik.de/europe-latest.osm.pbf
Downloading OSM extract from https://download.geofabrik.de/europe-latest.osm.pbf
+ curl -L https://download.geofabrik.de/europe-latest.osm.pbf --create-dirs -o /app/src/data/data.osm.pbf
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 270 100 270 0 0 2700 0 --:--:-- --:--:-- --:--:-- 2700
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
curl: (56) Recv failure: Connection reset by peer
Fri Apr 23 11:30:27 UTC 2021I use this code :
# Start timing
echo $(date)
SECONDS=0
#Stop the nominatim container if needed
docker stop nominatim
# change directory to docker folder
cd ~/nominatim-docker/3.6/
# display last commit hash for versionning
git log --format="%H" -n 1
# Build with no cache to have a fresh start
# docker build --no-cache -t nominatim . && \
docker run -it --rm --shm-size=8G \
-e PBF_URL=https://download.geofabrik.de/europe-latest.osm.pbf \
-e REPLICATION_URL=https://download.geofabrik.de/europe-updates/ \
-e IMPORT_WIKIPEDIA=false \
-e NOMINATIM_PASSWORD=very_secure_password \
-e THREADS=6 \
-e POSTGRES_MAX_WAL_SIZE=3GB \
-e IMPORT_STYLE=street \
-e IMPORT_GB_POSTCODES=true \
-p 8080:8080 \
--name nominatim \
mediagis/nominatim:3.6from nominatim-docker.
Related Issues (20)
- Poor response time after loading the docker HOT 1
- [TIP] Adding Wikipedia/Wikidata Rankings after Initial Import
- The configuration of "nominatim import" can not transfer to osm2pgsql.osm2pgsql execut slow. HOT 2
- Geometry Mismatch on World import in placex table HOT 5
- remote postgres host parse error HOT 3
- Import Style Customization HOT 4
- OSM import fails due to URL corruption HOT 2
- Replication logs not generating
- Error After Installation - How to troubleshoot HOT 11
- api request format HOT 1
- 漏洞 HOT 10
- External database gets recreated upon pod restart HOT 5
- nominatim reverse cli call with external database HOT 1
- Parallelize GitHub CI to cut down on overall timing HOT 1
- Cannot shutdown container if replication is not active
- Setup external PostgreSQL from dump and connect docker nominatim to it HOT 1
- Existing Full Planet GeoServer OSM HOT 4
- Replication is not updating
- psycopg2.OperationalError: server closed the connection unexpectedly HOT 2
- curl: (60) SSL certificate problem: self-signed certificate in certificate chain
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from nominatim-docker.