Comments (13)
@CloudNiner If the Dockerfile is working well for your use case, can you open a PR so I can add it to the repo?
from osmexpress.
thanks for the report - haven't dug into this yet but it looks like the same issue as google/s2geometry#87 - presumably need to link against brew openssl instead of system one
Will also work on updating the binary releases
from osmexpress.
I uploaded a new macOS binary here: https://github.com/protomaps/OSMExpress/releases/tag/0.0.4
I suspect this problem can be isolated by getting s2geometry to compile. It's difficult for me to reproduce though as I'm on MacPorts instead of Homebrew. Can you try building https://github.com/google/s2geometry - It should be cloned already in vendor/s2geometry ?
I suspect you may need to pass some configuration flags like this https://stackoverflow.com/a/16249265 to get S2's cmake to find the brew openSSL
from osmexpress.
Sorry about the delay in responding. Ultimately I was unsuccessful in getting a build on Catalina. The s2geometry issue you linked states that after a brew upgrade openssl
that the build should just work, which is not the case for me -- I've already got the latest openssl installed via brew.
Then I tried building s2geometry following vendor/s2geometry/README.md
:
cd vendor/s2geometry
mdkir build
cd build
cmake -DOPENSSL_INCLUDE_DIR=/usr/local/Cellar/[email protected]/1.1.1d/include/ ..
make
which fails with
/Users/cloudniner/src/OSMExpress/vendor/s2geometry/src/s2/s2shapeutil_testing.cc:20:10: fatal error: 'gtest/gtest.h' file not found
Which is expected since the README told me to install gtest. As far as I can tell there's no brew package for gtest so that's where I stopped rather than building from source and further complicating install.
I was also unable to run the pre-built binary you uploaded as 0.0.4, which failed with the error:
dyld: Library not loaded: /opt/local/lib/libssl.1.1.dylib
Referenced from: /Users/andrew/bin/osmx
Reason: image not found
[1] 48461 abort osmx
I found this stackoverflow which suggests using install_name_tool
on the binary. No luck with that either. Looks like homebrew removed openssl 1.0 formula: aisingapore/TagUI#635. Their workaround suggests curling a ruby script but I don't want to wreck my homebrew install so that's where I stopped on this one.
from osmexpress.
Instead of continuing with MacOS install woes I wrote a minimal Dockerfile based on Ubuntu 18.04. This image is 74mb in size so its definitely bigger than the executable, but its not unwieldy. If this is useful, I'd be happy to open a PR to add it to the repo.
from osmexpress.
@CloudNiner when building S2 can you use make s2
instead of make
which should ignore the test-related targets?
Thanks for the docker file example - can you verify that the Docker image works correctly on a large file e.g. osmx expand planet.osm.pbf planet.osmx
from osmexpress.
can you verify that the Docker image works correctly on a large file
How big is the expanded planet.osmx? My computer only has a 256gb drive.
from osmexpress.
Ultimately I think it's a shame that S2 depends on OpenSSL just for its Bignum class. Maybe it won't be too difficult to port it to a standalone library.
from osmexpress.
can you verify that the Docker image works correctly on a large file
How big is the expanded planet.osmx? My computer only has a 256gb drive.
It's about 600 GB.
from osmexpress.
:( I don't think I can run on the planet file then.
can you use
make s2
instead ofmake
That fails with
[100%] Linking CXX shared library libs2.dylib
ld: cannot link directly with dylib/framework, your binary is not an allowed client of /usr/lib/libcrypto.dylib for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[3]: *** [libs2.dylib] Error 1
make[2]: *** [CMakeFiles/s2.dir/all] Error 2
make[1]: *** [CMakeFiles/s2.dir/rule] Error 2
make: *** [s2] Error 2
from osmexpress.
PS I commented on the closed s2 issue you linked originally since it looks like we've gone full circle
from osmexpress.
@CloudNiner thanks. I added (experimental) next to Docker in the readme as I don't have any production experience running osmx inside it; two concerns pop out immediately
- the way the database (lmdb) works is by
mmap
ing the entire DB into memory, I'm not certain how this interacts with the Docker runtime - but if you're able to make this work for a extract it should be fine - Also, it should be fine in the non-Docker case if multiple processes open the same
.osmx
file simultaneously - the locking and transactions should be handled by the lmdb code but I'm not sure if this works correctly with docker and the filesystem
If you have success using it in Docker can you post it here?
from osmexpress.
Those are great points, thanks for the heads up.
I only got as far as generating an osmx file last night, which worked fine. It may be a bit until I have time to go further with this but I'll report back when able. Thanks for all your quick responses!
from osmexpress.
Related Issues (20)
- Query for multiple nodes, ways, or relations at once by ID HOT 3
- Segfault running query command with no arguments HOT 2
- augmented diff example program HOT 7
- get the approximate cell covering for a relation HOT 1
- What's the preferred linux package name for OSM Express? HOT 3
- Document how to build with system libs instead of vendored libs HOT 4
- (windows) MDB_Transaction Error when trying to work with Windows Subsystem for Linux HOT 3
- Add polygon extract queries to the Python API HOT 12
- Publish 0.0.4 osmx python bindings HOT 2
- Converting full planet pbf file fail HOT 4
- Include all metadata for locations
- S2CellUnion Expand for extract operation HOT 4
- Script to render examples/screenshot.png HOT 1
- Possible to use this to serve tiles? HOT 3
- Cannot extract node tags HOT 8
- Continuous growths of the osmx file with nightly updates HOT 5
- Crashes during parsing of malformed OSMX files HOT 2
- Master build failing on Ubuntu 22.04.1 LTS HOT 1
- Investigate string pools HOT 5
- Transactions will fail when db exceeds 1 TB
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from osmexpress.