Code Monkey home page Code Monkey logo

Comments (2)

wipfli avatar wipfli commented on May 19, 2024

wipfli/aws-planetiler#1

from planetiler.

msbarry avatar msbarry commented on May 19, 2024

What do people think of a single script that builds the arguments/JVM options based on the area being built and method used to run (docker, jar, maven build)

#!/usr/bin/env bash

# Usage: quickstart.sh {--docker, --jar, --source} {--planet,--area=monaco,massachusetts,etc.} [--memory=5g] other args...

set -o errexit
set -o pipefail
set -o nounset

JAVA="${JAVA:-java}"
METHOD="build"
AREA="monaco"
STORAGE="mmap"
PLANETILER_ARGS=("--download" "--force")
MEMORY=""
DRY_RUN=""
DOCKER_DIR="$(pwd)/data"

# Parse some args into vars, pass the rest through to planetiler
while [[ $# -gt 0 ]]; do
  case $1 in
    --jar) METHOD="jar" ;;
    --build|--source) METHOD="build" ;;
    --docker) METHOD="docker" ;;

    --dockerdir=*) DOCKER_DIR="${1#*=}" ;;
    --dockerdir) DOCKER_DIR="$2"; shift ;;

    --area=*) AREA="${1#*=}" ;;
    --area) AREA="$2"; shift ;;
    --planet) AREA="planet" ;;

    --memory=*) MEMORY="${MEMORY:-"-Xmx${1#*=}"}" ;;
    --memory) MEMORY="${MEMORY:-"-Xmx$2"}"; shift ;;
    --ram) STORAGE="ram" ;;

    --dry-run) DRY_RUN="true" ;;

    *) PLANETILER_ARGS+=("$1") ;;
  esac
  shift
done

PLANETILER_ARGS+=("--area=$AREA")
PLANETILER_ARGS+=("--storage=$STORAGE")

# Configure memory settings based on the area being built
case $AREA in
  planet)
    PLANETILER_ARGS+=("--nodemap-type=array" "--download-threads=20" "--download-chunk-size-mb=500")
    case "$STORAGE" in
      ram) MEMORY="${MEMORY:-"-Xmx150g"}" ;;
      mmap) MEMORY="${MEMORY:-"-Xmx30g -Xmn15g"}" ;;
    esac
    ;;
  monaco)
    # Use mini extracts for monaco
    PLANETILER_ARGS+=("--water-polygons-url=https://github.com/onthegomap/planetiler/raw/main/planetiler-core/src/test/resources/water-polygons-split-3857.zip")
    PLANETILER_ARGS+=("--water-polygons-path=data/sources/monaco-water.zip")
    PLANETILER_ARGS+=("--natural-earth-url=https://github.com/onthegomap/planetiler/raw/main/planetiler-core/src/test/resources/natural_earth_vector.sqlite.zip")
    PLANETILER_ARGS+=("--natural-earth-path=data/sources/monaco-natural_earth_vector.sqlite.zip")
    ;;
esac
# For extracts, use default nodemap type (sortedtable) and -Xmx (25% of RAM up to 25GB) and hope for the best.
# You can set --memory=5g if you want to change it.

JVM_ARGS="-XX:+UseParallelGC $MEMORY"

echo "METHOD=\"$METHOD\""
echo "JVM_ARGS=\"${JVM_ARGS}\""
echo "PLANETILER_ARGS=\"${PLANETILER_ARGS[*]}\""
echo "DRY_RUN=\"${DRY_RUN:-false}\""
echo ""

sleep 3

function run() {
  echo "$ $*"
  if [ "$DRY_RUN" != "true" ]
  then
    eval "$*"
  fi
}

function check_java_version() {
  if [ -z "$(which java)" ]; then
    echo "java not found on path"
    exit 1
  else
    OUTPUT="$($JAVA -jar "$1" --help 2>&1 || echo OK)"
    if [[ "$OUTPUT" =~ "UnsupportedClassVersionError" ]]; then
      echo "Wrong version of java installed, need at least 16 but found:"
      $JAVA --version
      exit 1
    fi
  fi
}

# Run planetiler
case $METHOD in
  docker)
    run docker run -e JAVA_TOOL_OPTIONS=\'"${JVM_ARGS}"\' -v "$DOCKER_DIR":/data ghcr.io/onthegomap/planetiler:latest "${PLANETILER_ARGS[@]}"
    ;;
  jar)
    run wget -nc https://github.com/onthegomap/planetiler/releases/latest/download/planetiler.jar
    check_java_version planetiler.jar
    run "$JAVA" "${JVM_ARGS}" -jar planetiler.jar "${PLANETILER_ARGS[@]}"
    ;;
  build)
    run ./mvnw -DskipTests --projects planetiler-dist -am clean package
    run "$JAVA" "${JVM_ARGS}" -jar planetiler-dist/target/*with-deps.jar "${PLANETILER_ARGS[@]}"
    ;;
esac

The alternative would be seperate scripts with the full commands inlined, something like:

  • quickstart-planet-docker.sh
  • quickstart-planet-jar.sh
  • quickstart-planet-from-source.sh
  • quickstart-extract-docker.sh
  • quickstart-extract-jar.sh
  • quickstart-extract-from-source.sh

from planetiler.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.