t-rex-tileserver / t-rex Goto Github PK

View Code? Open in Web Editor NEW

539.0 539.0 66.0 16.58 MB

t-rex is a vector tile server specialized on publishing MVT tiles from your own data

Home Page: https://t-rex.tileserver.ch/

License: MIT License

Rust 98.54% Makefile 0.85% Shell 0.05% HTML 0.15% Dockerfile 0.41%

database mapbox-gl-js openlayers tile-cache vector-tiles

t-rex's Introduction

t-rex

t-rex is a vector tile server specialized on publishing MVT tiles from your own data.

Features

Support for PostGIS databases and GDAL vector formats
Auto-detection of layers in data source
Built-in viewers for data display and inspection
Tile generation command with simple parallelization
Automatic reprojection to grid CRS
Support for custom tile grids

Presentations

T-rex, a vector tile server for your own data (FOSS4G 2017): slides
Vector Tiles - Introduction & Usage with QGIS (User meeting Bern 21.6.17): slides
Von WMS zu WMTS zu Vektor-Tiles (FOSSGIS 2017): Video
Workshop "Vector Tiles" (GEOSummit Bern 7.6.16): slides

Examples

basemap.de
Swiss Ornithological Institute, Birds of Switzerland

Usage

Quick tour

t_rex serve --dbconn postgresql://user:pass@localhost/osm2vectortiles

Tiles are then served at http://localhost:6767/{layer}/{z}/{x}/{y}.pbf

A list of all detected layers is available at http://localhost:6767/

Use a tile cache:

t_rex serve --dbconn postgresql://user:pass@localhost/osm2vectortiles --cache /tmp/mvtcache

Generate a configuration template:

t_rex genconfig --dbconn postgresql://user:pass@localhost/osm2vectortiles | tee osm2vectortiles.toml

Run server with configuration file:

t_rex serve --config osm2vectortiles.toml

Generate tiles for cache:

t_rex generate --config osm2vectortiles.toml

Increase log level:

t_rex serve --loglevel debug --dbconn postgresql://user:pass@localhost/osm2vectortiles

For developers

t-rex is written in Rust. Minimal required rustc version is 1.45.

Software Requirements

Ubuntu 20.04 (Focal Fossa):

sudo apt install cargo libssl-dev libgdal-dev clang

Build and run

Build:

cargo build

Run tests:

cargo test --all

Run server with DB connection:

cargo run -- serve --dbconn postgresql://t_rex:[email protected]:5439/t_rex_tests

Decode a vector tile:

curl --silent http://127.0.0.1:6767/ne_10m_populated_places/5/31/17.pbf | protoc --decode=vector_tile.Tile t-rex-core/src/mvt/vector_tile.proto

Database tests

Unit tests which need a PostgreSQL connection are ignored by default.

Start Test DB:

docker run -p 127.0.0.1:5439:5432 -d --name trextestdb --rm sourcepole/trextestdb

To run the database tests, declare the connection in an environment variable DBCONN:

export DBCONN=postgresql://t_rex:[email protected]:5439/t_rex_tests

Run the tests with

cargo test --all -- --ignored

Creating test database locally:

# Set Postgresql environment variables when needed: PGHOST, PGPORT, PGUSER, PGPASSWORD
cd data
make createdb loaddata

S3 tests

Unit tests which need a S3 connection are skipped by default.

Install MinIO Client.

Start Test S3

docker run -d --rm -p 9000:9000 -e MINIO_REGION_NAME=my-region -e MINIO_ACCESS_KEY=miniostorage -e MINIO_SECRET_KEY=miniostorage minio/minio server /data && sleep 5 && mc config host add local-docker http://localhost:9000 miniostorage miniostorage && mc mb local-docker/trex && mc policy set download local-docker/trex

To run the S3 tests, declare that there is a S3 available in an environment vaiable S3TEST:

export S3TEST=true

Run the tests with

cargo test --all-features --all -- --ignored

License

t-rex is released under the MIT License.

t-rex's People

Contributors

Stargazers

Watchers

Forkers

gijs fgcarto cuulee thomersch amandasaurus kyroskoh joostvenema lygstate wykks kiambogo wata909 chansuke cnfeigo danielquisbert jbg lnicola frodrigo adonig yuskesuzki apezel nimix myfreebrain farkal stvno yili9111 jpbede zjanetzek blesss mwjsanders paulyc sebastian-r-schmidt pdok jtvoorde vittrup88 arbakker malthe jeroentervoorde cpoolegla allenk summerdaydream chau-intl tobwen allanin pvgenuchten faizol 00mjk azorlogh christianbeilschmidt sharkandshark 4812571 procrastinatio dzfranklin woutervisscher k-bokka lukaslohoff rizwanalvi1 curtis18 fredj metabuilddev edouardpoitras solgunsevda robert-werner roger120981 simonpenel cosmosecho

t-rex's Issues

Rendering artifacts, square tiles

I noticed that I get rendering artifacts in MapBox GL and OpenLayers. Is there a setting to prevent the "square tiles" here I mean the edge around them?

Mapbox GL

OpenLayers

Warnings: Converting field 'createdAt'/ 'updatedAt' of type timestamptz to TEXT

Since I started using sequelizejs as ORM I (or sequelize...) added createdAt and updatedAt fields. When I start the server I suddenly get these warnings. Can I somehow get rid of these?

root@vps412437:/home/usr# t_rex serve --config osm.toml 
2017-07-14 21:47:00.297 INFO Reading configuration from 'osm.toml'
2017-07-14 21:47:00.334 WARN Layer 'datapoints': Converting field 'createdAt' of type timestamptz to TEXT
2017-07-14 21:47:00.334 WARN Layer 'datapoints': Converting field 'updatedAt' of type timestamptz to TEXT
2017-07-14 21:47:00.334 INFO Tile cache directory: /mnt/vdb/tmp/mvtcache
2017-07-14 21:47:00.337 WARN Layer 'datapoints': Converting field 'createdAt' of type timestamptz to TEXT
2017-07-14 21:47:00.337 WARN Layer 'datapoints': Converting field 'updatedAt' of type timestamptz to TEXT
2017-07-14 21:47:00.340 WARN Layer 'datapoints': Converting field 'createdAt' of type timestamptz to TEXT
2017-07-14 21:47:00.340 WARN Layer 'datapoints': Converting field 'updatedAt' of type timestamptz to TEXT
2017-07-14 21:47:00.341 WARN Layer 'datapoints': Converting field 'createdAt' of type timestamptz to TEXT
2017-07-14 21:47:00.341 WARN Layer 'datapoints': Converting field 'updatedAt' of type timestamptz to TEXT
Listening on http://127.0.0.1:1337
Ctrl-C to shutdown server

Enable SSL mode for PostgreSQL connections

I don't seem to be able to enable SSL in database connections. I'm running the following using t_rex 0.7.7-dev:

./t_rex genconfig --clip true --simplify true --dbconn 'postgres://***:***@***:***/***?sslmode=require'

And get the following output:

2017-07-26 13:17:28.530 ERROR Error opening a connection: Error reported by Postgres: FATAL: no pg_hba.conf entry for host "***", user "***", database "***", SSL off
[ logs repeated ]
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: InitializationError(Some("Error opening a connection: Error reported by Postgres: FATAL: no pg_hba.conf entry for host \"***\", user \"***\", database \"***\", SSL off"))', /checkout/src/libcore/result.rs:859
stack backtrace:
   0: std::sys::imp::backtrace::tracing::imp::unwind_backtrace
             at /checkout/src/libstd/sys/unix/backtrace/tracing/gcc_s.rs:49
   1: std::panicking::default_hook::{{closure}}
             at /checkout/src/libstd/sys_common/backtrace.rs:71
             at /checkout/src/libstd/sys_common/backtrace.rs:60
             at /checkout/src/libstd/panicking.rs:355
   2: std::panicking::rust_panic_with_hook
             at /checkout/src/libstd/panicking.rs:371
             at /checkout/src/libstd/panicking.rs:549
   3: std::panicking::begin_panic
             at /checkout/src/libstd/panicking.rs:511
   4: std::panicking::begin_panic_fmt
             at /checkout/src/libstd/panicking.rs:495
   5: core::panicking::panic_fmt
             at /checkout/src/libstd/panicking.rs:471
   6: core::result::unwrap_failed
   7: t_rex::datasource::postgis::PostgisInput::connected
   8: t_rex::webserver::server::service_from_args
   9: t_rex::webserver::server::gen_config
  10: t_rex::main
  11: main
  12: __libc_start_main
  13: <unknown>
1

Is there something I'm missing in order to specify a SSL connection?

what nodeno and nodes options do exactly?

After generating lots of cache tiles I get 'No space left on device' for basically every interaction with my machine. This seemed strange at first as my disk space was at 50% or so. After some digging it appears to be that I ran out of inodes on the /dev/vda1 partition:

root@vps412437:/home/usr# df -i
Filesystem      Inodes   IUsed  IFree IUse% Mounted on
udev            247614     403 247211    1% /dev
tmpfs           249890     520 249370    1% /run
/dev/vda1      1280000 1280000      0  100% /
tmpfs           249890       3 249887    1% /dev/shm
tmpfs           249890       3 249887    1% /run/lock
tmpfs           249890      16 249874    1% /sys/fs/cgroup
tmpfs           249890       4 249886    1% /run/user/0

It seems you need to create more inodes at system install to solve this (or of course have a bigger machine from the start...

From this background I was wondering what the cache generation options:
--nodeno Number of this nodes (0 <= n < nodes)
--nodes Number of generator nodes

do?

Use zoom token in osm2vectortiles example

Instead of z(!scale_denominator!) it's better to use the !zoom! token.

Support for multi-geomertry types

Cool project, having a lot of fun playing with it.

I encountered an issue while retrieving building data from a table. This table contains Polygons and Multipolygons, which seems to lead into decoding problems, as far as I understand. I tried looking into the code - I am a total Rust noob -, but it looks to me as if it isn't possible to mix types. Is this intended or did I misunderstand something?

Specifically the following log appears:
thread '<unnamed>' panicked at 'error retrieving column "polygon": Conversion(StringError("cannot convert geometry to Geometry"))', /root/.cargo/registry/src/github.com-88ac128001ac3a9a/postgres-0.11.9/src/rows.rs:204

Have linkable URLs for tileset demos

When viewing info, mapbox gl, openlayers, x-ray, and inspector for a tile set the URL does not change. If the URL changed, I could link a page to others to show them what I'm looking at. This would be useful when working with others.

Show zoom on x-ray view

It would be useful to show the zoom on the x-ray view

bbox for generating tiles

Hi! Great tool!
We try to generate pbf tiles for only the Netherlands and just a few zoom levels. When using t_rex generate it will always generate tiles for the whole world/gridset and has to start at zoom level 0.

Is it possible to include settings for a bounding box and zoom levels for which it will generate the tiles in? Or would it be possible to add these variables? Like for example in tl copy from https://github.com/mojodna/tl
-z 6 -Z 8 -b "-124.4096 32.5343 -114.1308 42.0095"

Thanks!

SSL support

Build with postgres feature openssl currently fails on Windows:

running: "C:\\Program Files (x86)\\Microsoft Visual Studio 12.0\\VC/bin\\amd64\\cl.exe" "/nologo" "/MD" "/Z7" "/FoC:\\projects\\t-rex\\target\\debug\\build\\openssl-sys-extras-6cdd9aaab3f99a18\\out\\src\\openssl_shim.o" "/c" "src/openssl_shim.c"
openssl_shim.c
src/openssl_shim.c(1) : fatal error C1083: Cannot open include file: 'openssl/hmac.h': No such file or directory
ExitStatus(ExitStatus(2))

Can not convert GEOMETRY type

When I try to run t_rex on a postgis db it can not render some of the tables. Those tables hold polygon features like buildings, landuse, etc. The error it shows on hte console is:

2017-08-07 12:54:34.841 ERROR Layer 'osm_buildings': Error converting between Postgres and Rust types: Unknown geometry type GEOMETRY
2017-08-07 12:54:34.841 ERROR Row { statement: Statement { name: "s0", parameter_types: [Float8, Float8, Float8, Float8], columns: [Column { name: "geometry", type_: Other(Other { name: "geometry", oid: 67041, kind: Simple, schema: "public" }) }, Column { name: "id", type_: Int4 }, Column { name: "osm_id", type_: Int8 }, Column { name: "name", type_: Varchar }, Column { name: "type", type_: Varchar }, Column { name: "height", type_: Varchar }] } }

Hope this is somewhat helpful.

unminified viewer.js

hey @pka, more a question than an issue.
I wanted to try to add some things on the default viewer but I could only find t-rex/t-rex-webserver/src/static/viewer.js which is a minified thing. do you have the non minified code somewhere?

cheers
Marco

Drop rustc-serialize dependency

It has been deprecated: announcement.

Is there any functionality you would require from Serde or another library before this would be possible?

Issues at lower zoom

I noticed following behavior and do not know 'what is going on':
When I worked with 50K points and generated tiles, the tiles for the lower zooms were ready in an instant (couple of seconds). Now I have 180K points and before it generates zoom 0 it scans features for 3 minutes or so.

Furthermore the points do not appear in the lower zoom, also not when cached (and after refreshing my browser cache of course).
I would expect when zoom 0 is cached that you do not see the request, but the request shows and I get following errors at lower zooms:

2017-07-02 12:17:42.913 INFO GET /customlayer/3/3/2.pbf
2017-07-02 12:17:48.136 INFO GET /customlayer/3/3/3.pbf
2017-07-02 12:17:49.160 INFO GET /customlayer/3/5/2.pbf
thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: GetTimeout(None)', /checkout/src/libcore/result.rs:859
note: Run with `RUST_BACKTRACE=1` for a backtrace.
2017-07-02 12:18:09.522 INFO GET /customlayer/3/5/3.pbf
thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: GetTimeout(None)', /checkout/src/libcore/result.rs:859
2017-07-02 12:18:10.105 INFO GET /customlayer/2/2/1.pbf
thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: GetTimeout(None)', /checkout/src/libcore/result.rs:859
2017-07-02 12:18:12.322 INFO GET /customlayer/2/1/1.pbf
thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: GetTimeout(None)', /checkout/src/libcore/result.rs:859
2017-07-02 12:18:12.950 INFO GET /customlayer/2/2/0.pbf
thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: GetTimeout(None)', /checkout/src/libcore/result.rs:859
2017-07-02 12:18:19.238 INFO GET /customlayer/2/1/0.pbf
2017-07-02 12:18:31.114 INFO GET /customlayer/2/3/1.pbf
thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: GetTimeout(None)', /checkout/src/libcore/result.rs:859
2017-07-02 12:18:42.357 INFO GET /customlayer/2/3/0.pbf
thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: GetTimeout(None)', /checkout/src/libcore/result.rs:859
2017-07-02 12:18:42.971 INFO GET /customlayer/1/1/0.pbf
thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: GetTimeout(None)', /checkout/src/libcore/result.rs:859
2017-07-02 12:18:49.252 INFO GET /customlayer/1/0/0.pbf
thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: GetTimeout(None)', /checkout/src/libcore/result.rs:859
2017-07-02 12:19:01.155 INFO GET /customlayer/1/0/1.pbf
thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: GetTimeout(None)', /checkout/src/libcore/result.rs:859
2017-07-02 12:19:12.382 INFO GET /customlayer/1/1/1.pbf
thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: GetTimeout(None)', /checkout/src/libcore/result.rs:859
2017-07-02 12:19:12.990 INFO GET /customlayer/0/0/0.pbf
thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: GetTimeout(None)', /checkout/src/libcore/result.rs:859
2017-07-02 12:19:19.276 INFO GET /customlayer/0/0/0.pbf
thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: GetTimeout(None)', /checkout/src/libcore/result.rs:859
2017-07-02 12:19:31.178 INFO GET /customlayer/0/0/0.pbf
thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: GetTimeout(None)', /checkout/src/libcore/result.rs:859
thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: GetTimeout(None)', /checkout/src/libcore/result.rs:859
thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: GetTimeout(None)', /checkout/src/libcore/result.rs:859
thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: GetTimeout(None)', /checkout/src/libcore/result.rs:859

Any suggestions are highly appreciated.

Add a basemap

Hello and first of all congrats for working on this project. It is on the of the best and easiest vector server to use. 👍

Since I use data for Albania, which is a very small country, sometimes is difficult to see the rendered data in the browser. Would it be possible to have a basemap for quick reference.

Cheers.

Extend Web mercator grid definition to zoom level 22

Hi pka,

I was wondering if it is possible to create group layers, ie. where the data in a layer comes from more than one table in PostGIS. We love your work and were hoping to use it in a project that requires nearly two dozen tables to be grouped into 4 displayable layers.

Is this possible? Any insights would be greatly appreciated.

Cheers,
Tom Cuthill

insert records

I would like to insert records (points) from the client browser into the postgis database. This is not included in t-rex. Is this something that could be added in the future? Could I just run another server (e.g. express.js) next to t-rex to provide me with this user data feature? I am on unknown territory...

'Missing' tiles at various zoom levels

First off, I wanted to say how impressed I am with this project - fantastic stuff! :)

When rendering some road data, I am noticing that there are a significant number of tiles missing (not rendering) at various zoom levels (in particular, lower zoom levels). If I zoom in quite a lot, then I get a better result. I have tested this in Mapbox, OpenLayer, and X-Ray/Inspector all with the same result. The only thing that I found in common with the missing tiles is that they contain >= 1000 features. I noticed that you added a limit of 1000 features when simplify is true - I tried to set this to false, as well as running 0.7.1 but I get the same result. The data is in the correct projection, and I can view it in QGIS from Postgis without issue.

Do you have any idea what could be causing this, and how I could remedy this?

For reference, this is OSM data that was inserted into Postgres with osm2pgsql default schema.

Mapbox:

Inspector:

v0.8.0-pre1 - OpenSSL version issues

On a fresh debian 9.1 install, I cannot run any t_rex command because of the ssl version installed.

t_rex: error while loading shared libraries: libssl.so.1.0.0: cannot open shared object file: No such file or directory

Here's what's installed by default:

/usr/lib/x86_64-linux-gnu/libssl3.so
/usr/lib/x86_64-linux-gnu/libssl.so.1.0.2
/usr/lib/x86_64-linux-gnu/libssl.so.1.1
OpenSSL 1.1.0f # openssl version

When trying to symlink 1.1 to 1.0.0 (just curious) it's telling me OpenSSL 1.0.0 is not installed.

What should I do?

Also, at this moment I don't need to use ssl. Is there a flag to skip it?

Thank you!

Add Last-Modified header & option for max-age

Currently there is Cache-Control:max-age=43200 but it's not appropriate if the source update frequently. It would be great to have an option to control this header.
Also Last-Modified would be usefull in all case, even if max-age is set. Maybe only when cache is enabled.

Generating tilesets with non-rectangular bounds

Currently in order to generate tiles we need to define the extent of tiles in rectangular terms.

We are very happy with t_rex and we are using it to generate tiles for a number of projects we have running. It is often the case that the tiles we need to generate cover very non-rectangular regions of Australia. I'm sure that other users will want to generate tiles over non-rectangular boundaries, particularly at higher zoom levels.

As a consequence, the tile generator runs for long periods, much of the time generating empty tiles which lie outside of the area of interest but within the single rectangular box bounding the tiles. It is very easy to specify the desired bound as a geometry in a table inside PostGIS. A wonderful enhancement to the generator would be to have some way of referencing the tile boundaries based on that geometry.

Thanks for all your efforts with t_rex and for all the benefits we and others have gained as a result!

Cheers,
Tom Cuthill

Respond to favicon.ico requests

Browsers will make GET requests to /favicon.ico, resulting in the warning 2017-05-31 14:21:01.824 WARN Get V4(192.168.1.2:62814) AbsolutePath("/favicon.ico") "File Not Found" Some(NotFound). It would be useful to either return a blank file, avoiding the 404 error, or an icon for t-rex.

Redis for caching

How would one go about using Redis for caching with t-rex?

drawing issues on Mapbox

T-rex is awesome. Everything seems to be working very well. However i'm having some issues with high-level zoom (above 12). I'm attaching a few screenshots. Was wondering if it was an issue with zoom level. But not sure

Support for other reference systems than EPSG 3857

Currently, only geometries in EPSG 3857 (Web mercator) are supported, despite an architecture which is targeted for generic reference system support. The main reason is a limitation of rust-postgis. It is planned to port the rust-postgis WKB reader (with support for SRID detection at runtime) to rust-geo, which solves other limitations as well.

Data driven style not working in built-in Mapbox GL viewer

I need to access a row returned from the executed SQL statement in a layer, and use it in the styling for that layer.

For example, lets say I have a layer defined as the following:

[[tileset]]
name = "osm"

[[tileset.layer]]
name = "line"
geometry_field = "geometry"
geometry_type = "LINESTRING"
fid_field = "osm_id"
simplify = true
  [[tileset.layer.query]]
  sql = """
    SELECT name, type, id as osm_id, ST_Union(geometry) AS geometry, colour
    FROM my_geom_table"""

where colour here is the hex value as text of the colour for that line (i.e. #ff00ff).

Then, in my styling guide for the layer, I need to set the colour for each line to be its value as is returned from the db. Ideally, something like this:

[[tileset.layer]]
name = "line"
# ...
[tileset.layer.style]
  type = "line"
  [tileset.layer.style.paint]
  line-color = [colour]

I have this functionality in Mapnik (moving from PNG raster tiles to vector tiles), as the following:
https://github.com/trifectalabs/roadquality/blob/master/resources/tilestache/palette.mss#L98
in my style guide, and rating_colour is referencing the returned row from the executed SQL: https://github.com/trifectalabs/roadquality/blob/master/resources/tilestache/project.mml#L240

I realize that this may not be as t-rex specific, but maybe more the TOML implementation of the Mapbox Style Spec, but can you offer any insight whether this is feasible, or something that could be added to the spec?

Thanks!

Strictly speaking, Google tile references should have their origin at the top left, not bottom left

According to the Google Maps API, its tile schema has an origin at the top left (see here: https://developers.google.com/maps/documentation/javascript/coordinates).

Here is web mercator schema in t_rex's source code:

Notice that the origin is set at the bottom left (TMS style) not the top left.

If the caller follows the google standard for referencing tiles, they will be inverted.

Cheers,
Tom

Configuration file documentation

I can't find any documentation on the configuration, just an example at https://github.com/pka/t-rex#configuration

I'm mainly interested in tileset.layer and tileset.layer.query, what properties can be set, and what their meanings are.

Directory layout?

The file directory cache uses a very simple Z/X/Y.pbf layout scheme. This is OK for many things, but you can wind up with a lot of file entries per directory. Especially if you're going to create all the vectortors for zoom 14. Most other tools which store tiles use a more split up format. I have a branch which uses the tile TileStash safe format (Z/xxx/xxx/yyy/yyy.pbf) format.

Would it be a good idea to change the directory layout?

Running t_rex from within a Docker container cannot connect

Thanks for building the server, it's really cool. I'm attempting to run the t_rex server from within docker but I can't connect to it from the host.

I suspect the problem is that the server runs on docker at 127.0.0.1:6767 but to make it connectable from the host it needs to run at 0.0.0.0:6767. This is how I have to run uwsgi when I want to run my python apps within docker

Can you suggest how I can get t_rex to run on 0.0.0.0

Thanks

Better seperation of layer definitions and deployment information

I'm writing layer definitions I'd like others to be able to use, but the config files are a mix of layer definitions (tileset stuff) and deployment information like IP addresses ([webserver] stuff). It would be good to have these independent so I can distribute layer definitions without making too many assumptions about someone's configuration.

Use libpq defaults connecting to postgresql

Right now if I want to use a socket to connect to PostgreSQL I need to set url = "postgresql://pnorman@%2Fvar%2Frun%2Fpostgresql/gis". This has two big flaws: I'm hard-coding the socket path, which varies depending on system, and the username, which is unsuitable for layer definitions I intend to distribute to others.

How can I write it so it gets the system defaults?

Ability to use more than two different queries per layer

I would like to be able to use multiple queries (in this case three), depending on the zoom level, because I use different generalised tables depending on zoom level.

Example:

[[tileset.layer]]
name = "waterareas"
srid = 3857
geometry_field = "geom"
geometry_type = "MULTIPOLYGON"
fid_field = "osm_id"

  [[tileset.layer.query]]
  sql = """
  SELECT osm_id, ST_Multi(geometry) AS geom
  FROM osm_waterareas
  WHERE geometry && !bbox!"""

  [[tileset.layer.query]]
  minzoom = 7
  maxzoom = 10
  sql = """
  SELECT osm_id, ST_Multi(geometry) AS geom
  FROM osm_waterareas_gen1
  WHERE geometry && !bbox!"""

  [[tileset.layer.query]]
  maxzoom = 6
  sql = """
  SELECT osm_id, ST_Multi(geometry) AS geom
  FROM osm_waterareas_gen0
  WHERE geometry && !bbox!"""

I tried this and that, but I could not achieve a consistent behaviour, I guess it isn't thought that way, right? Or is there a specific definition order I haven't understood yet?

Server halts after errors

When I zoom in quickly I get errors. For me these are 'acceptable' if the server would continue working after these, but it halts and becomes unresponsive (not after all errors, but it is common). Anything that can be done about that maybe? When I break with Ctrl+C and restart the server everything is working again.

Example of server output after which it frooze:


2017-06-29 05:57:57.946 INFO GET /customlayer/3/3/3.pbf
2017-06-29 05:57:57.949 INFO GET /customlayer/3/4/2.pbf
2017-06-29 05:57:58.446 WARN Get V4(92.110.243.159:48004) AbsolutePath("/customlayer/3/3/3.pbf") "Failed to send: Connection reset by peer (os error 104)" None
Error: Failed to send: Connection reset by peer (os error 104)
2017-06-29 05:57:58.657 INFO GET /customlayer/5/15/12.pbf
2017-06-29 05:57:58.960 INFO GET /customlayer/6/31/23.pbf
2017-06-29 05:57:58.961 INFO GET /customlayer/6/31/24.pbf
2017-06-29 06:13:53.458 WARN Get V4(92.110.243.159:46614) AbsolutePath("/customlayer/3/3/3.pbf") "Failed to send: Connection timed out (os error 110)" None
Error: Failed to send: Connection timed out (os error 110)
2017-06-29 06:13:53.459 INFO GET /customlayer/7/62/48.pbf
2017-06-29 06:14:14.450 WARN Get V4(92.110.243.159:48040) AbsolutePath("/customlayer/5/15/12.pbf") "Failed to send: Connection timed out (os error 110)" None
Error: Failed to send: Connection timed out (os error 110)
2017-06-29 06:14:14.450 INFO GET /customlayer/7/62/47.pbf
2017-06-29 06:14:14.451 INFO GET /customlayer/8/125/96.pbf
2017-06-29 06:14:43.122 WARN Get V4(92.110.243.159:48054) AbsolutePath("/customlayer/6/31/24.pbf") "Failed to send: Connection timed out (os error 110)" None
Error: Failed to send: Connection timed out (os error 110)
2017-06-29 06:14:43.123 INFO GET /customlayer/8/125/97.pbf
2017-06-29 06:14:43.123 INFO GET /customlayer/9/250/192.pbf
2017-06-29 06:14:44.658 WARN Get V4(92.110.243.159:48002) AbsolutePath("/customlayer/3/4/2.pbf") "Failed to send: Connection timed out (os error 110)" None
Error: Failed to send: Connection timed out (os error 110)
2017-06-29 06:14:44.659 INFO GET /customlayer/9/250/193.pbf
2017-06-29 06:30:21.116 WARN Get V4(92.110.243.159:48072) AbsolutePath("/customlayer/7/62/48.pbf") "Failed to send: Connection timed out (os error 110)" None
Error: Failed to send: Connection timed out (os error 110)
2017-06-29 06:30:21.127 INFO GET /customlayer/9/251/192.pbf
2017-06-29 06:30:21.138 INFO GET /customlayer/9/251/193.pbf
2017-06-29 06:30:21.140 INFO GET /customlayer/10/501/385.pbf
2017-06-29 06:30:42.099 WARN Get V4(92.110.243.159:48088) AbsolutePath("/customlayer/8/125/96.pbf") "Failed to send: Connection timed out (os error 110)" None
Error: Failed to send: Connection timed out (os error 110)
2017-06-29 06:30:42.103 INFO GET /customlayer/10/501/386.pbf
2017-06-29 06:31:10.770 WARN Get V4(92.110.243.159:48104) AbsolutePath("/customlayer/9/250/192.pbf") "Failed to send: Connection timed out (os error 110)" None
Error: Failed to send: Connection timed out (os error 110)
2017-06-29 06:31:10.772 INFO GET /customlayer/11/1002/772.pbf
2017-06-29 06:31:12.306 WARN Get V4(92.110.243.159:48106) AbsolutePath("/customlayer/9/250/193.pbf") "Failed to send: Connection timed out (os error 110)" None
Error: Failed to send: Connection timed out (os error 110)
2017-06-29 06:31:12.306 INFO GET /customlayer/11/1003/772.pbf

Handle non-mercator projections in viewer

Either disable viewers or support non-mercator projections...

thread 'main' panicked at 'no entry found for key'

When I add an 'extent' option I get an error, what is going wrong here?:

root@vps:/home/user# RUST_BACKTRACE=1 t_rex generate --config osm.toml --overwrite true --extent 4,52,5,53
2017-07-02 20:46:24.978 INFO Reading configuration from 'osm.toml'
2017-07-02 20:46:25.012 INFO Tile cache directory: /mnt/vdb/tmp/mvtcache
Generating tileset 'customlayer'...
Level 21: 0 / 1 [-------------------------------------------------------------------------------------------------------------------------------------------] thread 'main' panicked at 'no entry found for key', /checkout/src/libcore/option.rs:794
stack backtrace:
   0: <unknown>
   1: <unknown>
   2: <unknown>
   3: <unknown>
   4: <unknown>
   5: <unknown>
   6: <unknown>
   7: <unknown>
   8: <unknown>
   9: <unknown>
  10: main
  11: __libc_start_main
  12: <unknown>

Longer version:

root@vps:/home/usr# RUST_LOG=debug t_rex generate --config osm.toml --overwrite true --extent 4,52,5,53
2017-07-02 21:16:25.174 INFO Reading configuration from 'osm.toml'
2017-07-02 21:16:25.217 DEBUG detect_data_columns for layer customlayer with sql None
2017-07-02 21:16:25.218 DEBUG executing query: 
2017-07-02 21:16:25.218 DEBUG preparing query with name `s0`: SELECT * FROM customlayer
2017-07-02 21:16:25.219 DEBUG preparing query with name `__typeinfo`: SELECT t.typname, t.typtype, t.typelem, r.rngsubtype, t.typbasetype, n.nspname, t.typrelid FROM pg_catalog.pg_type t LEFT OUTER JOIN pg_catalog.pg_range r ON r.rngtypid = t.oid INNER JOIN pg_catalog.pg_namespace n ON t.typnamespace = n.oid WHERE t.oid = $1
2017-07-02 21:16:25.220 DEBUG executing statement __typeinfo with parameters: [16393]
2017-07-02 21:16:25.222 DEBUG Query for layer 'customlayer': SELECT wkb_geometry,"ogc_fid","image","url","city" FROM customlayer WHERE wkb_geometry && ST_MakeEnvelope($1,$2,$3,$4,3857)
2017-07-02 21:16:25.222 INFO Tile cache directory: /mnt/vdb/tmp/mvtcache
2017-07-02 21:16:25.222 DEBUG detect_data_columns for layer customlayer with sql None
2017-07-02 21:16:25.222 DEBUG executing query: 
2017-07-02 21:16:25.222 DEBUG preparing query with name `s0`: SELECT * FROM customlayer
2017-07-02 21:16:25.222 DEBUG preparing query with name `__typeinfo`: SELECT t.typname, t.typtype, t.typelem, r.rngsubtype, t.typbasetype, n.nspname, t.typrelid FROM pg_catalog.pg_type t LEFT OUTER JOIN pg_catalog.pg_range r ON r.rngtypid = t.oid INNER JOIN pg_catalog.pg_namespace n ON t.typnamespace = n.oid WHERE t.oid = $1
2017-07-02 21:16:25.223 DEBUG executing statement __typeinfo with parameters: [16393]
2017-07-02 21:16:25.225 DEBUG Filecache.write /mnt/vdb/tmp/mvtcache/customlayer.json
2017-07-02 21:16:25.225 DEBUG Filecache.write /mnt/vdb/tmp/mvtcache/customlayer.style.json
2017-07-02 21:16:25.225 DEBUG detect_data_columns for layer customlayer with sql None
2017-07-02 21:16:25.225 DEBUG executing query: 
2017-07-02 21:16:25.225 DEBUG preparing query with name `s0`: SELECT * FROM customlayer
2017-07-02 21:16:25.226 DEBUG preparing query with name `__typeinfo`: SELECT t.typname, t.typtype, t.typelem, r.rngsubtype, t.typbasetype, n.nspname, t.typrelid FROM pg_catalog.pg_type t LEFT OUTER JOIN pg_catalog.pg_range r ON r.rngtypid = t.oid INNER JOIN pg_catalog.pg_namespace n ON t.typnamespace = n.oid WHERE t.oid = $1
2017-07-02 21:16:25.226 DEBUG executing statement __typeinfo with parameters: [16393]
2017-07-02 21:16:25.228 DEBUG detect_data_columns for layer customlayer with sql None
2017-07-02 21:16:25.228 DEBUG executing query: 
2017-07-02 21:16:25.228 DEBUG preparing query with name `s0`: SELECT * FROM customlayer
2017-07-02 21:16:25.228 DEBUG preparing query with name `__typeinfo`: SELECT t.typname, t.typtype, t.typelem, r.rngsubtype, t.typbasetype, n.nspname, t.typrelid FROM pg_catalog.pg_type t LEFT OUTER JOIN pg_catalog.pg_range r ON r.rngtypid = t.oid INNER JOIN pg_catalog.pg_namespace n ON t.typnamespace = n.oid WHERE t.oid = $1
2017-07-02 21:16:25.229 DEBUG executing statement __typeinfo with parameters: [16393]
2017-07-02 21:16:25.230 DEBUG Filecache.write /mnt/vdb/tmp/mvtcache/customlayer/metadata.json
2017-07-02 21:16:25.230 DEBUG tile limits: Extent { minx: 4, miny: 52, maxx: 5, maxy: 53 }
Generating tileset 'customlayer'...
2017-07-02 21:16:25.230 DEBUG level 0: ExtentInt { minx: 0, miny: 0, maxx: 1, maxy: 1 }
Level 0: 0 / 1 [--------------------------------------------------------------------------------------------------------------------------------------------] 2017-07-02 21:16:25.230 DEBUG MVT tile request Extent { minx: -20037508.342789248, miny: -20037508.342789248, maxx: 20037508.342789248, maxy: 20037508.342789248 }
2017-07-02 21:16:25.230 DEBUG executing query: 
2017-07-02 21:16:25.230 DEBUG preparing query with name `s0`: SELECT wkb_geometry,"ogc_fid","image","url","city" FROM customlayer WHERE wkb_geometry && ST_MakeEnvelope($1,$2,$3,$4,3857)
2017-07-02 21:16:25.235 DEBUG preparing query with name `__typeinfo`: SELECT t.typname, t.typtype, t.typelem, r.rngsubtype, t.typbasetype, n.nspname, t.typrelid FROM pg_catalog.pg_type t LEFT OUTER JOIN pg_catalog.pg_range r ON r.rngtypid = t.oid INNER JOIN pg_catalog.pg_namespace n ON t.typnamespace = n.oid WHERE t.oid = $1
2017-07-02 21:16:25.235 DEBUG executing statement __typeinfo with parameters: [16393]
2017-07-02 21:16:25.237 DEBUG executing query: BEGIN
2017-07-02 21:16:25.237 DEBUG executing statement s0 with parameters: [-20037508.342789248, -20037508.342789248, 20037508.342789248, 20037508.342789248]
2017-07-02 21:16:25.390 DEBUG Reading features in layer customlayer
2017-07-02 21:21:01.856 DEBUG Feature count: 204316
2017-07-02 21:21:01.857 DEBUG executing query: ROLLBACK
2017-07-02 21:21:02.677 DEBUG Filecache.write /mnt/vdb/tmp/mvtcache/customlayer/0/0/0.pbf
Level 0: 1 / 1 [============================================================================================================================================] 2017-07-02 21:21:02.711 DEBUG level 1: ExtentInt { minx: 1, miny: 1, maxx: 2, maxy: 2 }
Level 1: 0 / 1 [--------------------------------------------------------------------------------------------------------------------------------------------] 2017-07-02 21:21:02.711 DEBUG MVT tile request Extent { minx: -0.000000003725290298461914, miny: -0.000000003725290298461914, maxx: 20037508.34278924, maxy: 20037508.34278924 }
2017-07-02 21:21:02.711 DEBUG executing query: 
2017-07-02 21:21:02.711 DEBUG preparing query with name `s0`: SELECT wkb_geometry,"ogc_fid","image","url","city" FROM customlayer WHERE wkb_geometry && ST_MakeEnvelope($1,$2,$3,$4,3857)
2017-07-02 21:21:02.713 DEBUG preparing query with name `__typeinfo`: SELECT t.typname, t.typtype, t.typelem, r.rngsubtype, t.typbasetype, n.nspname, t.typrelid FROM pg_catalog.pg_type t LEFT OUTER JOIN pg_catalog.pg_range r ON r.rngtypid = t.oid INNER JOIN pg_catalog.pg_namespace n ON t.typnamespace = n.oid WHERE t.oid = $1
2017-07-02 21:21:02.714 DEBUG executing statement __typeinfo with parameters: [16393]
2017-07-02 21:21:02.715 DEBUG executing query: BEGIN
2017-07-02 21:21:02.715 DEBUG executing statement s0 with parameters: [-0.000000003725290298461914, -0.000000003725290298461914, 20037508.34278924, 20037508.34278924]
2017-07-02 21:21:02.747 DEBUG Reading features in layer customlayer
2017-07-02 21:21:48.867 DEBUG Feature count: 98194
2017-07-02 21:21:48.867 DEBUG executing query: ROLLBACK
2017-07-02 21:21:49.239 DEBUG Filecache.write /mnt/vdb/tmp/mvtcache/customlayer/1/1/0.pbf
Level 1: 1 / 1 [============================================================================================================================================] 2017-07-02 21:21:49.252 DEBUG level 2: ExtentInt { minx: 2, miny: 2, maxx: 3, maxy: 3 }
Level 2: 0 / 1 [--------------------------------------------------------------------------------------------------------------------------------------------] 2017-07-02 21:21:49.252 DEBUG MVT tile request Extent { minx: -0.000000011175870895385742, miny: -0.000000011175870895385742, maxx: 10018754.171394609, maxy: 10018754.171394609 }
2017-07-02 21:21:49.253 DEBUG executing query: 
2017-07-02 21:21:49.253 DEBUG preparing query with name `s0`: SELECT wkb_geometry,"ogc_fid","image","url","city" FROM customlayer WHERE wkb_geometry && ST_MakeEnvelope($1,$2,$3,$4,3857)
2017-07-02 21:21:49.254 DEBUG preparing query with name `__typeinfo`: SELECT t.typname, t.typtype, t.typelem, r.rngsubtype, t.typbasetype, n.nspname, t.typrelid FROM pg_catalog.pg_type t LEFT OUTER JOIN pg_catalog.pg_range r ON r.rngtypid = t.oid INNER JOIN pg_catalog.pg_namespace n ON t.typnamespace = n.oid WHERE t.oid = $1
2017-07-02 21:21:49.255 DEBUG executing statement __typeinfo with parameters: [16393]
2017-07-02 21:21:49.256 DEBUG executing query: BEGIN
2017-07-02 21:21:49.256 DEBUG executing statement s0 with parameters: [-0.000000011175870895385742, -0.000000011175870895385742, 10018754.171394609, 10018754.171394609]
2017-07-02 21:21:49.281 DEBUG Reading features in layer customlayer
2017-07-02 21:22:37.580 DEBUG Feature count: 98194
2017-07-02 21:22:37.580 DEBUG executing query: ROLLBACK
2017-07-02 21:22:37.964 DEBUG Filecache.write /mnt/vdb/tmp/mvtcache/customlayer/2/2/1.pbf
Level 2: 1 / 1 [============================================================================================================================================] 2017-07-02 21:22:37.981 DEBUG level 3: ExtentInt { minx: 4, miny: 4, maxx: 5, maxy: 5 }
Level 3: 0 / 1 [--------------------------------------------------------------------------------------------------------------------------------------------] 2017-07-02 21:22:37.981 DEBUG MVT tile request Extent { minx: -0.000000003725290298461914, miny: -0.000000003725290298461914, maxx: 5009377.085697308, maxy: 5009377.085697308 }
2017-07-02 21:22:37.981 DEBUG executing query: 
2017-07-02 21:22:37.981 DEBUG preparing query with name `s0`: SELECT wkb_geometry,"ogc_fid","image","url","city" FROM customlayer WHERE wkb_geometry && ST_MakeEnvelope($1,$2,$3,$4,3857)
2017-07-02 21:22:37.983 DEBUG preparing query with name `__typeinfo`: SELECT t.typname, t.typtype, t.typelem, r.rngsubtype, t.typbasetype, n.nspname, t.typrelid FROM pg_catalog.pg_type t LEFT OUTER JOIN pg_catalog.pg_range r ON r.rngtypid = t.oid INNER JOIN pg_catalog.pg_namespace n ON t.typnamespace = n.oid WHERE t.oid = $1
2017-07-02 21:22:37.983 DEBUG executing statement __typeinfo with parameters: [16393]
2017-07-02 21:22:37.985 DEBUG executing query: BEGIN
2017-07-02 21:22:37.985 DEBUG executing statement s0 with parameters: [-0.000000003725290298461914, -0.000000003725290298461914, 5009377.085697308, 5009377.085697308]
2017-07-02 21:22:38.017 DEBUG Reading features in layer customlayer
2017-07-02 21:22:38.068 DEBUG Feature count: 2742
2017-07-02 21:22:38.068 DEBUG executing query: ROLLBACK
2017-07-02 21:22:38.075 DEBUG Filecache.write /mnt/vdb/tmp/mvtcache/customlayer/3/4/3.pbf
Level 3: 1 / 1 [============================================================================================================================================] 2017-07-02 21:22:38.075 DEBUG level 4: ExtentInt { minx: 8, miny: 8, maxx: 9, maxy: 9 }
Level 4: 0 / 1 [--------------------------------------------------------------------------------------------------------------------------------------------] 2017-07-02 21:22:38.076 DEBUG MVT tile request Extent { minx: -0.000000003725290298461914, miny: -0.000000003725290298461914, maxx: 2504688.5428486504, maxy: 2504688.5428486504 }
2017-07-02 21:22:38.076 DEBUG executing query: 
2017-07-02 21:22:38.076 DEBUG preparing query with name `s0`: SELECT wkb_geometry,"ogc_fid","image","url","city" FROM customlayer WHERE wkb_geometry && ST_MakeEnvelope($1,$2,$3,$4,3857)
2017-07-02 21:22:38.077 DEBUG preparing query with name `__typeinfo`: SELECT t.typname, t.typtype, t.typelem, r.rngsubtype, t.typbasetype, n.nspname, t.typrelid FROM pg_catalog.pg_type t LEFT OUTER JOIN pg_catalog.pg_range r ON r.rngtypid = t.oid INNER JOIN pg_catalog.pg_namespace n ON t.typnamespace = n.oid WHERE t.oid = $1
2017-07-02 21:22:38.078 DEBUG executing statement __typeinfo with parameters: [16393]
2017-07-02 21:22:38.079 DEBUG executing query: BEGIN
2017-07-02 21:22:38.079 DEBUG executing statement s0 with parameters: [-0.000000003725290298461914, -0.000000003725290298461914, 2504688.5428486504, 2504688.5428486504]
2017-07-02 21:22:38.101 DEBUG Reading features in layer customlayer
2017-07-02 21:22:38.156 DEBUG Feature count: 2742
2017-07-02 21:22:38.157 DEBUG executing query: ROLLBACK
2017-07-02 21:22:38.163 DEBUG Filecache.write /mnt/vdb/tmp/mvtcache/customlayer/4/8/7.pbf
Level 4: 1 / 1 [============================================================================================================================================] 2017-07-02 21:22:38.164 DEBUG level 5: ExtentInt { minx: 16, miny: 16, maxx: 17, maxy: 17 }
Level 5: 0 / 1 [--------------------------------------------------------------------------------------------------------------------------------------------] 2017-07-02 21:22:38.164 DEBUG MVT tile request Extent { minx: -0.000000003725290298461914, miny: -0.000000003725290298461914, maxx: 1252344.2714243233, maxy: 1252344.2714243233 }
2017-07-02 21:22:38.164 DEBUG executing query: 
2017-07-02 21:22:38.164 DEBUG preparing query with name `s0`: SELECT wkb_geometry,"ogc_fid","image","url","city" FROM customlayer WHERE wkb_geometry && ST_MakeEnvelope($1,$2,$3,$4,3857)
2017-07-02 21:22:38.165 DEBUG preparing query with name `__typeinfo`: SELECT t.typname, t.typtype, t.typelem, r.rngsubtype, t.typbasetype, n.nspname, t.typrelid FROM pg_catalog.pg_type t LEFT OUTER JOIN pg_catalog.pg_range r ON r.rngtypid = t.oid INNER JOIN pg_catalog.pg_namespace n ON t.typnamespace = n.oid WHERE t.oid = $1
2017-07-02 21:22:38.166 DEBUG executing statement __typeinfo with parameters: [16393]
2017-07-02 21:22:38.167 DEBUG executing query: BEGIN
2017-07-02 21:22:38.167 DEBUG executing statement s0 with parameters: [-0.000000003725290298461914, -0.000000003725290298461914, 1252344.2714243233, 1252344.2714243233]
2017-07-02 21:22:38.192 DEBUG Reading features in layer customlayer
2017-07-02 21:22:38.259 DEBUG Feature count: 2742
2017-07-02 21:22:38.259 DEBUG executing query: ROLLBACK
2017-07-02 21:22:38.267 DEBUG Filecache.write /mnt/vdb/tmp/mvtcache/customlayer/5/16/15.pbf
Level 5: 1 / 1 [============================================================================================================================================] 2017-07-02 21:22:38.268 DEBUG level 6: ExtentInt { minx: 32, miny: 32, maxx: 33, maxy: 33 }
Level 6: 0 / 1 [--------------------------------------------------------------------------------------------------------------------------------------------] 2017-07-02 21:22:38.268 DEBUG MVT tile request Extent { minx: -0.000000003725290298461914, miny: -0.000000003725290298461914, maxx: 626172.1357121617, maxy: 626172.1357121617 }
2017-07-02 21:22:38.268 DEBUG executing query: 
2017-07-02 21:22:38.269 DEBUG preparing query with name `s1`: SELECT wkb_geometry,"ogc_fid","image","url","city" FROM customlayer WHERE wkb_geometry && ST_MakeEnvelope($1,$2,$3,$4,3857)
2017-07-02 21:22:38.269 DEBUG executing query: BEGIN
2017-07-02 21:22:38.269 DEBUG executing statement s1 with parameters: [-0.000000003725290298461914, -0.000000003725290298461914, 626172.1357121617, 626172.1357121617]
2017-07-02 21:22:38.300 DEBUG Reading features in layer customlayer
2017-07-02 21:22:38.369 DEBUG Feature count: 2742
2017-07-02 21:22:38.369 DEBUG executing query: ROLLBACK
2017-07-02 21:22:38.378 DEBUG Filecache.write /mnt/vdb/tmp/mvtcache/customlayer/6/32/31.pbf
Level 6: 1 / 1 [============================================================================================================================================] 2017-07-02 21:22:38.379 DEBUG level 7: ExtentInt { minx: 64, miny: 64, maxx: 65, maxy: 65 }
Level 7: 0 / 1 [--------------------------------------------------------------------------------------------------------------------------------------------] 2017-07-02 21:22:38.379 DEBUG MVT tile request Extent { minx: -0.000000003725290298461914, miny: -0.000000003725290298461914, maxx: 313086.0678560771, maxy: 313086.0678560771 }
2017-07-02 21:22:38.379 DEBUG executing query: 
2017-07-02 21:22:38.380 DEBUG preparing query with name `s1`: SELECT wkb_geometry,"ogc_fid","image","url","city" FROM customlayer WHERE wkb_geometry && ST_MakeEnvelope($1,$2,$3,$4,3857)
2017-07-02 21:22:38.381 DEBUG executing query: BEGIN
2017-07-02 21:22:38.381 DEBUG executing statement s1 with parameters: [-0.000000003725290298461914, -0.000000003725290298461914, 313086.0678560771, 313086.0678560771]
2017-07-02 21:22:38.410 DEBUG Reading features in layer customlayer
2017-07-02 21:22:38.491 DEBUG Feature count: 2742
2017-07-02 21:22:38.491 DEBUG executing query: ROLLBACK
2017-07-02 21:22:38.498 DEBUG Filecache.write /mnt/vdb/tmp/mvtcache/customlayer/7/64/63.pbf
Level 7: 1 / 1 [============================================================================================================================================] 2017-07-02 21:22:38.500 DEBUG level 8: ExtentInt { minx: 128, miny: 128, maxx: 129, maxy: 129 }
Level 8: 0 / 1 [--------------------------------------------------------------------------------------------------------------------------------------------] 2017-07-02 21:22:38.500 DEBUG MVT tile request Extent { minx: -0.000000003725290298461914, miny: -0.000000003725290298461914, maxx: 156543.0339280367, maxy: 156543.0339280367 }
2017-07-02 21:22:38.500 DEBUG executing query: 
2017-07-02 21:22:38.500 DEBUG preparing query with name `s1`: SELECT wkb_geometry,"ogc_fid","image","url","city" FROM customlayer WHERE wkb_geometry && ST_MakeEnvelope($1,$2,$3,$4,3857)
2017-07-02 21:22:38.501 DEBUG executing query: BEGIN
2017-07-02 21:22:38.501 DEBUG executing statement s1 with parameters: [-0.000000003725290298461914, -0.000000003725290298461914, 156543.0339280367, 156543.0339280367]
2017-07-02 21:22:38.524 DEBUG Reading features in layer customlayer
2017-07-02 21:22:38.573 DEBUG Feature count: 2742
2017-07-02 21:22:38.573 DEBUG executing query: ROLLBACK
2017-07-02 21:22:38.580 DEBUG Filecache.write /mnt/vdb/tmp/mvtcache/customlayer/8/128/127.pbf
Level 8: 1 / 1 [============================================================================================================================================] 2017-07-02 21:22:38.580 DEBUG level 9: ExtentInt { minx: 256, miny: 256, maxx: 257, maxy: 257 }
Level 9: 0 / 1 [--------------------------------------------------------------------------------------------------------------------------------------------] 2017-07-02 21:22:38.581 DEBUG MVT tile request Extent { minx: -0.00000001862645149230957, miny: -0.00000001862645149230957, maxx: 78271.51696400344, maxy: 78271.51696400344 }
2017-07-02 21:22:38.581 DEBUG executing query: 
2017-07-02 21:22:38.581 DEBUG preparing query with name `s1`: SELECT wkb_geometry,"ogc_fid","image","url","city" FROM customlayer WHERE wkb_geometry && ST_MakeEnvelope($1,$2,$3,$4,3857)
2017-07-02 21:22:38.582 DEBUG executing query: BEGIN
2017-07-02 21:22:38.582 DEBUG executing statement s1 with parameters: [-0.00000001862645149230957, -0.00000001862645149230957, 78271.51696400344, 78271.51696400344]
2017-07-02 21:22:38.607 DEBUG Reading features in layer customlayer
2017-07-02 21:22:38.685 DEBUG Feature count: 2742
2017-07-02 21:22:38.686 DEBUG executing query: ROLLBACK
2017-07-02 21:22:38.692 DEBUG Filecache.write /mnt/vdb/tmp/mvtcache/customlayer/9/256/255.pbf
Level 9: 1 / 1 [============================================================================================================================================] 2017-07-02 21:22:38.696 DEBUG level 10: ExtentInt { minx: 512, miny: 512, maxx: 513, maxy: 513 }
Level 10: 0 / 1 [-------------------------------------------------------------------------------------------------------------------------------------------] 2017-07-02 21:22:38.696 DEBUG MVT tile request Extent { minx: -0.000000003725290298461914, miny: -0.000000003725290298461914, maxx: 39135.75848200545, maxy: 39135.75848200545 }
2017-07-02 21:22:38.696 DEBUG executing query: 
2017-07-02 21:22:38.696 DEBUG executing query: BEGIN
2017-07-02 21:22:38.696 DEBUG executing statement s0 with parameters: [-0.000000003725290298461914, -0.000000003725290298461914, 39135.75848200545, 39135.75848200545]
2017-07-02 21:22:38.698 DEBUG Reading features in layer customlayer
2017-07-02 21:22:38.751 DEBUG Feature count: 2742
2017-07-02 21:22:38.751 DEBUG executing query: ROLLBACK
2017-07-02 21:22:38.758 DEBUG Filecache.write /mnt/vdb/tmp/mvtcache/customlayer/10/512/511.pbf
Level 10: 1 / 1 [===========================================================================================================================================] 2017-07-02 21:22:38.762 DEBUG level 11: ExtentInt { minx: 1024, miny: 1024, maxx: 1025, maxy: 1025 }
Level 11: 0 / 1 [-------------------------------------------------------------------------------------------------------------------------------------------] 2017-07-02 21:22:38.762 DEBUG MVT tile request Extent { minx: -0.000000007450580596923828, miny: -0.000000007450580596923828, maxx: 19567.879240997136, maxy: 19567.879240997136 }
2017-07-02 21:22:38.762 DEBUG executing query: 
2017-07-02 21:22:38.763 DEBUG executing query: BEGIN
2017-07-02 21:22:38.763 DEBUG executing statement s0 with parameters: [-0.000000007450580596923828, -0.000000007450580596923828, 19567.879240997136, 19567.879240997136]
2017-07-02 21:22:38.764 DEBUG Reading features in layer customlayer
2017-07-02 21:22:38.843 DEBUG Feature count: 2742
2017-07-02 21:22:38.843 DEBUG executing query: ROLLBACK
2017-07-02 21:22:38.852 DEBUG Filecache.write /mnt/vdb/tmp/mvtcache/customlayer/11/1024/1023.pbf
Level 11: 1 / 1 [===========================================================================================================================================] 2017-07-02 21:22:38.864 DEBUG level 12: ExtentInt { minx: 2048, miny: 2048, maxx: 2049, maxy: 2049 }
Level 12: 0 / 1 [-------------------------------------------------------------------------------------------------------------------------------------------] 2017-07-02 21:22:38.864 DEBUG MVT tile request Extent { minx: -0.000000003725290298461914, miny: -0.000000003725290298461914, maxx: 9783.939620498568, maxy: 9783.939620498568 }
2017-07-02 21:22:38.864 DEBUG executing query: 
2017-07-02 21:22:38.864 DEBUG executing query: BEGIN
2017-07-02 21:22:38.864 DEBUG executing statement s0 with parameters: [-0.000000003725290298461914, -0.000000003725290298461914, 9783.939620498568, 9783.939620498568]
2017-07-02 21:22:38.866 DEBUG Reading features in layer customlayer
2017-07-02 21:22:38.940 DEBUG Feature count: 2742
2017-07-02 21:22:38.941 DEBUG executing query: ROLLBACK
2017-07-02 21:22:38.947 DEBUG Filecache.write /mnt/vdb/tmp/mvtcache/customlayer/12/2048/2047.pbf
Level 12: 1 / 1 [===========================================================================================================================================] 2017-07-02 21:22:38.948 DEBUG level 13: ExtentInt { minx: 4096, miny: 4096, maxx: 4097, maxy: 4097 }
Level 13: 0 / 1 [-------------------------------------------------------------------------------------------------------------------------------------------] 2017-07-02 21:22:38.948 DEBUG MVT tile request Extent { minx: -0.000000007450580596923828, miny: -0.000000007450580596923828, maxx: 4891.969810243696, maxy: 4891.969810243696 }
2017-07-02 21:22:38.948 DEBUG executing query: 
2017-07-02 21:22:38.948 DEBUG executing query: BEGIN
2017-07-02 21:22:38.948 DEBUG executing statement s0 with parameters: [-0.000000007450580596923828, -0.000000007450580596923828, 4891.969810243696, 4891.969810243696]
2017-07-02 21:22:38.949 DEBUG Reading features in layer customlayer
2017-07-02 21:22:39.000 DEBUG Feature count: 2742
2017-07-02 21:22:39.001 DEBUG executing query: ROLLBACK
2017-07-02 21:22:39.008 DEBUG Filecache.write /mnt/vdb/tmp/mvtcache/customlayer/13/4096/4095.pbf
Level 13: 1 / 1 [===========================================================================================================================================] 2017-07-02 21:22:39.009 DEBUG level 14: ExtentInt { minx: 8192, miny: 8192, maxx: 8193, maxy: 8193 }
Level 14: 0 / 1 [-------------------------------------------------------------------------------------------------------------------------------------------] 2017-07-02 21:22:39.009 DEBUG MVT tile request Extent { minx: -0.000000003725290298461914, miny: -0.000000003725290298461914, maxx: 2445.9849051237106, maxy: 2445.9849051237106 }
2017-07-02 21:22:39.009 DEBUG executing query: 
2017-07-02 21:22:39.009 DEBUG executing query: BEGIN
2017-07-02 21:22:39.009 DEBUG executing statement s0 with parameters: [-0.000000003725290298461914, -0.000000003725290298461914, 2445.9849051237106, 2445.9849051237106]
2017-07-02 21:22:39.010 DEBUG Reading features in layer customlayer
2017-07-02 21:22:39.065 DEBUG Feature count: 2742
2017-07-02 21:22:39.065 DEBUG executing query: ROLLBACK
2017-07-02 21:22:39.073 DEBUG Filecache.write /mnt/vdb/tmp/mvtcache/customlayer/14/8192/8191.pbf
Level 14: 1 / 1 [===========================================================================================================================================] 2017-07-02 21:22:39.073 DEBUG level 15: ExtentInt { minx: 16384, miny: 16384, maxx: 16385, maxy: 16385 }
Level 15: 0 / 1 [-------------------------------------------------------------------------------------------------------------------------------------------] 2017-07-02 21:22:39.073 DEBUG MVT tile request Extent { minx: -0.000000003725290298461914, miny: -0.000000003725290298461914, maxx: 1222.99245255813, maxy: 1222.99245255813 }
2017-07-02 21:22:39.073 DEBUG executing query: 
2017-07-02 21:22:39.074 DEBUG executing query: BEGIN
2017-07-02 21:22:39.074 DEBUG executing statement s0 with parameters: [-0.000000003725290298461914, -0.000000003725290298461914, 1222.99245255813, 1222.99245255813]
2017-07-02 21:22:39.074 DEBUG Reading features in layer customlayer
2017-07-02 21:22:39.131 DEBUG Feature count: 2742
2017-07-02 21:22:39.131 DEBUG executing query: ROLLBACK
2017-07-02 21:22:39.141 DEBUG Filecache.write /mnt/vdb/tmp/mvtcache/customlayer/15/16384/16383.pbf
Level 15: 1 / 1 [===========================================================================================================================================] 2017-07-02 21:22:39.141 DEBUG level 16: ExtentInt { minx: 32768, miny: 32768, maxx: 32769, maxy: 32769 }
Level 16: 0 / 1 [-------------------------------------------------------------------------------------------------------------------------------------------] 2017-07-02 21:22:39.142 DEBUG MVT tile request Extent { minx: -0.000000003725290298461914, miny: -0.000000003725290298461914, maxx: 611.4962262772024, maxy: 611.4962262772024 }
2017-07-02 21:22:39.142 DEBUG executing query: 
2017-07-02 21:22:39.142 DEBUG executing query: BEGIN
2017-07-02 21:22:39.142 DEBUG executing statement s1 with parameters: [-0.000000003725290298461914, -0.000000003725290298461914, 611.4962262772024, 611.4962262772024]
2017-07-02 21:22:39.143 DEBUG Reading features in layer customlayer
2017-07-02 21:22:39.198 DEBUG Feature count: 2742
2017-07-02 21:22:39.198 DEBUG executing query: ROLLBACK
2017-07-02 21:22:39.205 DEBUG Filecache.write /mnt/vdb/tmp/mvtcache/customlayer/16/32768/32767.pbf
Level 16: 1 / 1 [===========================================================================================================================================] 2017-07-02 21:22:39.206 DEBUG level 17: ExtentInt { minx: 65536, miny: 65536, maxx: 65537, maxy: 65537 }
Level 17: 0 / 1 [-------------------------------------------------------------------------------------------------------------------------------------------] 2017-07-02 21:22:39.206 DEBUG MVT tile request Extent { minx: -0.000000003725290298461914, miny: -0.000000003725290298461914, maxx: 305.74811313673854, maxy: 305.74811313673854 }
2017-07-02 21:22:39.206 DEBUG executing query: 
2017-07-02 21:22:39.206 DEBUG executing query: BEGIN
2017-07-02 21:22:39.206 DEBUG executing statement s1 with parameters: [-0.000000003725290298461914, -0.000000003725290298461914, 305.74811313673854, 305.74811313673854]
2017-07-02 21:22:39.207 DEBUG Reading features in layer customlayer
2017-07-02 21:22:39.256 DEBUG Feature count: 2742
2017-07-02 21:22:39.256 DEBUG executing query: ROLLBACK
2017-07-02 21:22:39.264 DEBUG Filecache.write /mnt/vdb/tmp/mvtcache/customlayer/17/65536/65535.pbf
Level 17: 1 / 1 [===========================================================================================================================================] 2017-07-02 21:22:39.264 DEBUG level 18: ExtentInt { minx: 131072, miny: 131072, maxx: 131073, maxy: 131073 }
Level 18: 0 / 1 [-------------------------------------------------------------------------------------------------------------------------------------------] 2017-07-02 21:22:39.264 DEBUG MVT tile request Extent { minx: -0.000000003725290298461914, miny: -0.000000003725290298461914, maxx: 152.87405656650662, maxy: 152.87405656650662 }
2017-07-02 21:22:39.264 DEBUG executing query: 
2017-07-02 21:22:39.264 DEBUG executing query: BEGIN
2017-07-02 21:22:39.265 DEBUG executing statement s1 with parameters: [-0.000000003725290298461914, -0.000000003725290298461914, 152.87405656650662, 152.87405656650662]
2017-07-02 21:22:39.265 DEBUG Reading features in layer customlayer
2017-07-02 21:22:39.320 DEBUG Feature count: 2742
2017-07-02 21:22:39.320 DEBUG executing query: ROLLBACK
2017-07-02 21:22:39.329 DEBUG Filecache.write /mnt/vdb/tmp/mvtcache/customlayer/18/131072/131071.pbf
Level 18: 1 / 1 [===========================================================================================================================================] 2017-07-02 21:22:39.330 DEBUG level 19: ExtentInt { minx: 262144, miny: 262144, maxx: 262145, maxy: 262145 }
Level 19: 0 / 1 [-------------------------------------------------------------------------------------------------------------------------------------------] 2017-07-02 21:22:39.330 DEBUG MVT tile request Extent { minx: 0.000000014901161193847656, miny: 0.000000014901161193847656, maxx: 76.43702830001712, maxy: 76.43702830001712 }
2017-07-02 21:22:39.330 DEBUG executing query: 
2017-07-02 21:22:39.330 DEBUG executing query: BEGIN
2017-07-02 21:22:39.330 DEBUG executing statement s1 with parameters: [0.000000014901161193847656, 0.000000014901161193847656, 76.43702830001712, 76.43702830001712]
2017-07-02 21:22:39.331 DEBUG Reading features in layer customlayer
2017-07-02 21:22:39.384 DEBUG Feature count: 2742
2017-07-02 21:22:39.384 DEBUG executing query: ROLLBACK
2017-07-02 21:22:39.392 DEBUG Filecache.write /mnt/vdb/tmp/mvtcache/customlayer/19/262144/262143.pbf
Level 19: 1 / 1 [===========================================================================================================================================] 2017-07-02 21:22:39.392 DEBUG level 20: ExtentInt { minx: 524288, miny: 524289, maxx: 524289, maxy: 524290 }
Level 20: 0 / 1 [-------------------------------------------------------------------------------------------------------------------------------------------] 2017-07-02 21:22:39.392 DEBUG MVT tile request Extent { minx: 0.000000014901161193847656, miny: 38.218514159321785, maxx: 38.218514159321785, maxy: 76.43702830001712 }
2017-07-02 21:22:39.392 DEBUG executing query: 
2017-07-02 21:22:39.393 DEBUG executing query: BEGIN
2017-07-02 21:22:39.393 DEBUG executing statement s0 with parameters: [0.000000014901161193847656, 38.218514159321785, 38.218514159321785, 76.43702830001712]
2017-07-02 21:22:39.393 DEBUG Reading features in layer customlayer
2017-07-02 21:22:39.449 DEBUG Feature count: 2742
2017-07-02 21:22:39.449 DEBUG executing query: ROLLBACK
2017-07-02 21:22:39.460 DEBUG Filecache.write /mnt/vdb/tmp/mvtcache/customlayer/20/524288/524286.pbf
Level 20: 1 / 1 [===========================================================================================================================================] 2017-07-02 21:22:39.460 DEBUG level 21: ExtentInt { minx: 1048576, miny: 1048578, maxx: 1048577, maxy: 1048579 }
Level 21: 0 / 1 [-------------------------------------------------------------------------------------------------------------------------------------------] 2017-07-02 21:22:39.460 DEBUG MVT tile request Extent { minx: -0.000000014901161193847656, miny: 38.21851412951946, maxx: 19.10925705730915, maxy: 57.327771198004484 }
2017-07-02 21:22:39.460 DEBUG executing query: 
thread 'main' panicked at 'no entry found for key', /checkout/src/libcore/option.rs:794
note: Run with `RUST_BACKTRACE=1` for a backtrace.

Proper config for # of threads

What is the 'rule of thumb' here? Does this have to be similar to your # of cores?

Fix invalid polygons after simplification, coord. transform or clipping

Some vector tile creation steps can produce invalid polygons:

Simplification
Coordinate transform
Clipping (tile buffer)

Error converting between Postgres and Rust types

Hi, this looks like a promising project. I am having nightmares trying to get node-mapnik output vector tiles properly, so when I saw your project I saw a glimpse of hope...

I have not managed to make it work, though (I am a complete amateur by the way), version pre7 fails with hundreds of lines of :

2016-08-15 00:03:44.847 ERROR Layer 'city_object': Error converting between Postgres and Rust types: cannot convert geometry to Geometry
2016-08-15 00:03:44.850 ERROR Row { statement: Statement { name: "s0", parameter_types: [Float8, Float8, Float8, Float8], columns: [Column { name: "geometry", type_: Other(Other { name: "geometry", oid: 116463, kind: Simple, schema: "public" }) }] } }

I tried to limit shapes to POLYGON, as referenced in #3, but it did not work.

I am intringued by the way as to why resort to rust to output vector tiles, where there are already existing tools (native mapnik C++, node bindings for mapnik, and python bindings at Mapzen) ? What are the benefits of going with rust (including serving tiles, seems far-fetched).

Error converting between Postgres and Rust types when using !zoom! token

2017-07-13 05:28:50.514 ERROR Query: SELECT ST_Multi(ST_SimplifyPreserveTopology(ST_Multi(way),$6::FLOAT8/2)) AS way,"name","ref","class","brunnel" FROM (  SELECT
      way,
      name,
      ref,
      highway AS class,
      CASE
        WHEN bridge IS NOT NULL AND bridge <> 'no' THEN 'bridge'
        ELSE CASE WHEN tunnel IS NOT NULL and tunnel <> 'no' THEN 'tunnel' END
      END AS brunnel
    FROM planet_osm_roads
    WHERE way && ST_MakeEnvelope($1,$2,$3,$4,3857)
      AND (highway = 'motorway'
        OR highway = 'trunk' AND $5 >= 7
        OR highway = 'primary' AND $5 >= 8
        OR highway = 'secondary' AND $5 >= 9)
  ) AS _q
2017-07-13 05:28:50.514 ERROR Param types: [Bbox, Zoom, PixelWidth]
2017-07-13 05:28:50.514 ERROR Param values: [-20037508.342789248, -20037508.342789248, 20037508.342789248, 20037508.342789248, 0, 156543.033928041]

The layer in question is

[[tileset.layer]]
name = "transport"
geometry_field = "way"
geometry_type = "LINESTRING"
srid = 3857
buffer-size = 128
simplify = true
  [[tileset.layer.query]]
  minzoom = 0
  maxzoom = 10
  sql = """
  SELECT
      way,
      name,
      ref,
      highway AS class,
      CASE
        WHEN bridge IS NOT NULL AND bridge <> 'no' THEN 'bridge'
        ELSE CASE WHEN tunnel IS NOT NULL and tunnel <> 'no' THEN 'tunnel' END
      END AS brunnel
    FROM planet_osm_roads
    WHERE way && !bbox!
      AND (highway = 'motorway'
        OR highway = 'trunk' AND !zoom! >= 7
        OR highway = 'primary' AND !zoom! >= 8
        OR highway = 'secondary' AND !zoom! >= 9)
  """

If I remove the !zoom! tokens, it works.

how to visually check pbf response?

I have a customlayer in postgres with 50000 points. I used to work with a server that could return mvt and json. When I visually wanted to check the response I simply used json and could see the curl result in the terminal. I now have migrated to t-rex and everything looks fine and dandy except that my points are not shown on the map... I am running t-rex on a remote server and trying to browse my customlayer with mapbox-gl-js on my local pc:

when I drag the map I can see t-rex giving responses for tiles that have not been queried before etc.
when on my local pc I do:
curl -s http://<remoteIPaddress>:8080/customlayer/4/2/6.pbf
It returns

So garbled output, but could be correct output.
There are no errormesages in the browser console, except when I zoom >22, which makes sense.

So how to proceed? How can I (visually) verify whether my points are actually in the pbf tiles, so I can check what is wrong?

My config.toml:

# t-rex configuration

[service.mvt]
viewer = true

[datasource]
type = "postgis"
url = "postgresql://usr:pw@localhost/dbname"

[grid]
# Predefined grids: web_mercator, wgs84
predefined = "web_mercator"

[[tileset]]
name = "customlayer"

[[tileset.layer]]
name = "customlayer"
table_name = "customlayer"
geometry_field = "wkb_geometry"
geometry_type = "POINT"
srid = 4326
#fid_field = "id"
#buffer_size = 10
query_limit = 100000
#[[tileset.layer.query]]
#sql = """SELECT ST_Transform(wkb_geometry,3857) AS wkb_geometry,"ogc_fid","image","city","url" FROM customlayer WHERE wkb_geometry && !bbox!"""

[cache.file]
base = "/tmp/mvtcache"

[webserver]
# Bind address. Use 0.0.0.0 to listen on all adresses.
bind = "<remoteIPaddress>"
port = 8080
threads = 4

Error with loading template

I have problems with the launch of t-rex server on Ubuntu 14.04 and Windows 7.
I import one polygonal layer in PostGIS database (EPSG:3857, singlepart features).
When I run command

~/t_rex serve --dbconn postgresql://postgres:postgres@localhost:5432/test (on Ubuntu)
t_rex.exe serve --dbconn postgresql://postgres:postgres@localhost:5432/test (on Windows)

and open URL http://127.0.0.1:6767/ in browser (or using command curl http://127.0.0.1:6767/) I receive error:

2016-06-22 12:37:03.337 INFO GET /
2016-06-22 12:37:03.337 WARN Get V4(127.0.0.1:50338) AbsolutePath("/") "Failed to compile template 'src/webserver/templates/index.tpl': Error { repr: Os { code: 2, message: "No such file or directory" } }" Some(InternalServerError)

Also I try run t-rex server with configuration file and get some error - I try change 'bind' and 'port' options in configuration file (When I change 'bind' and 'port' options in configuration file and I run t-rex server in console listening server remains unchanged 'Listening on http://127.0.0.1:6767').
What could be the problem?

thread '<unnamed>' panicked at 'no entry found for key'

I have this layer definition

[[tileset.layer]]
name = "transport"
geometry_field = "way"
geometry_type = "LINESTRING"
srid = 3857
buffer-size = 128
simplify = true
  [[tileset.layer.query]]
  minzoom = 6
  maxzoom = 10
  sql = """
  SELECT
      way,
      name,
      ref,
      highway AS class,
      CASE
        WHEN bridge IS NOT NULL AND bridge <> 'no' THEN 'bridge'
        ELSE CASE WHEN tunnel IS NOT NULL and tunnel <> 'no' THEN 'tunnel' END
      END AS brunnel
    FROM planet_osm_roads
    WHERE way && !bbox!
      AND (highway = 'motorway'
        OR highway = 'trunk' AND !zoom! >= 7
        OR highway = 'primary' AND !zoom! >= 8
        OR highway = 'secondary' AND !zoom! >= 9)
  """

and get this error when I try the X-Ray view

2017-07-13 05:16:51.608 INFO GET /vector_demo/1/0/1.pbf
thread '<unnamed>' panicked at 'no entry found for key', /checkout/src/libcore/option.rs:794
note: Run with `RUST_BACKTRACE=1` for a backtrace.

Turning on the backtrace gives

thread '<unnamed>' panicked at 'no entry found for key', /checkout/src/libcore/option.rs:794
stack backtrace:
   0: std::sys::imp::backtrace::tracing::imp::unwind_backtrace
             at /checkout/src/libstd/sys/unix/backtrace/tracing/gcc_s.rs:49
   1: std::panicking::default_hook::{{closure}}
             at /checkout/src/libstd/sys_common/backtrace.rs:71
             at /checkout/src/libstd/sys_common/backtrace.rs:60
             at /checkout/src/libstd/panicking.rs:355
   2: std::panicking::rust_panic_with_hook
             at /checkout/src/libstd/panicking.rs:371
             at /checkout/src/libstd/panicking.rs:549
   3: std::panicking::begin_panic
             at /checkout/src/libstd/panicking.rs:511
   4: std::panicking::begin_panic_fmt
             at /checkout/src/libstd/panicking.rs:495
   5: core::panicking::panic_fmt
             at /checkout/src/libstd/panicking.rs:471
   6: core::option::expect_failed
             at /checkout/src/libcore/option.rs:794
   7: <t_rex::datasource::postgis::PostgisInput as t_rex::datasource::datasource::DatasourceInput>::retrieve_features
   8: t_rex::service::mvt::MvtService::tile
   9: <T as nickel::middleware::Middleware<D>>::invoke
  10: <nickel::router::router::Router<D> as nickel::middleware::Middleware<D>>::invoke
  11: <nickel::server::ArcServer<D> as hyper::server::Handler>::handle
  12: hyper::server::listener::spawn_with::{{closure}}
  13: <F as alloc::boxed::FnBox<A>>::call_box
  14: std::sys::imp::thread::Thread::new::thread_start
             at /checkout/src/liballoc/boxed.rs:650
             at /checkout/src/libstd/sys_common/thread.rs:21
             at /checkout/src/libstd/sys/unix/thread.rs:84
  15: start_thread
  16: __clone

Empty tiles at zoomlevel 16 or greater

When I request tiles at zoomlevel 16, the coordinates of the Y-ax are way off. Below are 2 examples at level 15 and 16:

Zoomlevel 15: /15/16973/10716.pbf

DEBUG executing statement s5 with parameters: [720342.554559499, 6930698.228673499, 721565.5470120609, 6931921.221126061, 4.777314267823516, 18055.990933506204]

Zoomlevel 16: 16/33948/21433.pbf

DEBUG executing statement s4 with parameters: [721565.5470120609, -20037508.342789248, 722177.0432383418, -20036896.846562967, 2.388657133911758, 9027.995466753102]

pbf vs mvt

I used to have mvt tiles but now see pbf is returned. I think they are pretty similar?! Could you maybe comment why pbf is supported and mvt not?

minzoom/maxzoom by layer

How can set minzoom or maxzoom by layer ?
I see that you can customize sql query by zoom but I don't find a way to disable query by zoom.

points do not show (due to tileborder?!)

When I zooming in at a certain map area the points disappear and then reappear at a slightly higher zoom again. See pictures for example. Second picture is at higher zoom. I could add a lower zoom in which the points are shown before they disappear, but I forgot to do so. Hope it is clear enough.

no entry found for key

I get this:

root@vps:/home/usr# RUST_BACKTRACE=1 t_rex generate --extent=538432.3009181806,6851282.3420069255,538432.3009181806,6851282.3420069255 --minzoom=11 --maxzoom=21 --overwrite=true --config /home/usr/osm.toml
2017-09-18 11:36:59.178 INFO Reading configuration from '/home/usr/osm.toml'
2017-09-18 11:36:59.214 INFO Tile cache directory: /mnt/vdb/tmp/mvtcache
Generating tileset 'datapoints'...
Level 21: 0 / 1 [-------------------------------------------------------------------------------------------------------------------------------------------] thread 'main' panicked at 'no entry found for key', /checkout/src/libcore/option.rs:794
stack backtrace:
   0: <unknown>
   1: <unknown>
   2: <unknown>
   3: <unknown>
   4: <unknown>
   5: <unknown>
   6: <unknown>
   7: <unknown>
   8: <unknown>
   9: <unknown>
  10: main
  11: __libc_start_main
  12: <unknown>

any help on how to run this ?

i need to know how to run this on windows please ? i downloaded the exe file but when i click it, it goes so fast.
so i try to use the commands t_rex serve it gives that it's not recognizable, so any info about how should i run this ?

Dirty tiles and cache invalidation

When updating databases with OpenStreetMap data, you generally get a tile expiry list, a list which contains tiles that have changed (e.g. osm2pgsql --expire-tiles). When pre-rendering the world these tiles need to be re-rendered async to serving. When not pre-rendering, it is more complicated and most options involve "dirty tiles" or similar.

What options are there with t-rex for dealing with changed data?