Code Monkey home page Code Monkey logo

mbtiles-spec's Introduction

MBTiles Specification

MBTiles is a specification for storing arbitrary tiled map data in SQLite databases for immediate usage and for efficient transfer. MBTiles files, known as tilesets, must implement the specification below to ensure compatibility with devices.

Versions

Concept

MBTiles is a compact, restrictive specification. It supports only tiled data, including vector or image tiles and interactivity grid tiles. Only the Spherical Mercator projection is supported for presentation (tile display), and only latitude-longitude coordinates are supported for metadata such as bounds and centers.

It is a minimum specification, only specifying the ways in which data must be retrievable. Thus MBTiles files can internally compress and optimize data, and construct views that adhere to the MBTiles specification.

Unlike Spatialite, GeoJSON, and Rasterlite, MBTiles is not raw data storage. It is storage for tiled data, like rendered map tiles.

One MBTiles file represents a single tileset, optionally including grids of interactivity data. Multiple tilesets (layers, or maps in other terms) can be represented by multiple MBTiles files.

UTFGrid

The MBTiles specification previously contained the UTFGrid specification. It was removed in version 1.2 and moved into its own specification with synced version numbers, so MBTiles 1.2 is compatible with UTFGrid 1.2. The specs integrate but do not require each other for compliance.

License

The text of this specification is licensed under a Creative Commons Attribution 3.0 United States License. However, the use of this spec in products and code is entirely free: there are no royalties, restrictions, or requirements.

Authors

  • Eric Fischer (ericfischer)
  • Konstantin Kaefer (kkaefer)
  • Tom MacWright (tmcw)
  • Justin Miller (incanus)
  • Paul Norman (pnorman)
  • Blake Thompson (flippmoke)
  • Will White (willwhite)

mbtiles-spec's People

Contributors

e-n-f avatar flippmoke avatar kkaefer avatar tmcw avatar wrynearson avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

mbtiles-spec's Issues

Using 32 byte hashes as IDs uses an unnecessary amount of space

We have 5 columns that contain 32 byte string IDs. Using integer keys (let's assume 4 bytes per integer, although SQLite3's integer length is variable) instead, we could save 5 * (32 - 4) = 140 bytes per tile. Given world-light with 5592405 tiles, that'd save 5592405 * 140 = 782936700 bytes = 746 MB. (This calculation assumes interactivity which world-light doesn't have)

MbTiles and Openlayers.Layer.TMS not working

I tried these combinations (below), no errors but not display the tiles.

new OpenLayers.Layer.TMS("MapBox Tile Natural Earth 2", "http://localhost:8888/1.0.0/natural-earth-2/0.0.0.png",
{layername: 'natural-earth-2',
type: 'png',
transitionEffect: 'resize'
}),

2

new OpenLayers.Layer.TMS("MapBox Tile Natural Earth 2", "http://localhost:8888/1.0.0/natural-earth-2/0/0/0.png",
{layername: 'natural-earth-2',
type: 'png',
transitionEffect: 'resize'
}),

  1. with http://localhost:8080/1.0.0/natural-earth-2/0.0.0.png",
  2. with http://localhost/1.0.0/natural-earth-2/0.0.0.png",

1.0.0 is local directory in my ipad Xcode Openlayer project.

All the base layers are working e.g OSM, Google, Bing, ESRI street except Mapbox MbTiles.

I hope to get offline mbtiles with openlayers + spatialite + JSTS in ipad 2.

TIA.

Noli

grids vs tiles

From the spec it's quite unclear what the grids table is for. Is there an explanation that I missed?

Regarding .MBTiles to .DB Conversion

Hi.

I am using an Android Apps for Offline Mapping that uses .DB file for Map SQLite Database. I am trying to view my own Country Map by making .MBTiles using TileMill then rename the .MBTiles to .DB.

I observed the Map is not coming to the Apps. After doing several Zoom-out I can see the Map. Still while Zoom-In the Map the area gets divided into two Parts.

So far I think the problem is due to MBTiles Inverted Y-Axis. Please correct me if my thought is wrong.

I wanted to ask ... Is there any way to export OSM Data as .DB (not .MBTiles) by setting Min and Max Zoom level and Boundary?

Or any way to convert .MBTiles to .DB?

Thanks in advance.

Regards.

SQLite4 compatibility

As I understand SQLite3 and SQLite4 will be different at binary level. ( = breaking compatibility )

But according the mbtiles-spec 1.2 spec

  • "Tilesets are expected to be valid SQLite databases of version 3.0.0 or higher"

imho: the "or higher" can be misinterpret as a non compatible SQLite4 version.

so my proposal - define more clearly the SQLite3 compatibility.

  • "Tilesets are expected to be valid SQLite3 databases of version 3.x.x "

Uncaught (in promise) Error: Unimplemented type: 3

I have read that this error is being caused by the pbf files not being gzipped when being imported and saved. How exactly do I compress them in transit? I have installed the compression module for express and I gzips everything except these pbfs. Is there some sort of middleware that I am missing?

MBTiles - metadata - suggestion for OpenStreetMap support

In the 'Roadmap' text it is suggesting that support for the OpenStreetMap naming convention is being considered.

https://github.com/mapbox/mbtiles-spec/

'The format will switch tile ordering to the XYZ schema popularized by OpenStreetMap and away from the Tile Map Service specification.'

I would like to suggest the following:

add a 'metadata' name:
name='tile_row_type'

with (at the moment) 2 possible values:

'tms'
'osm'

A reader could then check for 'tile_row_type':

I am the author of the Bash-Shell script tileinfo.sh

http://wiki.openstreetmap.org/wiki/Slippy_map_tilenames#tileinfo.sh

this tool was developed as a helper to determine a tile address from a WSG84-position (for both tms and osm)
or to determine a WSG84-position from a tile address (for both tms and osm).

Today I have added the functionality of creating a .mbtile database from an existing tile-directory

  • assuming that either sqlite3 or spatialite is installed
  • a parameter can be given to tell the script that the tile-directory is a tms directory

Here I have added the metadata name 'tile_row_type', setting the value always to 'tms' - since all readers (at the moment) assume this. I will be doing the same for a QT based map library that reads tile-directorys and show the result on a map.

There are also Android applications, that together with their format (similar, yet different) also can read the present mbtiles format.

  • OruxMaps is the Application that I use mostly on my telefon/tablet and this application can read the created .mbtiles
    -- including the metadata names bounds and center - when listed: shows the extent of 'bounds' and when opened zooms to the position given in 'center'
    -- unfortunately, at the moment at least, a empty 'android_metadata' table must exist for this to work
    I have had contact with this developer and would like to convince him to switch over to the .mbtiles format.
    At the moment his application can create and fill a sqlite.db from an selected area.
    The created database uses the fields (z,x,y,image) and stores a 'CalibrationPoint' in an external xml file, which is used as the (oms) position 0,0.
    I am planing to work on a conversion function for this and will be using a 'calibration_point' in the metadata to calculate the offset from the default value of: '-180,-85.0511,180,85.0511'

For this reason I would also suggest adding a name 'calibration_point'

  • the default being '-180,-85.0511,180,85.0511'.

Adding these 2 values for reading applications to use would, I believe, encourage a wider use of the .mbtiles format.

Mark Johnson, Berlin Germany

Spec version

We might want to add a spec or schema version property to the metadata table and require individual mbtiles to specify which version of the spec they conform to. It could help applications handle any breaking changes in the future.

How to translate from coordinates to zoom_level, tile_column, tile_row?

I'm currently generating a .mbtiles file using TileMill and OpenStreetMaps data (with the OSM Bright project).
To optimize the file size I'm thinking of only leaving the high zoom tiles in the certain coordinates of the map. To know which tiles I can delete I need to know how to transform coordinates into: zoom_level, tile_column, tile_row pairs.

Has this been done before?
Can tiles be deleted without corrupting the .mbtiles file?

The mbtiles-spec is outdated, needs revison

The newest 1.2 spec is outdated. If you have motivation to update this spec, I will submit PR on the aspect of:

  1. add spec version field in metadata
  2. support for pbf
  3. considering migrate scheme from tms to zxy

use spatialite instead of sqlite

I believe that using spatiale could give many advantages

  • In addition to the tiles you will be able to add one or more vector and then use the power of the spatial sql offline
  • spatialite also offers rasterlite a feature that could help in performance

grid.json storage

exactly how the grids are stored in the file is not clearly defined making attempts at mbtile implementation which include grids difficult.

As far as I can tell they are spread across 3 fields in 2 views, with "keys":{ and "grid"{ including in the database but but "data":{ and grid( not plus 1 of the 3 fields is gziped.

Attribution in metadata

The MapBox iPad app can read text from an attribution key from the metadata table and display this text in the lower left corner. This is something we could document and include in the formal spec as an optional attribute.

Projection independence

The spec mentions "only the Spherical Mercator projection is supported for presentation".

Is this limitation necessary? It's just tiles so I would guess the actual projection of those tiles doesn't matter since mbtiles just stores them.

Write up legends

Write up legends in official/detailed language:

  1. Key/value in metadata table:
    • legend
    • [ self-contained html snippet ] of legend data
  2. Legend is served to the client by the service at 1.0.0/[layer]/legend.json, with a json object of the form:
    • { "legend": "[html snippet]" }

zxy from zoom_level, tile_column, tile_row in mbtiles file

Hi!

I want to get all the zxy of tiles present. So, I opened my mbtiles file in an sqlite3 viewer. I see that my zoom_level and tile_column values are the same as z and x in my mapbox-gl front end requests. But, my tile_row is very different from my y. How do I get my y from the tile_row?

Thanks :)

Security / performance ramifications of views

The implementation of views needs some thought. Mainly:

  • Allowing full Javascript allows tileset creators to run arbitrary code on any websites the tilesets are deployed on.
  • Some tilesets could benefit from server-side view-running, since a lot of information will be stored with each feature, and views being rendered on the server would reduce tremendously the amount of transfer & computation on the machine.

There are alternatives:

  • Ditching the idea of views and simply specifying non-semantic data
  • Supporting a subset of javascript, or building tools to scan for malicious code
  • Supporting a template language

For the time being, this can be pushed for ecosystem reasons, but we need to discuss this, because if the current approach continues, its security ramifications should be noted in the spec.

How do you allow the map to zoom more than the map actually has

We have an iPhone app that uses mapbox to display offline maps created using tilemill. In previous versions of Mapbox the user could zoom in as much as they wanted even though the map had only be created to a level 15 zoom. This was important to us as we overlay a GPX track and waypoints over the map and it is important we can zoom in to view that information. The map of course distorted but wasn't a major issue.

We have created various maps with tilemill however we are finding that to be able to zoom in to the level we require needs a map of 100s of megabytes which is not feasible for an iPhone app. In previous editions of Mapbox the user was able to zoom in further than the zoom set when creating the app.

Is there any way to zoom into the maps further? or are there any other alternatives to view offline maps on a mobile device?

Thanks.

Additional comment: Another option would be to hide all data at zoom levels above 17 so that the map would go blank above zoom 17. i.e. user can zoom in but once they past a certain point the map goes blank. Therefore not requiring the millions of tile images above zoom level 17. How could we do that in Tilemill?

service.md

We need to document interaction service as a spec.

layer.json service must provide json at the base path of its tiles to declare that it supports interaction. Base path is interpreted as the URL up to but not including z, x, and y coordinates, examples:

/1.0.0/foo/0/0/0.png
/1.0.0/foo/layer.json

/0/0/0.png
/layer.json

The json object keys

  • formatter Javascript formatter function represented as a string. If omitted, clients will assume that grid interaction is not supported by this layer.

  • legend HTML string. Legend for this layer.

    {
    "formatter": "function(data, options) { return data.NAME; }",
    "legend": "Countries of the World"
    }

Compatibility Grid

We should make a grid that shows which MBTiles features are supported by which implementations or clients.

Allow different tilesets in one file

I need to add several tilesets to one sqlite file. MBTiles does not support it.

Some sqlite tile schemas allow it by adding something like following:
a) add field tileset_id to tiles table
b) add field tileleset_id to metadata table

Update vector_layers to match tilejson-spec 3.0

The TileJSON specification v3.0 will be released soon - this version documents the de factor requirements for vector_layers by adapting the text from 1.3 of this specification.

A key difference is that the TileJSON specification does not require field values to be explicitly Number, String, or Boolean and suggests it should just be a string that describes the attribute. Here is what the mbtiles specification says:

fields (object): A JSON object whose keys and values are the names and types of attributes available in this layer. Each type MUST be the string "Number", "Boolean", or "String". Attributes whose type varies between features SHOULD be listed as "String".

And the tilejson-spec 3.0 proposes the following:

fields (object): A JSON object whose keys and values are the names and descriptions of attributes available in this layer. Each value (description) MUST be a string. The values MAY describe the underlying data type, such as "String", "Number", or "Boolean". Attributes whose type varies between features MAY be listed as "String".

The current mbtiles-spec will continue to be valid once 3.0 is released with this description (since they are strings), but the requirements are unnecessarily strict. In general, the mbtiles-specification is driven by encoders, so the current requirement is likely driven by the inability for computers to put together implicit descriptions other than determining the data type.

Once tilejson-spec is released, can we update the mbtiles-spec to be more relaxed and point to this description of "vector_layers" as the source of truth?

cc @ericfischer @GretaCB @flippmoke

Document optional client CSS

Interactivity clients can add CSS styles to parent containers. Map authors can use these styles to change how their content display on different devices and clients. Styles can look like:

  • .ipad .mapbox
  • .web .wax

Move or keep UTFGrid specification

It's uncertain whether we want to make UTFGrid part of the MBTiles spec or whether it should be its own spec project. The UTFGrid work doesn't have much interaction with the underlying storage.

EPSG:4326 projection support

The specification states that

The global-mercator (aka Spherical Mercator) profile is assumed

Is there any reason why it's limited by only global-mercator profile while global-geodetic profile is also supported in TMS specification? Can global-geodetic profile be also added to this specification?

Our server application generates TMS tilesets in EPSG:4326 (global geodetic), and I was hoping that some MBTiles implementation such as node-mbtiles can be used to store all tiles of a tileset in a single file. Suprisignly, it can't.

Unfortunately, we can't reproject our tilesets to EPSG:3857 due to they cover the whole globe territory [-180;-90; 180; 90]. As the result, all MBTiles implementations (I guess) are useless in this case.

Support GeoJSON

From what I can see, MBTiles would work perfectly with GeoJSON.

I'm just writing "geojson" in my DB instead of "png". I'd love this to be part of the spec, though.

Grids Creation Statement contains a typo

Hi,

The specs mention a GRIDS table but the create statement is for a GRID table (no final S).

Is:
CREATE TABLE grid (zoom_level integer, tile_column integer, tile_row integer, grid blob);

Should be:
CREATE TABLE grids (zoom_level integer, tile_column integer, tile_row integer, grid blob);

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.