Layer API about raster-foundry HOT 8 CLOSED

Thanks, Rob. My comments below:

Create Mosaic

• Do we need to anticipate some options for merging images with overlapping boundaries - perhaps Latest Image, Nearest Neighbor, Bilinear, Bicubic, and perhaps an option to use the image list sequence
• Should the Create Mosaic method also request generation of a tileset or should there be a Create Tiles method?
• Should we allow specification of an output SRID/EPSG?
• Whether generation of a tileset is supported with this method or a separate one, will likley need to support Tile options: Min/Max tile levels, tile format (JPG, PNG 8bit, PNG 24bit, etc.), tiling scheme (top-left origin vs. bottom-left origin)

Metadata Request

• For troubleshooting purposes, may be useful to return an database ID as well as a user and/or organization account
• Based on our UI wireframes, a single mosaic may have more than one set of tiles and the tilesets may have names
• It will likely be useful to return an upload date/time, mosaic start date/time and mosaic finish date/time
• Are there other image metadata info we'd want to return - we outlined a fair amount in the UI wireframes and the advisory board encouraged us to pull as much as we can from the TIFF header
• Should the TileJSON have an SRID/EPSG?

from raster-foundry.

Hi Robert, here's some responses to your comments:

Create Mosiac

Do we need to anticipate some options for merging images with overlapping boundaries - perhaps Latest Image, Nearest Neighbor, Bilinear, Bicubic, and perhaps an option to use the image list sequence

Merging images will initially be done by using the order of the image list to the API. Other tooling can be developed to set that list for an initial step, and as we move forward we will develop more options on how to merge images. So I would suggest that we try to complete a version that uses the image list for v0.1, and then add these as enhancements later, as we develop our prioritization algorithms that will do things like "prioritize the latest imagery that has cloudiness under 60%" or "do cloud removal on overlapping areas". I'm curious to hear your thoughts on how the resampling methods you list would play into the mosaicing.

Should the Create Mosaic method also request generation of a tileset or should there be a Create Tiles method?

I think I've made a mistake of naming...we've gotten a lot further in the planning around wireframes than I'm thinking of trying to hit in a quick, workable v0.1. In the former, we have a concept of layers (which you are calling tile sets), either off of OIN or user uploaded, and then projects that can mosaic from a number of layers. I'm thinking in v0.1 we don't have the multi-layer mosaics, and so this is really just talking about mosaicing one layer. So if I upload one image, it would be one layer, and I would have tile service endpoint to paint it on a map; it I throw a number of images at the import as one layer, then it will use image order to create the mosaic that can paint it on a map. Moving forward, we'll add the enhancements that will let users build additional mosaics that are derived from many layers.

Should we allow specification of an output SRID/EPSG

The web mappy-ness of the tile service requires that it be in EPSG:3857, and it seems like TileJSON is stating bounds in lat/lng. I think this is reasonable to start out with; when we later have endpoints that allow for exporting images as GeoTIFF's, for example, we can handle other CRSs.

Whether generation of a tileset is supported with this method or a separate one, will likley need to support Tile options: Min/Max tile levels, tile format (JPG, PNG 8bit, PNG 24bit, etc.), tiling scheme (top-left origin vs. bottom-left origin)

Good points. For min/max tile levels, we derive that information based on the original image resolution, and use "overzooming" to allow for serving out tiles of higher zoom levels beyond what the max zoom level for that imagery is. We could allow some user overrides on this. For tile format, I think starting with what works best for us and produces the best web maps would be a good thing to stick with for v0.1, but that would be a good option to support. If we wanted to support different tiling schemes, we could actually just create two endpoints that did some simple conversions. If this is a feature that people would be using in a v0.1, then we should put that in, or else leave it as an enhancement.

Metadata Request

For troubleshooting purposes, may be useful to return an database ID as well as a user and/or organization account

Good point. Once we get a better idea of the technical workflow of the creation, we'll know what information we can put in the response to help troubleshooting, and keep this in mind.

Based on our UI wireframes, a single mosaic may have more than one set of tiles and the tilesets may have names

This goes back to the point that this is describing a v0.1 version of the "mosaic", which I've mislabeled. I'm going to switch this issue to refer to "layers" instead (which you are calling tile sets, but I believe in the wireframes they are called layers).

It will likely be useful to return an upload date/time, mosaic start date/time and mosaic finish date/time

This is good information to record, although I'm wondering if that's more of an internal metric than the user wanting to know when the layer was created.

Are there other image metadata info we'd want to return - we outlined a fair amount in the UI wireframes and the advisory board encouraged us to pull as much as we can from the TIFF header

I think there's probably more metadata, but also perhaps more endpoints to serve them out...for example, if we wanted an endpoint that would speak to individual images in the layers, then that would provide information that was stated in the TIFF header. I agree that we want to give the user as much information as possible. I also think that, for now, if we hit the most important, necessary information, we can get some feedback on the most important information that is missing and prioritize those tasks.

Thank you for the comments. If we determine that some of these are enhancements that might not make it into v0.1, we should record them as issues so that we don't lose sight of them moving forward.

I also think it might me useful to switch from this being an issue to some API specs that are committed to the codebase, that we can write individual issues about and make PRs against. I'll do that soon.

from raster-foundry.

P.S. To Robert or anyone, like I said, I'm changing "Mosaic" to read "Layer"...I see that the TileJSON spec uses "tileset", if you think "TileSet" is better, or if there's a better name (because names are important), please let me know!

from raster-foundry.

Rob,

Thanks. In the wireframes and HTML prototypes, I believe we used the
following terminology:

• Layers = uploaded sets of images - the original TIFFs
• Tile Sets = TMS sets - each set of images could potentially have more
  than 1, though we may not need to support that at the beginning.

I think a "mosaic" is similar to a set of uploaded images, prior to being
tiled, and I think an API will likely need to support both concepts and
they will need to support a separate set of options.

Robert

On Sun, Jul 26, 2015 at 1:35 PM, Rob Emanuele [email protected]
wrote:

Hi Robert, here's some responses to your comments:
Create Mosiac

Do we need to anticipate some options for merging images with overlapping
boundaries - perhaps Latest Image, Nearest Neighbor, Bilinear, Bicubic, and
perhaps an option to use the image list sequence

Merging images will initially be done by using the order of the image list
to the API. Other tooling can be developed to set that list for an initial
step, and as we move forward we will develop more options on how to merge
images. So I would suggest that we try to complete a version that uses the
image list for v0.1, and then add these as enhancements later, as we
develop our prioritization algorithms that will do things like "prioritize
the latest imagery that has cloudiness under 60%" or "do cloud removal on
overlapping areas". I'm curious to hear your thoughts on how the resampling
methods you list would play into the mosaicing.

Should the Create Mosaic method also request generation of a tileset or
should there be a Create Tiles method?

I think I've made a mistake of naming...we've gotten a lot further in the
planning around wireframes than I'm thinking of trying to hit in a quick,
workable v0.1. In the former, we have a concept of layers (which you are
calling tile sets), either off of OIN or user uploaded, and then projects
that can mosaic from a number of layers. I'm thinking in v0.1 we don't
have the multi-layer mosaics, and so this is really just talking about
mosaicing one layer. So if I upload one image, it would be one layer, and I
would have tile service endpoint to paint it on a map; it I throw a number
of images at the import as one layer, then it will use image order to
create the mosaic that can paint it on a map. Moving forward, we'll add the
enhancements that will let users build additional mosaics that are derived
from many layers.

Should we allow specification of an output SRID/EPSG

The web mappy-ness of the tile service requires that it be in EPSG:3857,
and it seems like TileJSON is stating bounds in lat/lng. I think this is
reasonable to start out with; when we later have endpoints that allow for
exporting images as GeoTIFF's, for example, we can handle other CRSs.

Whether generation of a tileset is supported with this method or a
separate one, will likley need to support Tile options: Min/Max tile
levels, tile format (JPG, PNG 8bit, PNG 24bit, etc.), tiling scheme
(top-left origin vs. bottom-left origin)

Good points. For min/max tile levels, we derive that information based on
the original image resolution, and use "overzooming" to allow for serving
out tiles of higher zoom levels beyond what the max zoom level for that
imagery is. We could allow some user overrides on this. For tile format,
I think starting with what works best for us and produces the best web maps
would be a good thing to stick with for v0.1, but that would be a good
option to support. If we wanted to support different tiling schemes, we
could actually just create two endpoints that did some simple conversions.
If this is a feature that people would be using in a v0.1, then we should
put that in, or else leave it as an enhancement.
Metadata Request

For troubleshooting purposes, may be useful to return an database ID as
well as a user and/or organization account

Good point. Once we get a better idea of the technical workflow of the
creation, we'll know what information we can put in the response to help
troubleshooting, and keep this in mind.

Based on our UI wireframes, a single mosaic may have more than one set of
tiles and the tilesets may have names

This goes back to the point that this is describing a v0.1 version of the
"mosaic", which I've mislabeled. I'm going to switch this issue to refer to
"layers" instead (which you are calling tile sets, but I believe in the
wireframes they are called layers).

It will likely be useful to return an upload date/time, mosaic start
date/time and mosaic finish date/time

This is good information to record, although I'm wondering if that's more
of an internal metric than the user wanting to know when the layer was
created.

Are there other image metadata info we'd want to return - we outlined a
fair amount in the UI wireframes and the advisory board encouraged us to
pull as much as we can from the TIFF header

I think there's probably more metadata, but also perhaps more endpoints to
serve them out...for example, if we wanted an endpoint that would speak to
individual images in the layers, then that would provide information that
was stated in the TIFF header. I agree that we want to give the user as
much information as possible. I also think that, for now, if we hit the
most important, necessary information, we can get some feedback on the most
important information that is missing and prioritize those tasks.

Thank you for the comments. If we determine that some of these are
enhancements that might not make it into v0.1, we should record them as
issues so that we don't lose sight of them moving forward.

I also think it might me useful to switch from this being an issue to some
API specs that are committed to the codebase, that we can write individual
issues about and make PRs against. I'll do that soon.

—
Reply to this email directly or view it on GitHub
#5 (comment)
.

from raster-foundry.

Looks good for a 0.1, and I'll hold off on many of my further suggestions, as I'd love to see this exist and not bog it down with additional requirements. It can be 0.2, but I do think there is value in including the mosaic date range. Upload time is definitely less important.

from raster-foundry.

I think "Layer" is fine but I still think there is a distinction to be made
between the original set of images that are uploaded and the tile set that
is output.

Robert

On Sun, Jul 26, 2015 at 1:36 PM, Rob Emanuele [email protected]
wrote:

P.S. To Robert or anyone, like I said, I'm changing "Mosaic" to read
"Layer"...if you think "TileSet" is better, or if there's a better name
(because names are important), please let me know!

—
Reply to this email directly or view it on GitHub
#5 (comment)
.

from raster-foundry.

@rcheetham good point. I think the Layer can be the combination of the image set and tile set as it exists in RF, and any metadata associated with it.

@cholmes cool, I agree that seeing this exist would be a good first step and to keep it lightweight until after then, when we can refine and enhance. About mosaic date range...pulling time off of imagery is tough, because there's no standard on how that's encoded. Do you have any insight on how we'd handle that? Is it a matter of asking the user when the imagery is imported? Or is it common to place in GTiff GDAL metadata tags, so we could ask for the tag name? It would be nice to have the date range auto generated.

from raster-foundry.

@kdeloach and I came up with the following URL structure for the API. I'd appreciate any feedback.

Get all layers created by a user (that the requester has permission to access)
-GET /<username>/layers/

Post a layer (including tags)
-POST /<username>/layers/

Get a specific layer
-GET /<username>/<layer_slug>/

Delete a layer
-DELETE /<username>/<layer_slug>/

Get all layers
-GET /layers/

Optional query strings can be provided to the above endpoints for searching (including filtering by tag and favorite) and sorting

Get metadata for a specific layer
-GET /<username>/<layer_slug>/meta/

Post and delete favorites
-POST and DELETE /<username>/<layer_slug>/favorite/

from raster-foundry.