when using jpeg profile silently creates empty output if bit depth != 8 about rio-cogeo HOT 9 CLOSED

Thanks @alon this is interesting. I'll have a look at it

from rio-cogeo.

from rio-cogeo.

@alon

Is it possible to serve a COG without a tiling service

Absolutely, I'm usually using https://github.com/cogeotiff/rio-tiler (and https://github.com/vincentsarago/lambda-tiler) to do this but you could also check https://github.com/DHI-GRAS/terracotta or https://github.com/mojodna/marblecutter which provide the full architecture.

That's said, if you have int16 dataset you'll need custom script to rescale your data to int8.

from rio-cogeo.

@alon This is a rasterio problem and I'm not sure if there is a way around it.

Basically rasterio.shutil.copy https://github.com/mapbox/rasterio/blob/master/rasterio/shutil.pyx#L56 is swallowing GDAL warnings but I don't think there is a way to fix that.

cc @sgillies

from rio-cogeo.

@vincentsarago the errors occur during the execution of GDALCreateCopy at https://github.com/mapbox/rasterio/blob/master/rasterio/shutil.pyx#L137, right? Because we've released Python's global interpreter lock, the errors cannot be turned into exceptions. It might be possible to create a GDAL error handler that would break out of GDALCreateCopy and the nogil block using setjmp and longjmp, but I've never done that in Cython before and I'm hesitant to do so 😰 Validating the data type before copying is probably the best solution.

from rio-cogeo.

@sgillies thanks for the investigation.
That's said, I think GDAL doesn't throw error but just warnings (ERROR 1: JPEGSetupEncode:BitsPerSample 16 not allowed for JPEG)

Validating the data type before copying is probably the best solution

Yeah I think this is what we should do 🙏

from rio-cogeo.

🤔 @sgillies I've just encountered this with other options

# This command runs find but the result is empty (data is not copied)
$ rio cogeo create float64.tif float64cog.tif --co BLOCKXSIZE=256 --co BLOCKXSIZE=256 --cog-profile deflate --co ZLEVEL=6 --co PREDICTOR=2

while with gdal 
gdal_translate  float64.tif float64cog.tif -co BLOCKXSIZE=256 -co BLOCKYSIZE=256 -co COMPRESS=DEFLATE -co ZLEVEL=6 -co PREDICTOR=2
Input file size is 301, 520
0...10...20...30...40...50...60...70...80...90...100 - done.
ERROR 1: PredictorSetup:Horizontal differencing "Predictor" not supported with 64-bit samples
ERROR 1: PredictorSetup:Horizontal differencing "Predictor" not supported with 64-bit samples
ERROR 1: WriteEncodedTile/Strip() failed.
ERROR 1: PredictorSetup:Horizontal differencing "Predictor" not supported with 64-bit samples
ERROR 1: WriteEncodedTile/Strip() failed.
ERROR 1: PredictorSetup:Horizontal differencing "Predictor" not supported with 64-bit samples

I feel there are a lot of corner case I'll need to check ...

from rio-cogeo.

@alon

Is it possible to serve a COG without a tiling service

Absolutely, I'm usually using https://github.com/cogeotiff/rio-tiler (and https://github.com/vincentsarago/lambda-tiler) to do this but you could also check https://github.com/DHI-GRAS/terracotta or https://github.com/mojodna/marblecutter which provide the full architecture.

Right, I understand it is doable with a service which creates the tiles on demand, and then serves them:
COG -> TILER -> FRONTEND
And showing the artifacts:
COG: single file on disk
TILER: in memory / cached on disk tiles (png per X/Y/Z)

But I'd like to forgoe the tiler, and have
COG -> FRONTEND
where the arrow signifies HTTP traffic.

My current flow is:
Preprocessing: TIFF (not COG) -> tiles -> cloud (azure)
CLOUD -> FRONTEND (via leaflet)
The tiling stage is very slow, I'm using gdal2tiles with zoom levels 18, which is brutal for a 50k x 20k image, even though most of it is NODATA.

@vincentsarago I would happily discuss this offline, or publically but on a more appropriate place than this ticket :) I'm alon at pobox dotcom

from rio-cogeo.

@alon

COG -> FRONTEND

checkout https://github.com/geotiffjs/geotiff.js

I would happily discuss this offline, or publically but on a more appropriate place than this ticket :) I'm alon at pobox dotcom

We have a #cogeotiff Chanel on thespatialcommunity slack -> https://thespatialcommunity.org

from rio-cogeo.