zimeon / iiif Goto Github PK

View Code? Open in Web Editor NEW

55.0 55.0 15.0 32.38 MB

IIIF Image API reference implementation and Python library

License: GNU General Public License v3.0

Python 91.25% Shell 0.48% HTML 4.91% JavaScript 3.36%

iiif's People

Stargazers

Watchers

Forkers

jbaiter edsu edsilv rlskoeser hexaplant swalter62 juancobo glenrobson kkleutghen cul-it tsukiokabunko ahva-ubc suheilmustaklemtr lead311

iiif's Issues

python3 issue serving images

New merge for 0.6.1 (#14) has problems with iiif_testserver.py on python3. Even with the very latest Flask and dependencies, there is a problem with utf-8 confusion wrt images data:

(py3)simeon@RottenApple iiif>./iiif_testserver.py --debug
Installing pil IIIFManipulator at /1.0_pil/ v1.0 none
Installing pil IIIFManipulator at /1.1_pil/ v1.1 none
Installing pil IIIFManipulator at /2.0_pil/ v2.0 none
Starting test server on http://localhost:8000/ ...
...
IIIFHandler: image_request: starfish
Debugging middleware caught exception in streamed response at a point where response headers were already sent.
Traceback (most recent call last):
  File "/Users/simeon/miniconda3/envs/py3/lib/python3.5/site-packages/werkzeug/wsgi.py", line 761, in __next__
    data = self.file.read(self.buffer_size)
  File "/Users/simeon/miniconda3/envs/py3/lib/python3.5/codecs.py", line 321, in decode
    (result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xff in position 0: invalid start byte
werkzeug: 127.0.0.1 - - [22/Apr/2016 13:16:06] "GET /2.0_pil/starfish/full/256,256/0/default HTTP/1.1" 200 -

Need to be able to specify the URI prefix used to construct the @id in info.json in iiif_static.py

Currently there is no way to do this so the info.json will only work in relative contexts.

Extend iiif.info to deal with multiple tiles specs

The current code raises an exception if there are multiple tiles spec which have been allowed since v2.0. Internal model is based on the 1.1 model and needs to be migrated forward so that the single tilespec case of 1.1 is the special case.

Drop python 2.6 support and use current Pillow

Python 2.6 is EOL and not supported by Pillow 4 or 5. Have already dropped 3.3 in #22 so support should be 2.7, 3.4 and up.

200 width

Hi,

i try to create the /full/200,/0/native.jpg for a set of images.what's the right configuration?

this is a example call

/opt/iiif/iiif_static.py -e '/full/200,/0/default.jpg' --prefix http://[host]/preview --write-html demo-static -d /var/www/preview/is --api-version=1.1 /var/www/0900271180034.jpeg

Federico

PIL converter fails to handle mode 'I;16B' TIF image

Reported by @jgreidy:

lib-5433-1:sharedshelf-to-solr jgr25$ php image-to-iiif-s3-test.php --save
--s3path 190/1406367 --force -s 1406367
image url:
Array
(
    [url] => 
http://stor.artstor.org/stor/41f7f2fd-4df0-4d02-a056-ac0f584be3f7
    [ext] => tif
)
Traceback (most recent call last):
  File "/cul/share/iiif/iiif/iiif_static.py", line 82, in <module>
    main()
  File "/cul/share/iiif/iiif/iiif_static.py", line 73, in main
    sg.generate(source, identifier=opt.identifier)
  File "/cul/share/iiif/iiif/iiif/static.py", line 162, in generate
    self.generate_tile(region,size)
  File "/cul/share/iiif/iiif/iiif/static.py", line 183, in generate_tile
    m.derive(srcfile=self.src, request=r,
outfile=os.path.join(self.dst,path))
  File "/cul/share/iiif/iiif/iiif/manipulator.py", line 114, in derive
    self.do_format(self.request.format)
  File "/cul/share/iiif/iiif/iiif/manipulator_pil.py", line 127, in
do_format
    self.image.save(self.outfile,format=format)
  File "build/bdist.macosx-10.9-intel/egg/PIL/Image.py", line 1693, in save
  File "build/bdist.macosx-10.9-intel/egg/PIL/JpegImagePlugin.py", line
571, in _save
IOError: cannot write mode I;16B as JPEG
Caught exception: Error Processing iiif: Command failed: python
/cul/share/iiif/iiif/iiif_static.py --api-version 2.0 -d
/tmp/image-to-iiif-s3/190/1406367/iiif -p
https://s3.amazonaws.com/sharedshelftosolr.library.cornell.edu/public/190/1
406367 /tmp/image-to-iiif-s3/190/1406367/image.tif

Tidy error message when a PDF file is fed to iiif_static.py

@jgreidy reports a rather unhelpful error message if one feeds a PDF file to iiif_static.py:

Traceback (most recent call last):
  File "/cul/share/iiif/iiif/iiif_static.py", line 82, in <module>
    main()
  File "/cul/share/iiif/iiif/iiif_static.py", line 73, in main
    sg.generate(source, identifier=opt.identifier)
  File "/cul/share/iiif/iiif/iiif/static.py", line 127, in generate
    im.do_first()
  File "/cul/share/iiif/iiif/iiif/manipulator_pil.py", line 45, in do_first
    raise IIIFError(text=("PIL Image.open(%s) barfed:
%s"%(self.srcfile,str(e))))
iiif.error.IIIFError: <?xml version='1.0' encoding='UTF-8'?>
<error xmlns="http://library.stanford.edu/iiif/image-api/ns/">
<parameter>unknown</parameter>
<text>PIL Image.open(/tmp/image-to-iiif-s3/test/seapapers/image.pdf)
barfed: cannot identify image file
'/tmp/image-to-iiif-s3/test/seapapers/image.pdf'</text>
</error>
Caught exception: Error Processing iiif: Command failed: python
/cul/share/iiif/iiif/iiif_static.py --api-version 2.0 -d
/tmp/image-to-iiif-s3/test/seapapers/iiif -p
XXX/image.pdf

There is no support for PDF but a better message would be nice.

iiif_static.py tile sizes & mirador

I started this in a discussion over on iiif discussion list but perhaps it's easier to track here.

I generated tiles for a 5117 x 3467 JPG using your v0.6.0 branch:

 python iiif_static.py --api-version 2.0 001.jpg --dst 001

Which seemed to work fine. Then I put together a minimal manifest that references the tiles, and pushed it all up to GitHub, with the latest build of mirador (master):

https://edsu.github.io/mirador-test/

It almost seems to work, but Mirador appears to be requesting tile sizes that are slightly different than what are available. For example I can see in the console that Mirador is requesting these URLs that 404:

Did I configure the service incorrectly? Or perhaps there’s a mismatch between Mirador and iiif_static.py regarding tile sizes?

logging

I've been using the iiif module to generate static tiles and manifests. Thanks! I was wondering if it might be feasible for the print statements to be converted to logging statements and not clutter up the output of my command line program with lots of extraneous information?

Static tiles for images less than tile size don't include full size

Issue pointed out by @jgreidy. Example to repeat with a 200x129 image:

simeon@RottenApple iiif>rm -rf /tmp/200x129
simeon@RottenApple iiif>file testimages/200x129.png
testimages/200x129.png: PNG image data, 200 x 129, 8-bit/color RGB, non-interlaced
simeon@RottenApple iiif>iiif_static.py --dst=/tmp testimages/200x129.png 
/tmp / 200x129/full/100,/0/default.jpg
/tmp / 200x129/full/50,/0/default.jpg
/tmp / 200x129/full/25,/0/default.jpg
/tmp / 200x129/full/13,/0/default.jpg
/tmp / 200x129/full/6,/0/default.jpg
/tmp / 200x129/full/3,/0/default.jpg
/tmp / 200x129/full/2,/0/default.jpg
/tmp / 200x129/full/1,/0/default.jpg - zero size, skipped

Add option to generate extra static files in iiif_static.py

It would be nice to be able to generate specific thumbnails or other derivatives in addition to the normal set of static tiles. Should have syntax like:

iiif_static.py ...params... --extra='/full/90,/0/default.jpg' --extra='/0,0,10,10/full/90/default.png' img.jpg

For cases where region is full, size is w,h and rotation is 0 then the w,h size should also be added to sizes in the info.json.

static + Mirador

I'm just wondering if anyone has tested the output of iiif.static.IIIFStatic with latest Mirador. It seems like Mirador is fetching the info.json but for whatever reason it isn't able to retereve the correct tile sizes:

tiles = iiif.static.IIIFStatic(src=image_path, dst="./images/tiles", api_version="2.0")
tiles.generate(image_path, identifier=image_id)

For example after loading my manifest Mirador wants to fetch:

http://localhost:4000/images/tiles/369ba36037c2ceba9c57a27eaff39498/full/106,/0/default.jpg

But the closest I have to that on disk is:

images/tiles/369ba36037c2ceba9c57a27eaff39498/full/114,/0/default.jpg

Here's an example info.json that is getting created:

{

  "@context": "http://iiif.io/api/image/2/context.json",
  "@id": "DSC03389",
  "height": 2736,
  "profile": [
    "http://iiif.io/api/image/2/level0.json",
    {
      "formats": [
        "jpg"
      ],
      "qualities": [
        "default"
      ]
    }
  ],
  "protocol": "http://iiif.io/api/image",
  "sizes": [
    {
      "height": 684,
      "width": 912
    },
    {
      "height": 342,
      "width": 456
    },
    {
      "height": 171,
      "width": 228
    },
    {
      "height": 86,
      "width": 114
    },
    {
      "height": 43,
      "width": 57
    },
    {
      "height": 21,
      "width": 29
    },
    {
      "height": 11,
      "width": 14
    },
    {
      "height": 5,
      "width": 7
    },
    {
      "height": 3,
      "width": 4
    },
    {
      "height": 1,
      "width": 2
    },
    {
      "height": 1,
      "width": 1
    }
  ],
  "tiles": [
    {
      "height": 1024,
      "scaleFactors": [
        1,
        2
      ],
      "width": 1024
    }
  ],
  "width": 3648
}

Smaller dimension images do not get tiled as expected

I am opening this issue to understand the limitations the iiif_static.py module has when it comes to generating tile output for smaller sized images.

I've attached an example image (it's just a small version of the Google Small Business logo) and I attempt to tile it with the command

python3 /usr/local/bin/iiif_static.py -d .<some_output_directory> -p <some_base_path> --max-image-pixels 1000000000 ./google.jpg 
--verbose

When I look at the generated tile output, I only see

- full
- info.json

and no other subsequent folders, like I would for the tile output for a larger dimension image. Meanwhile, here is the tile output for a larger dimension image

My question is
Is there any error going on in the underlying tile process that I can attempt to check for, when an image is too small to be tiled? I have run the iiif_static.py module with the --verbose flag, but none of the output directly indicates that the image file is too small for tiling.

I'd like to catch such a scenario so that I can inform users that upload an image that the image is too small for tiling. Thank you!

Google Small Business

coverage segfaults on py3.3 only for one test

Have been unable to get the coverage part of tests to pass on 3.3, get:

test03_do_first (tests.test_manipulator_pil.TestAll)
Test first step. ... /home/travis/virtualenv/python3.3.5/lib/python3.3/site-packages/PIL/OleFileIO.py:400: ResourceWarning: unclosed file <_io.BufferedReader name='testimages/test1.png'>
  for keyword, var in list(vars().items()):
/home/travis/build.sh: line 45:  6666 Segmentation fault      (core dumped) coverage run --source=iiif --omit=iiif/manipulator_netpbm.py setup.py test
The command "coverage run --source=iiif --omit=iiif/manipulator_netpbm.py setup.py test" exited with 139.

even though python setup.py test works fine.

Make PIL manipulator detect what libraries were compiled in and advertise those formats

Builds on #7. PIL manipulator should detect what libraries were compiled in, should issue warnings for missing formats, and advertise the ones supported.

Pyhthon

"Pyhthon library for IIIF" is in the Github repository description 😄

Add webp output format

This is supported in Pillow: http://pillow.readthedocs.org/en/latest/handbook/image-file-formats.html?highlight=webp#webp

(Is not supported by netpbm: http://netpbm.sourceforge.net/doc/#otherothersoftware)

Fix unhelpful error for request /full/full/0/default.jpg with v3.0

> ./iiif_testserver.py 
root: Installing pil IIIFManipulator at /1.0_pil/ v1.0 none
root: Installing pil IIIFManipulator at /1.1_pil/ v1.1 none
root: Installing pil IIIFManipulator at /2.0_pil/ v2.0 none
root: Installing pil IIIFManipulator at /2.1_pil/ v2.1 none
root: Installing pil IIIFManipulator at /3.0_pil/ v3.0 none
root: Setup server on http://localhost:8000/ ...
 * Serving Flask app "iiif_testserver" (lazy loading)
 * Environment: production
   WARNING: This is a development server. Do not use it in a production deployment.
   Use a production WSGI server instead.
 * Debug mode: off
werkzeug:  * Running on http://localhost:8000/ (Press CTRL+C to quit)

the access http://localhost:8000/3.0_pil/starfish/full/full/0/default and the error is unhelpfully:

IIIF Image Server Error

Illegal size value (not enough values to unpack (expected 2, got 1)).

parameter=size
code=400

which is likely because it looks for a comma if not max. Probably should know about full and point out the specific issue to be helpful to any developer/tester.

Make manipulators advertise the formats they support

Should be a property that, if present, can be used then to fill out the info.json formats. Note also that the canonical request syntax always specifies the format so should consider how we might build in detection and redirect. See: http://iiif.io/api/image/2.0/#format

Don't generate directory for a zero size tile that is skipped

Pointed out by @jgreidy, example:

(py3)simeon@RottenApple iiif>wget http://stor.artstor.org/stor/1c7af029-cb44-48e2-ae10-0968a6f28dae
...
(py3)simeon@RottenApple iiif>./iiif_static.py --dst /tmp/iiif 1c7af029-cb44-48e2-ae10-0968a6f28dae
iiif_static.py: source file: 1c7af029-cb44-48e2-ae10-0968a6f28dae
iiif.static: Output directory /tmp/iiif/1c7af029-cb44-48e2-ae10-0968a6f28dae already exists, adding/updating files
... tiles ...
iiif.static: /tmp/iiif / 1c7af029-cb44-48e2-ae10-0968a6f28dae/full/351,/0/default.jpg
iiif.static: /tmp/iiif / 1c7af029-cb44-48e2-ae10-0968a6f28dae/full/351,168 -> 1c7af029-cb44-48e2-ae10-0968a6f28dae/full/351,
iiif.static: /tmp/iiif / 1c7af029-cb44-48e2-ae10-0968a6f28dae/full/175,/0/default.jpg
iiif.static: /tmp/iiif / 1c7af029-cb44-48e2-ae10-0968a6f28dae/full/175,84 -> 1c7af029-cb44-48e2-ae10-0968a6f28dae/full/175,
iiif.static: /tmp/iiif / 1c7af029-cb44-48e2-ae10-0968a6f28dae/full/88,/0/default.jpg
iiif.static: /tmp/iiif / 1c7af029-cb44-48e2-ae10-0968a6f28dae/full/88,42 -> 1c7af029-cb44-48e2-ae10-0968a6f28dae/full/88,
iiif.static: /tmp/iiif / 1c7af029-cb44-48e2-ae10-0968a6f28dae/full/44,/0/default.jpg
iiif.static: /tmp/iiif / 1c7af029-cb44-48e2-ae10-0968a6f28dae/full/44,21 -> 1c7af029-cb44-48e2-ae10-0968a6f28dae/full/44,
iiif.static: /tmp/iiif / 1c7af029-cb44-48e2-ae10-0968a6f28dae/full/22,/0/default.jpg
iiif.static: /tmp/iiif / 1c7af029-cb44-48e2-ae10-0968a6f28dae/full/22,10 -> 1c7af029-cb44-48e2-ae10-0968a6f28dae/full/22,
iiif.static: /tmp/iiif / 1c7af029-cb44-48e2-ae10-0968a6f28dae/full/11,/0/default.jpg
iiif.static: /tmp/iiif / 1c7af029-cb44-48e2-ae10-0968a6f28dae/full/11,5 -> 1c7af029-cb44-48e2-ae10-0968a6f28dae/full/11,
iiif.static: /tmp/iiif / 1c7af029-cb44-48e2-ae10-0968a6f28dae/full/5,/0/default.jpg
iiif.static: /tmp/iiif / 1c7af029-cb44-48e2-ae10-0968a6f28dae/full/5,3 -> 1c7af029-cb44-48e2-ae10-0968a6f28dae/full/5,
iiif.static: /tmp/iiif / 1c7af029-cb44-48e2-ae10-0968a6f28dae/full/3,/0/default.jpg
iiif.static: /tmp/iiif / 1c7af029-cb44-48e2-ae10-0968a6f28dae/full/3,1 -> 1c7af029-cb44-48e2-ae10-0968a6f28dae/full/3,
iiif.static: /tmp/iiif / 1c7af029-cb44-48e2-ae10-0968a6f28dae/full/1,/0/default.jpg - zero size, skipped
iiif.static: /tmp/iiif / 1c7af029-cb44-48e2-ae10-0968a6f28dae/info.json

where 1, was skipped, but still there is a dir:

(py3)simeon@RottenApple iiif>tree /tmp/iiif/1c7af029-cb44-48e2-ae10-0968a6f28dae/full
/tmp/iiif/1c7af029-cb44-48e2-ae10-0968a6f28dae/full
├── 1,
│   └── 0
├── 11,
│   └── 0
│       └── default.jpg
├── 11,5 -> 11,
├── 175,
│   └── 0
│       └── default.jpg
├── 175,84 -> 175,
├── 22,
│   └── 0
│       └── default.jpg
├── 22,10 -> 22,
├── 3,
│   └── 0
│       └── default.jpg
├── 3,1 -> 3,
├── 351,
│   └── 0
│       └── default.jpg
├── 351,168 -> 351,
├── 44,
│   └── 0
│       └── default.jpg
├── 44,21 -> 44,
├── 5,
│   └── 0
│       └── default.jpg
├── 5,3 -> 5,
├── 88,
│   └── 0
│       └── default.jpg
└── 88,42 -> 88,

26 directories, 8 files

Changing the base URI for the image api

I've deployed this as the image reference implementation but when you get the info.json the @id of the image starts with http://resync.library.cornell.edu/iiif_auth_test. Is there a way to set this either as part of the deployment or as a parameter? You can see an example of this by going to:

http://iiif.io/api/image/2.1/example/reference/detail/info.json

Ideally this would be either:

a config value I can set during the docker build
or a parameter to the info.json request e.g. http://host/2.0_pil?baseuri=http://iiif.io/api/image/2.1/example/reference

I see the comment in the code about taking it from the WSGI but that won't work in this case as its running from image-api.iiif.io and proxied through the iiif.io website. Unless I can set a variable in the WSGI config file.

For info this is the docker build file:

https://github.com/IIIF/docker-image-api-reference/blob/master/Dockerfile

Seadragon demo not functioning

zimeon/iiif version 1.0.6
Just a minor issue on the Seadragon demo. I manually installed the iiif package on Windows with Python 3.6. On running the demo it appeared the Seadragon elements were failing with a 404 exception.

Manually checking iiif\demo-static suggested the target seadragon200 was in fact a file rather than a folder. Copied in a new version of Seadragon and renamed the folder as seadragon200 and the demo functioned correctly. Possibly linked directories were used in the repository version.

Add sizes of scaled full tiles to the sizes field in info.json for static tile generator

We know what we calculate so they should be listed (rather than simply targetting OSD and relying on out-of-band understanding of OSD).

Bug in max*?

Looking here. If I understand the approach, you're using the original width and height to make the scale as accurate as possible, but piping w and h through a set of conditions to arrive at the largest possible size within the protential max_* constraints. So don't you want to do:

scale = float(self.max_width) / float(self.width)
w = int(w * scale + 0.5)  # instead of self.width
h = int(h * scale + 0.5)  # instead of self.height

I'm looking to steal this :magpie:, so would love to know if I'm missing something.

16bit TIFFs are munged in PIL converstion

Reported by @jgreidy. Image http://stor.artstor.org/stor/41f7f2fd-4df0-4d02-a056-ac0f584be3f7 is not handled well. "It looks like only the blackest black pixels came through."

Remove problematic test image test1_224x167.png

The test image testimages/test1_224x167.png seems to be a scaled version of the spec image, not 224x167 and RGBA not RGB:

> file testimages/test1_224x167.png 
testimages/test1_224x167.png: PNG image data, 605 x 451, 8-bit/color RGBA, non-interlaced

Not sure why it was included, PIL has problems with it and I don't see any reference to it so think we should remove.

Check compliance levels

Have old note to check to see that these are handled correctly, not sure what that means....

Throw helpful error if caret used in size for API version < 3.0

Current implementation will accept and ignore caret (^) prefex for size in API versions before 2.1. It should instead throw a helpful error.

Help explain/control large image warnings from Pillow

While working with iiif_static.py to generate tiles, @jgreidy ran into a DecompressionBombWarning with a large image:

Create directories.
/tmp/image-to-iiif-s3/78/203936
Local copy of image.
Making iiif tiles.
/usr/lib64/python2.6/site-packages/PIL/Image.py:2261: DecompressionBombWarning: Image size (166109750 pixels) exceeds limit of 89478485 pixels, could be decompression bomb DOS attack. DecompressionBombWarning)

The code still seemed to work and produce good tiles

Possible to generate new extra image sizes only?

Once again I find myself with a good use case for static iiif image content and I'm so glad the spec supports it and this library exists.

I'm wondering if there's a way to generate only the tiles that don't already exist? Specific use case is if I need to add a new extra size across a collection of images — I only want to generate the new one, not regenerate all the others. Maybe that logic could apply to any existing tiles by default? I don't have a sense of how much that would save in terms of processing time.

If it matters, I'm calling from python directly rather than using the command line script (modeled on what the script does). I may be able to contribute code if you're open to a PR and can suggest how this should be done.

Implementations must not reuse the access cookie value as the access token value, as it could be copied across domains when the access token is obtained from a malicious client.

zimeon / iiif Goto Github PK

iiif's People

Stargazers

Watchers

Forkers

iiif's Issues

Recommend Projects

Recommend Topics

Recommend Org