zimeon / iiif Goto Github PK
View Code? Open in Web Editor NEWIIIF Image API reference implementation and Python library
License: GNU General Public License v3.0
IIIF Image API reference implementation and Python library
License: GNU General Public License v3.0
New merge for 0.6.1 (#14) has problems with iiif_testserver.py on python3. Even with the very latest Flask and dependencies, there is a problem with utf-8 confusion wrt images data:
(py3)simeon@RottenApple iiif>./iiif_testserver.py --debug
Installing pil IIIFManipulator at /1.0_pil/ v1.0 none
Installing pil IIIFManipulator at /1.1_pil/ v1.1 none
Installing pil IIIFManipulator at /2.0_pil/ v2.0 none
Starting test server on http://localhost:8000/ ...
...
IIIFHandler: image_request: starfish
Debugging middleware caught exception in streamed response at a point where response headers were already sent.
Traceback (most recent call last):
File "/Users/simeon/miniconda3/envs/py3/lib/python3.5/site-packages/werkzeug/wsgi.py", line 761, in __next__
data = self.file.read(self.buffer_size)
File "/Users/simeon/miniconda3/envs/py3/lib/python3.5/codecs.py", line 321, in decode
(result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xff in position 0: invalid start byte
werkzeug: 127.0.0.1 - - [22/Apr/2016 13:16:06] "GET /2.0_pil/starfish/full/256,256/0/default HTTP/1.1" 200 -
Currently there is no way to do this so the info.json will only work in relative contexts.
The current code raises an exception if there are multiple tiles spec which have been allowed since v2.0. Internal model is based on the 1.1 model and needs to be migrated forward so that the single tilespec case of 1.1 is the special case.
Python 2.6 is EOL and not supported by Pillow 4 or 5. Have already dropped 3.3 in #22 so support should be 2.7, 3.4 and up.
Hi,
i try to create the /full/200,/0/native.jpg for a set of images.what's the right configuration?
this is a example call
/opt/iiif/iiif_static.py -e '/full/200,/0/default.jpg' --prefix http://[host]/preview --write-html demo-static -d /var/www/preview/is --api-version=1.1 /var/www/0900271180034.jpeg
Federico
Reported by @jgreidy:
lib-5433-1:sharedshelf-to-solr jgr25$ php image-to-iiif-s3-test.php --save
--s3path 190/1406367 --force -s 1406367
image url:
Array
(
[url] =>
http://stor.artstor.org/stor/41f7f2fd-4df0-4d02-a056-ac0f584be3f7
[ext] => tif
)
Traceback (most recent call last):
File "/cul/share/iiif/iiif/iiif_static.py", line 82, in <module>
main()
File "/cul/share/iiif/iiif/iiif_static.py", line 73, in main
sg.generate(source, identifier=opt.identifier)
File "/cul/share/iiif/iiif/iiif/static.py", line 162, in generate
self.generate_tile(region,size)
File "/cul/share/iiif/iiif/iiif/static.py", line 183, in generate_tile
m.derive(srcfile=self.src, request=r,
outfile=os.path.join(self.dst,path))
File "/cul/share/iiif/iiif/iiif/manipulator.py", line 114, in derive
self.do_format(self.request.format)
File "/cul/share/iiif/iiif/iiif/manipulator_pil.py", line 127, in
do_format
self.image.save(self.outfile,format=format)
File "build/bdist.macosx-10.9-intel/egg/PIL/Image.py", line 1693, in save
File "build/bdist.macosx-10.9-intel/egg/PIL/JpegImagePlugin.py", line
571, in _save
IOError: cannot write mode I;16B as JPEG
Caught exception: Error Processing iiif: Command failed: python
/cul/share/iiif/iiif/iiif_static.py --api-version 2.0 -d
/tmp/image-to-iiif-s3/190/1406367/iiif -p
https://s3.amazonaws.com/sharedshelftosolr.library.cornell.edu/public/190/1
406367 /tmp/image-to-iiif-s3/190/1406367/image.tif
@jgreidy reports a rather unhelpful error message if one feeds a PDF file to iiif_static.py:
Traceback (most recent call last):
File "/cul/share/iiif/iiif/iiif_static.py", line 82, in <module>
main()
File "/cul/share/iiif/iiif/iiif_static.py", line 73, in main
sg.generate(source, identifier=opt.identifier)
File "/cul/share/iiif/iiif/iiif/static.py", line 127, in generate
im.do_first()
File "/cul/share/iiif/iiif/iiif/manipulator_pil.py", line 45, in do_first
raise IIIFError(text=("PIL Image.open(%s) barfed:
%s"%(self.srcfile,str(e))))
iiif.error.IIIFError: <?xml version='1.0' encoding='UTF-8'?>
<error xmlns="http://library.stanford.edu/iiif/image-api/ns/">
<parameter>unknown</parameter>
<text>PIL Image.open(/tmp/image-to-iiif-s3/test/seapapers/image.pdf)
barfed: cannot identify image file
'/tmp/image-to-iiif-s3/test/seapapers/image.pdf'</text>
</error>
Caught exception: Error Processing iiif: Command failed: python
/cul/share/iiif/iiif/iiif_static.py --api-version 2.0 -d
/tmp/image-to-iiif-s3/test/seapapers/iiif -p
XXX/image.pdf
There is no support for PDF but a better message would be nice.
I started this in a discussion over on iiif discussion list but perhaps it's easier to track here.
I generated tiles for a 5117 x 3467 JPG using your v0.6.0 branch:
python iiif_static.py --api-version 2.0 001.jpg --dst 001
Which seemed to work fine. Then I put together a minimal manifest that references the tiles, and pushed it all up to GitHub, with the latest build of mirador (master):
https://edsu.github.io/mirador-test/
It almost seems to work, but Mirador appears to be requesting tile sizes that are slightly different than what are available. For example I can see in the console that Mirador is requesting these URLs that 404:
Did I configure the service incorrectly? Or perhaps thereβs a mismatch between Mirador and iiif_static.py regarding tile sizes?
I've been using the iiif module to generate static tiles and manifests. Thanks! I was wondering if it might be feasible for the print statements to be converted to logging statements and not clutter up the output of my command line program with lots of extraneous information?
Issue pointed out by @jgreidy. Example to repeat with a 200x129 image:
simeon@RottenApple iiif>rm -rf /tmp/200x129
simeon@RottenApple iiif>file testimages/200x129.png
testimages/200x129.png: PNG image data, 200 x 129, 8-bit/color RGB, non-interlaced
simeon@RottenApple iiif>iiif_static.py --dst=/tmp testimages/200x129.png
/tmp / 200x129/full/100,/0/default.jpg
/tmp / 200x129/full/50,/0/default.jpg
/tmp / 200x129/full/25,/0/default.jpg
/tmp / 200x129/full/13,/0/default.jpg
/tmp / 200x129/full/6,/0/default.jpg
/tmp / 200x129/full/3,/0/default.jpg
/tmp / 200x129/full/2,/0/default.jpg
/tmp / 200x129/full/1,/0/default.jpg - zero size, skipped
It would be nice to be able to generate specific thumbnails or other derivatives in addition to the normal set of static tiles. Should have syntax like:
iiif_static.py ...params... --extra='/full/90,/0/default.jpg' --extra='/0,0,10,10/full/90/default.png' img.jpg
For cases where region
is full
, size
is w,h
and rotation
is 0 then the w,h
size should also be added to sizes
in the info.json
.
I'm just wondering if anyone has tested the output of iiif.static.IIIFStatic with latest Mirador. It seems like Mirador is fetching the info.json but for whatever reason it isn't able to retereve the correct tile sizes:
tiles = iiif.static.IIIFStatic(src=image_path, dst="./images/tiles", api_version="2.0")
tiles.generate(image_path, identifier=image_id)
For example after loading my manifest Mirador wants to fetch:
http://localhost:4000/images/tiles/369ba36037c2ceba9c57a27eaff39498/full/106,/0/default.jpg
But the closest I have to that on disk is:
images/tiles/369ba36037c2ceba9c57a27eaff39498/full/114,/0/default.jpg
Here's an example info.json that is getting created:
{
"@context": "http://iiif.io/api/image/2/context.json",
"@id": "DSC03389",
"height": 2736,
"profile": [
"http://iiif.io/api/image/2/level0.json",
{
"formats": [
"jpg"
],
"qualities": [
"default"
]
}
],
"protocol": "http://iiif.io/api/image",
"sizes": [
{
"height": 684,
"width": 912
},
{
"height": 342,
"width": 456
},
{
"height": 171,
"width": 228
},
{
"height": 86,
"width": 114
},
{
"height": 43,
"width": 57
},
{
"height": 21,
"width": 29
},
{
"height": 11,
"width": 14
},
{
"height": 5,
"width": 7
},
{
"height": 3,
"width": 4
},
{
"height": 1,
"width": 2
},
{
"height": 1,
"width": 1
}
],
"tiles": [
{
"height": 1024,
"scaleFactors": [
1,
2
],
"width": 1024
}
],
"width": 3648
}
I am opening this issue to understand the limitations the iiif_static.py
module has when it comes to generating tile output for smaller sized images.
I've attached an example image (it's just a small version of the Google Small Business logo) and I attempt to tile it with the command
python3 /usr/local/bin/iiif_static.py -d .<some_output_directory> -p <some_base_path> --max-image-pixels 1000000000 ./google.jpg
--verbose
When I look at the generated tile output, I only see
- full
- info.json
and no other subsequent folders, like I would for the tile output for a larger dimension image. Meanwhile, here is the tile output for a larger dimension image
My question is
Is there any error going on in the underlying tile process that I can attempt to check for, when an image is too small to be tiled? I have run the iiif_static.py
module with the --verbose
flag, but none of the output directly indicates that the image file is too small for tiling.
I'd like to catch such a scenario so that I can inform users that upload an image that the image is too small for tiling. Thank you!
Have been unable to get the coverage part of tests to pass on 3.3, get:
test03_do_first (tests.test_manipulator_pil.TestAll)
Test first step. ... /home/travis/virtualenv/python3.3.5/lib/python3.3/site-packages/PIL/OleFileIO.py:400: ResourceWarning: unclosed file <_io.BufferedReader name='testimages/test1.png'>
for keyword, var in list(vars().items()):
/home/travis/build.sh: line 45: 6666 Segmentation fault (core dumped) coverage run --source=iiif --omit=iiif/manipulator_netpbm.py setup.py test
The command "coverage run --source=iiif --omit=iiif/manipulator_netpbm.py setup.py test" exited with 139.
even though python setup.py test
works fine.
Builds on #7. PIL manipulator should detect what libraries were compiled in, should issue warnings for missing formats, and advertise the ones supported.
"Pyhthon library for IIIF" is in the Github repository description π
This is supported in Pillow: http://pillow.readthedocs.org/en/latest/handbook/image-file-formats.html?highlight=webp#webp
(Is not supported by netpbm: http://netpbm.sourceforge.net/doc/#otherothersoftware)
> ./iiif_testserver.py
root: Installing pil IIIFManipulator at /1.0_pil/ v1.0 none
root: Installing pil IIIFManipulator at /1.1_pil/ v1.1 none
root: Installing pil IIIFManipulator at /2.0_pil/ v2.0 none
root: Installing pil IIIFManipulator at /2.1_pil/ v2.1 none
root: Installing pil IIIFManipulator at /3.0_pil/ v3.0 none
root: Setup server on http://localhost:8000/ ...
* Serving Flask app "iiif_testserver" (lazy loading)
* Environment: production
WARNING: This is a development server. Do not use it in a production deployment.
Use a production WSGI server instead.
* Debug mode: off
werkzeug: * Running on http://localhost:8000/ (Press CTRL+C to quit)
the access http://localhost:8000/3.0_pil/starfish/full/full/0/default and the error is unhelpfully:
IIIF Image Server Error
Illegal size value (not enough values to unpack (expected 2, got 1)).
parameter=size
code=400
which is likely because it looks for a comma if not max
. Probably should know about full
and point out the specific issue to be helpful to any developer/tester.
Should be a property that, if present, can be used then to fill out the info.json formats. Note also that the canonical request syntax always specifies the format so should consider how we might build in detection and redirect. See: http://iiif.io/api/image/2.0/#format
Pointed out by @jgreidy, example:
(py3)simeon@RottenApple iiif>wget http://stor.artstor.org/stor/1c7af029-cb44-48e2-ae10-0968a6f28dae
...
(py3)simeon@RottenApple iiif>./iiif_static.py --dst /tmp/iiif 1c7af029-cb44-48e2-ae10-0968a6f28dae
iiif_static.py: source file: 1c7af029-cb44-48e2-ae10-0968a6f28dae
iiif.static: Output directory /tmp/iiif/1c7af029-cb44-48e2-ae10-0968a6f28dae already exists, adding/updating files
... tiles ...
iiif.static: /tmp/iiif / 1c7af029-cb44-48e2-ae10-0968a6f28dae/full/351,/0/default.jpg
iiif.static: /tmp/iiif / 1c7af029-cb44-48e2-ae10-0968a6f28dae/full/351,168 -> 1c7af029-cb44-48e2-ae10-0968a6f28dae/full/351,
iiif.static: /tmp/iiif / 1c7af029-cb44-48e2-ae10-0968a6f28dae/full/175,/0/default.jpg
iiif.static: /tmp/iiif / 1c7af029-cb44-48e2-ae10-0968a6f28dae/full/175,84 -> 1c7af029-cb44-48e2-ae10-0968a6f28dae/full/175,
iiif.static: /tmp/iiif / 1c7af029-cb44-48e2-ae10-0968a6f28dae/full/88,/0/default.jpg
iiif.static: /tmp/iiif / 1c7af029-cb44-48e2-ae10-0968a6f28dae/full/88,42 -> 1c7af029-cb44-48e2-ae10-0968a6f28dae/full/88,
iiif.static: /tmp/iiif / 1c7af029-cb44-48e2-ae10-0968a6f28dae/full/44,/0/default.jpg
iiif.static: /tmp/iiif / 1c7af029-cb44-48e2-ae10-0968a6f28dae/full/44,21 -> 1c7af029-cb44-48e2-ae10-0968a6f28dae/full/44,
iiif.static: /tmp/iiif / 1c7af029-cb44-48e2-ae10-0968a6f28dae/full/22,/0/default.jpg
iiif.static: /tmp/iiif / 1c7af029-cb44-48e2-ae10-0968a6f28dae/full/22,10 -> 1c7af029-cb44-48e2-ae10-0968a6f28dae/full/22,
iiif.static: /tmp/iiif / 1c7af029-cb44-48e2-ae10-0968a6f28dae/full/11,/0/default.jpg
iiif.static: /tmp/iiif / 1c7af029-cb44-48e2-ae10-0968a6f28dae/full/11,5 -> 1c7af029-cb44-48e2-ae10-0968a6f28dae/full/11,
iiif.static: /tmp/iiif / 1c7af029-cb44-48e2-ae10-0968a6f28dae/full/5,/0/default.jpg
iiif.static: /tmp/iiif / 1c7af029-cb44-48e2-ae10-0968a6f28dae/full/5,3 -> 1c7af029-cb44-48e2-ae10-0968a6f28dae/full/5,
iiif.static: /tmp/iiif / 1c7af029-cb44-48e2-ae10-0968a6f28dae/full/3,/0/default.jpg
iiif.static: /tmp/iiif / 1c7af029-cb44-48e2-ae10-0968a6f28dae/full/3,1 -> 1c7af029-cb44-48e2-ae10-0968a6f28dae/full/3,
iiif.static: /tmp/iiif / 1c7af029-cb44-48e2-ae10-0968a6f28dae/full/1,/0/default.jpg - zero size, skipped
iiif.static: /tmp/iiif / 1c7af029-cb44-48e2-ae10-0968a6f28dae/info.json
where 1,
was skipped, but still there is a dir:
(py3)simeon@RottenApple iiif>tree /tmp/iiif/1c7af029-cb44-48e2-ae10-0968a6f28dae/full
/tmp/iiif/1c7af029-cb44-48e2-ae10-0968a6f28dae/full
βββ 1,
βΒ Β βββ 0
βββ 11,
βΒ Β βββ 0
βΒ Β βββ default.jpg
βββ 11,5 -> 11,
βββ 175,
βΒ Β βββ 0
βΒ Β βββ default.jpg
βββ 175,84 -> 175,
βββ 22,
βΒ Β βββ 0
βΒ Β βββ default.jpg
βββ 22,10 -> 22,
βββ 3,
βΒ Β βββ 0
βΒ Β βββ default.jpg
βββ 3,1 -> 3,
βββ 351,
βΒ Β βββ 0
βΒ Β βββ default.jpg
βββ 351,168 -> 351,
βββ 44,
βΒ Β βββ 0
βΒ Β βββ default.jpg
βββ 44,21 -> 44,
βββ 5,
βΒ Β βββ 0
βΒ Β βββ default.jpg
βββ 5,3 -> 5,
βββ 88,
βΒ Β βββ 0
βΒ Β βββ default.jpg
βββ 88,42 -> 88,
26 directories, 8 files
I've deployed this as the image reference implementation but when you get the info.json the @id
of the image starts with http://resync.library.cornell.edu/iiif_auth_test
. Is there a way to set this either as part of the deployment or as a parameter? You can see an example of this by going to:
http://iiif.io/api/image/2.1/example/reference/detail/info.json
Ideally this would be either:
I see the comment in the code about taking it from the WSGI but that won't work in this case as its running from image-api.iiif.io and proxied through the iiif.io website. Unless I can set a variable in the WSGI config file.
For info this is the docker build file:
https://github.com/IIIF/docker-image-api-reference/blob/master/Dockerfile
zimeon/iiif version 1.0.6
Just a minor issue on the Seadragon demo. I manually installed the iiif package on Windows with Python 3.6. On running the demo it appeared the Seadragon elements were failing with a 404 exception.
Manually checking iiif\demo-static suggested the target seadragon200 was in fact a file rather than a folder. Copied in a new version of Seadragon and renamed the folder as seadragon200 and the demo functioned correctly. Possibly linked directories were used in the repository version.
We know what we calculate so they should be listed (rather than simply targetting OSD and relying on out-of-band understanding of OSD).
Looking here. If I understand the approach, you're using the original width and height to make the scale as accurate as possible, but piping w
and h
through a set of conditions to arrive at the largest possible size within the protential max_*
constraints. So don't you want to do:
scale = float(self.max_width) / float(self.width)
w = int(w * scale + 0.5) # instead of self.width
h = int(h * scale + 0.5) # instead of self.height
I'm looking to steal this :magpie:, so would love to know if I'm missing something.
Reported by @jgreidy. Image http://stor.artstor.org/stor/41f7f2fd-4df0-4d02-a056-ac0f584be3f7 is not handled well. "It looks like only the blackest black pixels came through."
The test image testimages/test1_224x167.png seems to be a scaled version of the spec image, not 224x167 and RGBA not RGB:
> file testimages/test1_224x167.png
testimages/test1_224x167.png: PNG image data, 605 x 451, 8-bit/color RGBA, non-interlaced
Not sure why it was included, PIL has problems with it and I don't see any reference to it so think we should remove.
Have old note to check to see that these are handled correctly, not sure what that means....
Current implementation will accept and ignore caret (^) prefex for size in API versions before 2.1. It should instead throw a helpful error.
While working with iiif_static.py to generate tiles, @jgreidy ran into a DecompressionBombWarning with a large image:
Create directories.
/tmp/image-to-iiif-s3/78/203936
Local copy of image.
Making iiif tiles.
/usr/lib64/python2.6/site-packages/PIL/Image.py:2261: DecompressionBombWarning: Image size (166109750 pixels) exceeds limit of 89478485 pixels, could be decompression bomb DOS attack. DecompressionBombWarning)
The code still seemed to work and produce good tiles
Once again I find myself with a good use case for static iiif image content and I'm so glad the spec supports it and this library exists.
I'm wondering if there's a way to generate only the tiles that don't already exist? Specific use case is if I need to add a new extra size across a collection of images βΒ I only want to generate the new one, not regenerate all the others. Maybe that logic could apply to any existing tiles by default? I don't have a sense of how much that would save in terms of processing time.
If it matters, I'm calling from python directly rather than using the command line script (modeled on what the script does). I may be able to contribute code if you're open to a PR and can suggest how this should be done.
Need to have a good set of tests verifying that the algorithms are correctly followed.
Image APi v3.0 is currently still in draft status (http://prezi3.iiif.io/api/image/3.0/), work to support it is in branch version3
.
See http://iiif.io/api/auth/0.9/#a-implementation-notes, we recommend against it:
Implementations must not reuse the access cookie value as the access token value, as it could be copied across domains when the access token is obtained from a malicious client.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
π Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. πππ
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google β€οΈ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.