Image Resizer taking a long time with 'original' about image-resizer HOT 8 CLOSED

On a side note, I'm not sure this streaming approach is faster than if a singular GM command was built. Performance hasn't been so good compared to the old image server code so far.

from image-resizer.

What sort of setup are you seeing these long processing times on? Also what size of images are you using?

My dev setup (Macbook Air) does not show any significant slowdowns that I have seen. The typical use case at least for resizing images off of s3 was for file sizes of less than 5mb, are you processing images larger than that?

I would prefer to keep the original images being processed by the resize stream so we can auto-orient and remove the meta data. I will investigate performance though, if you could give me some parameters to test against that would be cool (image, setup etc).

In regards to streaming, much of what I did in this newer API was to cut out all the bottlenecks we were seeing previously. Writing to disk, especially on Heroku is slow and inconsistent and pushing data back to s3 was also very inconsistent. Keeping things in memory have thus far appeared to be a big win in both speed and efficiency. Using the CDN more effectively (removing the 302 redirects) was also a big step forward.

from image-resizer.

A 3mb PNG file on our dual-core VPS took about 15 seconds until I put that
code in to skip the resize stream and then it was down to about 200ms.
Interestingly, resizing down to 150px wide only took about 6 seconds. If
you email me at my username @ gmail.com I'll reply with the specific file I
used.

On Tue, Jun 24, 2014 at 11:02 AM, James Nicol [email protected]
wrote:

What sort of setup are you seeing these long processing times on? Also
what size of images are you using?

My dev setup (Macbook Air) does not show any significant slowdowns that I
have seen. The typical use case at least for resizing images off of s3 was
for file sizes of less than 5mb, are you processing images larger than that?

I would prefer to keep the original images being processed by the resize
stream so we can auto-orient and remove the meta data. I will investigate
performance though, if you could give me some parameters to test against
that would be cool (image, setup etc).

In regards to streaming, much of what I did in this newer API was to cut
out all the bottlenecks we were seeing previously. Writing to disk,
especially on Heroku is slow and inconsistent and pushing data back to s3
was also very inconsistent. Keeping things in memory have thus far appeared
to be a big win in both speed and efficiency. Using the CDN more
effectively (removing the 302 redirects) was also a big step forward.

—
Reply to this email directly or view it on GitHub
#14 (comment)
.

Jonathan

from image-resizer.

So I am still working on this but I have seen quite an improvement in using PngQuant instead of Optipng. For me the bottleneck was by far the optimize section, I was seeing things like this...

HEAD /elocal-h1000/test/sample_images/dont-version-invision.png 200 33049ms - 1.05mb
[resizer] new stream created!
[resizer] local filesystem - 4ms
[resizer] identify: no identify
[resizer] resize - 2317ms
[resizer] filter: none requested
[resizer] optimize:png - 28816ms
[resizer] original image size: 3008.838kb
[resizer] size saving: 63.26%
[resizer] checksum 274c6d9c9a44a50baa1d8a39a6287b525609fdda

PngQuant reduced the optimize time down to around 2700ms for the same request which is nice. What I dont understand yet is why the resize time is still lengthy and why that is your bottleneck.

from image-resizer.

Thanks for taking a look at this. PngQuant is lossy, so it's also not an
option for us because image fidelity is our business.

-Jonathan

On Wed, Jun 25, 2014 at 1:11 AM, James Nicol [email protected]
wrote:

So I am still working on this but I have seen quite an improvement in
using PngQuant instead of Optipng. For me the bottleneck was by far the
optimize section, I was seeing things like this...

HEAD /elocal-h1000/test/sample_images/dont-version-invision.png 200 33049ms - 1.05mb
[resizer] new stream created!
[resizer] local filesystem - 4ms
[resizer] identify: no identify
[resizer] resize - 2317ms
[resizer] filter: none requested
[resizer] optimize:png - 28816ms
[resizer] original image size: 3008.838kb
[resizer] size saving: 63.26%
[resizer] checksum 274c6d9c9a44a50baa1d8a39a6287b525609fdda

PngQuant reduced the optimize time down to around 2700ms for the same
request which is nice. What I dont understand yet is why the resize time is
still lengthy and why that is your bottleneck.

—
Reply to this email directly or view it on GitHub
#14 (comment)
.

Jonathan

from image-resizer.

@jrowny

So I tried a bunch of different things to try and smooth the original images through the resizer. I tried both returning a stream and buffer and it didnt seem to make much difference (actually very little).

Still no matter what I did the optimize step on my setup was bigger always the bottleneck. Switching to pngquant did obviously make a big difference, but given your use case precludes it I set the png optimizer to be a flag now. env.PNG_OPTIMIZER will allow you to select pngquant, optipng or null for nothing.

To skip the resize action for original images there is also a flag env.RESIZE_PROCESS_ORIGINAL which is a boolean.

I will continue to think about making this faster, but for now you should be able to configure your instance to meet your needs.

from image-resizer.

Oh all this is in v0.9.0

from image-resizer.

Thanks, I really appreciate your efforts!

On Thu, Jun 26, 2014 at 12:07 AM, James Nicol [email protected]
wrote:

Oh all this is in v0.9.0

—
Reply to this email directly or view it on GitHub
#14 (comment)
.

Jonathan

from image-resizer.