add estimate of gpu memory needed to guide about neural-style-transfer HOT 6 CLOSED

@7yl4r Due to the use of L-BFGS, very large images of the size 1000x1000 (for a square image) is too large, even for a 12 GB gpu ram card.

I suggest using Theano backend, and enable garbage collection. This should allow for slightly larger images to be styled, in exchange for speed. Even so, there is an upper limit to how large the image can be.

I found that for a 4 GB GPU, the upper limit on theano is 740x740 image. Tensorflow does not have garbage collection, I think. That may be the reason that it hits an OOM even at 700x700.

Even with a 12 GB card, I have not heard of anyone using L-BFGS for images larger than 1200x1200. However, with the adam optimizer, it is possible to use less memory to scale larger images, in exchange for quality. This repo doesn't support Adam optimizer, but if anyone wishes to add it, they can create a PR.

EDIT:

I really do suggest just using a 512 as the image size, and then upscale via a convolutional upscaling algorithm. waifu2x-caffe is added as a link at the bottom of the repo.

My workflow is basically -

• Create image at 512 image size
• Upscale 4x to 2048 px
• Apply denoising or sharpening if image quality is drastically poor, else leave it alone.

from neural-style-transfer.

How were the examples in the readme created? The first one is 2560x1600.

I think I'm misunderstanding something here.

from neural-style-transfer.

I added it to the edit, but to reiterate :

I really do suggest just using a 512 as the image size, and then upscale via a convolutional upscaling algorithm. waifu2x-caffe is added as a link at the bottom of the repo.

My workflow is basically -

• Create image at 512 image size
• Upscale 4x to 2048 px
• Apply denoising or sharpening if image quality is drastically poor, else leave it alone.

As you can see from the examples, waifu2x-caffe is a generally very useful tool to upscale stylized images. All the images in the readme were simply upscaled after generation. No cleaning or denoising applied.

from neural-style-transfer.

Interesting. I didn't realize you could achieve that quality upscaling.

Do you have any other tips for someone who is trying to generate the highest possible resolution image? Based on your suggestions my new plan is to see how large I can get using the theano backend and a cpu-capable machine to avoid GPU OOM errors and then upscale the output.

from neural-style-transfer.

@7yl4r CPU with this algorithm is a non-starter. It required over 4000 seconds per iteration on an i7 8 core machine (alienware laptop, so I can't speak for huge workstations with hundreds of cores and tons of RAM).

WIth Theano and garbage collection, and a 12 GB card, I have heard that image size of 1200x1200 can be achieved. Do note that this repo does not restrict to square images, so you are considering min image size = 1200 at this point.

So say you have a 1080p image then you will probably get a generated image of 1200 x 1920.

I can suggest a few "cheats", although this is only good for non academic purposes :
If you have Photoshop installed, go to Nik software and download the Photoshop addons. One of them called the Image Sharpening tools (Output sharpening) which allows you to sharpen the image which has been upscaled. You can also modify the "structure" parameter, which can improve the quality.

from neural-style-transfer.

I had run this implementation: https://colab.research.google.com/drive/1rDTE8Kssqdgc-evxO2JSkAE1YLIFRjwS (it is L-BFGS).
For 960x1280 like no prob. Same size gives out of memory error here.

from neural-style-transfer.