Dream Art

Dream Art combines inceptionism/DeepDream with "[A Neural Algorithm of Artistic Style][neural-art-arxiv]." The following examples were created with create-outputs.sh and show an input content image followed by the DeepDream content image, and then pairs of input artistic styles and the DeepDream content image with the style applied. See the gallery for more examples.

Tubingen

Original

DeepDream

Escher's "Hand with Reflecting Sphere"

DeepDream with Escher's Style

Golden Gate

Original

DeepDream

Kahlo's Self Portrait

DeepDream with Kahlo's Style

Brad Pitt

Original

DeepDream

TODO

TODO

DeepDream with TODO

How can I use this on my images without downloading anything?

Tweet the style and source image to @brandondamos with the hashtag #DreamArtRequest and I'll manually generate images for you. If there's enough interest, I'll create a bot to automatically process images with #DreamArtRequest on my server.

Inceptionism (DeepDream)

Inceptionism is introduced in Google Research's blog post and visualizes an artificial neural network by enhancing input images to elicit an interpretation. The following example from the blog post. See the original gallery or #DeepDream for more examples.

The current implementations are google/deepdream which uses Caffe and eladhoffer/DeepDream.torch which uses Torch.

A Neural Algorithm of Artistic Style

'[A Neural Algorthm of Artistic Style][neural-art-arxiv]' is a paper by Leon Gatys, Alexander Ecker, and Matthias Bethge released as a preprint on August 26, 2015. The current implementations are [jcjohnson/neural-style][neural-style] and [kaishengtai/neuralart][neural-art], which both use Torch.

Can't I just run the implementations separately? What does this repository provide?

Nontrivial toolchains and system configuration make the barrier to running current implementations high. This repository lowers the barrier by combining DeepDream.torch with [neural-style][neural-style] into Torch applications application that share the same model and run on the CPU and GPU.

Ideally, both of these implementations should ship on luarocks and provide a command-line interface and library that nicely load the models. When they do, this repository will be obsolete, but until then, this repository glues them together.

Setup

The following is from [jcjohnson/neural-style][neural-style].

Dependencies:

• torch7
• loadcaffe

Optional dependencies:

• CUDA 6.5+
• cudnn.torch

NOTE: If your machine does not have CUDA installed, then you may need to install loadcaffe manually like this:

git clone https://github.com/szagoruyko/loadcaffe.git
# Edit the file loadcaffe/loadcaffe-1.0-0.rockspec
# Delete lines 21 and 22 that mention cunn and inn
luarocks install loadcaffe/loadcaffe-1.0-0.rockspec

After installing dependencies, you'll need to run the following script to download the VGG model:

sh models/download_models.sh

This will download the original VGG-19 model. Leon Gatys has graciously provided the modified version of the VGG-19 model that was used in their paper; this will also be downloaded. By default the original VGG-19 model is used.

Usage

See th deepdream.lua -help and th neural-style.lua -help for the most updated docs.

dream-art(master*)$ ./deepdream.lua -help
-content_image Content target image [examples/inputs/tubingen.jpg]
-gpu           Zero-indexed ID of the GPU to use; for CPU mode set -gpu = -1 [0]
-num_iter      [100]
-num_octave    [8]
-octave_scale  [1.4]
-end_layer     [32]
-clip          [true]
-proto_file    [models/VGG_ILSVRC_19_layers_deploy.prototxt]
-model_file    [models/VGG_ILSVRC_19_layers.caffemodel]
-backend       nn|cudnn [nn]
-output_image  [out.png]

dream-art(master*)$ ./neural-style.lua -help
-style_image    Style target image [examples/inputs/seated-nude.jpg]
-content_image  Content target image [examples/inputs/tubingen.jpg]
-image_size     Maximum height / width of generated image [512]
-gpu            Zero-indexed ID of the GPU to use; for CPU mode set -gpu = -1 [0]
-content_weight [5]
-style_weight   [100]
-tv_weight      [0.001]
-num_iterations [1000]
-init           random|image [random]
-print_iter     [50]
-save_iter      [100]
-output_image   [out.png]
-style_scale    [1]
-pooling        max|avg [max]
-proto_file     [models/VGG_ILSVRC_19_layers_deploy.prototxt]
-model_file     [models/VGG_ILSVRC_19_layers.caffemodel]
-backend        nn|cudnn [nn]

Other Q&A

• How long does it take to process a single image? About 20 minutes with a Tesla K40.
• How much memory does this use? This depends on the neural network model and image size. On average, 4GB.

Licensing

All portions are MIT licensed by Brandon Amos unless otherwise noted.

This project uses and modifies the following open source projects and resources. Modifications remain under the original license.

[neural-art-arxiv]: http://arxiv.org/abs/1508.06576) [neural-style]: https://github.com/jcjohnson/neural-style [neural-art]: https://github.com/kaishengtai/neuralart