Code Monkey home page Code Monkey logo

deepdreamanim's Introduction

DeepDream Animator

This tool helps to create animations with deepdream. Extract frames from videos, process them with deepdream and then output as new video file. Frame blending option is provided, to ensure "stable" dreams across frames. A preview function make rapid iterations possible. Optionally guided dreaming & optical flow can be used.

deepdreamanim

How to

  1. Extract Video
    python dreamer.py --input myvideo/video.mp4 --output myvideo --extract 1

  2. Run DeepDream
    python dreamer.py --input myvideo --output myvideo/frames

  3. Create Video
    python dreamer.py --input myvideo/frames --output myvideo/deepdreamvideo.mp4 --create 1

(change "myvideo" to your directory/file name)

(change the path of the caffe model inside dreamer.py to where your model is )

deepdreamanim

Settings

Use Optical Flow & Guided Dreams & GPU
python dreamer.py --input myvideo --output myvideo/frames --octaves 4 --octavescale 1 --iterations 10 --jitter 32 --zoom 1 --stepsize 1.5 --blend 0 --layers inception_3b/output --gpu 1 --flow 1 --guide guide/flowers.png

Create a preview (currently does not work with flow)
python dreamer.py --input myvideo --output myvideo/frames --preview 600

Tweak settings
python dreamer.py --input myvideo --output myvideo/frames --preview 600 --octaves 4 --octavescale 1.4 --iterations 10 --jitter 32 --zoom 1 --stepsize 1.5 --blend 0.5 --layers inception_3a/output inception_3b/output

(Preview changes Images to width of choice, original files not changed)

(Try using multiple layers, it will cycle through them from frame to frame.)

Batch Processing

Use the above commands and stack them by putting a ";" inbetween commands.
python dreamer.py --input myvideo --output myvideo/frames;python dreamer.py --input myvideo2 --output myvideo2/frames

deepdreamanim

Examples

DeepDream Music Video Musiv Video

Optical flow + Guided Dreaming tests test

Creative Request

It would be very helpful for other deepdream researchers, if you could include the used parameters in the description of your youtube videos.

Requirements

  • Python
  • Caffe (and other deepdream dependencies)
  • FFMPEG
  • CV2 (if you use optical flow)

Audio

The tool currently does not handle audio. Check out DeepDreamVideo by @graphific

deepdreamanim

Credits

Samim | Samim.io | @samim

deepdreamanim's People

Contributors

samim23 avatar svelle avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

deepdreamanim's Issues

Processing frames in random order

When I run the image processing pass, the script seems to pull random frames from the frame directory instead of processing them sequentially:

Frame 158 of 1866
Frame Time: 1s
Estimated Total Time Remaining: 1708s (0:28:28)


Processing: myvideo/image-000659.png
0 0 inception_3b/output (360, 540, 3)
0 1 inception_3b/output (360, 540, 3)
...
3 9 inception_3b/output (360, 540, 3)


Saving Image As: myvideo/frames/frame_000159.png

this gives me an unwatchable mess when I try to re-stitch the frames.

strange error (windows 10)

I keep getting this error when trying to use it

(caffe) C:\Users\root\DeepDreamAnim>python dreamer.py --input myvideo --output myvideo/frames --preview 600
Traceback (most recent call last):
  File "dreamer.py", line 339, in <module>
    args.zoom, args.stepsize, args.blend, args.layers, args.guide, args.gpu, args.flow)
  File "dreamer.py", line 143, in main
    text_format.Merge(open(net_fn).read(), model)
IOError: [Errno 2] No such file or directory: '../../caffe/models/bvlc_googlenet/deploy.prototxt'

Error when trying to process frames

First up, big fan of your work, and I'm very excited about getting this working.

When I try to actually process the frames (I believe the command line should look something like) "C:\Anaconda>python dreamer.py --input frames\bp --output pic" it goes through some parts of the operation and then fails.

Traceback (most recent call last):
File "dreamer.py", line 339, in
args.zoom, args.stepsize, args.blend, args.layers, args.guide, args.gpu, args.flow)
File "dreamer.py", line 166, in main
img = PIL.Image.open(inputdir + '/' + vids[0])
IndexError: list index out of range

As I understand it, "indexerror: out of range" is that it usually is in reference to a line that isn't there, but line 339 is totally there, and that appears to be the bug reported in #3 , and the closing comment says that the index error is an image processing the last frame. Which is odd, because it isn't processing any frames at the moment.

The research I've done about the pil error has not turned up an answer that I've been able to put to solving the problem.
I've been talking to a friend who has had success with running deepdreamanim, he said that it could be related to my caffe paths, but as far as I can tell, he and I are using the same pathing.
Lines 134-138

Loading DNN model

model_name = 'bvlc_googlenet'
model_path = 'C:/anaconda/caffe/models/' + model_name + '/'
net_fn = model_path + 'deploy.prototxt'
param_fn = model_path + 'bvlc_googlenet.caffemodel'

I've also tried adding a 0 before each file name, but have not tried the files without the zero.
Any and all help is appreciated. Thanks.

guide image issue #2

having trouble using the guide command and I always get this outcome:

Traceback (most recent call last):
  File "dreamer.py", line 339, in <module>
    args.zoom, args.stepsize, args.blend, args.layers, args.guide, args.gpu, args.flow)
  File "dreamer.py", line 173, in main
    guideimg = PIL.Image.open(inputdir + '/' + guide)
  File "C:\Users\root\Anaconda2\envs\caffe\lib\site-packages\PIL\Image.py", line 1952, in open
    fp = __builtin__.open(fp, "rb")
IOError: [Errno 22] invalid mode ('rb') or filename: 'C:\\Users\\root\\Desktop\\input/C:\\Users\\root\\Desktop\\input\\image-000406.png'

Unable to install dependencies.

Please help, I'd love to try out your software. I am currently running a conda environment, and was able to install all dependencies except for caffe.

Flow behaves additive

When flow is enabled, specially for sequences where frames are similar (slow or no camera movement, partial/slow pixel movement) every frame becomes additive. Is this the normal behavior ?

When flow is turned off, everything seems okay over a long period of time (but there is a lot of jitter for sure) but when flow is on it becomes brighter and brighter every frame unless the changes between frames are too clear.

guide image issues

Hi, thanks for the excellent script - I've got it working fine except that when I try to set a guide image this error is thrown:

File "dreamer.py", line 21, in preprocess
return np.float32(np.rollaxis(img, 2)[::-1]) - net.transformer.mean['data']
ValueError: operands could not be broadcast together with shapes (4,301,600) (3,1,1)

I've tried a variety of .png images (even trying one of the source images) but I get this error every time.

Dreaming stop, ghostly image left

I tested your setup and it is all working nicely, GPU works too which is cool.

The problem is that, the dream just stops at some point. Only after few frames, and what left is the dream for previous frames as ghost image for the following frames.
https://youtu.be/xtkG_lftImg

Any idea why this happens? Could it be because of screensaver?

Stitching

Hi. After some installation experimentation, I've managed to get all the dependencies installed under Windows 7: Anaconda, FFmpeg, CUDA, CudNN, Caffe, + all the addons etc required. I've accomplished this following tutorials which are now linked on new wiki page in the DeepDreamAnim wiki.

So far so good. I stuck the dreamer.py file in the same parent directory as caffe and have managed to get through clipping up the volcano using:

C:\deepdream\DreamAnim>python dreamer.py -i C:\frames -o C:\frames_out

and then successfully dreampt onto those frames after swapping the places of the frames to new folders with:

C:\deepdream\DreamAnim>python dreamer.py -i C:\frames -o C:\frames_out

However, the last step generates an error message:


[image2 @ 0000000002f9f5c0] Pattern type 'glob' was selected but globbing is not supported by this libavformat build
C:\frames_out/*.jpeg: Function not implemented


This really won't be an issue in terms of generating videos for me, as I'm quite comfortable using myfavoritesoftware to do the stitching (like Blender).

Is this something in the dreamer.py or simply something to do with running this under Windows?

Warm Regards!

M

P.S. Great Job!

Earlier frames making an appearance later in longer sequences.

Currently dreaming a video with 1188 frames and I'm noticing weird things once I passed 100 frames.

Everything is in order with the input frames but the dreamed frames…

10099__octaves4_iterations10_octavescale1.5_netinception_4c-output_jitter32_stepsize1.5_blend0.5_renderTime9s_filenameimage-099.jpeg
10100__octaves4_iterations10_octavescale1.5_netinception_4c-output_jitter32_stepsize1.5_blend0.5_renderTime9s_filenameimage-100.jpeg
10101__octaves4_iterations10_octavescale1.5_netinception_4c-output_jitter32_stepsize1.5_blend0.5_renderTime9s_filenameimage-1000.jpeg
10102__octaves4_iterations10_octavescale1.5_netinception_4c-output_jitter32_stepsize1.5_blend0.5_renderTime9s_filenameimage-1001.jpeg
10103__octaves4_iterations10_octavescale1.5_netinception_4c-output_jitter32_stepsize1.5_blend0.5_renderTime9s_filenameimage-1002.jpeg
10104__octaves4_iterations10_octavescale1.5_netinception_4c-output_jitter32_stepsize1.5_blend0.5_renderTime9s_filenameimage-1003.jpeg
10105__octaves4_iterations10_octavescale1.5_netinception_4c-output_jitter32_stepsize1.5_blend0.5_renderTime9s_filenameimage-1004.jpeg
10106__octaves4_iterations10_octavescale1.5_netinception_4c-output_jitter32_stepsize1.5_blend0.5_renderTime9s_filenameimage-1005.jpeg
10107__octaves4_iterations10_octavescale1.5_netinception_4c-output_jitter32_stepsize1.5_blend0.5_renderTime9s_filenameimage-1006.jpeg
10108__octaves4_iterations10_octavescale1.5_netinception_4c-output_jitter32_stepsize1.5_blend0.5_renderTime9s_filenameimage-1007.jpeg
10109__octaves4_iterations10_octavescale1.5_netinception_4c-output_jitter32_stepsize1.5_blend0.5_renderTime9s_filenameimage-1008.jpeg
10110__octaves4_iterations10_octavescale1.5_netinception_4c-output_jitter32_stepsize1.5_blend0.5_renderTime9s_filenameimage-1009.jpeg
10111__octaves4_iterations10_octavescale1.5_netinception_4c-output_jitter32_stepsize1.5_blend0.5_renderTime9s_filenameimage-101.jpeg
10112__octaves4_iterations10_octavescale1.5_netinception_4c-output_jitter32_stepsize1.5_blend0.5_renderTime9s_filenameimage-1010.jpeg
10113__octaves4_iterations10_octavescale1.5_netinception_4c-output_jitter32_stepsize1.5_blend0.5_renderTime9s_filenameimage-1011.jpeg
10114__octaves4_iterations10_octavescale1.5_netinception_4c-output_jitter32_stepsize1.5_blend0.5_renderTime9s_filenameimage-1012.jpeg
10115__octaves4_iterations10_octavescale1.5_netinception_4c-output_jitter32_stepsize1.5_blend0.5_renderTime9s_filenameimage-1013.jpeg
10116__octaves4_iterations10_octavescale1.5_netinception_4c-output_jitter32_stepsize1.5_blend0.5_renderTime9s_filenameimage-1014.jpeg
10117__octaves4_iterations10_octavescale1.5_netinception_4c-output_jitter32_stepsize1.5_blend0.5_renderTime9s_filenameimage-1015.jpeg
10118__octaves4_iterations10_octavescale1.5_netinception_4c-output_jitter32_stepsize1.5_blend0.5_renderTime9s_filenameimage-1016.jpeg
10119__octaves4_iterations10_octavescale1.5_netinception_4c-output_jitter32_stepsize1.5_blend0.5_renderTime9s_filenameimage-1017.jpeg
10120__octaves4_iterations10_octavescale1.5_netinception_4c-output_jitter32_stepsize1.5_blend0.5_renderTime9s_filenameimage-1018.jpeg
10121__octaves4_iterations10_octavescale1.5_netinception_4c-output_jitter32_stepsize1.5_blend0.5_renderTime11s_filenameimage-1019.jpeg
10122__octaves4_iterations10_octavescale1.5_netinception_4c-output_jitter32_stepsize1.5_blend0.5_renderTime10s_filenameimage-102.jpeg
10123__octaves4_iterations10_octavescale1.5_netinception_4c-output_jitter32_stepsize1.5_blend0.5_renderTime11s_filenameimage-1020.jpeg

Every tenth frame after 100 ends in 3 numbers instead of 4 and those frames contain a frame from earlier in the sequence, blended in correctly but definitely misplaced.

Get an error towards the end of the script IndexError: list index out of range

I am a bit new to this, but when I ran it, it would run through the inception layers just fine how it normally would, towards the last few lines I got the following error.
Traceback (most recent call last):
File "dreamer.py", line 254, in
main(args.input, args.output, args.preview, args.octaves, args.octavescale, args.iterations, args.jitter, args.zoom,
args.stepsize, args.blend, args.layers)
File "dreamer.py", line 169, in main
img = PIL.Image.open(inputdir+'/'+vids[0])
IndexError: list index out of range

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.