brycedrennan / eulerian-magnification Goto Github PK
View Code? Open in Web Editor NEWA tool to discover hidden variation in video.
License: MIT License
A tool to discover hidden variation in video.
License: MIT License
I was trying to run show_frequencies(vid_data) as described but had two issues:
Hi, @brycedrennan
As a part of an academic project, we want to use this algorithm to detect respiration of babies and identifying the pattern of respiration in real-time with a help of simple mobile camera.This can be useful to various hospitals to replace the sensors used on infant body.
For now, the problem we are facing is the processing time this algorithm requires when a real-time video stream is passed into this.
I hope you can help us in figuring out the best approach that can give us real-time results.
Thanks :)
Implement improvements described at http://people.csail.mit.edu/nwadhwa/phase-video/
This repository has no license. Am I allowed to relicense it to some FOSS license GPL/BSD (I don't really care) to fork it and base some other functionality on it?
I received the following report via email:
"Thanks for all your work you have done on this code and permitting the public to use and help out with its development. I wanted to let you know that I found a small glitch that is causing problems implementing the function. It may be quite a simple fix. I noticed the pip package manager wouldn't recognize the 'eulerian_magnification' as currently spelled, but only as 'eularian_magnification'. I used the tarball to install the package on my linux machine, but when I import the function in python using the as-installed misspelled version everything goes well except when I use anything related to 'em.function_name' as the code appears to expect the correct spelling."
Hi is it possible to use this on a real time video stream?
Hello,
When using the module downloaded from pip, I get this error when trying out the sample code from the readme.
My video is an .mp4 file taken from the paper.
Hi,
I really don't understand what is the output of the algorithm? It seems to that the original video is not modified after the run and there is no new video created in my work directory. Maybe i need to do some other steps in order to receive the output?
Thanks!
In Readme.md
The spelling of eularian
is wrong.
Correct version :
import eularian_magnification as em
em.eularian_magnification('videos/baby.mp4', image_processing='gaussian', pyramid_levels=3, freq_min=50.0 / 60.0, freq_max=1.0, amplification=50)
With a 35" video (58MB), I get a frame_count of 1179, a height of 1080 and width of 1920.
Since you're creating a 4d matrix of uint8 zeros, this means:
In[1] > print "{} TB required!".format(1179 * 1080 * 1920 * 3 * 256.0 / 1024**4)
1.70765519142 TB required!
I would need to degrade my video to 320*180 in order to have a feasible (48.5GB) size, and I assume the result will be disastrous.
Do you have an idea for a better way to handle this magnification, which doesn't require 30872 times more memory than the input video size ?
Thanks for your suggestions
Hi
Just to explain, I'm very new to Python, and somewhat new to Ubuntu and OpenCV.
I've coded before, so I caught on quickly how the code works and began to implement it, but even when I run the example code, I seem to be getting a flash of bright light that complete blinds the image, followed by complete darkness, and so on.
I'm not too sure what the problem is, as I roughly understand the code and the setup. I thought it may have been the setup and that I don't have the correct versions installed or something, but figured that I would have gotten an error by now.
Also, with regards to the FFT plot displayed by the "show_frequencies" function --- What is this scale related to? I'm using the breathing baby video, and have my FFT spiking at 9? is that 9/30 Hz? I notice you're using a sampling time related to the fps (frames per second?), which is output in the Python Shell when running the "show_frequencies" function.
Any help would be appreciated.
Thank you very much.
Regards.
When I was reviewing the code for the gaussian_video function:
def gaussian_video(video, shrink_multiple):
"""Create a gaussian representation of a video"""
vid_data = None
for x in range(0, video.shape[0]):
frame = video[x]
gauss_copy = numpy.ndarray(shape=frame.shape, dtype="float")
gauss_copy[:] = frame
for i in range(shrink_multiple):
gauss_copy = cv2.pyrDown(gauss_copy)
if x == 0:
vid_data = numpy.zeros((video.shape[0], gauss_copy.shape[0], gauss_copy.shape[1], 3))
vid_data[x] = gauss_copy
return vid_data
It seems like it's just downsampling and overwriting gauss_copy 4 times (lines 106-107). I thought the algorithm saved all levels of the pyramid instead of just the lowest one?
The AVI written out is 8 kb and appears empty. The code works with the test videos from the MIT site. Those videos are:
960 x 624, H.264, and color profile SD (6-1-6) (from Get Info on the file on OS X).
My videos are
800 x 532, H.264 AAC, and color profile HD (1-1-1).
Using the 'file' command, both report "SO Media, MPEG v4 system, version 2".
I'm not sure what else might be causing the problem. Do you have any ideas why the output might be empty, or what else to check to see how the videos are different?
import eulerian_magnification as em
vid, fps = load_video_float(source_path)
em.eulerian_magnification(vid, fps,
freq_min=50.0 / 60.0,
freq_max=1.0,
amplification=50,
pyramid_levels=3
)
When I try to call the top function just like the usage following:
em.eulerian_magnification('media/face.mp4', image_processing='gaussian', pyramid_levels=3, freq_min=50.0 / 60.0, freq_max=1.0, amplification=50);
I find that the first parameter doesn't make sense because it's a string, not a video object. And in the eulerian_magnification function, there is no preprocess about the first parameter which should obtain an object like a video. So I get a error when the program comes to 'for frame_number, frame in enumerate(video)' in the pyramid.py(line 37) because the first parameter isn't an iterable object. What should I do solve this question?
Installed opencv by tapping hombrew/science
and doing brew install opencv
, then did pip install matplotlib scipy numpy
.
After opening up the file by going
python -i eulerian_magnify.py
I entered
eulerian_magnification('movie.mov', image_processing='gaussian', freq_min=50.0 / 60.0, freq_max=1.0, amplification=50)
and got
Loading will.mov
WARNING: Couldn't read movie file will.mov
Applying bandpass between 0.833333333333 and 1.0 Hz
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "eulerian_magnify.py", line 19, in eulerian_magnification
vid_data = temporal_bandpass_filter(vid_data, fps, freq_min=freq_min, freq_max=freq_max)
File "eulerian_magnify.py", line 61, in temporal_bandpass_filter
fft = scipy.fftpack.fft(data, axis=axis)
File "/usr/local/lib/python2.7/site-packages/scipy/fftpack/basic.py", line 230, in fft
n = tmp.shape[axis]
IndexError: tuple index out of range
It seems like opencv's binaries are already in the path.
I tried running your example.py
file with the face.mp4
file from http://people.csail.mit.edu/mrub/vidmag/. The show_frequencies
appears to work, but eulerian_magnification
produced a bad video file. VLC won't play it, and it is only 414 bytes.
Here is the log:
Loading media/face.mp4
/Users/Andreas/Sites/lies/eulerian_magnify.py:87: FutureWarning: comparison to `None` will result in an elementwise object comparison in the future.
if frame == None or x >= frame_count:
29
/usr/local/lib/python2.7/site-packages/numpy/core/numeric.py:462: ComplexWarning: Casting complex values to real discards the imaginary part
return array(a, dtype, copy=False, order=order)
Loading /Users/Andreas/Sites/lies/media/face.mp4
Applying bandpass between 0.833333333333 and 1.0 Hz
Amplifying signal by factor of 50
/Users/Andreas/Sites/lies/eulerian_magnify.py:147: ComplexWarning: Casting complex values to real discards the imaginary part
img[:] = g_video[x]
Still haven't had this working but I've noticed that some of the code is written eulErian-magnification while other parts say eulArian-Magnification. I'm new to python and coding so this might be a silly question: Do I need to CD to /media/face.mp4 to run the script? I keep getting a gaussian image error
eulerian_magnification() got an unexpected keyword argument 'image_processing'
I cant get it to work:
Loading eulerian_source_videos/baby.mp4
Traceback (most recent call last):
File "replicate_study.py", line 84, in <module>
replicate_study()
File "replicate_study.py", line 75, in replicate_study
pyramid_levels=pyramid_levels
File "/home/calle/eulerian-magnification/eulerian_magnification/base.py", line 27, in eulerian_magnification
vid_data = collapse_laplacian_video_pyramid(vid_pyramid)
File "/home/calle/eulerian-magnification/eulerian_magnification/pyramid.py", line 68, in collapse_laplacian_video_pyramid
return pyramid[0]
IndexError: list index out of range
Here is the exact way i tried to use it from a fresh ubuntu 1604 vm.
6 sudo apt install python3-pip
7 sudo apt install git
12 git clone https://github.com/brycedrennan/eulerian-magnification
13 cd eulerian-magnification/
16 pip3 install -r requirements.txt
19 pip3 install opencv
22 pip3 install matplotlib
25 sudo apt-get install python3-tk
28 sudo python3 setup.py install
31 pip3 install numpy scipy matplotlib
32 python3 replicate_study.py
33 history
Output files all seem to have zero size. I get this error when running the code:
eulerian_magnify.py:144: ComplexWarning: Casting complex values to real discards the imaginary part
img[:] = g_video[x]
System: OS X, Enthought Canopy Python, Python 2.7.3, 64-bit, OpenCV 2.4.2
Video (*_magnified.avi) size is 28 KB.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.