Comments (17)
from kinetics-i3d.
Ping! Any update on this?
I found that the pretrained model work better with the flow images which are extracted after resizing the rgb frames.
And I used OpenCV3.3 (or 4.0 may work) instead of 3.4 since I found there some difference for the cv::cuda::OpticalFlowDual_TVL1.
I have got compatible accuracy for the fusion results while the flow results is still slightly worse.
from kinetics-i3d.
Ping! Any update on this?
from kinetics-i3d.
Thank you very much for the feedback. This is helpful. I'll look into that.
from kinetics-i3d.
Has anyone tried calculating optical flow using python opencv? I can't seem to get good results with that preprocessing, but might also be my lack of understing about parameters to use. I'm using opencv 4.1.0:
optical_flow = cv2.optflow.createOptFlow_DualTVL1()
flow_frame = optical_flow.calc(prev, curr, None)
flow_frame = np.clip(flow_frame, -20, 20)
flow_frame = flow_frame / 20.0
Thanks for any comments!
from kinetics-i3d.
Actually, the code I used above works fine and produces good results on the example vid. But would still be nice to get pointers if this is missing something from the original preprocessing. Thanks
from kinetics-i3d.
Actually, the code I used above works fine and produces good results on the example vid. But would still be nice to get pointers if this is missing something from the original preprocessing. Thanks
I think your code is correct. But the python interface is too slow. Do you have any idea how to speedup? Actually, I used the C++ interface of OpenCV for flow extraction.
from kinetics-i3d.
Yes, it's slow, on my desktop flow calc runs at about 4 fps. I just wanted to reproduce for now. Don't know if it's possible to bring it up to 25 fps with Python - I'd guess that it isn't. Is the speed fine with C++?
from kinetics-i3d.
Hello, does anyone know how the frame sampling was done? Is it just nearest sampling?
from kinetics-i3d.
Yes, it's slow, on my desktop flow calc runs at about 4 fps. I just wanted to reproduce for now. Don't know if it's possible to bring it up to 25 fps with Python - I'd guess that it isn't. Is the speed fine with C++?
it is fine with C++. it runs at about 40~50fps on 2080ti with the opencv cuda interface.
from kinetics-i3d.
Hello, does anyone know how the frame sampling was done? Is it just nearest sampling?
Hi do you have any ideas? I am also wondering how the video resampling is done...
from kinetics-i3d.
Just to reply to the resampling question: This is what I did to my videos which were originally 1280x720 and 30fps (using ffmpeg on ubuntu command line):
ffmpeg -y -r 30 -i input.avi -r 25 -filter:v scale=456x256 -sws_flags bilinear output.avi
Output of this should be a bilinearly interpolated video with 25fps and smaller side of video at 256px, as described in the paper and/or README file.
from kinetics-i3d.
Hi Joao,
the preprocessing code has been released as part of mediapipe, see here:
https://github.com/google/mediapipe/tree/master/mediapipe/examples/desktop/media_sequence
If I follow the steps and use kinetics_dataset.py on v_CricketShot_g04_c01,avi, I do not get rgb and flow files. Please do elaborate on how to preprocess the avi file to generate the rgb and flow data.
Thanks for your help
from kinetics-i3d.
from kinetics-i3d.
Thanks Joao,
Just to clarify, I am following the steps outlined under "custom videos in the Kinetics format". I change VIDEO_PATH to point to the avi, build the media_sequence_demo and run kinetics_dataset.py. I do see an output file kinetics_700_custom_25fps_rgb_flow-00000-of-00001. I am not sure about the next step.
from kinetics-i3d.
I was hoping that it will generate the rgb & flow files in a format that I can then use as an input to evaluate_sample. Not sure if that is the intent of the release of the preprocessing code.
from kinetics-i3d.
from kinetics-i3d.
Related Issues (20)
- Optical flow rescaling HOT 5
- How to create a custom action recognition model HOT 2
- Inflating pre-trained models HOT 1
- Trying to create frozen graph, so i can convert it into tflite for android
- Training with different architectures
- customize actions class in the model HOT 1
- offline usage
- rgb.npy and flow.npy HOT 3
- training from scratch HOT 2
- which one is the checkpoint? HOT 5
- The problem of receptive field in I3D paper
- missing videos
- Does the video need to be cropped? HOT 11
- Calculation of TV L1 flow HOT 1
- dependencies issues HOT 1
- Struggling to learn using Opt. Flow HOT 2
- Incompatibility issues
- Run time of I3D on edge decives
- Is there Model File(.pth or .pt) that pretrained with Imagenet+Kinetics?
- I found the pth file which pretrained on Kinetics400 HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from kinetics-i3d.