Code Monkey home page Code Monkey logo

action-net's Introduction

ACTION-Net

Official implementation of ACTION-Net: Multipath Excitation for Action Recognition (CVPR'21)

By Zhengwei Wang, Qi She and Aljosa Smolic

Getting Started

  • EgoGesture data folder structure
|-frames
|---Subject01
|------Scene1
|---------Color1
|------------rgb1
|---------------000001.jpg
......
|-labels
|---Subject01
|------Scene1
|---------Group1.csv
......
  • Something-Something V2
|-frames
|---1
|------000001.jpg
|------000002.jpg
|------000003.jpg
......
  • Jester
|-frames
|---1
|------000001.jpg
|------000002.jpg
|------000003.jpg
......

Requirements

Provided in the action.Dockerfile

Annotation files

Annotation files are at this link. Please follow the annotation files to construct the frame path.

Usage

sh train_ego_8f.sh 0,1,2,3 if you use four gpus

Acknowledgment

Our codes are built based on previous repos TSN, TSM and TEA

Pretrained models

Currently, we do not provide the pretrained models since we reconstruct the structure and rename our modules of ACTION for public release. It should be able to get the similar performance indicated in the paper using the codes provided above.

(Update)

EgoGesture using 8f

Jester using 8f

Citation

If you find our work useful in your research, please cite:

@InProceedings{Wang_2021_CVPR,
author = {Wang, Zhengwei and She, Qi and Smolic, Aljosa},
title = {ACTION-Net: Multipath Excitation for Action Recognition},
booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
month = {June},
year = {2021}
}

action-net's People

Contributors

villawang avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

action-net's Issues

How to set the learning rate ?

Hello, I would like to know what is the basis for your fine-tuning of the learning rate ?
The experiment ?
Each adjustment of a parameter, an experiment is performed ?

return [
            {'params': first_conv_weight, 'lr_mult': 1, 'decay_mult': 1, 'name': "first_conv_weight"},
            {'params': first_conv_bias, 'lr_mult': 2, 'decay_mult': 0, 'name': "first_conv_bias"},
            {'params': normal_weight, 'lr_mult': 1, 'decay_mult': 1, 'name': "normal_weight"},
            {'params': normal_bias, 'lr_mult': 2, 'decay_mult': 0, 'name': "normal_bias"},
            {'params': bn, 'lr_mult': 1, 'decay_mult': 0, 'name': "BN scale/shift"},
            {'params': custom_weight, 'lr_mult': 1, 'decay_mult': 1, 'name': "custom_weight"},
            {'params': custom_bn, 'lr_mult': 1, 'decay_mult': 0, 'name': "custom_bn"},
            # for fc
            {'params': lr5_weight, 'lr_mult': 5, 'decay_mult': 1, 'name': "lr5_weight"},
            {'params': lr10_bias, 'lr_mult': 10, 'decay_mult': 0, 'name': "lr10_bias"},
        ]

`

`

model.py file function import error

In the models.py file under models line 132 'from models.temporal_shift_res2net import make_temporal_shift' and line 137 'from ops.non_local import make_non_local' error. Where should I import from

About the use of TSM in your work.

Thank you for your work. After I checked the source code, I have some questions. In model.py, I see that you use the shift operation of TSM model, but it is not explained in the paper. Is the test result obtained by adding the ACTION module you proposed to the TSM model?
Thanks for your reply.

Problems with PKL file generation

Hi, I'm a newbie. I would like to ask what is the code on the Jester dataset, something-something V2 dataset and generating the training, testing, and validating PKL files?

How to resume training from checkpoint?

Hi, thank you for your great work.
I tried to reproduce your work, but the machine was shut down.
So, I tried to resume the training process, but the epoch started at 0.
Is there any way to resume my training process from the checkpoint? (the epoch should start at the previous state)

Question of the paper

Hello! I want to ask a question about the paper.

捕获

Do you directly use the result of C3D:Resnext101 in their paper or train the model by yourself?

Thanks!

hyperparameters of 16 frames training.

Hello, I follow your suggestion and this is my hyperparameters of 16 frames training.

python3 train.py --is_train --is_shift \
--dataset EgoGesture --clip_len 16 \
--shift_div 8 --wd 1e-5 --dropout 0 \
--cuda_id $cuda_id --batch_size 16 --lr_steps 10 15 20 \
--lr 25e-4 --base_model resnet50 --epochs 25

The val accuracy is quiet lower than 8 frames result,which is 91.7 and 8 frames is 94.4.
I really want to reproduce the result of 16 frames in Egogesture. Is there any hyperparameter I need to change?

The result of CAM

Hi! Thank you for your wonderful project! Could you release the code for the CAM picture? I want to learn more about visualization.

Can you provide the code for generating pkl-file?

Thank you very much for providing the source code of this project, I want to use this project to experiment on the Something-Something V1, but the annotation file of the Something-Something V1 is in csv format, I am having difficulty with how to generate the pkl-file. Can you provide the code to generate the pkl-file or the pkl-file about the Something-Something V1. Thanks.

test accuracy

Hi, I have downloaded your jester-resnet pretrained model and jester annotations. I find the lables in test.pkl are all 0. When I run the test_jester_8f.sh, I get top1 =0, top5 = 100 witch mode in test.py is 'test' and use the jester_annotation test.pkl.
image

image

About Visualization

Hello, after reading your code and paper, I am very interested in the visualization part of your model(Figure 1.). Your input is five-dimensional data which include time dimension. How do you call the CAM module? Or could open source the code about CAM you have adjusted, thanks a lot.

Question about TSM result

In Jester dataset, TSM result in Table 1 of your paper is 94.4 (ResNet50 8frame 3crop 10clip),
But TSM result in origin paper(Table 1) is 97.0 (ResNet50 8frame full-resolution 2clips ).
I test the TSM by myself from the TSM official public code and it can also reach 97.0.
How cause this Gap between 94.4 and 97.0?

Pretrained models

could you give me the Pretrained models about mobileV2net in ssV2 for train

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.