ykotseruba / jaad Goto Github PK

Hi,

Many thanks for your very useful work!

Can I ask have you performed normalization for the pose data? I have observed some pose data and found the range is between 0 and 1. So I am wondering this.

Thanks a lot again!

Bests,
Xingchen

Hallo, recently two new python files have been added, and i could not find any readme about it.
jaad_data.py and jaad_eval.py why and how to use them.

Is there a script available that converts the .vvb and .mat information to a JSON, because i find once the information related to bounding boxes are stored in JSON, it is easier to work with in python.

Thnaks,
Kalinga

when I loaded video_0002, I met some problem in information of pedestrian1. The number of pose direction(158) in appearance is not the same as number of frames in ped_annotations(192), so how can we get the correspondence between the label of pose direction and ped_annotations, in other words, what is the corresponding frame index of pose direction ?

I have trined my model using label images annotation but i have not done in videos. Is it the same way or do i need to do anything extra in tensorflow API

In video number 102 there is a pedestrian crossing the street, but two pedestrians appear on the labels, so it is a tracking error, perhaps because it disappears and reappears in the scene (in the visible field of the camera)
PS: Labels on behavior data set are almost correct, change start_frame, from 134 to 137

Tries with matlabR2013a and R2016a versions of matlab. I am getting below errors.
Which version of matlab was used for this?

load('video_0001.mat')
Error using load
Unable to read MAT-file C:\Users\kara9147\Documents\MATLAB\video_0001.mat: not
a binary MAT-file.
Try LOAD -ASCII to read as text.

Thanks,
Kalinga

Hi @ykotseruba , For my model I need to extract pedestrian trajectory files as .pkl files (eg. 00131_0_109_608b.pkl). Can you help on how to generate the pickle files from the images extracted from the videos.

Hi. Thank you for your work. I'm trying to use the interface to extract the frames from the videos, which I placed in JAAD\JAAD_clips. I'm giving the root folder (\JAAD) as the data path, but the images are not being extracted to the images folder. The images folder and sub-folders are all created normally but the frames are saved in the root folder. And only the frames for video 1 are saved, while the rest aren't.
Could this be a Windows issue?

The following error occurs when using the script to generate the image
00000.png of each folder will report an error

Hi
Do you have intrinsic and extrinsic parameters available for all the cameras used for recording? if so can you share it ?

Hi,

I'm facing some errors while trying to generate the trajectory sequences using the pedestrian behaviour annotations (branch JAAD_2.0, commit be57a06).
The error can be reproduced with the following commands:

from jaad_data import JAAD
jaad_path = '/home/vito/workspace/JAAD/'
imdb = JAAD(data_path=jaad_path)
seq = imdb.generate_data_trajectory_sequence(image_set='train', seq_type='trajectory', sample_type='beh')

and here is the output:

---------------------------------------------------------                 
Generating action sequence data   
data_split_type: default          
height_rng: [0, inf]                                                      
squarify_ratio: 0                                                
kfold_params: {'num_folds': 5, 'fold': 1}                                 
subset: default                   
seq_type: trajectory                 
random_params: {'regen_data': False, 'val_data': True, 'ratios': None}    
fstride: 1                                                                
sample_type: beh                                                          
min_track_size: 15                                                        
---------------------------------------------------------
Generating database for jaad      
Getting annotations for video_0001                                        
Getting annotations for video_0002                                        
Getting annotations for video_0003                               
...
Getting annotations for video_0346
The database is written to /home/vito/workspace/JAAD/data_cache/jaad_database.pkl
---------------------------------------------------------        
Generating trajectory data
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/vito/workspace/JAAD/jaad_data.py", line 1031, in generate_data_trajectory_sequence
    sequence = self._get_trajectories(image_set, annot_database, **params)
  File "/home/vito/workspace/JAAD/jaad_data.py", line 1097, in _get_trajectories
    if annotations[vid]['ped_annotations'][pid]['attributes']['crossing'] == -1:
KeyError: 'crossing'
>>>
>>> import pdb
>>> pdb.pm()
> /home/vito/workspace/JAAD/jaad_data.py(1097)_get_trajectories()
-> if annotations[vid]['ped_annotations'][pid]['attributes']['crossing'] == -1:
(Pdb) list
1092                    ped_ids = [[pid]] * len(boxes)
1093
1094                    if params['sample_type'] == 'all':
1095                        intent = [[0]] * len(boxes)
1096                    else:
1097 ->                     if annotations[vid]['ped_annotations'][pid]['attributes']['crossing'] == -1:
1098                            intent = [[0]] * len(boxes)
1099                        else:
1100                            intent = [[1]] * len(boxes)
1101                    center = [self._get_center(b) for b in boxes]
1102
(Pdb) vid
'video_0001'
(Pdb) pid
'0_1_1b'
(Pdb)

It looks like annotations/ and annotations_attributes/ do not match.
For example, annotations_attributes/video_0001_attributes.xml lists two pedestrians: "0_1_2b" (old_id="pedestrian2") and "0_1_3b" (old_id="pedestrian1"). However annotations/video_0001.xml includes only two pedestrians with behavior: "0_1_1b" (old_id="pedestrian1") and "0_1_3b" (old_id="pedestrian2").

I'm afraid video_0001 is not the only mismatch... from the repo hisory I guess something must have happened with commit 3f19315. If I checkout the previous commit (0e819c6) the above python commands complete succesfully.

Could you please check if everything is ok with the annotations or tell me if I am using the interface in the wrong way?

Thanks!
Vito

Hi,
Do you provide vehicle odometry information for each frame as well?

For the video_0001
From annotaion i see for frame 363
hand_gesture

and for frame 364
reaction
action
hand_gesture

I am not able to locate what is this "reaction action" in the readme. Could you please elaborate on this. How these lables would be interesting, while predicting the Intention. I would like to use LSTM on the sequence of activities, but not sure about the activites such as "reaction action".

from jaad_data import JAAD
imdb = JAAD(data_path = jaad_path)
anno = imdb._get_annotations("video_0001")
for k,v in anno["ped_annotations"]["0_1_3b"]["behavior"].items():
print (k)

Outputs:
cross
reaction
look
nod
action
hand_gesture

I was expecting "walking" in this output. What are the possible values apart from above output.

Hello, I want to sequence analysis for the videos using predictions made by the neural network. I would like to know the error rate of the pedestrians. Please help me how to do that.

Is it possible to provide the steering angle of the vehicle?

Hi,
Thanks a lot for this amazing work. I have recently started working with JAAD data and had confusion on this line(

Thanks!

Hi,

Many thanks for your great work!

I have some questions about the annotation of pedestrian '0_5_12b', could you please kindly help to clarify?

From video_0005_attributes.xml, I know 0_5_12b is a female adult (the lady with a child).

In the video, this lady exists from frame 00000.png to 00158.png (she is fully occluded by a car from frame 00159). From around frame 00120.png she has occlusion.

However, in video_0005.xml, she exists from frame 12 to frame 206. Also, this file says she has occlusion from frame 12 to frame 77, and has no occlusion from frame 78 to 166, and then has part occlusion from frame 167 to frame 204.

So I think the image and annotations about '0_5_12_b' do not match.

Furthermore, in your latest benchmark work that provides pose data, pedestrian 'o_5_12b' still has pose data from frame 00158.png to frame 00173.png, but actually she is fully occluded from frame 00159.png.

I am not sure if I made any mistake here...Could you please check if you have sometime? That would be very helpful!

Many thanks for your help!

Bests,
Xingchen

Thank you so much for providing such a wonderful dataset.
I have a question. Is it possible to add pedestrian_unique_id on retinanet method as in the CSV results? Because, it just generates frames_id, bounding box, and pedestrian class without generating each pedestrian's unique ID.
If possible, may you teach or share here how to do it? Thank you, I would like to appreciate any help you can provide.