julimueller / dtld_parsing Goto Github PK

Scripts for loading and visualizing the DriveU Traffic Light Dataset (DTLD)

CMake 1.17% C++ 52.67% MATLAB 11.32% Python 34.84%

dtld_parsing's Introduction

Important information (05/2020)

05/2020: The label format changed from .yml to .json. This should help to load the dataset significantly faster (especially in Python). Moreover the following changed:

Class Identity is replaced by attributes dictionary. The attributes dictionary contains the keys "direction", "relevance", "occlusion", "orientation", "aspects", "state" and "pictogram". New since v2.0 is the "reflection" attribute.
Some labels may have the items ("aspects", "unknown"), ("state", "unknown") or ("pictogram", "unknown"). This differs to the labels in the yml format, which always had one default value even if the label information changed. Note that due to this your results on json max differ to .yml
MATLAB reader is deprecated. If you want to use .json in MATLAB you have to write your own reader. Do not hesitate to create a pull request.
API in C++ and Python slightly changed. Take a look into the code!

The amount of overall labels/images is NOT changed. The image data itself is also untouched.

For everybody who still wants to work with the old .yml files: You can simply checkout the state tagged as "v1". If you are not able to see the tag do not forget to fetch & pull.

The DriveU Traffic Light Dataset (DTLD): Introduction and Comparison with Existing Datasets

This repository provides code for parsing the DriveU Traffic Light Dataset (DTLD), which is published in the course of our 2018 ICRA publication "The DriveU Traffic Light Dataset: Introduction and Comparison with Existing Datasets".

Paper

Paper see https://ieeexplore.ieee.org/document/8460737.

Download the dataset

INFO (11/27/2018): The Dataset is online now!

The data can be downloaded from http://www.traffic-light-data.com/.

NEW v2 04/2021: json label format

.
├── DTLD                 # DTLD
    ├── Berlin           # Contains all Routes of Berlin
    ├── Bochum           # Contains all routes of Bochum
    ├── Bremen           # Contains all routes of Bremen
    ├── Dortmund         # Contains all routes of Dortmund
    ├── Duesseldorf      # Contains all routes of Duesseldorf
    ├── Essen            # Contains all routes of Essen
    ├── Frankfurt        # Contains all routes of Frankfurt
    ├── Fulda            # Contains all routes of Fulda
    ├── Hannover         # Contains all routes of Hannover
    ├── Kassel           # Contains all routes of Kassel
    ├── Koeln            # Contains all routes of Cologne
    ├── DTLD_labels_v1.0 # Old labels (v1.0) in yml-format
    ├── DTLD_labels_v2.0 # New labels (v2.0) in json-format
    ├── LICENSE          # License
    └── README.md        # Readme

DEPRECATED: DTLD_v1 (yml-Files)

.
├── DTLD                 # DTLD
    ├── Berlin           # Contains all Routes of Berlin
    ├── Bochum           # Contains all routes of Bochum
    ├── Bremen           # Contains all routes of Bremen
    ├── Dortmund         # Contains all routes of Dortmund
    ├── Duesseldorf      # Contains all routes of Duesseldorf
    ├── Essen            # Contains all routes of Essen
    ├── Frankfurt        # Contains all routes of Frankfurt
    ├── Fulda            # Contains all routes of Fulda
    ├── Hannover         # Contains all routes of Hannover
    ├── Kassel           # Contains all routes of Kassel
    ├── Koeln            # Contains all routes of Cologne
    ├── DTLD_labels      # Labels in yml-format
    ├── LICENSE          # License
    └── README.md        # Readme

Route structure

We separated each drive in one city into different routes

.
├── Berlin                # Berlin
    ├── Berlin1           # First route
    ├── Berlin2           # Second route
    ├── Berlin3           # Third route
    ├── ...

Sequence structure

We separated each route into several sequences. One sequence describes one unique intersection up to passing it. The foldername indicates date and time.

.
├── Berlin 1                    # Route Berlin1
    ├── 2015-04-17_10-50-05     # First intersection
    ├── 2015-04-17_10-50-41     # Second intersection
    ├── ...

Image structure

For each sequences, images and disparity images are available. Filename indicates time and date

.
├── 2015-04-17_10-50-05                                      # Route Berlin1
    ├── DE_BBBR667_2015-04-17_10-50-13-633939_k0.tiff        # First left camera image
    ├── DE_BBBR667_2015-04-17_10-50-13-633939_nativeV2.tiff  # First disparity image
    ├── DE_BBBR667_2015-04-17_10-50-14-299876_k0.tiff        # Second left camera image
    ├── DE_BBBR667_2015-04-17_10-50-14-299876_nativeV2       # Second disparity image
    ├── ...

Before starting

1. Check our documentation

Documentation is stored at /dtld_parsing/doc/. We give insights into the data and explain how to interpret it.

2. Change absolute paths

Do not forget to change the absolute paths of the images in all label files (.yml).

Using the dataset

C++

Clone the dtld_parsing respository

git clone https://github.com/julimueller/dtld_parsing

Build everything

1. cd dtld_parsing/c++/driveu_dataset/
2. mkdir build && cd build
3. cmake .. -DCMAKE_INSTALL_PREFIX="YOUR_PATH" && make -j12 install
4. driveu_test -label_file <label_file_path.yml> -calib_dir <path_to_calib> -data_base_dir <dtld_dir>

Note: "YOUR_PATH" has to be in LD_LIBRARY_PATH. DTLD_DIR is the directory where all .zips should be unpacked. The visualization should look like this

Python

UPDATE 03-26-2020: Python modules were changed to Python 3.

git clone https://github.com/julimueller/dtld_parsing
cd dtld_parsing
python3 setup.py install
cd python
python3 load_dtld.py --label_file <label_file_path.yml> --calib_dir <path_to_calib> --data_base_dir <dtld_dir>

Result should look like above

MATLAB

NOTE 05/2020: MATLAB support is deprecated and will likely not be added for the new label format (json).

Run main.m

Results should look like this

Citation

Do not forget to cite our work for the case you used DTLD

Citation:

@INPROCEEDINGS{8460737,
author={A. Fregin and J. Müller and U. Kreβel and K. Dietmayer},
booktitle={2018 IEEE International Conference on Robotics and Automation (ICRA)},
title={The DriveU Traffic Light Dataset: Introduction and Comparison with Existing Datasets},
year={2018},
volume={},
number={},
pages={3376-3383},
keywords={computer vision;image recognition;traffic engineering computing;DriveU traffic light dataset;traffic light recognition;autonomous driving;computer vision;University of Ulm Traffic Light Dataset;Daimler AG;Cameras;Urban areas;Benchmark testing;Lenses;Training;Visualization;Detectors},
doi={10.1109/ICRA.2018.8460737},
ISSN={2577-087X},
month={May},}

dtld_parsing's People

Contributors

Stargazers

Watchers

Forkers

atyshka shini-gami yeongjae etarakci-hvl jdeguia amirunpri2018 anzisheng darren-jl zaharenok rrkini yarwest arnoldfychen

dtld_parsing's Issues

Label map for Tensorflow

Hello,

Thank you for your work on parsing the dataset, and also for such a great dataset.

I was wondering how we need to create a label map for training using TensorFlow. TensorFlow requires a label map, which namely maps each of the used labels to an integer values. This label map is used both by the training and detection processes.

An example of it is below, assuming dataset contains three labels:

item {
id: 1
name: 'red'
}

item {
id: 2
name: 'green'
}

item {
id: 3
name: 'yellow'
}

I am bit confused about if we need to put all the labels, given we have a lot of attributes in the dataset or what could be the possible workaround? Any help is much appreciated.

The DTLD dataset's url was invalid

I have received the email that contain the download link, but the url was missing.

why don't all xx_k0.tiff show any image (all are black)

I downloaded Bochum data, but data with the label of xx_k0.tiff shows all black.

Using disparity images/maps to get real distance information

Hello,
Firstly, thank you for your amazing work. I am currently working on DTLD for my thesis. As mentioned in the title, I need to calculate real distance by using disparity images. Finding the depth is straightforward with focal length, baseline, and disparity matrix (considering parallel camera setup). However, I could not figure out how to calculate the 3D location of a point from the depth information. In OpenCV, there is the function cv2.reprojectImageTo3D(disparity_image, Q) where it requires 4x4 Q matrix (projection transformation matrix) and according to documentation, it can be obtained by cv2.stereoRectify(), but this one requires the following:

cameraMatrix1 – First camera matrix.
cameraMatrix2 – Second camera matrix.
distCoeffs1 – First camera distortion parameters.
distCoeffs2 – Second camera distortion parameters.
imageSize – Size of the image used for stereo calibration.
R – Rotation matrix between the coordinate systems of the first and the second cameras.
T – Translation vector between coordinate systems of the cameras.

Checking the calibration data published, I see that I need intrinsic matrix and distortion coefficients for the right camera too.
Could you help me about this task? Thank you very much.

Download issue

Currently, I cannot download from http://www.traffic-light-data.com/.
Can I download the dataset please?

python setup.py install command fails

When trying to run "python setup.py install" it gives me this error:

Traceback (most recent call last):
File "setup.py", line 1, in
from setuptools import setup, find_packages
ImportError: No module named setuptools

Saw that some modules were updated to Python 3.
Shouldn't we use the command "python3 setup.py install" instead in order to run it with python 3?

Why is the image I downloaded is .tiff, and there is no color, not RGB. How can I get RGB color images?

Why is the image I downloaded is .tiff, and there is no color, not RGB.
How can I get RGB color images?

class list

hello, I have seen the class annotition about the dataset as the following picture, do you know the class list or all class numbers?
for example: if the first bit of Viewpoint orientation is 2,3,4, apperantly, the behind annotation is none, so the class numbers is not 44745*7

Are images pre-rectified?

Hi Julian, I'm working on a fork of this repo to convert the dataset to Tensorflow records and had a quick question regarding calibration. Are the RGB (non-stereo) images provided pre-rectified or do we need to do that manually using the intrinsics provided? I don't see much warping in the sample images but I want to make sure that Tensorflow is ingesting properly rectified data.

EDIT: Looking through the python code, it seems that the bounding box coordinates are rectified using opencv but the images themselves are not. Would I be correct in assuming that the bounding boxes are unrectified and the images themselves are rectified? That would be a little counter-intuitive.

Display/load image in natural colors

Hello,
I am trying to load the images from this dataset in natural colors. I use the following code:

img_file = str(image_path)
img = cv2.imread(img_file, cv2.IMREAD_UNCHANGED)
img = cv2.cvtColor(img, cv2.COLOR_BAYER_GB2BGR)
img = np.right_shift(img, 4)
img = img.astype(np.uint8)
plt.figure(figsize=(20,20))
plt.imshow(img)

The "image_path" variable contains the path to one image ending with "k0.tiff". Unfortunately, when I display the images there seems to be a different color setting, as you can see in the attached image (yellow colors are showed as bluish).

Thank you for any help!

.tiff files are black

Hi,

I suspect there is something I'm doing wrong, but in the datasets all the ..._k0.tiff images are black.
Is there something I'm supposed to do to them first so that I can see them?

Thanks.

cannot connect to X server

i am trying to run and load the dataset but get the error: cannot connect to X server. i am running load_dtld.py with the proper path CLAs. i am currently loading only the berlin dataset, and i have changed the path and disp_path in the yaml file to point to the correct image location (based on the output it seems to be correct?). am i missing something?

Opening DriveuDatabase from File: data/dtld/Berlin_all.yml
data/dtld/Berlin/Berlin1/2015-04-17_10-50-05/DE_BBBR667_2015-04-17_10-50-13-633939_k0.tiff
...
Intrinsic Matrix:

[[2.29051e+03 0.00000e+00 1.06694e+03]
 [0.00000e+00 2.29051e+03 4.77152e+02]
 [0.00000e+00 0.00000e+00 1.00000e+00]]

Extrinsic Matrix:

[[ 0.0037 -0.0453  0.999   2.105 ]
 [-0.9998  0.0188  0.0047  0.049 ]
 [-0.0189 -0.9988 -0.0452  0.919 ]]

Projection Matrix:

[[2.26969958e+03 0.00000000e+00 1.05129465e+03 0.00000000e+00]
 [0.00000000e+00 2.26969958e+03 4.58462746e+02 0.00000000e+00]
 [0.00000000e+00 0.00000000e+00 1.00000000e+00 0.00000000e+00]]

Rectification Matrix:

[[ 9.99999e-01 -1.18600e-03  6.77000e-04]
 [ 1.18100e-03  9.99972e-01  7.35600e-03]
 [-6.86000e-04 -7.35500e-03  9.99973e-01]]

Distortion Matrix:

[[-0.036004  0.        0.        0.        0.      ]]

: cannot connect to X server

System

python 3.6
Ubuntu 16.04

TypeError: 'NoneType' object does not support item assignment

Hi, i got this problem during visualization of images. I did everything according to instructions:

python3 load_dtld.py --label_file Berlin_all.yml --calib_dir /dtld_parsing-master/calibration --data_base_dir /Berlin/

For some time everything was going well, i think:

2020-04-23 14:15:12.976 INFO load_dtld - main: Intrinsic Matrix:

[[2290.51     0.    1066.94 ]
 [   0.    2290.51   477.152]
 [   0.       0.       1.   ]]

2020-04-23 14:15:12.977 INFO load_dtld - main: Extrinsic Matrix:

[[ 0.0037 -0.0453  0.999   2.105 ]
 [-0.9998  0.0188  0.0047  0.049 ]
 [-0.0189 -0.9988 -0.0452  0.919 ]]

2020-04-23 14:15:12.978 INFO load_dtld - main: Projection Matrix:

[[2269.699585    0.       1051.294655    0.      ]
 [   0.       2269.699585  458.462746    0.      ]
 [   0.          0.          1.          0.      ]]

2020-04-23 14:15:12.979 INFO load_dtld - main: Rectification Matrix:

[[ 0.999999 -0.001186  0.000677]
 [ 0.001181  0.999972  0.007356]
 [-0.000686 -0.007355  0.999973]]

2020-04-23 14:15:12.980 INFO load_dtld - main: Distortion Matrix:

[[-0.036004  0.        0.        0.        0.      ]]

But then i got this error:

Traceback (most recent call last):
  File "load_dtld.py", line 106, in <module>
    main(parse_args())
  File "load_dtld.py", line 76, in main
    img_disp = img.visualize_disparity_image()
  File "/traffic_lights/dtld_parsing-master/python/dtld_parsing/driveu_dataset.py", line 206, in visualize_disparity_image
    img = self.get_disparity_image()
  File "/traffic_lights/dtld_parsing-master/python/dtld_parsing/driveu_dataset.py", line 189, in get_disparity_image
    img[img == 65535] = 0
TypeError: 'NoneType' object does not support item assignment

Do you have any advises for me? Some image didn't find or..?

How to get traffic light bounding box using gps latitude and longitude?

Kindly help in finding bounding box of traffic light using gps latitude and longitude

Error with pyyaml 6.0

python3 load_dtld.py --label_file /DriveU/dtld_data/v2.0/DTLD_all.json --calib_dir /DriveU/dtld_parsing/calibration --data_base_dir /DriveU/dtld_data

2021-12-19 17:23:08.352 INFO driveu_dataset - open: Opening DriveuDatabase from file: /DriveU/dtld_data/v2.0/DTLD_all.json Traceback (most recent call last): File "load_dtld.py", line 119, in <module> main(parse_args()) File "load_dtld.py", line 61, in main intrinsic_left = calibration.load_intrinsic_matrix( File "/DriveU/dtld_parsing/python/dtld_parsing/calibration.py", line 262, in load_intrinsic_matrix matrix = self.load_calibration_matrix(path) File "/DriveU/dtld_parsing/python/dtld_parsing/calibration.py", line 249, in load_calibration_matrix data = yaml.load(infile) TypeError: load() missing 1 required positional argument: 'Loader'

Convert to YOLO format

Hi. Is there a way to convert the DriveU dtld to YOLO format?
Thank you in advance