xeonqq / co-pilot Goto Github PK

Machine Learning Based Real-Time Traffic Light Alert on Your Car with Raspberrypi

Home Page: https://xeonqq.github.io/machine%20learning/rpi-coral-traffic-light-alert-en/

License: GNU General Public License v3.0

Python 97.52% Shell 1.10% Dockerfile 1.37%

maching-learning raspberry-pi traffic-light-detection ssd coral-tpu dashcam traffic-light-classification motion-detection surveillance

co-pilot's Introduction

Traffic light alert and Dashcam all in one.

Co-Pilot = Raspberrypi 3/4 + rpi camera + Google Coral TPU. Language support English/中文.

Features

Real time traffic light voice alert based on situation.
HD dashcam recording.
Surveillance mode, records only when motion is detected.
Auto deletion of old files when disk full.
One button for mode selection.

Buy me a coffee 😇

Watch the demo in car

What Co-Pilot sees	Voice alerts based on situation
	Ready-go / Green-go-go-go / Attention-red / Yellow-no-rush

Hardware Setup

Version	Front	Back
v0.1
v0.2: using min speaker
v0.3: custom designed housing

optional: RTC DS3231, to have correct date on the dashcam video and log.

Limitations

Currently works only with vertically placed traffic lights, optimized for Germany.
Delay of ~0.3 sec for each detection (Rpi 4 might have better performance, didn't have one to test)
Performance drops during night

Dependencies

echo "deb https://packages.cloud.google.com/apt coral-edgetpu-stable main" | tee /etc/apt/sources.list.d/coral-edgetpu.list
curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add -
apt-get update
udo apt install -y python3-opencv
apt-get install -y libedgetpu1-std
apt-get install -y python3-pycoral
apt-get install -y python3-tflite-runtime
python3 -m pip install -r requirements_pi.txt
sudo apt-get install libsdl2-mixer-2.0-0  libsdl2-2.0-0

# instal ffmpeg
cd ~ && git clone --depth 1 https://code.videolan.org/videolan/x264
cd x264
./configure --host=arm-unknown-linux-gnueabi --enable-static --disable-opencl
make -j4 && sudo make install
cd ~ && git clone git://source.ffmpeg.org/ffmpeg --depth=1
./configure --extra-ldflags="-latomic" --arch=armel --target-os=linux --enable-gpl --enable-omx --enable-omx-rpi --enable-nonfree
make -j4 && sudo make install

Run Co-Pilot

python3 -m src.main  --ssd_model models/ssd_mobilenet_v2_coco_quant_no_nms_edgetpu.tflite  --label models/coco_labels.txt --score_threshold 0.3 --traffic_light_classification_model models/traffic_light_edgetpu.tflite  --traffic_light_label models/traffic_light_labels.txt --blackbox_path=./

I use superviser to start co-pilot at RPI boot up.

Run Dashcam only mode

python3 -m src.dashcam

Run Surveillance mode

like dashcam mode, but record only if motion is detection

python3 -m src.dashcam --record_on_motion

Watch how the motion is detected under the hood:

Run task manager to be able to select any mode

python3 -m src.task_manager --blackbox_path=/mnt/hdd

A detailed description of the mode selection can be found in user manual.

Adjust volume

Once you've SSH'd into your Pi, type "alsamixer". This will bring up an interface within the terminal which will allow you to set the volume of the Raspberry Pi. Simply press the up and down arrow keys to either increase or decrease the volume. When you are done, press ESC.

Test

# under repo root folder
python3 -m pytest
# or
python3 -m tests.test_detection
python3 -m tests.test_classification

Reprocess with recorded video (On Host PC)

Build and run docker container

./build.sh
./linux_run.sh

In docker container

cd workspace
python3 -m src.reprocess  --ssd_model models/ssd_mobilenet_v2_coco_quant_no_nms_edgetpu.tflite  --label models/coco_labels.txt --score_threshold 0.3 --traffic_light_classification_model models/traffic_light_edgetpu.tflite  --traffic_light_label models/traffic_light_labels.txt --blackbox_path=./ --video recording_20210417-090028.h264.mp4 --fps 5

Both main and reprocess can be run without Coral TPU by specifying --cpu option.

References

SSD model is downloaded from https://github.com/google-coral/test_data/blob/master/ssd_mobilenet_v2_coco_quant_postprocess_edgetpu.tflite

co-pilot's People

Contributors

Stargazers

Watchers

Forkers

surejm damian-ntam softwareescarlata maxmarkov leo-ma0502

co-pilot's Issues

Suggest to loosen the dependency on transitions

Dear developers,

Your project co-pilot requires "transitions==0.8.10" in its dependency. After analyzing the source code, we found that the following versions of transitions can also be suitable without affecting your project, i.e., transitions 0.8.11. Therefore, we suggest to loosen the dependency on transitions from "transitions==0.8.10" to "transitions>=0.8.10,<=0.8.11" to avoid any possible conflict for importing more packages or for downstream projects that may use ddos_script.

May I pull a request to further loosen the dependency on transitions?

By the way, could you please tell us whether such dependency analysis may be potentially helpful for maintaining dependencies easier during your development?

Details:

Your project (commit id: 4da7ed6) directly uses 2 APIs from package transitions.

transitions.core.Machine.__init__, transitions.core.State.__init__

Beginning fromwhich, 22 functions are then indirectly called, including 16 transitions's internal APIs and 6 outsider APIs as follows:

[/xeonqq/co-pilot]
+--transitions.core.Machine.__init__
|      +--collections.deque
|      +--collections.OrderedDict
|      +--transitions.core.Machine.add_states
|      |      +--transitions.core.listify
|      |      +--transitions.core.Machine._create_state
|      |      +--transitions.core.Machine._add_model_to_state
|      |      |      +--functools.partial
|      |      |      +--transitions.core.Machine._checked_assignment
|      |      |      +--inspect.ismethod
|      |      +--collections.OrderedDict.keys
|      |      +--transitions.core.Machine.add_transition
|      |      |      +--transitions.core.Machine._create_event
|      |      |      +--transitions.core.Machine._add_trigger_to_model
|      |      |      |      +--transitions.core.Machine._checked_assignment
|      |      |      |      +--functools.partial
|      |      |      +--collections.OrderedDict.keys
|      |      |      +--transitions.core.listify
|      |      |      +--transitions.core.Machine._has_state
|      |      |      |      +--collections.OrderedDict.values
|      |      |      +--transitions.core.Machine._create_transition
|      |      +--functools.partial
|      +--transitions.core.Machine.add_transitions
|      |      +--transitions.core.listify
|      |      +--transitions.core.Machine.add_transition
|      +--transitions.core.Machine.add_ordered_transitions
|      |      +--collections.OrderedDict.keys
|      |      +--transitions.core._prep_ordered_arg
|      |      |      +--transitions.core.listify
|      |      +--transitions.core.Machine.add_transition
|      +--transitions.core.Machine.add_model
|      |      +--transitions.core.listify
|      |      +--transitions.core.Machine._checked_assignment
|      |      +--collections.OrderedDict.values
|      |      +--transitions.core.Machine.set_state
|      |      |      +--transitions.core.Machine.get_state
|      |      |      +--transitions.core.listify
|      |      +--functools.partial
|      |      +--transitions.core.Machine._add_trigger_to_model
|      |      +--transitions.core.Machine._add_model_to_state
+--transitions.core.State.__init__
|      +--transitions.core.listify

Since all these functions have not been changed between any version for package "transitions" from [0.8.11] and 0.8.10. Therefore, we believe it is safe to loosen the corresponding dependency.

RPi 4 issues

Hi!

Love the project! My new gf keeps not seeing traffic lights and was SO EXCITED to find this! Wanted to make it a gift but I'm stuck after playing with it for over 2 weeks. Just can't get it running.

I'm using a RPi4 4GB on 32bit Raspbian, current release.
Running in X as I thought perhaps it was needed to use the camera.
I had tried headless via SSH with the same issues.
Using an 8MP picam on the port by the pins.
I've tried Python 3.9 fully updated including pip and 3.7 without updating pip.

I'm unclear if the camera needs to be enabled as legacy in boot.cfg or not? I've tried the python setup as well, but co-pilot doesn't see the camera then. Either way I get good quality stills and video preview.

I don't have the Corel Accelerator, so added --cpu in cmd and the task manager

Also, don't have a button or LED so I disabled the LED function and removed all but the first "full" co-pilot options in task manager and set the index from -1 to 0.

I have solved a number of issues but now I'm stuck with the below errors.

Thank you for this awesome project and any assistance!

Trying with Python 3.9:

copilot@raspberrypi:~/co-pilot $ sudo python3 -m src.main --ssd_model models/ssd_mobilenet_v2_coco_quant_no_nms_edgetpu.tflite --label models/coco_labels.txt --score_threshold 0.3 --traffic_lig>
/usr/lib/python3/dist-packages/picamera/encoders.py:519: PiCameraAlphaStripping: using alpha-stripping to convert to non-alpha format; you may find the equivalent alpha format faster
warnings.warn(
Exception in thread Thread-4:
Traceback (most recent call last):
File "/usr/lib/python3.9/threading.py", line 954, in _bootstrap_inner
self.run()
File "/usr/lib/python3.9/threading.py", line 892, in run
self._target(*self._args, **self._kwargs)
File "/home/copilot/co-pilot/src/camera_capturer.py", line 32, in _run
self._camera.capture(
File "/usr/lib/python3/dist-packages/picamera/camera.py", line 1409, in capture
camera_port, output_port = self._get_ports(use_video_port, splitter_port)
File "/usr/lib/python3/dist-packages/picamera/camera.py", line 559, in _get_ports
self._check_camera_open()
File "/usr/lib/python3/dist-packages/picamera/camera.py", line 540, in _check_camera_open
raise PiCameraClosed("Camera is closed")
picamera.exc.PiCameraClosed: Camera is closed
Encountered unresolved custom op: edgetpu-custom-op.Node number 0 (edgetpu-custom-op) failed to prepare.

^CException ignored in: <module 'threading' from '/usr/lib/python3.9/threading.py'>
Traceback (most recent call last):
File "/usr/lib/python3.9/threading.py", line 1428, in _shutdown
lock.acquire()
KeyboardInterrupt:
copilot@raspberrypi:~/co-pilot $ Output file #0 does not contain any stream

Trying with Python 3.7:
sudo python3.7 -m src.main --ssd_model models/ssd_mobilenet_v2_coco_quant_no_nms_edgetpu.tflite --label models/coco_labels.txt --score_threshold 0.3 --traffic_light_classification_model models/>
File "/usr/local/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"main", mod_spec)
File "/usr/local/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/home/copilot/co-pilot/src/main.py", line 10, in
from .led import Led
File "/home/copilot/co-pilot/src/led.py", line 1, in
import RPi.GPIO as GPIO
ModuleNotFoundError: No module named 'RPi'