mcpcapital / harmonizeproject Goto Github PK

Hey,
I had this running on raspberryos for more than a year. Used it almost daily. I wanted to bump it to the latest version and I got this error. I started completely from scratch with a clean ubuntu server install on the rpi4. Same hardware that worked before.
After adding my user to the video group I was finally getting video, but not in the script.
I see the Input on videotest.py (installed gnome and used xforwarding...) but its showing the warning. Harmonize script itself wont work.

lsusb
Bus 003 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 003: ID eba4:7588 USB3.0 HD Audio Capture USB3.0 HD Video Capture
Bus 001 Device 002: ID 2109:3431 VIA Labs, Inc. Hub
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

v4l2-ctl -D
Driver Info:
        Driver name      : uvcvideo
        Card type        : USB3.0 HD Video Capture: USB3.0
        Bus info         : usb-0000:01:00.0-1.1
        Driver version   : 5.15.64
        Capabilities     : 0x84a00001
                Video Capture
                Metadata Capture
                Streaming
                Extended Pix Format
                Device Capabilities
        Device Caps      : 0x04200001
                Video Capture
                Streaming
                Extended Pix Format
Media Driver Info:
        Driver name      : uvcvideo
        Model            : USB3.0 HD Video Capture: USB3.0
        Serial           : HU123450
        Bus info         : usb-0000:01:00.0-1.1
        Media version    : 5.15.64
        Hardware revision: 0x00000328 (808)
        Driver version   : 5.15.64
Interface Info:
        ID               : 0x03000002
        Type             : V4L Video
Entity Info:
        ID               : 0x00000001 (1)
        Name             : USB3.0 HD Video Capture: USB3.0
        Function         : V4L2 I/O
        Flags            : default
        Pad 0x01000007   : 0: Sink
          Link 0x02000010: from remote pad 0x100000a of entity 'Processing 3' (Video Pixel Formatter): Data, Enabled, Immutable

v4l2-ctl --info -d /dev/video0 --list-formats-ext
Driver Info:
        Driver name      : uvcvideo
        Card type        : USB3.0 HD Video Capture: USB3.0
        Bus info         : usb-0000:01:00.0-1.1
        Driver version   : 5.15.64
        Capabilities     : 0x84a00001
                Video Capture
                Metadata Capture
                Streaming
                Extended Pix Format
                Device Capabilities
        Device Caps      : 0x04200001
                Video Capture
                Streaming
                Extended Pix Format
Media Driver Info:
        Driver name      : uvcvideo
        Model            : USB3.0 HD Video Capture: USB3.0
        Serial           : HU123450
        Bus info         : usb-0000:01:00.0-1.1
        Media version    : 5.15.64
        Hardware revision: 0x00000328 (808)
        Driver version   : 5.15.64
Interface Info:
        ID               : 0x03000002
        Type             : V4L Video
Entity Info:
        ID               : 0x00000001 (1)
        Name             : USB3.0 HD Video Capture: USB3.0
        Function         : V4L2 I/O
        Flags            : default
        Pad 0x01000007   : 0: Sink
          Link 0x02000010: from remote pad 0x100000a of entity 'Processing 3' (Video Pixel Formatter): Data, Enabled, Immutable
ioctl: VIDIOC_ENUM_FMT
        Type: Video Capture

        [0]: 'MJPG' (Motion-JPEG, compressed)
                Size: Discrete 1920x1080
                        Interval: Discrete 0.017s (60.000 fps)
                        Interval: Discrete 0.033s (30.000 fps)
                Size: Discrete 1280x720
                        Interval: Discrete 0.017s (60.000 fps)
                        Interval: Discrete 0.033s (30.000 fps)
                Size: Discrete 960x540
                        Interval: Discrete 0.017s (60.000 fps)
                        Interval: Discrete 0.033s (30.000 fps)
                Size: Discrete 800x450
                        Interval: Discrete 0.017s (60.000 fps)
                        Interval: Discrete 0.033s (30.000 fps)
                Size: Discrete 640x360
                        Interval: Discrete 0.017s (60.000 fps)
                        Interval: Discrete 0.033s (30.000 fps)

python3 videotest.py
Press ESC to exit.
[ WARN:0] global ./modules/videoio/src/cap_gstreamer.cpp (2075) handleMessage OpenCV | GStreamer warning: Embedded video playback halted; module v4l2src0 reported: Internal data stream error.
[ WARN:0] global ./modules/videoio/src/cap_gstreamer.cpp (1053) open OpenCV | GStreamer warning: unable to start pipeline
[ WARN:0] global ./modules/videoio/src/cap_gstreamer.cpp (616) isPipelinePlaying OpenCV | GStreamer warning: GStreamer: pipeline have not been created

./harmonize.py -v
--- Starting Harmonize Project ---
INFO: Detected Philips Hue - 6D35D3._hue._tcp.local. via mDNS at IP address: 172.17.172.8
INFO: Single Hue bridge detected on network via mDNS.
INFO: Hue bridge located at: 172.17.172.8
Checking whether Harmonizer application is already registered (Looking for client.json file).
INFO: Client data found from client.json file.
INFO: Client data valid: {'username': 'XXXXXXXXXXXXXXXX', 'clientkey': 'XXXXXXXXXXXXXXXXXXXXXXXX'}
Requesting bridge information...
INFO: The bridge is capable of streaming via APIv2. Firmware version 1955082050 detected...
Querying hue bridge for entertainment areas on local network.
groupid =  7
Using Entertainment area with group_id: 7
Selected Entertainment UDID: XXXXXXXXXXXXXXXXXXXXXXXX
INFO: 5 light(s) found in selected Entertainment area. Locations [x,y,z] are as follows:
 {'0': [-0.93, -0.28, 0.0], '1': [1.0, 0.43, 1.0], '2': [-1.0, 0.92, 0.0], '3': [0.95999, 0.68997, 0.0], '4': [0.91, 0.97, 0.0]}
Hue application id: 3c90b91a-6f58-4272-ae4b-987a6257ba27
Enabling streaming to your Entertainment area
{'data': [{'rid': 'XXXXXXXXXXXXXXXXXXXXXXX', 'rtype': 'entertainment_configuration'}], 'errors': []}
Starting computer vision engine...
OpenCV version: 4.5.4
--- INFO: Detected video capture card on /dev/video0 ---
[ WARN:0] global ./modules/videoio/src/cap_gstreamer.cpp (2075) handleMessage OpenCV | GStreamer warning: Embedded video playback halted; module v4l2src0 reported: Internal data stream error.
[ WARN:0] global ./modules/videoio/src/cap_gstreamer.cpp (1053) open OpenCV | GStreamer warning: unable to start pipeline
[ WARN:0] global ./modules/videoio/src/cap_gstreamer.cpp (616) isPipelinePlaying OpenCV | GStreamer warning: GStreamer: pipeline have not been created
Disabling streaming on Entertainment area...
{'data': [{'rid': 'XXXXXXXXXXXXXXXXXXXXXXXX', 'rtype': 'entertainment_configuration'}], 'errors': []}
ERROR: Unable to open capture device.

After selecting groupid, VIDEOIO ERROR: V4L2: setting property #38 is not supported

pi@raspberrypi:~/HarmonizeProject $ v4l2-ctl --list-devices
bcm2835-codec-decode (platform:bcm2835-codec):
/dev/video10
/dev/video11
/dev/video12

bcm2835-isp (platform:bcm2835-isp):
/dev/video13
/dev/video14
/dev/video15
/dev/video16

USB3.0 HD VIDEO: USB3.0 HD VIDE (usb-3f980000.usb-1.4):
/dev/video0
/dev/video1

[video4linux2,v4l2 @ 0x5dd1c0] Compressed: mjpeg : Motion-JPEG : 1920x1080 1600x1200 1360x768 1280x1024 1280x960 1280x720 1024x768 800x600 720x576 720x480 640x480
[video4linux2,v4l2 @ 0x5dd1c0] Raw : yuyv422 : YUYV 4:2:2 : 1920x1080 1600x1200 1360x768 1280x1024 1280x960 1280x720 1024x768 800x600 720x576 720x480 640x480

I have attempted both paths you list on a Raspberry Pi running Ubuntu 20.04 and neither build path produces a harmonize.py that starts.

Can you confirm that both install processes have been tested recently? I'm trying to understand why I'm having such trouble, given that you seem to have very clear instructions (they were easy to follow).

There are a couple packages that it says it can't find (when running your install script) so I don't know if they've been recently retired/changed or not. That's why I want to confirm that the install approaches have been tried recently from a clean install of the OS (that's where I started from)

Thanks!

Wanted to know if there's an option or a parameter one can tweak to increase the intensity of light through the script. It works great ofcourse but I've noticed that the lights are dimmer compared to the app version of hue sync. Apologies if this is something obvious that I'm missing.

Hello there. Yesterday I started setting up the harmonize project in my house. I bought everything I needed and after many hours of work I managed to make it work, I am very happy with the result. But the fact that I have to manually launch the harmonize.py file bothers me a lot as I have the whole house automated. So I installed supervised home assistant in a container on the raspberry itself and am trying in vain to integrate it. I tried with shell command but it constantly gives me error, probably because harmonize is in the root of the raspberry while home assistant is in the container. Is there any way to integrate and make these two software work together?

Hello,
I get the error:

"File "./harmonize.py", line 28, in

import mbedtls

File "/usr/local/lib/python3.7/dist-packages/mbedtls/init.py", line 8, in

import mbedtls.cipher as cipher

File "/usr/local/lib/python3.7/dist-packages/mbedtls/cipher/init.py", line 12, in

from . import AES, ARC4, ARIA, CHACHA20, DES, DES3, Blowfish, Camellia, DES3dbl

File "/usr/local/lib/python3.7/dist-packages/mbedtls/cipher/AES.py", line 11, in

from mbedtls.exceptions import TLSError

ModuleNotFoundError: No module named 'mbedtls.exceptions'"

Maybe you can help me

Error Log:

Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 917, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 865, in run
self._target(*self._args, **self._kwargs)
File "harmonize.py", line 226, in averageimage
coords[0] = ((coords[0])+1) * w//2 #Translates x value and resizes to video aspect ratio
NameError: name 'w' is not defined

Exception in thread Thread-3:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 917, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 865, in run
self._target(*self._args, **self._kwargs)
File "harmonize.py", line 309, in buffer_to_light
for i in rgb_bytes:
NameError: name 'rgb_bytes' is not defined

Got the same error using sudo and without.
changed the time at line 329/331 but still got the "w not defined" error

setup:
raspberry pi zero (got same error on Raspberry Pi 3b)
Roxio game capture

I get this message after a few seconds:

I will use the bridge at  192.168.1.75
Hue bridge not found. Mission failed, better luck next time

After it finds the bridge it only gives a few seconds to push the button (it never asks me to though - not sure if it is supposed to), but then it just says the bridge isn't found.

Any help would be appreciated.

Originally opened by @spapadim on August 4th 2020:
"Very cool and inspiring project! Always wanted one of these, but off-the-shelf prices are ridiculous..

First thing that immediately came to mind when I saw this: would this work with an HDMI-to-CSI adapter, such as eg https://www.amazon.com/dp/B0899L6ZXZ/ref=cm_sw_r_cp_apa_i_edwkFbJ7V4HT2 ?

Tl;dr: with GPU-accelerated frame processing, I'm hoping this could run even on a Zero..! :)

Other than potentially reducing cost a little bit (CSI adapter vs USB capture), the main advantage is that video downscaling (to compute average color intensities around border) could be offloaded to Pi's GPU (eg, see picamera Python module docs on "advanced configurations" -- GPU can even compute motion vector data quite efficiently, let alone just do pixel averaging). This could possibly allow the use of a smaller Pi, perhaps even a Zero, which would then really reduce cost.

The converters' frame rate is relatively low ("up to" 25fps), but I don't believe this matters here.

The only real catch is that these converters don't do HDCP. But, perhaps, if there is a reasonably priced splitter that can also do HDCP stripping on lower-res port..

I just wish my TV could do full HDMI pass-through (not just ARC to receiver but, eg, for chaining a second monitor), or my receiver could route to both zone's HDMI outputs simultaneously (rather than switching video and splitting just audio), but both of these features seem to be exceedingly rare.. sigh!

Finally, apologies in advance if using an issue for general commenting was inappropriate (feel free to close/delete, of so) -- but just my 2c. I'll probably take this up myself when (if?) I find some time."

i have an odroid lying around. can i use it instead of raspberry pi

Example image links in the README are broken with a 'Cannot proxy the given URL' message

Hey great script.

i use it on a raspberry 3b with a LogiLink Audio und Video Grabber USB 2.0

my questions is there a way to input more code for fine tuning to the script like values for:

• cropping
• saturation
• contrast
• brightness

thx

Hi,

I cannot seem to connect to my hue hub, I get the below error:

pi@raspberrypi:~/HarmonizeProject $ python3 ./harmonize.py
I will use the bridge at 192.168.5.24
Hue bridge not found. Mission failed, better luck next time

originally my hue hub was in 192.168.4.206, it’s all on the same network but I switched it and cleared the pi cache. I have also done it on a brand new install.

I noticed a closed issue where someone suggests deleting the config.json file but I do not have the file.

Any ideas would be appreciated.

Opened by @Bboy486 on August 13th

This is two issues in one. One is fixed.

1- The project sees the hue hub and when I press the button (or hold it down) I get an error that it has timed out have a nice day (something like that). I had to
cd harmonizeproject
and install from the folder.

2- After I did that I was able to add my hub. But I did receive the error below.
Success! I generated a username and client key to access the bridge's Entertainment API!
Traceback (most recent call last):
File "./harmonize.py", line 143, in
r = requests.get(url = baseurl+"/config")
NameError: name 'baseurl' is not defined

Originally reported by Unknown {Restoring old issues}

"Hello, I am trying to install this on my PI4 using Rasbian Buster Lite OS but it seems to encounter a lot of missing dependencies. What OS should I be using? I already purchased a HDMI - USB capture card but have been having difficulties getting this thing to run. Any help would be appreciated. Thanks"

Originally reported by @jtwizzy on August 12th 2020 {Restoring old issues}

"I am getting no module named Http_parser when running ./harmonize.py "

Originally reported by @restorept {Restoring old issues}

"Hello. I'm a beginner with Raspberry Pi. It is up and running properly. Wondering if there are specific step-by-step instructions for beginners. I have all the hardware necessary to complete this project but need specific step-by-step instructions on where to enter the coding within Raspberry Pi. Any help is most appreciated. Thank you!"

Hey not sure if this is an issue that can be resolved with the program or if it’s a HW issue but my Hue lights flicker even when the scene on the tv isn’t changing. Please see attached video.

Following up from #7 (comment)

In the instructions, there are some missing steps.
change cd harmonizeproject to cd HarmonizeProject
after sudo ./setup.sh you will need to give executable rights to ./harmonize.py
run - ls -la
If there is an X missing from harmonize.py then run the following:
sudo chmod +x harmonize.py

If you run ls -la, you'll see the file is now executable

now run sudo ./harmonize.py

I was looking at the trace and noticed a lot of corrupt jpeg data

LAstly, when I try to re-run the .py I get this error

sudo ./harmonize.py
I will use the bridge at 192.168.1.7
Hue bridge not found. Mission failed, better luck next time
pi@raspberrypi:~/HarmonizeProject $

WHEN IT DOES RUN I GOT THIS ERROR

You selected groupid 4
VIDEOIO ERROR: V4L2: Pixel format of incoming image is unsupported by OpenCV
Unable to stop the stream: Device or resource busy
Press return to stopException in thread Thread-2:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 917, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 865, in run
self._target(*self._args, **self._kwargs)
File "./harmonize.py", line 226, in averageimage
coords[0] = ((coords[0])+1) * w//2 #Translates x value and resizes to video aspect ratio
NameError: name 'w' is not defined

Exception in thread Thread-3:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 917, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 865, in run
self._target(*self._args, **self._kwargs)
File "./harmonize.py", line 309, in buffer_to_light
for i in rgb_bytes:
NameError: name 'rgb_bytes' is not defined

Hello,
I have implemented everything so far and it runs flawlessly. Only I do not know where to open the Hue app. Where and what do I have to do?
Thanks in advance!

I always get this error,
Script connects successfully, turns Hue Gradient dark and then shows colors of picture / first frame.

python3 harmonize.py -g 2 -i a.b.c.d -f bigbuckbunny.mp4

Then this shows up:

  File "/usr/lib/python3.9/threading.py", line 954, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.9/threading.py", line 892, in run
    self._target(*self._args, **self._kwargs)
  File "/HarmonizeProject/harmonize.py", line 378, in cv2input_to_buffer
    rgbframe = cv2.cvtColor(bgrframe, cv2.COLOR_BGR2RGB) #corrects BGR to RGB
cv2.error: OpenCV(4.5.1) ../modules/imgproc/src/color.cpp:182: error: (-215:Assertion failed) !_src.empty() in function 'cvtColor'```