furushchev / respeaker_ros Goto Github PK

View Code? Open in Web Editor NEW

67.0 4.0 58.0 43 KB

A ROS Package for Respeaker Mic Array

License: Apache License 2.0

CMake 3.34% Python 96.66%

respeaker_ros's Introduction

respeaker_ros

A ROS Package for Respeaker Mic Array

Supported Devices

Respeaker Mic Array v2.0

Preparation

Install this package

mkdir -p ~/catkin_ws/src && ~/catkin_ws/src
git clone https://github.com/furushchev/respeaker_ros.git
cd ~/catkin_ws
source /opt/ros/kinetic/setup.bash
rosdep install --from-paths src -i -r -n -y
catkin config --init
catkin build respeaker_ros
source ~/catkin_ws/devel/setup.bash

Register respeaker udev rules

Normally, we cannot access USB device without permission from user space. Using udev, we can give the right permission on only respeaker device automatically.

Please run the command as followings to install setting file:
```
roscd respeaker_ros
sudo cp -f $(rospack find respeaker_ros)/config/60-respeaker.rules /etc/udev/rules.d/60-respeaker.rules
sudo systemctl restart udev
```
And then re-connect the device.

Install python requirements

roscd respeaker_ros
sudo pip install -r requirements.txt

Update firmware

git clone https://github.com/respeaker/usb_4_mic_array.git
cd usb_4_mic_array
sudo python dfu.py --download 6_channels_firmware.bin  # The 6 channels version

How to use

Run executables

roslaunch respeaker_ros respeaker.launch
rostopic echo /sound_direction     # Result of DoA
rostopic echo /sound_localization  # Result of DoA as Pose
rostopic echo /is_speeching        # Result of VAD
rostopic echo /audio               # Raw audio
rostopic echo /speech_audio        # Audio data while speeching

You can also set various parameters via dynamic_reconfigure.

sudo apt install ros-kinetic-rqt-reconfigure  # Install if not
rosrun rqt_reconfigure rqt_reconfigure

To set LED color, publish desired color:

rostopic pub /status_led std_msgs/ColorRGBA "r: 0.0
g: 0.0
b: 1.0
a: 0.3"

Use cases

Voice Recognition

Notes

The configuration file for dynamic_reconfigure in this package is created automatically by reading the parameters from devices. Though it will be rare case, the configuration file can be updated as followings:

Connect the device to the computer.

Run the generator script.

rosrun  respeaker_ros respeaker_gencfg.py

You will see the updated configuration file at $(rospack find respeaker_ros)/cfg/Respeaker.cfg.

Author

Yuki Furuta <[email protected]>

License

Apache License

respeaker_ros's People

Contributors

Stargazers

Watchers

Forkers

uchagani auvilink ashtape jimmy-hu ericchai afdaniele roscn pal-robotics-forks dorothysdad jonasius 3spheresroboticsproject rwth-pem mikhailbertrand rionehome garaemon tutorgaming creedynz jst4apple timbrockmeyer dlozinski atonn alenying jackliang85 homeskies yarulik johnbarrett1964 komgcn jesusmartin86 loctranthanh epikem adacompnus botx-squad clam1110 wookook clearpathrobotics kit-happy-robot happy-isiyama fschoeller varungujjar findques jstar0525 rootadminwalker zacharykratochvil mailonghua sodapeter hgray2020 rchgrav nyxrobotics samaypashine qgsq heycms ioai-tech nu-parcs 360-robotics-solutions

respeaker_ros's Issues

action 'sound_play' is not initialized.

hey, when i run roslaunch respeaker_ros respeaker.launch, i got a mistake,
[ERROR] action 'sound_play' is not initialized.
any suggestions?
your sincerely

Sound source distance

rostopic echo /speech_localization # Result of DoA as Pose

Does this mean it can give out the sound source distance? (not only sound direction of arrival, but the actual distance).

Thanks a lot.

ImportError: No module named sound_play.msg

I launched the rspeaker node and i got import error
The error is as follows:
Traceback (most recent call last):
File "speech_to_text.py", line 11, in
from sound_play.msg import SoundRequest, SoundRequestAction, SoundRequestGoal
ImportError: No module named sound_play.msg

How to solve this issue?

Thank you in advance

ValueError: Invalid audio channels

hey
thanks for your nice work, but recently i have got a knotty problem when a run [roslaunch respeaker_ros respeaker.launch]. output log is shown below. could you give me some advices?Thank you in advance.
your sincerely

[INFO] [1567732215.569230]: Found 8: ReSpeaker 4 Mic Array (UAC1.0): USB Audio (hw:3,0) (channels: 0)
[WARN] [1567732215.569561]: 0 channel is found for respeaker
[WARN] [1567732215.569809]: You may have to update firmware.
Traceback (most recent call last):
File "/home/xbot/mengfei_ws/src/respeaker_ros/scripts/respeaker_node.py", line 428, in
n = RespeakerNode()
File "/home/xbot/mengfei_ws/src/respeaker_ros/scripts/respeaker_node.py", line 323, in init
self.respeaker_audio = RespeakerAudio(self.on_audio, suppress_error=suppress_pyaudio_error)
File "/home/xbot/mengfei_ws/src/respeaker_ros/scripts/respeaker_node.py", line 259, in init
input_device_index=self.device_index,
File "/usr/local/lib/python2.7/dist-packages/pyaudio.py", line 747, in open
stream = Stream(self, *args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/pyaudio.py", line 442, in init
self._stream = pa.open(*arguments)
ValueError: Invalid audio channels
Exception AttributeError: "'RespeakerAudio' object has no attribute 'stream'" in <bound method RespeakerAudio.del of <main.RespeakerAudio object at 0x7f6594adf410>> ignored
[respeaker_node-1] process has died [pid 6879, exit code 1, cmd /home/xbot/mengfei_ws/devel/lib/respeaker_ros/respeaker_node.py __name:=respeaker_node __log:=/home/xbot/.ros/log/18620894-d041-11e9-a3b9-2b9a926590f0/respeaker_node-1.log].
log file: /home/xbot/.ros/log/18620894-d041-11e9-a3b9-2b9a926590f0/respeaker_node-1.log

Any chance to support 4MICs HAT for Raspberry Pi

I have ROS installed on Raspberry Pi with 4MICs array. (4MICs HAT for Raspberry Pi)

Any chance this would support that hardware?

Thanks.

catkin build error

There are so many problelms,I would appreciate your help and advice!

error failed to recognize

I launched respeaker node and getting the following issue
[INFO] [1635785653.227701]: Speech detected for 2.152 seconds
[ERROR] [1635785653.913535]: Failed to recognize:[

respeaker_ros: No definition of [audio_common_msgs] for OS version [stretch]

Trying to install this on Raspberry Pi stretch (ROS installed) with

rosdep install --from-paths src -i -r -n -y

got this error:

respeaker_ros: No definition of [audio_common_msgs] for OS version [stretch]

Any help please.

thanks.

More issue with roslaunch

Hi, Mr. Yuki Furuta

Good job correction the usb permission issue.

Now that problem is gone, I now have this when issuing the roslaunch command.

roslaunch respeaker_ros respeaker.launch
... logging to /home/pi/.ros/log/35988528-6f8f-11e8-bfaf-b827eb05f16d/roslaunch-raspberrypi-955.log
Checking log directory for disk usage. This may take awhile.
Press Ctrl-C to interrupt
Done checking log file disk usage. Usage is <1GB.

started roslaunch server http://192.168.1.77:36061/

SUMMARY

PARAMETERS

/rosdistro: kinetic
/rosversion: 1.12.12

NODES
/
respeaker_node (respeaker_ros/respeaker_node.py)
static_transformer (tf/static_transform_publisher)

auto-starting new master
process[master]: started with pid [965]
ROS_MASTER_URI=http://192.168.1.77:11311

setting /run_id to 35988528-6f8f-11e8-bfaf-b827eb05f16d
process[rosout-1]: started with pid [978]
started core service [/rosout]
process[respeaker_node-2]: started with pid [981]
process[static_transformer-3]: started with pid [982]
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.rear
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.center_lfe
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.side
ALSA lib confmisc.c:1281:(snd_func_refer) Unable to find definition 'cards.bcm2835_alsa.pcm.surround51.0:CARD=0'
ALSA lib conf.c:4528:(_snd_config_evaluate) function snd_func_refer returned error: No such file or directory
ALSA lib conf.c:5007:(snd_config_expand) Evaluate error: No such file or directory
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM surround21
ALSA lib confmisc.c:1281:(snd_func_refer) Unable to find definition 'cards.bcm2835_alsa.pcm.surround51.0:CARD=0'
ALSA lib conf.c:4528:(_snd_config_evaluate) function snd_func_refer returned error: No such file or directory
ALSA lib conf.c:5007:(snd_config_expand) Evaluate error: No such file or directory
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM surround21
ALSA lib confmisc.c:1281:(snd_func_refer) Unable to find definition 'cards.bcm2835_alsa.pcm.surround40.0:CARD=0'
ALSA lib conf.c:4528:(_snd_config_evaluate) function snd_func_refer returned error: No such file or directory
ALSA lib conf.c:5007:(snd_config_expand) Evaluate error: No such file or directory
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM surround40
ALSA lib confmisc.c:1281:(snd_func_refer) Unable to find definition 'cards.bcm2835_alsa.pcm.surround51.0:CARD=0'
ALSA lib conf.c:4528:(_snd_config_evaluate) function snd_func_refer returned error: No such file or directory
ALSA lib conf.c:5007:(snd_config_expand) Evaluate error: No such file or directory
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM surround41
ALSA lib confmisc.c:1281:(snd_func_refer) Unable to find definition 'cards.bcm2835_alsa.pcm.surround51.0:CARD=0'
ALSA lib conf.c:4528:(_snd_config_evaluate) function snd_func_refer returned error: No such file or directory
ALSA lib conf.c:5007:(snd_config_expand) Evaluate error: No such file or directory
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM surround50
ALSA lib confmisc.c:1281:(snd_func_refer) Unable to find definition 'cards.bcm2835_alsa.pcm.surround51.0:CARD=0'
ALSA lib conf.c:4528:(_snd_config_evaluate) function snd_func_refer returned error: No such file or directory
ALSA lib conf.c:5007:(snd_config_expand) Evaluate error: No such file or directory
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM surround51
ALSA lib confmisc.c:1281:(snd_func_refer) Unable to find definition 'cards.bcm2835_alsa.pcm.surround71.0:CARD=0'
ALSA lib conf.c:4528:(_snd_config_evaluate) function snd_func_refer returned error: No such file or directory
ALSA lib conf.c:5007:(snd_config_expand) Evaluate error: No such file or directory
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM surround71
ALSA lib confmisc.c:1281:(snd_func_refer) Unable to find definition 'cards.bcm2835_alsa.pcm.iec958.0:CARD=0,AES0=4,AES1=130,AES2=0,AES3=2'
ALSA lib conf.c:4528:(_snd_config_evaluate) function snd_func_refer returned error: No such file or directory
ALSA lib conf.c:5007:(snd_config_expand) Evaluate error: No such file or directory
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM iec958
ALSA lib confmisc.c:1281:(snd_func_refer) Unable to find definition 'cards.bcm2835_alsa.pcm.iec958.0:CARD=0,AES0=4,AES1=130,AES2=0,AES3=2'
ALSA lib conf.c:4528:(_snd_config_evaluate) function snd_func_refer returned error: No such file or directory
ALSA lib conf.c:5007:(snd_config_expand) Evaluate error: No such file or directory
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM spdif
ALSA lib confmisc.c:1281:(snd_func_refer) Unable to find definition 'cards.bcm2835_alsa.pcm.iec958.0:CARD=0,AES0=4,AES1=130,AES2=0,AES3=2'
ALSA lib conf.c:4528:(_snd_config_evaluate) function snd_func_refer returned error: No such file or directory
ALSA lib conf.c:5007:(snd_config_expand) Evaluate error: No such file or directory
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM spdif
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.hdmi
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.hdmi
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.modem
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.modem
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.phoneline
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.phoneline
ALSA lib confmisc.c:1281:(snd_func_refer) Unable to find definition 'defaults.bluealsa.device'
ALSA lib conf.c:4528:(_snd_config_evaluate) function snd_func_refer returned error: No such file or directory
ALSA lib conf.c:4996:(snd_config_expand) Args evaluate error: No such file or directory
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM bluealsa
ALSA lib confmisc.c:1281:(snd_func_refer) Unable to find definition 'defaults.bluealsa.device'
ALSA lib conf.c:4528:(_snd_config_evaluate) function snd_func_refer returned error: No such file or directory
ALSA lib conf.c:4996:(snd_config_expand) Args evaluate error: No such file or directory
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM bluealsa
connect(2) call to /tmp/jack-1000/default/jack_0 failed (err=No such file or directory)
attempt to connect to server failed

Thanks again.

roslaunch Access denied (insufficient permissions

This is the new ReSpeaker Mic Array v2.0 with raspberry pi (raspbian stretch + ROS Kinetic).

Everything works fine until

roslaunch respeaker_ros respeaker.launch

File "/usr/local/lib/python2.7/dist-packages/usb/backend/libusb1.py", line 595, in _check
raise USBError(_strerror(ret), ret, _libusb_errno[ret])
usb.core.USBError: [Errno 13] Access denied (insufficient permissions)

Any ideas?

Thank you so much.

Something maybe wrong in respeaker_node.py

In the stream_callback function，buffer data was reshape with shape of (length,channels),but actually each channel i is data[i::channels];
also in the stream_callback function, buffer data is transformed into np.int16, but Audiodata.data in common_audio is uint8[], is there anything wrong with the inconsistent data type.

DOA degrees do not correspond to the annotated image

It seems that the published values of DOA range from -180 to 180 instead of 0 to 270 as shown in the README.
Also, the zero degree point is at another location.
Why is it so?

ROS2 support

I've created a ROS2 port with a closely matched API here: https://github.com/hcrlab/respeaker_ros

I'm posting this here because this repo is still the most popular ROS wrapper for the ReSpeaker, though it is not maintained, and never made the noetic transition (#31).

respeaker_node.py typo?

rostopic echo /speech_direction # Result of DoA
rostopic echo /speech_localization # Result of DoA as Pose

In respeaker_node.py these are written as : sound_direction / sound localization

I changed them and now everything finally works !

No module named sound_play.msg

Having issue after launching "roslaunch respeaker_ros respeaker.launch"

happened in speech_to_text.py: from sound_play.msg import

I installed all required package listed in requirement.txt, and have no problem of other respeaker functions except this one

DOA Accuracy Issue.

Hi,
This is not an issue with this ros package, and it is primarily about the Respeaker Mic array V2.
I was wondering whether anyone else is experiencing quite a significant inaccuracy in the DOA detection by Respeaker V2. I have used both this ros package and their own provided python code to get the DOA, but it is usually quite off in the order of about 90 to 180 degrees.
I would appreciate any hint on how to improve it in case anyone has experienced it.
Thank you so much in advance.

Respeaker core 2.0

Hi there does this code work with respeaker core 2.0??

Thank you

ImportError: dynamic module does not define init function (init_tf2)

When I launch respeaker.launch, it gives me error:

NODES
/
respeaker_node (respeaker_ros/respeaker_node.py)
speech_to_text (respeaker_ros/speech_to_text.py)
static_transformer (tf/static_transform_publisher)

ROS_MASTER_URI=http://localhost:11311

process[respeaker_node-1]: started with pid [68053]
process[static_transformer-2]: started with pid [68054]
process[speech_to_text-3]: started with pid [68055]
Traceback (most recent call last):
File "/home/gaurav/catkin_ws2/src/respeaker_ros/scripts/respeaker_node.py", line 12, in
import tf.transformations as T
File "/opt/ros/noetic/lib/python3/dist-packages/tf/init.py", line 30, in
from tf2_ros import TransformException as Exception, ConnectivityException, LookupException, ExtrapolationException
File "/opt/ros/noetic/lib/python3/dist-packages/tf2_ros/init.py", line 38, in
from tf2_py import *
File "/opt/ros/noetic/lib/python3/dist-packages/tf2_py/init.py", line 38, in
from .tf2 import *
ImportError: dynamic module does not define init function (init_tf2)
[respeaker_node-1] process has died [pid 68053, exit code 1, cmd /home/gaurav/catkin_ws2/devel/lib/respeaker_ros/respeaker_node.py __name:=respeaker_node __log:=/home/gaurav/.ros/log/a317667e-c4b7-11ec-bb11-8df75aa53f77/respeaker_node-1.log].
log file: /home/gaurav/.ros/log/a317667e-c4b7-11ec-bb11-8df75aa53f77/respeaker_node-1*.log
Traceback (most recent call last):
File "/home/gaurav/catkin_ws2/src/respeaker_ros/scripts/speech_to_text.py", line 5, in
import actionlib
File "/opt/ros/noetic/lib/python3/dist-packages/actionlib/init.py", line 28, in
from actionlib.action_client import *
File "/opt/ros/noetic/lib/python3/dist-packages/actionlib/action_client.py", line 56, in
import rospy
File "/opt/ros/noetic/lib/python3/dist-packages/rospy/init.py", line 49, in
from .client import spin, myargv, init_node,
File "/opt/ros/noetic/lib/python3/dist-packages/rospy/client.py", line 60, in
import rospy.impl.init
File "/opt/ros/noetic/lib/python3/dist-packages/rospy/impl/init.py", line 54, in
from .tcpros import init_tcpros
File "/opt/ros/noetic/lib/python3/dist-packages/rospy/impl/tcpros.py", line 45, in
import rospy.impl.tcpros_service
File "/opt/ros/noetic/lib/python3/dist-packages/rospy/impl/tcpros_service.py", line 54, in
from rospy.impl.tcpros_base import TCPROSTransport, TCPROSTransportProtocol,
File "/opt/ros/noetic/lib/python3/dist-packages/rospy/impl/tcpros_base.py", line 167
(e_errno, msg, *) = e.args
^
SyntaxError: invalid syntax
[speech_to_text-3] process has died [pid 68055, exit code 1, cmd /home/gaurav/catkin_ws2/devel/lib/respeaker_ros/speech_to_text.py audio:=speech_audio __name:=speech_to_text __log:=/home/gaurav/.ros/log/a317667e-c4b7-11ec-bb11-8df75aa53f77/speech_to_text-3.log].
log file: /home/gaurav/.ros/log/a317667e-c4b7-11ec-bb11-8df75aa53f77/speech_to_text-3*.log
^C[static_transformer-2] killing on exit
shutting down processing monitor...
... shutting down processing monitor complete
done

roslaunch problem on ubuntu 16.04

Hi, Mr. Furuta

In order to just make this work, I switched to a freshly installed Ubuntu 16.04 x86_64 + ROS

After carefully follow all the instructions, the roslaunch line spins out this error:

Traceback (most recent call last):
File "/home/asa/catkin_ws/src/respeaker_ros/scripts/respeaker_node.py", line 385, in
n = RespeakerNode()
File "/home/asa/catkin_ws/src/respeaker_ros/scripts/respeaker_node.py", line 284, in init
self.dyn_srv = Server(RespeakerConfig, self.on_config)
File "/opt/ros/kinetic/lib/python2.7/dist-packages/dynamic_reconfigure/server.py", line 81, in init
self._change_config(self.config, ~0) # Consistent with the C++ API, the callback gets called with level=~0 (i.e. -1)
File "/opt/ros/kinetic/lib/python2.7/dist-packages/dynamic_reconfigure/server.py", line 104, in _change_config
self.config = self.callback(config, level)
File "/home/asa/catkin_ws/src/respeaker_ros/scripts/respeaker_node.py", line 314, in on_config
config[name] = self.respeaker.read(name)
File "/home/asa/catkin_ws/src/respeaker_ros/scripts/respeaker_node.py", line 140, in read
response = struct.unpack(b'ii', response.tostring())
struct.error: unpack requires a string argument of length 8
[respeaker_node-2] process has died [pid 77173, exit code 1, cmd /home/asa/catkin_ws/src/respeaker_ros/scripts/respeaker_node.py __name:=respeaker_node __log:=/home/asa/.ros/log/ed105888-7003-11e8-a5dc-000c297e350d/respeaker_node-2.log].
log file: /home/asa/.ros/log/ed105888-7003-11e8-a5dc-000c297e350d/respeaker_node-2*.log

Did you happen to test it on ubuntu 16.04 ?

Thanks.

Latest respeaker-ros version

Just a note that for those googling I found that version of respeaker-ros that you get via apt comes from https://index.ros.org/p/respeaker_ros/ which in turn pulls from https://github.com/jsk-ros-pkg/jsk_3rdparty.

While that fixes some issues in order to get it working I issued a further Pull request (jsk-ros-pkg/jsk_3rdparty#283)

@furushchev given your repo still comes up as main hit and original source, it may be worth syncing / coordinating with jsk-ros-pkg?

Support for Melodic

Hello, I would like to know if you will upgrade your drive to support Melodic?

Thank you for you support