Add a noise reduction system to Rhasspy and eliminate extraneous sounds about rhasspy HOT 6 OPEN

I'd love to add such a system. Any suggestions on an open source model that can run close to realtime on a Raspberry Pi 4?

from rhasspy.

From several projects, such as: noise-suppression-for-voice pulseaudio, rnnoise-cli, NoiseTorch-ng on respeaker core I managed to run NoiseTorch-ng. NoiseTorch-ng works, removes noise and music, but words are sometimes swallowed, the load on respeaker is 30-40%, also the problem is that you need to run it via remote desktop, I run it through xrdp. According to the instructions, I did not immediately manage to install NoiseTorch-ng, below I will describe how to install and try NoiseTorch-ng

Installing NoiseTorch-ng

You need to install the Golang (Go) compiler first

Download the archive for the armv6 architecture (Raspberry, ReSpeaker)

wget https://go.dev/dl/go1.20.3.linux-armv6l.tar.gz

Unpack the archive in /usr/local

sudo tar -C /usr/local -xzf go1.20.3.linux-armv6l.tar.gz

To use Go, add the Go binary directory to your PATH environment variable. To add the Go binary directory to your PATH environment variable in your specific profile, run the following command

echo "export PATH=/usr/local/go/bin:${PATH}" | sudo tee -a $HOME/.profile

To load the new PATH configuration onto your current login session, run the following command

source $HOME/.profile

To verify that Go is installed correctly and that the PATH environment variable is configured correctly, run the following command in the terminal

go version

After installing the Golang (Go) compiler, we can compile NoiseTorch-ng for our device

git clone https://github.com/noisetorch/NoiseTorch
cd NoiseTorch
make

Installing NoiseTorch-ng

mkdir -p ~/.local/bin
cp ./bin/noisetorch ~/.local/bin/
cp ./assets/noisetorch.desktop ~/.local/share/applications
cp ./assets/icon/noisetorch.png ~/.local/share/icons/hicolor/256x256/apps

With gnome this can be done with:

gtk-update-icon-cache

Give it the required permissions with setcap:

sudo setcap 'CAP_SYS_RESOURCE=+ep' ~/.local/bin/noisetorch

Run noisetorch in a terminal on a remote desktop

~/.local/bin/noisetorch

If I try to run through the icon, an error will come out, so I run NoiseTorch-ng through the console on the desktop by entering the command ~/.local/bin/noisetorch

from rhasspy.

Is it possible to start using a virtual microphone for Rhasspy somehow? After all, a virtual microphone with noise reduction works perfectly. Noises are well cut off, yes, different sounds are heard, but the sound is clear and without noise. Only it does not work when Rhasspy is running, because when Rhasspy is running, the microphones disappear in PulseAudio and the virtual microphone also disappears.

How did I add noise reduction?

We display a list of devices where we can find our microphone

pactl list sources short

or with this command we can find our microphone

pactl list sources short|grep input |awk '{print $2}'|head -1

next, for the test, we will load the noise reduction module with the name MicrofoneNoise, only instead of alsa_input.platform-sound_0.seeed-8ch, specify your microphone

pacmd load-module module-echo-cancel source_name=alsa_input.platform-sound_0.seeed-8ch source_properties=device.description=MicrofoneNoise

We can see our virtual microphone through XRDP

or via the console by entering the command

pactl list sources short

When recording from a microphone, select Microphone Noise

If the record is clean and works well, then add it to /etc/pulse/default.pa

sudo nano /etc/pulse/default.pa

We go down to the very bottom and add a line

#Used to perform acoustic echo cancellation between a designated sink and source
load-module module-echo-cancel source_name=alsa_input.platform-sound_0.seeed-8ch source_properties=device.description=MicrofoneNoise
set-default-source MicrofoneNoise

I tried in asound.conf, which is located on the path /etc/asound.conf, but Rhasspy virtual microphone does not hear, and the hardware microphone hears

pcm.!default {
    type asym
    playback.pcm "speaker"
    capture.pcm "micnoise"
}


pcm.micnoise {
    type plug
    slave {
        rate 48000
        format S32_LE
        pcm "hw:MicrofoneNoise"
    }
}

pcm.speaker {
    type plug
    slave {
        rate 48000
        format S32_LE
        pcm "hw:seeed8micvoicec"
    }
}

from rhasspy.

I tried another noise reduction option and it seems to work not bad. In a noisy place, the speaker reacts and even correctly determines. Of course, this is not an ideal option, but it works

PULSE_PROP="filter.want=echo-cancel" /usr/bin/rhasspy --profile en 2>&1

from rhasspy.

I tested respeaker in various situations and I'll tell you what, yes, it PULSE_PROP="filter.want=echo-cancel" /usr/bin/rhasspy --profile en 2>&1 works. I'm thrilled. It certainly does not work perfectly, but it is much better than without filtering. The speaker began to react in noisy places, which was not the case before. This is my subjective opinion and it may not coincide with yours, but I am impressed with how noise reduction works, even if it is not perfect, but it works the same

My scenarios are how I use the column and will be evaluated in points from 1 to 10.

The noise filter is turned off

• My windows overlook the motorway and there is a lot of noise from cars coming from the street: activation 1 / speech recognition 3
• TV works: Activation 0 / Speech recognition 1
• Music playing: Activation 0 / Speech recognition 1
• Bathroom water noise: Activation 0 / Speech recognition 1

The noise filter is on

• My windows overlook the motorway and there is a lot of noise from cars coming from the street: activation 4 / speech recognition 6
• TV works: Activation 3 / Speech recognition 3
• Music playing: Activation 4 / Speech recognition 5
• Bathroom water noise: Activation 2 / Speech recognition 4

In order for noise reduction to work automatically, you need to add to autorun

[Unit]
Description=Rhasspy Service
After=network-online.target

[Service]
Type=simple
User=respeaker
Group=respeaker
WorkingDirectory=/home/respeaker/.config/rhasspy/
ExecStart=/bin/bash -lc 'PULSE_PROP="filter.want=echo-cancel" /usr/bin/rhasspy --profile en 2>&1 | cat'

StandardOutput=syslog
StandardError=syslog
SyslogIdentifier=rhasspy
Restart=on-failure
RestartSec=30

[Install]
WantedBy=multi-user.target

from rhasspy.

What I did above is not quite right. Yes, it began to work a little better, but it works unstable. It would be correct to add noise reduction to pyaudio hermes as it looks here

Recording file noise reduction
from scipy.io import wavfile
import noisereduce as nr
import pyaudio 
import time
import wave
rate, data = wavfile.read("001.wav")
_,noisy_part =  wavfile.read("noise.wav")
SAMPLING_FREQUENCY=16000
reduced_noise = nr.reduce_noise(y=data, y_noise=noisy_part, sr=SAMPLING_FREQUENCY)

FORMAT = pyaudio.paInt16  
CHANNELS = 1  
RATE = 16000  
RECORD_SECONDS = time  
WAVE_OUTPUT_FILENAME = "out_file.wav" 

with wave.open(WAVE_OUTPUT_FILENAME, 'wb') as wf:
    wf.setnchannels(CHANNELS)
    wf.setsampwidth(2)
    wf.setframerate(RATE)
    wf.writeframes(b''.join(reduced_noise))

I looked at the code in different files that are in the folder rhasspy-microphone-pyaudio-hermes and did not see the code import noisereduce as nr about noise reduction. I'm not a programmer and I don't know how to add noise reduction to pyaudio hermess. Can you try adding noise reduction to rhasspy-microphone-pyaudio-hermes?

There is a noise reduce library
https://pypi.org/project/noisereduce/

There is a ready-made example of noise reduction
https://www.programmersought.com/article/657910759835/

from rhasspy.