spatialaudio / python-rtmixer Goto Github PK

View Code? Open in Web Editor NEW

62.0 14.0 11.0 132 KB

:microphone: Reliable low-latency audio playback and recording with Python :snake:

Home Page: http://python-rtmixer.readthedocs.io/

License: MIT License

C 50.56% Python 47.07% Shell 2.37%

audio real-time low-latency python portaudio

python-rtmixer's Introduction

Realtime Audio Mixer for Python

Warning

This is work in progress!

Goal: Reliable low-latency audio playback and recording with Python, using PortAudio via the sounddevice module.

The audio callback is implemented in C (and compiled with the help of CFFI) and doesn't invoke the Python interpreter, therefore avoiding waiting for things like garbage collection and the GIL.

All PortAudio platforms and host APIs are supported. Runs on any Python version where CFFI is available.

Online documentation

https://python-rtmixer.readthedocs.io/

Source code repository

https://github.com/spatialaudio/python-rtmixer

Somewhat similar projects

python-rtmixer's People

Contributors

Stargazers

Watchers

Forkers

tgarc larsoner theonlyjoey blueff mparkg mgeier bburan tangliufeng ajunlonglive purplesparkle sadernalwis

python-rtmixer's Issues

Input Gain Data

This isn't an issue, per se, but I'm having trouble finding any documentation and hoping someone can help.

I'm trying to use rtmixer to develop software that sends a webhook request any time an input device's gain level crosses a predetermined threshold. I have to imagine this is possible using a combination of rtmixer and sounddevice, as the plot_input.py example file is able to plot the input gain of my mic.

Is there an attribute that I can call that will, at any given moment, return some representation of the relative (or absolute) input gain being collected by a stream?

Thanks in advance!

AttributeError: partially initialized module 'rtmixer' has no attribute 'Mixer' (most likely due to a circular import)

I'm trying to run the examples but I can't get working, it show me this error:
AttributeError: partially initialized module 'rtmixer' has no attribute 'Mixer' (most likely due to a circular import)

I'm on Mac and using python 3

Can't stop execution when coreaudiod restarts. Stale read

I am using examples/signal_processing.py

My system is macos, so I use coreaudio.
When I restart coreaudiod (when running script) by:
sudo launchctl stop com.apple.audio.coreaudiod && sudo launchctl start com.apple.audio.coreaudiod
or simply
sudo killall coreaudiod

process doesn't throw anything and stales in read loop. Throwing exception by myself doesn't help too (it doesn't throw and continues in stale mode).

Same if I delete virtual audio device I used in definition of mixer class.

Python: portaudio can't update audio device [Mac]

I install portaudio by homebrew (brew install portaudio) then I use sounddevice.query_devices() to get all available device on my Mac, but when I change device and recall sounddevice.query_devices() that not update new device.
I tried to use this :
sd._terminate()
Change device:
sd._initialize()
but I can't update new device when I recall sounddevice.query_devices()

How do I fix it in my project? Thanks for your answer!

installation on windows

Hi,

the code compiles without problems on linux and the example program runs fine

however I have not much experience with C;

I have added a stdbool.h file with:

typedef int bool;
#define false 0
#define true 1

but then I get the following error:

C:\Users\p\AppData\Local\Programs\Common\Microsoft\Visual C++ for Python\9.0\VC\Bin\cl.exe /c /nologo /Ox /MD /W3 /GS- /DNDEBUG -UNDEBUG -Isrc -Iportaudio/include -Iportaudio/src/common "-IC:\Program Files (x86)\PsychoPy2\include" "-IC:\Program Files (x86)\PsychoPy2\PC" /Tcbuild\temp.win32-2.7\Release\_rtmixer.c /Fobuild\temp.win32-2.7\Release\build\temp.win32-2.7\Release\_rtmixer.obj
cl : Command line warning D9025 : overriding '/DNDEBUG' with '/UNDEBUG'
_rtmixer.c
build\temp.win32-2.7\Release\_rtmixer.c(459) : error C2275: 'ring_buffer_size_t' : illegal use of this type as an expression
        portaudio/src/common\pa_ringbuffer.h(79) : see declaration of 'ring_buffer_size_t'
build\temp.win32-2.7\Release\_rtmixer.c(459) : error C2146: syntax error : missing ';' before identifier 'written'
build\temp.win32-2.7\Release\_rtmixer.c(459) : error C2065: 'written' : undeclared identifier
build\temp.win32-2.7\Release\_rtmixer.c(461) : error C2065: 'written' : undeclared identifier
build\temp.win32-2.7\Release\_rtmixer.c(480) : warning C4013: 'llround' undefined; assuming extern returning int
build\temp.win32-2.7\Release\_rtmixer.c(494) : error C2143: syntax error : missing ';' before 'type'
build\temp.win32-2.7\Release\_rtmixer.c(494) : error C2143: syntax error : missing ';' before 'type'
build\temp.win32-2.7\Release\_rtmixer.c(494) : error C2143: syntax error : missing ')' before 'type'
build\temp.win32-2.7\Release\_rtmixer.c(494) : error C2143: syntax error : missing ';' before 'type'
build\temp.win32-2.7\Release\_rtmixer.c(495) : error C2065: 'action' : undeclared identifier
build\temp.win32-2.7\Release\_rtmixer.c(496) : error C2143: syntax error : missing ';' before ')'
build\temp.win32-2.7\Release\_rtmixer.c(501) : error C2065: 'i' : undeclared identifier
build\temp.win32-2.7\Release\_rtmixer.c(501) : error C2223: left of '->next' must point to struct/union
build\temp.win32-2.7\Release\_rtmixer.c(503) : error C2065: 'i' : undeclared identifier
build\temp.win32-2.7\Release\_rtmixer.c(503) : error C2065: 'i' : undeclared identifier
build\temp.win32-2.7\Release\_rtmixer.c(503) : error C2223: left of '->next' must point to struct/union
build\temp.win32-2.7\Release\_rtmixer.c(505) : error C2065: 'i' : undeclared identifier
build\temp.win32-2.7\Release\_rtmixer.c(505) : error C2223: left of '->next' must point to struct/union
build\temp.win32-2.7\Release\_rtmixer.c(506) : error C2065: 'action' : undeclared identifier
build\temp.win32-2.7\Release\_rtmixer.c(506) : warning C4047: '=' : 'action *' differs in levels of indirection from 'int'
build\temp.win32-2.7\Release\_rtmixer.c(509) : error C2065: 'state' : undeclared identifier
build\temp.win32-2.7\Release\_rtmixer.c(509) : error C2223: left of '->actions' must point to struct/union
build\temp.win32-2.7\Release\_rtmixer.c(510) : error C2059: syntax error : 'while'
build\temp.win32-2.7\Release\_rtmixer.c(782) : error C2059: syntax error : 'return'
build\temp.win32-2.7\Release\_rtmixer.c(783) : error C2059: syntax error : '}'
build\temp.win32-2.7\Release\_rtmixer.c(903) : error C2065: '_cffi_types' : undeclared identifier
build\temp.win32-2.7\Release\_rtmixer.c(903) : error C2109: subscript requires array or pointer type
build\temp.win32-2.7\Release\_rtmixer.c(903) : error C2065: '_cffi_types' : undeclared identifier
build\temp.win32-2.7\Release\_rtmixer.c(903) : error C2109: subscript requires array or pointer type
build\temp.win32-2.7\Release\_rtmixer.c(903) : warning C4133: 'function' : incompatible types - from 'PyObject *' to '_cffi_ctypedescr *'
build\temp.win32-2.7\Release\_rtmixer.c(903) : warning C4047: 'function' : 'PyObject *' differs in levels of indirection from 'char **'
build\temp.win32-2.7\Release\_rtmixer.c(903) : warning C4024: 'function through pointer' : different types for formal and actual parameter 2
build\temp.win32-2.7\Release\_rtmixer.c(903) : error C2198: 'function through pointer' : too few arguments for call
build\temp.win32-2.7\Release\_rtmixer.c(909) : error C2065: '_cffi_types' : undeclared identifier
build\temp.win32-2.7\Release\_rtmixer.c(909) : error C2109: subscript requires array or pointer type
build\temp.win32-2.7\Release\_rtmixer.c(909) : error C2065: '_cffi_types' : undeclared identifier
build\temp.win32-2.7\Release\_rtmixer.c(909) : error C2109: subscript requires array or pointer type
build\temp.win32-2.7\Release\_rtmixer.c(909) : warning C4133: 'function' : incompatible types - from 'PyObject *' to '_cffi_ctypedescr *'
build\temp.win32-2.7\Release\_rtmixer.c(909) : error C2198: 'function through pointer' : too few arguments for call
build\temp.win32-2.7\Release\_rtmixer.c(949) : error C2065: '_cffi_types' : undeclared identifier
build\temp.win32-2.7\Release\_rtmixer.c(949) : error C2109: subscript requires array or pointer type
build\temp.win32-2.7\Release\_rtmixer.c(949) : error C2065: '_cffi_types' : undeclared identifier
build\temp.win32-2.7\Release\_rtmixer.c(949) : error C2109: subscript requires array or pointer type
build\temp.win32-2.7\Release\_rtmixer.c(949) : warning C4133: 'function' : incompatible types - from 'PyObject *' to '_cffi_ctypedescr *'
build\temp.win32-2.7\Release\_rtmixer.c(949) : warning C4047: 'function' : 'PyObject *' differs in levels of indirection from 'char **'
build\temp.win32-2.7\Release\_rtmixer.c(949) : warning C4024: 'function through pointer' : different types for formal and actual parameter 2
build\temp.win32-2.7\Release\_rtmixer.c(949) : error C2198: 'function through pointer' : too few arguments for call
build\temp.win32-2.7\Release\_rtmixer.c(955) : error C2065: '_cffi_types' : undeclared identifier
build\temp.win32-2.7\Release\_rtmixer.c(955) : error C2109: subscript requires array or pointer type
build\temp.win32-2.7\Release\_rtmixer.c(955) : error C2065: '_cffi_types' : undeclared identifier
build\temp.win32-2.7\Release\_rtmixer.c(955) : error C2109: subscript requires array or pointer type
build\temp.win32-2.7\Release\_rtmixer.c(955) : warning C4133: 'function' : incompatible types - from 'PyObject *' to '_cffi_ctypedescr *'
build\temp.win32-2.7\Release\_rtmixer.c(955) : error C2198: 'function through pointer' : too few arguments for call
build\temp.win32-2.7\Release\_rtmixer.c(988) : error C2065: '_cffi_types' : undeclared identifier
build\temp.win32-2.7\Release\_rtmixer.c(988) : error C2109: subscript requires array or pointer type
build\temp.win32-2.7\Release\_rtmixer.c(988) : error C2065: '_cffi_types' : undeclared identifier
build\temp.win32-2.7\Release\_rtmixer.c(988) : error C2109: subscript requires array or pointer type
build\temp.win32-2.7\Release\_rtmixer.c(988) : warning C4133: 'function' : incompatible types - from 'PyObject *' to '_cffi_ctypedescr *'
build\temp.win32-2.7\Release\_rtmixer.c(988) : warning C4047: 'function' : 'PyObject *' differs in levels of indirection from 'char **'
build\temp.win32-2.7\Release\_rtmixer.c(988) : warning C4024: 'function through pointer' : different types for formal and actual parameter 2
build\temp.win32-2.7\Release\_rtmixer.c(988) : error C2198: 'function through pointer' : too few arguments for call
build\temp.win32-2.7\Release\_rtmixer.c(994) : error C2065: '_cffi_types' : undeclared identifier
build\temp.win32-2.7\Release\_rtmixer.c(994) : error C2109: subscript requires array or pointer type
build\temp.win32-2.7\Release\_rtmixer.c(994) : error C2065: '_cffi_types' : undeclared identifier
build\temp.win32-2.7\Release\_rtmixer.c(994) : error C2109: subscript requires array or pointer type
build\temp.win32-2.7\Release\_rtmixer.c(994) : warning C4133: 'function' : incompatible types - from 'PyObject *' to '_cffi_ctypedescr *'
build\temp.win32-2.7\Release\_rtmixer.c(994) : error C2198: 'function through pointer' : too few arguments for call
build\temp.win32-2.7\Release\_rtmixer.c(1025) : error C2065: '_cffi_types' : undeclared identifier
build\temp.win32-2.7\Release\_rtmixer.c(1025) : error C2109: subscript requires array or pointer type
build\temp.win32-2.7\Release\_rtmixer.c(1025) : error C2065: '_cffi_types' : undeclared identifier
build\temp.win32-2.7\Release\_rtmixer.c(1025) : error C2109: subscript requires array or pointer type
build\temp.win32-2.7\Release\_rtmixer.c(1025) : warning C4133: 'function' : incompatible types - from 'PyObject *' to '_cffi_ctypedescr *'
build\temp.win32-2.7\Release\_rtmixer.c(1025) : warning C4047: 'function' : 'PyObject *' differs in levels of indirection from 'char **'
build\temp.win32-2.7\Release\_rtmixer.c(1025) : warning C4024: 'function through pointer' : different types for formal and actual parameter 2
build\temp.win32-2.7\Release\_rtmixer.c(1025) : error C2198: 'function through pointer' : too few arguments for call
build\temp.win32-2.7\Release\_rtmixer.c(1031) : error C2065: '_cffi_types' : undeclared identifier
build\temp.win32-2.7\Release\_rtmixer.c(1031) : error C2109: subscript requires array or pointer type
build\temp.win32-2.7\Release\_rtmixer.c(1031) : error C2065: '_cffi_types' : undeclared identifier
build\temp.win32-2.7\Release\_rtmixer.c(1031) : error C2109: subscript requires array or pointer type
build\temp.win32-2.7\Release\_rtmixer.c(1031) : warning C4133: 'function' : incompatible types - from 'PyObject *' to '_cffi_ctypedescr *'
build\temp.win32-2.7\Release\_rtmixer.c(1031) : error C2198: 'function through pointer' : too few arguments for call
build\temp.win32-2.7\Release\_rtmixer.c(1075) : error C2065: '_cffi_types' : undeclared identifier
build\temp.win32-2.7\Release\_rtmixer.c(1075) : error C2109: subscript requires array or pointer type
build\temp.win32-2.7\Release\_rtmixer.c(1075) : error C2065: '_cffi_types' : undeclared identifier
build\temp.win32-2.7\Release\_rtmixer.c(1075) : error C2109: subscript requires array or pointer type
build\temp.win32-2.7\Release\_rtmixer.c(1075) : warning C4133: 'function' : incompatible types - from 'PyObject *' to '_cffi_ctypedescr *'
build\temp.win32-2.7\Release\_rtmixer.c(1075) : warning C4047: 'function' : 'PyObject *' differs in levels of indirection from 'char **'
build\temp.win32-2.7\Release\_rtmixer.c(1075) : warning C4024: 'function through pointer' : different types for formal and actual parameter 2
build\temp.win32-2.7\Release\_rtmixer.c(1075) : error C2198: 'function through pointer' : too few arguments for call
build\temp.win32-2.7\Release\_rtmixer.c(1081) : error C2065: '_cffi_types' : undeclared identifier
build\temp.win32-2.7\Release\_rtmixer.c(1081) : error C2109: subscript requires array or pointer type
build\temp.win32-2.7\Release\_rtmixer.c(1081) : error C2065: '_cffi_types' : undeclared identifier
build\temp.win32-2.7\Release\_rtmixer.c(1081) : error C2109: subscript requires array or pointer type
build\temp.win32-2.7\Release\_rtmixer.c(1081) : warning C4133: 'function' : incompatible types - from 'PyObject *' to '_cffi_ctypedescr *'
build\temp.win32-2.7\Release\_rtmixer.c(1081) : error C2198: 'function through pointer' : too few arguments for call
build\temp.win32-2.7\Release\_rtmixer.c(1090) : error C2065: '_cffi_types' : undeclared identifier
build\temp.win32-2.7\Release\_rtmixer.c(1090) : error C2109: subscript requires array or pointer type
build\temp.win32-2.7\Release\_rtmixer.c(1090) : error C2065: '_cffi_types' : undeclared identifier
build\temp.win32-2.7\Release\_rtmixer.c(1090) : error C2109: subscript requires array or pointer type
build\temp.win32-2.7\Release\_rtmixer.c(1090) : warning C4133: 'function' : incompatible types - from 'PyObject *' to '_cffi_ctypedescr *'
build\temp.win32-2.7\Release\_rtmixer.c(1090) : warning C4047: 'function' : 'PyObject *' differs in levels of indirection from 'char **'
build\temp.win32-2.7\Release\_rtmixer.c(1090) : warning C4024: 'function through pointer' : different types for formal and actual parameter 2
build\temp.win32-2.7\Release\_rtmixer.c(1090) : error C2198: 'function through pointer' : too few arguments for call
build\temp.win32-2.7\Release\_rtmixer.c(1096) : error C2065: '_cffi_types' : undeclared identifier
build\temp.win32-2.7\Release\_rtmixer.c(1096) : error C2109: subscript requires array or pointer type
build\temp.win32-2.7\Release\_rtmixer.c(1096) : error C2065: '_cffi_types' : undeclared identifier
build\temp.win32-2.7\Release\_rtmixer.c(1096) : error C2109: subscript requires array or pointer type
build\temp.win32-2.7\Release\_rtmixer.c(1096) : warning C4133: 'function' : incompatible types - from 'PyObject *' to '_cffi_ctypedescr *'
build\temp.win32-2.7\Release\_rtmixer.c(1096) : error C2198: 'function through pointer' : too few arguments for call
build\temp.win32-2.7\Release\_rtmixer.c(1101) : error C2065: '_cffi_types' : undeclared identifier
build\temp.win32-2.7\Release\_rtmixer.c(1101) : error C2109: subscript requires array or pointer type
build\temp.win32-2.7\Release\_rtmixer.c(1101) : error C2065: '_cffi_types' : undeclared identifier
build\temp.win32-2.7\Release\_rtmixer.c(1101) : error C2109: subscript requires array or pointer type
build\temp.win32-2.7\Release\_rtmixer.c(1101) : warning C4133: 'function' : incompatible types - from 'PyObject *' to '_cffi_ctypedescr *'
build\temp.win32-2.7\Release\_rtmixer.c(1101) : warning C4047: 'function' : 'PyObject *' differs in levels of indirection from 'char **'
build\temp.win32-2.7\Release\_rtmixer.c(1101) : warning C4024: 'function through pointer' : different types for formal and actual parameter 2
build\temp.win32-2.7\Release\_rtmixer.c(1101) : error C2198: 'function through pointer' : too few arguments for call
build\temp.win32-2.7\Release\_rtmixer.c(1107) : error C2065: '_cffi_types' : undeclared identifier
build\temp.win32-2.7\Release\_rtmixer.c(1107) : error C2109: subscript requires array or pointer type
build\temp.win32-2.7\Release\_rtmixer.c(1107) : error C2065: '_cffi_types' : undeclared identifier
build\temp.win32-2.7\Release\_rtmixer.c(1107) : error C2109: subscript requires array or pointer type
build\temp.win32-2.7\Release\_rtmixer.c(1107) : warning C4133: 'function' : incompatible types - from 'PyObject *' to '_cffi_ctypedescr *'
build\temp.win32-2.7\Release\_rtmixer.c(1107) : error C2198: 'function through pointer' : too few arguments for call
build\temp.win32-2.7\Release\_rtmixer.c(1112) : error C2065: '_cffi_types' : undeclared identifier
build\temp.win32-2.7\Release\_rtmixer.c(1112) : error C2109: subscript requires array or pointer type
build\temp.win32-2.7\Release\_rtmixer.c(1112) : error C2065: '_cffi_types' : undeclared identifier
build\temp.win32-2.7\Release\_rtmixer.c(1112) : error C2109: subscript requires array or pointer type
build\temp.win32-2.7\Release\_rtmixer.c(1112) : warning C4133: 'function' : incompatible types - from 'PyObject *' to '_cffi_ctypedescr *'
build\temp.win32-2.7\Release\_rtmixer.c(1112) : warning C4047: 'function' : 'PyObject *' differs in levels of indirection from 'char **'
build\temp.win32-2.7\Release\_rtmixer.c(1112) : warning C4024: 'function through pointer' : different types for formal and actual parameter 2
build\temp.win32-2.7\Release\_rtmixer.c(1112) : error C2198: 'function through pointer' : too few arguments for call
build\temp.win32-2.7\Release\_rtmixer.c(1118) : error C2065: '_cffi_types' : undeclared identifier
build\temp.win32-2.7\Release\_rtmixer.c(1118) : error C2109: subscript requires array or pointer type
build\temp.win32-2.7\Release\_rtmixer.c(1118) : error C2065: '_cffi_types' : undeclared identifier
build\temp.win32-2.7\Release\_rtmixer.c(1118) : fatal error C1003: error count exceeds 100; stopping compilation
error: command 'C:\\Users\\p\\AppData\\Local\\Programs\\Common\\Microsoft\\Visual C++ for Python\\9.0\\VC\\Bin\\cl.exe' failed with exit

any ideas?

cheers
piotr

alternative architecture for low latency audio streaming

I just thought it might be relevant to discuss an alternative implementation of low latency audio streaming I've been working towards.

The general idea can be explained in a few points:

Audio callback is implemented in GIL-less C and does nothing but read and write data from and to a ring buffer (no processing)
Python user communicates data to the audio callback via a cffi wrapped portaudio ring buffer.
Audio processing is done using reader and writer threads. If low latency is not the highest priority, these can be implemented using python threads. Otherwise, they can be implemented using C or better yet, through Cython using with nogil.

My intention is to make a flexible architecture whose feature set can be extended relatively easily.

This is what I've been trying to achieve with this python module I've been working on: https://github.com/tgarc/pastream/tree/ringbuffer. Currently, I'm still using a few python threading synchronization primitives to synchronize with the audio callback since it made coding a bit simpler, but my intention is to move towards a scheme that doesn't require acquiring in the GIL in the callback.

Anyway, I just wanted to bounce these ideas off you and see if anything sticks :)

Ringbuffer underflow/overflow

I decided to make an issue here to summarize my issues/requests with rtmixer, perhaps for future fixes/updates.
In general, what I am trying to do is to execute an audio processing algorithm in real-time using rtmixer. I've already implemented this with jack-client, but, since rtmixer is better for many reasons, I'm trying to do the same with rtmixer.

As discussed previously in the jack-client repo, my main issue is that, in cases where my processing algorithm is too slow, the audio stops. The cause is that the ringbuffer is empty (underflow) and the playback action is removed by the rtmixer:

python-rtmixer/src/rtmixer.c

Lines 388 to 393 in d7095f2

    
           if (totalsize < (ring_buffer_size_t)frames) 
        
           { 
        
             // Ring buffer is empty or full 
        
             remove_action(actionaddr, state); 
        
             continue; 
        
           }

The thing here is that this happens even in cases where it shouldn't. What I mean is that, for a blocksize of 1024 samples and a sampling rate of 16 kHz for instance, the processing algorithm should have 60 ms available at the worst case. However, with a processing algorithm that takes about 40-50 ms I always get a ringbuffer underflow (I also put a printf inside the rtmixer.c if to make sure this was the cause). In fact, I measured the required time for the processing inside python and the worst case is that it takes 51 ms to execute. However, the audio playback always stops (ringbuffer underflows) and the 'weird' thing is that it always happens after 4 frames for my script (maybe it's not weird, I just haven't understood the reason yet). You can find the framework of my script at the end of this issue.
On the other hand, this is solved if I double the pre-filling of the queue, but that increases latency and in my case is the last solution. I also tried increasing the latency from 'low' to 'high' and also the MixerAndRecorder blocksize to 0 but nothing helped (in fact setting blocksize=0 gives immediately a ringbuffer underflow and I still haven't found the cause). Still, what's troubling me is whether the buffer underflow makes sense for a processing algorithm that is at least 10 ms faster than the limitation.
From various timings that I did there seems to be some delay each time in the filling of the input queue with the first frame, which may be causing this buffer underflow (I haven't found the exact reason yet). However, if I increase the pre-filling (also for the input queue) and set the blocksize=0 it works. Of course if a slow processing for a frame occurs it stops (because of a buffer underflow), but that makes sense as we said. Correct me if got this wrong but, by setting blocksize=0, no matter how much I pre-fill the queues I would assume that after a while the playback catches up to the case where no pre-fill existed (the latency difference is lost).

Another thing that would be really good is if there was a feature added to the API that could allow for the playback to continue after ringbuffer overflows/underflows. My C programming skills are not that great so I wasn't able to check somehow the buffer underflows and implement this from scratch, but I guess it would make sense to have a flag in the python API that will allow ringbuffer over/underflow in the rtmixer playback.

#!/usr/bin/env python

from __future__ import division, print_function
from time import time,sleep
import rtmixer
import sys
import numpy as np

blocksize = 1024

latency = 'low'
samplerate = 16000
channels = 1
qin_size = 4
q_size = 4*qin_size

stream = rtmixer.MixerAndRecorder(
    channels=channels, blocksize=blocksize, #blocksize=0?
    latency=latency, samplerate=samplerate)
with stream:
    print('  input latency:', stream.latency[0])
    print(' output latency:', stream.latency[1])
    print('            sum:', sum(stream.latency))
    print('requested delay:',timeout)

    samplesize = 4
    assert {samplesize} == set(stream.samplesize)

    qin = rtmixer.RingBuffer(samplesize * channels, qin_size * blocksize)
    record_action = stream.record_ringbuffer(qin)

    q = rtmixer.RingBuffer(samplesize * channels, q_size * blocksize)
    buffer = np.zeros((blocksize,1),dtype='float32') # or q_size*blocksize?
    q.write(buffer)
    play_action = stream.play_ringbuffer(q)

    try:
        while True:
            while qin.read_available < blocksize:
                if record_action not in stream.actions:
                    break
                sleep(0.001)
            if record_action not in stream.actions:
                break
            read, buf1, buf2 = qin.get_read_buffers(blocksize)
            t = time()
            buffer = np.frombuffer(buf1, dtype='float32')
            noisy[0,:,0] = buffer
            # processing of 'noisy' is performed here and 'clean' is computed
            buffer = clean.ravel() #.astype('float32')
            qin.advance_read_index(blocksize)
            while q.write_available < blocksize:
                if play_action not in stream.actions:
                    print('Ringbuffer underflow')
                    break
                sleep(0.001)
            if play_action not in stream.actions:
                print('Ringbuffer underflow')
                break
            q.write(buffer)
            print(time()-t) # measure processing time
    except KeyboardInterrupt:
        print('\nInterrupted by User')

Jitter in the actual_time of the action

Hi,

I am currently working on a project where I'd like to sync some timestamps, if possible with sub-millisecond precision. I cannot use the allow_belated = False because that would end up not doing the action (running on Windows and with other application open which may delay too much).

Therefore I needed to find the time at which the sound is really played. To play the signal I use a RingBuffer (for looping capabilities) and register the action. I then wait until the action has an actual time and use it as my ground 0.

stream = rtmixer.Mixer()
action = stream.play_ringbuffer(ringbuffer)

while action.actual_time == 0:
    pass

realStart = action.actual_time

However if I use a signal that is set (for example 2 sines of 500ms that start at 0.5s and 1.5s respectively) and use LSL to record both the output of my sound card and timestamps that signal the start and stop of the previous sines, I get a jitter in the accuracy of the actual_time around 6ms.
I can only think of it coming from the actual_time as the measurement within the loop are set timestamps.

Q: Bit-perfect output

I want to be able to reliably get 32-bit integer values out of the sound card digital channel, such as 256 or 1024 (or higher powers of 2). This is problematic because float32 cannot represent all integers up to 2 ** 31 - 1 without losing precision:

np.float32(2 ** 31 - 1) == np.float32(2147483647) == 2147483600

So far I've tried staying in float64 precision and scaling by 1. / float(2 ** 31 - 1) to try to avoid as much precision loss as possible before forcing into float32 format for rtmixer but it seems like some triggers are still being missed.

Using sounddevice and int32 format does not suffer from this problem. Hence more reliable solutions for rtmixer would be:

Support double, which can represent 32-bit integers without loss of precision, and be careful about conversion to int32 for PortAudio (or whatever format it wants, not sure if it will take double directly and convert).
Support int32 directly.

(2) seems more in line with what sounddevice does, in that it already supports int32 format directly, so seems like a path of least resistance.

Either way, we could duplicate all callback code to achieve double or int32 this but it's ugly. To avoid code dup it seems reasonable to move to Cython (my preference) or C++ templates (I probably could not do this) for the callback implementation. If this seems reasonable, I can try:

Converting the current C code to Cython using cdef in one PR (don't change functionality, just change how the code is written and compiled), then
Use Cython fused types to implement an int32 callback.

Thoughts @mgeier ?

BUG: Jitter on Windows machines

I have tested python-rtmixer on Linux and found (with a scope) sub-millisecond jitter between a parallel port trigger and the audio start, assuming I call:

mixer.play_buffer(..., start=mixer.time + fixed_delay)

with a fixed_delay that is sufficiently large, i.e. probably bigger than the sound card ALSA buffer (on Linux with my built-in sound card I use 10 ms but could probably go lower).

On Windows, however, I cannot get this behavior on any backend (WDM, MME, WASAPI) or multiple sound cards (motherboard built-in Realtek ALC1220, or some USB sound cards). There is always some jitter, even when I set the fixed delay. I can see the delay works, because as I increase it, the latency increases. But jitter never goes down.

Is PortAudio/rtmixer on Windows supposed to behave this way? Any ideas why this might be happening?

arm-linux-gnueabihf-gcc: error: portaudio/src/common/pa_ringbuffer.c: No such file or directory

I have tried to install the latest version of rtmixer in two computers and in both cases I get the error below. Specifically, I did:
git clone https://github.com/spatialaudio/python-rtmixer
cd python-rtmixer
python3 setup.py develop --user

An the error produced is:

running develop
running egg_info
writing dependency_links to src/rtmixer.egg-info/dependency_links.txt
writing src/rtmixer.egg-info/PKG-INFO
writing requirements to src/rtmixer.egg-info/requires.txt
writing top-level names to src/rtmixer.egg-info/top_level.txt
reading manifest file 'src/rtmixer.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'portaudio/LICENSE.txt'
warning: no files found matching 'portaudio/index.html'
warning: no files found matching 'portaudio/src/common/pa_ringbuffer.h'
warning: no files found matching 'portaudio/src/common/pa_memorybarrier.h'
writing manifest file 'src/rtmixer.egg-info/SOURCES.txt'
running build_ext
generating cffi module 'build/temp.linux-armv7l-3.5/_rtmixer.c'
already up-to-date
building '_rtmixer' extension
arm-linux-gnueabihf-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -g -fdebug-prefix-map=/build/python3.5-6waWnr/python3.5-3.5.3=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -UNDEBUG -Isrc -Iportaudio/include -I/usr/include/python3.5m -c build/temp.linux-armv7l-3.5/_rtmixer.c -o build/temp.linux-armv7l-3.5/build/temp.linux-armv7l-3.5/_rtmixer.o
arm-linux-gnueabihf-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -g -fdebug-prefix-map=/build/python3.5-6waWnr/python3.5-3.5.3=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -UNDEBUG -Isrc -Iportaudio/include -I/usr/include/python3.5m -c portaudio/src/common/pa_ringbuffer.c -o build/temp.linux-armv7l-3.5/portaudio/src/common/pa_ringbuffer.o
arm-linux-gnueabihf-gcc: error: portaudio/src/common/pa_ringbuffer.c: No such file or directory
arm-linux-gnueabihf-gcc: fatal error: no input files
compilation terminated.
error: command 'arm-linux-gnueabihf-gcc' failed with exit status 1

With the command python3 -m pip install -e . --user I get a similar error:

Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Obtaining file:///home/pi/python-rtmixer
Requirement already satisfied: CFFI>=1 in /home/pi/.local/lib/python3.5/site-packages (from rtmixer==0.0.0) (1.11.5)
Requirement already satisfied: pa_ringbuffer in /home/pi/.local/lib/python3.5/site-packages (from rtmixer==0.0.0) (0.1.3)
Requirement already satisfied: sounddevice>0.3.9 in /home/pi/.local/lib/python3.5/site-packages (from rtmixer==0.0.0) (0.3.12)
Requirement already satisfied: pycparser in /home/pi/.local/lib/python3.5/site-packages (from CFFI>=1->rtmixer==0.0.0) (2.19)
Installing collected packages: rtmixer
  Found existing installation: rtmixer 0.0.0
    Uninstalling rtmixer-0.0.0:
      Successfully uninstalled rtmixer-0.0.0
  Running setup.py develop for rtmixer
    Complete output from command /usr/bin/python3 -c "import setuptools, tokenize;__file__='/home/pi/python-rtmixer/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" develop --no-deps --user --prefix=:
    running develop
    running egg_info
    writing requirements to src/rtmixer.egg-info/requires.txt
    writing top-level names to src/rtmixer.egg-info/top_level.txt
    writing dependency_links to src/rtmixer.egg-info/dependency_links.txt
    writing src/rtmixer.egg-info/PKG-INFO
    reading manifest file 'src/rtmixer.egg-info/SOURCES.txt'
    reading manifest template 'MANIFEST.in'
    warning: no files found matching 'portaudio/LICENSE.txt'
    warning: no files found matching 'portaudio/index.html'
    warning: no files found matching 'portaudio/src/common/pa_ringbuffer.h'
    warning: no files found matching 'portaudio/src/common/pa_memorybarrier.h'
    writing manifest file 'src/rtmixer.egg-info/SOURCES.txt'
    running build_ext
    generating cffi module 'build/temp.linux-armv7l-3.5/_rtmixer.c'
    already up-to-date
    building '_rtmixer' extension
    arm-linux-gnueabihf-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -g -fdebug-prefix-map=/build/python3.5-6waWnr/python3.5-3.5.3=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -UNDEBUG -Isrc -Iportaudio/include -I/usr/include/python3.5m -c build/temp.linux-armv7l-3.5/_rtmixer.c -o build/temp.linux-armv7l-3.5/build/temp.linux-armv7l-3.5/_rtmixer.o
    arm-linux-gnueabihf-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -g -fdebug-prefix-map=/build/python3.5-6waWnr/python3.5-3.5.3=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -UNDEBUG -Isrc -Iportaudio/include -I/usr/include/python3.5m -c portaudio/src/common/pa_ringbuffer.c -o build/temp.linux-armv7l-3.5/portaudio/src/common/pa_ringbuffer.o
    arm-linux-gnueabihf-gcc: error: portaudio/src/common/pa_ringbuffer.c: No such file or directory
    arm-linux-gnueabihf-gcc: fatal error: no input files
    compilation terminated.
    error: command 'arm-linux-gnueabihf-gcc' failed with exit status 1
    
    ----------------------------------------
  Rolling back uninstall of rtmixer
Command "/usr/bin/python3 -c "import setuptools, tokenize;__file__='/home/pi/python-rtmixer/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" develop --no-deps --user --prefix=" failed with error code 1 in /home/pi/python-rtmixer/

I didn't have any problems installing rtmixer before, I guess that the portaudio submodule moved or is inaccessible? Sorry for keeping you busy!

installation fails on Linux (can't find portaudio.h)

pip install rtmixer fails on Linux 20.04. Setup:

$ conda create -c conda-forge -n testrtmixer python=3
$ conda activate testrtmixer
$ pip install rtmixer

This yields the following error:

    build/temp.linux-x86_64-3.8/_rtmixer.c:569:10: fatal error: portaudio.h: No such file or directory
      569 | #include <portaudio.h>
          |          ^~~~~~~~~~~~~
    compilation terminated.

...but the library is definitely installed:

$ apt search ^libportaudio2
Sorting... Done
Full Text Search... Done
libportaudio2/focal,now 19.6.0-1build1 amd64 [installed]
  Portable audio I/O - shared library

...and yet portaudio.h is not there:

$ sudo find / -name portaudio.h  # no output except a couple "permission denied" lines in `/run/usr`

If I install portaudio through conda:

$ conda install -c conda-forge portaudio  # it also installs alsa-lib
$ pip install rtmixer

it still yields the same error. This time I'll paste the full output:

Building wheels for collected packages: rtmixer
  Building wheel for rtmixer (setup.py) ... error
  ERROR: Command errored out with exit status 1:
   command: /opt/miniconda3/envs/testrtmixer/bin/python3.8 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-ciif4x1a/rtmixer/setup.py'"'"'; __file__='"'"'/tmp/pip-install-ciif4x1a/rtmixer/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-419j4fk6
       cwd: /tmp/pip-install-ciif4x1a/rtmixer/
  Complete output (21 lines):
  running bdist_wheel
  running build
  running build_py
  creating build
  creating build/lib.linux-x86_64-3.8
  copying src/rtmixer.py -> build/lib.linux-x86_64-3.8
  running build_ext
  generating cffi module 'build/temp.linux-x86_64-3.8/_rtmixer.c'
  creating build/temp.linux-x86_64-3.8
  building '_rtmixer' extension
  creating build/temp.linux-x86_64-3.8/build
  creating build/temp.linux-x86_64-3.8/build/temp.linux-x86_64-3.8
  creating build/temp.linux-x86_64-3.8/portaudio
  creating build/temp.linux-x86_64-3.8/portaudio/src
  creating build/temp.linux-x86_64-3.8/portaudio/src/common
  gcc -pthread -B /opt/miniconda3/envs/testrtmixer/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -Isrc -Iportaudio/include -I/opt/miniconda3/envs/testrtmixer/include/python3.8 -c build/temp.linux-x86_64-3.8/_rtmixer.c -o build/temp.linux-x86_64-3.8/build/temp.linux-x86_64-3.8/_rtmixer.o --std=c99
  build/temp.linux-x86_64-3.8/_rtmixer.c:569:10: fatal error: portaudio.h: No such file or directory
    569 | #include <portaudio.h>
        |          ^~~~~~~~~~~~~
  compilation terminated.
  error: command 'gcc' failed with exit status 1
  ----------------------------------------
  ERROR: Failed building wheel for rtmixer
  Running setup.py clean for rtmixer
Failed to build rtmixer
Installing collected packages: rtmixer
    Running setup.py install for rtmixer ... error
    ERROR: Command errored out with exit status 1:
     command: /opt/miniconda3/envs/testrtmixer/bin/python3.8 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-ciif4x1a/rtmixer/setup.py'"'"'; __file__='"'"'/tmp/pip-install-ciif4x1a/rtmixer/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-m7ceidgi/install-record.txt --single-version-externally-managed --compile --install-headers /opt/miniconda3/envs/testrtmixer/include/python3.8/rtmixer
         cwd: /tmp/pip-install-ciif4x1a/rtmixer/
    Complete output (21 lines):
    running install
    running build
    running build_py
    creating build
    creating build/lib.linux-x86_64-3.8
    copying src/rtmixer.py -> build/lib.linux-x86_64-3.8
    running build_ext
    generating cffi module 'build/temp.linux-x86_64-3.8/_rtmixer.c'
    creating build/temp.linux-x86_64-3.8
    building '_rtmixer' extension
    creating build/temp.linux-x86_64-3.8/build
    creating build/temp.linux-x86_64-3.8/build/temp.linux-x86_64-3.8
    creating build/temp.linux-x86_64-3.8/portaudio
    creating build/temp.linux-x86_64-3.8/portaudio/src
    creating build/temp.linux-x86_64-3.8/portaudio/src/common
    gcc -pthread -B /opt/miniconda3/envs/testrtmixer/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -Isrc -Iportaudio/include -I/opt/miniconda3/envs/testrtmixer/include/python3.8 -c build/temp.linux-x86_64-3.8/_rtmixer.c -o build/temp.linux-x86_64-3.8/build/temp.linux-x86_64-3.8/_rtmixer.o --std=c99
    build/temp.linux-x86_64-3.8/_rtmixer.c:569:10: fatal error: portaudio.h: No such file or directory
      569 | #include <portaudio.h>
          |          ^~~~~~~~~~~~~
    compilation terminated.
    error: command 'gcc' failed with exit status 1
    ----------------------------------------
ERROR: Command errored out with exit status 1: /opt/miniconda3/envs/testrtmixer/bin/python3.8 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-ciif4x1a/rtmixer/setup.py'"'"'; __file__='"'"'/tmp/pip-install-ciif4x1a/rtmixer/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-m7ceidgi/install-record.txt --single-version-externally-managed --compile --install-headers /opt/miniconda3/envs/testrtmixer/include/python3.8/rtmixer Check the logs for full command output.

What am I doing wrong here?

Execute multiple actions at exactly the same time

Currently, if a common start time that's far enough in the future (and allow_belated=False) is used for multiple actions, all those actions will start at exactly the given time (or not at all if they are too late).

Calculating an appropriate start time that's "as soon as possible" but still late enough to make sure none of the actions are late is basically impossible.

I would like to be able to start multiple actions at the same time, without having to come up with an appropriate start time.

I've already prepared for that in the callback function, which allows having linked lists of actions in the action queue, which are checked in this loop:

python-rtmixer/src/rtmixer.c

Lines 72 to 91 in 6147a3a

    
           do 
        
           { 
        
             struct action* next = new_action->next; 
        
             if (new_action->type == CANCEL) 
        
             { 
        
               new_action->next = state->actions; 
        
               state->actions = new_action; 
        
             } 
        
             else 
        
             { 
        
               new_action->next = NULL; 
        
               while (*last_action_addr) 
        
               { 
        
                 last_action_addr = &((*last_action_addr)->next); 
        
               } 
        
               *last_action_addr = new_action; 
        
             } 
        
             new_action = next; 
        
           } 
        
           while (new_action);

This guarantees that all actions in the same list are handled in the same invocation of the callback function. And if they have time=0 (or any time less than the current time) and allow_belated=True, they will all be guaranteed to start at the beginning of the same audio block (assuming the callback finishes in time).

So far so good, the problem is how to implement this on the Python side without duplicating a lot of code.

I imagine that for each action there would be a pair of Python methods, one that immediately enqueues the action (like the current methods), and one that only prepares the action and concatenates it with other actions that have been "prepared" before into a linked list.
When all desired actions have been "prepared", the user can then "trigger" all actions at once by enqueuing the whole list.

I think the first type of methods could be implemented in terms of the second type by immediately "triggering" at the end of the call.

Ideally, the current functionality should be unaffected by this.

compilation errors; c99 requirement

So I'm getting two compilation errors. First one is a straightfoward C99 requirement:

  build/temp.linux-x86_64-2.7/_rtmixer.c: In function 'callback':
  build/temp.linux-x86_64-2.7/_rtmixer.c:495: error: 'for' loop initial declarations are only allowed in C99 mode
  build/temp.linux-x86_64-2.7/_rtmixer.c:495: note: use option -std=c99 or -std=gnu99 to compile your code
  build/temp.linux-x86_64-2.7/_rtmixer.c:567: error: 'for' loop initial declarations are only allowed in C99 mode
  build/temp.linux-x86_64-2.7/_rtmixer.c:667: error: 'for' loop initial declarations are only allowed in C99 mode
  build/temp.linux-x86_64-2.7/_rtmixer.c:682: error: 'for' loop initial declarations are only allowed in C99 mode
  build/temp.linux-x86_64-2.7/_rtmixer.c:712: error: 'for' loop initial declarations are only allowed in C99 mode
  build/temp.linux-x86_64-2.7/_rtmixer.c:722: error: 'for' loop initial declarations are only allowed in C99 mode
  build/temp.linux-x86_64-2.7/_rtmixer.c:742: error: 'for' loop initial declarations are only allowed in C99 mode
  build/temp.linux-x86_64-2.7/_rtmixer.c:752: error: 'for' loop initial declarations are only allowed in C99 mode
  error: command 'gcc' failed with exit status 1

Which can be fixed by just setting extra_compile_args=['-std=c99'] in rtmixer_build.py.

The other bug is a little more confusing:

_rtmixer.c: In function 'callback':
_rtmixer.c:517: error: 'struct action' has no member named 'action'
_rtmixer.c:518: error: 'struct action' has no member named 'action'
_rtmixer.c:569: error: 'struct action' has no member named 'action'
_rtmixer.c:661: error: 'struct action' has no member named 'buffer'
_rtmixer.c:705: error: 'struct action' has no member named 'ringbuffer'
_rtmixer.c:729: error: 'struct action' has no member named 'ringbuffer'
_rtmixer.c:735: error: 'struct action' has no member named 'ringbuffer'
_rtmixer.c:759: error: 'struct action' has no member named 'ringbuffer'
_rtmixer.c: In function '_cffi_checkfld_struct_action':
_rtmixer.c:1586: error: 'struct action' has no member named 'buffer'
_rtmixer.c:1587: error: 'struct action' has no member named 'ringbuffer'
_rtmixer.c:1588: error: 'struct action' has no member named 'action'
_rtmixer.c: At top level:
_rtmixer.c:1682: error: 'struct action' has no member named 'buffer'
_rtmixer.c:1683: error: 'struct action' has no member named 'buffer'
_rtmixer.c:1685: error: 'struct action' has no member named 'ringbuffer'
_rtmixer.c:1686: error: 'struct action' has no member named 'ringbuffer'
_rtmixer.c:1688: error: 'struct action' has no member named 'action'
_rtmixer.c:1689: error: 'struct action' has no member named 'action'

Looks like each of these fields are inside a union inside the action struct. Maybe this is a result of platform dependent behavior, not really sure.

I'm running this on red hat 6.7 with python 2.7.11, cffi 1.9.1, gcc 4.4.7 (yup our compiler is old)

EDIT
ahah, I spoke a little too soon. Apparently anonymous structs/unions are not allowed in C99 (however they are allowed in gnu99). See this SO post.

ENH: Wheel builds

@mgeier if you can add me with write permissions (probably?) I can set up Azure CIs. I've done it for a number of repos and it's a bit of a pain, so having me do it is probably the easiest thing. And I'll need the email you use(d) to sign up for an Azure account in order to give you permissions.

	if (totalsize < (ring_buffer_size_t)frames)
	{
	// Ring buffer is empty or full
	remove_action(actionaddr, state);
	continue;
	}

	do
	{
	struct action* next = new_action->next;
	if (new_action->type == CANCEL)
	{
	new_action->next = state->actions;
	state->actions = new_action;
	}
	else
	{
	new_action->next = NULL;
	while (*last_action_addr)
	{
	last_action_addr = &((*last_action_addr)->next);
	}
	*last_action_addr = new_action;
	}
	new_action = next;
	}
	while (new_action);