Comments (23)
Okay I got a Raspberry Pi and I think I've figured it out. It's that the P/Invokes of the ALSA APIs are assuming 64-bit while Raspbian is 32-bit ARM.
We'll get the fix into the next release. I updated the little sample test app here. To patch this in \psi you could change these two P/Invokes to take uint
(instead of ulong
) and to return int
. That will then only work on 32-bit. We'll be fixing this in a more complete way in the next release.
Thanks again for bringing this bug to our attention!
from psi.
Hi Nicolas,
The Linux versions of the AudioCapture
and AudioPlayer
\psi components are very thin wrappers over the ALSA APIs and depend on the asound
shared object. The symptom of a missing asound
lib seems consistent with something being wrong with your libasound2
installation (apt install libasound2-dev
).
An approach might be to test your audio hardware and asound
installation outside of \psi first. You may record and playback audio with the arecord
and aplay
command line utilities. For example, to record and playback a 10 second test clip:
arecord -f S16_LE -d 10 -r 16000 -D hw:1,0 test.wav
aplay test.wav
You can list available capture (arecord -L
) and playback (aplay -L
) hardware to determine device names. You may want to experiment with sample rates and formats to ensure that your settings are correct (though, you'd be getting a different error if that were the root issue here).
Once in \psi, the AudioCapture
and AudioPlayer
components each take configuration details, as I see you're using, including the DeviceName
(default "plughw:0,0"
) and Format
(default 16KHz, 1 channel, 16-bit PCM).
Hopefully this is helpful. Please let us know how this goes!
from psi.
Hi Ashley,
Thanks a lot for your feedback!
I've tried to run the arecord
and aplay
on the host directly and that seems to work without any issues.
Also the microphone is listed under the available capture hardware list.
I could imagine that there's a package missing in my Container. I've added a few more and now I've installed the following:
- alsa-utils
- libasound2
- libasound2-dev
- libasound2-data
- libasound2-plugins
Could you think of any other package I could try? Or do you have any other idea what could go wrong?
from psi.
I'm at a loss, honestly. I don't see how arecord
and aplay
could work without asound
installed. Our code is merely importing functions from asound
.
Is the libasound.so
in your /lib
or /usr/lib
?
find /usr/lib -name *asound*
Or, if not, is its location listed in /etc/ld.so.conf
or seen by ldconfig -p | grep asound
?
from psi.
It looks like libasound2-dev
wasn't installed properly. However after fixing that, I get the following Exception:
ALSA lib pcm_hw.c:1713:(_snd_pcm_hw_open) Invalid value for card
Unhandled Exception: System.AggregateException: One or more errors occurred. (Pipeline 'default' was terminated because of one or more unexpected errors (Open failed.)) ---> System.AggregateException: Pipeline 'default' was terminated because of one or more unexpected errors (Open failed.) ---> System.ArgumentException: Open failed.
at Microsoft.Psi.Audio.LinuxAudioInterop.Open(String name, Mode mode, Int32 rate, Int32 channels, Format format, Access access)
at Microsoft.Psi.Audio.AudioCapture.Start(Action`1 notifyCompletionTime)
at Microsoft.Psi.Executive.PipelineElement.<Activate>b__46_0()
at Microsoft.Psi.Scheduling.Scheduler.ExecuteAndRelease(SynchronizationLock synchronizationObject, Action action, SchedulerContext context)
--- End of inner exception stack trace ---
at Microsoft.Psi.Pipeline.ThrowIfError()
at Microsoft.Psi.Pipeline.Run(ReplayDescriptor descriptor)
at AudioAnalyzer.Program.Init() in /app/Program.cs:line 93
--- End of inner exception stack trace ---
at System.Threading.Tasks.Task.Wait(Int32 millisecondsTimeout, CancellationToken cancellationToken)
at AudioAnalyzer.Program.Main(String[] args) in /app/Program.cs:line 36
I guess that indicates that asound
is unable to find the Device. When I run arecord -l
I get the following Device:
**** List of CAPTURE Hardware Devices ****
card 1: Device [USB PnP Sound Device], device 0: USB Audio [USB Audio]
Subdevices: 1/1
Subdevice #0: subdevice #0
In my understanding the default DeviceName should be correct: plughw:0,0
. However since the card is 1 I also tried the following: plughw:1,0
. With both I get the same error.
Do you see any other reason why the ALSA lib pcm_hw.c:1713:(_snd_pcm_hw_open) Invalid value for card
could come up?
from psi.
Seems like I didn't set the Container Options correctly that's why Alsa probably wasn't able to find the Device. However after fixing that I still get the following not very detailed Exception:
Unhandled Exception: System.AggregateException: One or more errors occurred. (Pipeline 'default' was terminated because of one or more unexpected errors (Open failed.)) ---> System.AggregateException: Pipeline 'default' was terminated because of one or more unexpected errors (Open failed.) ---> System.ArgumentException: Open failed.
at Microsoft.Psi.Audio.LinuxAudioInterop.Open(String name, Mode mode, Int32 rate, Int32 channels, Format format, Access access)
at Microsoft.Psi.Audio.AudioCapture.Start(Action`1 notifyCompletionTime)
at Microsoft.Psi.Executive.PipelineElement.<Activate>b__46_0()
at Microsoft.Psi.Scheduling.Scheduler.ExecuteAndRelease(SynchronizationLock synchronizationObject, Action action, SchedulerContext context)
--- End of inner exception stack trace ---
at Microsoft.Psi.Pipeline.ThrowIfError()
at Microsoft.Psi.Pipeline.Run(ReplayDescriptor descriptor)
from psi.
We're down to a configuration problem I believe. What name and other parameters work when using arecord
?
The failure to open could be due to the name, mode, rate, channels, format, ... I'd try to get it working with arecord
first and then use those settings in \psi. Maybe less opaque errors.
BTW, I believe that arecord -L
(with capital L
) will display the names (e.g hw:0,0
, hw:0,1
) as well. Example:
$ aplay -L
default:CARD=CK804
NVidia CK804, NVidia CK804
Default Audio Device
front:CARD=CK804,DEV=0 # hw:0,0
NVidia CK804, NVidia CK804
Front speakers
surround40:CARD=CK804,DEV=0 # hw:0,1
NVidia CK804, NVidia CK804
4.0 Surround output to Front and Rear speakers
from psi.
The following works to record a Wave file:
arecord -f S16_LE -d 10 -r 16000 -D hw:1,0 test.wav
I use the following configuration in my code:
Format = WaveFormat.Create(WaveFormatTag.WAVE_FORMAT_PCM, 44100, 16, 1, 2, 88200)
I don't get the names with arecord -L
:
arecord -L
null
Discard all samples (playback) or generate zero samples (capture)
default:CARD=Device
USB PnP Sound Device, USB Audio
Default Audio Device
sysdefault:CARD=Device
USB PnP Sound Device, USB Audio
Default Audio Device
front:CARD=Device,DEV=0
USB PnP Sound Device, USB Audio
Front speakers
surround21:CARD=Device,DEV=0
USB PnP Sound Device, USB Audio
2.1 Surround output to Front and Subwoofer speakers
surround40:CARD=Device,DEV=0
USB PnP Sound Device, USB Audio
4.0 Surround output to Front and Rear speakers
surround41:CARD=Device,DEV=0
USB PnP Sound Device, USB Audio
4.1 Surround output to Front, Rear and Subwoofer speakers
surround50:CARD=Device,DEV=0
USB PnP Sound Device, USB Audio
5.0 Surround output to Front, Center and Rear speakers
surround51:CARD=Device,DEV=0
USB PnP Sound Device, USB Audio
5.1 Surround output to Front, Center, Rear and Subwoofer speakers
surround71:CARD=Device,DEV=0
USB PnP Sound Device, USB Audio
7.1 Surround output to Front, Center, Side, Rear and Woofer speakers
iec958:CARD=Device,DEV=0
USB PnP Sound Device, USB Audio
IEC958 (S/PDIF) Digital Audio Output
dmix:CARD=Device,DEV=0
USB PnP Sound Device, USB Audio
Direct sample mixing device
dsnoop:CARD=Device,DEV=0
USB PnP Sound Device, USB Audio
Direct sample snooping device
hw:CARD=Device,DEV=0
USB PnP Sound Device, USB Audio
Direct hardware device without any conversions
plughw:CARD=Device,DEV=0
USB PnP Sound Device, USB Audio
Hardware device with all software conversions
However I guess if arecord with 1,0 works I have to use plughw:1,0 as DeviceName.
This leaves me with the following Exception:
Unhandled Exception: System.ArgumentException: Read recovery failed.
at Microsoft.Psi.Audio.AudioCapture.<Start>b__15_0()
at System.Threading.Thread.ThreadMain_ThreadStart()
at System.Threading.ThreadHelper.ThreadStart_Context(Object state)
at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
--- End of stack trace from previous location where exception was thrown ---
at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
at System.Threading.ThreadHelper.ThreadStart()
from psi.
To summarize, the following works on your hardware:
arecord -f S16_LE -d 10 -r 16000 -D hw:1,0 test.wav
And the following throws with "Read recovery failed." in \psi:
Format = WaveFormat.Create(WaveFormatTag.WAVE_FORMAT_PCM, 44100, 16, 1, 2, 88200)
\\ BTW, Format = WaveFormat.Create16BitPcm(44100, 1) is equivalent
I'm curious, did arecord
show a message such as "Warning: rate is not accurate (requested = 16000Hz, got = 44100Hz)"? Is that why 44100
is used in the \psi config? Does arecord -r 44100 ...
work?
Also does Name = "hw:1,0"
fail in \psi and/or does -D plughw:1,0
work with arecord
? Basically, trying the exact same settings across arecord
and \psi.
Assuming none of this works, the next thing I can think of is to clone and build \psi locally and debug to see what the actual err
number is from the ALSA APIs here. I've added a work item for myself to add these error numbers to the exceptions we throw throughout to make this easier in the future.
from psi.
The following works with arecord:
arecord -f S16_LE -d 10 -r 44100 -D hw:1,0 test.wav
And the following fails in psi:
DeviceName = "plughw:1,0",
Format = WaveFormat.Create16BitPcm(44100, 1)
Yes exactly I get the warning regarding the 16000Hz that's why I'm using 44100.
And yes I guess I'm referring to the same device.
Also I've analyzed the Header of the wav file I generated with arecord and it looks as expected:
Sample Rate: 44100
Avg Bytes Per Sec: 88200
Bits Per Sample: 16
Num Channels: 1
I'll try to clone the library and check the error number.
from psi.
I cloned the library and added Logs in the LinuxAudioInterop class.
The Read Method call (line 414) returns the following code:
-4574459702921920256
The Recover Method call (line 417) returns the following code:
-256
Hope that helps.
from psi.
Humm... I'm not able to make sense of those. I wish I could repro on my end. My next step would be to try to get a simple C program to run using just the ALSA APIs and then figure out what is wrong with the \psi wrapper. This could very well be a bug in \psi and I feel like I'm having you do my job for me at this point :)
If possible, could you try building and running this sample app?
https://github.com/ashleyF/linux-audio
from psi.
I'm able to run your sample app without any error:
./audioTest
Bit-width: 16
Capturing...
Buf 0: 2 5 3 4 1 6 ...
Buf 1: 13 13 11 12 17 6 ...
Buf 2: 9 9 5 11 9 8 ...
.......
To make sure the problem is not due to an incorrect Device mapping in my container I've built a little sample Net Core Console App to run it on the host directly.
Here's the Code of the sample app:
class Program
{
static void Main(string[] args)
{
Console.WriteLine("Hello World!");
using (var pipeline = Pipeline.Create())
{
var audioCapture = new AudioCapture(pipeline, new AudioCaptureConfiguration
{
DeviceName = "plughw:1,0",
Format = WaveFormat.Create16BitPcm(44100, 1)
});
audioCapture.Do(async (buffer, envelope) => await ProcessAudioMessage(buffer, envelope));
pipeline.Run();
}
}
private static async Task ProcessAudioMessage(AudioBuffer buffer, Envelope envelope)
{
Console.WriteLine($"Received Audio Bytes {buffer.Length}");
}
}
However I get the same output (added some more logs):
Hello World!
plughw:1,0
44100
1
S16LE
Read error number -4574459702921920256
Recover error number -256
Unhandled exception. System.ArgumentException: Read recovery failed.
at Microsoft.Psi.Audio.AudioCapture.<Start>b__15_0() in C:\Users\nicolas.iten\Downloads\psi-master\psi-master\Sources\Audio\Microsoft.Psi.Audio.Linux\AudioCapture.cs:line 138
at System.Threading.ThreadHelper.ThreadStart_Context(Object state)
at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
--- End of stack trace from previous location where exception was thrown ---
at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
at System.Threading.ThreadHelper.ThreadStart()
Aborted
from psi.
Just tested on a different Linux Device (Debian, arm7 32bit).
It still doesn't work but I get a different error number:
./Audio
Hello World!
plughw:0,0
44100
1
S16LE
Read error number 4294967552
Unhandled exception. System.ArgumentException: Read failed.
at Microsoft.Psi.Audio.AudioCapture.<Start>b__15_0() in C:\Users\nicolas.iten\Downloads\psi-master\psi-master\Sources\Audio\Microsoft.Psi.Audio.Linux\AudioCapture.cs:line 138
at System.Threading.ThreadHelper.ThreadStart_Context(Object state)
at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
--- End of stack trace from previous location where exception was thrown ---
at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
at System.Threading.ThreadHelper.ThreadStart()
Aborted
from psi.
Can I see the code producing this "Read error number" message? It seems like some fixednum conversion or sign extending or some such problem with reporting the error code. In fact, a positive error number from snd_pcm_readi()
doesn't even make sense. Only negative numbers are errors. Positive numbers are supposed to be the number of bytes read (but 40GB?!).
None of these error codes are making any sense (e.g. snd_strerror(...)
returns "Unknown error ..."
).
Also, does the sample C app work on this new hardware?
from psi.
It's the same code I posted 2 comments ago. I did all my tests with that one.
arecord works on both devices and if I remember correctly so does your sample C app.
Do you have any idea what else we could test or install?
I'm starting to get out of ideas and I need to get this running as soon as possible.
from psi.
I just added a .NET project to the Linux Audio test repo. This is not using \psi, but is using the same p/invokes directly to the ALSA APIs. There must be a difference between the calls being made by the working C app and either this .NET sample (if it fails) or \psi. Examining the code though, I'm failing to see it...
from psi.
I tested the .NET sample and this one returns the following output and error code:
ALSA Test App
Opened
Params
Params initialized
Params set access
Params set format
Params set rate
Params set channels
Set params
Free params
Prepare handle
Error: Read failed. -4574459702921920384
We even tested with different microphones/soundcards but always the same result.. It works with arecord and the c app but not with the C# apps.
from psi.
Looking closely at the C (working) vs. C# (broken) samples, the only difference I see is that the C version is actually not freeing the snd_pcm_hw_params_t
while the C# version is. Also, one uses "plughw:0,0"
and the other "plughw:1,0"
but I assume you changed that.
Perhaps, could you try uncommenting this line in the C code. If that fails, then perhaps you could try commenting out this line in the C# sample and/or this line in the \psi component.
I'd be interested to hear the result! I guess the theory would be that something under the covers may be holding a reference?
from psi.
Yes, I noticed that as the only difference between the C# and the C sample. And I tried to comment that line out in C# but sadly that didn't change anything. I even checked if all the parameters match, which is the case.
Also I made sure to use the same DeviceName for both apps of course.
Since I lost a lot of time with that issue and I had to come up with a solution I implemented a workaround. If the app is running on Windows I'm using the Psi.Audio Library to read and process the data. However if it's running on a Linux system I'm now using arecord to get the data and Psi.Audio only to process it.
So currently I'm not further investigating the problem and I came up with a solution.
If there's something else I can test out to help fixing the problem let me know!
from psi.
I'm glad you have somewhat of a workaround; capturing outside of \psi with arecord
. The one last thing I might ask is, could you tell me the hardware and Linux distro version (Raspbian, correct? On Raspberry Pi [version?], external microphone?), etc. that you're using and I can perhaps try to repro this.
Thanks so much for pushing on this. We've added ALSA error output to our component (next release) and will hopefully figure this bug out at well.
from psi.
Thank's for putting in so much of effort to try to help me!
We're using the following distro:
Distributor ID: Raspbian
Release: 10
Codename: buster
And the following hardware:
Raspberry Pi 3 Model B Plus Rev 1.3
Microphone: Adafruit Mini USB Microphone
I will keep myself updated regarding this and I'm looking forward to use the psi library in future projects. If you need further information or if I can help in any way you can reach out to me.
from psi.
This is now fixed in the latest release. We're closing this issue now. Thanks once again for bringing this to our attention.
from psi.
Related Issues (20)
- Troubles with RemoteImporter HOT 6
- Will the implementation of "Continual Learning about Objects in the Wild: An Interactive Approach" be opensource? HOT 2
- PSI on Oculus Quest (Android) HOT 4
- Publicizing StereoKitTransforms properties HOT 1
- `NullReferenceException` in MixedReality.cs HOT 1
- Thread-safety bug for the pipeline's stopping state HOT 1
- HoloLens 2 ir cam intrinsics HOT 4
- Fix missing project Microsoft.Psi.Audio.Windows in solution HoloLensCapture.sln
- \psi Studio's 3D panel not rendering properly when using the integrated GPU (but ok with an external monitor) HOT 1
- [Bug]: Hitting an exception when attempting to start hololenscaptureapp in hololens2 using VS2022 HOT 5
- Memory usage of SharedImagePool keeps growing HOT 3
- Poor Serialization Performance for the Diagnostics's `TrackMessageSize` HOT 4
- [Question] Subpipline stop without stopping parent pipline HOT 5
- .NET Native Support
- PsiStoreReader has an inconsistent output for indexed entries on different projects (DotNet vs. UWP) same x64 architecture HOT 2
- Could not load file or assembly 'Microsoft.Psi.Media_Interop.Windows.x64, Version=0.18.72.1, Culture=neutral, PublicKeyToken=null' or one of its dependencies. HOT 2
- PsiStudio not displaying correctly properties with new visualizers HOT 4
- Connecting windows C# and Ubuntu py with interop HOT 2
- If operator to compare to ip producer variables in order to send a specific frame HOT 2
- [whoisusing] MIT Interactive Robotics Group
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from psi.