Forward audio about scrcpy HOT 289 CLOSED

Hi,

I would like to share with you my work-in-progress branch: audio. This branch can be rebased/rewritten at any time.

I implemented it, and it works fine for me! 🎉

I would like early feedbacks so that any major issue can be fixed.

Requirements

• Linux (for now, I have problems to make it work on Windows and MacOS)
• a device supporting AOA2
• the device must be connected via USB (not adb connect)

How to

Install libusb:

sudo apt install libusb-1.0-0 libusb-1.0-0-dev

Checkout the audio branch:

git fetch
git checkout origin/audio

Use a debug build to report problems:

meson d
ninja -C d

Run with audio forwarding enabled:

./run d --forward-audio

The audio should be forwarded.

When closed, the audio forwarding may not be totally disabled. It may require to restart the current audio track (in VLC for example). If it is not sufficient, unplug the device. At worst (but it should not be necessary), disable/enable USB debugging.

How it works

Scrcpy connects to the device over USB to send AOA requests so that the computer becomes an "USB audio accessory" for the device.

This creates a new audio input source on the computer (which can be seen in PulseAudio for example).

Scrcpy records this input source and immediately plays it to the default output source.

Thank you for your feedbacks.

from scrcpy.

Adding this would really making streaming games would so much better. So far all the other existing solutions kinda suck.

from scrcpy.

I just implemented a PoC (separated of scrcpy) to enable audio forwarding from the device to the computer: aoa-audio.

On Debian stable, it "works": the device sound is played on the computer.

This is quite manual though, and several issues must be solved before something similar could be implemented in scrcpy.

Any help is welcome 😉

from scrcpy.

Oh, you're right, I missed the REMOTE_SUBMIX audio source.

from scrcpy.

Is it possible Android Q could make this easier/more reliable? https://developer.android.com/preview/features/playback-capture

from scrcpy.

Even from an app, the source REMOTE_SUBMIX does not work:

AudioRecord: Could not get audio input for record source 8, sample rate 44100, format 0x1, channel mask 0xc, session 73, flags 0

(while it works with MIC for example).

from scrcpy.

(I have no idea how to make it work on Windows at all.)

from scrcpy.

What is the plan of merging this excellent feature into the main branch?

from scrcpy.

As suggested by Juan, one possibility to investigate is using AOA.

I already implemented something using HID over AOA in the past (I even have a publicly available sample), it should be quite similar.

from scrcpy.

AudioRecord may only record input sources (like microphone), not the device audio output.

from scrcpy.

In theory, it should now also work on Windows (in MSYS2/mingw64), with the libusb package:

pacman -S mingw-w64-x86_64-libusb

However, when it try to communicate to open the device over USB on Windows, libusb_open() returns -3, which is (according to libusb_strerror()):

Access denied (insufficient permissions)

I have no idea how to get the permissions granted. Any ideas?

from scrcpy.

On macOS you can install libusb with the following command: brew install libusb

On a MBP 15, macOS 10.13.3 it is not working.

I start the app while a podcast is playing on the device.
After the app starts I get some noise coming out from the computer's speakers, but nothing that I can recognise. At the same time the device's speakers stop emitting a sound.
After about 10 seconds, the scrcpy app stops itself and the podcast sound comes back on the device's speaker.

This behaviour is 100% reproducible on my side.

Here are the command line output:

2018-03-24 23:01:14.865 scrcpy[5291:36915] DEBUG: Device serial is ENU7N15B03004574
2018-03-24 23:01:14.866 scrcpy[5291:36915] DEBUG: USB device with serial ENU7N15B03004574 found: 18d1:4ee7
2018-03-24 23:01:14.867 scrcpy[5291:36915] DEBUG: Device AOA version: 2
2018-03-24 23:01:14.867 scrcpy[5291:36915] INFO: Audio forwarding enabled
2018-03-24 23:01:14.883 scrcpy[5291:36915] INFO: Waiting 2s for USB reconfiguration...
2018-03-24 23:01:17.144 scrcpy[5291:36915] INFO: Selecting input audio source: Built-in Microphone
d/server/scrcpy-server.jar: 1 file pushed. 2.9 MB/s (23985 bytes in 0.008s)
2018-03-24 23:01:17.918 scrcpy[5291:36915] DEBUG: Starting decoder thread
2018-03-24 23:01:17.918 scrcpy[5291:36915] DEBUG: Starting controller thread
2018-03-24 23:01:18.845 scrcpy[5291:36915] INFO: OpenGL shaders: ENABLED
2018-03-24 23:01:18.845 scrcpy[5291:36915] INFO: Created renderer: opengl
2018-03-24 23:01:18.849 scrcpy[5291:36915] INFO: Initial texture: 1440x2560
2018-03-24 23:01:20.662 scrcpy[5291:36915] DEBUG: Unknown touch device id -841524992, cannot reset
2018-03-24 23:01:25.184 scrcpy[5291:37036] DEBUG: End of frames
2018-03-24 23:01:25.188 scrcpy[5291:36915] DEBUG: Video decoder stopped
2018-03-24 23:01:25.188 scrcpy[5291:36915] DEBUG: quit...
2018-03-24 23:01:25.200 scrcpy[5291:36915] DEBUG: Server terminated
2018-03-24 23:01:25.408 scrcpy[5291:36915] DEBUG: USB device with serial ENU7N15B03004574 found: 18d1:4ee7
2018-03-24 23:01:25.409 scrcpy[5291:36915] DEBUG: Device AOA version: 2
2018-03-24 23:01:25.409 scrcpy[5291:36915] INFO: Audio forwarding disabled```

from scrcpy.

@rom1v

For your notice, I set up a VM using Virtual Box and a Ubuntu 18 image and it works.
I can mirror the screen from my tablet with forwarded USB to the VM.
The quality is not so good as on my real Win PC but I think this comes frome the USB.

Now I cloned the VM, so that I can try the audio stuff. 👍

Michael

from scrcpy.

@Ajedi32 Ah great! To be investigated…

from scrcpy.

I developed a separate tool to forward audio over USB (Linux-only): USBAudio (details).

from scrcpy.

That's a great idea! I'll give it a shot when I get a chance (someone will likely beat me to it) although I'm struggling to understand the playback capture docs. It almost looks like you can only capture audio from a specific app (as opposed to recording the entire system.) I'm going to do more research tonight

from scrcpy.

I may be wrong but here is an example of what I thought:

https://github.com/tzutalin/ReadPCMData/blob/master/src/com/da/readpcmtest/MainActivity.java

Sorry I should have added this on my previous comment.

from scrcpy.

I don't think so.

from scrcpy.

@rom1v I tried this on Xperia XA1 Ultra and it works(scrcpy with sound I mean). 👍

from scrcpy.

@n0v4c0r3 Same issue, see #14 (comment).

Instead of guessing the audio device from the name, we should list all audio devices first, enable AOA, then list all audio devices after, then take the diff.

from scrcpy.

I had a bit of a duh moment... I can use the Bluetooth on the computer to get the audio. USB would be ideal but Bluetooth actually works really well

from scrcpy.

I don't understand why scrcpy can't do this when Vysor Pro can. If it takes downloading an app to the device, then that'd be totally fine with me.

from scrcpy.

Is it even possible without a custom rom?

from scrcpy.

May it be possible to use AudioRecord api server side?
Problem is that AudioRecord requires an Activity context, I don't know if we can spawn one by using app_process.

from scrcpy.

Arf, just calling new AudioRecord(…) segfaults, probably because there is no Context (fault addr 0x0).

Note that AudioRecord itself does not require a Context explicitly.

from scrcpy.

Yes... :(
That's why I was wondering if we could create one even if we are using app process.

Audio support is really tricky :/

from scrcpy.

Thank you for the test 👍

Selecting input audio source: Built-in Microphone

It did not select the right input source, either because none were created, either because for now it just selects the last one, assuming it's the most recent (so it should probably be the input source from the device).

I'll check that. Thank you.

from scrcpy.

For your information I've done 2 tests one with a headset connected to the computer and another one without any headset plugged in.

Good luck

from scrcpy.

@eyal-lezmy I investigated on Mac, and it appears that enabling "audio accessory" on the device does not create a new audio input source on the computer.

So unfortunately, I think that the feature will be Linux-only…

I updated the branch to reject the input source if its name does not contain the device model (so that it does not open the built-in microphone for example).

from scrcpy.

Ok so heres a wholly stupid question. Would it work via the Linux environment in windows 10?

from scrcpy.

Would it work via the Linux environment in windows 10?

You tell me 😉

from scrcpy.

Did you test it? Does it work correctly for you?

There are several problems:

• it's a bit hacky/fragile;
• it only works on Linux (but not necessarily a problem);
• if the computer consumes audio at a slightly different rate from the one it is produced by the device (which is not unlikely), the drift will increase and sound becomes very bad, and there is no mechanism to compensate.

So for now, I don't merge it.

Maybe I could, mentionning that it is very experimental...

from scrcpy.

Hi, thanks for the package! I've been enjoying it a lot
I wanted to test the audio forwarding feature out on my machines, but I cannot work out why the program is failing to start with the forward audio enabled.

On my Archlinux machine, I have both libusb and libusb-compat installed besides the dependencies already listed in the AUR package for scrcpy. The testing scrcpy pacakge was compiled with the pre-built server.jar. When attempting to start scrcpy audio forwarding with my Nokia6.1 (Android 8.1, July 1st 2018 patch), the program exits with the following error message:

DEBUG: Device AOA version: 2
INFO: Audio accessory enabled
INFO: Waiting 2s for USB reconfiguration...
DEBUG: Audio input #0: Built-in Audio Analog Stereo
DEBUG: setpriority() failed
DEBUG: Device model is: Nokia 6.1
ERROR: Cannot find the Android accessory audio input source
DEBUG: Disabling audio accessory...
DEBUG: USB device with serial PL2GAMR832306185 found: 18d1:2d05
DEBUG: Device AOA version: 2

Without the --forward-audio flag, the program runs fine.

from scrcpy.

@yimuchen Could you test manually with pactl and ffplay:

$ pactl list short sources
0   alsa_output.pci-0000_00_05.0.analog-stereo.monitor module-alsa-card.c  s16le 2ch 44100Hz   SUSPENDED
1   alsa_input.pci-0000_00_05.0.analog-stereo   module-alsa-card.c s16le 2ch 44100Hz   SUSPENDED

$ ffplay -vn -f pulse -i alsa_input.usb-LGE_Nexus_5_05f5e60a0ae518e5-01.analog-stereo

from scrcpy.

@rom1v I'm a bit new to writing this sort of code, Can you help me with how I should get the string for my device, the alsa_input.usb-LGE_Nexus_5_05f5e60a0ae518e5-01.analog-stereo for my phone?

The output of pactl is:

0       alsa_output.pci-0000_00_1b.0.analog-stereo.monitor      module-alsa-card.c      s16le 2ch 48000Hz       SUSPENDED
1       alsa_input.pci-0000_00_1b.0.analog-stereo       module-alsa-card.c      s16le 2ch 44100Hz       SUSPENDED

On a side note, as a test, I edited the get_matching_audio_device in app/src/audio.c, so that the program uses the found Audio input #0: Built-in Audio Analog Stereo as the audio device, it seems to be able to pass the microphone input of my phone to my PC (I can hear myself talking to the phone from hy PC's headphone), but not the internal audio.

from scrcpy.

Can you help me with how I should get the string for my device, the alsa_input.usb-LGE_Nexus_5_05f5e60a0ae518e5-01.analog-stereo for my phone?

Sorry, I copied incomplete instructions I kept in some textfile.

Instead, follow the steps I wrote in the README of aoa-audio.

from scrcpy.

@rom1v Thanks for instructions! I think the issue that I have is that the udev rules for my phone isn't available yet. The vendor/product ID under lsusb is constantly in flux, and changes everytime I run the ./audio package. I guess I'll try again when I find and updated udev rule : /

from scrcpy.

@rom1v
First, I want to thank you for this awesome application.
It is exactly what I was looking for to mirror the screen of my Huawei Mediapad M5 on the PC and in very high quality.
Neither google cast nor miracast can keep up with the image quality
Better even as an airplay from the Ipad Air 2 to the PC.
I use it with adb over wlan with this cmd command:
scrcpy.exe -s 192.168.10.74:5555 -b 32M -m 1735

So far, I transmit the sound either via cable, which is awkward because the M5 has only a USB-C and no headphone jack, or I use bluetooth but that is not so clever.

Since I work exclusively with Windows and unfortunately with linux little on the hat, I would be glad to test the addressed by you "experimental audio inclusive version". With the knowledge that it does not work 100% yet and that there can be problems.

Thanks again for this great application.
Regards
Michael

from scrcpy.

Since I work exclusively with Windows and unfortunately with linux little on the hat, I would be glad to test the addressed by you "experimental audio inclusive version". With the knowledge that it does not work 100% yet and that there can be problems.

On Windows, it does not work at all. And anyway, when it works, it's only over USB.

from scrcpy.

Hello rom1v,

thanks for reply, if I can help testing on windows I can use USB that will no problem.
What I would say is, that your great app works over WLAN better as the "big known players" and thats great..... RESPECT to your work !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

would it work in a VM???? Will adb work in a VM when USB is passed through??
I did not have any experience with adb on a linux system, I have some experience on Windows
I root Samsung S5 S6 S8, Nexus 6 and the tablet Nexus 7, also Huawei P10 but the mediapad its just new so I dont will do it 😂 all using adb and fastboot.

Michael

from scrcpy.

What I would say is, that your great app works over WLAN better as the "big known players" and thats great..... RESPECT to your work !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

Glad you like it 😉

would it work in a VM???? Will adb work in a VM when USB is passed through??

I didn't try.

from scrcpy.

Hello again,

First:
I have managed that the quality in the VM is quiet the same as on my real Windows PC. I have to enable USB-3.0 mode in the VM.

Second:
as you described I created a debug build.
If I run .\run d it works like the normal build with scrcpy,
but if I use the "-a" flag I got this error messages:

wot@wot-VirtualBox:~/scrcpy$ ./run d -s  NNF6R18525002604  -m 600 -a

DEBUG: Enabling audio accessory...
DEBUG: USB: cannot open device 1d6b:0003 (Access denied (insufficient permissions))
DEBUG: USB device with serial NNF6R18525002604 found: 12d1:107e
DEBUG: Device AOA version: 2
INFO: Audio accessory enabled
INFO: Waiting 2s for USB reconfiguration...
DEBUG: Audio input #0: Eingebautes Tongerät Analog Stereo
DEBUG: setpriority() failed
error: device 'NNF6R18525002604' not found
ERROR: "getprop model" returned with value 1
ERROR: Cannot read Android device model
ERROR: Cannot find the Android accessory audio input source
DEBUG: Disabling audio accessory...
DEBUG: USB: cannot open device 1d6b:0003 (Access denied (insufficient permissions))
DEBUG: USB: cannot open device 80ee:0021 (Access denied (insufficient permissions))
DEBUG: USB: cannot open device 1d6b:0002 (Access denied (insufficient permissions))
ERROR: Cannot find USB device having serial NNF6R18525002604
WARN: Cannot disable audio forwarding
wot@wot-VirtualBox:~/scrcpy$ 

Where can I look into???

MIchael

from scrcpy.

@ml1969 you probably need to add the udev rules for Android device (just install the android-tools-adb package on Debian-based distribution).

from scrcpy.

After I executed the command "./audio 18d1 4ee1 1", can I get the PCM data from the usb handle?

from scrcpy.

@likezjuisee no, it just adds a new input audio source on your computer, that you can play or record.

from scrcpy.

If I want to record the audio data from the android, could you give me some advice? The audio data's format is PCM?

from scrcpy.

After adding udev rules, I got this:

DEBUG: No serial provided, request it to the device
DEBUG: Device serial is ZY224P7LF8
DEBUG: Enabling audio accessory...
DEBUG: USB device with serial ZY224P7LF8 found: 18d1:2d01
DEBUG: Device AOA version: 2
INFO: Audio accessory enabled
INFO: Waiting 2s for USB reconfiguration...
DEBUG: Audio input #0: Built-in Audio Analog Stereo
DEBUG: setpriority() failed
DEBUG: Device model is: moto x4
ERROR: Cannot find the Android accessory audio input source
DEBUG: Disabling audio accessory...
DEBUG: USB device with serial ZY224P7LF8 found: 18d1:2d05
DEBUG: Device AOA version: 2

from scrcpy.

If I want to record the audio data from the android, could you give me some advice? The audio data's format is PCM?

Once AOA audio is enabled, you have a new input source on your computer (e.g. in pulseaudio). You can then play or record it as any input source.
See

• https://github.com/rom1v/aoa-audio
• https://askubuntu.com/questions/834961/sox-and-rec-without-the-mic/835012#835012

You can probably record it with audacity too.

from scrcpy.

@nrfwmd Yes, it does not work with all devices or all computers. That's why I did not merge this branch.

from scrcpy.

@rom1v will rooting my device help me with this?

from scrcpy.

What I really think important for this function is to have a compiled Windows version. Even it only works for few models.
After I advertised this awesome tool to the Chinese community, especially the community of mobile game streamers, all them love it. And all of them asked how can they forward audio. Apparently, most common users are using Windows and this is not a unique phenomenon in some countries. It would be great if we can give them a touchable hope. In addition, spreading your company name to the millions of viewers of these streamers surely is a good and free adverting.

from scrcpy.

So question, if I run this app within an ubuntu VM on a windows host machine would it work O.o (all I want to do is watch this app that only streams to 'mobile devices')

from scrcpy.

@IdrisDose See #14 (comment).

from scrcpy.

If I want to record the audio data from the android, could you give me some advice? The audio data's format is PCM?

Once AOA audio is enabled, you have a new input source on your computer (e.g. in pulseaudio). You can then play or record it as any input source.
See

• https://github.com/rom1v/aoa-audio
• https://askubuntu.com/questions/834961/sox-and-rec-without-the-mic/835012#835012

You can probably record it with audacity too.

I will try, thanks.

from scrcpy.

I rebased the audio on the current master. There were conflicts and things to rewrite, the previous audio branch was written above v1.1. The old one is still available in audio_old.

Nothing changes in the audio forwarding implementation, but it is now implemented above the last version of scrcpy.

from scrcpy.

I exclusively use Gnu/Linux (Manjaro) as I hate windows.
Scrcpy is very good alternative to connect app on windows although it doesn't support audio forwarding like connect app so I use Bluetooth to forward my android audio
Would it be possible to use it via wireless display(cast)?

from scrcpy.

Can you help me with how I should get the string for my device, the alsa_input.usb-LGE_Nexus_5_05f5e60a0ae518e5-01.analog-stereo for my phone?

Sorry, I copied incomplete instructions I kept in some textfile.

Instead, follow the steps I wrote in the README of aoa-audio.

Hmm, I've tested with ffplay and it works as expected. However I still get this error

 ❯ ./run d --forward-audio
d/server/scrcpy-server.jar: 1 file pushed. 3.4 MB/s (69550 bytes in 0.019s)
DEBUG: No serial provided, request it to the device
DEBUG: Device serial is 851QFDSF2259C
DEBUG: Enabling audio accessory...
DEBUG: USB: cannot open device 8087:8001 (Access denied (insufficient permissions))
DEBUG: USB: cannot open device 1d6b:0002 (Access denied (insufficient permissions))
DEBUG: USB: cannot open device 17ef:1012 (Access denied (insufficient permissions))
DEBUG: USB: cannot open device 1d6b:0003 (Access denied (insufficient permissions))
DEBUG: USB: cannot open device 04ca:703c (Access denied (insufficient permissions))
DEBUG: USB: cannot open device 8087:0a2a (Access denied (insufficient permissions))
DEBUG: USB: cannot open device 138a:0017 (Access denied (insufficient permissions))
DEBUG: USB device with serial 851QFDSF2259C found: 18d1:2d01
DEBUG: Device AOA version: 2
INFO: Audio accessory enabled
INFO: Waiting 2s for USB reconfiguration...
DEBUG: Audio input #0: Built-in Audio Analog Stereo
DEBUG: Audio input #1: SDM636-MTP _SN:95743F4E Analog Stereo
DEBUG: Device model is: MEIZU E3
ERROR: Cannot find the Android accessory audio input source
DEBUG: Disabling audio accessory...
DEBUG: USB device with serial 851QFDSF2259C found: 18d1:2d05
DEBUG: Device AOA version: 2

pacman -S android-udev is installed.

from scrcpy.

Cannot find the Android accessory audio input source

The problem is that there is no link between the USB device and the input source created when we enable AOA. So it attempts to guess, based on the names.

Currently, it checks whether the audio device name contains the device model:

It seems it is not the case for your device.

Could you add a debug log to print the name variable (the audio device name)?

from scrcpy.

DEBUG: Audio input #0: Built-in Audio Analog Stereo
DEBUG: Audio input #1: SDM636-MTP _SN:95743F4E Analog Stereo
DEBUG: Device model is: MEIZU E3
DEBUG: Device name is: SDM636-MTP _SN:95743F4E Analog Stereo
DEBUG: Device name is: Built-in Audio Analog Stereo
ERROR: Cannot find the Android accessory audio input source
DEBUG: Disabling audio accessory...
DEBUG: USB device with serial 851QFDSF2259C found: 18d1:2d05
DEBUG: Device AOA version: 2

from scrcpy.

btw, phone calls don't get forwarded when I was testing ffplay. Only music does.

from scrcpy.

OK, so the audio device name is SDM636-MTP _SN:95743F4E and the device model is MEIZU E3.

I don't know how we can match the audio device.

from scrcpy.

lemme try hard coding then

from scrcpy.

hmm, what the heck are these https://la.wentropy.com/GXbK

from scrcpy.

hmm, what the heck are these https://la.wentropy.com/GXbK

#17 (comment)

from scrcpy.

That aur package doesn't help. Well it seems to be harmless errors though.

from scrcpy.

so, i did everything as per instruction bit i dont know why audio is still not coming.
Here's some of the details about it:-

vasu97@BlackPearl:~/scrcpy$ ./run d --forward-audio
[100%] /data/local/tmp/scrcpy-server.jar
DEBUG: No serial provided, request it to the device
DEBUG: Device serial is 307c9a4c
DEBUG: Enabling audio accessory...
DEBUG: USB: cannot open device 05c8:036e (Access denied (insufficient permissions))
DEBUG: USB: cannot open device 0bda:b001 (Access denied (insufficient permissions))
DEBUG: USB: cannot open device 8087:8000 (Access denied (insufficient permissions))
DEBUG: USB: cannot open device 1d6b:0002 (Access denied (insufficient permissions))
DEBUG: USB: cannot open device 1d6b:0003 (Access denied (insufficient permissions))
DEBUG: USB: cannot open device 1d57:ad04 (Access denied (insufficient permissions))
DEBUG: USB: cannot open device 1c4f:0002 (Access denied (insufficient permissions))
DEBUG: USB device with serial 307c9a4c found: 18d1:2d01
DEBUG: Device AOA version: 2
INFO: Audio accessory enabled
INFO: Waiting 2s for USB reconfiguration...
DEBUG: setpriority() failed
DEBUG: Audio input #0: Built-in Audio Analog Stereo
DEBUG: Audio input #1: OnePlus Analog Stereo
DEBUG: Device model is: ONEPLUS A5000
ERROR: Cannot find the Android accessory audio input source
DEBUG: Disabling audio accessory...
DEBUG: USB device with serial 307c9a4c found: 18d1:2d05
DEBUG: Device AOA version: 2

well this is it and i dont know whats wrong.

from scrcpy.

@vasu97 See #14 (comment).

OnePlus Analog Stereo does not contain ONEPLUS A5000.

from scrcpy.

┌─[✗]─[novacore@parrot]─[~/Desktop/scrcpy]
└──╼ $sudo ./run d --forward-audio
d/server/scrcpy-server.jar: 1 file pushed. 2.4 MB/s (26987 bytes in 0.011s)
DEBUG: No serial provided, request it to the device
DEBUG: Device serial is fa41fce77d63
DEBUG: Enabling audio accessory...
DEBUG: USB: device 0cf3:e500 has no serial number available
DEBUG: USB: device 0438:7900 has no serial number available
DEBUG: USB device with serial fa41fce77d63 found: 18d1:2d05
DEBUG: Device AOA version: 2
INFO: Audio accessory enabled
INFO: Waiting 2s for USB reconfiguration...
DEBUG: Audio input #0: Built-in Audio Analog Stereo
DEBUG: Audio input #1: Android Analog Stereo
DEBUG: Device model is: havoc_land
ERROR: Cannot find the Android accessory audio input source
DEBUG: Disabling audio accessory...
DEBUG: USB: device 0cf3:e500 has no serial number available
DEBUG: USB: device 0438:7900 has no serial number available
DEBUG: USB device with serial fa41fce77d63 found: 18d1:2d05
DEBUG: Device AOA version: 2
DEBUG: Event controller stopped

DEVICE: REDMI 3S
device code: land
android 9

any suggestions for me?

from scrcpy.

Mine doesn't even seem to get that far (Oculus Go)

[parker@e55 scrcpy]$ ./run d -a
d/server/scrcpy-server.jar: 1 file pushed. 2.1 MB/s (19186 bytes in 0.009s)
INFO: Audio accessory enabled
INFO: Waiting 2s for USB reconfiguration...
ERROR: No audio input source found

Using the 1.5 server jar.

EDIT: Ok so I screwed up the last one

[parker@e55 scrcpy]$ ./run d --forward-audio
d/server/scrcpy-server.jar: 1 file pushed. 1.7 MB/s (19186 bytes in 0.011s)
DEBUG: No serial provided, request it to the device
DEBUG: Device serial is 1KWPH802328184
DEBUG: Enabling audio accessory...
DEBUG: USB: cannot open device 046d:c24c (Access denied (insufficient permissions))
DEBUG: USB: cannot open device 2516:0004 (Access denied (insufficient permissions))
DEBUG: USB: cannot open device 8087:0024 (Access denied (insufficient permissions))
DEBUG: USB: cannot open device 1d6b:0002 (Access denied (insufficient permissions))
DEBUG: USB: cannot open device 058f:6362 (Access denied (insufficient permissions))
DEBUG: USB device with serial 1KWPH802328184 found: 2833:0083
DEBUG: Device AOA version: 2
INFO: Audio accessory enabled
INFO: Waiting 2s for USB reconfiguration...
ERROR: No audio input source found
DEBUG: Disabling audio accessory...
DEBUG: USB device with serial 1KWPH802328184 found: 2833:0083
DEBUG: Device AOA version: 2

from scrcpy.

Tried the standalone and the device never shows up

[parker@e55 aoa-audio]$ ./audio 2833 0083 1
Device 2833:0083 found. Opening...
Setting audio mode: 1
SUCCESS
[parker@e55 aoa-audio]$ pactl list short sources
0       alsa_output.pci-0000_01_00.1.hdmi-stereo-extra3.monitor module-alsa-card.c      s16le 2ch 44100Hz       SUSPENDED

Right after enabling aoa the MTP disconnects and reconnects

from scrcpy.

Please help me how to install aoa-audio step by step on windows 10 tq

from scrcpy.

On macOS you can install libusb with the following command: brew install libusb

On a MBP 15, macOS 10.13.3 it is not working.

I start the app while a podcast is playing on the device.
After the app starts I get some noise coming out from the computer's speakers, but nothing that I can recognise. At the same time the device's speakers stop emitting a sound.
After about 10 seconds, the scrcpy app stops itself and the podcast sound comes back on the device's speaker.

@eyal-lezmy What are the commands for forwarding audio to mac after installing libusb?

from scrcpy.

On latest updated Debian Stretch with Samsung Galaxy S7 Edge no success like this:

./run d --forward-audio
d/server/scrcpy-server.jar: 1 file pushed. 3.5 MB/s (27439 bytes in 0.008s)
DEBUG: No serial provided, request it to the device
DEBUG: Device serial is ce0117115189590705
DEBUG: Enabling audio accessory...
DEBUG: USB device with serial ce0117115189590705 found: 04e8:6860
DEBUG: Device AOA version: 2
INFO: Audio accessory enabled
INFO: Waiting 2s for USB reconfiguration...
DEBUG: setpriority() failed
DEBUG: Audio input #0: Built-in Audio Analog Stereo
DEBUG: Audio input #1: SAMSUNG_Android Analog Stereo
error: insufficient permissions for device: verify udev rules.
See [http://developer.android.com/tools/device.html] for more information.
ERROR: "getprop model" returned with value 1
ERROR: Cannot read Android device model
ERROR: Cannot find the Android accessory audio input source
DEBUG: Disabling audio accessory...
DEBUG: USB: cannot open device 18d1:2d05 (Access denied (insufficient permissions))
DEBUG: USB: cannot open device 062a:4101 (Access denied (insufficient permissions))
DEBUG: USB: cannot open device 8087:0020 (Access denied (insufficient permissions))
DEBUG: USB: cannot open device 1d6b:0002 (Access denied (insufficient permissions))
DEBUG: USB: cannot open device 18a5:0243 (Access denied (insufficient permissions))
DEBUG: USB: cannot open device 1d6b:0003 (Access denied (insufficient permissions))
DEBUG: USB: cannot open device 1d6b:0002 (Access denied (insufficient permissions))
DEBUG: USB: cannot open device 8087:0020 (Access denied (insufficient permissions))
DEBUG: USB: cannot open device 1d6b:0002 (Access denied (insufficient permissions))
ERROR: Cannot find USB device having serial ce0117115189590705
WARN: Cannot disable audio forwarding

from scrcpy.

On Pop!_OS 19.04
Device: Xiaomi Mi 9 Android Ver. 9
No success.

./run d --forward-audio
d/server/scrcpy-server.jar: 1 file pushed. 0.6 MB/s (19734 bytes in 0.032s)
INFO: Audio accessory enabled
INFO: Waiting 2s for USB reconfiguration...
error: device '66d13e79' not found
ERROR: "getprop model" returned with value 1
ERROR: Cannot read Android device model
ERROR: Cannot find the Android accessory audio input source
ERROR: Cannot find USB device having serial 66d13e79
WARN: Cannot disable audio forwarding

from scrcpy.

Not the best solution. But it works. If it can help .
( Nice about 0-5 m, max 10-15 meters)

from scrcpy.

I assume the custom Bluetooth link is lower latency than normal?

Also can you explain the setup. I see the links in the other post but am not quite sure of what's going on.

from scrcpy.

i would like to fool shazam with sending audio from linux (mpv+pulseaudio) to android and discovering the song.

from scrcpy.

@257 That would be completely the reverse. The current work is trying to get from the device to computer, not the other way around. You could at that point just play the music on the speakers and let the phone pick it up.

from scrcpy.

@parkerlreed right.
although speaker is not an option here. i'm listening to radio at work with my headphones.
it would be cool though if audio could be fed into android/shazam. any idea?

from scrcpy.

@257 Is this an Internet radio station streaming on the computer? Most will also encode the stream with metadata.

Or if it's streaming, just load it on the phone directly.

from scrcpy.

@parkerlreed no, good old bbc music 6 i fetch with get_iplayer

from scrcpy.

@rom1v Would it be possible to forward audio from voice call to the PC?

from scrcpy.

What's the status of this for Windows? It would be great to capture audio from line-in / mic when recording video via scrcpy. At the moment if you want to stream Oculus Quest to a PC and record it then you'd have to run scrcpy and sound recorder at the same time

from scrcpy.

Even from an app, the source REMOTE_SUBMIX does not work:

AudioRecord: Could not get audio input for record source 8, sample rate 44100, format 0x1, channel mask 0xc, session 73, flags 0

(while it works with MIC for example).

REMOTE_SUBMIX seems working as system apps only, maybe making a magisk module?

take a look: https://gitlab.com/vijai/screencam-magisk-module

from scrcpy.

Tried running usbaudio in the linux subsystem for Windows. Couldn't get it running.

• meson is not in ubuntu trusty's package manager
• meson can be installed with python 3.5 (which must be installed from an external ppa) python3.5 -m pip install --user meson
• meson when run with meson x --buildtype=release will complain that ninja is not found
• you can download ninja and add it to /user/bin but meson will still complain that it cannot find ninja, running ninja will complain that it doesn't have root privileges
• running sudo meson x --buildtype=release appears to no longer complain that it ninja is missing but sits stalled for an eternity and never actually completes

from scrcpy.

Was able to get it running by downloading an older version of nijna (1.5.1 to be exact) the end result is:

[ERROR] Could not initialize AOA

from scrcpy.

Android Q added an API to capture audio playback. I'm not familiar with the scrcpy codebase yet, but would it be possible to create a helper app that could record audio and stream that to the scrcpy desktop app?

from scrcpy.

Android Q added an API to capture audio playback.

Yes 👍 (#14 (comment))

I'm not familiar with the scrcpy codebase yet

If you have an Android Q device and want to investigate, creating a "standalone Java app" (cf here) outside scrcpy which captures audio packets (and just log them) would help a lot 👍

I currently don't have time to investigate (and I don't have any Android Q device for the moment), but once the investigation part is done (with a PoC which receives the packets), I could probably work on the scrcpy-specific part.

from scrcpy.

Hi,

I would like to share with you my work-in-progress branch: audio. This branch can be rebased/rewritten at any time.

I implemented it, and it works fine for me!

I would like early feedbacks so that any major issue can be fixed.

Requirements

• Linux (for now, I have problems to make it work on Windows and MacOS)
• a device supporting AOA2
• the device must be connected via USB (not adb connect)

How to

Install libusb:

sudo apt install libusb-1.0-0 libusb-1.0-0-dev

Checkout the audio branch:

git fetch
git checkout origin/audio

Use a debug build to report problems:

meson d
ninja -C d

Run with audio forwarding enabled:

./run d --forward-audio

The audio should be forwarded.

When closed, the audio forwarding may not be totally disabled. It may require to restart the current audio track (in VLC for example). If it is not sufficient, unplug the device. At worst (but it should not be necessary), disable/enable USB debugging.

How it works

Scrcpy connects to the device over USB to send AOA requests so that the computer becomes an "USB audio accessory" for the device.

This creates a new audio input source on the computer (which can be seen in PulseAudio for example).

Scrcpy records this input source and immediately plays it to the default output source.

Thank you for your feedbacks.

For me it's not working, I'm using a Lenovo T430 I purchased on FB marketplace. The guy I bought It from refurbished it a bit, here are the spec's output from the terminal:hardwareinfo.html.txt

and here's the program output from the terminal:

big-dog-boy@ben:~$ sudo apt install libusb-1.0-0 libusb-1.0-0-dev
[sudo] password for big-dog-boy: 
Sorry, try again.
[sudo] password for big-dog-boy: 
Reading package lists... Done
Building dependency tree       
Reading state information... Done
libusb-1.0-0 is already the newest version (2:1.0.22-2).
libusb-1.0-0 set to manually installed.
The following packages were automatically installed and are no longer required:
  fonts-liberation2 fonts-opensymbol gstreamer1.0-gtk3 libabw-0.1-1
  libboost-date-time1.67.0 libboost-filesystem1.67.0 libboost-iostreams1.67.0
  libboost-locale1.67.0 libcdr-0.1-1 libclucene-contribs1v5 libclucene-core1v5
  libcmis-0.5-5v5 libcolamd2 libe-book-0.1-1 libeot0 libepubgen-0.1-1
  libetonyek-0.1-1 libexttextcat-2.0-0 libexttextcat-data libfreehand-0.1-1
  libglew2.1 libgpgmepp6 liblangtag-common liblangtag1 libmessaging-menu0
  libmhash2 libmspub-0.1-1 libmwaw-0.3-3 libmythes-1.2-0 libneon27-gnutls
  libodfgen-0.1-1 liborcus-0.14-0 libpagemaker-0.0-0 libraptor2-0 librasqal3
  librdf0 librevenge-0.0-0 libsdl2-image-2.0-0 libsuitesparseconfig5
  libvisio-0.1-1 libwpd-0.10-10 libwpg-0.3-3 libwps-0.4-4 libxmlsec1
  libxmlsec1-nss libyajl2 linux-headers-5.0.0-23
  linux-headers-5.0.0-23-generic linux-image-5.0.0-23-generic
  linux-modules-5.0.0-23-generic linux-modules-extra-5.0.0-23-generic lp-solve
  uno-libs3 ure
Use 'sudo apt autoremove' to remove them.
The following NEW packages will be installed:
  libusb-1.0-0-dev libusb-1.0-doc
0 upgraded, 2 newly installed, 0 to remove and 28 not upgraded.
Need to get 233 kB of archives.
After this operation, 1,793 kB of additional disk space will be used.
Get:1 http://us.archive.ubuntu.com/ubuntu disco/main amd64 libusb-1.0-0-dev amd64 2:1.0.22-2 [61.7 kB]
Get:2 http://us.archive.ubuntu.com/ubuntu disco/main amd64 libusb-1.0-doc all 2:1.0.22-2 [171 kB]
Fetched 233 kB in 1s (309 kB/s)         
Selecting previously unselected package libusb-1.0-0-dev:amd64.
(Reading database ... 218828 files and directories currently installed.)
Preparing to unpack .../libusb-1.0-0-dev_2%3a1.0.22-2_amd64.deb ...
Unpacking libusb-1.0-0-dev:amd64 (2:1.0.22-2) ...
Selecting previously unselected package libusb-1.0-doc.
Preparing to unpack .../libusb-1.0-doc_2%3a1.0.22-2_all.deb ...
Unpacking libusb-1.0-doc (2:1.0.22-2) ...
Setting up libusb-1.0-doc (2:1.0.22-2) ...
Setting up libusb-1.0-0-dev:amd64 (2:1.0.22-2) ...
big-dog-boy@ben:~$ git fetch

Command 'git' not found, but can be installed with:

sudo apt install git

big-dog-boy@ben:~$ sudo apt install git
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following packages were automatically installed and are no longer required:
  fonts-liberation2 fonts-opensymbol gstreamer1.0-gtk3 libabw-0.1-1
  libboost-date-time1.67.0 libboost-filesystem1.67.0 libboost-iostreams1.67.0
  libboost-locale1.67.0 libcdr-0.1-1 libclucene-contribs1v5 libclucene-core1v5
  libcmis-0.5-5v5 libcolamd2 libe-book-0.1-1 libeot0 libepubgen-0.1-1
  libetonyek-0.1-1 libexttextcat-2.0-0 libexttextcat-data libfreehand-0.1-1
  libglew2.1 libgpgmepp6 liblangtag-common liblangtag1 libmessaging-menu0
  libmhash2 libmspub-0.1-1 libmwaw-0.3-3 libmythes-1.2-0 libneon27-gnutls
  libodfgen-0.1-1 liborcus-0.14-0 libpagemaker-0.0-0 libraptor2-0 librasqal3
  librdf0 librevenge-0.0-0 libsdl2-image-2.0-0 libsuitesparseconfig5
  libvisio-0.1-1 libwpd-0.10-10 libwpg-0.3-3 libwps-0.4-4 libxmlsec1
  libxmlsec1-nss libyajl2 linux-headers-5.0.0-23
  linux-headers-5.0.0-23-generic linux-image-5.0.0-23-generic
  linux-modules-5.0.0-23-generic linux-modules-extra-5.0.0-23-generic lp-solve
  uno-libs3 ure
Use 'sudo apt autoremove' to remove them.
The following additional packages will be installed:
  git-man liberror-perl
Suggested packages:
  git-daemon-run | git-daemon-sysvinit git-doc git-el git-email git-gui gitk
  gitweb git-cvs git-mediawiki git-svn
The following NEW packages will be installed:
  git git-man liberror-perl
0 upgraded, 3 newly installed, 0 to remove and 28 not upgraded.
Need to get 4,969 kB of archives.
After this operation, 33.9 MB of additional disk space will be used.
Do you want to continue? [Y/n] y
Get:1 http://us.archive.ubuntu.com/ubuntu disco/main amd64 liberror-perl all 0.17027-2 [26.6 kB]
Get:2 http://us.archive.ubuntu.com/ubuntu disco/main amd64 git-man all 1:2.20.1-2ubuntu1 [835 kB]
Get:3 http://us.archive.ubuntu.com/ubuntu disco/main amd64 git amd64 1:2.20.1-2ubuntu1 [4,107 kB]
Fetched 4,969 kB in 9s (584 kB/s)                                              
Selecting previously unselected package liberror-perl.
(Reading database ... 218911 files and directories currently installed.)
Preparing to unpack .../liberror-perl_0.17027-2_all.deb ...
Unpacking liberror-perl (0.17027-2) ...
Selecting previously unselected package git-man.
Preparing to unpack .../git-man_1%3a2.20.1-2ubuntu1_all.deb ...
Unpacking git-man (1:2.20.1-2ubuntu1) ...
Selecting previously unselected package git.
Preparing to unpack .../git_1%3a2.20.1-2ubuntu1_amd64.deb ...
Unpacking git (1:2.20.1-2ubuntu1) ...
Setting up liberror-perl (0.17027-2) ...
Setting up git-man (1:2.20.1-2ubuntu1) ...
Setting up git (1:2.20.1-2ubuntu1) ...
Processing triggers for man-db (2.8.5-2) ...
big-dog-boy@ben:~$ git fetch
fatal: not a git repository (or any of the parent directories): .git
big-dog-boy@ben:~$ git checkout origin/audio
fatal: not a git repository (or any of the parent directories): .git

from scrcpy.

Sorry practically all the program output is striked through, I don't know what happened.

from scrcpy.

It's ticks ` and not tildes ~

```
code block
```

from scrcpy.

@BigDogBoy The git commands must be run from the directory where you cloned scrcpy:

git clone https://github.com/Genymobile/scrcpy
cd scrcpy

Anyway, this branch is old. If you want to try this "hack", consider testing USBaudio instead.

from scrcpy.

Any news for Windows?

from scrcpy.

yeah. windows? i think it would be able to work on WSL but it doesnt use adb (that i know of), or have any other server on the windows side of things to be able to access the device. unless we could give wsl direct usb access, we would need something on the windows side to relay.

from scrcpy.

yeah. windows? i think it would be able to work on WSL but it doesnt use adb (that i know of), or have any other server on the windows side of things to be able to access the device. unless we could give wsl direct usb access, we would need something on the windows side to relay.

I tried on WSL I was able to compile the project but it wouldn't run. #14 (comment)

from scrcpy.

I tried on WSL I was able to compile the project but it wouldn't run. #14 (comment)

well it probably doesnt help that it cant access the device 😅
really, does anyone know how to do that, not just as a storage device? is that even possible? 🤔

from scrcpy.

I cant seem to even run ./run d alone (run scrcpy worked tho)
What i did: listed all devices, enable AOA to my phone then run .run d -a
Here is the result. DO u have any idea?

ichirou2910@IK-Inspiron-7559~/scrcpy> sudo ./run d -a
d/server/scrcpy-server.jar: 1 file pushed. 0.5 MB/s (22546 bytes in 0.040s)
DEBUG: No serial provided, request it to the device
DEBUG: Device serial is 5200f616b4c2257f
DEBUG: Enabling audio accessory...
DEBUG: USB: device 8087:0a2a has no serial number available
DEBUG: USB: device 1bcf:28b0 has no serial number available
DEBUG: USB device with serial 5200f616b4c2257f found: 18d1:2d05
DEBUG: Device AOA version: 2
INFO: Audio accessory enabled
INFO: Waiting 2s for USB reconfiguration...
[server] ERROR: Exception on thread Thread[main,5,main]
java.lang.IllegalArgumentException: Expecting 6 parameters
	at com.genymobile.scrcpy.Server.createOptions(Server.java:71)
	at com.genymobile.scrcpy.Server.main(Server.java:132)
	at com.android.internal.os.RuntimeInit.nativeFinishInit(Native Method)
	at com.android.internal.os.RuntimeInit.main(RuntimeInit.java:341)
DEBUG: Audio input #0: Built-in Audio Analog Stereo
DEBUG: Audio input #1: SAMSUNG_Android Analog Stereo
DEBUG: Device model is: SM-J730G
ERROR: Cannot find the Android accessory audio input source
DEBUG: Disabling audio accessory...
DEBUG: USB: device 8087:0a2a has no serial number available
DEBUG: USB: device 1bcf:28b0 has no serial number available
DEBUG: USB device with serial 5200f616b4c2257f found: 18d1:2d05
DEBUG: Device AOA version: 2

from scrcpy.

https://www.incentivespro.com/downloads.html#usb-redirector-linux
GASP! Does anyone know how to get this to work on wsl? M' getting,
*** INSTALLATION FAILED! ??? Kernel sources or kernel headers directory not found. Please install the corresponding package first.

from scrcpy.

In theory, it should now also work on Windows (in MSYS2/mingw64), with the libusb package:

pacman -S mingw-w64-x86_64-libusb

However, when it try to communicate to open the device over USB on Windows, libusb_open() returns -3, which is (according to libusb_strerror()):

Access denied (insufficient permissions)

I have no idea how to get the permissions granted. Any ideas?

Hi, I managed to enable audio accessory mode on Android by changing my phone's usb driver to libusbK using Zadig and your aoa-audio, new audio device appear, but i don't know what the next step is, because first, there is a problem with the audio device driver (showing warning sign that the device cannot start code:10) and second, i'm unable to make pulseaudio on windows to work and i don't know what program that have similar function as pulseaudio on windows.

from scrcpy.

from scrcpy.