Currently the video capture module converts the frames to I420 and the H264 encoder converts the frames to NV12. These conversions are not needed if the camera provides the frames in NV12.
Additionally, the MF might support pixel formats other than NV12 without requiring double conversions.

This issue was fixed in m80 by the patch file: 0005-Fixing-UWP-build-for-time_utils.cc.patch

Describe the bug
Camera selection doesn't work, the first camera is always used

To Reproduce

1. Call webrtc::VideoCaptureFactory::Create(device_id) with device_id being any video capture device other than the first one enumerated.

Expected behavior
device_id is actually used and cameras other that the first enumerated one can be opened.

Desktop (please complete the following information):

• OS: Windows (UWP)

Additional context
It appears VideoCaptureWinRT::_deviceUniqueId is assigned but never used.

Describe the bug

1. When opening the cloned Project and build on Visual Studio. During the build Visual Studio could find the nugget package WIN2D, CPP/Winrt
2. .Visual Studio fails to build after a deprecated C++17 feature being used by abseil
3. Visual Studio can't find webrtc headers and compiled library generated with instructions from the GitHub for m84.

In case line 329 is reached, meaning that a suitable MediaFrameSource could not be found, the code will go into an infinite loop. It should be advancing the iterator before looping.

Hi,

I have a question regarding the old sdk and since that repo is deprecated I thought I might ask here. I also wonder if you would suggest me to move on to the new sdk instead. If this is the wrong place please let me know. I could not find anywhere in the old repo to ask this question.

I have been using the now deprecated werbrtc uwp sdk. It has worked well most of the time and have made webrtc implementation in uwp a lot easier for me. :) At this moment however I am having an issue with packet loss causing video framerate to drop permanently. It shows in that video works great until a packet or a few are lost. When this happens framerate drop and delay in video are introduced. This drop in quality is never improved until a new call is made.
Audio is never an issue though. I was just wondering if this was a known issue and if there are any suggestions on how I would solve it? Since I only bind my mediaelement to the incoming track I really have no control over the incoming frames as far as I know.

Any suggestion or idea on how to solve my problem would be greatly appreciated. Would it be recommended to move onto using this sdk instead or is it too early?

Thanks for the great work!
Regards
Gustav

Im building Windows app using .net core, hope this is a Good choice.

I'm getting the following problem when I try to compile VideoConferencing project:

Package 'Microsoft.WinRTC.libwebrtc.84.0.13000001-alpha' is not found on source

Could you share a road map or a list of regressions/missing features?

Installing pywin in the depot_tools is required for building for Arm64.

The current patches fix WebRTC for x86 and x64, but not Arm64. It would be good have an offering without relying on clang-cl.

The MixedReality SDK is blocked on use of winrtc, as noted here.

The MixedReality-WebRTC from Microsoft also provides a wrapper on libwebrtc. What Is the difference between that project and this one?

Is this summary correct?

• winrtc provides a WinRT interface to libwebrtc (plus XAML control and nuget package)
• mrwebrtc (Mixed Reality WebRTC) provides a UWP interface to libwebrtc ` (plus nuget package)

Is there any likelihood that this project will ever work with Microsoft Teams so that a native app can make/receive calls to Teams given that Teams is using WebRTC under the hood?

A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
As is, WebRTC.org doesn't build for Modern Windows. WebRTC.org does have a port for Win32, but it doesn't build for Modern Windows (ie. UWP).

It would be good to have patch files fixing WebRTC.org in order to have it building for x64 and x86.

It causes header misalignment across projects not using Ninja.

VideoCaptureWinRTInternal::StopCapture uses a 250 milliseconds timeout when invoking media_frame_reader_->StopAsync method. This actually seems to cause a crash whenever the timeout is actually reached, and I guess that this isn't the wanted behavior.

https://chromium-review.googlesource.com/c/chromium/src/+/2117732

SIMD optimizations had to be turned off during the port of WebRTC m84 over WinRTC. Leveraging the changes from m80 weren't enough and SIMD had to be turned off to unblock partner. Turning off the libjpeg-turbo SIMD's for ARM64 didn't impact use of the ARM64 device during tests.

I'm developing an application in ReactNative for Windows for Hololens and need to use WebRTC. How can i do it? I've found repo with WinRTC and it seems good for me. But how to connect with this project from RN ? Is there any example?

Does this project support WPF?

Deploying the Arm64 patches successfully produce an app capable of establishing a P2P connection. However, after few seconds, the app crashes.

Is your feature request related to a problem? Please describe.
I'm the creator of a open-source remote support solution: https://github.com/lucent-sea/Remotely

For remote control on the Windows agent, I'm using MixedReality WebRTC's external video source to stream the desktop (in combination with SharpDX to capture the frames). It'd be nice to have desktop capture included in the WinRTC API, which I'd use to replace MR WebRTC.

Describe the solution you'd like
The ideal solution would look similar to how Electron does it: https://www.electronjs.org/docs/api/desktop-capturer

There'd be an API to enumerate display devices and to return a video stream of a specified display, then add it as a track to the peer connection.

Describe alternatives you've considered
I'm currently using MixedReality.WebRTC's ExternalVideoTrackSource and SharpDX.

Additional context
Add any other context or screenshots about the feature request here.

Hi all,

Thanks for this great project. I managed to build webrtc project with patches, but I couldn't find documentation to build nuget packages to ease development to be able to work with sample project. So if you could provide such documentation it will be great

Is your feature request related to a problem? Please describe.
We're currently using webrtc-uwp-sdk and its windows-specific bits in a project that needs hardware-accelerated encoding and decoding. I've seen the issue for importing the decoder already and wanted to ask if the encoder is going to be imported into this project as well.

It'd be nice if we could upgrade to a newer WebRTC version because I'd like to test the new Insertable Streams API.

At the moment the only way to use MF H.264 is to pass specific codec factories to the PeerConnectionFactory initialization. This means that you can't mix MF H.264 and other codecs, since there is no factory supporting both.

It should be easy enough to make a proxy factory forwarding to either the MF H.264 one or the default one depending on the codec requested. This is what WebRTC-UWP used to do AFAIK (so this is technically a regression).

See #29 (comment)

Allocating a new buffer for each frame has a significant performance hit on devices with limited CPU. See webrtc-uwp/webrtc-apis@ced46e4 for a fix. Use webrtc::I420BufferPool in common_video/include/i420_buffer_pool.h.

Greetings,

1. I have successfully build all.sln after applying patch M84. (no errors, tried twice, ninja and VS2019, followed steps to the letter)
2. I have opened VideoConferencing.sln and both sample projects loaded successfully.
3. After a while I saw that paths in projects were configured for winrtc project to be on the same hierarchical level as webrtc project folder. I assured that that is the case.
4. I see that some folders and nuget packets are missing, packages are not being able to be restored.
5. There is no C:\webrtc\src\out\msvc\uwp\Release\x64\ packages folder

as required in project file for multiple files, example

<Import Project="..\..\..\webrtc\src\out\msvc\uwp\Release\x64\packages\Microsoft.Windows.CppWinRT.2.0.200703.9\build\native\Microsoft.Windows.CppWinRT.props" Condition="Exists('..\..\..\webrtc\src\out\msvc\uwp\Release\x64\packages\Microsoft.Windows.CppWinRT.2.0.200703.9\build\native\Microsoft.Windows.CppWinRT.props')" />

6. There is no \packages\Microsoft.WinRTC.libwebrtc.84.0.13000001-alpha\build\native\Microsoft.WinRTC.libwebrtc.props in Samples directory as required in VideoConferencing project.

As I can see something is wrong with paths in Microsoft.WinRTC.libwebrtc.uwp.nuspec. It seams that package,cmd in .pipelines directory never executes + paths are wrong.

Am I missing something?

Thank you in advance.

Building after applying the patches in the m84 directory generates plenty of errors. See attached log for a comprehensive list (the used args.gn is also attached).
errors.zip

The patch at patches_for_WebRTC_org\m84\3001-Removing-code-that-only-runs-on-Win32-from-the-build.patch doesn't apply to rtc_base/platform_thread_types.cc on the m84 branch - the corresponding code is significantly different.

Removing that part of the patch makes the patch go through. libwebrtc builds for UWP (x64 Debug at least) after that, but I am not sure if the unpatched code actually works as intended on UWP too.

Describe the bug
When trying to compile the solution I'm getting the following error message: "Cannot open include file: 'api/media_stream_interface.h': No such file or directory Microsoft.WinRtc.Simple.VideoConferencing"

To Reproduce
Steps to reproduce the behaviour:

1. Download source code from master branch
2. Open solution MyFirstWinRtc
3. Rebuild solution
4. See error

Expected behaviour
Compile successfully.

Screenshots

Desktop (please complete the following information):

• OS: Windows 10
• Visual Studio 2019 V16.6.2

I am running MR-WebRTC built on top of the m84 version of WinRTC, from the Unity editor (so Win32 platform) on x64. The issue seems to happen both when using Debug and Release.

When calling StopCapture() on a webrtc::VideoCaptureModule previously created with webrtc::VideoCaptureFactory::Create(const char* deviceUniqueIdUTF8), I get this warning

(help_functions_winrt.cc:430): Waiting in a non-MTA thread. Deadlocks might occur.

and after that the thread indeed hangs forever and the process freezes.

The callstack is:

mrwebrtc.dll!webrtc::videocapturemodule::WaitForAsyncAction(ABI::Windows::Foundation::IAsyncAction * p_async_action=0x000001ec79b79d30) Line 430	C++
mrwebrtc.dll!webrtc::videocapturemodule::VideoCaptureWinRTInternal::StopCapture() Line 407	C++
mrwebrtc.dll!webrtc::videocapturemodule::VideoCaptureWinRT::StopCapture() Line 675	C++
(MR-WebRTC stuff follows)

The value of apt_type is APTTYPE_MAINSTA.

I seem to get this both in Debug and in Release (in Release without the error message).

I'm running Windows 2004 on a MacBook Pro 2019, using VMware Fusion.
Built-in Camera app works properly with VMware virtual camera drivers, but VideoCaptureWinRTInternal fails to read frames from it.
All the initialization part goes pretty well, the implementation automatically selects kYUV2 over kI420 (that would be my preference), but then VideoCaptureWinRTInternal::FrameArrived fails to decode the bitmap: bitmap_buffer->GetPlaneCount returns 1 rather than 2 causing the whole thing to fail.
This seems to happen also on physical devices (to say, not VMs), but I can't confirm it's the same issue.

Describe the bug
Audio recording doesn't shutdown properly, and therefore cannot be restarted.

To Reproduce
Steps to reproduce the behavior:

1. Attach an audio track to an audio transceiver
2. Establish a connection
3. Remove the audio track from the transceiver
4. Re-add the audio track on the transceiver

Expected behavior
In step 3., audio stops being transmitted. Then in step 4. it starts again and the remote peer can hear the microphone recording again like after step 2.

Desktop (please complete the following information):

• OS: win32

Additional context
AudioDeviceWindowsCore::RecordingIsInitialized() returns false after AudioDeviceWindowsCore::StopRecording() was called, but the _audioClient is left untouched, still initialized. When the WebRTC engine calls AudioDeviceWindowsCore::StartRecording() again in step 4., the code attempts to initialize _audioClient a second time, which logically fails with AUDCLNT_E_ALREADY_INITIALIZED.

To Reproduce
Steps to reproduce the behavior:

1. Follow the instructions here
2. Build with Visual Studio 2019
3. Search for "Permissions denied" or "The command "call ninja [...]" exited with code 1" in the output logs
4. See error

Expected behavior
Building correctly

Logs
12> c:\webrtc\src\third_party\boringssl\src\crypto\bio\socket.c : fatal error C1083: Non è possibile aprire il file generato dal compilatore: 'c:\webrtc\src\out\msvc\uwp\Release\x64\obj\third_party\boringssl\boringssl\socket.obj': Permission denied
12>ninja : warning : premature end of file; recovering
12>c:\webrtc\src\third_party\boringssl\src\crypto\bio\socket.c : fatal error C1083: Non è possibile aprire il file generato dal compilatore: 'c:\webrtc\src\out\msvc\uwp\Release\x64\obj\third_party\boringssl\boringssl\socket.obj': Permission denied
12>c:\webrtc\src\out\msvc\uwp\Release\x64\obj\api\audio_codecs\builtin_audio_encoder_factory.vcxproj(66,5): error MSB3073: The command "call ninja.exe -C c:\webrtc\src\out\msvc\uwp\Release\x64\ obj/api/audio_codecs/builtin_audio_encoder_factory.lib" exited with code 1.

There are not all the logs that Visual Studio prints but it's full of them just like these.

Desktop (please complete the following information):

• OS: Windows 10
• VisualStudio Version: Microsoft Visual Studio Community 2019 Version 16.7.2

Additional context
I also tried running everything as administrator, but it didn't work.

It would be great if WinRTC could have built-in module for screen capture based on GraphicsCapturePicker and Direct3D11CaptureFramePool.

Cameras on different devices support different capabilities including the set of supported capture resolutions, frame rate for video captures, and whether HDR or variable frame rate captures are supported.
It is up to the device driver to expose this capability via camera profiles or not. WinRTC should support all the camera configurations exposed by the UWP platform.
For more info: https://docs.microsoft.com/en-us/windows/uwp/audio-video-camera/camera-profiles

Hello! First of all thank you for making libwebrtc easy to compile on Windows. I have a need and I'm not intimately familiar with WinRTC/libwebrtc to know so I'm asking:

Perhaps this is possible, and if so, I'd like to know how, but I can't find the right interface or object to access the RTP video data as-is as it comes in?

I was looking at your AppRtc sample and saw there is an OnFrame callback:

In my scenario I am receiving H264 video tracks over WebRTC and I would like to read the RTP payload data as it comes and I don't want it to be decoded, I want to handle it myself with my own video pipeline (which is already able to decode H264 quickly, so I really don't want to decode it and encode it again...)

How would I access that with WinRTC? Any clue or hint will be appreciated.

I build lib follow to https://docs.microsoft.com/en-us/winrtc/getting-started.
T added lib in project use VS 2017 with mediasoupclient.
Code:
auto* device = new mediasoupclient::Device();

if(device->CanProduce("audio"))
{
   cout <<"sdfgasdg";
}
return 0;

and error:
Severity Code Description Project File Line Suppression State
Error LNK2019 unresolved external symbol ___std_reverse_copy_trivially_copyable_4 referenced in function "float * __cdecl std::reverse_copy<float const *,float *>(float const *,float const *,float *)" (??$reverse_copy@PBMPAM@std@@YAPAMPBM0PAM@Z) mediasoup D:\callSys\mediasoup\build\webrtc.lib(auto_correlation.obj) 1

Describe the bug
When opening the sample app from the cloned Winrtc folder document the path to the Webrtc.lib is incorrect which result in an link error.

When compiling a VS 2019 project including api\video_codecs\video_encoder.h from m80 with the /permissive- option, the included header absl\container\internal\compressed_tuple.h fails to compile.

This seems to be a VS bug and it's fixed in the preview: https://developercommunity.visualstudio.com/content/problem/841781/error-c3546-there-are-no-parameter-packs-available-1.html

We should point this out in the docs and/or suggest a minimum VS version.

Regarding the requirements section of the patch manual where ARM and ARM64 is mentioned: I added the build tools for all platforms, as I want to compile for ARM/ARM64/x86/x64.
The setting up step works for all other platforms but ARM.

Writing just "arm" instead of "arm64" leads to the following error:
>gn gen --ide=vs2019 out\msvc\uwp\Release\arm --filters=//:webrtc "--args=is_debug=false use_lld=false is_clang=false rtc_include_tests=false rtc_build_tools=false rtc_win_video_capture_winrt=true target_os=\"winuwp\" rtc_build_examples=false rtc_win_use_mf_h264=true enable_libaom=false rtc_enable_protobuf=false target_cpu=\"arm\" " ERROR at //third_party/nasm/nasm_assemble.gni:79:3: Assertion failed. assert(defined(invoker.sources), "Need sources defined for $target_name") ^----- Need sources defined for boringssl_asm See //third_party/boringssl/BUILD.gn:54:3: whence it was called. nasm_assemble("boringssl_asm") { ^------------------------------- See //rtc_base/BUILD.gn:915:15: which caused the file to be included. deps += [ "//third_party/boringssl" ] ^------------------------ Traceback (most recent call last): File "D:/Git/webrtc/src/build/vs_toolchain.py", line 569, in <module> sys.exit(main()) File "D:/Git/webrtc/src/build/vs_toolchain.py", line 565, in main return commands[sys.argv[1]](*sys.argv[2:]) File "D:/Git/webrtc/src/build/vs_toolchain.py", line 378, in CopyDlls raise Exception('Unknown target_cpu: ' + target_cpu) Exception: Unknown target_cpu: arm

The third party tool boringssl does not support arm/arm64 for windows, just linux and iOS, if I understand their build file correctly. However, building works for ARM64, so I am lost where to ask for ARM support. Here at the winRTC project?
Or is there already a patch / workaround to build for ARM? The MixedReality-WebRTC project builds for ARM, so it should be possible...

The video synchronization functions recently got reviewed and now have log messages and check for async completeness before and after waiting for the awaitable event. Unfortunately, there is no current way to have these functions in a common WinRT common library. Until the common library is in place, let's duplicate the code from the video to the audio module.

WebRTC has a new release drop (m81). Not all patches contributed so far rolled over or were merged yet.

Currently the revert script is reverting to a specific commit id. The script should revert to the branch label (ie,: branch-heads/3987)

It is possible to have an async operation to complete before the code have opportunity to define the callback for put_Completed. In that situations, the waitable event timesout. It would be good to check if the async operation successfully completed after timingout and not fail the synchronization.

https://github.com/microsoft/winrtc/blob/master/patches_for_WebRTC_org/m80/0000-WebRTC-doesn-t-need-C-CX.patch enables C++17 unconditionally for UWP builds, for the sake of facilitating integration with WinRT/C++ code.

Google is building everything with C++14 AFAIK. There are issues in mixing C++17 and earlier when including/linking webrtc.lib, some absl classes are defined to std ones when C++17 is enabled and will make linking fail. Also building UWP with C++17 and Windows Desktop with C++14 has the potential to be a source of mistakes/inconsistencies.

Since C++17 is not strictly necessary on UWP it should at least be made optional. E.g. the line that uses it can be moved in a different patch to be applied optionally.