Code Monkey home page Code Monkey logo

symphonymediabridge's Introduction

FINOS - Incubating

Symphony Media Bridge

The Symphony Media Bridge (SMB) is a media server application that handles audio, video and screen sharing media streams in an RTC conference system.

In RTC conferencing systems, when more than two participants are in a conference there is usually a media server component involved. Each participant in the conference will send their audio and video streams to the media server. The media server is then responsible for sending the correct media streams to each receiving participant.

Conferencing media servers are typically divided into two categories, multipoint conference units (MCU) and selective forwarding units (SFU). MCUs decode, mix and transcode media streams delivering a single or a few output streams to each receiver. SFUs do not decode or transcode media streams from participants, but forwards a selection of the incoming media streams to each receiver in some intelligent way. Not all incoming streams are forwarded, but typically more than one stream to each receiver.

SMB is an SFU at its core, but has some hybrid MCU like solutions. Video streams are forwarded and not transcoded, but participants can request a mixed, transcoded audio stream instead of forwarded streams. SMB can also be run in a mode where multiple streams are forwarded, but the contents of each stream can vary between different participants in the conference. This allows for larger conferences than SFUs can typically handle.

Written as a high performance native application with efficient resource management, SMB is designed to scale efficiently as the number of conferences and participants grow.

Documentation

Installing

The Supported platforms for the release builds are

  • Ubuntu Server 20.04 LTS

The following additional dependencies have to be installed:

apt-get install libc++-dev libc++abi-dev libsrtp2-1 libmicrohttpd12 libopus0

Alternative 1. Installing the .deb package

1. Download the .deb file from a release

2. Install

dpkg -i finos-rtc-smb_<version>.deb

3. Start with the default empty config file

smb /etc/finos-rtc-smb/config.json

Alternative 2. Download the .tar.gz file

1. Download and extract the .tar.gz file from a release

2. Add realtime priority capabilities to the smb binary

setcap CAP_SYS_NICE+ep smb

3. Start with the default empty config file

./smb config.json

Development setup

The Symphony Media Bridge is a cmake based project that can be built and run for development purposes on Linux as well as MacOSX. Here are instruction on what dependencies are needed to build and run locally. SMB currently only supports compiling with clang/llvm and linking with libc++ on both Linux and MacOSX.

Building for Ubuntu Linux 20.04

1. Install the required dependencies

apt-get install cmake llvm lld clang lldb python3 git libc++-dev libc++abi-dev libssl-dev libsrtp2-dev libmicrohttpd-dev libopus-dev libunwind-dev

2. Set Clang as compiler

export CC=clang && export CXX=clang++

3. Generate the Makefile

cmake -DCMAKE_BUILD_TYPE=Debug .

Building for MacOSX

1. Install the required dependencies using brew

brew install cmake srtp [email protected] libmicrohttpd opus

2.1 Generate the Makefile

cmake -DCMAKE_BUILD_TYPE=Debug -G "Unix Makefiles" .

2.2 Alternatively generate Xcode project files

cmake -G "Xcode" .

Running the application

./smb config.json

Running the local test suite

./UnitTest
./UnitTest2

UnitTest2 contains integration tests that run in emulated network environment and emulated time. It is very close to end to end tests without adding external RTC agents.

Contributing

  1. Fork it (https://github.com/finos/SymphonyMediaBridge/fork)
  2. Create your feature branch (git checkout -b feature/fooBar)
  3. Read our contribution guidelines and Community Code of Conduct
  4. Commit your changes (git commit -am 'Add some fooBar')
  5. Push to the branch (git push origin feature/fooBar)
  6. Create a new Pull Request

NOTE: Commits and pull requests to FINOS repositories will only be accepted from those contributors with an active, executed Individual Contributor License Agreement (ICLA) with FINOS OR who are covered under an existing and active Corporate Contribution License Agreement (CCLA) executed with FINOS. Commits from individuals not covered under an ICLA or CCLA will be flagged and blocked by the FINOS Clabot tool (or EasyCLA). Please note that some CCLAs require individuals/employees to be explicitly named on the CCLA.

Need an ICLA? Unsure if you are covered under an existing CCLA? Email [email protected]

License

Copyright 2021 Symphony LLC

Distributed under the Apache License, Version 2.0.

SPDX-License-Identifier: Apache-2.0

symphonymediabridge's People

Contributors

danielgronberg avatar maoo avatar marcusspangenberg avatar olofkallander avatar pcarlsym avatar reddvl1980 avatar ricardomdomingues avatar stepanaksenov avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

symphonymediabridge's Issues

Transport::runTicks is no longer called

In PR #172 EngineMixer::runTransportTicks was removed. This means Transport::runTick() is never called anywhere anymore.

Was this the intent of the change, otherwise it seems like a possible issue. If runTick() is never to be used anymore, the function in Transport should be removed. Right now it just looks wrong.

Director undefined behaviour

Issues after #123 was merged. Was there a profiling run done before and after this? Otherwise I don't see the purpose of introducing these changes just for the sake of optimizing.

Without digging deeper I can see some issues:

1

We have some situations where there is no video forwarded to a participant. I can see in the log that the value of desiredHighestEstimatedPinnedLevel is out of bounds:

2022-09-22 07:53:05.964 INFO [0x7f5c2246f700][StreamDirector-9] setUplinkEstimateKbps 1269, endpointIdHash 628728810038321742 desiredLevel 12 > level 2, scale up

12 is not a valid level value.

2

Undefined behaviour in the code

size_t pinnedQuality, unpinnedQuality;
getVideoQualityLimits(participantStream, pinnedQuality, unpinnedQuality);
inline void getVideoQualityLimits(const ParticipantStreams& participantStreams,
        size_t& outPinnedQuality,
        size_t& outUnpinnedQuality) const
    {
        const auto maxVideoStreams = std::min(std::max(1ul, _lowQualitySsrcs.size()), (unsigned long)_lastN - 1);
        if (maxVideoStreams < 1)
        {
            return;
        }

If maxVideoStreams is < 1 the function returns and the code above continues with the values of pinnedQuality and unpinnedQuality unset.

RTP payload not supported

Support Question

Hello, i have in logs:

2024-03-19 09:02:06.426 INFO [0x7f20180aad80][TransportFactory] opened main media port at 172.31.0.2:12000
2024-03-19 09:02:06.427 INFO [0x7f20180aad80][main] Build: Release 697d35d
2024-03-19 09:02:12.234 INFO [0x7f1f7d7fa700][RequestHandler] Incoming request [1001] POST /conferences/
2024-03-19 09:02:12.235 INFO [0x7f1f7d7fa700][MixerManager] Create mixer, last-n 5
2024-03-19 09:02:12.247 INFO [0x7f1f7d7fa700][MixerManager] Mixer-2 id=15567660543410975965, local 596846179 slides 2777252046 pins 3778981342 3827951542 940096512 70977836
2024-03-19 09:02:12.247 INFO [0x7f1f7d7fa700][ApiRequestHandler] Allocate conference 15567660543410975965, mixer Mixer-2, last-n -1, global-port t
2024-03-19 09:02:12.247 INFO [0x7f1f7d7fa700][RequestHandler] Outgoing response [1001] 200
2024-03-19 09:02:12.249 INFO [0x7f1f667fc700][RequestHandler] Incoming request [1002] POST /conferences/15567660543410975965/ingest
2024-03-19 09:02:12.251 INFO [0x7f1f667fc700][Transport-3] SRTP client: SrtpClient-4
2024-03-19 09:02:12.251 INFO [0x7f1f667fc700][Mixer-2] Created bundle transport, endpointId ingest, endpointIdHash 5765349304897777532, transport Transport-3 (0x7f1f4800b538), ice controlling t
2024-03-19 09:02:12.251 INFO [0x7f1f667fc700][Mixer-2] Created bundled audioStream id 1128390118524043092, endpointId ingest, endpointIdHash 5765349304897777532, transport Transport-3
2024-03-19 09:02:12.251 INFO [0x7f1f667fc700][Mixer-2] Created bundled videoStream id 2392350303525710656, endpointId ingest, endpointIdHash 5765349304897777532, transport Transport-3
2024-03-19 09:02:12.251 INFO [0x7f1f667fc700][RequestHandler] Outgoing response [1002] 200
2024-03-19 09:02:12.255 INFO [0x7f1f677fe700][RequestHandler] Incoming request [1003] POST /conferences/15567660543410975965/ingest
2024-03-19 09:02:12.256 WARN [0x7f1f677fe700][RequestHandler] Outgoing response [1003] 400. Error message: rtp payload '' not supported

how can i debug this? thx

Api format request/response mismatch - payload parameters

In the API, the format for the parameters object was changed from

"parameters": [
{"name": "minptime", "value "10"}
],

to

"parameters": {
"minptime": "10"
},

in the requests, but the responses still use the old format.

The reasoning behind the name/value format was to be able to define JSON objects in JS/Java for parameters instead of having to reflect the key/value pairs. It would be good if it at least it was the one or the other. The examples/simpleserver code would need updating as well if the new format is to be used.

TcpEndpoint-SendJob-run, How do I ensure the sequence of SENDING TCP packets

there are three tcp packets to be sent,

  1. app call TcpEndpoint->sendTo three times for packet1 , packet2, packet3;
  2. TcpEndpoint---_sendJobs.addJob(*this, packet, target, allocator) called three times;
  3. if app run 8 workThread for jobManager, How do I ensure the sequence of SENDING TCP packets for each tcp connection, packet1, packet2, packet3, may not be packet2, packet3, packet1?

Segfault when ice.singlePort set to 0 in config

Bug Report

Steps to Reproduce:

Set ice.singlePort to 0 in config.json
Start application

Expected Result:

Starting without single UDP port, using the port range instead.

Actual Result:

Segfault in:

std::vector::begin() vector:1408
transport::TransportFactoryImpl::registerIceListener(transport::Endpoint::IEvents &, const std::string &) TransportFactory.cpp:515
bridge::Bridge::initialize(std::shared_ptr<…>) Bridge.cpp:173
bridge::Bridge::initialize() Bridge.cpp:103
main main.cpp:151
__libc_start_call_main 0x00007f11e4e58d90
__libc_start_main_impl 0x00007f11e4e58e40
_start 0x0000563af1e66955

Environment:

Ubuntu Linux x64

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.