#20

I'd like to embed a zoom-rs client in an existing web / electron app, written in javascript using a non-yew UI framework. That would mean decoupling the encoding/decoding code and control mechanisms from the yew UI code, factoring them out to create an "engine" that could be used in other contexts. (Something like this would be needed also for #49 (comment) which suggests using electron on MacOS.)

I'm imagining another top-level directory client-engine/ that would hold that code. It would be set it up so that it would both:

• define a local crate that could be imported into the yew-ui app
• compile into a wasm npm module that could be imported into javascript apps.

The engine code wouldn't require specific HTML and wouldn't create any HTML. It would just provide a control API for establishing connections etc, it would take ids of the canvas elements to render into, and it would use callbacks to communicate interesting events back to the app.

My approach would be to first factor out the client engine code but keep it within yew-ui/src/-- and once that looks good, snip it out and move it into that other directory and set up the cargo & npm mechanisms. The factoring could be done incrementally with PRs along the way, allowing reviews/suggestions as it goes and avoiding a massive shock to the code.

What do you think? I'm happy to take this on.

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

Describe the solution you'd like
Set MacOS min version to 13.4 using the tauri plist key: LSMinimumSystemVersion

Describe the bug
A clear and concise description of what the bug is.
If you try to load the app on browsers that do not support web codecs, the page crashes silently.

To Reproduce
Load the app with an old browser like Safari 15,(webcodecs are supported until 16.4)
then you'll see a trace like this:

remove ability to scroll from insights.

Following up from #118, here are some thoughts about things to put into a future version of the protocol. Putting them out here for discussion...

Explicit "join" and "leave" messages for each participant:

Currently the app only realizes that there's a new participant because messages start coming in, and the app never knows when a participant leaves. Having explicit "join" and "leave" messages would make it easier to have apps handle participants' coming and going with a nice UX.

The "join" message could potentially include useful arbitrary metadata, such as the participant's full name, location, time zone, avatar, etc.

Arbitrary data sources or "channels"(better name?) for each participant:

Currently the code is hardwired for each participant to have a video channel, an audio channel, and a screen-share channel. And there's some awkwardness around screen-share not necessarily being there, and there's no cleanup when they're gone.

Instead, what if the protocol allowed arbitrary data channels, so that different applications could use different data channels? Some applications might not do screen sharing; Some applications might allow a user to do screen sharing for more than one window simultaneously; Some applications might allow a user to play a video; etc.

Each channel could have explicit "start", "pause", and "end" messages. The "start" message would include an identifier, the data type (audio/video/etc. see below :), and potentially other useful arbitrary metadata like codec, resolution, etc. The "pause" message would allow the app to have UX showing when remote peers have paused. The "end" message would allow the app to clean up.

As far as the protocol is concerned, the identifier could be an arbitrary string. Any given app would use whatever identifiers made sense to it. And the app would likewise know what metadata it was interested in sending/receiving.

Other data types for channels:

In addition to audio & video channels, could imagine supporting/allowing other channel data types. Such as:

• Text -- For chats. The app could show all text messages from all users in a single chat window. Could even have multiple text channels, with the "start" metadata indicating who the intended recipients are.

• "Action codes" -- potentially allowing things like raising hand, thumbs up, voting, etc. An app would define its own codes, possibly just a bit of arbitrary json

• File transfer -- "start" the channel with metadata for filename, mimetype, total size, checksum; send the file data (sending and receiving apps can show progress bar); "end" when the transfer is complete

• etc....?

Actually, does the protocol actually care what the data types are? Or is the data type really just an arbitrary string to be interpreted by the app? As long as an app knows how to serialize/encode the data into MediaPackets to send and how to parse/decode them when receiving it should be fine, I think? That said, the ecosystem would presumably include standard common data types with encoders/decoders for each.

...

What do you think?

Is your feature request related to a problem? Please describe.
In order to bump the max number of peers, do not send me video for peers that I can't see on my screen.

Is your feature request related to a problem? Please describe.
Every repetitive simple work should be automated to save more resources.

Describe the solution you'd like
All Contibutors

Describe alternatives you've considered
undefined

Additional context
undefined

We need to forward websocket pings to all hosts so they know that those users are connected while muted and not showing video

Hello. I use part of the project code in my project, the code responsible for transferring the video and audio stream, I transfer chunks via webtrc. The video works without errors, while decoding the audio I get the error "Uncaught Error: closure invoked recursively or after being dropped". I've been struggling with this problem for three days now. Maybe you also faced this problem?

See title

Seems like Safari is missing some classes that our App relies on, we need to find a way around it or at least show an error:

We currently use the chrome default mic and camera which is incredibly annoying

Part of #19

Data seems to be flowing much slower than with the WebSockets so this task is to figure out why

The two peers exchange their public keys over the insecure channel. It is essential that they verify the integrity and authenticity of the received public key, as a Man-in-the-Middle (MitM) attacker could replace the keys with their own. This verification can be done using a trusted third party, digital signatures, or some other secure method.

The aspect ratio is hardcoded.

Instead, the code should lazily adapt to the aspect ratio of the decoded images.

if off, stop encoding and sending video

Currently you have to go to chrome://flags to get the app to work by allowing the insecure origin to bypass the security check. We could get the app to work on other browsers that have WebCodecs API like Firefox and not have to customize the browser if the server and frontend supported a TLS certificate

Right now the UI is sending raw audio data and the packets are huge. We should use an audio encoder to make the packets smaller

Is your feature request related to a problem? Please describe.
Alpine image is popular thanks to its lightweight and having the bare minimum required Linux to run. I found several Dockerfile still using the Debian Bullseye Slim image. It's just a size issue.

Describe the solution you'd like
By using Alpine, we can save a minimum of 30% of the image size.

Describe alternatives you've considered
undefined

Additional context

Reference: https://dev.to/mohammadfaisal/how-i-reduced-docker-image-size-from-143-gb-to-224-mb-53nc

Add a helm chart for deploying the service to kubernetes

Is your feature request related to a problem? Please describe.
It should be possible to use tailwindcss with yew and i think we should consider to use it instead of writing plain css!
I should be able to set this up aswell if you agree!

#20

We do not request users to authenticate, that means that you can choose any user id that you feel like, and join any call. I feel that authentication and authorization are solved problems and I feel proud that as a company we focused on the hard problems like supporting WebTransport and End to End Encryption.

Let us know which kind of OAuth providers would you like us to integrate with, such as Login with Apple, Mastodon, Login with Google etc.

Right now there are a bunch of performance bottlenecks that prevent us from scaling the service.

Find'em and kill'em

Is your feature request related to a problem? Please describe.
The current docker image built by Dockerfile.actix and Dockerfile.yew is huge because we use it for the active development process as runtime inside docker-compose.yml, it's not 'old device' friendly tho.

Describe the solution you'd like
We can create separated those Dockerfile and docker-compose.yml for development and production.

Describe alternatives you've considered
undefined

Additional context
https://levelup.gitconnected.com/create-an-optimized-rust-alpine-docker-image-1940db638a6c
https://dev.to/mohammadfaisal/how-i-reduced-docker-image-size-from-143-gb-to-224-mb-53nc

Currently, we have environment variables for configuring the yew-ui at runtime, but in order to bundle the static assets those need to be set. We should find a better way to provide these values that either don't need configuration or if they do then it is achievable with the bundled static assets

https://bitnami.com/stack/matomo/helm

I've already mentioned: Yes they [zoom] will. They[zoom] are from land of "freedom" where suing people is common.
Besides that - zoom is a registered trademark in this context.

On the UI we should show when peers are muted

https://docs.rs/nnnoiseless/latest/nnnoiseless/struct.DenoiseState.html

1. Right now the UI dockerfile has the backend endpoint harcoded.

This is bad because people will have to modify this image to change that value to host it elsewhere.

Modify the build process so that the UI App accepts the parameter at runtime, if this is impossible, then document this kink so that others are aware of it.

Is your feature request related to a problem? Please describe.
The UI does not show when there are connection issues with the server. For example, if the server is not running it appears that the UI is able to connect to it when behind the scenes the connection is actually failing.

Describe the solution you'd like
It would be nice to show the user that there is a problem connecting with the server somehow, then make the "close" button go back to the "connect" state.

Part of #19

We seem to be receiving packets out of order so they need to be ordered when being decoded and played back to the canvas

Following up this issue: #59
I think we should consedere drop postgresql to use cockroachdb.
Cockroachdb use the same postgresql driver so it should be as easy as changing the docker image used.
And it will allow better and easier scalability since all cockroachdb node can be write node

AES-GCM (Advanced Encryption Standard with Galois/Counter Mode) is a symmetric encryption algorithm that combines two things: the AES algorithm and the GCM (Galois/Counter Mode) mode of operation. It provides both encryption and authentication, ensuring both the confidentiality and the integrity of the data.

Here's a brief look at the two main components:

AES (Advanced Encryption Standard): This is a widely-used symmetric encryption algorithm. Symmetric means that the same key is used both for encrypting the plaintext and decrypting the ciphertext. AES can use key sizes of 128, 192, or 256 bits.

GCM (Galois/Counter Mode): GCM is a mode of operation that turns a block cipher like AES into a stream cipher. It also provides an authentication tag, which allows the recipient to verify the integrity of the message, ensuring that it hasn't been tampered with.

Why AES-GCM?
AES-GCM is popular because it's efficient and secure. The efficiency comes from the use of counter mode, which allows parallelization, meaning that blocks can be encrypted or decrypted simultaneously, taking full advantage of modern multi-core processors.

The authentication part of AES-GCM ensures that if anyone alters a bit of the encrypted message, the decryption will detect this change and reject the message. This is important in many applications to prevent tampering with the encrypted data.

Where is AES-GCM Used?
AES-GCM is commonly used in various protocols like TLS (for secure web browsing), IPsec (for VPNs), and many others. It's a widely accepted standard and has been analyzed extensively by the cryptographic community.

Security Considerations
While AES-GCM is generally considered secure, it must be used correctly. The initialization vector (IV) must be unique for each encryption performed with the same key; otherwise, it can lead to serious vulnerabilities. Proper implementation and key management practices must be followed to maintain the security guarantees provided by AES-GCM.

Is your feature request related to a problem? Please describe.
Would you be open to create a desktop app using tauri ?
It wont add to much work or maintenance overall!
And i can setup it if you agree

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

Describe the solution you'd like
A clear and concise description of what you want to happen.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

We will move away from websockets and use webtransport instead!!!

I already added webtransport to wasm-bindgen: rustwasm/wasm-bindgen#3344

I am working on adding webtransport to h3: hyperium/h3#183

After this is done, we will have to swap our server.

          This is naive error handling, at least we need to let the user know that they need to refresh the page

Originally posted by @darioalessandro in #130 (comment)

Do we have this api in the project?

Great that you're taking an RFC approach to this project! I'm writing this proposal as an issue rather than a PR since it doesn't quite fit into the existing RFC doc.

I’d like to propose steering this project towards a broader overall goal: Not only the "world class video conferencing system" as a black box (albeit an open-source one!), but instead creating an “open video conferencing” ecosystem — i.e. open source definitions and building blocks that would support video conferencing that could be embedded in various apps — as well of course as the video conferencing system itself.

For possible applications, imagine things like telemedicine apps or face-to-face customer service portals, where the video chat would be integrated rather than requiring a participant to say the equivalent of “I’ll send you a zoom link”.

That is ideally, there’d be a collection of separate parts, likely each in its own repo:

• The key foundational part: A documented, versioned communication protocol. (This repo could include the protobuf-generated type definitions)
• Front-end communication modules with documented, versioned APIs (that are tied to specific protocol versions)
• Back-end communication modules with documented, versioned APIs (that are tied to specific protocol versions)
• Possibly a separate crate of yew components that provide video conferencing UI widgets
• Of course, the repo that puts it all together into the complete video conferencing system.

I think organizing the project this way would be a great service to the open-source world — and I think it would have the benefit of making it easy for those with expertise in some aspect of the overall problem space to make contributions, and so making it easier for the project to keep growing and improving. And it would open things up to the community creating a growing ecosystem of related tools. (E.g., say, a video player that generates data following the communication protocol so that it can be hooked in as a peer in a conference.) Also it would allow people to build interoperable parts in other languages if they needed to.

Of course in #74 I’ve been working on the front-end communication module aspect, but the suggestion here is to take things to the next level :)

Full disclosure, I’m working on a startup (sorry, no details now — still stealth mode) that will have an app/service with embedded video chat (not telemedicine or customer service :), and I’d love to be able to build on your work and use as much open source as I can! For now I have bandwidth to keep working on this, but I can’t promise how long I’ll be able to keep it up.

What do you think?