element-hq / element-call Goto Github PK
View Code? Open in Web Editor NEWGroup calls powered by Matrix
Home Page: https://call.element.io
License: GNU Affero General Public License v3.0
Group calls powered by Matrix
Home Page: https://call.element.io
License: GNU Affero General Public License v3.0
We should move to using device events instead of room events for signalling/keep alive messages. This data doesn't is ephemeral and doesn't need to be stored. It also can cause problems if too many events are sent in the room and signalling messages are not fetched when joining. By moving to device events, we will stop storing these signalling events and will not run up against bugs due to the number of events fetched on join.
When we transitioned to to-device events users who leave a call without properly removing the call id from their member state event get stuck where they will not be called by other users in a call if their username is also lexicographically lesser than the other members currently in the room.
Here's an example:
I have a call I'd like to join with 3 users: User 1, User 2, and User 3
All three users join the call.
One user drops but does not reset their member event to remove the call. The members events now look like this:
{
"@user1": { "m.calls": ["123"] },
"@user2": { "m.calls": ["123"] },
"@user3": { "m.calls": ["123"] },
}
User 3's video tile still appears on User 1 and User 2's screen until the WebRTC call drops.
User 3 enters the call again 2 minutes later and the member state is still set as it was before.
User 3 expects users 1 and 2 to start the call with them because they have lexicographically lesser user ids. They never receive a call invite because to users 1 and 2, that user's WebRTC call was disconnected and their state event never changes.
I see a couple different solutions to this:
Tried to join General on robertlong.dev from both macOS Chrome and Mobile Safari (iOS 15) but looks like signalling didn't work; the other call didn't turn up at all. Reproducible.
Currently we display the room initials in a default room avatar, but don't show room avatars when they exist. Now that we support a public room list and signing in with other homeservers we need to support room avatars.
Safari is not sending its webcam feed reliably. At first I thought this was due to reusing the user's MediaStream for each outbound call. This issue is outlined here. I made sure that these streams are now cloned, however it's still sometimes not sending the stream. What's even more odd is that when you change your webcam during this state, your local feed goes blank and the other participants can see you.
It would be nice to be able to set room aliases in the UI so that urls are more memorable.
The way we determine active speaker / presenter should be updated with the following behaviors:
Video tiles should be able to locally be made bigger/smaller. You should be able to drag/drop tiles into these larger spots to have them take up the expanded space.
MSC 3401 specifies a sources array that allows for a single user to join a group call from multiple devices.
Describe the bug
When leaving the call via navigation, the webcam indicator doesn't turn off.
To Reproduce
Steps to reproduce the behavior:
Expected behavior
The webcam indicator should turn off.
Desktop (please complete the following information):
Additional context
To properly disable the webcam status indicator, all video media stream tracks need to be stopped by calling .stop()
on them. I thought I was doing this, but that's most likely not the case.
Rather than flickering up the name of the user who's currently speaking, i think it might be better to show a 'vector green' #0DBD8B
glow of a few pixels around the border of the tile, CSS fading in & out over ~100ms to avoid it being too shouty.
Similarly, if the remote user is muted, we should show that they are muted constantly rather than just on hoverover.
You should be able to set/change your user avatar in the conference UI. The avatars should be shown when your webcam is disabled or if you are using audio only.
Right now we need to link in the matrix-js-sdk and matrix-react-sdk projects because they are not exporting their dependencies properly. It would be nice to be able to use a git branch on the staging server as well. I think in order to do that we should use a prepare
script in the package's package.json like so. We'll also probably need typescript and webpack aliases set up for these projects to reference typescript source files in development.
We should add indicators to the participant's nametag that shows if a user has muted themselves. We can use the CallFeed state/events for this.
Most of the conference call logic currently resides in this project. We've proven this concept now and the the conference call logic should make its way back into the matrix-js-sdk.
MSC3401 is set up so that you should be able to dial the conference from multiple devices as the same user, but this doesn't seem to be wired up right. Therefore it's very easy to get very confused (c.f. #37) if you try to join the same room under the same user from multiple devices.
At least for ConferenceCallManager
and similar as I assume it will become a part of the js-sdk
which is close to 100% TS
We should determine if a user is currently in the call via timeouts, likely by sending ping/pong to-device messages to the currently connected device.
We should add Sentry for error reporting in production
You should be able to change your display name from the microphone/webcam screen and in the call in the settings modal
You should be able to share your screen in a conference room.
You should be able to pick which audio output device to use if your browser supports it (Chrome)
Facepiles show who is currently in a room that you've already joined. This will not work for public rooms until we have proper room peeking.
The debugger panel currently takes up a fixed height. It'd be great to make the top of the panel draggable so it can be resized.
Currently participants are laid out in a naive fashion and there are updated designs with more intuitive per-device layouts. We need to update the layout algorithm to reflect these.
It would be nice to be able to send a link to a user and have them join the conference all as a guest, without needing them to register.
I recently added support for group call participants, however they are not being rendered for participants in rooms when you first view the home page, making them relatively useless. There must be a bug with emitting the participants changed event or something along those lines of announcing participant changes.
You should be able to change your webcam/microphone/audio device when joining a call and during the call. Changing what audio output device is used will be limited to Chrome only.
It should be possible to join a room by only allowing microphone permissions. We could request microphone and video permissions separately to do this.
When you or someone else is speaking there's a green highlight around your tile. This highlight goes away too quickly during pauses and is quite distracting. It should only fade away after a second of inactivity and the opacity should smoothly fade out over a longer period of time.
Prior to switching to to-device signalling, logging in with an account that you use with Element would prompt you to receive 1-1 group call rings. We need to test to ensure that this is fixed with the to-device messages.
In the case where you are joining a call with an active screenshare and you are responsible for calling the other user, your call will not succeed.
Thank you @SimonBrandner for the bug report:
When joining a call where someone is screen-sharing, if we are the party sending the invite, the call will fail. This is caused by the fact that our offer doesn't include anything for the screen-sharing track/stream, the remote party then immediately tries negotiate but we ignore that because we're the impolite party and we don't have a stable connection at that moment.
One solution I found is to hangup the call and let the other party which is screen-sharing send the offer and let us answer. While this covers most cases, there could be a case where one party would be only sending video and the other only audio, so they would never agree.
Maybe there is a sane way to start re-negotiation right after the call connects? Or maybe there is a generally saner way to do this? ๐ค
We should talking indicators to the user's nametag.
I think we can use matrix-org/matrix-js-sdk#1865 for talking indicators once it is finished.
To-device signalling is currently sent via plaintext. We should switch to sending them with Olm
I noticed that all of my remote video feeds froze when I went to screenshare. I think maybe it's stopping them or not updating properly when renegotiating, audio kept working fine. I remember seeing in matrix-js-sdk something about stopping video feeds when screensharing, so maybe this isn't supported yet? If not I can add support for it.
From MSC3401:
When initiating a 1:1 call, the m.call.invite is sent to * devices of the intended target user.
Once the user answers the call from the device, the sender should rescind the other pending to-device messages, ensuring that other devices don't get spammed about long-obsolete 1:1 calls. XXX: We will need a way to rescind pending to-device msgs.
Subsequent candidates and other events are sent only to the device who answered.
Rather than prompting guest users to enter a display name, we can randomly generate a guest username for them so that joining requires one less step. They can then change it once in the room in the settings modal or in the microphone/webcam setup screen.
vendor.9b9f7923.js:30 Uncaught TypeError: i is not a function
at index.d73c20e2.js:1
at Object.Nv (vendor.9b9f7923.js:30)
at Uv (vendor.9b9f7923.js:30)
at Dv (vendor.9b9f7923.js:30)
at bd (vendor.9b9f7923.js:30)
at Ud (vendor.9b9f7923.js:30)
at vendor.9b9f7923.js:30
at qc (vendor.9b9f7923.js:34)
at Mv (vendor.9b9f7923.js:30)
at jd (vendor.9b9f7923.js:30)
We should have a view where a presenter is made the focus of the call. By default this will use an algorithm to detect who is the active speaker. Otherwise users can be pinned as a speaker in the room. Users can also locally focus a user and have them as the presenter.
This is highly dependent on #9
If we don't need to worry about signalling messages not being received due to the number of debug messages, we can safely turn debug messages back on.
We should use hash routing #
for the single page app which will better hide the room id from the server. This also lets us use proper room aliases in room urls and simplifies the reverse proxy setup.
When users join & part the conference we should clearly play vworping sound effects
Currently we do not allow for accounts other than the server the client is configured for.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.