The speaking state is still sent only through data channels, as it is
not currently handled by other clients when sent through signaling
messages.
Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
Note that this implicitly send the current state to remote participants
when the local participant joins, as in that case all the remote
participants already in the call join from the point of view of the
local participant
Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
This is not possible when Janus is used, as Janus only allows
broadcasting data channel messages to all the subscribers of the
publisher connection.
Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
The LocalStateBroadcaster observes changes in the
LocalCallParticipantModel and notifies other participants in the call as
needed. Although it is created right before joining the call there is a
slim chance of the state changing before the local participant is
actually in the call, but even in that case other participants would not
be notified about the state due to the MessageSender depending on the
list of call participants / peer connections passed to it, which should
not be initialized before the local participant is actually in the call.
There is, however, a race condition that could cause participants to not
be added to the participant list if they join at the same time as the
local participant and a signaling message listing them but not the local
participant as in the call is received once the CallParticipantList was
created, but that is unrelated to the broadcaster and will be fixed
in another commit.
Currently only changes in the audio, speaking and video state are
notified, although in the future it should also notify about the nick,
the raised hand or any other state (but not one-time events, like
reactions). The notifications right now are sent only through data
channels, but at a later point they will be sent also through signaling
messages as needed.
Similarly, although right now it only notifies of changes in the state
it will also take care of notifying other participants about the current
state when they join the call (or the local participant joins).
Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
This is the counterpart of CallParticipantModel for the local
participant. For now it just stores whether audio and video are enabled
or not, and whether the local participant is speaking or not, but it
will be eventually extended with further properties.
It is also expected that the views, like the button with the microphone
state, will update themselves based on the model. Similarly the model
should be moved from the CallActivity to a class similar to
CallParticipant but for the local participant. In any case, all that is
something for the future; the immediate use of the model will be to know
when the local state changes to notify other participants.
Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
Data channel messages are expected to be sent only to peer connections
with "video" type, which provide the audio and video tracks of the
participant (and, in fact, peer connections for screen shares do not
even have data channels enabled in the WebUI).
Note that this could change if at some point several audio/video tracks
are sent in the same peer connection, or if "speaking" messages are
added to screen shares, but that will be addressed if/when that happens.
Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
For now it just provides support for sending a data channel message to
all participants, so notifying all participants when the media is
toggled or the speaking status change can be directly refactored to use
it.
While it would have been fine to use a single class for both MCU and no
MCU they were split for easier and cleaner unit testing in future
stages.
Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
"hasMCU" (which has always been the wrong name, because it is an SFU
rather than an MCU, but it is wrong even in the signaling server so for
now the legacy name is kept) was set again and again whenever the call
participant list changed. Now it is set instead once its value is known,
that is, when it is known that the internal signaling server is used (as
no "MCU" is used in that case), or when the connection with the external
signaling server is established, as its supported features are not known
until then.
This change should have no effect in the usages of "hasMCU", as it is
used when the call participant list change, which will happen only after
joining the call in the signaling server, or when sending "isSpeaking"
and toggling media, in both cases guarded by "isConnectionEstablished",
which will be true only once "performCall" was called or if the call is
active with other participants.
Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
The SDP constraints for publisher connections when the MCU is used were
set for all connections. Those constraints set "OfferToReceiveAudio" and
"OfferToReceiveVideo" to false, which disables receiving audio and video
when the local participant is the one sending the offer. Therefore,
audio and video was not received when the MCU was not used and the local
participant was the one initiating the connection.
The "OfferToReceiveXXX" configurations have no effect when set on an
answer (and thus are not even set, an empty MediaConstraints is used in
that case). However, when "OfferToReceiveVideo = false" is set the video
transceiver is explicitly stopped (which is used to avoid receiving
video when joining a call with audio only). Therefore, as
"OfferToReceiveVideo = false" was always set, video was never received
in subscriber connections when the MCU is used, or connections initiated
by the other peer when the MCU is not used.
Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
The SDP constraints should be set when the MCU is used, but only for
publisher connections; receiver connections should use the general SDP
constraints.
Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
When the data channel is not open yet data channel messages are queued
and then sent once opened. "onStateChange" is called from the WebRTC
signaling thread, while "send" can be called potentially from any
thread, so to send the data channel messages in the same order that they
were added new messages need to be enqueued until all the pending
messages have been sent. Otherwise, even if there is synchronization
already, it could happen that "onStateChange" was called but, before
getting the lock, "send" gets it and sends the new message before the
pending messages were sent.
Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
Adding and disposing remote data channels is done from different
threads; they are added from the WebRTC signaling thread when
"onDataChannel" is called, while they can be disposed potentially from
any thread when "removePeerConnection" is called. To prevent race
conditions between them now both operations are synchronized.
However, as "onDataChannel" belongs to an inner class it needs to use a
synchronized statement with the outer class lock. This could still cause
a race condition if the same data channel was added again; this should
not happen, but it is handled just in case.
Moreover, once a data channel is disposed it can be no longer used, and
trying to call any of its methods throws an "IllegalStateException". Due
to this, as sending can be also done potentially from any thread, it
needs to be synchronized too with removing the peer connection.
State changes on data channels as well as receiving messages are also
done in the WebRTC signaling thread. State changes needs synchronization
as well, although receiving messages should not, as it does not directly
use the data channel (and it is assumed that using the buffers of a
disposed data channel is safe). Nevertheless a little check (which in
this case requires synchronization) was added to ignore the received
messages if the peer connection was removed already.
Finally, the synchronization added to "send" and "onStateChange" had the
nice side effect of making the pending data channel messages thread-safe
too, as before it could happen that a message was enqueued when the
pending messages were being sent, which caused a
"ConcurrentModificationException".
Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
Getting the label is no longer possible once the data channel has been
disposed. This will help to make the observer thread-safe.
Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
Data channel messages can be sent only when the data channel is open.
Otherwise the message is simply lost. Clients of the
PeerConnectionWrapper do not need to be aware of that detail or keep
track of whether the data channel was open already or not, so now data
channel messages sent before the data channel is open are queued and
sent once the data channel is opened.
Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
Data channel messages are expected to be sent using the "status" data
channel that is locally created. However, if another data channel was
opened by the remote peer the reference to the "status" data channel was
overwritten with the new data channel, and messages were sent instead on
the remote data channel.
In current Talk versions that was not a problem, and the change makes no
difference either, because since the support for Janus 1.x was added
data channel messages are listened on all data channels, independently
of their label or whether they were created by the local or remote peer.
However, in older Talk versions this fixes a regression introduced with
the support for Janus 1.x. In those versions only messages coming from
the "status" or "JanusDataChannel" data channels were taken into
account. When Janus is not used the WebUI opens the legacy
"simplewebrtc" data channel, so that data channel may be the one used to
send data channel messages (if it is open after the "status" data
channel), but the messages received on that data channel were ignored by
the WebUI. Nevertheless, at this point this is more an academic problem
than a real world problem, as it is unlikely that there are many
Nextcloud servers with Talk < 16 and without HPB being used.
Independently of all that, when the peer connection is removed only the
"status" data channel is disposed, but none of the remote data channels
are. This is just a variation of an already existing bug (the last open
data channel was the one disposed due to being the last saved reference,
but the rest were not) and it will be fixed in another commit.
Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
The legacy name was a bit strange, so now it is renamed to just "send"
as the parameter type ("DataChannelMessage") gives enough context.
Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
This implicitly fixes trying to send the initial state on the latest
remote data channel found (which is the one stored in the "dataChannel"
attribute of the "PeerConnectionWrapper") when any other existing data
channel changes its status to open. Nevertheless, as all this will be
reworked, no unit test was added for it.
Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
