Messages sent to the internal signaling server need to be serialized
twice, first the signaling message and then the wrapper as a whole. Due
to this the NCMessageWrapper was not actually used.
For simplicity the manual serialization was kept rather than adding
something like "NCMessageWrapperToSend" where a serialized signaling
message could be set before serializing it.
Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
NCMessageWrapper is used only for messages sent and received by the
internal signaling server. However, it is unused by the external
signaling server, except for getting the NCSignalingMessage, which is
the common message for both signaling servers.
Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
Note that the thread used to handle the participant list messages from
the external signaling server does not change; the EventBus subscriber
mode was "BACKGROUND", but as the message was posted from a WebSocket
handler, which runs in a worker thread rather than in the main thread,
the subscriber was executed in the same thread as the poster.
Also note that the removed "userId" remark was not fully accurate;
although some external signaling messages do actually use "userid" those
currently handled to process the users do not, they always use "userId"
(as documented in the SignalingMessageReceiver).
Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
Note that the thread used to handle and notify messages from the
external signaling server does not change; the EventBus subscriber mode
was "BACKGROUND", but as the message was posted from a WebSocket
handler, which runs in a worker thread rather than in the main thread,
the subscriber was executed in the same thread as the poster.
Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
This will no longer log an error if the room type of the received
message is neither "video" nor "screen". However, that should never
happen, and it would be useful only while debugging, so it is fine to
lose that.
Note that the check is not added to SignalingMessageReceiver itself to
keep it as generic as possible (and due to the low value of adding it as
explained above). Nevertheless, if needed in the future it would be
possible to add a special listener that receives raw messages in order
to validate them and log the errors, if any.
Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
Although "unshareScreen" is technically bound to a specific peer
connection it is instead treated as a general message on the call
participant.
Nevertheless, call participant messages will make possible (at a later
point) to listen to events like "raise hand" or "mute" (which, again,
could be technically bound to a specific peer connection, but at least
for now are treated as a general message on the call participant).
Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
Unlike the WebRtcMessageListener, which is bound to a specific peer
connection, an OfferMessageListener listens to all offer messages, no
matter which peer connection they are bound to. This can be used, for
example, to create a new peer connection when a remote offer for which
there is no previous connection is received.
Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
Eventually all signaling related code should be moved to a Signaling
class that abstracts the differences between the internal and external
signaling servers, including how messages are sent and listened to. In
the meantime a temporary SignalingMessageReceiver implementation is
added to CallActivity to be able to start using it.
Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
"peerConnectionWrapper" needs to be defined to enter the if and execute
the switch, so just return before the switch if "peerConnectionWrapper"
is null.
Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
The message type is set for all signaling messages. On the other hand,
the payload type is only set for offers and answers (and, if the message
was sent by the Android app, also for candidates). However, in all those
cases the payload type just duplicates the message type, so the message
type can be assigned directly rather than falling back to it if there is
no payload type.
Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
When the HPB is used the signaling messages can be received even when
the local participant is not currently in the call (for example, when
starting the call timed out without other participant joining, or when
reconnecting due to the publisher connection failing). Therefore if the
local participant is not in the call it should not try to establish a
connection with the other participants and disconnect them instead.
Moreover, if the connection is tried to be established when not
in the call the HPB will prevent that, and the PeerConnectionWrapper
will stay in a limbo state waiting for an offer to be sent. If the local
participant then joins the call the PeerConnectionWrapper will already
exist for the other participants, so no new connections will be created,
but those previous connections will never be finally established.
Additionally, as the signaling messages can be received before the join
call response the participant list could be received while the call
state is "RECONNECTING" or "PUBLISHER_FAILED". In those cases, as long
as the local participant is already in the call, the participant list
should be processed as if the call state was already "JOINED" (otherwise
the connections were not established either).
For simplicity the participant list is now ignored only when the call
state is "LEAVING"; this means that the participant list would be also
processed in the "CONNECTION_TIMEOUT" state, but the signaling message
should not be received anyway in that case.
Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
If the call is hung up with a view shutdown (which finishes the
activity) there is no need to do any further processing on the
participant list.
Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
When a call is joined the call flags of the local participant change, so
this causes a signaling message to be sent by the server. When the HPB
is used the signaling message is sent through a WebSocket, which is
already connected before joining the call. Therefore, in some cases the
signaling message can be received through the WebSocket even before the
response to the HTTP "joinCall" request.
If there are other participants in the call the call state is changed to
"IN_CONVERSATION" when the signaling message is processed. However, in
the case described above the call state was then set to "JOINED", which
automatically traverses to "CONNECTION_TIMEOUT" if no other call state
was set in 45 seconds. Due to all this the call was joined and the
connections with the other participants were established, but they were
not visible in the UI (although they could be heard) and after 45
seconds the call was left.
To prevent that now the call state is changed to "JOINED" if it was not
already changed to "IN_CONVERSATION" in the meantime.
Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
Fresco is replaced with Coil everywhere to make it possible to set 'minSdkVersion'
to 23. But Coil is not used directly to avoid splintering the dependency
everywhere in the code. Coil is wrapped by extension functions for 'ImageView'.
Some shared functionality is moved from 'DisplayUtils' into the
'ImageViewExtensions'.
The exisiting initialization of Coil has also be changed. The usage of the self
initialized OKHttp client is removed. If this one is added the
caching of the http client is used by Coil additionally to memory and
disk cache.
Resolves: #2227, #2376
Signed-off-by: Tim Krüger <t@timkrueger.me>
E/AndroidRuntime: FATAL EXCEPTION: main
Process: com.nextcloud.talk2, PID: 10874
java.lang.NullPointerException: uri param can not be null.
at android.media.MediaPlayer.setDataSource(MediaPlayer.java:1058)
at android.media.MediaPlayer.setDataSource(MediaPlayer.java:1021)
at com.nextcloud.talk.activities.CallActivity.playCallingSound(CallActivity.java:2643)
Signed-off-by: Marcel Hibbe <dev@mhibbe.de>
When the web socket is abruptly closed it is connected again and the
call is rejoined. However, the call was rejoined in a background thread,
so an exception was thrown when trying to modify the views, which
prevented the call from being joined again.
Besides that the call state needs to be explicitly changed, as if the
web socket was connected again while in a call the state would be
already "JOINED" or "IN_CONVERSATION", which prevents the signaling
settings from being fetched again after the permissions check, and
therefore also prevented the call from being joined again.
Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
The ParticipantDisplayItems were associated to both the session ID and
the video stream type ("video" or "screen"), but the code that gets them
was not updated to include the video stream type in the key.
Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
ParticipantDisplayItems are not associated to a full participant but to
each of the single connections that the participant may have (video and
screen). However, when they are added to the map only the session ID is
used as key. Due to this when a participant starts a screen share the
ParticipantDisplayItem for the screen share overwrites the item for the
video, and once the screen share is stopped the old item is not
restored. Moreover, if a participant is already sharing a screen when
the local participant joins whether the video or the screen share is
shown is undefined and depends on which connection is established first.
To solve that the ParticipantDisplayItems are now associated with both
the session ID and the video stream type ("video" or "screen"). Due to
this both the video and the screen share of the remote participant are
shown in the grid view; in the future it might be better to only show
the screen share, or allow switching between screen share and video, or
show the screen share in full screen and hide the grid... but for now,
as a quick fix, this is good enough :-)
Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
When pulling signaling messages failed the source observable was
immediately subscribed again, which immediately triggered another pull.
Rather than hammering the server or a flaky network with new requests
again and again now further requests are performed with an incremental
delay (up to 16 seconds).
The delay is increased only when several requests fail in a row, and it
is reset as soon as a request succeeds.
Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
When the internal signaling server is used the observable to pull
signaling messages is subscribed again after each completion, but in
case of an error it was retried only 3 times. Those 3 times are not in a
row, though, but in total for the whole observable, no matter how many
times it was subscribed again.
Due to the limitation on retries in a long call with a flaky connection
pulling the signaling messages could fail more than 3 times, which
caused the observable to finish with an error and therefore stop further
pullings. In this situation the Android app would not notice if other
participants joined or left the call, and thus it would not establish a
connection with them or stop it. To prevent that now the number of
retries is unlimited (although the retry is still stopped if the local
participant is no longer in the call).
Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
The URL for the avatar depends on whether the call participant is a user
or a guest and, if it is a guest, on its nick. Although the user id of a
participant does not change if the participant is a guest the nick may
be changed during a call, so the avatar URL needs to be updated as well.
As the avatar URL is fully derived from other properties it is now
calculated internally in the ParticipantDisplayItem and calculated when
any of the properties it depends on changes (also for the user id for
completeness, as technically the item could be reused for a different
participant with a different user id, even if it is not currently done).
Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
The new sessions are computed by substracting the old sessions (those
for which a PeerConnectionWrapper exists) from the sessions currently in
the call. However, when "oldSessions" was used for that it no longer
contained the old sessions, it only contained the sessions which were no
longer in the call. As those sessions are mutually exclusive with the
sessions currently in the call nothing was substracted from
"newSessions", and it ended being the sessions currently in the call
instead.
When the HPB is not used the list of participants in the conversation is
periodically updated every 30 seconds if no other signaling message was
received in the meantime. As the layout for a participant overrides any
previous layout for that participant this periodically reset the layout
of all participants in the call, as they were all treated as new
sessions.
When the HPB is used the list of participants in the conversation is
updated only when something changed. However, similarly to the previous
case, when that happens the layout of all participants in the call is
also reset for the same reason.
To solve that now "oldSessions" is not modified, so it contains the
sessions for which a PeerConnectionWrapper exists, and substracting it
from "newSessions" now gives only the new sessions.
The other usages of "newSessions" besides creating the connection and
setting up the layout, that is, getting the peers in the call and
changing the call status to "In conversation", should be safe if
executed only when there are new sessions rather than when there are
participants in the call but they did not change.
Resolves: #2486
Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
The old sessions are used to know which connections need to be ended
because they are no longer in the call. However, if a participant is no
longer in the call but there was no connection yet then there is no
connection that needs to be ended; any existing connection will be added
when looping through the connection list, and if it needs to be stopped
it will be found when substracting the sessions that are currently in
the call.
The old sessions are also used to find the new sessions in the call. Due
to an issue in how that is computed "newSessions" currently store the
sessions in the call rather than only the new sessions (this will be
addressed in a following commit). Nevertheless, in both cases any
session not in the call for which there is no connection either will not
make any difference in the computed "newSessions" (as they are mutually
exclusive, so they will never be removed from "newSessions" when
substracting the old sessions).
Due to all that it is not needed to store sessions not in call as old
sessions / sessions to end.
Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
When a new participant is found and the layout for that participant is
set up the participant data might have not been fetched yet. If that
happens the user ID can not be got from the data and therefore a guest
avatar would be shown for that participant, even if that participant is
a normal user.
However, the signaling message that is used to find new participants
already includes the user ID, so it is now explicitly given and, if not,
then it is got from the participant data (which is needed when handling
the establishment of a connection, as in that case the event does not
contain the user ID).
Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
When setting the participant layout, if the HPB is not used, the user ID
is got from the participant list data fetched when a new participant is
found. However, as the participant layout is setup as soon as a new
participant is found the data may have not been received yet, which
ended in a crash (NullPointerException). Now the access to the
participant object is guarded to prevent that.
Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
So that my future self and other valued developrs don't must also
research that the variable 'isPTTActive' is renamed to
'isPushToTalkActive'.
Signed-off-by: Tim Krüger <t@timkrueger.me>
With this implementation the can publish audio & video permission are
set during the creation of the 'CallActivity'. This permissions are
fixed for the complete call. If the permissions are changed by a
moderator the call must be left and joined again.
Resolves: #1783
Signed-off-by: Tim Krüger <t@timkrueger.me>
During the migration from Java to Kotlin this was not done and resulted
in
BundleKeys.INSTANCE.getKEY_CALL_VOICE_ONLY()
instead of
BundleKeys.KEY_CALL_VOICE_ONLY
Signed-off-by: Tim Krüger <t@timkrueger.me>
