2015 Q4 webrtc standards update
•
17 gefällt mir•6,307 views
Evolution of WebRTC APIs and feature from peer-to-peer to object model in 2015. Include latest updates from November meetings in Japan. Beta, non-complete version. Contact agouaillard@gmail.com for more.
Empfohlen
Empfohlen
Weitere ähnliche Inhalte
Was ist angesagt?
Was ist angesagt? (20)
Ähnlich wie 2015 Q4 webrtc standards update
Ähnlich wie 2015 Q4 webrtc standards update (20)
Mehr von Alexandre Gouaillard
Mehr von Alexandre Gouaillard (9)
Kürzlich hochgeladen
Kürzlich hochgeladen (20)
2015 Q4 webrtc standards update
- 1. December 2015 Standards Update a.k.a One year of standards awesomeness Dr. Alex Gouaillard © Dr. Alex Gouaillard @ Citrix Systems, 2015. Document provided under CC BY-‐NC 4.0
- 2. Overview © Dr. Alex Gouaillard @ Citrix Systems, 2015. Document provided under CC BY-‐NC 4.0 • Early 2015 WebRTC • Object Models APIs • Glue (Transceiver) • ICE and opMmizaMons • Simulcast • Others in webrtc 1.0
- 3. Early 2015, peer-‐to-‐peer webRTC Model © Dr. Alex Gouaillard @ Citrix Systems, 2015. Document provided under CC BY-‐NC 4.0 hRp://www.html5rocks.com/en/tutorials/webrtc/basics/
- 4. Signaling server Early 2015, peer-‐to-‐peer webRTC Model PeerConnecMon JavaScript APP (Sender) JavaScript APP (Receiver) PeerConnecMon Media Stream Data Channel CreateOffer SetLocal SetRemote SetLocal SetRemote Offer (SDP) Answer (SDP) Candidates (SDP) Media Stream Data Channel OnCandidate AddCandidate CreateAnswer ICE Bindings + tests © Dr. Alex Gouaillard @ Citrix Systems, 2015. Document provided under CC BY-‐NC 4.0
- 5. Signaling server Early 2015, peer-‐to-‐peer webRTC Model OmniPotent PeerConnecMon PeerConnecMon JavaScript APP (Sender) JavaScript APP (Receiver) PeerConnecMon Media Stream Data Channel ICE UA DTLS RTP Codec CreateOffer Gather Transport SetLocal SetRemote ICE UA DTLS RTP Codec Gather Transport SetLocal SetRemote Offer (SDP) Answer (SDP) Candidates (SDP) Media Stream Data Channel OnCandidate AddCandidate CreateAnswer ICE Bindings + tests © Dr. Alex Gouaillard @ Citrix Systems, 2015. Document provided under CC BY-‐NC 4.0
- 6. Early 2015 webRTC Model PC internals © Dr. Alex Gouaillard @ Citrix Systems, 2015. Document provided under CC BY-‐NC 4.0 hRp://www.w3.org/TR/webrtc/
- 7. Early 2015 webRTC Model PC internals Never Used. © Dr. Alex Gouaillard @ Citrix Systems, 2015. Document provided under CC BY-‐NC 4.0 hRp://www.w3.org/TR/webrtc/
- 8. Early 2015 webRTC Model PC internals 1 state, 4 different cases … © Dr. Alex Gouaillard @ Citrix Systems, 2015. Document provided under CC BY-‐NC 4.0 hRp://www.w3.org/TR/webrtc/
- 9. Early 2015 webRTC Model PC internals Mutually Exclusive Paths © Dr. Alex Gouaillard @ Citrix Systems, 2015. Document provided under CC BY-‐NC 4.0 hRp://www.w3.org/TR/webrtc/
- 10. stable have-‐local-‐offer stable OFFER ANSWER Create PC Create offer SetLocal(offer) Create PC SetRemote(offer) Create answer SetLocal(answer) stable Have-‐remote-‐ offer stable SetRemote(answer) addRemoteStream Early 2015 webRTC Model PC internals Alternate view Caller Callee Internet © Dr. Alex Gouaillard @ Temasys Comm., 2014. Document provided under CC BY-‐NC 4.0 © Dr. Alex Gouaillard @ Citrix Systems, 2015. Document provided under CC BY-‐NC 4.0
- 11. Early 2015 webRTC Model PC internals ICE States visible from JS © Dr. Alex Gouaillard @ Citrix Systems, 2015. Document provided under CC BY-‐NC 4.0 hRp://www.w3.org/TR/webrtc/
- 12. Early 2015 webRTC Model PC internals ICE Not always reachable (not implemented) © Dr. Alex Gouaillard @ Citrix Systems, 2015. Document provided under CC BY-‐NC 4.0 hRp://www.w3.org/TR/webrtc/
- 13. Early 2015 webRTC Model PC internals ICE 1 state several failures: -‐ ICE failure, -‐ DTLS Failure, … -‐ For mulMple tracks, if one track fail, the state is “failed”, no way to tell which one failed © Dr. Alex Gouaillard @ Citrix Systems, 2015. Document provided under CC BY-‐NC 4.0 hRp://www.w3.org/TR/webrtc/
- 14. DONE FROZEN IN CALL CONNECTION HANDSHAKE (SDP O/A) . new checking connected disconnected failed Completed close new gathering complete CALLER SIG-‐SERVER CALLEE stable have-‐local-‐offer stable Close OFFER ANSWER Create PC Add local stream(s) Create offer <modify sdp> SetLocal(offer) Sending offer Create PC SetRemote(offer) addRemoteStream(s) Add local stream(s) Create answer <modify sdp> SetLocal(answer) Send answer stable Have-‐remote-‐ offer stable new gathering complete onIceCandidate <Filter candidates> Send candidate <Filter candidates> addIceCandidate onIceCandidate <Filter candidates> Send candidate <Filter candidates> addIceCandidate new checking connected disconnected failed Completed close Close © Temasys CommunicaMons, pvt, ltd, 2014 Document provided under CC BY-‐NC 4.0 CANDIDATES Peer ConnecMon ICE ConnecMon ICE Gathering SetRemote(answer) addRemoteStream BYE
- 15. Early 2015, peer-‐to-‐peer webRTC Model Open QuesMons 1. How do I handle bandwidth preferences 2. How do I check fingerprint (MITM aRacks prevenMon) 3. How do I handle codecs and their parameters => Hum, ……. Just modify the SDP and pray it works. 4. How to know which pair of candidate is being used? 5. How do I restrict the ICE gathering (not to expose my IPs) 6. How do I get more info about ICE failures? 7. How do I make the (ICE) handshake faster? 8. How do I get rid of JSEP, SDP, O/A, …..? 9. Can I switch front / self view without renegoMaMon? => No can’t do. © Dr. Alex Gouaillard @ Citrix Systems, 2015. Document provided under CC BY-‐NC 4.0
- 16. Overview © Dr. Alex Gouaillard @ Citrix Systems, 2015. Document provided under CC BY-‐NC 4.0 • Early 2015 WebRTC • Object Models APIs • Glue (Transceiver) • ICE and OpMmizaMons • Simulcast • Others in webrtc 1.0
- 17. Q2 2015, New W3C WebRTC charter • => ORTC – WebRTC merge • NoMon of Object Model API (ORTC inspired) © Dr. Alex Gouaillard @ Citrix Systems, 2015. Document provided under CC BY-‐NC 4.0 hRp://ortc.org/wp-‐content/uploads/2014/08/ortc.html
- 18. Object Models? • Each Object Does only one thing! • Each Object can be probed for capabiliMes (JS) • Parameters can be get/set dynamically (JS) © Dr. Alex Gouaillard @ Citrix Systems, 2015. Document provided under CC BY-‐NC 4.0 #8 (Edge) hRp://ortc.org/wp-‐content/uploads/2014/08/ortc.html
- 19. Object Model: Example RtpSender -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ -‐ MediaStreamTrack track -‐ DTLSTransport transport -‐ DTLSTransport rtcpTransport -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ Promise<void> replaceTrack( track ) RtpCapabiliMes getCapabiliMes() RtpParameters getParameters() Void setParameters( RtpParameters ) © Dr. Alex Gouaillard @ Citrix Systems, 2015. Document provided under CC BY-‐NC 4.0
- 20. Object Model: Example RtpSender -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ -‐ MediaStreamTrack track -‐ DTLSTransport transport -‐ DTLSTransport rtcpTransport -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ Promise<void> replaceTrack( track ) RtpCapabiliMes getCapabiliMes() RtpParameters getParameters() Void setParameters( RtpParameters ) Switch one track for another without renegoMaMon All the ICE magic and the EncrypMon happen therein. If bundled, both points to the same object. © Dr. Alex Gouaillard @ Citrix Systems, 2015. Document provided under CC BY-‐NC 4.0 #9
- 21. Object Model: Example CapabiliMes and Parameters RtpSender -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ -‐ MediaStreamTrack track -‐ DTLSTransport transport -‐ DTLSTransport rtcpTransport -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ Promise<void> replaceTrack( track ) RtpCapabiliMes getCapabiliMes() RtpParameters getParameters() Void setParameters( RtpParameters ) RtpCapabiliMes [readonly] -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ -‐ Seq<RtpCodecParameters> codecs -‐ Seq<RtpHeaderExtParameters> headerExtensions RtpParameters -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ -‐ seq<RtpEncodingParameters> encodings -‐ seq<RtpHeaderExtParameters> headerExtensions -‐ RTCRtcpParameters rtcp -‐ Seq<RtpCodecParameters> codecs Where most of the Codec and streaming magic happen now! Can change parameters during a call, through a JS API. © Dr. Alex Gouaillard @ Citrix Systems, 2015. Document provided under CC BY-‐NC 4.0
- 22. The RtpSender Object: RtpParameters in Detail RtpParameters -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ -‐ seq<RtpEncodingParameters> encodings -‐ seq<RtpHeaderExtParameters>headerExtensions -‐ RTCRtcpParameters rtcp -‐ Seq<RtpCodecParameters> codecs RtpEncodingParameters -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ -‐ unsigned long ssrc -‐ RTCRtxParameters rtx -‐ RTCFecParameters fec -‐ boolean acMve -‐ RTCPriorityType priority -‐ unsigned long maxBitrate -‐ RTCDegradaMonPreference degradaMonPreference © Dr. Alex Gouaillard @ Citrix Systems, 2015. Document provided under CC BY-‐NC 4.0 DeacMvates sending (saves bandwidth when muFng) PrioriMze streams Cap Bandwidth Chose to impact spaMal or temporal resoluMon (or balanced) in case of degradaMon. #1 RtpCodecParameters -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ -‐ unsigned short payloadType -‐ DOMString mimeType -‐ unsigned long clockRate -‐ unsigned short channels = 1 -‐ DOMString sdpFmtpLine #3
- 23. DtlsTransport Object DtlsTransport -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ -‐ readonly IceTransport transport -‐ readonly DtlsTransportState state -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ -‐ EventHandler onstatechange -‐ seq<ArrayBuffer> getRemoteCerMficates() © Dr. Alex Gouaillard @ Citrix Systems, 2015. Document provided under CC BY-‐NC 4.0 The Ice Object is separated New State Machine for DTLS (encrypMon) errors. ConnecMng Access The fingerprints to check against MITM aRacks. #2 #6a
- 24. IceTransport Object IceTransport -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ -‐ readonly RTCIceRole role -‐ readonly RTCIceComponent component -‐ readonly RTCIceConnecMonState state -‐ readonly RTCIceGatheringState gatheringState -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ -‐ seq<RTCIceCandidate> getLocalCandidates() -‐ seq<RTCIceCandidate> getRemoteCandidates() -‐ RTCIceCandidatePair getSelectedCandidatePair() -‐ RTCIceParameters getLocalParameters () -‐ RTCIceParameters getRemoteParameters() -‐ seq<ArrayBuffer> getRemoteCerMficates() -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ -‐ EventHandler onstatechange -‐ EventHandler ongatheringstatechange -‐ EventHandler onselectedcandidatepairchange © Dr. Alex Gouaillard @ Citrix Systems, 2015. Document provided under CC BY-‐NC 4.0 Same State machine, but only for ICE this Mme. #6b Controlling Controlled Rtp Rtcp Selected Pair (dynamic) #5
- 25. RtpReceiver Object RtpReceiver -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ -‐ MediaStreamTrack track -‐ DTLSTransport transport -‐ DTLSTransport rtcpTransport -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ RtpCapabiliMes getCapabiliMes() Seq<RtpContribuMngSource> getContribuMngSources () © Dr. Alex Gouaillard @ Citrix Systems, 2015. Document provided under CC BY-‐NC 4.0 Same as sender. Provide SSRC and audio levels of each contribuMng source, across Mme. RtpContribuMngSource -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ -‐ DOMHighResTimeStamp Mmestamp -‐ unsigned long source -‐ byte audioLevel
- 26. Early 2015, peer-‐to-‐peer webRTC Model Open QuesMons Have a final API. • How do I handle bandwidth preferences • How do I check fingerprint (MITM aRacks prevenMon) • How to know which pair of candidate is being used? • How do I get more info about ICE failures? • How do I get rid of JSEP, SDP, O/A, …..? • Can I switch front / self view without renegoMaMon? Have some API, can do beLer or more. • How do I handle codecs and their parameters • How do I restrict the ICE gathering (not to expose my IPs) Not addressed yet • How do I make the (ICE) handshake faster? © Dr. Alex Gouaillard @ Citrix Systems, 2015. Document provided under CC BY-‐NC 4.0
- 27. Overview © Dr. Alex Gouaillard @ Citrix Systems, 2015. Document provided under CC BY-‐NC 4.0 • Early 2015 WebRTC • Object Models APIs • Glue (Transceiver) • ICE and OpMmizaMons • Simulcast • Others in webrtc 1.0
- 28. WebRTC 1.0 as Object Model + JSEP Problems: 1. m-‐lines in SDP are intrinsically bi-‐direcMonal 2. m-‐lines in SDP have aRributes that fit neither sender nor receiver object model (mid), or that would be duplicated. How to model the equivalent to a m-‐line on top of the Object Model (so we can throw it later if we decide to get rid of SDP, JSEP, ….) ? © Dr. Alex Gouaillard @ Citrix Systems, 2015. Document provided under CC BY-‐NC 4.0
- 29. RtpTransceiver Object represents a combinaFon of an RtpSender and an RtpReceiver that share a common mid RtpTransceiver -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ -‐ DOMString mid -‐ RtpSender sender -‐ RtpReceiver receiver -‐ boolean stopped -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ -‐ void stop() -‐ void setCodecPreferences (seq<RtpCodecCapability> codecs) © Dr. Alex Gouaillard @ Citrix Systems, 2015. Document provided under CC BY-‐NC 4.0 Convenience Method to set up codecs at the Transceiver level. Depending on the sequence of API calls, allow for elegant manipulaMon of codec choices pre or post SDP O/A. #3 RtpTransceiverInit -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ -‐ Boolean send = true -‐ Boolean receive = true -‐ seq<MediaStream> streams Replaces offerToReceive constraints Track to Stream referencing this track NOTE ALEX : PeerConnecFon.AddTransceiver( RtpTransceiverInit )
- 30. WebRTC 1.0 as Object Model + JSEP Drawing? Explain the pain of the rollback. © Dr. Alex Gouaillard @ Citrix Systems, 2015. Document provided under CC BY-‐NC 4.0
- 31. Overview © Dr. Alex Gouaillard @ Citrix Systems, 2015. Document provided under CC BY-‐NC 4.0 • Early 2015 WebRTC • Object Models APIs • Glue (Transceiver) • ICE and OpMmizaMons • Simulcast • Others in webrtc 1.0
- 32. ICE is HOT ! Burning hot … -‐ Best NAT/Firewall tech. -‐ Fully automated! -‐ Lot of ongoing work => new WG at IETF However -‐ #1 Pain source for large scale soluMons (dixit cisco, unify, …) -‐ SMll slower than telephony calls on mobile -‐ Hand-‐over on mobile problemaMc © Dr. Alex Gouaillard @ Citrix Systems, 2015. Document provided under CC BY-‐NC 4.0
- 33. What’s your flavor? Full, Trickle, Lite Feature flavor Full Trickle Lite Make connecMvity checks Yes Yes NO Answer connecMvity checks Yes Yes Yes Controlling or controled Both Both Controlled Candidates send in the Offer/Answer Yes No ? Flag Ice-‐opMon: trickle a=ice-‐lite hRps://tools.iew.org/html/drax-‐iew-‐mmusic-‐trickle-‐ice-‐02 hRps://webrtchacks.com/trickle-‐ice/ © Dr. Alex Gouaillard @ Citrix Systems, 2015. Document provided under CC BY-‐NC 4.0 Trickle: From 10s to 1S Q: How to get 100ms?
- 34. Faster ICE -‐ Start even before the call! -‐ ICE pool -‐ Pre-‐gathering -‐ Simplify and fasten the pairing / checking © Dr. Alex Gouaillard @ Citrix Systems, 2015. Document provided under CC BY-‐NC 4.0
- 35. Warmup / early media Make everything async, don’t wait -‐ ICE, DTLS, and media ready before signaling is (warmup) -‐ Transceiver + replace track -‐ StarMng sending media as you’re sending the answer (early media) -‐ Transceiver awesomeness -‐ [use the generic drawing and point to each case] © Dr. Alex Gouaillard @ Citrix Systems, 2015. Document provided under CC BY-‐NC 4.0
- 36. API for default ICE servers -‐ Browsers can now propose “default” STUN/ TURN server to applicaMons. © Dr. Alex Gouaillard @ Citrix Systems, 2015. Document provided under CC BY-‐NC 4.0
- 37. Overview © Dr. Alex Gouaillard @ Citrix Systems, 2015. Document provided under CC BY-‐NC 4.0 • Early 2015 WebRTC • Object Models APIs • Glue (Transceiver) • ICE and OpMmizaMons • Simulcast • Others in webrtc 1.0
- 38. Reminder MulM-‐cast vs Simulcast vs SVC © Dr. Alex Gouaillard @ Citrix Systems, 2015. Document provided under CC BY-‐NC 4.0 A B C MulMcast • Several tracks • Decodable separately • Bandwidth management separated HD SD QVGA Simulcast • Several tracks • Coming from the same source • Decodable separately • Smart bandwidth management possible 1 SSRC Δ2 Δ1 QVGA SVC Encoding • Several tracks • Coming from the same source • Not Decodable separately (Except base layer) • Smart bandwidth management mandatory • Less bandwidth, more resilience. 1 SSRC E E E E E E SVC Enc. D D D HD SD QVGA
- 39. Reminder State of MulM-‐cast signaling © Dr. Alex Gouaillard @ Citrix Systems, 2015. Document provided under CC BY-‐NC 4.0 From the beginning of webrtc, one have been able to send mulMple media streams in separate peer connecMons, but it consumes much more ports, and there is no sync between those tracks (lip sync, ..). A B C E E E MulMcast • Several tracks • Decodable separately • Bandwidth management separated
- 40. RTP/RTCP within a PeerConnecMon Use case: 2 media tracks (audio and video), how many ports do I need? Less ports would useful for -‐ BeRer connecMon rate esp. over mobile net. -‐ OpMmize turn/media server resources -‐ Client side ports constraints © Dr. Alex Gouaillard @ Citrix Systems, 2015. Document provided under CC BY-‐NC 4.0
- 41. RTP/RTCP within a PeerConnecMon Default 4 A V RTP RTCP hLp://tools.ieV.org/html/draW-‐ieV-‐mmusic-‐sdp-‐bundle-‐negoFaFon hLp://datatracker.ieV.org/doc/rfc5761/ © Dr. Alex Gouaillard @ Cosmo ConsulFng., 2015. Document provided under CC BY-‐NC 4.0
- 42. RTP/RTCP Default 4 Bundle 2 A V RTP RTCP A V RTP RTCP a=group:BUNDLE hLp://tools.ieV.org/html/draW-‐ieV-‐mmusic-‐sdp-‐bundle-‐negoFaFon hLp://datatracker.ieV.org/doc/rfc5761/ © Dr. Alex Gouaillard @ Cosmo ConsulFng., 2015. Document provided under CC BY-‐NC 4.0
- 43. RTP/RTCP Default 4 Bundle 2 Rtcp-‐mux 2 A V RTP RTCP A V RTP RTCP A V RTP RTCP a=rtcp-‐mux a=group:BUNDLE hLp://tools.ieV.org/html/draW-‐ieV-‐mmusic-‐sdp-‐bundle-‐negoFaFon hLp://datatracker.ieV.org/doc/rfc5761/ © Dr. Alex Gouaillard @ Cosmo ConsulFng., 2015. Document provided under CC BY-‐NC 4.0
- 44. RTP/RTCP Default 4 Bundle 2 Rtcp-‐mux 2 Both 1 A V RTP RTCP A V RTP RTCP A V RTP RTCP A V RTP RTCP a=rtcp-‐mux a=group:BUNDLE hLp://tools.ieV.org/html/draW-‐ieV-‐mmusic-‐sdp-‐bundle-‐negoFaFon hLp://datatracker.ieV.org/doc/rfc5761/ © Dr. Alex Gouaillard @ Cosmo ConsulFng., 2015. Document provided under CC BY-‐NC 4.0
- 45. RTP/RTCP -‐ BeRer connec. rate esp. over mobile net. (10%) -‐ OpMmize turn/media server resources (factor 4!) -‐ Client side ports constraints (c’est la vie) -‐ Beware of components / ports / opMons changes during O/A More examples of SDPs: hLp://tools.ieV.org/id/draW-‐nandakumar-‐rtcweb-‐sdp-‐01.html © Dr. Alex Gouaillard @ Cosmo ConsulFng., 2015. Document provided under CC BY-‐NC 4.0
- 46. RTP/RTCP – extrapolaMon to n tracks Default 2n Bundle n Rtcp-‐mux n Both 1 A V RTP RTCP A V RTP RTCP A V RTP RTCP A V RTP RTCP a=rtcp-‐mux a=group:BUNDLE hLp://tools.ieV.org/html/draW-‐ieV-‐mmusic-‐sdp-‐bundle-‐negoFaFon hLp://datatracker.ieV.org/doc/rfc5761/ © Dr. Alex Gouaillard @ Cosmo ConsulFng., 2015. Document provided under CC BY-‐NC 4.0
- 47. Reminder State of MulM-‐cast signaling For mulFple streams in a single peer connecFon! © Dr. Alex Gouaillard @ Citrix Systems, 2015. Document provided under CC BY-‐NC 4.0
- 48. Reminder State of MulM-‐cast signaling For mulFple streams in a single peer connecFon! © Dr. Alex Gouaillard @ Cosmo ConsulFng., 2015. Document provided under CC BY-‐NC 4.0 © Dr. Alex Gouaillard @ Citrix Systems, 2015. Document provided under CC BY-‐NC 4.0 1. For historical reason, a specific non standard signaling has been used by chrome for almost two years (Plan B). It has become the de-‐facto standard (jitsi, …) 2. The real spec is “Unified Plan”, now implemented in FF, and soon in Chrome.
- 49. Plan B: group per media type (chrome current, hangout, jitsi) a=group:BUNDLE sdparta_0 sdparta_1 a=msid-‐semanMc:WMS * m=audio 9 RTP/SAVPF 109 9 0 8 a=mid:sdparta_0 a=ssrc:323910839 msid:{d57d3917-‐64e9-‐4f49-‐ad‚-‐b049d165c312} {920e9ffc-‐728e-‐0d40-‐a1b9-‐ebd0025c860a} a=ssrc:323910839 cname:{72b9ff9f-‐4d8a-‐5244-‐b19a-‐bd9b47251770} m=video 9 RTP/SAVPF 120 a=mid:sdparta_1 a=ssrc:2917595157 msid:{d57d3917-‐64e9-‐4f49-‐ad‚-‐b049d165c312} {35eeb34f-‐f89c-‐3946-‐8e5e-‐2d5abd38c5a5} a=ssrc:2917595157 cname:{72b9ff9f-‐4d8a-‐5244-‐b19a-‐bd9b47251770} a=ssrc:2325911938 msid:{3a2bfe17-‐c65d-‐364a-‐af14-‐415d90bb9f52} {aa7a4ca4-‐189b-‐504a-‐9748-‐5c22bc7a6c4f} a=ssrc:2325911938 cname:{72b9ff9f-‐4d8a-‐5244-‐b19a-‐bd9b47251770} hLps://tools.ieV.org/html/draW-‐uberF-‐rtcweb-‐plan-‐00 © Dr. Alex Gouaillard @ Cosmo ConsulFng., 2015. Document provided under CC BY-‐NC 4.0 © Dr. Alex Gouaillard @ Citrix Systems, 2015. Document provided under CC BY-‐NC 4.0
- 50. Unified plan: 1 media => 1 m-‐line => 1 mid (FF current, next chrome, real standard) a=group:BUNDLE sdparta_0 sdparta_1 sdparta_2 a=msid-‐semanMc:WMS * m=audio 9 RTP/SAVPF 109 9 0 8 a=mid:sdparta_0 a= msid:{d57d3917-‐64e9-‐4f49-‐ad‚-‐b049d165c312} {920e9ffc-‐728e-‐0d40-‐a1b9-‐ebd0025c860a} a=ssrc:323910839 cname:{72b9ff9f-‐4d8a-‐5244-‐b19a-‐bd9b47251770} m=video 9 RTP/SAVPF 120 a=mid:sdparta_1 a= msid:{d57d3917-‐64e9-‐4f49-‐ad‚-‐b049d165c312} {35eeb34f-‐f89c-‐3946-‐8e5e-‐2d5abd38c5a5} a=ssrc:2917595157 cname:{72b9ff9f-‐4d8a-‐5244-‐b19a-‐bd9b47251770} m=video 9 RTP/SAVPF 120 a=mid:sdparta_2 a= msid:{3a2bfe17-‐c65d-‐364a-‐af14-‐415d90bb9f52} {aa7a4ca4-‐189b-‐504a-‐9748-‐5c22bc7a6c4f} a=ssrc:2325911938 cname:{72b9ff9f-‐4d8a-‐5244-‐b19a-‐bd9b47251770} hLps://tools.ieV.org/html/draW-‐roach-‐mmusic-‐unified-‐plan-‐00 © Dr. Alex Gouaillard @ Cosmo ConsulFng., 2015. Document provided under CC BY-‐NC 4.0 © Dr. Alex Gouaillard @ Citrix Systems, 2015. Document provided under CC BY-‐NC 4.0
- 51. Reminder MulM-‐cast vs Simulcast vs SVC © Dr. Alex Gouaillard @ Citrix Systems, 2015. Document provided under CC BY-‐NC 4.0 A B C MulMcast • Several tracks • Decodable separately • Bandwidth management separated HD SD QVGA Simulcast • Several tracks • Coming from the same source • Decodable separately • Smart bandwidth management possible 1 SSRC Δ2 Δ1 QVGA SVC Encoding • Several tracks • Coming from the same source • Not Decodable separately (Except base layer) • Smart bandwidth management mandatory • Less bandwidth, more resilience. 1 SSRC E E E E E E SVC Enc. D D D HD SD QVGA
- 52. Simulcast: Use case for webRTC 1.0 -‐ Use Case for webrtc 1.0: SFU -‐ Browser send simulcast, does not receive simulcast (in webrtc 1.0) Desk. HD SFU SD QVGA Cr. Book Mobile Desk. HD SD QVGA © Dr. Alex Gouaillard @ Citrix Systems, 2015. Document provided under CC BY-‐NC 4.0
- 53. Simulcast (and SVC later): 3 layers, 3 working groups, 2 standard bodies -‐ Separate the -‐ “on-‐the-‐wire” info (1), -‐ JS API (2), -‐ and signaling (3) Signaling server Web server Bob (device) Alice (browser) Media (3) (2) (1) © Dr. Alex Gouaillard @ Cosmo ConsulFng., 2015. Document provided under CC BY-‐NC 4.0 © Dr. Alex Gouaillard @ Citrix Systems, 2015. Document provided under CC BY-‐NC 4.0
- 54. Simulcast: changes on the wire -‐ On the wire work at IETF (AVTEXT): hLps://tools.ieV.org/html/draW-‐pthatcher-‐mmusic-‐rid-‐02 RID in RTP header extensions © Dr. Alex Gouaillard @ Citrix Systems, 2015. Document provided under CC BY-‐NC 4.0
- 55. Simulcast: JS API -‐ JS API aRached to the Transceiver object through the TransceiverInit, using ScaleResoluMonDownBy keyword: var sender = pc.addTransceiver( {sendEncodings: [ {rid: “F”}, {rid: “H”, scaleResoluMonDownBy: 2}, {rid: “Q”, scaleResoluMonDownBy: 4} ]} ).sender; © Dr. Alex Gouaillard @ Citrix Systems, 2015. Document provided under CC BY-‐NC 4.0
- 56. Simulcast: Offer SDP • Signaling (If all goes well in MMUSIC.... ) hLps://tools.ieV.org/html/draW-‐pthatcher-‐mmusic-‐rid-‐02 • hLps://tools.ieV.org/html/draW-‐ieV-‐mmusic-‐sdp-‐simulcast-‐03 // In the client’s descripMon a=rid:F send a=rid:H send a=rid:Q send a=simulcast rid:F,H,Q // In the server’s descripMon a=rid:F recv a=rid:H recv a=simulcast rid:F,H © Dr. Alex Gouaillard @ Citrix Systems, 2015. Document provided under CC BY-‐NC 4.0 HD SD QVGA 1 SSRC E E E
- 57. Overview © Dr. Alex Gouaillard @ Citrix Systems, 2015. Document provided under CC BY-‐NC 4.0 • Early 2015 WebRTC • Object Models APIs • Glue (Tranceiver) • ICE and OpMmizaMons • Simulcast • Others in webrtc 1.0
- 58. Screen Capture API simplificaMon -‐ No more disMncMon between screen, windows and apps. => no more constraints, flat list. • Screen Capture hnp://w3c.github.io/mediacapture-‐screen-‐share/ © Dr. Alex Gouaillard @ Cosmo ConsulFng., 2015. Document provided under CC BY-‐NC 4.0
- 59. Record your streams (local), but not all streams (remote). hRp://www.w3.org/TR/mediastream-‐recording/ Media Stream Recording (Client side) © Dr. Alex Gouaillard @ Cosmo ConsulFng., 2015. Document provided under CC BY-‐NC 4.0
- 60. Use <AUDIO> or <VIDEO> as a source ! hRp://w3c.github.io/mediacapture-‐fromelement/ Media Capture from DOM Elements && Image Capture (a.k.a. snapshot) © Dr. Alex Gouaillard @ Cosmo ConsulFng., 2015. Document provided under CC BY-‐NC 4.0
- 61. Send different streams to different outputs hRp://w3c.github.io/mediacapture-‐output/ Audio output devices API © Dr. Alex Gouaillard @ Cosmo ConsulFng., 2015. Document provided under CC BY-‐NC 4.0
- 62. hRp://www.w3.org/TR/webrtc-‐stats/ IdenMfiers for WebRTC's StaMsMcs API Full stats about data channel, codecs, FEC, … © Dr. Alex Gouaillard @ Cosmo ConsulFng., 2015. Document provided under CC BY-‐NC 4.0
- 63. Transferable DataChannels a.k.a. DataChannels in workers © Dr. Alex Gouaillard @ Citrix Systems, 2015. Document provided under CC BY-‐NC 4.0
- 64. SECURITY: IP Address Leakage 4 levels for candidate gathering -‐ All [with consent] -‐ Restricted I (host + 1918) [default] -‐ Restricted II (no host) [pref/extension] -‐ Only proxy/VPN [pref/extension] © Dr. Alex Gouaillard @ Citrix Systems, 2015. Document provided under CC BY-‐NC 4.0
- 65. Conclusion WebRTC 1.0 is Awesome [Evil Grin] © Dr. Alex Gouaillard @ Citrix Systems, 2015. Document provided under CC BY-‐NC 4.0