The number of devices and applications capable to connecting to the internet is growing day by day. Video calls, messages, photos, and audio generate lots of data traffic between devices, a representation is shown in Figure 1. Due to this, applications need to overcome the existing limitations, for instance some of them could be bandwidth, resolution, framerate, and high latency.
Fortunately, RidgeRun has been developing streaming protocol solutions to improve communication data through the network. This blog presents the RidgeRun GStreamer plugin named Kinesis WebRTC, how it works, advantages, metrics, and examples.
Figure 1. Data traffic
To understand what the Kinesis WebRTC plugin is, it is imperative to separate them as different concepts: WebRTC and Amazon Kinesis. WebRTC is an open standard specification for real-time communications (RTC). It works with mobile applications and web browsers, via simple APIs by handling peer to peer communication, providing real time data exchange by peering techniques and low latency media streaming.
It supports video, voice, and generic data to be sent between peers, allowing developers to build powerful voice and video-communication solutions. The majority of applications that involve WebRTC need the ability to have a person communicate with others in real time and in a conversational manner.
On the other hand, Amazon Web Services (AWS) is a cloud platform that offers featured services from data centers around the world, and one of them is Amazon Kinesis. This service is capable of collecting, processing, and analyzing real-time streaming data. It also incorporates a standards-compliant WebRTC implementation, so you can use Amazon Kinesis Video Streams with WebRTC to securely live streaming media, and perform two-way audio and video interaction between any camera IoT device and WebRTC compliant mobile or web players.
And this is exactly the gap where the RidgeRun Kinesis WebRTC plugin lives.
RidgeRun Kinesis WebRTC
So, the question is, how can we exploit the interaction between cloud services and the embedded platforms? The answer is by innovating new solutions such as the RidgeRun Kinesis WebRTC. This solution is a GStreamer plugin that uses Amazon Kinesis Video Streams with WebRTC SDK to provide a WebRTC compliant master endpoint. It takes advantage of the AWS connection and WebRTC capabilities in a GStreamer pipeline in order to create streaming solution applications. This can be done by the user, without any line of coding. A “plug and play” element, which is kind of the RidgeRun product’s signature. It can be built and installed in an embedded platform out of the box and start running and testing.
Currently the element supports sending and receiving OPUS encoded audio and VP8 encoded video format. However, it could be extended to support other formats included in the Kinesis API.
How does it work?
It uses a signaling channel hosted by AWS to set up and configure a peer-to-peer connection with each of the viewers connected to the same signaling channel. This configuration is done through an “offer” that is sent by the peer. Underneath that, the offer contains the necessary information to start and establish a session, which contains features to receive and send audio and/or video.
Finally, the Kinesis WebRTC element starts pushing audio/video buffers to the RTC transceiver and waits for peer buffers to push them downstream in the pipeline.
When the peer disconnects, the signaling informs through a message and the peer connection session is removed from the element. Figure 2, represents the process in a high level way and shows the applications possibilities.
Figure 2. Work process and its potential applications
Users can create applications that involve unified communications – voice and video calling, 1:1 or group sessions. Users can exploit this benefit and use the Kinesis WebRTC element to build real-time vision apps by integrating video analytics capabilities. Examples of these applications could be:
Virtual events (webinars, meetings)
Online education
Telehealth
Remote first-response applications
Some benefits of using this plugin are:
Communicate regardless of the network topologies
Embedded in the WebRTC protocol there are mechanisms that allow two peers to communicate, regardless of the network topologies they may be connected to.
Security
The data transmitted between the peers is secured using DTLS and controlled using SRTP and SCTP. It can securely stream media from lots of different edge devices and use AWS to ingest their data for playback, storage, analytics, machine learning, and other processing functionalities.
Smartphones, security cameras, RADARs, LIDARs, dash cams, drones, and depth sensors are examples of data sources that could be connected to the Kinesis WebRTC element.
Universality
It works out-of-the-box on the most popular browsers, without the need of installing external plug-ins, special codecs or additional software.
Examples and metrics
The plugin has the ability to send audio and video as separate streams, but it can send them combined as well. In order to demonstrate the functionality of the plugin, we had set a pipeline running on a x86 environment.
On the other side, we had used KVS Signaling Channel WebRTC test page and the AWS Kinesis as the endpoint caller. Below you can find the pipelines used for three different application types: send only video, send only audio, and send audio and video.
gst-launch-1.0 videotestsrc is-live=true ! queue ! video/x-raw,width=640,height=480,framerate=30/1 ! vp8enc error-resilient=partitions keyframe-max-dist=10 auto-alt-ref=true cpu-used=5 deadline=1 ! queue ! kinesiswebrtcbin channel=$SIGNALING_CHANNELSnippet code 1. Send Video only pipeline
gst-launch-1.0 audiotestsrc is-live=TRUE ! queue leaky=2 max-size-buffers=400 ! audioconvert ! audioresample ! opusenc ! audio/x-opus,rate=48000,channels=2 ! queue ! kinesiswebrtcbin channel=$SIGNALING_CHANNELSnippet code 2. Send Audio only pipeline
gst-launch-1.0 videotestsrc is-live=true ! queue ! video/x-raw,width=640,height=480,framerate=30/1 ! vp8enc error-resilient=partitions keyframe-max-dist=10 auto-alt-ref=true cpu-used=5 deadline=1 ! queue ! perf ! kinesiswebrtcbin channel=$SIGNALING_CHANNEL name=bin audiotestsrc is-live=TRUE ! queue leaky=2 max-size-buffers=400 ! audioconvert ! audioresample ! opusenc ! audio/x-opus,rate=48000,channels=2 ! queue ! bin.Snippet code 3. Send Audio and Video pipeline
One important measurement to show is the glass to glass latency, these numbers are post on Figure 3. This was tested on a NVIDIA Jetson Xavier AGX and a x86 computer. It can be appreciated that the latency is really low, it moves between a range of 142ms and 159ms. This latency is highly dependent on the video source, GStreamer pipeline, platform processing power and network topology. For your use case it could be higher or lower. RidgeRun can help you to optimize latency in your custom application.
Figure 3. Glass to glass latency
Figure 4. Kinesis WebRTC AWS Kinesis Video Stream
Figure 5. KVS WebRTC Test Page
Figure 4 and 5 show the plugin working from the AWS perspective and using a KVS Signaling Channel WebRTC test page.
Conclusion
As we have shown, this plugin is an innovative, power saving consumer way to create streaming solutions that can target embedded platforms. Connecting people with situations using embedded platforms, sending live streaming information from the active source can be done now, easier than ever before.
Do not hesitate and feel free to ask for an evaluation version to start creating new products together!
You can find more information and more examples in our developer wiki https://developer.ridgerun.com/wiki/index.php/GstKinesisWebRTC_A_GStreamer_Amazon_Kinesis_WebRTC_Plugin.
For purchase details, please go to our online store in the following link https://shop.ridgerun.com/collections/all.