Alongside Jetpack 6, a new addition to the Jetson ecosystem was added for the Orin AGX and Orin NX, that enables developers and users to quickly and easily deploy edge applications, Metropolis Microservices. Metropolis Microservices is a bundle of pre-made services and a platform for development and management of Edge AI applications. The core idea is to provide a modular approach to solve different issues, using a microservices architecture. The main concept is to create smaller pieces of software that have an API to expose their functionality to the rest of the software stack. It is available starting from JP6 for the Orin AGX and Orin NX platforms.
NVIDIA with the Metropolis Microservices toolkit provides three main component classes. The first class contains a set of pre-made services that are readily available to be used and just need to be configured. Another set of services that can be used by developers to leverage the platform and use its capabilities to the fullest extent, but still keeping the modularity that a microservices based architecture provides. Lastly there are the BSP services, these are the lower level set of services that are closer to the hardware.
Application services
NVIDIA provides a set of premade applications that can be used out of the box that enables quick and easy deployment of edge applications with very little hassle. These applications cover very basic and common needs that most edge deployments need. For example, the need for a storage management system, or live video monitoring. For those purposes NVIDIA provides three software pieces:
VST(Video Storage Toolkit)
Deepstream application
Analytics microservices.
VST
Video Storage Toolkit is a basic piece on most edge deployments, it allows for efficient management and storage of video streams on Jetson. It allows the user to add, remove and control cameras, it also provides access to file-based streams to retrieve information and allows concurrent video playback. With Video wall support for live video monitoring. It supports autodiscovery of ONVIF S-profile devices and enables hardware h264 and h265 encoding of the incoming streams. On image 1, we see the VST dashboard by accessing the url http://JETSON_IP:30080/vst/#/vst/dashboard.
We can use the rtsp-sink source to emulate an IP camera and generate a stream that VST is able to read and attach to. By using the following GStreamer pipeline:
gst-launch-1.0 videotestsrc is-live=true ! "video/x-raw,height=720,width=1280,framerate=30/1" ! nvvidconv ! nvv4l2h264enc insert-sps-pps=true idrinterval=30 ! "video/x-h264,mapping=/stream1" ! rtspsink service=6000In this example we are just taking the video from the videotestsrc element, encoding it to h264 and sending it with the rtspsink element. After we create the stream, then we can set it up on VST, as shown on figure 2.
After doing so, we can access the stream and play it back on the VST WebUI, as seen on figure 3.
DeepStream
To make use of the streams that VST manages, NVIDIA provides the Deepstream microservice, an application that enables the use of PeopleNet model for object detection and tracking using the NvDCG plugin. The application can be set to use the efficient DLA cores on the Jetson to provide real-time video processing. Alongside the Deepstream microservice there is the SDR (Sensor Distributing and Routing) service, a service that allow the autodiscovery of VST streams and can add them of the fly making it very easy to use.
Analytics Microservices
This module includes a gamut of microservices that provide analytics for zones that are monitored by cameras and provides notifications with timestamps, bounding boxes for the events. With this microservice stack users can set-up trip-wire, FOV and ROI monitoring. The three of them can be easily configured with rules, for example flow-rate alerts, occupancy alerts, object tracking after the event or simply just single event notifications. All of it is exposed through a simple Web API for other microservices or applications to use and retrieve the detected events.
Platform services
Alongside the application services, NVIDIA provides another set of microservices that enable access and control to the platform. Developers can leverage these microservices to enable their applications to use the Jetson and its capabilities to the fullest. Developers can control Network configurations, Storage setup, Redis bus, Ingress service and more. Included on this stack there are also monitoring microservices that enable developers to use tools like Grafna or Prometheus to set alerts. For example we have the jetson-monitoring microservice that provides this nice web based monitoring tab, where everything on the Jetson can be monitored, this can be accessed on the url http://JETSON_IP:3000/grafana/d/rYdddlPWk/system-metrics, and we can see the dashboard as shown on figure 4.
NVStreamer
NVIDIA provides a tool to generate RTSP streams from sample files, making it easier to test the microservice stack and other software that requires a RTSP stream. It has a simple web based UI that users can enter and set up on either Jetson devices or even on x86 hosts. Users can then upload files and generate any number of streams. For example running the NVIDIA sample file we generate a stream and add it to VST. We can see the main NVStreamer dashboard running on the Jetson by accessing the url http://JETSON_IP:31000/#/streamer/dashboard, as shown on figure 5.
So now on VST we can have a video wall with multiple sources, and get an end result with all video streams, and that will look like figure 6.
And then we can use monitoring services to check the system status, as can be seen on figure 7
How can RidgeRun help?
By using NVIDIA’s Metropolis Microservices, multiple possibilities open up for customers, and RidgeRun could become your engineering partner for success, getting the most out of this new framework. RidgeRun’s expertise on RTSP, Deepstream, ONVIF, and Video Processing could be leveraged to speed up your product time to market, optimize your application, and have an NVIDIA Jetson technology expert alongside in your development journey. Together we can make use of the microservices architecture and offerings from NVIDIA to build robust end to end edge systems that leverage the Jetson’s potential to the fullest.
For technical questions or any service inquiry, please send an email to support@ridgerun.com or send a message through https://www.ridgerun.com/contact.