Multi-Sensor System Demo Platform Shows How FPGAs And Xilinx IP Enable Reliable, Real-Time Accuracy
Looking both ways before you cross the street is a good way to avoid catastrophe. Almost instantly, your brain can analyze information from your optic nerve, determine it’s an oncoming car, and stop your leg muscles from moving you in its way.
For the car, the same process has a few more steps.
It takes a massive amount of computing capacity to analyze information from digital cameras, determine size and speed from differences in the contrast of individual pixels, identify a human moving into the path of the vehicle, and synthesize that data into a message to the braking system.
In the end, you just hope the message arrives on time.
Legacy embedded systems, even those reliant on multi-core processing technology, lack the capacity to process large amounts of video data and fuse it with live GPS or satellite positioning information in order to perform time-sensitive control of a vehicle. FPGAs and Xilinx’s MPSoC technology provide an answer, allowing some of those demands on the processing system to be handed over to custom logic in the hardware. The Multi-Sensor System Demo Platform by DornerWorks shows a real-time multi-stream video system enabled by Xilinx IP running almost entirely on custom logic and hardware resources.
With the specialized IP demonstrated on the Zynq UltraScale+ MPSoC ZCU102 Evaluation Board, four separate video streams are brought in over a single DisplayPort interface. The streams are scaled and mixed with a software-defined video overlay using a video mixer with alpha blending and windowing. The resultant video stream can be output over HDMI, DisplayPort, HD-SDI, or a custom analog FMC. The stream can also be compressed and encoded using the Video Codec Unit (on the ZCU106, for example) and sent deterministically over Ethernet using the DornerWorks AVB MAC IP.
- Xilinx Video Mixer v3.0
- Xilinx Video Processing Subsystem v2.0
- Xilinx Video Test Pattern Generator v8.0
- Xilinx HDMI Transmitter Subsystem v3.1
- Xilinx DisplayPort RX Subsystem v2.0
- Xilinx Video PHY Controller v2.2
The block diagram below shows how the system works.
This demonstration was first shown off on the ZCU102 at the 2019 Xilinx Worldwide Sales Conference in Los Angeles. It can accelerate designs for video conferencing, surveillance, Advanced Driver Assisted Systems (ADAS), and streaming and encoding applications. Moreover, a system preconfigured with this amount of custom IP could help kickstart business development on video-based products.
DornerWorks is currently porting the demo to the ZCU106 and integrating the DornerWorks’ AVB MAC IP that will further enhance this platform by providing the capability to stream compressed video over Ethernet with high accuracy (+/-8ns) and minimal footprint. This IP can enable reliable data fusion for autonomous systems in the automotive or industrial markets, or anywhere else you need real-time video streaming synchronously over Ethernet.
Schedule a free consultation with DornerWorks and we will develop a plan to integrate this technology into your products. With a time-synchronous video streaming solution, you can lead the market by delivering the real-time results your customers deserve.
Engineer David Verbree contributed to this post.