The Stereolabs ZED Camera is introducing SDK 2.0 which includes support for the Jetson TX1 and Jetson TX2. Looky here:
The ZED Camera is a stereo depth sensor which contains two 4 megapixel imagers. For each frame, a composite of the left and right images (side by side) are sent over USB 3.0 to a host computer. The host computer then constructs a depth image using a GPU.
The small package and relatively light weight make the ZED an interesting choice for robotics and other mobile applications.
There are a variety of other APIs in the ZED SDK which do further processing. These functions include such interesting tasks as 6-axis Positional Tracking (visual odometery, no markers needed) and large-scale 3D environment mapping.
Stereolabs has a Github repository with a lot of code for project support, which is worth checking out.
The Jetson acts as the host computer for the ZED in this article.
Stereolabs recently updated their website to facilitate better interaction. Currently, you can get the SDK here.
Installing the SDK is straightforward. Download it from the website, set the permissions, and then run the installer from a terminal. The installer gives the option of also installing sample code, useful for development purposes.
While downloading and installing the SDK was simple, actually getting the ZED to work was quite challenging in the development environment here.
First, we started using the ZED camera here over a year ago. Before most of the tools could work, a firmware upgrade was needed for the camera.
The firmware must be flashed from a PC. The PC here is an older, slower machine and doesn’t appear to care much for all that is new and shiny. It runs Windows 8. The Stereolabs package was downloaded from their website and installed on the PC. The ZED was connected to the PC, and then the dreaded ‘MSVCP120.dll is missing‘ message popped up.
This is odd, as the installer made it a point to ask for confirmation when installing the related Microsoft Visual C++ Redistributable Package. Having had to support this very issue over the last decade or so, I knew how to fix it. As a developer, I understood. As a consumer, I was annoyed.
Note: In the reorganization of the website, Stereolabs now has a page for troubleshooting this very issue. Unfortunately it wasn’t around when I first encountered the issue.
The second installation issue is the ZED Camera calibration. The procedure itself is simple enough. Unfortunately the calibration is effected by the lighting conditions when the test is being run. So much so that about the only time a good calibration could be made is when the room was dark. In my house, the only time it’s dark is at night. So calibrating at night was the only option.
Worse, there are times after running what appears to be a successful calibration that the tools cannot ‘find’ the configuration files. My guess is that it actually found the calibration file, but the file contained a bad range of values and is rejected. In either case, as a developer I understand. As a consumer and user, not so much.
And to be clear, I did a fresh install of the OS, downloaded the ZED SDK, and tried to calibrate the camera a dozen times. I tried restoring calibrations that I knew worked previously, and they were not recognized either.
Once the SDK is installed and the ZED is calibrated, there are several tools and examples to explore. The ZED Depth Viewer provides a depth map and point cloud viewer, and a status on the number of frames that are being processed per second (Hz).
The frame rates for the Jetson TX2 and Jetson TX1:
|Jetson TX1 vs Jetson TX2 – ZED Depth Viewer
720p @ 60 fps – bigger is better
|Jetson TX1 (Maximum Clocks)||30|
|Jetson TX2 (Mode Max-Q)||33|
|Jetson TX2 (Mode Max-P)||43|
As we can see from the table, in low power mode the Jetson TX2 is about 10% faster than the Jetson TX1 in maximum performance mode. In maximum performance mode the TX2 is almost 50% faster. Remember that you must factor in that the input stream is 60 fps. Interesting.
So there you have it, our first look on the TX2 with the ZED API 2.0. We will be using this camera on the current iteration of the Jetson RACECAR, so we should be able to see how it works in an actual application.