Jetson Nano – RealSense Depth Camera

Getting full support for the Intel RealSense Depth Camera on the NVIDIA Jetson Nano Developer Kit is simplified by using a couple of installation scripts. Looky here:

Introduction

In an earlier article, we installed an Intel RealSense Tracking Camera on the Jetson Nano along with the librealsense SDK. We’ve have used the RealSense D400 cameras a lot on the other Jetsons, now it’s time to put them to work on the Jetson Nano.

For best performance and support of the RealSense Depth Camera features, Intel recommends modifying the Linux kernel and modules.

To remind you of the different cameras available, here’s a couple of the more popular models which points out their features:

Intel® RealSenseTM Depth Camera D415

  • Intel® RealSenseTM Vision Processor D4
  • Up to 1280×720 active stereo depth resolution
  • Up to 1920×1080 RGB resolution
  • Depth Diagonal Field of View over 70°
  • Dual rolling shutter sensors for up to 90 FPS depth streaming
  • Range 0.3m to over 10m (Varies with lighting conditions)

Intel® RealSenseTM Depth Camera D435/D435i 

  • Intel® RealSenseTM Vision Processor D4
  • Up to 1280×720 active stereo depth resolution
  • Up to 1920×1080 RGB resolution
  •  Depth Diagonal Field of View over 90°
  • Dual global shutter sensors for up to 90 FPS depth streaming
  • Range 0.2m to over 10m (Varies with lighting conditions)
  • Intel® RealSenseTM Depth Camera D435i includes Inertial Measurement Unit (IMU) for 6 degrees of freedom (6DoF) data

For robotics applications, the D435 is popular due to its global shutter and wide field of view.

Software Installation

To interface with the camera,  Intel provides the open source library librealsense. On the JetsonHacksNano account on Github, there is a repository named installLibrealsense. The repository contains convenience scripts to install librealsense.

In order to use the install script, you will either need to create a swapfile to ease an out of memory issue, or modify the install script to run less jobs during the make process. In the video, we chose the swapfile route. To install the swapfile:

$ git clone https://github.com/jetsonhacksnano/installSwapfile
$ cd installSwapfile
$ ./installSwapfile.sh
$ cd ..

You’re now ready to install librealsense.

$ git clone https://github.com/jetsonhacksnano/installLibrealsense
$ cd installLibrealsense
$ ./installLibrealsense.sh

While the installLibrealsense.sh script has the option to compile the librealsense with CUDA support. If you want to add CUDA support to the librealsense SDK, add a -c switch to the shell script:


$ ./installLibrealsense.sh -c

The location of librealsense SDK products after installation:

  • The library is installed in /usr/local/lib
  • The header files are in /usr/local/include
  • The demos and tools are located in /usr/local/bin

Kernel and Modules

For the RealSense Depth Cameras, you will find that performance is much better if you apply the patches to the kernel models. Note: If you have a D435i, the camera will not be detected without the patches.

This video covers the differences between only installing librealsense versus librealsense plus the kernel modifications. Looky here:

To install the kernel and module patches, build them, and then install first switch to the installLibrealsense directory then:

You’re now ready to install librealsense.


$ ./patchUbuntu.sh

This will start the patch, build and install process. On a micro SD card this will take ~ one hour, 20 minutes. Note: If you compile the kernel and modules on a USB SSD, remember to copy the new Image to the /boot directory of the SD card which you boot from.

Demos

Go to the demos and tools directory in /usr/local/bin, and checkout the realsense-viewer application and all of the different demonstrations! There are a wide variety of code samples for different uses in the librealsense SDK. When you go to program against the SDK, you’ll benefit from having a good catalog with which to work.

16 Comments

  1. Hi,

    I followed your instructions with a fresh Jetson Nano install. it all seems to work well but my D435i does not connect. I can see the device in cheese and via libusb.

    When i run realsense-viewer it does not show the d435i

    it says ;

    (ds5-factory.cpp:678) DS5 group_devices is empty.
    5 times.

    Is there any step before or after i should be performing?

    Thanks for all your work!

      • is there any way to check where they may have gone wrong- because this is exactly what happened to me too- and I’ve updated the FW and tried re-flashing the sd card as well to no avail.

        • I’ve modified patchUbuntu.sh as follows and my D435i works fine with Nano.

          patchUbuntu.sh Line 143

          Before :
          sudo ./scripts/configureKernel.sh

          After:
          sudo ./scripts/configureKernelUSB.sh

          • Thank you very much for sharing this. This gave me the clue as to what was wrong with the scripts, which I was able to fix relatively easily after this receiving this information. I really appreciate your contribution, it saved me a lot of time coming up with a fix!

    • Hi,

      I had the exact same issue from a fresh Jetson Nano install… Flashed fresh SD card. Setup Nano. Installed swap file and reboot. Run installLibrealsense script. Wait 1-2 hours. Run patchUbuntu script. Wait 1-2 hours. Reboot. Test Camera D435i…”(ds5-factory.cpp:678) DS5 group_devices is empty.” and not recognized by realsense viewer.

      So wondering if something had silently failed during the kernel rebuild, I ran the patchUbuntu script a second time, and now it works.

      So you might want to try that too.

      BTW.. Jim, thanks so much for posting this info and scripts. It’s been really frustrating trying to get things up and running on the Nano and the info on your website has been invaluable.

      Cheers,
      Will

      • Thank you very much for the kind words. Your description allowed me to fix the scripts for the D435i.

        I understand the pain in getting everything to work, it’s taken me about 3 long weeks to get the RealSense cameras working properly. And this is after having set them up on other Jetsons in the past!

        However, once the scripts are written and in this case people test them out, a lot of people benefit. When the official release of the Jetson Module and new motherboard comes out, it will probably be worth it to come up with an disk image with librealsense pre-installed. Thanks for the helping!

  2. Hi, Curious if this will work with the r200? Or will following the tx1/2 tutorial for the r200 work for the nano as well?

    • This is for librealsense2, the R200 requires the legacy librealsense. You would need to figure out how to patch the kernel and modules for version 4.9. Thanks for reading!

  3. Hello, i found the following issue and could not solve it.

    Current working directory: /usr/src/kernel/kernel-4.9
    bash: scripts/config: No such file or directory
    make: *** No rule to make target ‘olddefconfig’. Stop.

    Thank you so much !!

Leave a Reply

Your email address will not be published.


*