Intel RealSense Camera Installation – NVIDIA Jetson TX1

Intel RealSense cameras can use an open source library called librealsense as a driver for the Jetson TX1 development kit. Looky here:

Background

Note: This article is intended for intermediate users who are comfortable with Linux kernel development, and can read and modify simple shell scripts if needed.

In earlier articles we talked about the Intel RealSense R200 Camera, which is a relatively inexpensive RGBD device in a compact package. The camera uses USB 3.0 to communicate with a computer.

Intel has made available an open source library, librealsense on Github. librealsense is a cross platform library which allows developers to interface with the RealSense family of cameras, including the R200. Support is provided for Windows, Macintosh, and Linux.

There are two major parts to getting the R200 camera to work with the Jetson. First, operating system level files must be modified to recognize the camera video formats. When doing development on Linux based machines you will frequently hear the terms “kernel” and “modules”. The kernel is the code that is the base of the operating system, the interface between hardware and the application code.

A kernel module is code that can be accessed from the kernel on demand, without having to modify the kernel. These modules provide ancillary support for different types of devices and subsystems.

A module is compiled code which is stored as a file separately from the kernel, typically with a .ko extension. The advantage of having a module is that it can be easily changed without having to modify the entire kernel. We will be building a module called uvcvideo to help interface with the RealSense camera. Normally uvcvideo is built-in to the kernel, we will designate it as a module as part of our modification. We will modify uvcvideo to recognize the RealSense camera data formats.

The second part of getting the R200 to work with the Jetson TX1 is to build and install librealsense.

Kernel and Module Building

Note: In the video above, the installation was performed on a newly flashed L4T 24.2.1 TX1 using JetPack 2.3

We have covered building the kernel for the Jetson TX1 in a previous article. Here are the major steps involved:

The script files to build the kernel on the Jetson TX1 are available on the JetsonHacks Github account in the buildJetsonTX1 repository.

$ git clone https://github.com/jetsonhacks/buildJetsonTX1Kernel.git
$ cd buildJetsonTX1Kernel

The script getKernelSources.sh gets the kernel sources from the NVIDIA developer website, then unpacks the sources into /usr/src/kernel.

$ ./getKernelSources.sh

After the sources are installed, the script opens an editor on the kernel configuration file. In the video, the local version of the kernel is set. The stock kernel uses -tegra as its local version identifier. Make sure to save the configuration file when done editing.

Next, patchAndBuildKernel.sh, patches one of the sources files to more easily compile the kernel:

$ ./patchAndBuildKernel.sh

and proceeds to build the kernel and modules using make. The modules are then installed in /lib/modules/3.10.96[local version name]

Install librealsense

A convenience script has been created to help with this task in the installLibrealsense repository on the JetsonHacks Github account.

$ cd $HOME
$ git clone https://github.com/jetsonhacks/installLibrealsenseTX1.git
$ ./installLibrealsense.sh

This will build the librealsense library and install it on the system. This will also setup udev rules for the RealSense device so that the permissions will be set correctly and the camera can be accessed from user space.

USB Video Class Module

Note: This step assumes that the kernel is located in /usr/src/kernel and that the kernel is to be installed on the board. If this is not your intent, modify the script accordingly. applyUVCPatch.sh has the command to patch the UVC driver with the RealSense camera formats.

The third major step is to build the USB Video Class (UVC) driver as a kernel module. This is can be done using the script:

$ ./buildPatchedKernel.sh

The buildPatchedKernel script will modify the kernel .config file to indicated that the UVC driver should be built as a module. Next, the script patches the UVC driver to recognize the RealSense camera formats. Finally the script builds the kernel, builds the kernel modules, installs the modules and then copies the kernel image to the /boot directory.

Note: The kernel and modules should have already been compiled once before performing this step. Building the kernel from scratch as described in the ‘Kernel and Module Building’ section above pulls a few shenanigans to get things to build properly the first time through.

One more minor point of bookkeeping. In order to save power, the Jetson TX1 will auto-suspend USB ports when not in use for devices like web cams. This confuses the RealSense camera. In order to turn auto-suspend off, run the following script:

$ ./setupTX1.sh

Once finished, reboot the machine for the new kernel and modules to be loaded.

Conclusion

So there you have it. This has been a little bit more involved than some of our other projects here, but if you are interested in this kind of device, well worth it.

Notes

There are several notes on this project:

  • In the video above, the installation was done on a Jetson TX1 running L4T 24.2.1 immediately after being flashed by JetPack 2.3
  • These scripts only support the 64-bit L4T series, 24.X
  • One difference between L4T 24.2 and L4T 24.2.1 is that a soft link issue with Mesa drivers has been resolved. If you are using L4T 24.2, you may have to:

    $ cd /usr/lib/aarch64-linux-gnu
    $ sudo rm libGL.so
    $ sudo ln -s /usr/lib/aarch64-linux-gnu/tegra/libGL.so libGL.so

  • QtCreator and Qt 5 are installed as dependencies in the librealsense part of the install. There are QtCreator project files located in the librealsense.qt directory. The project files build the library and example files. If you do not use QtCreator, consider modifying the installer script to take QtCreator out.
  • The librealsense examples are located in librealsense/bin after everything has been built.
  • These scripts install librealsense version v1.11.0 (last commit 74ff66da50210e6b9edc3157411bad95c209740f)
  • The RealSense R200 is the only camera tested at this time.
  • Examples using librealsense are located in librealsense/bin

12 Comments on Intel RealSense Camera Installation – NVIDIA Jetson TX1

  1. Hi,

    We have successfully made the streaming of Intel RealSense SR300 camera in L4T R24.1 by modifying the script and L4T R24.2 using the steps given above.

    Thanks kangalow for the detailed steps provided.

  2. Firstly, thanks for the detail explanation!

    However, when I follow the steps, and connect the Intel RealSense R200 to usb3.0 port of Astro carrier board for jetson TX1. The dmesg shows “can not enable port 2. Maybe USB cable bad?” The board can recognize normal usb webcam, but not the RealSense.

    Since it works well on windows, so I think is not the cable problem. Any suggestions is appreciated!!

  3. Hi,
    Unfortunately I do not have any experience with the Astro Carrier Board. If you have it plugged into the USB 3.0 port, it should enumerate properly assuming that USB 3.0 is enabled. Hope you get it figured out!

    • Thanks for your quick reply!

      I found that in the video the uvcvideo is auto loaded at the start up. However, in my case, I have to run “sudo modprobe uvcvideo” manually after reboot.

      (I am using the method stated in http://qiita.com/furushchev/items/a95acd598b7106fd52ba to boot from SATA SSD. It seems that jetson TX1 will first examine the extconfig inside bootable emmc partition, so that I have to manually copied the generated kernel to /media/ubuntu/xxxxx/boot to boot the correct kernel.)

      I am not sure if it is normal that we should manually loading uvcvideo driver?

      Thanks in advance!!

      • In the method above, the UVC Video driver is marked as an external module and built. After it is built, it is copied to the correct place in the driver file hierarchy. As you note, it is a file called: ‘uvcvideo.ko’. The local version of the OS determines placement, so it’s important that the uvcvideo module be built using the same local version as the kernel that is being used.

        With that said, is it possible that you have ucvideo.ko in the wrong place? I’m not sure where all of the drivers are loaded from when booting from the SSD, but it may be that the uvcvideo.ko may not be seen during the booting process. When booting from the internal flash as shown in the examples, it just loads like all of the other modules.

  4. when I run buildJetsonTX1Kernel/scripts/installKernelSources.sh
    at the end it doesnt open the terminal.
    I have done this before and it worked any idea on how to open this manually?
    thanks heaps for the videos

Leave a Reply

Your email address will not be published.


*


Skip to toolbar