Intel RealSense Camera librealsense – NVIDIA Jetson TX Dev Kits

Intel RealSense cameras can use an open source library called librealsense as a driver for the Jetson TX1 and TX2 development kits. Looky here:

Background

With the release of L4T 28.1, both the Jetson TX1 and Jetson TX2 run on a Linux 4.4 version kernel. The 4.4 kernel has built-in support of the RealSense camera formats in the UVC video module. Earlier versions of L4T running an earlier kernel version required that the kernel be rebuilt before using a RealSense camera. Here’s an earlier article for reference.

That’s great news, as it simplifies installation of the RealSense driver library, librealsense. Without having to rebuild the kernel, life be good.

Install librealsense

A convenience script has been created to help with this task in the installLibrealsense repository on the JetsonHacks Github account.

For the Jetson TX2:

$ cd $HOME
$ git clone https://github.com/jetsonhacks/installLibrealsenseTX2.git
$ cd installLibrealsenseTX2
$ ./installLibrealsense.sh

For the Jetson TX1:

$ cd $HOME
$ git clone https://github.com/jetsonhacks/installLibrealsenseTX1.git
$ cd installLibrealsenseTX1
$ ./installLibrealsense.sh

This will build the librealsense library and install it on the system. This will also setup udev rules for the RealSense device so that the permissions will be set correctly and the camera can be accessed from user space. There is a patch applied during the installation which fixes an issue with the UVC video module not being recognized. This issue has been addressed upstream in the librealsense repository, but is not in a release just yet.

Note: At this point, both of the Jetson scripts are identical because the same kernel is being used. Prior to the release of L4T 28.1, the Jetson TX2 and Jetson TX1 ran different kernel versions.

Notes

Here are some notes:

  • In the video above, the installation was done on a Jetson TX2 running L4T 28.1 immediately after being flashed by JetPack 3.1
  • Librealsense now uses CMake as its build system.
  • QtCreator and Qt 5 are installed as dependencies in the librealsense part of the install. There are QtCreator project files located in the librealsense.qt directory. The project files build the library and example files. If you do not use QtCreator, consider modifying the installer script to remove QtCreator.
  • These scripts install librealsense version v1.12.1
  • The RealSense R200 is the only camera tested at this time.
  • Some people have reported issues when using the camera directly attached to the Jetson USB 3.0 port. A powered USB 3.0 hub is recommended
  • The RealSense cameras are USB 3.0 devices. If the camera is attached to a USB hub, the USB hub must support USB 3.0
  • Examples using librealsense are located in ~/librealsense/build/examples
  • Intel RealSense Stereoscopic Depth Cameras is a comprehensive overview of the stereoscopic Intel RealSense RGBD imaging systems

25 Comments

  1. HI I tested it with the the Tx2 with jetpack 3.1 and it works perfectly well

    The example work with the ZR300 and R200 cameras.

    but when i tried to install ros-kinetic-librealsense on top it , it gave the following error:
    awk: cannot open /proc/1/sched (no such file of directory)
    INFO: Docker enviroment not supported for udevadm:skippin.
    processing trigger for libc-bin(2.23-0ubuntu3) …

    was the ros-kinetic-librealsense package installed properly?

    the next step of compiling ros-kinetic-realsense-camera also works and all launch files are created but, the zr300 camera is not detected in the rgbd.launch files

  2. Hi!

    I have follow your intructions to install librealsense in my TX1 (L4T 28.1), but it didn’t work. The library is compiled and installed, but it fails to detect my SR300 camera. Do you have any idea of which could be the problem?

    Also, I have tried to install the new Intel Realsense SDK version (SDK 2.0) with the same results. Have you tried this version or are yo planning to do it soon?

    Any way, thanks you very much for all your videos and tutorials.

    • Hi Pedro.

      I just tried realsense SDK2 on TX1. On running samples, it showed me error: RealSense error calling rs2_pipeline_start(pipe:0x4bd5b0):
      Requested pixel format is not natively supported by the Linux kernel and likely requires a patch for fourcc code INZI!
      Alternatively please upgrade to kernel 4.12 or later. Last Error: Inappropriate ioctl for device.

      Can you please suggest how you managed to run sdk2 on TX1?
      P.S On realsense SKD 1.12, my depth camera is unable to extract data, if you can help here.

  3. Hi. I have tried your script on TX1 flashed with Jetpack 3.1. It does work with R200 but not with S300; I am using a USB3 hub. In librealsense examples, few depth samples are working but with flickering results.

    Do you think I would need to rebuild kernel or would there be any solution for it?

  4. I am having the same problem as Haroon. Brand new TX2 with Jetpack 3.1, freshly flashed w/ LT28.1, SR300, using the script provided that checks out v1.12.1. Flickering Depth, no IR, and RGB.

    I also tried to install the SDK2 on a TX2, and I got the same error as Haroon again:
    Requested pixel format is not natively supported by the Linux kernel and likely requires a patch for fourc code INZI!
    Alternatively please upgrade to kernel 4.12 or later. Last Error: Inappropriate ioctl for device.

    The RGB camera works, but no IR or Depth. Any ideas about what might be happening?

    • Unfortunately I do not have a SR300, so I cannot reproduce or help solve your issue. You will have to figure out how to patch the kernel with the appropriate video mode INZI

      • Haroon figured out enough to get the RealSense SDK 2.0 with the Jetson TX2. You can find out more here: https://github.com/syedharoonalam/buildJetsonTX2Kernel

        I found myself having to apply the realsense-camera-formats.patch manually first, and then applying Haroon’s patch. My guess is that I could have probably skipped the realsense-camera-formats.

        I’m still having a major issue with the RealSense cam not picking up frames right away, it times out about 50% of the time on the USB port. I tried canceling autosuspend in the extlinux.conf file, getting a 60W powered external hub, and neither helped.

        Any ideas why this would be happening?

  5. HI I tested it with the the Tx2 with jetpack 3.1 L4T 28.1
    But it didn’t work,when I run “cpp-capture”, it returned me the error information like

    RealSense error calling rs_create_context(api_version:11201):
    UVCIOC_CTRL_QUERY:UVC_GET_CUR error 56, Invalid request code

    I tested with R200 camera.How can I fix it? Thank you!

    • Hey, did u finally fix this problem? I met a familiar error with u. The error info is:
      ***
      rs.warn: Camera head contents version != 12, data may be missing/incorrect
      Capturing DEPTH at 480 x 360, fov = 55.6 x 43.2, distortion = NONE
      Capturing COLOR at 640 x 480, fov = 54.5 x 41.9, distortion = MODIFIED_BROWN_CONRADY
      Capturing INFRARED at 480 x 360, fov = 55.6 x 43.2, distortion = NONE
      Capturing INFRARED2 at 480 x 360, fov = 55.6 x 43.2, distortion = NONE
      RealSense error calling rs_start_source(device:0x48b840, source:VIDEO):
      VIDIOC_S_FMT error 5, Input/output error
      ***

      I used a R200 on Tx2 with L4T 28.2

  6. I have tried your example with SR300 on Jetson TX2 and the installation went good, but when I run the examples, I see no images, I only get to see a a screen, half black and half white with cpp-config.ui and a pale blue screen with pointcloud example. What am I missing, why I am not able to see images, any help?

  7. I got this to work with the r300, but am unable to get it to work with the r400 (specifically the r435). I got an email from Intel suggesting that I use realsenselib2, whereas this script apparently uses v1.12.1. I tried blindly using version 2.7.7 (the earliest version 2 available) and this didn’t work either. Suggestions?

  8. I’ve tried Jetpack 3.2 with success.
    –then had to revert back to 3.1 as installing a Zed camera keep complaining I needed CUDA 8–when the release notes says it’s CUDA 9 compliant. Nonetheless:

    Download librealsense2 from github and run the new realsense-viewer from the tools section. Should work with the D4xx series as I have it working here in both 3.1/3.2. And absolutely make sure you have the camera connected to a powered hub otherwise you’ll get uvc errors. Do not run the libRS modified uvcvideo.ko script–it will segfault on modprobe. Lastly make sure you have no other camera brands connected. I had the Zed connected and libRS tried to initialize it. cpp-config.ui is going away to the realsense-viewer app.

    Also ,make sure your user is added to the plugdev and video groups.

  9. I found this invaluable last year when I was using an R200 camera. Thanks!
    Are you planning to cover the install of the D435? Maybe also checking it’s integration with JetPack 3.2 and ROS? That would be fantastic!!
    Best Regards, Mike

  10. I have successfully installed the D435 on regular Ubuntu, but continue to fail on the TX2. During compilation I get a fatal error:

    /home/nvidia/librealsense/src/image.cpp:23:19: fatal error: cpuid.h: No such file or directory
    compilation terminated.
    CMakeFiles/realsense2.dir/build.make:374: recipe for target ‘CMakeFiles/realsense2.dir/src/image.cpp.o’ failed
    make[2]: *** [CMakeFiles/realsense2.dir/src/image.cpp.o] Error 1
    CMakeFiles/Makefile2:67: recipe for target ‘CMakeFiles/realsense2.dir/all’ failed
    make[1]: *** [CMakeFiles/realsense2.dir/all] Error 2
    Makefile:127: recipe for target ‘all’ failed
    make: *** [all] Error 2

    Any ideas welcome, as the whole point of this is to get the D435 in the air! 🙂

  11. Hi, can you help me? I have a problem about uvcvideo.(realsense r200)

    My environment is :
    NVIDIA Jetson TX2
    L4T 28.1.0 [ JetPack 3.1 ]
    Board: t186ref
    Ubuntu 16.04.4 LTS
    Kernel Version: 4.4.38-tegra
    CUDA 8.0.84

    dmesg,
    [14561.687162] uvcvideo: version magic ‘4.4.131+ SMP preempt mod_unload aarch64’ should be ‘4.4.38-tegra SMP preempt mod_unload aarch64’
    [14561.688253] uvcvideo: version magic ‘4.4.131+ SMP preempt mod_unload aarch64’ should be ‘4.4.38-tegra SMP preempt mod_unload aarch64’
    [14561.696997] uvcvideo: version magic ‘4.4.131+ SMP preempt mod_unload aarch64’ should be ‘4.4.38-tegra SMP preempt mod_unload aarch64’
    [14827.937962] uvcvideo: version magic ‘4.4.131+ SMP preempt mod_unload aarch64’ should be ‘4.4.38-tegra SMP preempt mod_unload aarch64’

    lsusb, can find r200
    Bus 002 Device 004: ID 8086:0a80 Intel Corp.
    Bus 002 Device 002: ID 8564:4100 Transcend Information, Inc.
    Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
    Bus 001 Device 004: ID 10c4:8105 Cygnal Integrated Products, Inc.
    Bus 001 Device 003: ID 2a7a:9a18
    Bus 001 Device 002: ID 8564:4100 Transcend Information, Inc.
    Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

    lsmod,
    Module Size Used by
    videobuf2_vmalloc 6326 0
    fuse 89008 2
    bcmdhd 7625819 0
    pci_tegra 74691 0
    bluedroid_pm 13564 0

    Thanks you.

  12. Hi all,
    I had issues in the installation,

    Package qtcreator is not available, but is referred to by another package.
    This may mean that the package is missing, has been obsoleted, or
    is only available from another source
    However the following packages replace it:
    qtchooser

    E: Package ‘qtcreator’ has no installation candidate
    Reading package lists… Done
    Building dependency tree
    Reading state information… Done
    Package cmake-curses-gui is not available, but is referred to by another package.
    This may mean that the package is missing, has been obsoleted, or
    is only available from another source
    However the following packages replace it:
    cmake-doc

    Somebody have a solution?

    • There’s not much information here. Which script where you running when you encountered the issue. The qtcreator package is in the universe repository, you should be able to load it. Which Jetson are you running the script on, and which version of L4T are you running?

Leave a Reply

Your email address will not be published.


*