Installation of the Intel RealSense Cameras on the Jetson TK1 is made possible by the use of the open source library librealsense. Looky here:
Note: This article is intended for intermediate users who are comfortable (or want to be) with Linux kernel development, and can read and modify simple shell scripts if needed.
In a previous post, Intel RealSense Camera, we discussed adding the RealSense R200 camera to the Jetson TK1.
Intel has made available an open source library, librealsense on Github. librealsense is a cross platform library which allows developers to interface with the RealSense family of cameras, including the R200. Support is provided for Windows, Macintosh, and Linux.
There are two parts to getting the R200 camera to work with the Jetson. First, operating system level files must be modified to recognize the camera video formats. When doing development on Linux based machines you will frequently hear the terms “kernel” and “modules”. The kernel is the code that is the base of the operating system, the interface between hardware and the application code.
A kernel module is code that can be loaded into the kernel image at will, without having to modify the kernel. These modules provide ancillary support for different types of devices and subsystems. The code for these modules is either in the kernel itself, in which case it is called ‘built-in’, or designated to built as a module. When built as a module the compiled code is stored separately from the kernel, typically with a .ko extension. The advantage of having a module is that it can be easily changed without having to rebuild the entire kernel. We will be building a module called uvcvideo to help interface with the RealSense camera.
The second part of getting the R200 to work is to build and install librealsense.
Kernel and Module Building
Note: In the video above, the installation was performed on a newly flashed L4T 21.4 TK1 using JetPack 2.2.
In order to build the kernel and the modules, download the kernel sources. The sources should be placed into /usr/src
A convenience script has been created to help with this task in the installLibrealsense repository on the JetsonHacks Github account.
$ git clone https://github.com/jetsonhacks/installLibrealsense.git
$ cd installLibrealsense/UVCKernelPatches
The above commands gets the repository and runs a script which downloads the kernel sources. Note: The convenience files are simple shell scripts which automate the tasks described above. They are provided as a guideline on how to accomplish the task, but due to various issues (such as network or directory layout) you may need to modify them to suit your needs.
Once the kernel sources have been downloaded and decompressed into the /usr/src directory, a configuration editor opens. In the configuration editor, set the local version of the kernel to that of the current configuration. The current local version number is available through the command:
$ uname -r
The local version number consists of the digits following the 40 in this case, i.e. -gdacac96.
Remember the – sign, it is important! This identifier is used to ensure that the module matches the build of the kernel and should match exactly. Place the local version number in the field:
General Setup -> Local version – append to kernel release:
Next, we will be modify the USB Video Class (UVC) to understand RealSense video formats.
The option to compile UVC as a module is located in:
Device Drivers -> Multimedia Support -> Media USB Adapters -> USB Video Class (UVC)
Once you find the entry, right-click on the entry until you see a small circle. The circle indicates that the option will be compiled as a module. Save the configuration file.
Patching the UVC module
A patch file is provided to apply on the module source and a shell script is provided to apply the patch. Again, these are convenience files, you may have to modify them for your particular situation.
Next, compile the kernel and module files:
This takes several minutes as the kernel and modules are built and the modules installed. Once the build is complete, you have a couple of options. The first option is to make a backup of the new kernel and modules to place them on a host system to flash a Jetson system with the new kernel. We will not be covering that here, but for reference:
The second option is to copy the kernel over to the boot directory. A convenience script is provided:
In addition to copying the new kernel into the boot directory, the newly built module, uvcvideo, is added to the file /etc/modules to indicate that the module should be loaded at boot time.
The RealSense cameras require USB 3.0. The USB port is set for USB 2.0 from the factory. Also, the stock kernel uses what is called ‘autosuspend’ to minimize power usage for USB peripherals. This is incompatible with most USB video devices. If you have not changed these settings on your TK1, a convenience script has been provided:
Now reboot the system.
Building and Installing librealsense
Once the machine has finished rebooting, open a Terminal:
$ cd installLibrealsense
This will build librealsense and install it on the system. This will also setup udev rules for the RealSense device so that the permissions will be set correctly and can be accessed from user space. Once installation is complete, you will be able to play with the examples. For example:
$ cd ~/librealsense/bin
The example allows you to set the camera parameters. Hit the ‘Start Capture’ button to start the camera.
There are Qt Creator files in librealsense which may be used to build the examples and librealsense itself. A convenience script, ./installQtCreator.sh , is provided in the installLibrealsense directory to install Qt Creator.
Intel RealSense Stereoscopic Depth Cameras is a comprehensive overview of the stereoscopic Intel RealSense RGBD imaging systems
So there you have it. This has been a little bit more involved than some of our other projects here, but if you are interested in this kind of device, well worth it.