This article has been superseded. Please see: Install Kinect V2 – Part II
The article below is for historical purposes only, and does not reflect the current state of the project.
Open source drivers (OpenKinect libfreenect2) are available for the Kinect for Windows V2. Most installations are for PC based, x86 machines. The Jetson TK1, being an ARM V7 machine poses some special challenges for porting the source over. Most of those issues have been addressed, and an installation script is now available. Looky here:
While most of the porting issues are straightforward, some special code needed to be written to get good performance out of the Kinect sensor on the Jetson. There are also some hardware issues that need to be addressed.
Fortunately Lingzhu Xiang added Tegra support for the Jetson to libfreenect2 which addressed the performance issues. The code is available on Github at: https://github.com/xlz/libfreenect2. Here is the “how to”: Jetson TK1 HOWTO. The Tegra code allows frame processing at 60Hz. Way excellent!
You might remember from an earlier post (Kinect V2 with libfreenect2) that there were issues using the built in USB controller. So a Syba 19-Pin USB 3.0 Header Mini PCI-Express Card with Female USB 3.0 Cable SD-MPE20142 was acquired. Note: It is a full size Mini PCI-Express card, so it does hang over the edge of the Jetson when installed. Also, the card requires a floppy drive power connector. I found one at a local Frys electronics store that hooked into the Jetson onboard Molex power connector.
Also, remember that in order to use the Kinect for Window V2 with the Jetson, the Kinect needs to be initialized with a Windows 8.1 machine.
This installation was run on a Jetson with L4T 21.2 with OpenCV and CUDA 6.5 installed.
(See JetPack installation). After the flash installation, the Post Flash Setup procedure was performed, which prepares the Jetson for desktop operation. After using JetPack and performing the post flash setup, the Jetson is prepared as the starting point for libfreenect installation.
Another note, the Jetson feature of USB autosuspend needs to be disabled for the Kinect V2 to work correctly. This is taken care of by the post flash setup mentioned above.
Issues, Issues, Issues …
When I tried to install Lingzhu Xiang’s code, I ran into several issues. The installation script below to install libfreenect2 addresses these issues.
The first issue is that the original installation recommended installing opencv-dev. Unfortunately this overwrites the hardware accelerated Tegra OpenCV library that JetPack installs. This was easy enough to rectify, the only real issue was that the include file turbojpeg.h was missing. I built a little repository for the installation that was forked from Lingzhu Xiang’s and added turbojpeg.h to it. Also, the link to libturbojpeg.so needed to be moved around a little.
The second issue is with libusb. It appears as if libusb/libfreenect2 was changed after the original repository was published. In this installer, there is a patch included for the libusb headers (usbfs.h) which sets the MAX_ISO_BUFFER_LENGTH to be 49152.
Note: Earlier versions of libfreenect2 worked with the built in USB 3.0 of the Jetson, as Lingzhu Xiang reported. The current version (even with the patch) does not. This is an outstanding issue.
By adding the file 90-kinect2.rules to the udev permissions, Kinect V2 is given permission to run in user space. This is included in the script setup.
For the demo, a colormap of the depth map is added. By pressing one of the number keys, different colormaps can be used.
Note: The NVIDIA Gstjpeg headers are needed to enable hardware acceleration and JPEG decoding. These headers are downloaded using the installation script and placed into the libfreenect/depends directory.
The installation script is available on Github:
|# Install Prerequisites|
|#opencv-dev -> opencv4tegra-dev -> already installed from Jetpack|
|sudo apt-get install -y build-essential libturbojpeg libtool autoconf libudev-dev cmake mesa-common-dev freeglut3-dev libxrandr-dev doxygen libxi-dev libjpeg-turbo8-dev|
|# Get libfreenect2|
|git clone https://github.com/jetsonhacks/libfreenect2.git|
|# Copy nv_headers into libfreenect2/depends|
|tar -xvf gstjpeg_src.tbz2 gstjpeg_src/nv_headers|
|mv gstjpeg_src/nv_headers/ libfreenect2/depends/|
|# Cleanup the empty folder|
|# build libusb and glfw|
|#fix up libturbojpeg.so linker doesn't get upset|
|sudo ln -s /usr/lib/arm-linux-gnueabihf/libturbojpeg.so.0.0.0 /usr/lib/arm-linux-gnueabihf/libturbojpeg.so|
|# Build Protonect example|
|make && sudo make install|
|# Install the udev rule so that you always have read/write permission to the Kinect V2|
|sudo cp extras/90-kinect2.rules /etc/udev/rules.d/90-kinect2.rules|
|/bin/echo -e "\e[1;32mFinished.\e[0m"|
Download it to the Jetson. In the video above, the script is placed into the Home (~/) directory.
$ sh installLibfreenect2.sh
This will install the code dependencies, download the libfreenect source and dependent libraries, and compile them.
Once the installation is complete, you can reboot the Jetson. This will enable the udev Kinect rules, which allows user access to the Kinect. If you do not reboot the Jetson, you will have to use the ‘sudo’ command before running the demo.
Switch to the demo directory:
$ cd libfreenect2/examples/protonect/bin
(This assumes that libfreenect2 was installed in your home directory). To run the demo:
If all went well, you should see the depth and rgb images.