With a USB firmware patch and an updated libfreenect2, the Microsoft Kinect V2 now runs on the Jetson TX1 Development Kit. Looky here:
For a stretch there it was not possible to run the open source Kinect V2 driver libfreenect2 on the Jetson TX1 because of an issue with the USB firmware. Fortunately NVIDIA has issued a firmware patch
(see the Jetson TX1 Forum, USB 3 Transfer Failures ) which fixes the issue. As you might recall, Microsoft now offers the Kinect V2 as a two part kit a Xbox One Kinect Sensor Bar along with a Kinect Adapter for Windows. You will need both a Kinect Xbox One sensor and the adapter for use with the Jetson, or the discontinued Kinect for Windows. The Kinect Adapter for Windows converts the output from the Kinect to USB 3.0. The advantage of this setup is that you can use the Kinect sensor from your Xbox One, or at least have an excuse to get a Xbox One + Kinect for “research” purposes.
The installLibfreenect2 repository on the JetsonHacks Github account contains convenience scripts for installing libfreenect2 and the USB firmware patch. First, get the repository:
$ git clone https://github.com/jetsonhacks/installLibfreenect2
Second, install libfreenect2 and compile the library and examples:
$ cd installLibfreenect2
Third, you will need to patch the USB firmware:
After installing the USB firmware patch, it is necessary to reboot the machine in order for the firmware changes to take effect.
When the machine reboots, you can run the example:
$ cd ~/libfreenect2/build/bin
The installation of libfreenect2 in the video is on L4T 24.1, flashed by JetPack 2.2. CUDA is required. Both 32 bit and 64 bit versions of L4T are shown in the video, installation of libfreenect2 and the firmware patch is the same in both cases.
The Mesa libs installed by libfreenect2 overwrite libGL.so, which causes issues. The fix as of this writing (July, 2016) is to link libGL.so to the Tegra version.
The repository contains a script jetson_max_l4t.sh which sets the CPU and GPU to maximum clock values. This will increase the Jetson TX1 performance at the cost of power consumption.
L4T 23.X Notes
The JPEG decompressor under L4T 23.X produces RGBA format, where as the Protonect viewer consumes BGRA format. This makes the video appear with a purple hue.
Update 8-28-16 In the NVIDIA Jetson Dev Forum user kassinen wrote:
Actually you can fix it by changing the line 56 from viewer.h
typedef ImageFormat<4, GL_RGBA, GL_BGRA, GL_UNSIGNED_BYTE> F8C4;
typedef ImageFormat<4, GL_RGBA, GL_RGBA, GL_UNSIGNED_BYTE> F8C4;
The repository contains a patch which adds a simplistic algorithm to rearrange the bytes appropriately. If you intend to use this script with L4T 23.X, you will need to uncomment the line:
# patch -p 1 -i $PATCHDIR/bgra.patch
Also, if you plan to use this library in production L4T 23.X code, your application should consider writing specialized code to do the RGBA→BGRA conversion more efficiently.
For L4T 24.1, there is no patch applied as the JPEG decompressor produces BGRA format natively.