Porting to a new platform is rarely easy. The steps are pretty straightforward, but all the niggles add up to make it easy to fiddle away the better part of a couple days for even easy projects. For a new development board, this is especially true. In my case, I haven’t been around Linux recently enough to be entirely comfortable.
With that said, here’s all the tips and tricks I did to get OpenNI2 running on the Jetson to interface with the Occipital Structure Sensor. It might not be optimal, and it may not be absolutely correct, but it appears to work. We all know, to look good is to be good. Appearances are everything.
Remember this is all pretty low level stuff, drivers for the sensor, usb interface, etc. So there’s a lot of little stuff that needs to be done.
For the sake of the video, I installed the dependencies before going on camera. You should be able to install the dependencies without any issues. Here’s the steps I took (the steps should follow the steps in the video):
# Install the dependencies for OpenNI2
$ sudo apt-get install -y g++ python libusb-1.0-0-dev freeglut3-dev doxygen graphviz
# Added 6-15-2015; Sometimes libudev is missing, add it just to be sure
$ sudo apt-get install libudev-dev
# Navigtate to where you would like to build, then:
$ git clone https://github.com/occipital/OpenNI2.git
$ cd OpenNI2
# Set the compile flags to build for the Jetson
$ gedit ThirdParty/PSCommon/BuildSystem/Platform.Arm
Change:
CFLAGS += -march=armv7-a -mtune=cortex-a9 -mfpu=neon -mfloat-abi=softfp #-mcpu=cortex-a8
to:
CFLAGS += -march=armv7-a -mtune=cortex-a15 -mfpu=neon-vfpv4 -mfloat-abi=hard
# Save the file
# You should also modify ThirdParty/PSCommon/BuildSystem/CommonCppMakefile
# to add the lpthread library.$ gedit ThirdParty/PSCommon/BuildSystem/CommonCppMakefile
Here's the diff:
--- OpenNI2-2.2.0.30/ThirdParty/PSCommon/BuildSystem/CommonCppMakefile.old 2014-03-28 19:09:11.572263107 -0700
+++ OpenNI2-2.2.0.30/ThirdParty/PSCommon/BuildSystem/CommonCppMakefile 2014-03-28 19:09:55.600261937 -0700
@@ -95,6 +95,9 @@
OUTPUT_NAME = $(EXE_NAME)
# We want the executables to look for the .so's locally first:
LDFLAGS += -Wl,-rpath ./
+ ifneq ("$(OSTYPE)","Darwin")
+ LDFLAGS += -lpthread
+ endif
OUTPUT_COMMAND = $(CXX) -o $(OUTPUT_FILE) $(OBJ_FILES) $(LDFLAGS)
endif
ifneq "$(SLIB_NAME)" ""# Save the file
# Add Samples and Tools to the build chain if desired
$ gedit Makefile
# Add these two lines to the end of the file:
core_samples: $(CORE_SAMPLES)
tools: $(ALL_TOOLS)
# Save the file
# You're ready to build!
$ make
$ make core_samples # this probably isn't necessary, they should already be built
#GLUT_SUPPORTED tells the make to compile NiViewer for OpenGL
$ GLUT_SUPPORTED=1 make tools# Build and install libfreenect
$ sudo apt-get install cmake freeglut3-dev pkg-config build-essential libxmu-dev libxi-dev libusb-1.0-0-dev -y$ git clone git://github.com/OpenKinect/libfreenect.git
$ cd libfreenect
$ mkdir build
$ cd build
$ cmake ..
$ make
$ sudo make install
# Build the OpenNI2 driver
$ cmake .. -DBUILD_OPENNI2_DRIVER=ON
$ make
# Copy libfreenect to the OpenNI2 driver directory
$ Repository=../../Bin/Arm-Release/OpenNI2/Drivers
$ cp -L lib/OpenNI2-FreenectDriver/libFreenectDriver* ${Repository}
# You may have to set permissions to be able to use the Sensor
$ sudo usermod -a -G video ubuntu
# Switch back to OpenNI2 directory here
# Now copy the libraries and include files to /usr
$ sudo cp -r Include /usr/include/openni2
$ sudo cp -r Bin/Arm-Release/OpenNI2 /usr/lib/
$ sudo cp Bin/Arm-Release/libOpenNI2.* /usr/lib/
# Create a package config file
# this will enable ubuntu to find the location of the drivers, libraries and include files.
$ sudo gedit /usr/lib/pkgconfig/libopenni2.pcand fill it with this:
prefix=/usr
exec_prefix=${prefix}
libdir=${exec_prefix}/lib
includedir=${prefix}/include/openni2Name: OpenNI2
Description: A general purpose driver for all OpenNI cameras.
Version: 2.2.0.0
Cflags: -I${includedir}
Libs: -L${libdir} -lOpenNI2 -L${libdir}/OpenNI2/Drivers -lDummyDevice -lOniFile -lPS1080.so# To make sure it is correctly found, run
$ pkg-config --modversion libopenni2# which should give the same version as defined in the file above (2.2.0.0)
# Linux has used the udev system to handle devices such as USB connected peripherals.
$ cd Packaging/Linux
$ sudo cp primesense-usb.rules /etc/udev/rules.d/557-primesense-usb.rules
At this point, you should be good to go. Switch over to the Bin/Arm-Release directory and run NiViewer to make sure it works. I’ll point out that you’ll need to plug in the sensor and have it running before NiViewer will work. It might be embarrassing to try to figure out why it’s not working when it’s not plugged in. It’s even more embarrassing if you happen to be on camera when you’re trying to demo. Don’t ask me how I know that. Also, please see the note below.
Note on using ASUS XTION PRO
If you have an ASUS XTION Pro, Walter Lucetti has step by step instructions for installation of OpenNI2 here:
Note on USB Autosuspend
There is an issue with the default setting for USB ports being turned off when not in use. The Jetson does this to save energy. If this happens when the Structure is connected, the Jetson will not recognize the Structure. A solution is to execute commands automatically on every boot-up that alter this behavior. You can put your commands into the “/etc/rc.local” boot up script. For example, run this on your board:
$ sudo gedit /etc/rc.local
Then add this near the bottom of the file but before the “exit” line:
# Disable USB auto-suspend, since it disconnects some devices such as webcams on Jetson TK1.
echo -1 > /sys/module/usbcore/parameters/autosuspend
Save the file.
You then reboot the machine so that the changes take effect.
$ sudo reboot
NOTE: If you would like to do this more selectively, you can enter this command on the command line:
sudo bash -c ‘echo -1 > /sys/module/usbcore/parameters/autosuspend’
which will cause auto suspend to be turned off until the next boot cycle.
Note that you can also modify the kernel firmware. That’s beyond the scope of this article, but you can read more about it here: Installing Grinch LinuxForTegra (L4T) on NVIDIA Jetson TK1
55 Responses
Hi, I’m following your guide to install OpenNI2 on my Jetson TK1 to be used with the Asus XTion PRo Live on my robot.
I have a doubt: do I need libfreenect driver? I thoughtit was needed only for Microsoft Kinect… doesn’t it?
Anyway, good job… this is a really clear guide!!!
Hi Myzhar,
The libfreenect driver is a device driver for PrimeSense based devices. The Kinect, the Asus XTion Pro Live, and the Occipital Structure Sensor are all based on the PrimeSense chipsets. It is my understanding that you can use libfreenect as the driver, and must configure it as an OpenNI2 driver as noted above. Of course you could always try it without it, and see what happens.
When they first started, libfreenect and OpenNI were two “competing” projects and were not at all compatible. Over the last couple of years, people started integrating OpenNI with libfreenect as the device driver. It is unclear to me what the minimal configuration is. I believe that ROS also has some integration with depth cameras too, including the ASUS.
Your robot looks like it’s coming along, it was great to see the videos from Makers Faire!
This article is superb! I was trying to install OPENNI one my TK1 as well as on my pcduino for days without luck! something used to fail. But then your article came along and installation was a breeze. Can’t thank you enough. I got my Xtion working on tk1 now (atleast the NiViewer is showing depth). Now I’ll get to building a depth sensing robot based on the tk1.
I’ll also try to use your instructions on the pcduino and see if it works.
Thanks again
You’re welcome and good luck! Hope you can share details about your Robot when you get it working.
Thank you so much for doing this! One additional dependency I found that I was missing was libudev which I got with apt-get install libudev-dev.
Hi edu,
Thank you for reading the article!
The dependency is apparently from a change made in the source since this article was published. I have added it to the above code snippet. Thanks for pointing this out!
Hi, I having trouble installing openni2 on my jetson.
When I run the command
cp -L lib/OpenNI2-FreenectDriver/libFreenectDriver* ${Repository}
I get this error
cp: target ‘”../../Bin/Arm-Release/OpenNI2/Drivers”’ is not a directory
Any help would be great.
Hi Kalvik,
You’ll need to check if the directory is actually there. This step is at here in the video: https://youtu.be/Bn9WqbYtNBw?t=6m4s
Of course the full path will be different on your machine. You should be in the ‘OpenNI2/libfreenect2/build’ directory. The full repository path looks to be ‘OpenNI2/Bin/Arm-Release/OpenNI2/Drivers’
You should check to make sure that it there.
I am trying to make this work for libfreenect2. What would be my command for it?
cp -L lib/OpenNI2-FreenectDriver/libFreenectDriver* ${Repository}
I went the the
lib/OpenNI2-FreenectDriver/
and the folder contains 3 files with the name libFreenectDriver with different extensions.
So it is the right directory right?
But I still get the error
Hmm, you’re trying to copy the binary files into the directory ‘OpenNI2/Bin/Arm-Release/OpenNI2/Drivers’
Your error message says that the directory does not exist. Is that the case?
And I went to the full repository path and the file exists and has 4 files with .so extensions
I went to the ‘OpenNI2/Bin/Arm-Release/OpenNI2/Drivers’ directory and it exists but I keep getting the error. So I went ahead and copies the files from the ‘lib/OpenNI2-FreenectDriver/’ file to ‘OpenNI2/Bin/Arm-Release/OpenNI2/Drivers’ and then when I run
$ pkg-config –modversion libopenni2
from OpenNI2 directory the terminal dosent show me anything.
Hi Kalvik,
It looks like there’s some formatting issues with the code. Sorry, horrible teething pains, but thank you for catching it.
Try:
Repository=../../Bin/Arm-Release/OpenNI2/Drivers
$ cp -L lib/OpenNI2-FreenectDriver/libFreenectDriver* ${Repository}
and:
$ pkg-config –-modversion libopenni2
There were no quotes in the Repository line. There are two hyphens instead of one in the pkg-config command.
Reading DIFFs gives me a headache, not to mention I still might do it wrong.
Any way you can take:
$ gedit ThirdParty/PSCommon/BuildSystem/CommonCppMakefile
Here’s the diff:
— OpenNI2-2.2.0.30/ThirdParty/PSCommon/BuildSystem/CommonCppMakefile.old 2014-03-28 19:09:11.572263107 -0700
+++ OpenNI2-2.2.0.30/ThirdParty/PSCommon/BuildSystem/CommonCppMakefile 2014-03-28 19:09:55.600261937 -0700
@@ -95,6 +95,9 @@
OUTPUT_NAME = $(EXE_NAME)
# We want the executables to look for the .so’s locally first:
LDFLAGS += -Wl,-rpath ./
+ ifneq (“$(OSTYPE)”,”Darwin”)
+ LDFLAGS += -lpthread
+ endif
OUTPUT_COMMAND = $(CXX) -o $(OUTPUT_FILE) $(OBJ_FILES) $(LDFLAGS)
endif
ifneq “$(SLIB_NAME)” “”
And just post what needs to be there ?
Thanks,
JT
Hi jtilghman,
Unfortunately I’m working on several projects right now with tight deadlines and won’t have time to do it. Sorry.
Hi Kangalow. I would like to know what changes need to be made for OpenNI2 to work with the Jetson TX1.
Hi Jesse,
I believe that it should work on the TX1 as written. However, I have not tried it.
I tried what you told me and the result I get is that the host platform cannot be determined. I also tried changing the cortex number with no success. Please help me with this issue.
Sorry, I don’t have one available to work with.
I’m also trying to compile. As far as I know, Tx1 use cortex-a57, so at least you need to change that line.
I am also sitting with a TX1 here and have the same problem. When I am going to save the Makefile i get this;
(gedit:10873): Gtk-CRITICAL **: _gtk_widget_captured_event: assertion ‘WIDGET_REALIZED_FOR_EVENT (widget, event)’ failed
(gedit:10873): Gtk-CRITICAL **: _gtk_widget_captured_event: assertion ‘WIDGET_REALIZED_FOR_EVENT (widget, event)’ failed
ubuntu@tegra-ubuntu:~/Downloads/OpenNI2$ gedit Makefile
ubuntu@tegra-ubuntu:~/Downloads/OpenNI2$ make
ThirdParty/PSCommon/BuildSystem/CommonDefs.mak:22: *** Can’t determine host platform. Stop.
Same problem here. Did you ever end up solving it?
Hi, Jesse!
Could you run that on Jetson TX1?
I have this problem: ThirdParty/PSCommon/BuildSystem/CommonDefs.mak:22: *** Can’t determine host platform. Stop.
Can someone help me with that?
Hi ,I have the same problem. any clue?
thanks
Same problem here. Did you ever end up solving it?
Quick question. Is OpenNI2 the only platform that can run the Structure sensor with the Jetson TX1?
I believe that it just uses libfreenect
You may have more success asking questions about the Structure sensor in their forums, I have limited experience with the device.
hello,when I run this command that”cp -L lib/OpenNI2-FreenectDriver/libFreenectDriver* ${Repository}”, my full command is “haha@ubuntu:~/libfreenect/build$ cp -L lib/OpenNI2-FreenectDriver/libFreenectDriver* ${Repository}” Unfortunatelly,the error says that “cp: target ‘”../../Bin/Arm-Release/OpenNI2/Drivers”’ is not a directory” i don’t know if my path is wrong.BTW,i have read the question like mine,but I still can’t handle this trouble,thx
Hi
When I tried to run NiViewer, I got the follwoing error. Do you know how to fix them?
$ sudo ./NiViewer
OpenNI2-FreenectDriver: Using libfreenect v0.6.0
OpenNI2-FreenectDriver: Found device freenect://0
Found sibling device [same parent]
Failed to set the LED of K4W or 1473 device: -4
OpenNI2-FreenectDriver: Opening device freenect://0
[Stream 70] Negotiated packet size 1920
Failed to submit isochronous transfer 0: -1
Failed to submit isochronous transfer 1: -1
Failed to submit isochronous transfer 2: -1
Failed to submit isochronous transfer 3: -1
Failed to submit isochronous transfer 4: -1
Failed to submit isochronous transfer 5: -1
Failed to submit isochronous transfer 6: -1
Failed to submit isochronous transfer 7: -1
Failed to submit isochronous transfer 8: -1
Failed to submit isochronous transfer 9: -1
Failed to submit isochronous transfer 10: -1
Failed to submit isochronous transfer 11: -1
Failed to submit isochronous transfer 12: -1
Failed to submit isochronous transfer 13: -1
Failed to submit isochronous transfer 14: -1
Failed to submit isochronous transfer 15: -1
write_register: 0x0105 <= 0x00
write_register: 0x0006 <= 0x00
write_register: 0x0012 <= 0x03
write_register: 0x0013 <= 0x01
write_register: 0x0014 <= 0x1e
write_register: 0x0006 <= 0x02
write_register: 0x0017 <= 0x00
[Stream 80] Negotiated packet size 1920
Failed to submit isochronous transfer 0: -1
Failed to submit isochronous transfer 1: -1
Failed to submit isochronous transfer 2: -1
Failed to submit isochronous transfer 3: -1
Failed to submit isochronous transfer 4: -1
Failed to submit isochronous transfer 5: -1
Failed to submit isochronous transfer 6: -1
Failed to submit isochronous transfer 7: -1
Failed to submit isochronous transfer 8: -1
Failed to submit isochronous transfer 9: -1
Failed to submit isochronous transfer 10: -1
Failed to submit isochronous transfer 11: -1
Failed to submit isochronous transfer 12: -1
Failed to submit isochronous transfer 13: -1
Failed to submit isochronous transfer 14: -1
Failed to submit isochronous transfer 15: -1
write_register: 0x000c <= 0x00
write_register: 0x000d <= 0x01
write_register: 0x000e <= 0x1e
write_register: 0x0005 <= 0x01
write_register: 0x0047 <= 0x00
ubuntu@tegra-ubuntu:/home/xiaodong/OpenNI/OpenNI2/Bin/Arm-Release$ sudo ./NiViewer
OpenNI2-FreenectDriver: Using libfreenect v0.6.0
OpenNI2-FreenectDriver: Found device freenect://0
Found sibling device [same parent]
Failed to set the LED of K4W or 1473 device: -4
OpenNI2-FreenectDriver: Opening device freenect://0
[Stream 70] Negotiated packet size 1920
Failed to submit isochronous transfer 0: -1
Failed to submit isochronous transfer 1: -1
Failed to submit isochronous transfer 2: -1
Failed to submit isochronous transfer 3: -1
Failed to submit isochronous transfer 4: -1
Failed to submit isochronous transfer 5: -1
Failed to submit isochronous transfer 6: -1
Failed to submit isochronous transfer 7: -1
Failed to submit isochronous transfer 8: -1
Failed to submit isochronous transfer 9: -1
Failed to submit isochronous transfer 10: -1
Failed to submit isochronous transfer 11: -1
Failed to submit isochronous transfer 12: -1
Failed to submit isochronous transfer 13: -1
Failed to submit isochronous transfer 14: -1
Failed to submit isochronous transfer 15: -1
write_register: 0x0105 <= 0x00
write_register: 0x0006 <= 0x00
write_register: 0x0012 <= 0x03
write_register: 0x0013 <= 0x01
write_register: 0x0014 <= 0x1e
write_register: 0x0006 <= 0x02
write_register: 0x0017 <= 0x00
[Stream 80] Negotiated packet size 1920
Failed to submit isochronous transfer 0: -1
Failed to submit isochronous transfer 1: -1
Failed to submit isochronous transfer 2: -1
Failed to submit isochronous transfer 3: -1
Failed to submit isochronous transfer 4: -1
Failed to submit isochronous transfer 5: -1
Failed to submit isochronous transfer 6: -1
Failed to submit isochronous transfer 7: -1
Failed to submit isochronous transfer 8: -1
Failed to submit isochronous transfer 9: -1
Failed to submit isochronous transfer 10: -1
Failed to submit isochronous transfer 11: -1
Failed to submit isochronous transfer 12: -1
Failed to submit isochronous transfer 13: -1
Failed to submit isochronous transfer 14: -1
Failed to submit isochronous transfer 15: -1
write_register: 0x000c <= 0x00
write_register: 0x000d <= 0x01
write_register: 0x000e <= 0x1e
write_register: 0x0005 <= 0x01
write_register: 0x0047 <= 0x00
What machine and OS version are you trying to make this work on?
Jetson TK1, Ubuntu 14.04.
Kinect 1473.
I followed your instruction. With one additional step
1. I upgraded the libusb as following
$ apt-add-repository ppa:floe/libusb && apt-get update && apt-get install libusb-1.0-0-dev
https://github.com/OpenKinect/libfreenect2/issues/390
so I moved libusb-1.0,17 –> libusb-1.0.20
Thanks!
Thanks!
Thanks!
I also flushed jetpack 3.1 before I started to build openNI
This what I see from lsusb
Bus 003 Device 040: ID 045e:02bf Microsoft Corp.
Bus 003 Device 039: ID 045e:02be Microsoft Corp.
Bus 003 Device 038: ID 045e:02c2 Microsoft Corp.
The article here is about installing OpenNI2 on a Occipital Structure Sensor. I do not know what needs to be changed to have it work for a Kinect. You can try the instructions here for a similar hardware solution:
http://myzharbot.robot-home.it/blog/software/configuration-nvidia-jetson-tk1/asus-xtion-pro-live-openni2-compilation-install-instructions/
How about this link?
https://jetsonhacks.com/2014/07/14/jetson-tk1-microsoft-kinect-using-openkinect/
It looks like Occipital has a download page for OpenNI, that includes precomipiled binaries for 2.2.0.33 on ARM (https://structure.io/openni ). Am I better off building from source on Jetson, or just downloading / installing the prebuilt packages?
I do not know. I have not tried the precompiled binaries.
Tried following the above instructions, but when it comes time to run “Make”, I get the following error. Any ideas?
ThirdParty/PSCommon/BuildSystem/CommonDefs.mak:22: *** Can’t determine host platform. Stop.
Well, I updated CommonDefs.mak to be hardcoded to the “arm” platform, now I’m getting these errors:
===
g++: error: unrecognized command line option ‘-mfpu=neon-vfpv4’
g++: error: unrecognized command line option ‘-mfloat-abi=hard’
I guess it doesn’t help that I’m trying to follow these instructions on a TX2. But it’s the closest to an actual working installation guide out there…
Eureka! Turns out, some kind sould has sorted it all out for TX1 (Which is close enough to TX2, I guess). Instead of cloning out of the canonical OpenNI2 repo, just use this instead: https://github.com/mikeh9/OpenNI2-TX1 . Then you can install the ros wrapper from https://github.com/fangchangma/openni2_ros_wrapper , and AWAY WE GO!
Hi,Could you install OpenNI on TX2?I got the same issue as you.
Hi Kangalow. I would like to know what changes need to be made for OpenNI2 to work with the Xtion on Jetson TX2.
I do not know. What did you try?
Hi Kanglow,
I am installing libfreenect2 and OPENNI2 to make Kinect V2 work. I had issues while executing the below statement.
cp -L lib/OpenNI2-FreenectDriver/libFreenectDriver* ${Repository}
the location of the OpenNI2 libraries on my local is as follows:
/usr/lib/OpenNI2/Drivers
There are four files uner it
and my repository location is
../../Bin/x64-Release/OpenNI2/Drivers
I executed the following command:
cp -L lib/OpenNI2/Drivers* ${Repository}
Now it throws me the following error
cp: cannot stat ‘lib/OpenNI2/Drivers*’: No such file or directory
Can you please let me know what mistake i am doing here? Thank you
I don’t understand your question. Are you on a Jetson TK1?
Are you in the /usr directory when you try the copy?
Hello Kanglow,
I am using the instructions you posted above to make Kinect2 working on my machine. I have installed OPENNI2 and when i am trying to install libfrenect2 (the latest one, not the old one – libfreenect), i am stuck at one the steps.
I am at the following location
root@shankar:~/OpenNI2/libfreenect2/build$
I am executing the following command at the above location
cp -L lib/OpenNI2/Drivers* ${Repository}
and i am getting error stated above
What kind of machine is “your machine”?
Its alienware desktop
The instructions given are for a ARM-64 embedded development system, not a x86 desktop. Please follow the instructions for your type of machine.