Running OpenCV on Jetson Dev Kits has more nuance than many people expect. Looky here:
OpenCV is the most popular computer vision library in the world. There are many gems of algorithms and downright computer vision goodness in there.
At the same time, OpenCV is what we call in computer lingo a “rich” environment. That usually means that it is configurable in many different and sometimes baffling ways depending on your perspective.
Any particular build of OpenCV has many options. Obviously which processor the library is built for (such as ARM 64 for the Jetsons), but also which media libraries, video handling libraries, 3rd party libraries it incorporates and so on.
The default version of OpenCV that NVIDIA supplies with the Jetsons (L4T 32.2.1/L4T 4.2.2) is OpenCV 3.3.1.
Point 1 – What Version of OpenCV is available
The version of OpenCV is important for a couple of reasons. First the version tells you which features are available. Recently OpenCV 4.X has come on the scene, version OpenCV 3.X is the mature, stable branch.
The second reason the version is important is that on the Jetson, the default OpenCV version on the Jetson is different than the one in the Ubuntu repositories. Many applications will load the older version of OpenCV when they install via apt-get. This may overwrite the default version, or cause confusion during application loading.
One of the main differences between the version in the Ubuntu repository and the Jetson default version is GStreamer support. The Jetson default has it, the other doesn’t. That makes it more difficult to interface with the Jetson CSI camera.
You can find the OpenCV version with dpkg
$ dpkg -l | grep libopencv
Note that there may be multiple installations of OpenCV on the Jetson, so you may have to examine library paths, and search through /usr/lib and /usr/local/lib to find out what is really going on. Some people are really sneaky and place it other places too.
Point 2 – Python 3 and OpenCV
Does Python 3 work with the default OpenCV? The short answer is yes. However you will need to install numpy first.
$ sudo apt-get install python3-numpy
Of course, if you want to use pip3 instead to install numpy, that’s a fine route to take.
Point 3 – Does OpenCV has feature XYZ
You have OpenCV on your machine, but you don’t know what features it supports.
OpenCV itself has a function, getBuildInformation. This is easy to access from Python 2.7 (this is in the default Jetson Image). From a Terminal:
>>> import cv2
>>> print cv2.getBuildInformation()
This will list to the console a large amount of information, which tells how OpenCV is built, what modules are available, media and video support and so on.
Looking at the build information, we see that the default version of OpenCV on the Jetsons have GStreamer support, but no CUDA support.
Point 4 – Python and CUDA and OpenCV
From Point 3, you can find out if the version of the OpenCV library you are using has Python support. That’s straightforward. We can also tell if CUDA support is built it. That’s straightforward.
What is not straightforward is if there is Python support for CUDA in the OpenCV library. Again, there’s a short answer. If you build OpenCV with CUDA support, version 4.X can build Python wrappers for GPU functions. Version 3.X does not.
There may be ways around this, but it’s not simple.
Point 5 – Other Packages May Not Play Nice
As we discussed in Point 1, applications may load different versions of OpenCV, depending on how they were built. Typically they load a version from the Ubuntu repository when you install them from the Ubuntu repository.
If this happens, then you usually have a different feature set than the default one from the Jetson. This is probably the major point of confusion people report as they do not know that the default OpenCV has been overwritten.
Just because you have OpenCV CUDA enabled, this does not automatically mean that you get better performance. There are many factors, here’s a few.
First, not all OpenCV functions have CUDA code available. Second, some algorithms do not benefit from parallel execution. Third, the processing pipeline may not be able to take advantage of parallel execution.
Also, the amount of time it takes to setup for CUDA execution (memory transfers and other setup), may take longer than the execution speed that you gain from the CUDA code.
One more thing, think of this as double bonus, this is from James Bowley:
The benefit you will get from moving your processing to the GPU with OpenCV will depend on the function you call and configuration that you use, in addition to your processing pipeline. That said from, what I have observed, on average the CUDA functions are much much quicker than their CPU counterparts.
I hope you are able to get something out of this little discussion. We get many questions about this topic, I think having this discussion available here will make life simpler.