Jetson Nano + Raspberry Pi Camera

The NVIDIA Jetson Nano Developer Kit plugs and plays with the Raspberry Pi V2 Camera! Looky here:

Background

Since the introduction of the first Jetson in 2014, one of the most requested features has been Raspberry Pi camera support. The Jetson Nano has built in support, no finagling required.

The Jetson family has always supported MIPI-CSI cameras. MIPI stands for Mobile Industry Processor Interface, the CSI stands for Camera Serial Interface. This protocol is for high speed transmission between cameras and host devices. Basically it’s a hose straight to the processor, there isn’t a lot of overhead like there is with something like say a USB stack.

However, for those folks who are not professional hardware/software developers, getting access to inexpensive imaging devices through that interface has been, let’s say, challenging.

This is for a couple of reasons. First, the camera connection and wiring is through a connector to which most hobbyists don’t have good access. In addition, there’s a lot of jiggering with the drivers for the camera in the Linux kernel along with manipulation of the device tree that needs to happen before imaging magic occurs. Like I said, pro stuff. Most people take the path of least resistance, and simply use a USB camera.

Raspberry Pi Camera Module V2

At the same time, one of the most popular CSI-2 cameras is the Raspberry Pi Camera Module V2. The camera has a ribbon connector which connects to the board using a simple connector. At the core, the RPi camera consists of a Sony IMX-219 imager, and is available in different versions, with and without an infrared filter. Leaving out the infrared filter in the Pi NoIR camera (NoIR= No Infrared) allows people to build ‘night vision’ cameras when paired with infrared lighting. And they cost ~ $25, lots of bang for the buck!

Are they the end all of end all cameras? Nope, but you can get in the game for not a whole lot of cash.

Note: The V1 Raspberry Pi Camera Module is not compatible with the default Jetson Nano Install. The driver for the imaging element is not included in the base kernel modules.

Jetson Nano

Here’s the thing. The Jetson Nano Developer Kit has a RPi camera compatible connector! Device drivers for the IMX 219 are already installed, the camera is configured. Just plug it in, and you’re good to go.

Installation

Installation is simple. On the Jetson Nano J13 Camera Connector, lift up the piece of plastic which will hold the ribbon cable in place. Be gentle, you should be able to pry it up with a finger/fingernail. Once loose, you insert the camera ribbon cable, with the contacts on the cable facing inwards towards the Nano module. Then press down on the plastic tab to capture the ribbon cable. Some pics (natch):

Make sure that the camera cable is held firmly in place after closing the tab. Here’s a pro tip: Remove the protective plastic film which covers the camera lens on a new camera before use. You’ll get better images (don’t ask me how I know).

Testing and some Codez

The CSI-Camera repository on Github contains some sample code to interface with the camera. Once installed, the camera should show up on /dev/video0. On the Jetson Nano, GStreamer is used to interface with cameras. Here is a command line to test the camera:

$ gst-launch-1.0 nvarguscamerasrc ! 'video/x-raw(memory:NVMM),width=3820, height=2464, framerate=21/1, format=NV12' ! nvvidconv flip-method=0 ! 'video/x-raw,width=960, height=616' ! nvvidconv ! nvegltransform ! nveglglessink -e

This requests GStreamer to open a camera stream 3820 pixels wide by 2464 high @ 21 frames per second and display it in a window that is 960 pixels wide by 616 pixels high. The ‘flip-method’ is useful when you need to change the orientation of the camera because if flips the picture around for you. You can get some more tips in the README.md file in the repository.

There are also a couple of simple ‘read the camera and show it in a window’ code samples, one written in Python, the other C++.

The third demo is more interesting. The ‘face_detect.py’ script runs a Haar Cascade classifier to detect faces on the camera stream. You can read the article Face Detection using Haar Cascades to learn the nitty gritty. The example in the CSI-Camera repository is a straightforward implementation from that article. This is one of the earlier examples of mainstream machine learning.

Notes

A Logitech C920 webcam is used in the video through the Cheese application.

Demonstration environment:

  • Jetson Nano Developer Kit
  • L4T 32.1.0
  • Raspberry Pi Camera Module V2

31 Comments

  1. Wondering if the OV5647 cams (like Raspberry Pi Camera v1) will also be compatible with the #JetsonNano Dev Kit? Would you happen to know?
    Thanks in advance

        • Did you managed it to work, the night vision one ??
          I have tried the “original” raspberry v2.1 cam, this one is working.
          But a second one called “Raspberry Pi Full HD Kamera Madul” with night vision from electreeks.de just want work at all.
          Cheers!

          • For the default Jetson Nano Image, you must use an IMX219 based sensor, such as the Raspberry Pi 1.2 NoIR camera. The camera you mention is a OV5647 sensor. Thanks for reading!

            • Thank you clearing this out kanglow!
              I have the Raspberry Pi Noir Kamera-Modul V2 connected and working, unfortunately the pictures are blue in the middle and red outside.
              maybe it is just broken…
              funny thing is, nvidia AI says “jelly fish” all the time
              haha..

  2. Awesome article! Thanks. Would you happen to know the gstreamer command line to operate a second CSI camera on the nano?

  3. Hi there!

    I tried to do it like you! And thanks for your time and work to make this report. But I got some Issues and dont know how to solve it cause I am fairly new to Linux.
    I think thats what causes my trouble:

    (Argus) Error FileOperationFailed: Connecting to nvargus-daemon failed: Die Struktur muss bereinigt werden (in src/rpc/socket/client/SocketClientDispatch.cpp, function openSocketConnection(), line 201)
    (Argus) Error FileOperationFailed: Cannot create camera provider (in src/rpc/socket/client/SocketClientDispatch.cpp, function createCameraProvider(), line 102)
    Error generated. /dvs/git/dirty/git-master_linux/multimedia/nvgstreamer/gst-nvarguscamera/gstnvarguscamerasrc.cpp, execute:515 Failed to create CameraProvider
    FEHLER: Von Element /GstPipeline:pipeline0/GstEglGlesSink:eglglessink0: Output window was closed

    It would be great if someone can tell me what I have to take care with ..

    Greetings Nico

  4. Thanks a lot i could fix it by taking another sd-card something went wrong with it… now everything works like described…

      • Thanks for great work!
        I would like to connect my custom camera to jetson nano. My question is for connecting a different camera to nano. Can you tell me which kernel file be edited for a different camera. I have all the data for my customly made camera. I think it is not impossible to do that.

  5. You have great tutorials, I really appreciate them!

    When I run the gst-launch command in the terminal it works fine, no problem. But when I tried your python example (simple_camera.py) it could not open the camera. I found out that the pre-installed opencv-3.2.0 was built without gstreamer support:

    Python 3.6.7 (default, Oct 22 2018, 11:32:17)
    [GCC 8.2.0] on linux
    Type “help”, “copyright”, “credits” or “license” for more information.
    >>> import cv2
    >>> print(cv2.getBuildInformation())

    GStreamer: NO

    I used a newly flashed SD card with Jetpack. Do you have any idea on how to proceed?

    • Thank you for the kind words. Did you try with Python 2.7? For Python 3.6, there are a lot of steps needed to install for OpenCV, it is possible that the original OpenCV was overwritten.

  6. In this article you use the camera utility gst-launch-1.0. In several of the Nvidia demos they use another utility nvgstcapture-1.0.

    Can you explain the differences? I am very new to Linux so this may be obvious to more experienced users

  7. hello,
    thank you for all your tutorials,they really help me a lot!
    I’m a TX2 user and wonder if PRI v2 cam can work on my TX2 with the latest Jetpack 4.2 version?
    I have a Auvidea J20 board ,a 6-csi-cam-add-on module.When I plug everything on my TX2 dev-kit and make some “sudo”-command(5 i2cset command) as Auvidea reference manual said.The GPIO light switch on but still no “/dev/video”.
    I found someone having the same problem with J20 and PRIV2cam,and some questioned on nvidia Forums(mainly on 2017 and 2018).Still I don’t get much useful help.
    I know that Ridgerun have done these things and nicely write a wiki on how to make it work.But it costs 2,499$ to buy their drivers.That’s far from i can afford.I don’t know what should I do now.
    I’m appreciated for all the advise or guide.
    John

  8. In case this helps anyone trying to stream over X11 to a remote desktop:
    ssh -X $USER_NAME@$NANO_IP -C “gst-launch-1.0 nvarguscamerasrc ! ‘video/x-raw(memory:NVMM),width=3820, height=2464, framerate=21/1, format=NV12’ ! nvvidconv flip-method=2 ! ‘video/x-raw,width=960, height=616’ ! nvvidconv ! ximagesink”

  9. Any thoughts on why a Raspberry Pi cam 2.1 is very red and grainy? simple_camera and face-detect work.

    Regards, Geordy

Leave a Reply

Your email address will not be published.


*