JetsonHacks

Jetson Xavier NX – Run from SSD

It is a simple task to a sea of gigabyte goodness to the NVIDIA Jetson Xavier NX by adding a NVMe SSD. This is the best performance upgrade you can make for your Jetson. Looky here:

Background

Even though they share the same form factor, the Jetson Xavier NX Developer Kit has a nice addition in comparison to the Jetson Nano. A M.2 Key M slot. With the M.2 Key M slot, we can easily add a Solid State Drive (SSD). The M.2 Key M slot uses the Non-Volatile Memory Express (NVMe) protocol that runs over PCIe. 

There are two types of SSDs that use Key M. When you install a SSD card in the slot, the card needs to be the NVMe PCIe type.

The SSD is typically 5-20x faster than the flash memory in a SD card, so anytime you do disk access things speed up considerably.

This process works on the Jetson AGX Xavier too. See Notes below.

Note: Before upgrading using Over The Air (OTA) updates, read the notes below in Upgrading.

Materials and Tools

In the video, we install a Western Digital 500GB NVMe SSD. There are several different sizes and brands of these types of devices, we also get good results with the Samsung variety. We also use our trusty iFixit Pro Tech Toolkit which contains a variety of useful tools for just this purpose.

Hardware Installation

The M.2 Key M connector is on the underside of the carrier board. A retaining screw is at the end of the M.2 slot. Remove the retaining screw. From the factory, the screw may have some blue thread locker on it, which may require a little elbow grease to start the removal process.

Remove Retaining Screw

Next install the the SSD card into the Key M connector, and then use the same retaining screw to hold it in place.

Install SSD and secure with retaining screw

Disk Configuration

After the hardware install, it is time to configure the disk under Ubuntu. Hook the Xavier back up to a keyboard, mouse and monitor. In the video above, we use mostly GUI tools, please refer to the video for a walk through.

The basic steps are to format the disk, and then create partition(s). In the video, we just allocate most of the disk space to one big ol’ partition. However you may want to be a little fancier. Being Linux, there are tools for doing this through the command line, as all the pros will tell you.

At this point, you can now begin using the SSD. It will show up on your desktop as an unmounted drive.

Running from SSD

We go a little further here. After the Jetson boots the image from the SD card, we switch the rootfs to point to the SSD. In effect, the system will now run from the SSD, the SD card is only there to boot the system.

You should do this process directly after creating a new SD card.

On the JetsonHacks account on Github, there is a repository rootOnNVMe. Clone the repository:

$ git clone https://github.com/jetsonhacks/rootOnNVMe

and switch over to that repository’s directory:

$ cd rootOnNVMe

Next, copy the rootfs of the eMMC/SD card to the SSD

$ ./copy-rootfs-ssd.sh

Finally, we will add a service which will run a script when the system starts up. The script will “pivot the root” to the SSD so that the system will run from the SSD.

$ ./setup-service.sh

There’s a chapter in the video ‘Tech Talk Time’ that walks through how the script works. After setting up the service, reboot for the changes to take effect.

The performance of your system will feel much snappier after this simple modification.

Boot Notes

These script changes the rootfs to the SSD after the kernel image is loaded from the eMMC/SD card. For the Xavier NX, you will still need to have the SD card installed for booting. As of this writing, the default configuration of the Jetson NX does not allow direct booting from the NVMe.

Upgrading

Note: (7/9/2020) There are currently issues with the Over The Air (OTA) updates with the JetPack 4.4 release. Please see the article: JetPack 4.4 Over The Air (OTA) Update issues. NVIDIA is working on the issues, but currently we do not recommend upgrading to JetPack 4.4 production release via the OTA update.

Once this service is installed, the rootfs will be on the SSD. If you upgrade to a newer version of L4T using OTA updates (using the NVIDIA .deb repository), you will need to also apply those changes to the SD card that you are booting from.

Typically this involves copying the /boot* directory and /lib/modules/<kernel name>/ from the SSD to the SD card. If they are different, then modules load will be ‘tainted’, that is, the modules version will not match the kernel version.

Special Shout Out!

This is taken from the NVIDIA Jetson AGX Xavier forum https://forums.developer.nvidia.com/t/how-to-boot-from-nvme-ssd/65147/22, written by user crazy_yorik (https://forums.developer.nvidia.com/u/crazy_yorick). Thank you crazy_yorik!

Notes

Installation of the SSD is different on the AGX Xavier. See: Install NVMe SSD on NVIDIA Jetson Developer Kit. After formatting the drive, software installation is the same as shown in the article.