Edge Impulse Firmware for Sony's Spresense

Edge Impulse enables developers to create the next generation of intelligent device solutions with embedded machine learning. This repository contains the Edge Impulse firmware for Sony's Spresense development board. This device supports all of Edge Impulse's device features, including ingestion, remote management and inferencing.

Note: Do you just want to use this development board to collect data with Edge Impulse? No need to build this firmware. View the getting started guide for a pre-built firmware image and flashing instructions. Or, you can use the data forwarder to capture data from any sensor.

Requirements

Hardware

• Sony's Spresense Main board.

Optional add-ons:

• Spresense Extension board - to connect external sensors.
• CXD5602PWBCAM1 camera add-on - if you want to add sight to your Spresense.
• CXD5602PWBCAM2W HDR camera add-on - if you want to add sight to your Spresense.
• Spresense Sensor EVK-70 add-on with an accelerometer - if you plan on building models that depend on motion.
• Spresense Wi-Fi / iS110B module - for wireless connectivity.

Software

• Edge Impulse CLI.
• GNU Make.
• GNU ARM Embedded Toolchain 9-2019-q4-major - make sure arm-none-eabi-gcc is in your PATH.
• Python 3.

Building and flashing the application

1. Run make from the root folder to build the project:
   • Standalone Spresense:

   $ make -j
   

   • Spresense + Commonsense:

   $ make commonsense -j
   

2. Connect the board to your computer using USB.
3. Flash the board:

   $ make flash
   

Build and flash with Docker

1. Build the Docker image:

   $ docker build -t spresense-build .
   

2. Build the application by running the container as follows:
   Windows

   $ docker run --rm -it -v "%cd%":/app spresense-build /bin/bash -c "make -j"
   

   Linux, macOS

   $ docker run --rm -it -v $PWD:/app:delegated spresense-build /bin/bash -c "make -j"
   

3. Connect the board to your computer using USB.
4. Flash the board:

   $ make flash
   

   Or if you don't have make installed:

   $ tools/flash_writer.py -s -d -b 115200 -n build/firmware.spk
   

Connecting to the board

Edge Impulse Studio

You can use the edge-impulse-daemon to connect with Edge Impulse studio. Documentation can be found here.

Running your impulse

The edge-impulse-run-impulse tool is used to start and run the impulse on your board. Documentation can be found here.

WARNING

The nuttx stdint.h defines int32 as unsigned long, whereas the stdlib.h that ships with ARM GCC defines int32 as unsigned int. These are the same size (https://developer.arm.com/documentation/dui0472/k/C-and-C---Implementation-Details/Basic-data-types-in-ARM-C-and-C--), so from a stack perspective, it doesn't matter, but a C++ linker will treat a different in int32 as a function overload (so you'll get a missing function error from the linker if you're not careful)

Notes

• The video driver doesn't like to change resolutions after being opened. This prevents lazy initialization and using the same handle for the entire life of program.
• Thus, we open and close on entering and exiting snapshot stream, and on each snapshot for ingestion or inference

Tips

Override the ARM GNU Toolchain

If you have multiple toolchains installed, you can override the compiler via:

$ CROSS_COMPILE=~/toolchains/gcc-arm-none-eabi-9-2019-q4-major/bin/arm-none-eabi- make -j

Troubleshoot

The latest Sony Spresense SDK requires to have the bootloader on device updated to the latest version. If you encounter following message during startup the bootloader is outdated:

cxd56_farapiinitialize: Mismatched version: loader(2015) != Self(20585)
cxd56_farapiiitialize: Please update loader and gnssfw firmwares!!

Uploading the Sony Spresense bootloader is done using the Arduino IDE. Follow the Spresense Arduino Library Getting Started Guide for instructions.