Deploying Models
With a completed model, it's time to test it out on real hardware.
- Select & Download the model
- Test it in simulation (Optional)
- Deploy it to the target hardware
Download the Model
Once training has successfully completed, there may be multiple models available. These will typically involve a tradeoff between accuracy vs flash footprint & processing time.
Once selected, download the model locally. It comes bundled as the model source files & weights and a standalone executable that can be used to test the model on your local machine.
Test the Model in Simulation (Optional)
Downloading the model contains two key components:
- The ML algorithms C code, to be deployed to the embedded device.
- An standalone executable (for each OS) that can be run on your local machine.
Using the standalone executable downloaded with the model, you can test the model in a simulated environment on your local machine. The model accepts a CSV input and requires similar configuration as the model training (like specifying the label and session columns when applicable).
When deploying the model to the target hardware, sometimes the model can be incorrectly input or configured. It's also useful to sanity check the model's behaviour in a controlled environment, allowing you to test standardised inputs and validate the outputs are identical in the both the simulation and the target hardware environments. These tests help identify any discrepancies in the model before deploying to firmware early on and isolating firmware issues.
| acc_x | acc_y | acc_z | acc_mag |
|---|
| acc_mag | acc_x | acc_y | acc_z |
|---|
Deploy the Model to the Target Hardware
Once you are satisfied with the model's performance in simulation, you can deploy it to the target hardware. Follow the hardware-specific instructions to transfer the model files and executable to the device.
Refer to the sample in: samples/algorithms/imu_algo.
Insert the algorithm into nrf_edgeai_generated folder into the imu_algo/gesture_algorithm subfolder.
The sample contains two main.c files:
- The first being the general application setting up hardware, sampling accelerometer,
- The second containing the gesture algorithm and used as a standalone extension added to the first dynamically.
However, there are a few points that need to be updated to accomodate the model:
Model input count
Check nrf_edgeai_user_model.c for the parameter USER_UNIQ_INPUTS_NUM.
This indicates the number of input values the model expects.
In imu_algo/gesture_algorithm/main.c, update the USER_UNIQ_INPUTS_NUM value to the corresponding value above.
Model Input type
Similarly, check the model configuration for the input type: INPUT_FEATURE_DATA_TYPE.
In the main.c, search for the flt_32 input_sample and update the
type depending on the INPUT_FEATURE_DATA_TYPE, using
flt32_t, int16_t or int8_t.
Input model order
Similarly, the model is fed by copying values to input_sample, before being provided to the model e.g.
input_sample[0] = mag(acc_sample->x, acc_sample->y, acc_sample->z);
input_sample[1] = (float)acc_sample->x;
input_sample[2] = (float)acc_sample->y;
input_sample[3] = (float)acc_sample->z;
nrf_edgeai_feed_inputs(user_model, input_sample, USER_UNIQ_INPUTS_NUM);
These will need to be updated depending on the input order of the model.
For accelerometer values, refer to the acc_sample->``x/y/z.
Here, it's simpler to calculate magnitudes manually.
Additionally, gyro data can be integrated easily using e.g.
/* Feed data into the model.*/
const struct imu_sample *acc_sample;
const struct imu_sample *gyro_sample;
...
imu_data = zbus_chan_const_msg(chan);
acc_start = imu_data->accelerometer.offset;
gyro_start = imu_data->gyroscope.offset;
inference_window_start = imu_data->accelerometer.timestamp_ticks;
for (int i = 0; i < imu_data->accelerometer.num; i++) {
/* This model expects Floating Point inputs, and a magnitude */
acc_sample = &imu_data->samples[acc_start + i];
gyro_sample = &gyro_data->samples[gyro_start + i];
input_sample[0] = sqrtf((float)acc_sample->x * (float)acc_sample->x +
(float)acc_sample->y * (float)acc_sample->y +
(float)acc_sample->z * (float)acc_sample->z);
input_sample[1] = (float)acc_sample->x;
input_sample[2] = (float)acc_sample->y;
input_sample[3] = (float)acc_sample->z;
input_sample[4] = sqrtf((float)gyro_sample->x * (float)gyro_sample->x +
(float)gyro_sample->y * (float)gyro_sample->y +
(float)gyro_sample->z * (float)gyro_sample->z);
input_sample[5] = (float)gyro_sample->x;
input_sample[6] = (float)gyro_sample->y;
input_sample[7] = (float)gyro_sample->z;
/* Feed this sample pair into the model's windowing buffer */
res = nrf_edgeai_feed_inputs(user_model, input_sample, USER_UNIQ_INPUTS_NUM);
if (res == NRF_EDGEAI_ERR_INPROGRESS) {
continue;
}
...
Output Class Names
Depending on which classes the model will output, update the class labels at the top of the file.
const char *class_strings[] = {
"Double Shake", "Double Tap", "Triple Tap", "CW rotate", "CCW rotate", "Idle", "Unknown",
};
Output class probabilities.
The sample model outputs the class probabilities as floating point values. However, depending on the model, it can output integer probabilities (uint8 or uint16).
If so update the probability extraction to accomodate this:
printk("class: %s (%d) %.3f\n", class_strings[predicted_class], predicted_class,
(double)user_model->decoded_output.classif.probabilities
.p_f32[predicted_class]);
Changing user_model->decoded_output.classif.probabilities.p_f32 to the appropriate type.
Test the model on the hardware to ensure it performs as expected.