Data collection

About your Tag

To capture raw data and send annotate data to the tag we will need to be able to connect and run commands on the tag. To do this we will need to know the tag's Infuse ID. Follow these instructions to get the tag's Infuse ID.

You can also enter it below to substitute it into subsequent commands.

Data Collection

While running an experiment we need to save the raw sensor data during the experiment. This leaves us with two main options:

• Storing raw sensor data locally on the tag's local flash (preferably SD card or external flash).
• Streaming the raw sensor data live over Bluetooth (or serial).

Streaming the data may be more simple, but can be susceptible to data loss in the case of signal interference or range constraints. Conversely, storing the data on the tag is more reliable, however getting the data off the tag requires a bit more effort.

For this demonstration, we will use the streaming approach, especially since the nRF54L15 Tag does not have external flash loaded by default, limiting how much accelerometer data can be recorded.

We can also try this out ahead of the experiment:

infuse bt_log --id 0x123456789ABCDEF --csv ~/csv_data --data

  [11:44:12.950] Connected to 123456789ABCDEF (DATA)
  Opening new ~/csv_data/123456789ABCDEF_ACC_4G.csv
  1581889511.712 TDF: ACC_4G[10]
  Opening new ~/csv_data/123456789ABCDEF_BATTERY_STATE.csv
  1581889512.725 TDF: BATTERY_STATE
  1581889511.895 TDF: ACC_4G[18]
  1581889512.232 TDF: ACC_4G[32]
  1581889512.837 TDF: ACC_4G[2]
  1581889512.857 TDF: ACC_4G[29]
  1581889513.395 TDF: ACC_4G[23]

Annotation Labels

While recording the raw data, we will want to send annotations to indicate that this raw data is useful and what was occuring. This will become the labels in the labelled dataset.

This can be done via the annotation RPC (Command), which logs a timestamped annotation TDF (Sensor data).

To simplify the process the infuse annotate_events tool can be used. It simplifies:

• Handling timing between the two systems when creating annotations to minimise delay.
• Maintaining a constant connection (to reduce down time connecting to the device).
• Pre-configurable or manually typed labels.

Time Configuration

Because many Infuse-IoT embedded devices are not networked, they do not always have a method to obtain the current time and even if they do, it will drift during operation. In this demo to keep things simple we will use the default behaviour which will prompt users to set the device time before an experiment. This is recommended since it improves the tool's reliability.

Loggers

When we send the annotation we need that to be saved somewhere ideally alongside our data. This could either be on the device (on either the onboard or removeable storage) or sent to a different device (over Bluetooth, UDP or Serial).

For the nRF54L15 Tag, we will be streaming the raw data over Bluetooth, so the easiest way to log this data will be over the Bluetooth interface by providing the --gatt argument. However, if utilising the on device storage, --onboard or --external arguments can be used to store data to the corresponding onboard or removeable flash loggers.

For the nRF52L15 Tag, streaming the data will require annotations to be sent over the Bluetooth interface using --gatt.

Labels

Before we can send annotations to the device, we need a set of labels we want to classify. This can be whatever we want such as the behaviours we're aiming to classify such as:

• Swipe right
• Swipe left
• Double shake
• Double tap
• Rotation clockwise
• Rotation counter-clockwise
• Idle
• Unknown

However, the Nordic Edge-AI Lab requires datasets to be formatted in a particular way. Rather than having a window of time series data labelled as a specific event, Nordic Edge AI lab's datasets require:

• A label for each time series sample (not window).
• Option to provide an additional metadata field for the recording window.

This provides some interesting optimisations such as applying sliding windows during the recording window, which can help the training process to accomodate with the fact that windows of data at runtime are random (not perfectly aligned to the start and end of an event).

As a result marking the start and end of a series of gestures greately simplifies the process. Therefore each of the labels above will have a start and a stop label.

Running the tool will allow the user to type each label in, however this is slow, especially if it's the same labels each time. Instead, we can supply a .json file containing the labels we want to use and supply this file instead.

[
  "swipe_right start",
  "swipe_left start",
  "double_shake start",
  "double_tap start",
  "triple_tap start",
  "rotation_cw start",
  "rotation_ccw start",
  "idle start",
  "unknown start",
  "swipe_right end",
  "swipe_left end",
  "double_shake end",
  "double_tap end",
  "triple_tap end",
  "rotation_cw end",
  "rotation_ccw end",
  "idle end",
  "unknown end"
]

Create the labels.json label file with the same contents.

Test Run

Ensuring your gateway and gateway scripts are still running, start the infuse bt_log tool again (if not currently running).

Now use the following command to try to send some annotations to the device:

infuse annotate_events --id 0x123456789ABCDEF --preset-labels labels.json --gatt

infuse annotate_events --id 0x123456789ABCDEF --preset-labels labels.json --onboard

infuse annotate_events --id 0x123456789ABCDEF --preset-labels labels.json --external

Once ready, select the annotation you want to send using the arrow keys and hit ↵ to send it.

An ANNOTATION TDF should appear in the infuse bt_log console and the corresponding .csv.

You're now all set to run an experiment.

Start Logging

Repeat the process before starting to log data with:

infuse bt_log --id 0x123456789ABCDEF --data --csv ~/csv_data

Start the annotation Tool

And then running the annotation tool to get ready:

infuse annotate_events --id 0x123456789ABCDEF --preset-labels labels.json --gatt

infuse annotate_events --id 0x123456789ABCDEF --preset-labels labels.json --onboard

infuse annotate_events --id 0x123456789ABCDEF --preset-labels labels.json --external

Start labelling

Select a behaviour to perform e.g. "Double Tap" and send the start annotation Perform the behaviour repeatedly for a given duration

Stop labelling

When finished, send the corresponding end annotation.

Repeat

Repeat the process for the remaining labels. For the "Unknown" class, perform random or other motions unlike the other gestures (like dropping, rolling, shaking, etc).