# Data collection

> Collect raw sensor data to create a dataset

The next step is to collect data to form a dataset. This entails:

1. Using a board with an accelerometer running Infuse-IoT firmware.
2. Build and Flash the tag with a suitable accelerometer firmware.
3. During an experiment, log accelerometer values to the data logger or the GATT peripheral
4. and utilise Python tools to annotate the raw data.
5. Use Python tools to download the data logger data, or save the live GATT peripheral data
6. Preprocess data (calculate the magnitudes, etc)
7. Sample windows of data based on events for a balance dataset
8. Save the dataset

<steps level="2">

## Boards and devices

When selecting a board to use it must:

- Have an accelerometer (or other sensor to provide suitable data) and
- Have a way to log the sensor data either via

  - The [data logger](https://docs.dev.infuse-iot.com/latest/embedded/app_features/data_logger.html) subsystem, or
  <warning>
  
  Using the data logger is preferred, however ensure the capacity is suitable for the sensor and experiment. For example 100Hz accelerometer data occupies at least 72kB per hour. Ensure a suitable size is available to prevent existing data being overwritten.
  
  </warning>
  - A reliable [ePacket interface](https://docs.dev.infuse-iot.com/latest/embedded/app_features/epacket.html#interfaces)
  like Serial or Bluetooth to stream the data live.
  <warning>
  
  Although a Serial link is more reliable, a cable may change the movement profile.
  Utilising the bluetooth is the most convenient method to stream the data wirelessly.
  
  </warning>

Suitable boards include the Thingy:52/53/91 and the nRF54L15 Tag.

<tabs sync="tagBoard">
<tabs-item label="Default">

For this demo, the [nRF54L15 Tag](https://www.nordicsemi.com/Products/Development-hardware/nRF54L15-Tag)
will be used as the sensor board.<br />

The Zephyr board target is `nrf54l15tag/nrf54l15/cpuapp`.

Alternately, you can use a suitable Custom Tag Board instead.

</tabs-item>

<tabs-item label="Custom Tag Board">

Enter the board name/target of the custom board to update commands below

<config-input default="nrf54l15tag/nrf54l15/cpuapp" label="Tag target name" name="tagTarget">



</config-input>
</tabs-item>
</tabs>

---

A second board will be required to be used as a gateway (unless using `infuse native_bt`).
This just requires a board with Bluetooth (e.g. nRF52840DK, nRF54L15DK, etc.)

<tabs sync="gatewayBoard">
<tabs-item label="Default">

For this demo, the [nRF54L15DK](https://www.nordicsemi.com/Products/Development-hardware/nRF54L15-DK)
will be used as the gateway board. In Zephyr this target is referred to as `nrf54l15dk/nrf54l15/cpuapp`.

</tabs-item>

<tabs-item label="Custom Gateway Board">

If using a custom board for the gateway device, enter the board name to update commands below

<config-input default="nrf54l15dk/nrf54l15/cpuapp" label="Gatway target name" name="gatewayTarget">



</config-input>
</tabs-item>
</tabs>

<tip>

When using the nRF54L15 Tag as the sensor board and nRF54L15DK as the gateway,
to avoid draining the non-rechargeable CR2032 battery,
the nRF54l15 Tag can be powered by the DK while connected via the JTAG programming port.
This can be done by connecting a suitable power pin (Port P2's `VDD:IO`) to the `VDD SWD0`.

<warning>

Do not use the DK to supply power to the nRF54L15 tag while the 3V CR2032 Coin Cell battery is connected

</warning>

This avoid draining the battery while programming, testing or downloading logger data over Bluetooth.

</tip>

## Hardware Preparation

Ensure both devices have been provisioned by following the [provisioning guide](/guides/embedded-iot/how-tos/provision-a-tag).
This step can be skipped if both devices are already provisioned.

<tip>

If using the nF54L15DK, ensure the nRF54L15 Tag is not plugged in while trying to provision or program the nF54L15DK.

</tip>

## Build and Flash the Gateway

With your gateway device connected, follow the steps in [Flashing a Gateway](/guides/embedded-iot/how-tos/flash-a-gateway)
to program your gateway if it is not already running Gateway firmware.

## Build and Flash the tag

Then build the `task_runner: imu` application sample for the tag:

<tabs sync="tagBoard">
<tabs-item label="Default">

```bash
west build -b nrf54l15tag/nrf54l15/cpuapp infuse-sdk/samples/task_runner/imu
```

</tabs-item>

<tabs-item label="Custom Tag">
<dynamic-block language="bash" template="west build -b {tagTarget} infuse-sdk/samples/task_runner/imu
">



</dynamic-block>
</tabs-item>
</tabs>

<tip>

To program the nRF54L15 Tag, keep the gateway connected and insert the nRF54L15 Tag into the "Debug Out" port.

</tip>

And flash the tag with the following command.

<tabs sync="tagBoard">
<tabs-item label="Default">

```bash
west flash -d build/nrf54l15tag/nrf54l15/cpuapp/imu
```

</tabs-item>

<tabs-item label="Custom Tag">
<dynamic-block language="bash" template="west flash -d build/{tagTarget}/imu
">



</dynamic-block>
</tabs-item>
</tabs>

## Prepare your Gateway

Follow the [guide to connect to your gateway](/guides/embedded-iot/how-tos/connect-gateway) using the Python Tools.

<tip>

The nRF54L15DK will appear as a "Serial (UART/USB)" device.

</tip>

<tip>

Alternately, connect directly to the tag if using `infuse native_bt` See
[Native Bluetooth](/docs/infuse-iot-sdk/python-tools/gateways#native-bluetooth)
for more details.

</tip>

## Prepare for an experiment

Before we can run the experiment there's a few steps to take to get ready beforehand.

<steps level="3">

### About your Tag

To capture raw data and send annotate data to the tag we will need to be able to connect and run commands on the tag.
To do this we will need to know the tag's Infuse ID. Follow [these instructions](/guides/embedded-iot/how-tos/tag-infuse-id)
to get the tag's Infuse ID.

You can also enter it below to substitute it into subsequent commands.

<config-input default="0x123456789ABCDEF" label="Tag's Infuse ID (hex)" name="infuseId" type="text">



</config-input>

### Data Collection

While running an experiment we need to save the raw sensor data during the experiment.
This leaves us with two main options:

- Storing raw sensor data locally on the tag's local flash (preferably SD card or external flash).
- Streaming the raw sensor data live over Bluetooth (or serial).

Streaming the data may be more simple, but can be susceptible to data loss in the case of signal interference or range constraints.
Conversely, storing the data on the tag is more reliable, however getting the data off the tag requires a bit more effort.

For this demonstration, we will use the streaming approach, especially since the nRF54L15 Tag does not
have external flash loaded by default, limiting how much accelerometer data can be recorded.

We can also try this out ahead of the experiment:

<tabs sync="dataLogging">
<tabs-item label="Streaming data Live" icon="i-lucide-bluetooth">

With your gateway connected and running (ensuring the Python Gateway script is also running)

Choose a suitable location to save the logs:

<config-input default="~/csv_data" label="Folder path to save to" name="savePath" type="text">



</config-input>

And run the following command (in another terminal) to connect to the Tag,
start streaming accelerometer data and saving it to
your machine:

<dynamic-block language="bash" template="infuse bt_log --id {infuseId} --csv {savePath} --data
">



</dynamic-block>

Once connected, there should be TDF messages appearing in the terminal, something similar to:

```text
[11:44:12.950] Connected to 123456789ABCDEF (DATA)
  Opening new ~/csv_data/123456789ABCDEF_ACC_4G.csv
  1581889511.712 TDF: ACC_4G[10]
  Opening new ~/csv_data/123456789ABCDEF_BATTERY_STATE.csv
  1581889512.725 TDF: BATTERY_STATE
  1581889511.895 TDF: ACC_4G[18]
  1581889512.232 TDF: ACC_4G[32]
  1581889512.837 TDF: ACC_4G[2]
  1581889512.857 TDF: ACC_4G[29]
  1581889513.395 TDF: ACC_4G[23]
```

This will create a new `.csv` file in <dynamic-snippet template="{savePath}">



</dynamic-snippet>

 for each type of TDF.
This includes:

- Accelerometer Data (<dynamic-snippet template="{infuseId}_ACC_XG.csv">



</dynamic-snippet>

, where X is 2, 4, 8, or 16)
- Annotations (<dynamic-snippet template="{infuseId}_ANNOTATION.csv">



</dynamic-snippet>

)
- Battery Voltages (<dynamic-snippet template="{infuseId}_BATTERY_STATE.csv">



</dynamic-snippet>

)
and more.
<tip>

This tool will continue to log new data until you close it with <kbd value="meta">



</kbd>

 <kbd value="c">



</kbd>

.

</tip>
</tabs-item>

<tabs-item label="Data Logger" icon="i-lucide-hard-drive">

Coming Soon

</tabs-item>
</tabs>

### Annotation Labels

While recording the raw data, we will want to send annotations to indicate that this raw data is useful and what was occuring.
This will become the labels in the labelled dataset.

This can be done via the `annotation` RPC (Command),
which logs a timestamped `annotation` TDF (Sensor data).

To simplify the process the `infuse annotate_events` tool can be used. It simplifies:

- Handling timing between the two systems when creating annotations to minimise delay.
- Maintaining a constant connection (to reduce down time connecting to the device).
- Pre-configurable or manually typed labels.

#### Time Configuration

Because many Infuse-IoT embedded devices are not networked, they do not always have a method to obtain the current
time and even if they do, it will drift during operation.
In this demo to keep things simple we will use the default behaviour which will prompt users to set the device time
before an experiment.
This is recommended since it improves the tool's reliability.

<tip>

Depending on the circumstances of the experiment, the tool can be configured to forcible set, or skip
setting the device time.

<warning>

If the tag's time is not set altogether, it can cause the tool to hang.<br />


Only use `--skip-time` if you know what you're doing.

</warning>
</tip>

#### Loggers

When we send the annotation we need that to be saved somewhere ideally alongside our data.
This could either be on the device (on either the onboard or removeable storage) or sent to a different device (over Bluetooth, UDP or Serial).

For the nRF54L15 Tag, we will be streaming the raw data over Bluetooth, so the easiest way to log this data will be over the Bluetooth interface by providing the `--gatt` argument.
However, if utilising the on device storage, `--onboard` or `--external` arguments can be used to store data to the corresponding onboard or removeable flash loggers.

For the nRF52L15 Tag, streaming the data will require annotations to be sent over the Bluetooth interface using `--gatt`.

#### Labels

Before we can send annotations to the device, we need a set of labels we want to classify.
This can be whatever we want such as the behaviours we're aiming to classify such as:

- Swipe right
- Swipe left
- Double shake
- Double tap
- Rotation clockwise
- Rotation counter-clockwise
- Idle
- Unknown

However, the Nordic Edge-AI Lab requires datasets to be formatted in a particular way.
Rather than having a window of time series data labelled as a specific event, Nordic Edge AI lab's datasets require:

- A label for each time series sample (not window).
- Option to provide an additional metadata field for the recording window.

This provides some interesting optimisations such as applying sliding windows during the recording window, which
can help the training process to accomodate with the fact that windows of data at runtime are random (not perfectly aligned to the start and end of an event).

As a result marking the start and end of a series of gestures greately simplifies the process.
Therefore each of the labels above will have a start and a stop label.

Running the tool will allow the user to type each label in, however this is slow, especially if it's the same labels each time.
Instead, we can supply a `.json` file containing the labels we want to use and supply this file instead.

<code-collapse>
<file-code file="2.guides/embedded-iot/embedded-algorithms/labels.txt" filename="labels.json">



</file-code>
</code-collapse>

Create the `labels.json` label file with the same contents.

<warning>

These sorts of labels only work when you can start and stop specific behaviours on demand.
<br />


Simlarly, some behaviours might be more specific and require more fine tuned labelling.
Usually, a label may be required on a per-event basis, making the tool much more valuable under
those circumstances.

</warning>

### Test Run

Ensuring your gateway and gateway scripts are still running, start the `infuse bt_log` tool again
(if not currently running).

Now use the following command to try to send some annotations to the device:

<tip>

Place your `labels.json` in the present working directory, or update the path in the command.

</tip>

<tabs sync="dataLogging">
<tabs-item label="Streaming data Live" icon="i-lucide-bluetooth">
<dynamic-block language="bash" template="infuse annotate_events --id {infuseId} --preset-labels labels.json --gatt
">



</dynamic-block>
</tabs-item>

<tabs-item label="Data Logger" icon="i-lucide-hard-drive">
<tabs sync="logger">
<tabs-item label="Onboard Flash">
<dynamic-block language="bash" template="infuse annotate_events --id {infuseId} --preset-labels labels.json --onboard
">



</dynamic-block>
</tabs-item>

<tabs-item label="Removable Flash">
<dynamic-block language="bash" template="infuse annotate_events --id {infuseId} --preset-labels labels.json --external
">



</dynamic-block>
</tabs-item>
</tabs>
</tabs-item>
</tabs>

Once ready, select the annotation you want to send using the arrow keys and hit <kbd value="enter">



</kbd>


to send it.

An `ANNOTATION` TDF should appear in the `infuse bt_log` console and the corresponding `.csv`.

<tip>

Select the `Exit` option or enter <kbd value="meta">



</kbd>

 <kbd value="c">



</kbd>

 to close the tool.

</tip>

You're now all set to run an experiment.

<tip>

It may be useful to delete the existing `.csv` files before starting an actual experiment.
Re-running the logging script will not overwrite existing data, instead appending to it.

</tip>
</steps>

## Running an Experiment

It's now time to run the experiment.

<steps level="3">

### Start Logging

Repeat the process before starting to log data with:

<tabs sync="dataLogging">
<tabs-item label="Streaming data Live" icon="i-lucide-bluetooth">
<dynamic-block language="bash" template="infuse bt_log --id {infuseId} --data --csv {savePath}
">



</dynamic-block>
</tabs-item>

<tabs-item label="Data Logger" icon="i-lucide-hard-drive">

Coming Soon

</tabs-item>
</tabs>

### Start the annotation Tool

And then running the annotation tool to get ready:

<tabs sync="dataLogging">
<tabs-item label="Streaming data Live" icon="i-lucide-bluetooth">
<dynamic-block language="bash" template="infuse annotate_events --id {infuseId} --preset-labels labels.json --gatt
">



</dynamic-block>
</tabs-item>

<tabs-item label="Data Logger" icon="i-lucide-hard-drive">
<tabs sync="logger">
<tabs-item label="Onboard Flash">
<dynamic-block language="bash" template="infuse annotate_events --id {infuseId} --preset-labels labels.json --onboard
">



</dynamic-block>
</tabs-item>

<tabs-item label="Removable Flash">
<dynamic-block language="bash" template="infuse annotate_events --id {infuseId} --preset-labels labels.json --external
">



</dynamic-block>
</tabs-item>
</tabs>
</tabs-item>
</tabs>

### Start labelling

Select a behaviour to perform e.g. "Double Tap" and send the start annotation
Perform the behaviour repeatedly for a given duration

### Stop labelling

When finished, send the corresponding end annotation.

### Repeat

Repeat the process for the remaining labels. For the "Unknown" class, perform random or other motions
unlike the other gestures (like dropping, rolling, shaking, etc).

<tip>

The process can be repeated for the same label to collect additional data.<br />


Especially under different conditions such as holding the tag with a different orientation
or other people performing the actions.

</tip>
</steps>

With this the data collection process is complete. This process can be repeated to increase and diversify the
raw data we use to create the training dataset.

The next step is to take the raw data collected and turn it into a training dataset.

</steps>