This demo project is the integration of Infineon's Machine learning: Imagimob MTBML deployment and Avnet's IoTConnect ModusToolbox™ Basic Sample. It demonstrates the ModusToolbox™ Machine Learning (MTBML) flow algorithm with "baby cry" model detection by using data from the pulse-density modulation to pulse-code modulation (PDM/PCM) which is sent to the model to detect whether a baby is crying or not. The AI model's recognized class is then reported to IoTConnect.
This project currently supports the CY8CKIT-062S2-AI The project supports only Eclipse with GCC_ARM compiler.
To quickly evaluate this project without compiling code follow the step-by-step instructions in the QuickStart Guide.
- GNU Arm® Embedded Compiler (GCC_ARM) - Default value of TOOLCHAIN
The code has been developed and tested with MTB 3.2, with Eclipse, and the boards below:
- PSoC™ 6 CY8CKIT-062S2-AI(
CY8CKIT-062S2-AI)
Watch an overview video of creating a new project with IoTConnect in ModusToolbox™ then follow the steps below.
To build the project, please refer to the IoTConnect ModusToolbox™ Basic Sample Developer Guide and note the following:
- Once ModusToolbox has been installed, the ModusToolbox™ for Machine Learning software should be installed as well.
- If using the model generator, you will need to install QEMU and set up the relevant environment variables per the Machine Learning User Guide
- Over-the-air updates are not currently supported.
- Use the psoc6aibm-device-template.json Device Template instead of the Basic Sample's template.
Once the board connects to IoTConnect, it will start processing microphone input and attempt to detect a baby crying sound. This can be tested by placing the board next to the PC speakers and playing an audio file like this one.
The application sends data once every second by default (see commands below), but the AI model recognizes sounds several times per second.
If a baby cry s recognized, the baby_cry_detected attribute will be reported as true for at least 3 seconds after the actual detection.
The actual confidence percentage detected at the time of sending the telemetry data will be reported as confidence_baby_cry.
Note that due to the fact that confidence_baby_cry "lingers" as true for up to 3 seconds so we do not miss a report, confidence value reported below the threshold may be reported along with confidence_baby_cry being true. This behavior is intended.
One can also supply a smaller number than 1000 (milliseconds) as a parameter to the set-reporting-interval command at runtime to increase the reporting interval. In that case the confidence value will be more in sync.
The following commands can be sent to the device using the IoTConnect Web UI:
| Command | Argument Type | Description |
|---|---|---|
board-user-led |
String (on/off) | Turn the board LED on or off |
set-detection-threshold |
Number (eg. 50) | Set baby cry detection threshold (default 85) as percentage. |
set-reporting-interval |
Number (eg. 4000) | Set telemetry reporting interval in milliseconds. By default, the application will report every 1000ms. |
demo-mode |
String (on/off) | Enable demo mode. In this mode the application will send telemetry to IoTConnect for a longer period |
To evaluate other Infineon/Imagimob models with IoTConnect, use the links below:
The project can be modified to support the CY8CKIT-028-SENSE shield and similar boards. If introducing support for other boards, please make note that .cyignore currently ignores shield files and that CY_IGNORE logic in the makefile has no effect.
See the list here