poc:2025:ondevice-sleepstaging [DokuWiki]

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
====== On-Device Prediction of Sleep Stages ======


===== Quick Summary =====


  * **Status**:  ❌ Abandoned  
  * **Duration**: August 19 2025
  * **Type**: Hardware/Possible Product  
  * **Repository**: N/A
  * **Team**: Fabricio

**What we're exploring**: 
The feasibility of on-device predictions of sleep stages. Important aspects are a.) realtime processing b.) feasibility on an ESP32 c.) expected performance regressions.
This is relevant due to a.) limitations on the iPhone for compute b.) decisions on how to process sleep stages etc. on our devices, e.g. what hardware?, rather in the cloud?


===== Key Findings =====

==== ESP Model: ESP32-S3 ====
The [[ https://www.espressif.com/en/products/socs/esp32-s3 | ESP32 S3 ]] is optimized for neural networks.
Documentation can be found [[ https://www.espressif.com/sites/default/files/documentation/esp32-s3_datasheet_en.pdf | here ]].

==== Hardware Limitations ====
The main limitation is presumably the memory.
   * The ESP32 has a total of 512KB of SRAM. This has to be shared with other processes.
   * While there is an option to upgrade with external RAM up to an additional 4MB (i.e. 4MB accessed at once), this does not seem to be recommended due to a hard decrease in performance.
   * The main issue with Convolutional-Architectures would be the explosion in intermediate activations if many filters are used.
   * This is a bit difficult to manually tune and explore.
   * int8 is apparently the preferred format.. Not sure if convolutions work that well on that quantization
   * Unclear how the DSP interacts with the NN..
   * Dynamic USleep would be probably out of scope

==== Possible Solution Approaches ====
   * One would probably need a MLOps pipeline like [[ https://www.edgeimpulse.com/blog/ | https://www.edgeimpulse.com/blog/ ]] for getting solutions in any reasonable amount of time.
   * TensorFlow Lite has suport for Microcontrollers: [[ https://github.com/tensorflow/tflite-micro |TFLite Micro]].
   * [[ https://github.com/espressif/esp-nn | Espressive-NN ]] provides optimized NN Kernels for their ESP32 devices.

==== Surprises & Insights ====
  * It's barely deep learning anymore. Much more focus on DSP.
  * I have absolutely no idea what happens with the performance on such small models. One would need a full development cycle for a new sleep staging model, completely decoupled from recent advances in sleep staging.

===== Recommendations =====
**Should we proceed?**: No, currently out of scope.

===== External Resources =====
  * [[https://www.espressif.com/sites/default/files/documentation/esp32-s3_datasheet_en.pdf|ESP32 S3 Datasheet]]
  * [[https://arxiv.org/abs/2212.03332|Edge Impulse MLOps Arxiv Article]]