A Video-based End-to-end Pipeline for Non-nutritive Sucking Action Recognition and Segmentation in Young Infants
We present an end-to-end computer vision pipeline to detect non-nutritive sucking (NNS) -- an infant sucking pattern with no nutrition delivered -- as a potential biomarker for developmental delays, using off-the-shelf baby monitor video footage. One barrier to clinical (or algorithmic) assessment o...
Gespeichert in:
Hauptverfasser: | , , , , , , , , , , , , |
---|---|
Format: | Artikel |
Sprache: | eng |
Schlagworte: | |
Online-Zugang: | Volltext bestellen |
Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
Zusammenfassung: | We present an end-to-end computer vision pipeline to detect non-nutritive
sucking (NNS) -- an infant sucking pattern with no nutrition delivered -- as a
potential biomarker for developmental delays, using off-the-shelf baby monitor
video footage. One barrier to clinical (or algorithmic) assessment of NNS stems
from its sparsity, requiring experts to wade through hours of footage to find
minutes of relevant activity. Our NNS activity segmentation algorithm solves
this problem by identifying periods of NNS with high certainty -- up to 94.0\%
average precision and 84.9\% average recall across 30 heterogeneous 60 s clips,
drawn from our manually annotated NNS clinical in-crib dataset of 183 hours of
overnight baby monitor footage from 19 infants. Our method is based on an
underlying NNS action recognition algorithm, which uses spatiotemporal deep
learning networks and infant-specific pose estimation, achieving 94.9\%
accuracy in binary classification of 960 2.5 s balanced NNS vs. non-NNS clips.
Tested on our second, independent, and public NNS in-the-wild dataset, NNS
recognition classification reaches 92.3\% accuracy, and NNS segmentation
achieves 90.8\% precision and 84.2\% recall. |
---|---|
DOI: | 10.48550/arxiv.2303.16867 |