This study presents a multimodal system designed to support autism therapy by integrating a humanoid robot with sensorised smart objects and computer vision techniques. The system enables the digitalisation of standard therapeutic activities while providing quantitative measures of children's behaviour during interaction tasks. Specifically, motor interaction features (e.g., latency, acceleration, and time to peak) are extracted from smart objects, as well as gaze-related metrics (e.g., stationery and transition entropy) are obtained from RGB-D camera data. An exploratory study involving children with Autism Spectrum Condition (ASC) was conducted across multiple therapy sessions. Participants were grouped according to their Childhood Autism Rating Scale (CARS) scores, and a Principal Component Analysis (PCA) was applied to investigate differences in interaction patterns over time. Results suggest that the proposed system captures meaningful behavioural dimensions by combining motor- and visual- derived features. The PCA representation reveals distinguishable patterns in the distribution of participants according to ASC severity. These findings indicate that the extracted metrics enable a quantitative comparison of behavioural performance and interaction profiles between ASC severity. Overall, the proposed approach highlights the potential of integrating robotics and sensing technologies to provide objective support for the assessment and monitoring of autism therapy. Future work will focus on larger cohorts to further validate these preliminary findings.
Pugi, L., Fiorini, L., Borghese, N., Ciardo, F., Ricciardelli, P., Actis-Grosso, R., et al. (2026). Beyond Observation: A Data-Driven Approach to Autism Therapy Measuring the Interaction with Social Robot and Smart Objects. In 2026 IEEE 35th International Conference on Robot and Human Interactive Communication (RO-MAN) (pp.1579-1584) [10.1109/RO-MAN72286.2026.11715007].
Beyond Observation: A Data-Driven Approach to Autism Therapy Measuring the Interaction with Social Robot and Smart Objects
Ciardo F.;Ricciardelli P.;Actis-Grosso R.;
2026
Abstract
This study presents a multimodal system designed to support autism therapy by integrating a humanoid robot with sensorised smart objects and computer vision techniques. The system enables the digitalisation of standard therapeutic activities while providing quantitative measures of children's behaviour during interaction tasks. Specifically, motor interaction features (e.g., latency, acceleration, and time to peak) are extracted from smart objects, as well as gaze-related metrics (e.g., stationery and transition entropy) are obtained from RGB-D camera data. An exploratory study involving children with Autism Spectrum Condition (ASC) was conducted across multiple therapy sessions. Participants were grouped according to their Childhood Autism Rating Scale (CARS) scores, and a Principal Component Analysis (PCA) was applied to investigate differences in interaction patterns over time. Results suggest that the proposed system captures meaningful behavioural dimensions by combining motor- and visual- derived features. The PCA representation reveals distinguishable patterns in the distribution of participants according to ASC severity. These findings indicate that the extracted metrics enable a quantitative comparison of behavioural performance and interaction profiles between ASC severity. Overall, the proposed approach highlights the potential of integrating robotics and sensing technologies to provide objective support for the assessment and monitoring of autism therapy. Future work will focus on larger cohorts to further validate these preliminary findings.I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.


