This study presents a multimodal system designed to support autism therapy by integrating a humanoid robot with sensorised smart objects and computer vision techniques. The system enables the digitalisation of standard therapeutic activities while providing quantitative measures of children's behaviour during interaction tasks. Specifically, motor interaction features (e.g., latency, acceleration, and time to peak) are extracted from smart objects, as well as gaze-related metrics (e.g., stationery and transition entropy) are obtained from RGB-D camera data. An exploratory study involving children with Autism Spectrum Condition (ASC) was conducted across multiple therapy sessions. Participants were grouped according to their Childhood Autism Rating Scale (CARS) scores, and a Principal Component Analysis (PCA) was applied to investigate differences in interaction patterns over time. Results suggest that the proposed system captures meaningful behavioural dimensions by combining motor- and visual- derived features. The PCA representation reveals distinguishable patterns in the distribution of participants according to ASC severity. These findings indicate that the extracted metrics enable a quantitative comparison of behavioural performance and interaction profiles between ASC severity. Overall, the proposed approach highlights the potential of integrating robotics and sensing technologies to provide objective support for the assessment and monitoring of autism therapy. Future work will focus on larger cohorts to further validate these preliminary findings.

Pugi, L., Fiorini, L., Borghese, N., Ciardo, F., Ricciardelli, P., Actis-Grosso, R., et al. (2026). Beyond Observation: A Data-Driven Approach to Autism Therapy Measuring the Interaction with Social Robot and Smart Objects. In 2026 IEEE 35th International Conference on Robot and Human Interactive Communication (RO-MAN) (pp.1579-1584) [10.1109/RO-MAN72286.2026.11715007].

Beyond Observation: A Data-Driven Approach to Autism Therapy Measuring the Interaction with Social Robot and Smart Objects

Ciardo F.;Ricciardelli P.;Actis-Grosso R.;
2026

Abstract

This study presents a multimodal system designed to support autism therapy by integrating a humanoid robot with sensorised smart objects and computer vision techniques. The system enables the digitalisation of standard therapeutic activities while providing quantitative measures of children's behaviour during interaction tasks. Specifically, motor interaction features (e.g., latency, acceleration, and time to peak) are extracted from smart objects, as well as gaze-related metrics (e.g., stationery and transition entropy) are obtained from RGB-D camera data. An exploratory study involving children with Autism Spectrum Condition (ASC) was conducted across multiple therapy sessions. Participants were grouped according to their Childhood Autism Rating Scale (CARS) scores, and a Principal Component Analysis (PCA) was applied to investigate differences in interaction patterns over time. Results suggest that the proposed system captures meaningful behavioural dimensions by combining motor- and visual- derived features. The PCA representation reveals distinguishable patterns in the distribution of participants according to ASC severity. These findings indicate that the extracted metrics enable a quantitative comparison of behavioural performance and interaction profiles between ASC severity. Overall, the proposed approach highlights the potential of integrating robotics and sensing technologies to provide objective support for the assessment and monitoring of autism therapy. Future work will focus on larger cohorts to further validate these preliminary findings.
paper
Autism Spectrum Condition; Human Robot Interaction; smart objects; Social robotics;
English
35th International Conference on Robot and Human Interactive Communication, RO-MAN 2026 - 24 August 2026 - 28 August 2026
2026
2026 IEEE 35th International Conference on Robot and Human Interactive Communication (RO-MAN)
9798319510556
2026
1579
1584
none
Pugi, L., Fiorini, L., Borghese, N., Ciardo, F., Ricciardelli, P., Actis-Grosso, R., et al. (2026). Beyond Observation: A Data-Driven Approach to Autism Therapy Measuring the Interaction with Social Robot and Smart Objects. In 2026 IEEE 35th International Conference on Robot and Human Interactive Communication (RO-MAN) (pp.1579-1584) [10.1109/RO-MAN72286.2026.11715007].
File in questo prodotto:
Non ci sono file associati a questo prodotto.

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/10281/630121
Citazioni
  • Scopus 0
  • ???jsp.display-item.citation.isi??? ND
Social impact