Validating Personas presents a significant challenge in current scientific literature, relying on costly external validation methods involving users, experts or simulations. This study aims to address this challenge by introducing a novel method to perform personas validation based on data already available during their development. The proposed method starts from previously developed Personas, and trains a clustering model on the entire dataset. Subsequently, a training-test split is repeated twenty times, with each iteration involving the training of a clustering model on the training set. Labels for the test set are then predicted by both models, and consensus is assessed through clustering consensus metrics such as Adjusted Rand Index, Adjusted Mutual Information, homogeneity, completeness and V- measure averaged across the twenty iterations. The proposed method is evaluated using a dataset comprising 1070 subjects regarding their willingness to vaccinate against COVID-19. Three unbalanced Personas are derived from previous studies, and are used as input to this novel method. Clustering consensus metrics reveal a fair agreement between clustering results, with average ARI of 0.3391, average AMI of 0.3435, average homogeneity of 0.3531, average completeness of 0.3473 and average V-measure of 0.3499. In conclusion, the proposed method exhibits robust capabilities in generalizing beyond the training data by accurately classifying new samples in their respective Personas, suggesting the possibility of reducing costs when compared to existing methods in current literature.
Tauro, E., Gorini, A., Bilo, G., Caiani, E. (2024). A Data-Driven Method to Perform Personas Validation Using Clustering Consensus Metrics. In Advances in Digital Health and Medical Bioengineering Proceedings of the 11th International Conference on E-Health and Bioengineering, EHB-2023, November 9–10, 2023, Bucharest, Romania – Volume 1: Medical Devices, Measurements, and Artificial Intelligence Applications (pp.494-501). Springer Science and Business Media Deutschland GmbH [10.1007/978-3-031-62502-2_57].
A Data-Driven Method to Perform Personas Validation Using Clustering Consensus Metrics
Bilo G.;
2024
Abstract
Validating Personas presents a significant challenge in current scientific literature, relying on costly external validation methods involving users, experts or simulations. This study aims to address this challenge by introducing a novel method to perform personas validation based on data already available during their development. The proposed method starts from previously developed Personas, and trains a clustering model on the entire dataset. Subsequently, a training-test split is repeated twenty times, with each iteration involving the training of a clustering model on the training set. Labels for the test set are then predicted by both models, and consensus is assessed through clustering consensus metrics such as Adjusted Rand Index, Adjusted Mutual Information, homogeneity, completeness and V- measure averaged across the twenty iterations. The proposed method is evaluated using a dataset comprising 1070 subjects regarding their willingness to vaccinate against COVID-19. Three unbalanced Personas are derived from previous studies, and are used as input to this novel method. Clustering consensus metrics reveal a fair agreement between clustering results, with average ARI of 0.3391, average AMI of 0.3435, average homogeneity of 0.3531, average completeness of 0.3473 and average V-measure of 0.3499. In conclusion, the proposed method exhibits robust capabilities in generalizing beyond the training data by accurately classifying new samples in their respective Personas, suggesting the possibility of reducing costs when compared to existing methods in current literature.
I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.
