In recent years, chatbots have gained widespread adoption thanks to their ability to assist users at any time and across diverse domains. However, the lack of large-scale curated datasets limits research on their quality and reliability. This paper presents TOFU-D, a snapshot of 1,788 Dialogflow chatbots from GitHub, and COD, a curated subset of TOFU-D including 185 validated chatbots. The two datasets capture a wide range of domains, languages, and implementation patterns, offering a sound basis for empirical studies on chatbot quality and security. A preliminary assessment using the Botium testing framework and the Bandit static analyzer revealed gaps in test coverage and frequent security vulnerabilities in several chatbots, highlighting the need for systematic, multi-platform research on chatbot quality and security.

Masserini, E., Clerissi, D., Micucci, D., Mariani, L. (2026). Assessing Task-based Chatbots: Snapshot and Curated Datasets for Dialogflow. In MSR '26: Proceedings of the 23rd International Conference on Mining Software Repositories (pp.682-686) [10.1145/3793302.3793325].

Assessing Task-based Chatbots: Snapshot and Curated Datasets for Dialogflow

Masserini E.;Clerissi D.;Micucci D.;Mariani L.
2026

Abstract

In recent years, chatbots have gained widespread adoption thanks to their ability to assist users at any time and across diverse domains. However, the lack of large-scale curated datasets limits research on their quality and reliability. This paper presents TOFU-D, a snapshot of 1,788 Dialogflow chatbots from GitHub, and COD, a curated subset of TOFU-D including 185 validated chatbots. The two datasets capture a wide range of domains, languages, and implementation patterns, offering a sound basis for empirical studies on chatbot quality and security. A preliminary assessment using the Botium testing framework and the Bandit static analyzer revealed gaps in test coverage and frequent security vulnerabilities in several chatbots, highlighting the need for systematic, multi-platform research on chatbot quality and security.
paper
Chatbot, Dialogflow, Dataset, GitHub;
English
23rd International Conference on Mining Software Repositories, MSR 2026 - 13 April 2026 - 14 April 2026
2026
MSR '26: Proceedings of the 23rd International Conference on Mining Software Repositories
9798400724749
2026
682
686
open
Masserini, E., Clerissi, D., Micucci, D., Mariani, L. (2026). Assessing Task-based Chatbots: Snapshot and Curated Datasets for Dialogflow. In MSR '26: Proceedings of the 23rd International Conference on Mining Software Repositories (pp.682-686) [10.1145/3793302.3793325].
File in questo prodotto:
File Dimensione Formato  
Masserini et al-2026-MSR '26-VoR.pdf

accesso aperto

Tipologia di allegato: Publisher’s Version (Version of Record, VoR)
Licenza: Creative Commons
Dimensione 738.62 kB
Formato Adobe PDF
738.62 kB Adobe PDF Visualizza/Apri

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/10281/628045
Citazioni
  • Scopus 1
  • ???jsp.display-item.citation.isi??? ND
Social impact