Artificial intelligence systems are commonly evaluated through an error-eradication paradigm in which better systems minimize visible failure and support increasingly seamless automation. From a socio-technical perspective, however, the absence of visible error can weaken oversight by concealing the conditions under which AI outputs should be questioned, repaired, or rejected. We introduce productive fallibility as a property of human–AI configurations in which residual fallibility becomes detectable, interpretable, actionable, and contestable. We distinguish destructive, hidden, and productive configurations of error and develop a five-level framework spanning cognitive, interactional, organizational, ethical, and learning functions. Illustrative vignettes from healthcare, education, and organizational decision-making show how interfaces, workflows, and governance arrangements can transform otherwise concealed fallibility into opportunities for verification, human repair, escalation, and organizational learning without legitimizing unsafe failure. We conclude with design principles for inspectability, repairability, calibrated reliance, distributed responsibility, human reflection, and learning loops.

Cabitza, F., Marconi, L. (2026). From Error Eradication to Productive Fallibility: A Socio-Technical Perspective on Human–AI Oversight. In Proceedings of the 12th International Workshop on Socio-Technical Perspectives in Information Systems: Socio-Technical Systems and Organisation (STPIS-STO 2026) co-located with 12th International Conference on Socio-Technical Perspectives in Information Systems (STPIS 2026) (pp.306-322). CEUR-WS.org.

From Error Eradication to Productive Fallibility: A Socio-Technical Perspective on Human–AI Oversight

Cabitza, F.
Primo
;
Marconi, L.
Secondo
2026

Abstract

Artificial intelligence systems are commonly evaluated through an error-eradication paradigm in which better systems minimize visible failure and support increasingly seamless automation. From a socio-technical perspective, however, the absence of visible error can weaken oversight by concealing the conditions under which AI outputs should be questioned, repaired, or rejected. We introduce productive fallibility as a property of human–AI configurations in which residual fallibility becomes detectable, interpretable, actionable, and contestable. We distinguish destructive, hidden, and productive configurations of error and develop a five-level framework spanning cognitive, interactional, organizational, ethical, and learning functions. Illustrative vignettes from healthcare, education, and organizational decision-making show how interfaces, workflows, and governance arrangements can transform otherwise concealed fallibility into opportunities for verification, human repair, escalation, and organizational learning without legitimizing unsafe failure. We conclude with design principles for inspectability, repairability, calibrated reliance, distributed responsibility, human reflection, and learning loops.
paper
Human–AI interaction; productive fallibility; socio-technical systems; human oversight; calibrated reliance; contestability; resilience engineering
English
The 12th International Workshop on Socio-Technical Perspectives in Information Systems: Socio-Technical Systems and Organisation (STPIS-STO 2026) co-located with 12th International Conference on Socio-Technical Perspectives in Information Systems (STPIS 2026) - September 16-17, 2026
2026
Tarantino, L; Bednar, P; Vallo Hult, H; Kävrestad, J; D’Aloisio, G; Rajanen, M
Proceedings of the 12th International Workshop on Socio-Technical Perspectives in Information Systems: Socio-Technical Systems and Organisation (STPIS-STO 2026) co-located with 12th International Conference on Socio-Technical Perspectives in Information Systems (STPIS 2026)
2026
4325
306
322
https://ceur-ws.org/Vol-4325/
open
Cabitza, F., Marconi, L. (2026). From Error Eradication to Productive Fallibility: A Socio-Technical Perspective on Human–AI Oversight. In Proceedings of the 12th International Workshop on Socio-Technical Perspectives in Information Systems: Socio-Technical Systems and Organisation (STPIS-STO 2026) co-located with 12th International Conference on Socio-Technical Perspectives in Information Systems (STPIS 2026) (pp.306-322). CEUR-WS.org.
File in questo prodotto:
File Dimensione Formato  
Cabitza-Marconi-2026-CEUR-VoR.pdf

accesso aperto

Tipologia di allegato: Publisher’s Version (Version of Record, VoR)
Licenza: Creative Commons
Dimensione 1.61 MB
Formato Adobe PDF
1.61 MB Adobe PDF Visualizza/Apri

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/10281/630481
Citazioni
  • Scopus ND
  • ???jsp.display-item.citation.isi??? ND
Social impact