Artificial intelligence systems are commonly evaluated through an error-eradication paradigm in which better systems minimize visible failure and support increasingly seamless automation. From a socio-technical perspective, however, the absence of visible error can weaken oversight by concealing the conditions under which AI outputs should be questioned, repaired, or rejected. We introduce productive fallibility as a property of human–AI configurations in which residual fallibility becomes detectable, interpretable, actionable, and contestable. We distinguish destructive, hidden, and productive configurations of error and develop a five-level framework spanning cognitive, interactional, organizational, ethical, and learning functions. Illustrative vignettes from healthcare, education, and organizational decision-making show how interfaces, workflows, and governance arrangements can transform otherwise concealed fallibility into opportunities for verification, human repair, escalation, and organizational learning without legitimizing unsafe failure. We conclude with design principles for inspectability, repairability, calibrated reliance, distributed responsibility, human reflection, and learning loops.
Cabitza, F., Marconi, L. (2026). From Error Eradication to Productive Fallibility: A Socio-Technical Perspective on Human–AI Oversight. In Proceedings of the 12th International Workshop on Socio-Technical Perspectives in Information Systems: Socio-Technical Systems and Organisation (STPIS-STO 2026) co-located with 12th International Conference on Socio-Technical Perspectives in Information Systems (STPIS 2026) (pp.306-322). CEUR-WS.org.
From Error Eradication to Productive Fallibility: A Socio-Technical Perspective on Human–AI Oversight
Cabitza, F.Primo
;Marconi, L.
Secondo
2026
Abstract
Artificial intelligence systems are commonly evaluated through an error-eradication paradigm in which better systems minimize visible failure and support increasingly seamless automation. From a socio-technical perspective, however, the absence of visible error can weaken oversight by concealing the conditions under which AI outputs should be questioned, repaired, or rejected. We introduce productive fallibility as a property of human–AI configurations in which residual fallibility becomes detectable, interpretable, actionable, and contestable. We distinguish destructive, hidden, and productive configurations of error and develop a five-level framework spanning cognitive, interactional, organizational, ethical, and learning functions. Illustrative vignettes from healthcare, education, and organizational decision-making show how interfaces, workflows, and governance arrangements can transform otherwise concealed fallibility into opportunities for verification, human repair, escalation, and organizational learning without legitimizing unsafe failure. We conclude with design principles for inspectability, repairability, calibrated reliance, distributed responsibility, human reflection, and learning loops.| File | Dimensione | Formato | |
|---|---|---|---|
|
Cabitza-Marconi-2026-CEUR-VoR.pdf
accesso aperto
Tipologia di allegato:
Publisher’s Version (Version of Record, VoR)
Licenza:
Creative Commons
Dimensione
1.61 MB
Formato
Adobe PDF
|
1.61 MB | Adobe PDF | Visualizza/Apri |
I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.


