Catherine Thompson (ctthompso@student.unimelb.edu.au)
University of Melbourne, Faculty of Engineering & IT, School of Computing & Information Systems, 2025
Abstract
Digital ecosystems are increasingly integral to everyday human activity and the conduct of our societies. While these ecosystems have positive impacts, they are also capable of generating systemic and societal scale harms. Digital ecosystems – complex sociotechnical systems with a technology artefact at their core – evolve more rapidly than efforts to regulate them. Whether accidentally or by design, their operation is not always benign. Yet, the interactions that shape their evolving natures have still to be systematically investigated. In particular, the dynamics that promote and sustain trustworthiness in a digital ecosystem have not been rigorously examined, and the indicators that signal untrustworthiness are yet to be surfaced. These deficiencies are especially troubling in high-risk contexts such as systems of social protection and other forms of e-government. This interdisciplinary research explores trustworthiness and its absence in digital ecosystems through the lens of the question: “What forces and relationships shape the trustworthiness of a digital ecosystem, and how might we apprehend them?”. It seeks to address the challenge that interpreting function and dysfunction in a complex system presents for systems theorists. It aims to develop exploratory approaches to studying trustworthiness and its absence in conditions of complex causality; to test their explanatory power; and to inspire ecosystems that are more intentionally trustworthy. The research draws on sociotechnical, complexity and systems theories to explain the nature of digital environments, and on philosophical theories of trust, trustworthiness and truth to explain core concepts. It integrates insights from the information systems (IS) reference disciplines of sociology and management and organisation studies, as well as contributions from science, technology and society (STS) studies, safety science and criminology. It also engages with emerging research fields, including algorithmic and data justice and institutional gaslighting. Critical realism (CR) provides the research paradigm and many of the techniques on which this qualitative enquiry is founded. The research considers a single case study – the egregious Australian e-government welfare scandal Robodebt – through the lens of our exploratory theoretical framework to develop causal accounts of the forces that influenced this ecosystem’s untrustworthy evolution. Our findings illuminate how key system roles and their interactions determine ecosystem trustworthiness. We show how opaque ecosystems may signal their true nature and deviance, and we highlight the frailties of formal governance regimes, the mechanisms by which truth narratives may emerge or be suppressed, and the surprising centrality of the role of the Truthteller. We advance an argument for structured moral evaluation as a key system corrective, and from it develop the outline of a virtue ethics theory of technology and explore its implications. The principal contribution of this research to theory is a CR-informed theoretical framework of trustworthiness in the digital world, including in its absence. Several framework elements represent theoretical contributions in their own right. In particular, the new phenomenon of untrustworthy technology as an entry to the IS dark side canon, and a fully rounded, interdisciplinary conceptualisation of the system role and action of the Truthteller. In addition, a governance model contributes to the discourse on regulating digital-world phenomena, and a playbook of truth suppression and denial techniques, relevant to digital and non-digital settings, completes the theoretical contribution. Methodologically, this research advances the use of critical realism in IS as a research paradigm for developing causal explanations of complex sociotechnical realities at different levels of analysis. A sensemaking model developed for the research contributes to extending the emerging fields of algorithmic and data justice with causal reasoning Smaller methodological contributions include a novel use of dynamic capabilities to explore the microfoundations of the truthtelling journey. Several of the findings will have practical application for policymakers, system designers and other practitioners. The Regulatory pyramid offers a blueprint for professionals with responsibilities for developing regulatory approaches to emerging technologies and their evolving use cases. We also describe and collate indicators of system untrustworthiness to provide an assessment tool for use by risk professionals in sensitive and regulated environments, as well as by designers seeking to create systems that are more intentionally trustworthy. Lastly, we propose an approach to the systemic design of trustworthy new digital ecosystems, together with concrete actions to improve the trustworthiness of those that already exist.
Methods
Qualitative