Chapter 7. Living well with brokenness in an inclusive research culture: what we can learn from failures and processes in a Digital Humanities lab

Technical failures, living well and learning with brokenness

KDL has internal monitoring mechanisms to check on the health of servers and applications are in place with upgrades and patching occurring at regular intervals, often accelerated or intensified by incoming cybersecurity alerts or other vulnerabilities the team is made aware of. Despite these regular maintenance cycles, co-habitation with technology is prone to changes and breaks.

A recurring vehicle via which the KDL team engages with ‘brokenness’ is via colleagues or the public reporting that a site or a functionality is down or not working. Given the size of public-facing digital resources we maintain (currently circa ninety) and the fact that some of those project architectures date back to the end of the 1990s, requests or problems occur quite frequently. In the last year (April 2023–April 2024), around ten issue reports were recorded per month as illustrated in Figure 7.1 (King’s Digital Lab 2019; De Roure et al. 2022). Once in the pipeline, these signals of brokenness are documented, discussed and addressed within the team with predominant involvement of Lab Manager or Project Manager and Systems Manager.

If the broken functionality is reported for a project that is under Service Level Agreement (SLA) with the lab – that is a project for which terms and costs of services such as web hosting are agreed – KDL have the responsibility to investigate and act as soon as possible.1 It is noteworthy that the standard SLA language (King’s Digital Lab 2019) acknowledges brokenness as a constant with usually at least a day of ‘bug fix time’ –human probing needed to analyze the problem and initiate a repairing action – accounted for every year of maintenance.

For an RSE team deeply engaged with the mechanics of technical systems and processes, brokenness is a risk with very high probability. Hence mitigations or countermeasures are a necessity. While solving some of these problems of brokenness can be at times hard and painful, this is also the stakes of the game, the ‘beauty’ of working with things that one learns to adjust and tweak. Some of the mitigations are inscribed into evolving policies and governance documentation – including SLAs but also in other phases of projects’ lifecycles, such as in the formal Product Quote signed prior to any collaboration. For example, snippets of the SLA currently in use reads as following:

The evolving nature of the online security environment can present challenges to maintaining complex software systems over the long term. KDL works to balance the interests of project partners with the security of KCL’s IT network but this is not always possible. In the unlikely event of an extended outage, KDL will notify project partners and extend this SLA by at least the length of the outage. If the project cannot be securely updated for technical reasons KDL reserves the right to end the SLA […]

[...] servers are closely monitored, but it is important that we are notified in advance if any significant increases in traffic are expected (due to the resource being profiled on national television, for example). KDL cannot be held responsible for outages caused by significant predictable increases in site traffic.

Other mitigations are enacted via monitoring processes such as the management of requests outlined above and exemplified in Figure 7.2.

However, these ad-hoc repairs, maintenance cycles and governance documents do not address other elements of brokenness that are endemic to the design of software such as testing processes to assess – and on purpose break! – parts of a solution or entire systems to identify vulnerabilities and make them more robust prior to deployment. Brokenness can indeed be a very valuable feature of intermediate or ephemeral products such as proof of concept and scale models built to test a method or a technology. In the tradition of design methods (Norman 1988), this risk-averse and playful attitude to incompletion and brokenness is part of constructive and creative iterations in prototyping, where the makers of mockups and prototypes embrace the temporary, craftlike, half-broken status of these artefacts to learn, experiment, test and evolve an idea in practice as Figure 7.3 exemplifies (Ciula and Smithies 2023).

Therefore, while change and uncertainty in a research production context affect long tail digital scholarly outputs as well as ephemeral experiments, the implications are different. There is a productive tension between embracing generative technical brokenness which is part of the evolving technical production process (such as in Figure 7.3) – very akin to the learning by doing spirit in maker labs but also to the production of intermediate artifacts in experimental research – and mitigating its destabilising effects when technical products have reached maturity and are expected to continue to be in use, as it is often the case for digital research outputs. The holistic Archiving and Sustainability programme KDL has developed over the years (King’s Digital Lab 2024) outlines criteria for maintenance and associated options, in particular around material obsolescence and degradation or ‘sunsetting’ of digital products.

These days, Software Development Lifecycles (SDLC) for new projects in KDL embed open discussions on shared responsibilities and forward planning from the outset, embracing a holistic approach to infrastructure aware of technical constraints and options, human efforts and collective costs of repairing, patching and tinkering. Archiving or static-first development approach has become a humble ambition – to reach a baseline of what should stay rather than assuming all components of a fully-fledged solution deserve to last.2 In this sobering perspective (Tucker 2022), one could say that failure to understand the financial, technical and human costs of sustaining a rich portfolio of digital projects, has turned into an opportunity for the Lab, namely the opportunity to review critically and creatively its SDLC – where brokenness remains endemic, nonetheless.

KDL team members’ reflections over the relatively recent history of KDL have resulted in processes to address digital (material) obsolescence, change and uncertainty in a research production context which spans long tail digital scholarly outputs as well as ephemeral experiments. Brokenness has become an accepted constant, with mitigations and countermeasures not only enacted by local expertise in daily operations (Figure 7.2) but also inscribed in (evolving) processes (Figure 7.4), contractual obligations and institutional policies. Rather than attempt to circumnavigate failure, these efforts make brokenness emerge as endemic of the digital RTPs creative way of working – underpinned by processes of repairing, patching and tinkering.

The role of experts in a broken research culture

I referred above to the human costs of sustaining and maintaining digital resources cognisant of the implications this has for the field as a whole. If we are serious about embedding the digital into the arts and humanities knowledge production cycle, the role of digital RTPs is essential to this endeavour (Ciula 2022; Pawlicka-Deger 2022). Failure to recognise the fundamental importance of these roles for research and training has caused several issues to the quality standards and research culture of the discipline that we should tackle with some urgency. What follows are what I with others (Bergel et al. 2020; McGillivrary et al. 2020; Romanova et al. 2021; Gambell et al. 2023) see as some examples of these failures, which are the result of substantial disinvestment in digital research infrastructures that unfortunately span the academic sector beyond DH.

• Mismatch between increasingly digital lifecycle of projects and the career pipelines that would provide the expertise to design, develop and maintain those lifecycles. This failure bears a question and a call for action addressed mainly to those who lead teaching and training programmes: are we working towards creating the profiles of the digital RTPs we need? DH could contribute quite substantially to shaping holistic and diverse profiles by giving weight not only to data and systems but to process-oriented teaching and training and by including design and analysis skills in education strategies.

• Precarity of labour associated with technical roles. Here the overtly open critique is addressed to senior management of the higher education sector (a sector I am conscious I represent) as well as of comparable independent research organisations operating in the arts, humanities and cultural heritage research domains: can digital RTPs in these domains become more mainstream than a few laboratories and ad hoc figures with no defined roles and career progression? While we cannot compete with industry remunerations, benefits including inclusive and stimulating research environments, career paths and opportunities ought to be defined and offered systematically across institutions.

• Outdated, if not discriminatory, research cultures that are unable or unwilling to value the integral role technical objects play in knowledge production and therefore to value complementary digital research outputs including software (Hidden REF n.d.; Tasovac et al. 2023) and those experts that have an intimate knowledge of how these objects work or break (Smithies et al. 2023). This borderline discriminatory research culture goes beyond DH and is reflected in our academic promotions systems and evaluation frameworks but also in funding streams and, more subtly, yet with substantial implications, in attitudes, habits and behaviours.

• Individualist research focus which discourages creative and collaborative re-use and reduces collective impact to unidirectional dissemination activities. Linked to the point above, this relates to the difficulty of reforming research and education environments to work with (and not only for) communities. Experts cognisant of the importance of participatory and imaginative design methods are needed to achieve this reform and ultimately contribute to living well with technology collectively.3

It is encouraging to see – thanks in part to DH community lobbying and activism – national funders and institutions in the UK have become sensitive to some of these failures; programmes and policies are emerging to address some of them.4 However, some systemic failures inscribed in the research academic culture of fields where digital RTPs’ roles are essential, such as DH, call for a more radical programme of change that integrate lessons learned by teams such as KDL and invest in fixing a destructive rather than constructive kind of brokenness, which leads to the devaluing of technical expertise and ultimately to poor research. If, in the case of KDL, the failure to understand the financial, technical and human costs of developing and sustaining a rich portfolio of digital projects has found some process-oriented mitigations, the responsibility to address the failure of consolidating the role of digital RTPs via adequate research and education strategies lies beyond process improvements practices in the local context. Responsible collective action – informed by DH-inspired values – is needed to foster the ethics of critical technical development, the shaping of new generations of digital RTPs and the reformation of a truly inclusive research culture.

References

Alan Turing Institute. ‘Data/Culture: Building Sustainable Communities around Arts and Humanities Datasets and Tools’. Accessed 18 October 2024. https://www.turing.ac.uk/research/research-projects/dataculture-building-sustainable-communities-around-arts-and-humanities.

Alan Turing Institute. ‘People in Data’. Accessed 18 October 2024. https://www.turing.ac.uk/research/research-projects/people-data.

Bergel, Giles, Pip Willcox, Guyda Armstrong, James Baker, Arianna Ciula, Nicholas Cole, Julianne Nyhan, et al. ‘Sustaining Digital Humanities in the UK’. Zenodo, 25 September 2020. https://doi.org/10.5281/zenodo.4046267.

Caton, Paul, and Brian Maher. ‘After the Grant’. In Sixth Annual Conference for Research Software Engineering. Newcastle Upon Tyne, 2022. https://virtual.oxfordabstracts.com/event/3101/submission/107.

Ciula, Arianna. ‘Exposing Legacy Project Datasets in Digital Humanities: KDL Experience’. King’s Digital Lab - Thoughts and Reflections from the Lab (blog), 7 July 2020. https://www.kdl.kcl.ac.uk/blog/legacy-project-datasets/.

———. ‘What Does the Role of Research Technology Professional Mean to King’s Digital Lab?’ King’s Digital Lab - Thoughts and Reflections from the Lab (blog), 13 October 2022. https://kdl.kcl.ac.uk/blog/rtp-rse/.

Ciula, Arianna, Øyvind Eide, Cristina Marras, and Patrick Sahle. Modelling Between Digital and Humanities: Thinking in Practice. Open Book Publishers, 2023. https://doi.org/10.11647/obp.0369.

Ciula, Arianna, and James Smithies. ‘Sustainability and Modelling at King’s Digital Lab: Between Tradition and Innovation’. In On Making in the Digital Humanities: Essays on the Scholarship of Digital Humanities Development in Honour of John Bradley, edited by Julianne Nyhan, Geoffroy Rockwell, Stefan Sinclair, and Alexandra Ortolja-Baird, 78–104. London: University College London Press, 2023. https://doi.org/10.14324/111.9781800084209.

De Roure, David, John Moore, Kevin Page, Toby Burrows, David Beavan, Timothy Hobson, Giles Bergel, et al. ‘DigiSpec: Scoping Future Born-Digital Data Services for the Arts and Humanities: Case Reports’, 31 August 2022. https://doi.org/10.5281/zenodo.4716148.

Gambell, Sarah, Paul Gooding, Lorna Hughes, Michelle Doran, Órla Murphy, Charlotte Tupman, Jane Winters, et al. ‘Communicating the Value and Impact of Digital Humanities in Teaching, Research, and Infrastructure Development’. Zenodo, 3 November 2021. https://doi.org/10.5281/zenodo.5643727.

Hidden REF. n.d. ‘The Hidden REF: Celebrating All Research Outputs’. Accessed 18 October 2024. https://hidden-ref.org/.

‘iDAH Research Software Engineering (RSE) Steering Group Working Paper’. Zenodo, 20 June 2023. https://doi.org/10.5281/zenodo.8060003.

‘iREAL: Inclusive Requirements Elicitation for AI in Libraries – Inclusive Requirements Elicitation for AI in Libraries’. Accessed 18 October 2024. https://irealrai.org/.

Jakeman, Neil. ‘Safeguarding an Inheritance and Ensuring a Legacy’. King’s Digital Lab - Thoughts and Reflections from the Lab (blog), 19 August 2020. https://kdl.kcl.ac.uk/blog/sdlc-for-rse/.

Jakeman, Neil, and Elliott Hall. ‘Empowering Artistic Innovation’. King’s Digital Lab - Thoughts and Reflections from the Lab (blog), 8 April 2024. https://kdl.kcl.ac.uk/blog/glow3-blog/.

King’s Digital Lab. ‘A Software Development Life Cycle for Research Software Engineering’. sdlc-for-rse, 2019. https://github.com/kingsdigitallab/sdlc-for-rse.

King’s Digital Lab. ‘Archiving and Sustainability’. Accessed 18 October 2024. https://kdl.kcl.ac.uk/about/archiving-and-sustainability/.

McGillivray, Barbara, Beatrice Alex, Sarah Ames, Guyda Armstrong, David Beavan, Arianna Ciula, Giovanni Colavizza, et al. ‘The Challenges and Prospects of the Intersection of Humanities and Data Science: A White Paper from The Alan Turing Institute’. The Alan Turing Institute, 4 August 2020. https://doi.org/10.6084/m9.figshare.12732164.v5.

Norman, Donald A. The Psychology of Everyday Things. Basic Books, 1988.

Pawlicka-Deger, Urszula. ‘Digital Humanities Needs Equality between Humanists and Technicians’. Times Higher Education (THE) (blog), 9 July 2022. https://www.timeshighereducation.com/blog/digital-humanities-needs-equality-between-humanists-and-technicians.

Romanova, Natasha, Arianna Ciula, James Smithies, Neil Jakeman, Órla Murphy, Charlotte Tupman, Jane Winters, et al. ‘Capacity Enhancement in Digital Humanities in the United Kingdom and Ireland: Training and Beyond’. Zenodo, 15 July 2021. https://doi.org/10.5281/zenodo.5105938.

Smithies, James, and Arianna Ciula. ‘Humans in the Loop: Epistemology & Method in King’s Digital Lab’. In Routledge International Handbook of Research Methods in Digital Humanities, edited by Kristen Schuster and Stuart Dunn, 155–72. Routledge, 2020. https://doi.org/10.4324/9780429777028.

Smithies, James, Patrick ffrench, and Arianna Ciula. ‘Droit de Cité: The Digital Lab as Digital Milieu’. In Digital Humanities and Laboratories: Perspectives on Knowledge, Digital Research in the Arts and Humanities Infrastructure and Culture, edited by Urszula Pawlicka-Deger and Christopher Thompson, 52–66. Routledge, 2023. https://doi.org/10.4324/9781003185932.

Smithies, James, Anna Maria Sichani, Carina Westling, Pam Mellen, and Arianna Ciula. ‘Managing 100 Digital Humanities Projects: Digital Scholarship & Archiving in King’s Digital Lab’. Digital Humanities Quarterly 13, no. 1 (2019). http://www.digitalhumanities.org/dhq/vol/13/1/000411/000411.html.

‘STEP-UP: A Strategic TEchnical Platform for University Technical Professionals’. Accessed 18 October 2024. http://step-up.ac.uk/.

Tasovac, Toma, Laurent Romary, Erzsébet Tóth-Czifra, Rahel C. Ackermann, Daniel Alves, Sally Chambers, Mike Cosgrave, et al. ‘The Role of Research Infrastructures in the Research Assessment Reform: A DARIAH Position Paper’, June 2023. https://hal.science/hal-04136772.

Tucker, Joanna. ‘Facing the Challenge of Digital Sustainability as Humanities Researchers’. Journal of the British Academy 10 (2022): 93–120. https://doi.org/10.5871/jba/010.093.

UK Research and Innovation (UKRI). 2021. ‘Technician Commitment Action Plan’. https://www.ukri.org/wp-content/uploads/2022/03/UKRI-010322-TechnicianCommitmentActionPlan.pdf.