“To err is human but to really foul things up requires a computer”
Software guru Dr Chris Luke on the lessons that healthcare organisations must learn from the Post Office scandal in the UK
One of the few lessons that I recall from the days when I studied ‘management theory’ in the NHS is that useful truism: “To err is human but to really foul things up requires a computer.”
Of course, as a young and eager Senior Registrar, I regarded this as a source of great amusement, but the comic aspect of information technology was lost for me (in 1992 to be precise) when I undertook a survey of the IT systems in all large UK ‘A&E Departments’, as they were then called, for the British Association for Accident and Emergency Medicine, the forerunner of the Royal College of Emergency Medicine.
Back then, I thought there was only one advantage of this time-consuming and slightly tedious project: I became familiar with the name, address and lead consultant of every British emergency department at a time when I was looking for a consultant post myself.
But, in fact, the real and enduring benefit for me was a devout scepticism when it came to the relationship between software and healthcare.
The findings of the ‘IT in the ED’ survey were pretty unremarkable: there was a huge range of newish computerised record systems, none of which was particularly user-friendly and clinicians were often unconvinced of their practical merits, aside from a few medics who had a particular ‘interest’ (either using them for publication material or having been personally involved in the development of the software).
And, in the thirty-odd years since, I confess that my reservations about ‘computer systems’ in the health service have only grown. Again, the reason is simple. Few of the systems that I’ve worked with in large and small hospitals on either side of the Irish Sea have been reliable, straightforward or time-saving.
There was a diversity (each institution having its own) which was baffling – and sometimes downright suspicious – and there seemed to be a universal failure to ‘keep up’ with the changing times and requirements (in one large public hospital, we waited almost a decade for an upgrade in basic word-processing software).
Notwithstanding my own bias, however, there’s a quartet of IT stories in the public domain that should make all clinicians vigilant about the information technology that’s ‘made available’ to them.
In 2013, for instance, the NHS abandoned an electronic health record (EHR) system that had failed to deliver on the promises of its makers and their political and clinical champions (i.e. of a ‘paperless’ ecosystem), at a cost of over £10bn, with one member of the Public Accounts Committee at Westminster describing the affair ‘as one of the worst and most expensive contracting fiascos in the history of the public sector.’
Secondly, there are the umpteen reports from the USA of physicians spending up to six exhausting hours a day completing EHRs, ‘logging large amounts of clinically irrelevant data’ and becoming burned-out as a result.
One author of a paper studying the burden of electronic record-keeping, Dr James Kahn, argued that ‘while the EHR can improve access to medical information, it is a data-hungry beast that exacts a huge toll as measured in hours, clinical focus, communication quality, and quite likely clinical outcomes’. (The problem – as we saw in the UK – is that EHRs are big business.)
Thirdly, I sympathise deeply with the staff in Irish hospitals, GP surgeries and ambulatory services who said that the infamous May 2021 cyberattack on the HSE’s IT systems nationwide with ‘Conti’ ransomware (which caused many computers to be shut down) was by far the most stressful low point of the pandemic, and a source of even more worry and work than the relatively ‘simple’ management of suspected coronavirus cases (as people ransacked bins for bits of paper containing any information about the patients in front of them).
In addition to the distressing publication of sensitive personal data on the (Dark) Web by the suspected Russian hackers, contact tracing, appointment systems, scanning and many other functions were disabled for weeks.
Even in September 2021, only 95 per cent of systems had been restored, and one gets some sense of the difficult choices that had to be made from the estimated costs involved: a ransom of up to €20 million was reportedly demanded but denied (as per official policy) and, as the crisis subsided, Paul Reid, CEO of the HSE, suggested the whole affair would cost them in the region of €100 million.
Mr Reid rightly praised the staff for their ‘resilience’ in dealing with the effects of the cyber-attack and attributed its occurrence (and the lack of secure IT defences) to a fragmented and ‘siloed’ system that had evolved organically after the amalgamation of previous health boards, hospital groups and community healthcare organisations.
After the event, the Irish public were reassured that urgent efforts would be made to appoint suitably expert personnel and systems of defence, but only time will tell if these will be enough to rebut future malicious attacks.
It is the fourth exemplar of IT ‘pathology’, however, which is the source of my gravest concern. This is the so-called ‘Great Post Office/Horizon Scandal’ which was recently described by Prime Minister Sunak as ‘the worst miscarriage of justice in Britain this century’.
It is hard (without reading a book like The Great Post Office Scandal, by Nick Wallis) to comprehend the course of this sorry saga. But if you’re not familiar with the story, you work with computers, and you think that they are ultimately a necessary evil, then I recommend you try to watch the recent ITV 4-part series, Mr Bates vs The Post Office.
This powerful drama brilliantly illustrates the truly Kafkaesque nightmare into which thousands of sub-postmasters were thrust in the 25 years after a new IT accounting system (‘Horizon’, from the Japanese company, Fujitsu) was installed in thousands of post offices around the UK (including Northern Ireland), and it began to ‘reveal’ losses of many thousands of pounds.
By way of example, one poor victim was totting up her day’s takings when a £20,000 ‘shortfall’ suddenly appeared, and nothing she did on the keyboard could explain – or erase – the figure.
The same experience occurred to 4,000 other sub-postmasters who – instead of having the IT ‘checked’ – were promptly accused by the Post Office of theft, fraud and false accounting.
Subsequently, about 900 of them were convicted, with little hard evidence, effort to audit the system or acknowledgment of a staggeringly widespread level of so-called deception.
Meanwhile, hundreds more sub-postmasters ended up ‘paying back’ the Post Office thousands of pounds that had vanished into the Horizon software, even though the computer-generated nature of the losses was obvious in hindsight.
What has emerged lately (courtesy of investigative journalists) is that the powers-that-be in Fujitsu and the Post Office were aware of serious bugs in the Horizon system from the outset, but – faced with what the notorious British judge, Lord Denning, once referred to as an ‘appalling vista’ (i.e. the exposure of a terrible injustice for all to see) – they preferred to pick off, persecute and prosecute their employees, to see them ostracised, imprisoned and ruined (with resulting suicides and family breakdown), and to deploy expensive PR and legal tactics for years to suppress the facts.
In short, the ‘administration’ (i.e. successive CEOs and middle managers) knew about the literally fatal flaws in Horizon for years before they were – under duress – finally acknowledged. And – to date – compensation has been received by just a fraction of victims.
Meanwhile, Ministers came and went and did nothing, and it has been suggested that real compensation will come to £4 billion or more.
I’m not a very enthusiastic capitalist, but I am passionate about fairness, social justice, and public service. It has long seemed to me that IT systems are promoted primarily by those with a vested interest.
That’s understandable and of little interest to me when it’s confined to the private sector, but it is profoundly consequential when it involves the public sector.
The hype that accompanies the launch of many IT-in-healthcare systems (like Babylon Health, a briefly profitable ‘digital health service provider’ in the UK, with an alluring app, ostensibly aimed at the worried wealthy) belies the often-hidden costs to patients and professionals, regardless of whether it fails or ‘works’.
The latter can mean endless form-filling for the benefit of the ‘higher-ups’ at the expense of actual work, while the former may see people in the C-suites do almost anything to cover up embarrassing failings in the acquisition or performance of dodgy software.
The Great Post Office/Horizon Scandal demonstrates clearly that incompetent senior decision-makers with dubious morals can combine with dodgy tech to perpetrate appalling injustices. And, as we move forward into the brave new world of AI-driven healthcare, I’m reminded of that other indispensable motto for the general user of the new machines: artificial intelligence is no match for human stupidity.