1. Home
  2.  » 
  3. Health
  4.  » 
  5. Coronavirus
  6.  » Coding that led to lockdown was ‘totally unreliable’ and a ‘buggy mess’, say experts

Hannah Boland and Ellie Zolfagharifard – The Telegraph May 16, 2020

Professor Neil Furguson. Click to enlarge

The Covid-19 modelling that sent Britain into lockdown, shutting the economy and leaving millions unemployed, has been slammed by a series of experts.

Professor Neil Ferguson’s computer coding was derided as “totally unreliable” by leading figures, who warned it was “something you wouldn’t stake your life on”.

The model, credited with forcing the Government to make a U-turn and introduce a nationwide lockdown, is a “buggy mess that looks more like a bowl of angel hair pasta than a finely tuned piece of programming”, says David Richards, co-founder of British data technology company WANdisco.

“In our commercial reality, we would fire anyone for developing code like this and any business that relied on it to produce software for sale would likely go bust.”

The comments are likely to reignite a row over whether the UK was right to send the public into lockdown, with conflicting scientific models having suggested people may have already acquired substantial herd immunity and that Covid-19 may have hit Britain earlier than first thought. Scientists have also been split on what the fatality rate of Covid-19 is, which has resulted in vastly different models.

Up until now, though, significant weight has been attached to Imperial’s model, which placed the fatality rate higher than others and predicted that 510,000 people in the UK could die without a lockdown.

It was said to have prompted a dramatic change in policy from the Government, causing businesses, schools and restaurants to be shuttered immediately in March. The Bank of England has predicted that the economy could take a year to return to normal, after facing its worst recession for more than three centuries.

The Imperial model works by using code to simulate transport links, population size, social networks and healthcare provisions to predict how coronavirus would spread. However, questions have since emerged over whether the model is accurate, after researchers released the code behind it, which in its original form was “thousands of lines” developed over more than 13 years.

In its initial form, developers claimed the code had been unreadable, with some parts looking “like they were machine translated from Fortran”, an old coding language, according to John Carmack, an American developer, who helped clean up the code before it was published online. Yet, the problems appear to go much deeper than messy coding.

Neil Ferguson

·         2001

Professor Ferguson worked on research that led to the mass culling of hundreds of thousands of farm animals during the 2001 foot and mouth disease epidemic. His work was later challenged in reviews by other academics.

·         2002

Predicted that between 50 and 50,000 people could die from exposure to BSE (mad cow disease) in beef. He also predicted that number could rise to 150,000 if there was a sheep epidemic as well. In the UK, there have been fewer than 200 deaths from the human form of BSE

·         2005

Prof Ferguson claimed that up to 200 million people could be killed from bird flu. In an interview with the Guardian, he compared it to the 1918 Spanish flu, adding: “There are six times more people on the planet now so you could scale it up to around 200 million people probably.” Only several hundred people died worldwide.

·         2009

The Imperial team predicted that swine flu had a case fatality rate of between 0.3 per cent and 1.5 per cent, with his most likely estimate landing at 0.4 per cent. A Government estimate based on his advice was that a reasonable worst-case scenario was the disease would lead to 65,000 UK deaths. Swine flu killed just 457 people in the UK, with the chief medical officer concluding in 2009 that the actual mortality rate was 0.026 per cent.

·         March 2020

A paper produced by Prof Ferguson’s team predicted that the coronavirus pandemic could lead to 250,000 deaths in the UK unless stringent lockdown measures were implemented. His research is said to have convinced the Government to change direction.

·         May 5 2020

Professor Ferguson quits the Scientific Advisory Group on Emergencies (Sage) after The Telegraph confronts him over his married lover visiting him at his home, thereby breaking the lockdown rules

Many have claimed that it is almost impossible to reproduce the same results from the same data, using the same code. Scientists from the University of Edinburgh reported such an issue, saying they got different results when they used different machines, and even in some cases, when they used the same machines.

“There appears to be a bug in either the creation or re-use of the network file. If we attempt two completely identical runs, only varying in that the second should use the network file produced by the first, the results are quite different,” the Edinburgh researchers wrote on the Github file.

After a discussion with one of the Github developers, a fix was later provided. This is said to be one of a number of bugs discovered within the system. The Github developers explained this by saying that the model is “stochastic”, and that “multiple runs with different seeds should be undertaken to see average behaviour”.

However, it has prompted questions from specialists, who say “models must be capable of passing the basic scientific test of producing the same results given the same initial set of parameters…otherwise, there is simply no way of knowing whether they will be reliable.”

It comes amid a wider debate over whether the Government should have relied more heavily on numerous models before making policy decisions.

Writing for telegraph.co.uk, Sir Nigel Shadbolt, Principal at Jesus College, said that “having a diverse variety of models, particularly those that enable policymakers to explore predictions under different assumptions, and with different interventions, is incredibly powerful”.

Like the Imperial code, a rival model by Professor Sunetra Gupta at Oxford University works on a so-called “SIR approach” in which the population is divided into those that are susceptible, infected and recorded. However, while Gupta made the assumption that 0.1pc of people infected with coronavirus would die, Ferguson placed that figure at 0.9pc.

That led to a dramatic reversal in government policy from attempting to build “herd immunity” to a full-on lockdown. Experts remain baffled as to why the government appeared to dismiss other models.

Continues …