{"id":206879,"date":"2020-05-08T07:13:54","date_gmt":"2020-05-08T07:13:54","guid":{"rendered":"http:\/\/www.thetruthseeker.co.uk\/?p=206879"},"modified":"2020-05-10T07:42:49","modified_gmt":"2020-05-10T07:42:49","slug":"code-review-of-fergusons-model","status":"publish","type":"post","link":"https:\/\/www.thetruthseeker.co.uk\/?p=206879","title":{"rendered":"Code Review of Ferguson\u2019s Model"},"content":{"rendered":"

Sue Denim (a pseudonym) \u2013 Lock Down Sceptics May 7, 2020<\/strong><\/h1>\n

Imperial finally released a derivative of Ferguson\u2019s code. I figured I\u2019d do a review of it and send you some of the things I noticed. I don\u2019t know your background so apologies if some of this is pitched at the wrong level.<\/span><\/p>\n

My background.<\/strong>\u00a0I wrote software for 30 years. I worked at Google between 2006 and 2014, where I was a senior software engineer working on Maps, Gmail and account security. I spent the last five years at a US\/UK firm where I designed the company\u2019s database product, amongst other jobs and projects. I was also an independent consultant for a couple of years. Obviously I\u2019m giving only my own professional opinion and not speaking for my current employer.<\/span><\/p>\n

The code.<\/strong> It isn\u2019t the code Ferguson ran to produce his famous Report 9. What\u2019s been released on GitHub<\/a><\/strong><\/span> is a heavily modified derivative of it, after having been upgraded for over a month by a team from Microsoft and others. This revised codebase is split into multiple files for legibility and written in C++, whereas the original program was \u201ca single 15,000 line file that had been worked on for a decade<\/a>\u201d<\/strong><\/span> (this is considered extremely poor practice). A request for the original code was made 8 days ago but ignored<\/a><\/strong><\/span>, and it will probably take some kind of legal compulsion to make them release it.\u00a0 Clearly, Imperial are too embarrassed by the state of it ever to release it of their own free will, which is unacceptable given that it was paid for by the taxpayer and belongs to them.<\/span><\/p>\n

The model.<\/strong>\u00a0 What it\u2019s doing is best described as \u201cSimCity without the graphics\u201d. It attempts to simulate households, schools, offices, people and their movements, etc. I won\u2019t go further into the underlying assumptions, since that\u2019s well explored elsewhere.<\/span><\/p>\n

Non-deterministic outputs.<\/strong>\u00a0Due to bugs, the code can produce very different results given identical inputs. They routinely act as if this is unimportant.<\/span><\/p>\n

This problem makes the code unusable for scientific purposes, given that a key part of the scientific method is the ability to replicate results. Without replication, the findings might not be real at all \u2013 as the field of psychology has been finding out to its cost.\u00a0Even if their original code was released, it\u2019s apparent that the same numbers as in Report 9 might not come out of it.<\/span><\/p>\n

Non-deterministic outputs may take some explanation, as it\u2019s not something anyone previously floated as a possibility.<\/span><\/p>\n

The documentation says:<\/span><\/p>\n

The model is stochastic. Multiple runs with different seeds should be undertaken to see average behaviour.<\/em><\/span><\/p>\n

\u201cStochastic\u201d is just a scientific-sounding word for \u201crandom\u201d<\/strong><\/span><\/a>. That\u2019s not a problem if the randomness is intentional\u00a0pseudo-randomness<\/strong><\/span><\/a>, i.e. the randomness is derived from a starting \u201cseed\u201d which is iterated to produce the random numbers. Such randomness is often used in Monte Carlo techniques<\/strong><\/span><\/a>. It\u2019s safe because the seed can be recorded and the same (pseudo-)random numbers produced from it in future. Any kid who\u2019s played Minecraft is familiar with pseudo-randomness because Minecraft gives you the seeds it uses to generate the random worlds, so by sharing seeds you can share worlds.<\/span><\/p>\n

Clearly, the documentation wants us to think that, given a starting seed, the model will always produce the same results.<\/span><\/p>\n

Investigation reveals the truth: the code produces critically different results, even for identical starting seeds and parameters.<\/span><\/p>\n

I\u2019ll illustrate with a few bugs. In issue 116<\/a><\/strong><\/span>\u00a0a UK \u201cred team\u201d at Edinburgh University reports that they tried to use a mode that stores data tables in a more efficient format for faster loading, and discovered \u2013 to their surprise \u2013 that the resulting predictions varied by around 80,000 deaths after 80 days:<\/span><\/p>\n

\"\"<\/p>\n

 <\/p>\n

Continues \u2026<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"

For tech geeks and software professionals: a former senior Google software engineer offers a devastating analysis of the code that Neil Ferguson used in his report, which led to the Covid-19 lockdown. <\/p>\n","protected":false},"author":39262,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"_et_pb_use_builder":"","_et_pb_old_content":"","_et_gb_content_width":"","footnotes":""},"categories":[61,105],"tags":[],"class_list":["post-206879","post","type-post","status-publish","format-standard","hentry","category-Commentary","category-coronavirus"],"acf":[],"_links":{"self":[{"href":"https:\/\/www.thetruthseeker.co.uk\/index.php?rest_route=\/wp\/v2\/posts\/206879","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.thetruthseeker.co.uk\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.thetruthseeker.co.uk\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.thetruthseeker.co.uk\/index.php?rest_route=\/wp\/v2\/users\/39262"}],"replies":[{"embeddable":true,"href":"https:\/\/www.thetruthseeker.co.uk\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=206879"}],"version-history":[{"count":4,"href":"https:\/\/www.thetruthseeker.co.uk\/index.php?rest_route=\/wp\/v2\/posts\/206879\/revisions"}],"predecessor-version":[{"id":206896,"href":"https:\/\/www.thetruthseeker.co.uk\/index.php?rest_route=\/wp\/v2\/posts\/206879\/revisions\/206896"}],"wp:attachment":[{"href":"https:\/\/www.thetruthseeker.co.uk\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=206879"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.thetruthseeker.co.uk\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=206879"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.thetruthseeker.co.uk\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=206879"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}