PassW0rd – 13th May 2020 (NHS App)

Speaker A: This program is brought to you by Resonance FM. If you like what you hear, please support our work by making a donation at resonancefm.com/donate.

Speaker B: Hello, I’m Peter Warren, and I’ve got a bit of a cough. I should get it checked out. In this edition of Password on Resonance FM, I’m exploring how that cough could be diagnosed as coronavirus or not using smartphone apps and artificial intelligence. Can technology cure a pandemic and free us all from lockdown? There are promising signs that it can. And there are some doubts.

Speaker C: This is the starter of a full meal. Don’t even think for a moment that is the thing. But I’ve also seen, unfortunately, simplifications, rhetoric of the app is the savior. That is insane.

Speaker D: If it’s bungled in the roll-up, then people might kind of scorn it for good and say, well, I tried once, it didn’t download very well, or I downloaded it, I got a bunch of notifications when I hadn’t even left the house. I’m not going to give that another shot.

Speaker E: This is proximity events, not risk, that is being measured. The phones, by signal strength and so on, can see whether you’re— roughly speaking— how far you might be between you and another person. But it can’t tell, for instance, whether you shook hands, whether you embraced, whether the person was inside or outside, whether the person was coughing, whether they were wearing a face mask. None of this is recorded, and all of those are pretty significant in terms of what risk is involved. It’s not just a matter of time, it’s the nature of the contact.

Speaker B: Some people believe the COVID tracing apps will come at a high price in terms of loss of privacy. I’m one of them. Some parts of Great Britain are deploying smartphone apps and AI as a way to unlock the lockdown without causing a spike of new infections. In the UK, Baroness Harding has just been appointed to oversee the process. Now that name rings a bell. An alarm bell. Dido Harding was the CEO of TalkTalk, the telecoms company that was hacked by cybercriminals in 2015 when 156,959 customers’ bank and phone account details were stolen. When asked if the affected customer data was encrypted or not, she replied, “The awful truth is I don’t know.” A keen horsewoman and racehorse owner, after she left TalkTalk, she joined the board of the Jockey Club and was subsequently appointed head of NHS Improvement. At the time of the TalkTalk hack, employees and contractors at TalkTalk told me that Harding had been actively engaged in cost-cutting at the company that cybersecurity was one of her targets. In Switzerland, Professor David Atienza Alonso developed an app called Coughid at the University of Lausanne’s Embedded Systems Laboratory. He explained how it works to Passwords Jane Wyatt.

Speaker C: We have to record a number of seconds where we typically need to have at least between 3 to 5 coughs. These coughs are recorded directly with the phone, so we just need to have a microphone that is minimum quality, so you have to record not too far from the microphone of your smartphone, and then you send us the recording and this recording will be automatically analysed.

Speaker A: I see, and what are the characteristics of a COVID cough?

Speaker C: So the COVID cough, that’s the point, is that there is not yet any standard signature, so there are similar exercises being done all over the world. So the current exercise that we have been doing is with the doctors, both in Lausanne or in Switzerland in general, because we have also hospitals in the German part and also in Geneva, trying to analyze the cough. So it’s a very dry cough that has short intervals, but I cannot give you more references because it doesn’t really have a pattern yet.

Speaker A: And how accurate can it be, do you think? Do you have any results that show it is accurate, that you can diagnose a cough in this way?

Speaker C: So the idea of this app is to do a prescreening. So it cannot be used as a single-diagnose mechanism because COVID is a disease that is multi-parametric. So first of all, like I said, we only are aware that 2/3 of the people that have COVID will have a cough. So basically, we cannot— even if we are perfect on the recognition, we can only target to recognize around 66% of the people that may have it. Now, having said that, because there is a very high variability, what we are hoping to target is something in the range of 70% accuracy within the 66%. So I would argue that we can detect around 50% if we are actually lucky with the generalization. These are results that we currently have for a small dataset, but the only way to really get with a real absolute value is when we have a large volume of data we can see the results for a large set of the population.

Speaker B: As we heard, David Atienza is aware of privacy and data protection concerns. His team includes several clinicians who abide by the ethics of the Hippocratic Oath taken by all healthcare workers to do no harm. There’s no such oath for technologists. However, the Council of Europe has drawn up some guidelines Sophie Kwasny, head of the Council of Europe’s Data Protection Unit, outlines the organisation’s concerns.

Speaker A: This has been done with international cooperation, so researchers from different countries in Europe are trying to work together and trying to build a system that would be the least intrusive possible, and putting in guarantees in terms of anonymisation, decentralisation, minimisation of the data, transparency, which is is also a key feature, the fact that you would rather trust something that you know is fully public and that can be scrutinized by computer scientists instead of something and a code that cannot be fully checked. So it’s great that there has been this initiative of researchers trying to propose something that would be the less intrusive possible. The same actually happened in the US with some of the MIT researchers and teams putting up a proposal. And then you’ve had alternative proposals from the big internet players that I mentioned, Apple and Google teaming up to propose something. What is also quite noticeable for the European part, and when I say Europe now, I mean the EU, is that the EU very rapidly in addressing the COVID crisis has tried to provide coordination and common guidance to its member states. And so there was a toolbox developed by the European Commission on precisely those apps that was also complemented by a specific data protection guidance. They are called the guidelines adopted by the group of data protection authorities of the 27 EU member states, trying to address those issues and trying to provide orientations as to the way to go.

Speaker B: I understand that in Austria that the Austrian government has taken the records from the mobile phone companies and is using those. Is that true, do you know?

Speaker A: I’m not specifically aware of the Austrian case, but the use of mobile data held by telecom operators is not new. And Same thing there at EU level. We have seen very early in the crisis, we’ve seen Thierry Breton requesting that telecom operators share their mobility data and mobile data with the European Commission, but in a great aggregated anonymized manner. So this has been a call done at EU level. This is something that is done at national level. Myself, I live in France and I know that Orange, which was the historic national operator for telecoms, has a partnership with the National Institute for Research where, in those circumstances, the mobile data is shared in an aggregated manner to enable research on the spread and, you know, movements of population and things like that. So this is being done. The question is how is it done, what are the safeguards, safeguards that are in place in terms of aggregation of data and making it impossible for the data to lead to a particular individual. Regarding mobile data, there is the group, it’s the Association of GSM Operators, which is a global association that already at the time of the Ebola crisis had been issuing guidance on what to do with telecom data, how it can be used, and what are the precautions and safeguards you need to put in place so that there are no risks for the individuals and for the right to data protection. So I would say this is not something new, and there are manners in which you can use this data without interfering with the rights of the individuals.

Speaker B: Now, one of the issues— I spoke to some of the people who are actually actively involved in these exercises, and I made the point to them that one of the things that you want to do to understand what’s happening is you want to hold on to that data for quite a long time to be able to get the maximum value from it, perhaps even after the crisis is over. What’s the situation with the retention of that data?

Speaker A: We’re speaking about an exceptional measure here in a state of emergency, and this by definition means that the processing of such data needs to be temporary. And it needs to be specifically linked to the purpose. And here, if it’s contact tracing and listing the proximity between people, then you need that as long as there is a possibility of infection. So it’s 2 weeks, basically, as research shows. You should not keep the data for longer. So this is something that comes out very clearly. In the positions of human rights defenders and privacy defenders and rule of law defenders is that such exceptional mechanisms can only function with the limited period of time that needs to be defined from the start of the system. And you need to have a deletion of all data once you’ve reached that term.

Speaker B: But Sophie, somebody will say Hang on a moment, we want to know what to do in the event of there being another pandemic, because it has been suggested that this is only going to be the first, and we need to know what, what to do very quickly. And so just to be able to analyze the data for the duration of the pandemic, it might not give us the best opportunity to deal with that future possibility.

Speaker A: But then don’t present the tool as a contact tracing tool, then present it as a monitoring tool of the pandemic and the spread of the infection. But then you enter into much more difficulties, I would say, because the way the contact tracing app has been, has been designed as the least intrusive is really because it’s focusing on a specific, specific objective, which is to indicate the proximity between persons. If now you want to use this data to do something else, that’s a completely different story. And I think you take the risk of completely endangering and making the wider public lose complete trust in those tools, if they had trust to start with.

Speaker B: Sophie Kwasny. If, like her, you live in France, you can call a free hotline number and a specially trained chatbot will ask you about your symptoms and your location and tell you whether you need to be tested for COVID and, if so, how to get the test. It’s called AlloCovid. France’s Assemblée nationale has just passed new emergency regulations. They include collecting personal health data without consent if necessary, and storing it for up to 3 months. As with all of these measures, the response to the pandemic is fast-moving and anything can happen. My sources tell me the UK government’s considering a wide range of very controversial options. They’ve been discussing using GPS data from mobile phone companies to track people’s movements and examined whether they can seize that data under the Data Retention and Investigatory Powers Act. The conclusion was that for the government to do so, they would have to redraft the bill, but the option is still there, even though the move would provoke a furious response from privacy campaigners. Professor Tim Spector of King’s College London launched the COVID-19 symptom tracker app on March 23rd— with some 3 million people using it in Scotland, Wales, and the United States since then. However, NHS England refused to endorse the tracker and instead is rolling out its own smartphone application. On the Isle of Wight— We all need to play our part in the next stage of the fight against coronavirus.

Speaker A: The more people who download and use the NHS COVID-19 app— The more lives can be saved. Go to COVID19.nhs.uk to download the app now.

Speaker B: On the Isle of Wight, the scientist leading the app’s development admitted that it cannot run on many phones. Dr. Garant-Lewis said the app will only work with phones which run Bluetooth Low Energy and have at least Apple’s iOS 11 or Google’s Android 8 software. Software installed. This will exclude older phones and the newest Huawei devices running HarmonyOS. Lewis was speaking on BBC Radio Solent after some listeners contacted the radio station to say that the app was not working on their devices. Meanwhile, in Estonia, Europe’s most digitally advanced nation, e-health is a way of life and the chatbot will answer your COVID questions. Hi, I’m Suuv. I am a robot and I’m here to help you find information about changes in state services during the emergency situation in Estonia. Before we start chatting, please read here how I will handle the information you are sharing with me.

Speaker A: Hi, can I travel to Estonia from Great Britain at this time?

Speaker F: At this time, Estonia is only letting in Estonian citizens or residents of Estonia with a residence permit or right of residence, including holders of the so-called grey passport (stateless persons). The temporary restriction on the crossing of the border shall remain in force until the order is amended. Its necessity shall be assessed at least every 2 weeks. Other persons are denied entry to Estonia and are returned from the border. Starting from 17 March, everyone arriving in Estonia will People have to stay in self-isolation in their place of stay for 14 days.

Speaker A: Do I have to wear a mask in Estonia?

Speaker B: There is a lack of masks in Estonia and it is hard to get them. People who still want to wear a mask can make their own cloth masks.

Speaker F: A self-made cloth mask somewhat reduces the risk of an infected person spreading the virus to others and partly also protects the wearer of the mask herself if she does not have the virus yet.

Speaker B: In some places, people willingly share their intimate health data. In Belgium, people have no problem giving their information. According to the head of Google in Belgium, Thierry Hertz, they regard donating data as akin to giving blood. Still, as I have said, there are doubts about this technological fix for England. Carly Kind is a director of the London-based Ada Lovelace Institute. Institute, which organized a rapid reaction conference to investigate the apps and the AI that is being used against coronavirus. They called it Exit Through the App Store. After hearing the experts, Carly Kind is skeptical about whether any of the apps will work.

Speaker D: I think it’s a really, it’s a really good question whether or not it should be mandatory. I think there’s some strong arguments in favor of it being mandatory. So for example, as I said, it only works if a large percentage of the population actually use it. So therefore, there may be a reason why the government would make it compulsory to use it. On the other hand, we don’t actually have the evidence yet that it works at all. And in the meantime, it raises a number of risks. Some of them are privacy related. Some of them are around exclusion or particular groups of people being left out of the app. So if they’re going to make it mandatory, it would have to work perfectly. And we’d have to see the evidence of that. And there’d have to be a lot of transparency around it. And then there’d have to be a lot of protection. So we don’t yet have a piece of legislation that governs the use of this app in the UK. Other countries like Australia have adopted or in the process of adopting a law to govern the use of the app. It says things like employers can’t make their employees download the app as a condition of returning to work, that type of thing. So if we were to head down the mandatory route, I think there’s a lot of steps remaining in order to make sure that it would be done in the most kind of lawful and protective way possible. But I don’t disagree with you that maybe from an effectiveness point of view, maybe a mandatory approach would be better. It just would have to be balanced against other concerns and safeguards.

Speaker C: Isn’t there more—

Speaker B: this is making policy on the hoof, of course it is, because it’s a pandemic, we’ve never had this before, and so what we’re trying to do is to roll out an app in seconds flat. Yeah. Surely because we’re learning from this situation as we go, we’ll have to learn about the rolling out of the app as we go.

Speaker D: Yeah, I think that’s right. We are absolutely making policy on the hoof here. Some of that expediency is warranted by the situation, but in the case of digital contact tracing app, it’s not going to be an overnight fix. It’s not going to solve things so that we can all go back to work next week. What we’re talking about here is a technical solution which might help keep the pandemic under control for for the length of time it requires to develop a vaccine. So there we’re talking 18 months, 2 years this app will be needed for potentially if it’s useful, if it’s effective. So rushing things out in a number of weeks doesn’t really benefit us much. And in fact, it might actually have some disadvantages. In particular, if it’s bungled in the roll-up, then people might kind of scorn it for good and say, well, I tried once, it didn’t download very well, or I downloaded it, I got a bunch of notifications when I hadn’t even left the house. I’m not gonna give that another shot. And while the NHS may be improving the app as we go, if people kind of get burned by their first experience with it, we might actually ruin the long-term ability of the app to be a useful tool for the government. So I think there’s some arguments there in favor of a kind of moderate approach and not rushing into it until those things have been thought through.

Speaker B: There is one argument, one school of thought that says this is a unique opportunity for the government to get trust with the population for technology, the technology that it wants to use in the future. Yeah, they have to get this right.

Speaker D: Yeah, I think that’s it. That’s another really good point, actually, which is this type of relationship between NHSX and the British population, i.e., using an app to prompt certain behavior or to to enable instantaneous reporting of health conditions. We can imagine all sorts of ways that that might be useful in the future. So this is a really good chance to, to kind of gain the trust of people in that type of digital interaction with the National Health Service. Again, if it’s done badly, is it just going to ruin this, ruin an opportunity in the long term? Are people going to be burned and that’ll kind of set the whole programme of activity back by a number of years? And I think that’s another really strong reason why going about this the right way with a lot of transparency is worth it for the NHS in the long term.

Speaker B: The NHS has not had a good track record of developing technology. Just about every single project that it’s worked on has failed for years.

Speaker A: And there’s a—

Speaker D: this app happens against the backdrop of that history, and including recent problems like the Google DeepMind and Royal Free Hospital kind of data sharing agreement, which again really had kind of ruined people’s trust for data sharing between the NHS and private companies, even though there may be some benefits to be gained from those. So really it is about taking a slow approach in order to rebuild and keep that trust.

Speaker G: Okay.

Speaker B: How about though, it doesn’t look good at the moment that there are two competing systems. There’s one in Scotland which Dr. Spector has been developing and which the NHS rejected, and there’s another one in England. I mean, if you’re running almost competing systems. That doesn’t look very good, does it? It doesn’t look as if you’re actually going to be really data sharing in the interests of the UK as a whole.

Speaker D: No, that’s right. I think there’s a few things in what you’ve just said. So there’s the NHSX app, and then there’s different approaches or different concerns being raised by the devolved nations on the one hand, and then we also have this kind of Apple and Google system that’s being developed, and that’s also also competing in inverted commas against the NHS app. And certainly the messages we’re hearing from the government is that they have a very clear idea about what they want from this app, and they’re going to push through that through NHSX. And they think that, that the kind of value of that will be demonstrated such that everybody will come around to their approach. I think kind of in principle, that’s a nice idea that they could design the perfect app and then everybody else will get on board. In practice, I think we’re seeing there’s a range of reasons why that might not ultimately be the case. And the Financial Times reported over the weekend that they are actually developing a second backup app in case their app doesn’t really kind of pass muster in the next few weeks. Some of the reasons why that might happen are around the functionality. So some people are saying that in the Isle of Wight trials, we’re seeing the app doesn’t work as well on iPhones, for example. Some of it is around privacy concerns, so the Apple and Google app having a better, more decentralized system, which is better for privacy privacy, although maybe not better from a public health point of view. And then just kind of differing thoughts on how these types of technologies should be rolled out and used in, in the different nations. So I think it’s not clear that the NHS app is going to be kind of rise to the top of that yet. And that in the process, again, I think we’re actually going to see as these kind of debates unroll in public, it feels like people might lose patience. Or lose interest in that app in the meantime. So I think we need to— or the NHS needs to be quite careful about how they navigate the next few weeks.

Speaker B: What’s more, I’ve been talking to government insiders who themselves admit that the source code underlying the COVID app is badly written and the data drawn from the apps has too much noise, which means in technological speak that the data they produce is not clear enough to give meaningful results for telling people to get tested, stay at home, or go to work as normal. That’s one of the reasons why the government’s considering drawing on our GPS data. Bluetooth is simply not accurate enough. The problem is because the phone itself generates the Bluetooth radio wave, and this can be variable in distance, can cause false positives, i.e., record a contact when one wasn’t there, The app also has to send details of contacts back to a central database, and it can only do this by using the mobile phone, and this risks exposing the phone number. According to former government insiders intimately involved with the NHS and the development of ID systems, the phone number can be forced out of the data and the phone number’s owner can be identified by fuzzy matching it with data from other sources. And on top of these doubts about anonymisation and technical quality, there are also concerns about privacy, inclusion, and data retention. Luciano Floridi is the Oxford Professor of Philosophy and Ethics and advises the European Union on apps.

Speaker C: We can go through a whole list. There’s a challenge of privacy, there is a challenge of trust, and I think there’s a challenge of social justice. The one about privacy is obvious, has been all over the place. Which data are collected? How are they minimized? And what risks are involved in the collection of data that inevitably might be personal? Now I think that the technology in place is good enough to tackle the privacy challenge. And I say good enough. That’s also true if you look at the several data authorities around Europe. I didn’t mean European Union, Europe at large. Sense that with the technology, the minimization of data privacy is something that we can protect and is paramount. So the next question becomes a matter of trust. Whom do you trust with this technology, which of course is crucial? And the final point of our conversation, which actually is closer to my heart, if I may say so, is the one about social justice. Who can use it? What skills are required? What What kind of, shall we say, support is presupposed to be there? What kind of phone? What kind of expertise? Do I know how to turn on and off a Bluetooth, etc.? And we must remember, across all the world, across the UK, across Italy, across Europe, there is a digital divide. And there is a whole segment of the population that might find this challenging. Now, social justice, if you look to, potential, and I haven’t seen this happening yet, so I’m talking in a sort of speculative way, of attaching incentives that the app is more widely spread, the uptake is higher. How do you incentivize people? You make it convenient. You give more to those who have it. Well, you can see that the social justice is a very big issue, and I’ve been saying this loudly in Italy as well. Where there’s a big chunk of the population, 25%, is not online on the internet. And so those who have the app, have the right sort of hardware, the right software, the right skills, are, as it were, on the good side of the digital divide. They also get the advantage of being able to work, socialize, move around, et cetera. And those who don’t, they have the extra disadvantage. I think that we should address this quite clearly. It’s a problem. I mean, there isn’t an easy solution. But I’ve suggested already in several contexts that whatever incentives you have, and I close here, should be external to the use of the app. I might give you an example if you like. It’s a bit of a joke, so, but I hope it gives you a sense of what I mean. I have the app, I can use it, I can show it. It does not mean that I have, you know, first-class citizenship, but maybe I win a lottery ticket. Once a week, it’s totally detached from my social interactions, my ability to work, to see your children, et cetera, but it does provide an incentive. So maybe I put on the app because I also have X amount of lottery ticket, whatever, and that kind of external incentive, not endogenous, or something similar to that would qualify as a good incentive, but it wouldn’t transform that into an exacerbation of the digital divide, if you see what I mean. So I think we need to be a little bit smart in terms of how the nudging might be working here.

Speaker B: So, I mean, it’s a good point, isn’t it? And it’s a point a lot of people have made. And also that currently it would appear that the app only works on more modern mobile phones. Which of course means that you have to be able to buy a smartphone, which you perhaps will not be able to because elderly people quite often are constrained by their budgets.

Speaker C: Or the skills as well. Let’s put that on the table as well. I mean, I might have a mobile phone that is— I mean, I can speak for myself, like my mobile phone does more things than I can possibly imagine. And every now and then I just go, oh, you ‘Oh, amazing.’ So I know plenty of people who have a Bluetooth— So imagine the Bluetooth on your mobile phone stops working and you need to, say, turn on and off the Bluetooth. Do you know where to look for? And it might be trivial. I mean, honestly, it is trivial for people who, like, say, you and I are totally used to it. But it can be challenging for other people.

Speaker B: There is a certain amount of technological blindness here, isn’t there? Because we’re used to having these mobile phones in front of our faces all of the time, we seem to think the solution must be on the mobile phone.

Speaker C: I’ve seen very good points made around in the UK, in Italy, countries at the forefront of the tragedy, where the app, when it’s presented properly, is always as a minor part of a wider strategy. It’s always about, no, this is, if you like, the starter of a full meal. Don’t even think for a moment that is the thing. But I’ve also seen, unfortunately, simplifications, rhetoric of the app is the savior. That is insane. There is no chance that this technology is going to put us on a safe foot, full stop, by itself. It’s not a panacea, it’s not a savior. Can it help? Done properly, If, if, if, if the uptake is very high, if the technology is robust, if the minimization of data is there, if the trust is there, if the privacy is protected, quite a bit of number of ifs, can be done. Yes, absolutely. That’s why I’m committed to make it happen. If all that happens, then you can make a difference. How big?

Speaker B: Small.

Speaker C: Not insignificant, small. And in this battle, we need all the weapons we can. So I’m on board. But please, you know, if you have this widespread to other people, the message must be it’s a component of a much bigger strategy where the real bits are medical and social. That’s where we really win the battle. This is a support.

Speaker B: Professor Luciano Floridi, you’re listening to Password with me, Peter Warren, on Resonance FM. And after this, you can hear DJ Ritu with A World in London. In the world of J.R.R. Tolkien’s Lord of the Rings, there is a magical spherical stone that allows the person gazing into its swirling colours to see events happening far away, and it exerts a powerful fascination over that person so that it becomes impossible to live without it Or not to have your gaze drawn to it. What’s it about, the Seven Stars and Seven Stones? About the palantíri of the kings of old, said Gandalf. And what are they? The name meant ‘that which looks far away.’ The Orthanc Stone was one. Then it was not Made— not made— Pippin hesitated— by the Enemy? No, nor by Saruman. It is beyond his art, and beyond Sauron’s too. The palantíri come from beyond Westernesse, from Eldamar. The Noldor made them. Fëanor himself maybe wrought them in days so long ago that the time cannot be measured in years. But there is nothing that Sauron cannot turn to evil uses. Alas for Saruman! It was his downfall, as I now perceive. Perilous to us all are the devices of an art deeper than we possess ourselves. And what on earth, or Middle-earth, has a palantír got to do with tracing apps for COVID-19? Well, Palantir is the name of a multi-billion-dollar American company based in Silicon Valley. It’s taking care of the data that will be collected by the NHS England COVID-19 app. Palantir was mentioned by whistleblower Christopher Wylie when he gave his evidence to the House of Commons Select Committee on the 27th of March 2018, about the Cambridge Analytica scandal. That was when millions of people’s personal information taken from Facebook surveys was used to target political advertising. The company denied any formal involvement but admitted that one of its employees had had a working relationship with Cambridge Analytica. According to our sources, Palantir’s expertise lies in being able to force patterns out of data. And that includes de-anonymising it. We asked Palantir’s Paula Kift, head of privacy, to comment on its role in the COVID apps, but received no reply. So here’s what the company says on its website: Palantir Technologies Inc.

Speaker A: and our subsidiaries are committed to complying with data protection legislation, including the data protection regime in introduced by the General Data Protection Regulation (EU Regulation) 2016/679.

Speaker B: We process some personal data in order to run our business.

Speaker A: There are a limited number of circumstances where we may share your information with third parties, for example, pursuant to a court order, if we’re part of a merger, or with our business partners and service providers who support our business or collaborate with us.

Speaker B: We take the security of your information seriously and take steps to keep your information secure.

Speaker A: Please read the full privacy and security statement below, and you can always contact us if you have questions.

Speaker B: That has a hollow ring for Jim Killock, director of the Open Rights Group, which is campaigning for greater government transparency about its response to the coronavirus pandemic.

Speaker E: Pandemic. They are a company that nobody trusts because they are a surveillance company first and foremost. And their reputation globally has been made through WikiLeaks revealing their activities, identifying journalists and others to security services. I mean, this is a pretty poor start, right? If that’s how you treat privacy, if that’s what you think privacy means, removing it so that people can be got at by the state for reasons that are less than legitimate, then that doesn’t really give you a head start for handling mass data in analytic systems that’s health data and other things. You know, not least you have the implication of could this be reused elsewhere? Might it be identified for purposes that are less than ethical? And government promised at the beginning of all of this when it said Palantir is being signed up that it wouldn’t be handling personal data. Now, the 1st of April or thereabouts, the government released a notice, a data processing notice, saying that Palantir was now going to be able to process anonymized data. So a question is then begged: what data exactly is that that’s being processed? What personal data is this that is anonymized? How Is that anonymized, and how is that processed and handled by Palantir? I have no information about that. I don’t know how that’s handled, what that data is at all. I couldn’t say whether that relates to the NHS app, but these are things that ought to be immediately clarified because they’re handling personal data. Anonymization is usually something quite suspect that just means not currently identified, not currently mapped to individuals, could be in the future. That tends to be what anonymized datasets mean, and it depends on how rich that dataset is on how easy it is to re-identify it to individuals. And then finally, it depends on the motivation of the person with the data in their hands and the data governance around that, whether that ever does get mapped back to individuals.

Speaker B: It has been suggested that the issue with much of this is that you start to adopt these technologies or you use them under these circumstances and they continue being used afterwards and that this process doesn’t get reversed. You could arguably say, well, if it was a good thing, then it’s a good thing to keep on being able to analyze the data to get that value from it. Do you know, do we want to go back to the times of the 20th century? Century?

Speaker E: I mean, the 20th century had a few problems. I think we have to learn from the past, right? And you don’t just grant people huge powers on the basis of emergencies or difficult problems, thinking that that’s the solution. Because what happens is the means are the ends. And if you create a society without privacy and without trust of individuals, then you end up with a very paranoid sort of society to live in. So you have to look a little bit to the long term and just think, well, you know, what do we want society to look like? I think this is a very challenging time, but I don’t think the thing that is going to really appear on the other side of this is that the problem was data and privacy. I think that what people say afterwards is, you know what, the problem was being underprepared, not thinking about these problems globally, not cooperating globally, and running social systems which were full of vulnerabilities and were not able to cope with crises and were actually had to be re-engineered even to be scaled up, you know. And here we’re talking about everything from the NHS through the social care system through the benefits system through the tax system. All of these things have have turned out to be deeply problematic. But I think actually it’s more about the social relationships and how we treat each other and how we work together as a society. That’s going to be much, much more in people’s minds, I think, at the other side of this than, you know, questions of whether we have too much data privacy.

Speaker B: But to be fair to Palantir, it’s not always easy to balance privacy and public health. There’s a whole other debate, which I’m not going to explore here, about vaccination. The public health argument says that it’s everyone’s duty to get vaccinated because only then will deadly diseases be eradicated, or at least, if they’re not eradicated, then weak or old people’s lives will be saved by herd immunity. It’s for the greater good, an ethical solution. The ethical questions around COVID apps are more complex, as Dr. Jonathan Cave of Warwick University, who’s been advising those working on the NHS app on behalf of the Turing Institute, has been telling me.

Speaker G: I don’t think it’s entirely privacy-driven. There are many other concerns. So for example, if we had a contact tracing app connected to a sort of South Korean style, let’s say, more urgent, or German style, more urgent form of enforced self-isolation, then I could certainly make use of that to inflict harm on you, maybe even to my commercial or otherwise advantage, by causing the system to regard you as somebody who needed to lock themselves away in their house. And so if I install an app on my phone which exposes me to the risk that through error or malevolent behavior I’m locked down in my house when otherwise I could be free to go about my business, I might therefore not adopt that app or take measures to frustrate it. So in that case, it’s not the data privacy with which we would be concerned, but our actual ability to go about our lives. Now, I don’t think that’s entirely a privacy thing. Another element is that the SIM card data are not held by the government. The government doesn’t provide our SIMs. The SIMs in our cars, in our phones, and so on, which— and the government is using all sorts of locational data. For example, those graphs that they show about how many people have driven on which roads and so on. Those are based on the fact that all the cars have SIM cards in them too, and ANPRS and other things. But there are commercial concerns and security concerns as well as privacy concerns. So I agree that there needs to be a more profound ethical commercial policy discussion around these issues. I don’t think, however, that it is a trade-off between privacy and other rights any more than the trade-off between managing the epidemic and the economic consequences is a straightforward monotone trade-off. I think there are intricate connections there, and many of them are behaviorally linked. That I think in countries where there’s a more coercive form of control, You don’t really need to tell people much except get in your house and don’t do anything or install this software. So in Xinjiang in China, where they were before the virus ever came up, what they wanted to do is to control the Uyghur population. And so they forced everybody to install in their cars GPS software that tracks them and reported to a central data system. So they could understand the movements of people there. So these are not isolated fears. They’re not, I think, a single issue. But I do agree that particularly— and it’s not just if this becomes a permanent fact of life. When we begin to unlock from this thing, the manner of our unlocking will depend on our ability to have in place things that provide people assurance. So if Boris Johnson tells us that we can all go back to work, most of us won’t. And when we do restart the economy, many businesses and many jobs that have been on life support will actually disappear. And there will be a need for substantial structural change, which has huge ethical consequences. This is all tied to the collection and use of this information. Because if they decide that certain areas, for example, certain groups of people— I see that both you and I are gray in the head. If they decide that we have to be locked away for our own protection, that will have a consequence. But I agree that at the moment, the urgency of just feeling we’ve got to manage this somehow has led us into policies which, in the case of Things like the furloughing program or the, the business interruption loans program have enormous costs, and we haven’t even begun to think about how we’re going to deal with those. The same is true with the collection of this information or the change in people’s attitudes to make them accept that as a matter of public duty, information which they had regarded as private is no longer private. I mean, that’s a kind of rewriting of the social contract, and it will not affect everybody equally, and there will be groups, as we see in America, right? I mean, there are certainly people in America who have suddenly discovered that the government’s ability to pursue public health policies restricts some important First or Second Amendment right that all of a sudden they think is much more important. Than the right to live, or in particular, somebody else’s right to live.

Speaker B: When you say it rewrites the social contract, the social contract is a philosophical idea that there is no actual contract between me and the government.

Speaker G: No, but we don’t have a constitution in this country either, and nonetheless it is real. So I think that in that sense, when the government, or when businesses collect information about us. And so if, for example, I have a loyalty card from a large supermarket, that’s collecting all kinds of information about me. There are certain limits, crash barriers if you like, that are placed on that, but they tend to be limits on the data themselves. The many ways in which these data can be used could either infringe my privacy if I’m targeted directly or could allow the people who collect those data to gain value which in some sense might rightfully be mine. Now the law addresses the most serious, what it regards as the most serious or urgent aspects of this, but leaves the other to sort of social interaction and convention. And as things change, for example, as more and more people came on the internet and children came on and different types of content came on, certain things that were not illegal suddenly became matters of law or of duties of care. And that’s what the Online Harms Bill is really all about, because it’s not about the commission online of crimes that are already on the statute books. It’s about harmful actions that are not yet crimes. So in other words, they’re not part of the written contract, but they’re part of the penumbra of unwritten understandings and conventions, shared ethical values, or things we think are shared that have suddenly, through a change of technology— and the virus is a technology, I suppose— suddenly become salient and need to be nailed down a little bit more. Because polite social interaction, being English about the whole thing, manifestly doesn’t work.

Speaker B: And the last word must go to the man himself, the government’s own advisor on ethics, Roger Taylor, the head of the new Centre for Data Ethics and Innovation.

Speaker F: If you’re saying government should try and build trust by only working with companies that have not been criticized. There’s a sort of PR aspect to that. I can see what you’re talking about, but I’m more interested in actually what are the actual rules and what are the governance— because I do think if we have good governance and we have effective rules, then we don’t have to say, oh, we can’t work with this company because it might cause bad PR. That wouldn’t just be the consideration because you say, no, we have good governance. We know if we have a company come in and analyze some data, and we tell them that this is to be used in an anonymous fashion to do the following things. We know we can govern it. We’re not, I mean, there is obviously an issue around you want to work with fit and proper companies that have appropriate governance and appropriate levels of oversight. But I don’t think Palantir falls short of being a well-governed company in that score. It’s purely a PR issue. And it does concern me if trust becomes a PR issue rather than an issue around the fundamentals of trustworthy governance, and that is what we should be focusing on.

Speaker B: I agree completely. However, you know, I mean, you know as a journalist that trust and PR are quite often indissolubly linked. Have you been consulted over the NHS app?

Speaker F: So just to say where we are on that, the CDEI was asked early on by NHSX to So we did two things when this happened. The first thing we sort of said was, if anyone, do people need help? And we seconded quite some people over to NHSX to a range of different jobs, not just on the app, but just over the place. The second thing we then did was also, we were asked for advice on ethical governance, and we said, you should, you know, you want to put in place some ethical governance, we’d suggest creating a separate ethical board, board to review the situation, that the ethics board should kind of publish its advice to government, and then government can decide. But obviously it’s going to be government’s decision what it does. And that structure has now been put in place. An ethics board has been appointed, it’s been announced. Jonathan Montgomery is chairing it, and that board has met a few times. I sit on that board. We have had set out a sort of initial parameters that we think the NHSX should be working within and considering, and what we think the key questions are that needs to be addressed to ensure that this is done in a way that is ethical, and that it, as it were, considers both what is the most effective response to the virus against and alongside how it is— privacy is protected, how transparency is ensured so that the people understand people understand how, what is going on, and how there is appropriate accountability and governance for decisions about the app, not least because there will be a, you know, this will develop over time, so there’s a real issue about encouraging people to download an app, and then being clear that what are the rules if something then changes about how the app operates, if a new feature is added, or some aspect of the mechanism changes, how is that governed, who makes that decision, and How are people informed and are able to decide whether they wish to continue to, to use the app? So we’ve set out the, the issues. Those are currently with NHSX, and that is an ongoing process.

Speaker B: So yes, we, we’ve been, we have been consulted. And so you’ve been consulted, and now you’re waiting for them to come back to you and say how they want your future role to be, is that?

Speaker F: Oh no, no, sorry. So there’s now an ongoing discussion between the development of the app and the ethics advisory board about the design and operation of the app.

Speaker B: In Belgium, the Belgians have for a while been encouraged to give data like blood. That’s obviously the sort of situation that you want to get to, isn’t it?

Speaker F: Well, I think the blood analogy is interesting in the sense that I think—

Speaker B: Oh, and also, just to interrupt, Luciano Floridi told me that he was going to give his data up when he died.

Speaker F: Yes, I mean, I like these ideas. They’re interesting. Particularly the notion that, as it were, because there’s a lot of discussion about whether it’s a good idea to pay people, as it were, that they should get a financial return on their data, or whether they shouldn’t. And particularly in areas like issues around public health, I think there was a very similar debate about blood donation early on, and I think, I can’t remember the name of the economist who pointed out that actually you’re, then there’s been sort of various behavioural science since then, but this is a lot, way before that became fashionable, pointing out that if you pay people people for it, it will quite likely reduce the supply because people won’t feel it’s something they’re obliged to do, it’s something that they— and the value of it will be quite low. So he points out actually making it something that people volunteer is probably a better way of encouraging people to donate blood. And there is, I think, I think that analogy does have some real pertinence to the— particularly the use of data by public sector organisations for issues of societal benefit, of general public benefit. I do think that is an interesting way to think about it. I think there’s an element of it though that goes beyond the donation element, and this goes to elements where people’s rights are potentially infringed by not having access to data about other people. And so a clear example of this would be algorithmic bias. So if you feel that an algorithm is systematically biased against a particular group of people. So, for example, let’s just imagine— I don’t think this is happening, I’m just making this up to be really clear, right? But, you know, so people worry about, for example, there’s been lots of evidence that credit algorithms in the US have been biased against Black and ethnic minority applicants. Now, if you imagine a situation where there was a sense that ICU beds, the algorithm that was determining you know, the way our decisions were made about allocation of ICU beds was ethnically biased. You know, I think people have a right— if I’m from an ethnic minority, I think I have a right to know whether that is happening and to challenge it. But for me to be able to know whether that is happening, I have to know how that algorithm has been applied not just to me, but how it’s been applied to everybody else as well. So I need to know the information about how other decisions about other people were made for me to be able to exercise my rights to not be discriminated against. And so that is a scenario in which it’s perhaps not going to be enough to simply ask people to donate data. It’s where people have— other people have a right to know what decisions were made about me. Not about me individually or personally, as it were, only in an anonymous way, but they do have a right to be able to know how that decision system operated across other people, to know whether or not they are being treated treated fairly. And that’s quite an important principle that we need to, we need to be very clear about when we talk about these issues.

Speaker B: I mean, one of the things about this is a lot of people, to go to something that you mentioned earlier, have said, hang on, we should suspend privacy for the duration of a pandemic that’s threatening the lives of a lot of people. That really, to supply information should be mandatory, that it seems to have worked It seems to have worked in South Korea. Is that being considered? Has it been considered that, for example, that you could get a lot of data from the mobile phones of people and that you should use the GPS systems rather than this Bluetooth thing, which seems a bit fuzzy?

Speaker F: So just to be clear on this, I mean, they’re under the existing laws about communicable diseases. At an individual level, somebody can be required to sort of disclose information in order to protect other people. So if somebody’s infectious— I don’t think this does not apply to COVID at the moment, but there are legal structures that can, as it were, require people to do that. That’s not where we are at the moment with COVID and I don’t think that that is a situation we’re going to get into, no.

Speaker B: We did ask the Department of Health and the Information Commissioner’s Office for an interview in order to provide the transparency and clarity that would reassure the public, but no one came back to us. Of course, we must all make up our own minds about tracing, tracking, and testing apps and weigh up whether we mind losing some of our data privacy in order to prevent the spread of infection and save lives. One thing is clear, and that is the apps will only be effective if they work, technologically speaking, and if people trust them enough to use them. As Gandalf warned in Lord of the Rings, “Perilous to us all are the devices of an art deeper than we possess ourselves.” In the meantime, I’m sticking to the tried and tested ways of staying healthy in the corona pandemic. I’m washing my hands with some priceless hand sanitiser I got from some elves on the darknet and wearing a mask. That way, I hope to be back next month with another edition of Password, researched by Future Intelligence, produced by Blue Buffery, and written by Jane Wyatt. Thanks for listening.

Speaker A: Goodbye. This program has been brought to you by Resonance FM. If you like what you heard, please support our work by making a donation at resonancefm.com/donate.