PassW0rd – 9th May 2018

Speaker A: This program is brought to you by Resonance 104.4 FM.

Speaker B: If you like what you hear and want to support our work, please make a donation at fundraiser.resonance.fm.

Speaker C: Hello and welcome to Password on Resonance FM with me, Jane Wyatt, and me, Peter Wyatt. In the next hour, we’re exploring the latest in-depth research commissioned by the House of Lords, the UK’s upper house of parliament. It aims to lay down the rules that will create the right conditions for Britain to benefit from the artificial intelligence revolution and even become potentially a world leader in AI.

Speaker D: I was at the House of Lords for the launch of the report by the AI Select Committee. It’s chaired by Lord Clement-Jones, the Liberal Democrat peer. So, okay, you’ve released this report on AI. Um, AI is a bit of a nebulous subject, isn’t it? What are your big concerns about AI?

Speaker A: Well, our big concerns are about the risk that public trust will be damaged if we don’t get the ethical framework right. There are huge opportunities across the UK. You know, we’ve got a fabulous research community, we’ve got a really good startup culture, but if we don’t get the ethics right, we don’t basically make sure the public trust AI in the form of algorithms, machine learning, the use of their data, that they’re not fully educated in terms of digital understanding, then we’re going to have a problem in the future.

Speaker D: Okay, so we have this issue at the moment with Cambridge Analytica. How is it that you think that the government should head off this use of AI, and how also should they educate people about it?

Speaker A: Well, we think that they should adopt this ethical framework so that there’s a very clear understanding of the purposes of research, that AI is beneficial for people and mankind, and that people are fully educated in the use of AI, that it is transparent and explainable. So there are certain principles that definitely apply. And then of course it’s very important that our regulators in different sectors have the powers that they need. Classic example with Cambridge Analytica was the fact that the Information Commissioner waited 5 days before she was able to examine the books, so to speak, at Cambridge Analytica. Well, we need to give her more powers, quite frankly. And, you know, that’s the sort of power that the Competition and Markets Authority have. They can do those sorts of dawn raids, and the Information Commissioner should be able to do that because our personal data is very, very important.

Speaker D: One of the very interesting points that was made here was that Professor Cave said that there shouldn’t be any nation competitiveness in all of this. However, That’s a bit inevitable, isn’t it? Two weeks ago, President Macron, in an interview for Wired, said that France was aiming to become the major AI power in the world. How are we going to stop that?

Speaker A: Well, I think you can square that circle. Of course we’re going to be competitive in terms of providing the right environment for AI developers, providing the right environment for AI researchers at our terrific universities, and so on. But what we mustn’t be competitive about is a race to the bottom in ethical terms. That’s what we mustn’t compete over. We need to have a common set of principles by which AI is developed, um, so that in a sense what we’ve got is a common understanding of the purposes of AI and the transparency and explainability whichever nation develops it.

Speaker D: To achieve that, you’re going to need the cooperation of the technology companies though, aren’t you? Because A lot of people will say, oh well, if there’s regulations, then we’ll just go to another country and we’ll avoid them. But so you’ve actually got to get the technology companies to buy into these ideas too.

Speaker A: Yes, I agree with that. But of course, technology companies decide where they base themselves, and the UK is a fantastic place for them to base themselves. So, you know, they’re going to want to comply with the sort of ethics that we establish. I’m quite sure of that. Many of them are already developing their own ethical codes through the AI partnership. You know, there are many, many good elements that are happening within the tech majors. We do make the point about data monopolies, of course, and many others have made that point, and we’ve just got to make sure that they don’t abuse a dominant position over their use of data.

Speaker D: I suppose the only— the other fascinating point about this is technology is already making some new jobs, isn’t it? These are jobs for ethnicists, and there are jobs for philosophers.

Speaker A: And for linguists and so on. And funny enough, when I come across undergraduates who are doing non-scientific subjects, I say, look, you too can be involved in all this new technology in AI, because AI essentially will rely more and more on creative skills and humanities skills, so to speak. And that is going to be extremely important.

Speaker E: Yes.

Speaker A: So we’re not just talking about computer science, machine learning skills and so on.

Speaker C: Lord Clement-Jones. We’ve had him in the Back of Our Passwords studio before, haven’t we, Pete? That was back in 2013 when we were working on our research that was published in the report called ‘Can We Make the Digital World Ethical?’ We presented it in Brussels and in Paris, and now it seems the British government is also taking note.

Speaker D: Well, the House of Lords has now caught up with our thinking and even produced a set of guidelines that have a lot in common with some of the recommendations that we’ve made. The report sets out some key principles. You could call them the Five Commandments. One, artificial intelligence should be developed for the common good and benefit of humanity. Two, artificial intelligence should operate on principles of intelligibility and fairness. 3. Artificial intelligence should not be used to diminish the data rights of individuals, families, or communities. 4. All citizens have the right to be educated to enable them to flourish mentally, emotionally, and economically alongside artificial intelligence. 5. The autonomous power to hurt, destroy, or deceive human beings should never be vested in artificial intelligence.

Speaker C: Well, those commandments sound all well and good, but what do they mean for our jobs, healthcare, and data privacy? What do they actually mean in practice?

Speaker D: Experts don’t all agree about that, but I’ve been talking and listening to some of the top brains in AI. These are researchers at the point where neuroscience and engineering meet, like Dr. Demi Hassabis, the Chief Executive Officer of Google DeepMind. He’s launched a lecture series for the Royal Society called You and AI.

Speaker C: Dr. Hassabis works for Google, right? So what is Google planning to do with artificial intelligence? I know it has an ethics board and its motto is “Don’t do evil,” so hopefully Dr. Hassabis sees AI as a force for good.

Speaker F: I think already the systems we have today can be usefully used in science. In fact, we’ve seen that by work we’ve done, some work we’ve done, and many other groups are using some of these systems I’ve already talked about, deep learning and reinforcement learning, in all sorts of very interesting scientific domains. So, it’s been used to discover new exoplanets by analyzing data from telescopes. AI systems have been used for controlling plasma in nuclear fusion reactors. We’ve been working on, and others, on how it can help with quantum chemistry problems., and also it’s been used a lot in healthcare domains. So actually, we have a partnership with Moorfields to help the radiographers quickly triage retinopathy scans, so scans of the retina to look for macular degeneration. So, it was very, very— 4 extremely diverse fields, and I could have, you know, done many slides on different applications that are currently going on with AI, and I think this is just the beginning. One of the things I’m most excited about is applying it to the problem of protein folding. So this is the, this is the problem of you get an amino acid sequence, 2D sequence of the protein structure, and you need to figure out the 3D structure the protein will eventually fold into. And that’s really key to a lot of disease and drug discovery because the 3D structure of the protein governs how it will behave. So this is a huge long-standing scientific challenge in biology, and we’re working quite hard with a project team on this, with some collaborators from the Crick. Other scientific applications I see coming up is helping with things like drug design, the design of new materials, and in biotechnology in areas like genomics. And in fact, if I was to boil down the kinds of problems, the properties of problems that are well suited to the AI we already have today, let alone what we’re going to create in the future, I think it comes down to 3 key properties. Property 1, it’s got to be a massive combinatorial search space. So, that’s kind of got to be inherent in the problem. Secondly, can you specify a clear objective function or metric to hill climb against, to optimize against? It’s almost like a score, if you like, a score in a game. You have to be able to have some kind of score of how well you’re doing towards the final goal. And then you either need lots of data to learn from, actual real data, or an accurate and efficient simulation or simulator, so you can generate a lot of data in the way that we do with our game systems. So, as long as you satisfy those three constraints, properties, I think the AI systems we already have today could potentially be usefully deployed in those areas, and I think there’s actually a lot of areas in science that already would fit these desired properties. And then, of course, there’s all sorts of applications to the real world that we’re working on in combination with Google, including with healthcare. We work with the NHS in many projects. You’re making the assistant on the phone more intelligent, and also in areas like education and personalized education. And I think AI is set to revolutionize a lot of these other sectors. So, just to sort of sum up now, you know, one of the reasons that I’ve spent my whole career on AI is that I’ve always felt that it’s a kind of meta-solution to many other problems that face us today. You know, if you think about how the world is today, one of the big challenges is the amount of information that we’re confronted with and that we’re producing as a society. So, and I mean that both in our personal lives in terms of like choosing, you know, our entertainment, to science, where there’s just so much data now being produced from something like CERN, or in genomics, you know, how do we make sense of it all? And indeed, the systems that we would like to understand better and have more control over are incredibly complex systems. You know, think about climate, or the nuclear fusion systems, you know, these incredibly complex systems that are, in some cases, are bordering on chaos systems, and so they’re very difficult for us to describe with equations and to understand, even the top human scientists and experts. So, you know, for a long time, big data was the buzzword, you know, before AI is now the buzzword. AI was the, you know, big data was the buzzword, and I think that actually, in a way, big data can be seen as the problem. You know, if you think about it from an industry point of view, All companies have tons of data now and talk about big data. The problem is how do they get the insights out of that data and how do they make use of all of that data to be useful to their customers and their clients and so on. And I think AI is the answer to help find the structure and insights in all of that unstructured data. And in fact, you can think of— I think one way to think about intelligence is as a process, an automatic process that converts unstructured information into useful actionable knowledge. I think AI could help us automate that process. For me, my personal dream and a lot of the dream of my team is to make AI-assisted science possible, or even perhaps create AI scientists that can work in tandem with their human expert counterparts.

Speaker D: Well, there’s a problem with this, according to Xavier Ruiz of the Open Rights Group, who gave evidence to the House of Lords along with a panel of privacy experts. I interviewed him, and he picked up on the third point that the Lords Select Committee was concerned about: data ownership. He was concerned that the health trusts and similar organizations working with AI companies like Google Seems to be a poor relation in this process at the moment.

Speaker H: Open data, but when it comes to artificial intelligence, it’s a bit unclear what an open artificial intelligence that has been, you know, supported by public funding, it would look like. So if we look at the negotiations between DeepMind, which is a Google subsidiary, and several NHS trusts, In general, the discussion that you see there is that the trusts are more or less perceived as having no bargaining power. They say, you know, look at us, we need help for medicine to advance. And this is something that people from, you know, doctors tell us, you know, we need help, we need Google to come and help us. So Google comes almost like a knight in shining armor. With their AI tools takes a lot of the data from the, from the NHS and comes up with solutions that can improve, for example, the diagnosis of eye disease or breast cancer screening, or the algorithms for the actual internal hospital algorithms, which are the decision-making processes for how to treat patients.

Speaker G: Now, the—

Speaker H: I mean, our understandings are actually that In that exchange, hospitals could deal with Google, or they could deal with IBM, or they could deal with a group of open source developers, or they could try to do something in-house. But the one thing that is absolutely necessary for any AI system there is the data. I mean, this is the data that is the, you know, the most critical element in developing these tools. So at the moment, what we are concerned is that we are seeing a situation where without pretty much any transparency or public procurement due process whatsoever. Hospitals are going to Google to ask for help. Google is coming and giving a service more or less for free, but they are obtaining insights and they are first to market and they are determining how IT systems, including how IT systems work in the NHS, because they need to convert data structures to work with their with their AI. So we think that this whole area basically is very, very problematic. I mean, public sector bodies, if they are going to contract AI, should follow a proper procurement process and they should follow government policies, you know, which at the moment are a bit in retreat, you know, but they, they’re still fairly widespread accepted, you know, around making software available and trying to ensure that any investment can be reused as much as possible.

Speaker A: And they shouldn’t.

Speaker H: So they should try to enforce a different exchange. And also, the idea, as we know now, I think if something is free, you really need to look behind the curtain. I think that it was okay for Google to come up 15 years ago, you know, and offer a free product, a free search engine. I think that is very, very different in 2018 for Google to come and offer a free AI service to hospitals, and because clearly It’s not free. Google is making money either today or planning to make money tomorrow. And this is not just in cloud Google. This is just to say that’s how business works. And I think that is completely naive to sign a contract for free in 2018 with a large internet company. So I mean, we think that definitely the role of the data should be— I mean, should be made more explicit. And if you actually look at some of those contracts, the ones that we’ve seen, they have clauses that say that any developed IP, any derived IP in the course of the contract, and they list copyright patents or whatever you name it, it will be the exclusive property of Google. So, and that is obviously really, really problematic because not even in a normal ICT contract you would probably get such a, you know, one-sided exchange, you know. And I think that because they’re doing it for free, you know, they feel that they can claim that. But obviously if the NHS putting the data is not really for free.

Speaker C: Isn’t that just scaremongering? Why should we be afraid of algorithms that were written by human beings?

Speaker D: There are dangers, and it’s not just me saying that. Back in 1970, Marvin Minsky of the Massachusetts Institute of Technology remarked that when robots ruled the world, If we are lucky, they’ll keep us as pets. Now, he may have been saying that slightly tongue-in-cheek and with a twinkle in his eye, but much more recently, and in fact 2 weeks before he died, Cambridge professor Stephen Hawking repeated his earlier warning that artificial intelligence already has the power to potentially achieve domination over humans. Professor John Harris is one of the founder directors of the International Association of Bioethics. He’s a British bioethicist and a philosopher. He’s the Lord Alliance Professor of Bioethics and the Director of the Institute for Science, Ethics and Innovation at the University of Manchester. And he told me that he believes that AI threatens humanity, so we need to restrain it, but we’ve gotta sort this out because we really need it.

Speaker J: The fears that are, I think, most, both most pressing in a way, though they’re some way off, pressing in the sense of important and most interesting are the possibilities of actually creating artificial intelligence. That is to say, machines that can think in ways either comparable to ourselves or indeed in superior ways to ourselves. Machines that can outthink us. Now, there are very good reasons which I could go into as to why we might— it might be in our interest to create such machines, but it is in those that will not just choose between a given set of alternatives, but actually decide what their own alternatives are, that are the danger and the interest.

Speaker G: Isn’t this the issue though with any technology? You’re an expert on bioethics. Isn’t there always going to be a risk with any research that anything that you do could really quite seriously upset a balance?

Speaker J: Oh yes, of course, and you have anything that can be used can be abused. And that, you know, applies from kitchen knives or the stone axe to vast computer systems or machinery or systems of transport, air travel and so forth. Of course that is true and we always have to make the calculation, not is it safe because nothing is safe, is it safe enough given the balance of benefit and burden that is likely to come with it.

Speaker G: So therefore, we need AI to help us prevent our world colliding with the sun or losing out to the sun. But the AI that we’re developing to help us do that might have other decisions about where it’s going.

Speaker J: And that is the classic dilemma. Yes. But in a sense, that’s not a new dilemma. We have that with every generation of our children. Our children may decide to run the world in ways that we had not foreseen that they would choose to do, or run our little segments of it. So this is not a new problem, but what we are doing is creating, for the sake of it solving some bigger problems for us, the sorts of creatures that might, because of their abilities to help us solve our big problems, might find that they have even bigger problems that they’re more interested in.

Speaker C: Well, machines are not above the law, are they? There are people who own them and people who program them. Can’t we just pass some new laws?

Speaker D: It’s not as easy as that. We’ve given them rights over us, often without thinking and without even knowing. Xavier Ruiz of the Open Rights Group says, we’ve been giving control of our destiny to machines for a while now. Those terms and conditions, those legally binding clauses we agree to when we tick the box, give away our privacy and our rights over our own data. While Hassabis says, this is one of the reasons we need AI, to help us through this ever-rising mountain of data. According to Ruiz though, privacy and the potential loss of it is something that we have to keep on top of.

Speaker H: There has been a debate about data for a long time. I mean, if you look at GDPR, this Data Protection Regulation coming into force this month, the preparation started many, many, many years ago before Cambridge Analytica was, I think, even created, possibly. You know, there were people not just in Brussels, in the UK, everywhere, I mean, who have been concerned about this. Certain policymakers have been trying to drive more control because they could see increasingly that the whole data, the data economy is being developed around the concept of surveillance and lack of, you know, accountability. So there, I mean, I think that, yeah, there are a few people who have started looking at this for a long time. I think what is quite hard is that until now it was quite— people like ourselves were saying that there were risks, but the risks were always seen as something that may happen tomorrow. They weren’t very clear. We were saying, look, you know, you don’t want to have companies knowing so much about people. And the response we got is, oh, I don’t care if Google knows about my emails because I have nothing to hide. And they’re saying what we were saying, said, well, it may not be important whether they know about your emails, but if they know about everyone’s emails, you know, that is a system systemic issue. And I think that now we’re starting to see some of the consequences of the systemic factors. And the very positive thing is that finally privacy is not seen as an individual value opposed to the collective values, which has been until now the way that it was described. For example, around the Snowden leaks 5 years ago now, you know, we were always told, oh, you know, but, you know, it’s, you know, we need to balance the privacy of the individuals versus the security of society as a whole. And now we are seeing now that what we need to balance is the, you know, accountability and security and democratic values of society as a whole versus the potential control, you know, that widespread data spying, you know, enables. So in that sense, I think it’s positive. And to be honest, it’s better late than never. I mean, I think that we are not going to complain too much now if If people start becoming privacy geeks today, you know, like, we won’t, I mean, I won’t complain about late conversions, to be honest, if they contribute to move the, to change the landscape that we are seeing.

Speaker D: So it seems to me that what we need is a new form of society that takes into account the role that artificial intelligence is going to play in the very near future. Asabis of Google DeepMind, has some ideas.

Speaker F: From a neuroscience point of view, one of my dreams as well is to try and better understand our own human minds. And I think building AI in this neuroscience-inspired way, and then, and then sort of comparing that construct, that algorithmic construct, with the way the human mind works, will potentially shed some light, I think, on some of the unique properties of our own minds. Things like creativity, dreaming, perhaps even the big question of consciousness. I think, you know, AI holds enormous promise for the future, and I think these are incredibly exciting times to sort of be alive and working in these fields. But, you know, this is— with all this sort of potential also comes a lot of responsibility. You know, we believe at DeepMind, as with all powerful technologies, AI must be built responsibly and safely and used for the benefit of everyone in society. And we’ve been thinking about that from the very beginning of DeepMind. And this requires lots of things that we’re actively engaged with right now with the wider community. Research on the impact of the technology, how to control this technology and deploy it. And we need a diversity of voices, both in the development and the use of the technology and meaningful public engagement. And we’ve just launched our own ethics and society team at DeepMind that’s involved in working with many outside stakeholders to figure out the best way to go about deploying and using these types of technologies to benefit everyone in society. And we’ve also been involved on industry scale across the whole field in co-founding the Partnership on AI, which is a cross-industry collaboration with for-profit and nonprofit companies, some of the biggest companies in the world coming together to talk about this and try and agree some best practices and some protocols around how to research this technology and how to engage the public with it.

Speaker D: According to Xavier Ruiz, we need data democracy, and for that We need widespread understanding of artificial intelligence. We need to educate people.

Speaker H: So the government is proposing to set up a new data ethics and innovation body, which is more or less, I mean, it has been discussed in terms of data ethics, artificial intelligence, and algorithms. You know, I think they’re bundling up this whole, all these new problem areas around IT. Into this new body. So there is an expectation that this new center or body will be providing that function. I think what we said to the committee was that when we were asked whether there should be a new regulator, you know, which is not exactly the same that’s being proposed in terms of the center, just to be clear, our response was that the most important thing was for many regulators to be AI literate. So the Information Commissioner, the Electoral Commissioner, Ofcom, you know, any regulator that is going to encounter AI. And this could even be like finance regulators, you know, who are now dealing with, um, you know, algorithmic trading, or retail regulators, you know, you’re going to regulate competition and, you know, all of a sudden now Tesco decides to go big on AI, which they already do quite a bit, you know. To understand, but they start becoming, crossing some lines. You know, they, you know, basically lots of, maybe not the whole of the population needs to be absolutely AI literate, but we think that, I think that many, many regulators and enforcement bodies need to become a lot more savvy around not just AI but technology in general.

Speaker C: Yeah. Well, how are we going to educate people about the rights and responsibilities of dealing with artificial intelligence?

Speaker D: There’s a Scottish inventor called Alexander Sandy Enoch. He’s built a robot that uses AI to teach people computer code. It’s called Marty, and it works like this. Professor John Harris also believes that, yes, AI can be used for the benefit of humanity.

Speaker J: We may need artificial intelligence of the sort that I’ve described— intelligence actually capable of thinking and deciding things for itself. Itself, to help us with scientific problems that we simply cannot solve. There was a wonderful remark by Martin Rees, who was president of the Royal Society and is an astrophysicist, who said, there may be problems that creatures with our evolutionary history can neither think about nor solve. Now, we may need machines to solve those, and one of those problems which is clearly there but maybe some way off, is the problem of the eventual demise of our planet. At some point, the sun will die, and with it, all possibility of life on this planet. Originally, Stephen Hawking calculated that that would be 7.6 billion years ahead, but not long before he died, he reran the calculations and said publicly that we might only have 1,000 more years before the Earth became uninhabitable. Now we need to start thinking and getting assistance in thinking about how our species is to continue, if it is to continue, beyond that sort of horizon. There’s no safe path here. It would be as dangerous to put a hold on all of this research as it is to pursue it. What we have to try to do, and it’s not always possible, but what we have to try to do is pursue it as safely and as cautiously as is warranted. But the difficulty is knowing how cautious we should be and how much time we’ve got to solve these problems.

Speaker G: So we’re quite literally between the devil and the deep red sun.

Speaker J: Well, yes, we are quite literally there, and it’s probable that it won’t happen in your lifetime or mine, but may be our grandchildren, and there is no point in preparing for a world which does not include our grandchildren or our great-grandchildren. And of course, if we have artificial intelligence, the grandchildren and the great-grandchildren of these intelligent machines, which will almost certainly become capable of replicating themselves and living alongside us.

Speaker G: Artificial intelligence should operate on principles of intelligibility and fairness. How on earth is that going to be possible if we’re going to build some system that is going to be more intelligent than us? How are we going to understand something that is more intelligent than us? Surely that’s pie in the sky.

Speaker J: Well, if and when these entities exist, yes, we probably will not be able to understand them, but no system of any description has been— has ever been built with a complete assurance of intelligibility and fairness. Do we have a fair system of government? Do we have a fair system of resource allocation? Do we have a fair system of allocation to scarce resources like the NHS? No, we don’t. We have to try to make it fairer. We shouldn’t just give up because it’s not fair. But the idea that we might make something, anything these days, completely intelligent to everybody, intelligible to everybody, and completely fair in access and so on is not possible. Now that’s a classic challenge and every government since there were governments has faced that challenge of how it satisfies the needs of the people and their desires and so forth.

Speaker G: Isn’t there an essential paradox here though? Because what we’re trying to do is to impose on an intelligence that we think will probably become more intelligent than us. Being intelligent when we’re actually quite stupid and quite inhuman. We seem to think that we’re going to lose our humanity to a system that is going to be more intelligent than us and simply point out to us that we’re inhuman.

Speaker J: Well, some aspects of our humanity we could probably well afford to lose. Indeed, we probably could well afford to make sure that we lose those aspects. But the other issue is that Artificial intelligence, machine intelligence, will not pass what I call the Shylock test. Shylock is Shakespeare’s famous character in The Merchant of Venice who said memorably, “We understand one another because,” and I quote, “if you prick us, do we not bleed? If you tickle us, do we not laugh?” And he ended that speech by saying, “And if you wrong us, shall we not revenge?” Now, our problem with artificial intelligence and their problem with us will be understanding creatures who are so differently constructed, so different, turned on by so many different sorts of things. How can we imagine creatures you can’t tickle and don’t bleed? How can we imagine what’s really good for them and how will they use their imagination to know what is really good for us and indeed to come to care about what is good for us.

Speaker C: What about us as individuals and Britain as a player on the world stage? How does AI help us to compete internationally in what they call the knowledge economy?

Speaker D: Actually, the House of Lords report and all the experts I’ve interviewed are quite optimistic on that score. For example, Demis Hassabis points out that Google came to London to acquire DeepMind and didn’t move its experts back to Silicon Valley.

Speaker F: And all this is happening, you know, for us right here in the centre, in the heart of London, you know, our home. We’re very proudly British company, and, you know, we work here at King’s Cross with our colleagues at the Crick Institute and the Alan Turing Institute, which is based in the British Library, all around. And King’s Cross is becoming quite a hotbed, and UCL of course is around there, of AI research. And, you know, as Andy mentioned at the start, we should leverage in the UK all of our incredible strengths, these amazing universities that we have here, Oxford, Cambridge, UCL, Imperial, and others that have incredible strength in computer science. And, you know, I feel very strongly that DeepMind needs to play its part in encouraging and supporting this AI ecosystem through sponsorships, scholarships, and internships, and actually lectures given by DeepMind staff. And I’m very passionate about establishing the UK as one of the world leaders in AI, and I think we have an amazing position. And we should really be building on our heritage in computing that starts with, you know, actually Charles Babbage inventing computing really 100 years before its time in some senses. And then of course that continued with Alan Turing, who famously laid down the fundamentals of computing in the ’40s and ’50s. Then the World Wide Web, you know, with people like Sir Tim Berners-Lee instrumental in creating the internet. And I feel like, you know, the next thing in the lineage of those types of technologies is artificial general intelligence. And I think the UK has a huge part to play in that, and I hope DeepMind will play its part in that too.

Speaker D: So London’s well placed to be the world centre for AI. Britain has a long tradition of innovation in this field. Charles Babbage and Ada Lovelace with the Difference Engine made the first mechanical computer. Alan Turing, is often referred to as the father of computing. Sir Tim Berners-Lee, the founder of the World Wide Web. And another figure who is less well known but may just become as famous, Dr. Donald Michie, the man who began artificial intelligence research in Edinburgh. Dr. Stephen Cave, head of the Leverhulme Centre for the Future of Intelligence at Cambridge University, believes companies need to rebuild trust and that countries should not see AI as something that confers competitive advantage.

Speaker B: I think what’s striking about this report, of all of the many reports that have come out on AI in the last couple of years, is how much it emphasizes the importance of taking an ethical, responsible approach. We’ve seen in recent weeks with the scandals around social media just how important that is. If we don’t take a responsible approach to developing this technology, then there will be a backlash, and that will undermine the kind of public trust that businesses and technology needs in order to really fulfill its potential.

Speaker D: Do you think that ethics can be put into technology? We mentioned this to some technology— to technologists about 4 years ago, and they laughed at us.

Speaker B: It’s about building the right kind of framework, the the right kind of regulatory framework, the right kind of codes of conduct, about educating developers and technologists who of course have a passion for solving problems and doing exciting things, educating them about the impact of their technology. They’re good people too. They want to benefit society. They don’t want to build killer robots. And so it’s about helping people to use the technology for maximum benefit.

Speaker D: When you say lose trust, what is the loss trust that you’re worried about?

Speaker B: People are nervous about AI. It’s a technology that has such deep resonances, gives rise to such deep anxieties about killer robots and robots taking our jobs and alienating us from the world around us as we get replaced by gadgets instead of people. And we have to face up to those fears if we’re going to use real practical AI to actually make our lives easier. We have to take these fears seriously and implement the kind of measures that show people that their worst-case scenarios, their worst fears, aren’t going to come true. And so build the trust that’s going to allow people to make use of this technology, to rely on this technology, so that we can use it.

Speaker D: But so ironically, what you’re saying is that we need to completely rebuild our relationships with technology.

Speaker B: I think there are lots of good examples rules for using technology responsibly. But there are times when the debate gets out of control, like it did with genetically modified organisms, where it became very polarized very quickly. There are issues that are intrinsically really controversial, like nuclear power, where the effects of nuclear waste might last for millions and millions of years. But there are technologies that we have gotten used to— cars. They’re extremely dangerous. They hurtle about the streets at, you know, however many miles an— they could easily kill people inside them or outside them. We found ways of managing this powerful technology for the greater good, and now we don’t think about it. We don’t think about doing it ethically and responsibly, but of course we do. We take safety seriously and, and so forth. So we can draw on these precedents for developing technology safely, setting standards and building trust and developing regulatory frameworks. There are ways of doing it right.

Speaker D: But some people would say that we only actually find out what the bad things— when something goes wrong, and then we regulate after the event.

Speaker B: Things will go wrong, of course. We are trying to prepare for a technology that’s developing incredibly quickly and that will impact on many different areas of life. But that shouldn’t stop us from trying to do the best that we can to prepare the way, to build up centers and capacity for thinking what it means to implement this technology responsibly. And we can even develop mechanisms of foresight, asking researchers to think seriously about how their technology could be used either for good or indeed for ill, and prepare for that before the worst happens.

Speaker C: Okay, but what about the House of Lords’ fifth commandment? AI, thou shalt not kill. How can we stop drones and Google’s Big Dog military robot and automatic soldiers from waging Waging war without taking orders from a human.

Speaker D: Well, Professor Noel Sharkey, the Emeritus Professor of Artificial Intelligence and Robotics at the University of Sheffield, fears that the national competition, if it does take place, might unleash an arms race for autonomous weapons, killing machines that can operate without human agency. He’s a signatory to the open letter to the governments of the world calling for a freeze on autonomous weapons development. And so am I.

Speaker E: Well, I wouldn’t say I’m an active critic of AI, but yes, I, I do criticize the application of it. I thought the report was wonderful, actually. I’m so pleased to see it because I’ve been reading all the UK documents about robotics and autonomous systems coming out from the government, and I find in one footnote the word ethical mentioned and nowhere else. And this report puts Britain in a moral leadership role in AI. It doesn’t solve anything, but I think the recommendations are absolutely wonderful.

Speaker D: One of the comments that Professor Cave made was that we shouldn’t really be country competitive about this. He said that if there is an AI arms race, it’ll be a race to the bottom. What do you think about that? That’s gonna be very difficult to achieve.

Speaker E: It depends on what it’s for. If it’s commercial increase of course there will be a race. But I noticed that last week 25 countries, including the UK and Europe, signed up a cooperative arrangement between them for AI. I mean, my biggest concern, and where I appear in the report quite extensively, is in the area of autonomous weapons systems. And the UK have provided in 2011 a science fiction definition which has been holding them back at negotiations at the UN, which I’ve been attending because they say these will never exist. And of course they won’t, because they’re saying pretty much that they have to be sentient to be autonomous weapons— completely out of step with the rest of the world. And here we are, and this report makes that really clear. They took my report to them, pushed the UK now, and they’ve given them 8 months to rewrite that definition and make it in line with the rest of the world. So I’m just over the moon about that.

Speaker D: I mean, that is an issue, isn’t it, in terms of of this conversation about AI and weapons systems. I mean, the Russians, for example, have said we’re going to develop them come what may, haven’t they?

Speaker E: They’ve been reported to have said that. At the UN, the Russians’ official statement is, what are you talking about? These weapons don’t exist. And yet we see them developing like crazy. We see Kalashnikov making announcements. We see the Armata T-14 tank and Russians with, you know, Terminator-like things with machine guns, which are ridiculous, but they’re publicity stunt. So it’s very hard to pin down their position. And of course, the international community, if we get a full prohibition of a ban— we got China this week at the UN to say that it would prohibit these weapons and supported our ban. If the international community do it, Russia can’t really do it or face more sanctions.

Speaker D: Although chemical weapons are theoretically banned, and we seem to have found that they’re actually still being used.

Speaker E: They are, and look what happens. The international community stand up to the mark and kick up a fuss about it and stigmatize anybody who uses them. I mean, I can’t say I approve of airstrikes, but nonetheless, it shows that there’s— it changes everything when you— somebody uses chemical weapons because they’re prohibited by the international law. And the same thing if autonomous weapons are prohibited by the international law, anybody who uses them will be sanctioned.

Speaker D: Although then there is that alarming situation of if somebody does use an autonomous weapon, you’re going to need to have an autonomous weapon to take it out, aren’t you?

Speaker E: No, not necessarily. A missile will take it out quite, quite easily, and so will those brand new laser weapons they’re using on drones. So they could be taking out if there’s only a few. Obviously, if everybody’s using them, there’s an arms race in proliferation, then that makes it much harder. But if it’s just one country, we can handle that quite easily.

Speaker C: But will the governments listen? Think of all the money that’s invested in military hardware and software. Will they be prepared to stop this arms race?

Speaker D: Listen to this. Professor Andy Hopper, Head of Computer Science at Cambridge University, takes a robust stance against politicians.

Speaker B: We’re independent of government, we’re independent of industry, and we’re independent of universities. So we can speak truth to power as appropriate in all those dimensions. And so you can be sure that we’ll make every effort not just to be trusted, but by using that independence to be trustworthy in this debate which is going to continue for some while.

Speaker C: Hmm, strong words, but is it realistic, Pete? Can you build in moral values to companies and technologies?

Speaker D: I put that question to Fred Cate, distinguished professor of law at Indiana University. He’s on Microsoft’s Trustworthy Computing Academic Advisory Board. He’s also a member of the US’s National Academy of Sciences Committee on Technical and Privacy Dimensions of Information for Terrorism Prevention and Other National Goals.

Speaker I: I think the answer to that is yes, it is possible. Although you then immediately have to take a step back and say, what do we mean by ethics? And so, you know, I think all companies, even companies that don’t care at all about ethics, care a lot about attracting customers and not causing offense that will drive away customers., and that may often lead them to behavior that actually turns out to be ethical. Again, not because they were pursuing ethics, but rather because they were pursuing something that they viewed as necessary to the transaction. I think the question you’re asking is can we go a step beyond that and say, can a company build in processes or values, if you will, that might not be in their best corporate interest or their shareholders’ interest, but that nevertheless serve ethical values. And again, I think the answer to that is yes. I think it is much harder, and I think we see many examples of companies that don’t do that, but I don’t think that makes it impossible, and I think we have seen that.

Speaker G: Let’s take one example, Cambridge Analytica. It’s a name that’s now known right the way around the world because of the activities of this company in the UK, which apparently was taking data from Facebook and using that information to try to influence the American election, if everything we’re told is true. People said that’s unethical. Cambridge Analytica’s now closed. It says that it wasn’t behaving unethically. It was also saying that those ethical principles don’t exist at the moment, so how could it be doing something wrong if it’s not prescribed? Is that a position?

Speaker I: Is that an argument? No, I actually don’t think that’s a valid argument. In other words, I think what Cambridge Analytica is trying to say is they don’t think they broke the law. And frankly, I have no idea whether they broke the law or not, but I don’t think that is clear whether or not they did. But that is a different question than did they act ethically. And so there are lots of things that can be done that we do in our daily lives, I think that companies do, that reflect— you might call them professional standards. They reflect individual values. They reflect ethics, something other than just am I doing what the law says.. And sometimes we see those standards actually enacted into law. So then you have no choice. And so, for example, you know, requiring what in the U.S. we call institutional review boards before you do research involving humans. And, you know, we started with ethical principles and then we enacted them into law. But I don’t think that’s the full extent of ethics. It’s not just what’s enacted into law.

Speaker G: It’s really what goes beyond law. One of the things that people are saying now, and you’ve touched upon it, is that if we are to develop artificial intelligence, if we are to develop artificial intelligence technology, that there is a need to rebuild the trust between the individual and technology companies, and also for technology companies to actually completely reforge their relationship because They should be developing AI for the common good and benefit of humanity, not necessarily for their balance sheet.

Speaker I: Yeah, I can understand the attractiveness of that argument. I think it has little practical value. In other words, we live on both sides of the Atlantic in a very commercial economy. It is driven by companies and customers making decisions. Not necessarily by the notion that any of us act in our everyday lives consistently in what we view as the greater good. And to think that suddenly we’re going to have a profound philosophical transformation and expect companies to use that as their guide stone, I think, is unrealistic. And frankly, I don’t think it’s very helpful to think about that. I do think it’s helpful to think about both the role of law and the role of clear ethical standards or professional standards so that it’s easier for companies to try to evaluate when making a decision. Am I making a decision that’s going to cause greater harm than it’s going to cause benefit, or that’s going to cause harm that I may not be aware of? And, you know, this is— ethics review boards can help with this. We see some companies creating ethics officers. But I have to say, I think the role of trust here is often I wouldn’t say overstated. I just think it’s misunderstood. For all of the people who say they don’t trust technology companies, they’re still going out and buying their products and using their services. And so the level of the trust deficit either isn’t big enough or lack of trust is not enough to stop people from not using something they think is useful in their lives. Kate says he doesn’t think many companies are being honest about what they are doing with data? I think they are not in many cases, perhaps in most cases, stating clearly what they are doing. I think it is very valuable for them to do so, and I’m delighted to see law compel them to do so. But I would just say in many cases it is not going to make a difference. And it’s ironic. We’ve seen a lot of testing around security, for example. And we have loads of studies showing that when browsers first started bringing out real security tools, tools that told you you were going to an unsafe website, when Internet Explorer changed the browser bar color to red to alert you that you were going, when you got pop-up messages saying, “This is a known phishing site. Are you sure you want to go?” The vast majority of users ignored those clear, powerful warnings and went ahead and did what they wanted anyway. And so I think we are likely to see the same thing around other forms of, if you will, trust.— so that even clearer disclosures about what Facebook does with your data isn’t going to stop people who think Facebook is important in their lives from using it.

Speaker C: Well, the case of Cambridge Analytica is just one example of what can happen when they’re not honest about how personal data culled from Facebook and search engines is put to another use— in their case, in order to influence the outcome of the United States presidential election and the UK’s referendum on leaving the European Union.

Speaker D: The trouble is that people still go to websites even though they are told they are bad. Fred Cate thinks that AI may help by teaching us to be a little more high-minded and that AI could be taught to guard our privacy the way that the Lords want.

Speaker G: Okay, but in that case then,— do you think that there’s a need for an AI system to be developed that looks after your interests against Facebook and says, “Hey, you shouldn’t be doing this to my— the person I’m looking after, because they don’t really understand the ramifications of this, and I want to extract these promises from you.” Do you think that that’s a possible role for AI?

Speaker I: Yes, yes I do, and I think it’s a really important role. And, you know, we often talk about AI as a threat to privacy. It can certainly be a threat to privacy. But it could also be a great enabler of privacy. It could be the thing that helps guard against the inclination most of us have to make bad decisions or to make uninformed decisions. And so I think that’s an incredibly important tool, and frankly, for both privacy and security.

Speaker D: Still, we have to question why education about AI as opposed to education about other things. It’s a point that Xavier Ruiz makes, and this is what we really need to think about. We’ve suddenly decided to get ethical and moral with machines because they threaten our superiority, but we’ve done such a bad job of ethics and morals ourselves. Ruiz says that we should teach people about ethics and philosophy from the start in schools. He also says We should teach regulators about technology. As Kate just says, well, really, we should ensure that we have the best education.

Speaker I: Why education about AI as opposed to education about other things? Why the right to participate in decision-making with AI as opposed to the right to participate in decision-making generally? And I think it’s important that we not become so transfixed by AI that we lose sight of the fact that the values questions are not just linked to AI.

Speaker C: They’re linked to society and to humanity in general. Let’s face it, we humans are the ones who write the algorithms, train them, and program the computers.

Speaker D: So aren’t all our attitudes and failings just going to be embedded in the code that’s written? Fred Cate’s also worried about this. He is making the point that once again we’re being lazy and abrogating our responsibility. That we are wanting ethics put into the machines so that we don’t have to be bothered with ethics. The need, according to Kate, is to have a more ethical society. The use of technology in and by society, according to Kate, is simply a measure of the society itself. If you want ethical technology in your society, then your society has to be ethical, and that is how you will use your technology.

Speaker I: It does what it’s told. So I think AI is not going to be at heart better than we are. And so although I think it is a very useful tool and a tool that we already rely on and will rely on much more in the future to keep us safe and to help address important issues, at the same time, there are lots of issues we know how to address right now when we’re just not willing to do it. There’s not a country with a democracy that thinks that its leaders are doing what the people most need. We see corruption throughout government and industry and elsewhere, even though we have the tools to detect it and prevent it. So I don’t think AI is going to save us from ourselves to the extent that we as societies, that we as people are interested in trying to move forward I think AI will be a critical tool for doing that. And then it will be incredibly useful for all sorts of things that aren’t so high-minded. You know, they may make us just a little bit more productive, or they may make our entertainment a little more tailored, or they may do other things that are valuable in the sense that people value them, but not necessarily high-minded.

Speaker C: You know, this reminds me a bit of the Pilgrim Fathers. They set sail for the New World because they wanted to build a better society based on their religion and ethical values. But they took with them all their own human frailties, and then they just reproduced in North America a version of Europe that’s turned out, if anything, to be more violent and unethical than the Old World.

Speaker D: But it doesn’t have to be like that. The House of Lords points out that these are very early days, And if we act now to regulate the use of artificial intelligence, we can make this new world a better place. That’s all from this month’s Password, but you can find out more about the House of Lords Select Committee on AI at our website, Future Intelligence, and download the podcasts from our eminent interviewees. Password is made by me, Pete Whirring, with Jane Wyatt and producer Lou Bufferett.

Speaker B: Thanks for listening and goodbye. This program has been brought to you by Resonance 104.4 FM. If you liked what you heard and want to support our work, please make a donation at fundraiser.resonance.fm.