PassW0rd – 10 May 2023 (AI Ethics)

Speaker A: This program is brought to you by Resonance FM. If you like what you hear, please support our work by making a donation at resonancefm.com/donate.

Speaker B: Hello and welcome to Password on Resonance FM with May Blue Buckley. The programme about the impact of technology on our lives. In today’s episode, we look at the ethics of artificial intelligence and explore the debate that has begun about how it should be used. It’s a topic close to the hearts of those involved in the website Future Intelligence, which makes the Password Radio programme, because on the 10th of June, the Harry Potter publisher Bloomsbury released a book on the subject, co-written by our usual presenter, Peter Warren. The book AI on Trial examines the role that AI has already begun to play in our lives, where it is going to be deployed in the very near future, and the radical impact that it could have on equality in our society, and why we all really need to know about AI. Here’s Peter Warren on what AI is already doing and where people are aiming to deploy it in the future.

Speaker C: AI is here now. We touch on AI every single day. If you’re using a mobile phone, that’s driven by AI. Whatever’s routing the mobile phone around, sending the messages around, all of that is AI. If you get on a train, you’re engaging with AI because the AI is routing the trains around. If you look, you’ll see that all of the signal boxes that used to be there that people used to have on their train sets, they’re all boarded up now. That’s because electronic relays are actually controlled by AI. They’re controlling where the trains are going. You turn a light switch on, AI controls the electricity network. AI is in the computer networks. It’s literally running our lives already.

Speaker B: If the AI is there, why can’t I see it? Why isn’t it obvious?

Speaker C: Well, you can’t see it because people tend to think of AI as something from the film. The film industry’s done us a bit of a disservice because AI actually does very boring tasks at the moment. It’s very mundane. It sits in the networks, as I was saying earlier. It sits behind the scenes in our world.

Speaker B: So how fast is this process going to go?

Speaker C: The process is accelerating enormously. One of the things about AI is it’s a software program. There are these things called algorithms that people are very, very scared of, or they don’t know anything about. They need huge amounts of data. That’s why whenever you go onto a website, it asks you whether you are prepared to give it your data. It’s your data, information about you, what you’re interested in, what you’re doing, that it’s interested in all of the time. Google, for example, has the highest number of data points of any organization. It’s collecting information about you. It’s profiling you. It’s finding out what sort of a person you are, because that information is very, very valuable. It’s very valuable to advertisers. It’s very valuable to governments. They know the sort of person that you are. They know the sort of things that you’re interested in. Perish the thought, we don’t like to think that we’re like other people, but some people actually are.

Speaker B: Looking forward, how is AI going to work with us?

Speaker C: Well, that’s the big problem. The big problem with AI is whether we make it work for us or whether it works against us. At the moment, what you’re seeing, and there is an incredible temptation for large organizations to do this, to an extent you’re seeing people trying to manipulate our behavior, trying to push us around. The AI can be used to actually suggest to particular personality types, ‘Oh, you’ll be interested in this.’ Now, you might think that that’s not directly manipulation, but what you can do is you can start to make people buy things based upon who they are. The converse of that, and it’s very, very difficult to actually identify the difference in this, is that one of the things that people want to do is they want to make the AI help you. They want the AI to say, you need to be educated. I’m going to help educate you. I’m going to I’m going to find a career for you based upon what I know about you. I’m going to be very, very supportive. I’m going to be your pal on the phone. Some of the large technology companies are trying to do this at the moment. And you’re seeing more and more of this augmented AI where the AI system is looking to try to be helpful to you. It’s trying to find you jobs. It’s trying to suggest things that might make your life better. The problem is that we have to actually ensure that it’s doing that and it’s not manipulating us. It’s very frightening because what we have to think about is the world that we’re going to go into and what we’re having to think about is how much of our identity we are going to retain in that world and how much self-determination we’re going to retain. One of the things that is very, very important is that we stop seeing AI as the Terminator-style robots. The AI that you need to be more worried about is the AI that’s in the system. Soon you will have Internet of Things sensors that will be in chairs. There will be Internet of Things sensors everywhere in every part of our lives, and they will all be pulling information about us. That they would be able to find out how many times you go to the pub, for instance. They would be able to find out how overweight you could be. We have to have a big debate about this incursion into our lives and how far we want it to go, and also who is the recipient of that information.

Speaker B: So positives of AI, but also terrifying. Do we fight this? Do we embrace it?

Speaker C: Well, what we have to do is, in a sense, we have to develop something that is akin to some of the supranational organizations that we have for moderating things like genes or gene therapy or genetic development. We have to also have some body that oversees this to make sure that it does what is good.— and that’s going to be a significant debate in and of itself, because what some people consider to be societal good is not necessarily what other people do. There’s going to have to be that debate about what is in the best interests of people, and then there is going to be another debate about whether the best interests of people are in the best interests of the government.

Speaker B: The relationship with technology and the information and data that we generate that will also create a huge range of threats to us and bring the ethics of humanity ironically to the fore as we debate just what it’s right to use AI for and where it should not go. Here’s Mark Deem, a lawyer on technology issues and Peter Warren’s co-author on AI on Trial, on the risks of unfettered AI. Unfettered AI?

Speaker D: Well, we’ve all probably seen the headlines which characterize AI in a very sensationalized way. The real risks of unfettered AI operate at a lesser but nonetheless pernicious level, which is AI presents us with some very good and real opportunities which, if we can harness them, not only can we see some fundamental economic growth that we might be able to achieve some societal good, which is becoming increasingly important in these sort of fractured and dislocated times. I think it’s fair to say that from a technical point of view, there may be a lot of technologists who would welcome the opportunity for the technology to develop in an unfettered way, and they may certainly make out a use case for that in a sandbox environment. But the real problem here is if it’s unfettered and it’s in a live environment, that we really run the risks not only of harm occurring, but that we are not able to achieve the real promise that is offered by the technology.

Speaker C: And presumably it could also mean that we could manipulate people. We could get very ruthless or very unethical companies saying, hey, I’m gonna use my AI to find everything I can about people and move them around and make them buy stuff.

Speaker D: Exactly right. And information and data that is available and put out there by ourselves at this moment in time could be used for unforeseen circumstances or unforeseen purposes. So it’s not simply the case of us saying that just because we have consented for someone else to hold on to our data, that that should be sufficient. Because if those individuals that are using that data have ulterior motives and perhaps malevolent motives, then there’s very little that we can do once we’ve given our consent to stop that data being used against us.

Speaker C: Do you think we should make people understand a lot more about AI? Do you think we should make the population at large aware of what’s actually going on? They don’t know what’s working behind their mobile phones, do they?

Speaker D: They don’t, and unfortunately the prism through which they’re seeing AI is that sensationalist way that I started off our discussion, talking about killer robots, killer drones, and that is very unhelpful. You combine that sensationalism that we see in the print media, in the broadcast media, in the films, and in Hollywood with the the level of understanding, or perhaps a lack of understanding or education that we have of the way in which the technology works. And it makes for quite a heady brew because a lot of people think about AI in terms of the output, or thinking about in terms of a robot, rather than perhaps the more mundane thing that AI actually is, which is the combination of statistics and computing power. And those statistics being driven by data and by algorithms. And once seen in that light, perhaps a less sexy light in many ways, you can actually start to understand and get under the bonnet of what’s really going on here, what we really need to do. But so long as we see it through the sensationalized prism that has been presented to us and unfortunately don’t get beyond that, there is a real risk here.

Speaker C: AI could be a real force for good, couldn’t it? It could really develop some societal change. We could educate people, we could make their lives better. If we really wanted to do it.

Speaker E: Absolutely.

Speaker D: The power of AI and combining the power of AI with human intelligence in many ways to augment what we’re doing at the moment is where the real opportunity in the short to medium term exists. It really has an ability to impact all different aspects of our lives. And certainly, as I said, in these fractured and dislocated times where we have economies that are troubled, we have societies that are perhaps less connected than they might otherwise be. That if we’re going to get through this, it could be the technology that really unlocks that. But that requires us to really use the technology in the right and appropriate manner.

Speaker C: How do we go about controlling AI? What do we do?

Speaker D: The most important thing is to actually really get under the bonnet and work out what AI is, because if we just seek to control what the output of any AI is, I think we’re going to be going down the wrong path. As I’ve said, AI is very much a combination of statistics and computing power, and the statistics are driven by data capture and also by algorithms. Now, within that data capture, within those algorithms, is bias. And bias is not necessarily a good or a bad thing. It just reflects who we are as humans. By our very nature, we are flawed, but we are generally free-thinking and therefore vulnerable to have thoughts or preferences which may be considered unacceptable to many. And so really what we’re doing when we’re looking at trying to control this is we’re trying to make sure those potential flaws, those biases, are not in any way amplified in a way that could be detrimental to a certain part of society. So rather than look at the output of the AI and try to control and predict what the output might be, I think actually what the controls should look like is trying to understand and temper what is going into the technology so that it doesn’t operate against a certain, perhaps protected part of society.

Speaker C: In a sense, what you’re saying is that we need to create an example because we need to show what good AI looks like to set up a model for the rest of the world, in a sense.

Speaker D: And therein lies the problem, because what may be seen to be good AI for us may not necessarily be seen to be good AI for someone else, somewhere else in the world. And it’s that qualitative judgment of the quality or the how good AI may be, or the purpose for which AI is being used, is where the real problem comes.

Speaker B: So AI is going to be everywhere, and indeed it already is, according to Professor Stuart Russell, a professor of computer science at the University of California, Berkeley, an adjunct professor of neurological surgery at the University of California, San Francisco.

Speaker C: Probably the—

Speaker F: your most frequent encounter with AI is in social media. Every time you are recommended to watch a video on YouTube or your Facebook feed has something in it, those are being suggested by machine learning algorithms. Every time you use a search engine, the answers are being generated by machine learning algorithms, natural language understanding algorithms, other kinds of AI reasoning systems. So it’s really pervasive in the online world, and I think fairly soon— not immediately, but fairly soon— the most visible application will be cars driving around by themselves.

Speaker C: Everybody always makes this discrimination that there is general AI and deep learning AI. The AI that we’re talking about in social media, a lot of people are saying that’s the deep learning AI. Is that right?

Speaker F: Well, the algorithms actually are mostly proprietary. I think it’s a mixture. It uses reinforcement learning technology that goes back decades and decades and decades, but the ability of the algorithm to take input in the form of vast quantities of text or imagery or videos that’s really facilitated by deep learning methods. But I wouldn’t say it’s either or. I think most solutions these days are some combination of the two. Machine learning is one of the oldest parts of artificial intelligence. I think the word was first used in the late 1950s. Alan Turing in 1950 said that machine learning would be the way that we create AI systems in the future.

Speaker C: Now, on that, the big worry that everybody talks about, everybody discusses, it’s in so many books, it’s in so many films, is that AI will evolve and that it will take over. That there is this notion of the singularity that gets bandied around all of the time. Some people are saying actually it doesn’t really need to, that all you need to do is put together a pyramid of all of these deep learning systems and you put another AI system on top of that and you essentially achieve that notion of wanting to get all of the information and understanding that you need or desire?

Speaker F: I don’t think we’re actually very close to having AI systems that exceed human capabilities in general-purpose learning and decision-making. AI systems don’t function particularly well in the real world. I mean, if they did, we would have self-driving cars, but even that problem is is currently still too difficult for the algorithms. And a lot of it is because they don’t have the kind of common sense that human beings have to work out what to do in situations they haven’t seen before. And so the ability to run a company or teach a child to read or milk a cow that’s being recalcitrant, I don’t think we’re particularly close to having those kinds of capabilities. And it really will require breakthroughs. But I think there is a valid point in what you say in that it actually doesn’t require that individual machines be superhuman in order to present a risk of serious harm to our society. And I think some people argue that those simple machine learning algorithms that are recommending news articles and videos to watch, are already having really serious negative impact on our world. And one reason, of course, is that there are billions of copies of those algorithms, so they can have a very large impact. And the other is that the algorithms are programmed to optimize a fixed objective, which is typically related to the amount of clicking that you do, because that’s what generates revenue. And in pursuit of that objective, the algorithms have learned to manipulate human beings to turn them into more voracious consumers of predictable content so that the algorithms will in future be able to guarantee that you’ll click on something because they have turned you into a consumer of that type of content. And this is completely unconscious in the sense that the algorithms don’t know that humans exist or have brains or anything else. You’re just a stream of clicks as far as the algorithm is concerned, and they want future streams of clicks to be more profitable, and so that’s what they do. So you can start to see how this loss of control that people are afraid of happens, that you have algorithms that are fixated on some objective, and in pursuing that objective, the side effects are serious or even terminal.

Speaker B: Professor Stuart Russell, with his co-author Peter Norvig, has produced the most popular textbook in the field of AI. ‘Artificial Intelligence: A Modern Approach,’ used in more than 1,500 universities in 135 countries. He’s also a noted opponent of the use of AI in autonomous weapon systems and their manufacture, and he’s not alone. Professor Toby Walsh of the University of New South Wales is also a noted opponent of the use of AI and robots in warfare. And has been at the forefront of the campaign to ban them, something he debates passionately in AI on Trial.

Speaker G: Thousands of my colleagues have got rather strong views about this, that this will take us to very dangerous territory, that it will cross, and there’s plentiful legal, technical, moral issues to be concerned about. One of the mistakes I think people make when you start discussing this is to think that The concerns that people like myself working in the field have are fixed in time, and actually the concerns are going to change over time. The concerns I have today with current capabilities are often about the inadequacies that would be handing over control to machines that would be making lots of mistakes and killing lots of the wrong people. But equally, I can also see in 10, 20, or 30 or 40 years’ time when the technologies are much more capable, then we are much greater concerns, some of which are about the fact that technologies will be much more capable and that that will transform the way we fight war in a way that humans will no longer be in charge. And the possibility that we might end up with flash wars happening because we put these complex systems out into the real world and they interact, you know, putting competitive systems out in the real world interacting with each other. And we know ends badly. We’ve already seen examples of how that ends badly. It’s called the stock market. And we already had to put in circuit breakers and things to make sure that these complex systems don’t behave in undesirable ways. Well, that’s okay in the stock market because if something goes wrong, as they do, the circuit breakers kick in and they stop all the transactions and they say none of that took place and they give everyone back their money and say, okay, we’ll start again. Well, you can’t do that if that was the border between North Korea and South Korea and you’ve just started a nuclear war because the algorithm has got into feedback loops and faster than humans could intervene, they actually started fighting a war.

Speaker C: But as we’ve mentioned already, will the wars be as the wars in the past have been? Won’t they be people fighting for your heartbeat? Won’t they be people trying to use technology in a so much more sophisticated way that the war isn’t necessarily noticeable, and also that those things that occur on the stock market will actually be considered to be wars.

Speaker G: I think you’re quite right there. World War III will start on the internet. It’s clear it’s going to start in cyberspace before it starts in the physical space. People are going to take down infrastructure, the water system, the power plants, the hospitals, Everything that’s connected is going to be brought down. So the world will notice, well, you’ll know that World War III started because everything will stop. And that’s going to impact all of us, but it will then spread out into the physical world. If you look at the trajectory of warfare, it’s increasingly against civilian populations. It’s not fighting on battlefields with one army against another. We no longer fight most of our wars, not wars against regular armies. They’re very asymmetrical against opponents who don’t fight in that way, and increasingly they’re against civilian populations. You only have to look at Syria and elsewhere and to see that wars are increasingly fought in and around and against civilian populations. And it’s certainly not going to be my robots against your robots, and the idea that some people have that, oh, we can get people out of the battlefield and it will make warfare a much easier thing is, I think, an incredibly dangerous and wrong-headed thought.

Speaker B: That was Professor Toby Walsh and why he thinks that AI-controlled weapon systems should be banned. It could be too late. Already there are reports that AI-controlled weapon systems have been used against people. You’re listening to Password on Resonance FM, and after us, DJ Ritu and A World in London. Here on Password, we’re exploring the pressing need to make the world know more about AI. In May 2021, the New Scientist printed a piece based on a United Nations report that stated that autonomous drones had been used to kill soldiers in the conflict between Syria and Turkey. Despite that, military ethics committees in both France and the US have cleared the technology for use by their armed forces. The French Military Ethics Committee gave permission for its armed forces to develop augmented soldiers and is considering medical treatments, prosthetics, and implants that improve physical, cognitive, perceptive, and psychological capacities and could allow for location tracking or connectivity with weapon systems and other soldiers. What that basically means is that soldiers will be directly linked, presumably by encrypted radio waves, to each other and their equipment. The committee said that France needs to maintain operational superiority of its armed forces in a challenging strategic context while respecting the rules governing the military, humanitarian law, and the fundamental values of our society. As a result, it’s forbidden any modification that would affect a soldier’s ability to manage the use of force or affect their sense of humanity. The US position is similar. It says it has a moral obligation to develop AI weapons.

Speaker C: This is what they said: The National Security Commission on Artificial Intelligence NSCAI, humbly acknowledges how much remains to be discovered about AI. Nevertheless, we know enough about AI today to begin with two convictions. First, the rapidly improving ability of computer systems to solve problems and to perform tasks that would otherwise require human intelligence, and in some instances exceed human performance, is world-altering. AI technologies are the most powerful tools in generations for expanding knowledge, increasing prosperity, and enriching the human experience. AI is also the quintessential dual-use technology. The ability of a machine to perceive, evaluate, and act more quickly and accurately than a human represents a competitive advantage in any field. Civilian or military. AI technologies will be a source of enormous power for the companies and countries that harness them. Second, AI is expanding the window of vulnerability the United States has already entered. For the first time since World War II, America’s technological predominance, the backbone of its economic and military power, is under threat. China possesses the might, talent, and ambition to surpass the United States as the world’s leader in AI in the next decade if current trends do not change. Simultaneously, AI is deepening the threat posed by cyberattacks and disinformation campaigns that Russia, China, and others are using to infiltrate our society, steal our data, and interfere in our democracy. The limited uses of AI-enabled attacks to date represent the tip of the iceberg. Meanwhile, global crises exemplified by the COVID-19 pandemic and climate change highlight the need to expand our conception of national security and find innovative AI-enabled solutions.

Speaker B: But the renowned US military hawk, Brigadier General Robert Spalding, thinks that the dangers posed by AI go even further than that. According to the Brigadier, robot planes, tanks, and troops are the least of our worries because Spalding says that we are now in an arms race that involves data and our relationship with technology, and one that has AI at its heart.

Speaker H: And in the 21st century, it’s also how you collect intelligence and drive influence. Now, we spend in the United States $800 billion on aircraft carriers, on F-35s, on nuclear subs, on tanks, on an incredible array of military capability. None of that matters in the 21st century when you can influence at the individual level using the data they collect on people.

Speaker C: I mean, in a sense, what you’re saying is that the war of the future then is already going to be fought through these networks and through these AI systems. It’s a battle for hearts and minds. It’s not going to be with missiles. It’s going to be with data.

Speaker H: And it’s not the battle of the future. It’s happening now. In 2016, the Russians used artificial intelligence bots, social media networks, and big data to create protests in the United States on both sides of the aisle. Right up to the Taiwan elections, the Chinese were using the same kind of technology and techniques within Taiwan, in Malaysia, in the Philippines. It’s happening in Europe today, all over Europe, all over the world really. These tools are being used not only to create economic value for the companies that possess them, but also if those companies happen to be in totalitarian regimes, to create influence for those regimes. This was documented by Samantha Hoffman, a researcher out of Australian Strategic Policy Institute when she talked about Global Tone Communication Corporation, a big data and AI company in China that does language translation in 65 languages. Its technology is built into Huawei products. It collects 2 to 3 petabytes of data per year and then sends that data not just for translating languages, but also sends it to the People’s Liberation Army intelligence arm and the propaganda arm of the Chinese Communist Party. This is not the future. This is happening now.

Speaker C: You talk about all of that data. Can the Chinese process that amount of data?

Speaker H: This is exactly what I was asked by the intelligence community, or actually told by the intelligence community in 2017. How can the Chinese possibly essentially process all that data? I said, that’s what Facebook, Amazon, and Google’s business model is based on, being able to process that data and turn it into clicks. They are experts at it. And you know what? The Chinese have all that technology and capability.

Speaker B: So Brigadier Spalding, like the US National Security Commission on AI, says that we are now in a battle for hearts and minds, a battle of ideology with AI and its use at its center. But the cybersecurity expert Dr. Matthy Axella of F-Secure says that what’s right and wrong is subjective. You can encode ethics into AI, but doing so also involves an ethical choice, and deciding whether it’s ethical or not can depend on which side of a wall you are. In Russia or China, they definitely do not share the same perspective as many of those in the West.

Speaker I: What I’m saying is basically like if you were to say like the stereotypical, some years ago, like machine learning expert is a PhD in computer science and knows the underlying fundamentals. Is it enough for those people to understand the ethics? Actually, probably not, because there’s a lot of people applying machine learning in these existing packages that don’t really think about even the underlying models. And that’s kind of, in a way, the democratization of data science and machine learning is both a huge opportunity, but then there’s also slightly a risk, because when people, in a way, common understanding sometimes is that actually it’s more than it really is. I forget whose quote it was, but modern AI is basically complex curve fitting. So that’s actually pretty true. So most of them are really complicated, kind of simple curve fitting solutions that optimize a certain boundary between decision boundaries and try to make the most out of it. So when you don’t also think about that, that’s one of these challenges potentially. Like if you don’t really have the right expectations on what your algorithm can do, and then you point it towards even a good task and a positive thing, like for example these recruiting examples, if you don’t understand in a way that your data will be biased or is biased in that situation, if you don’t see that coming in a way, if you don’t think about it in advance, then you might reach totally different outcomes than you intended, and that can be arguably, well, again, we’re in a bit kind of Ethics isn’t my strongest point, but, but kind of even if your intentions are good, you can also reach outcomes that are nothing like you hoped.

Speaker C: That’s a very good point, isn’t it?

Speaker E: Because it’s a—

Speaker C: in China, for example, there may be a completely different objective that you think is good, and somebody in America could also think that they have an objective that is good, and someone in Europe could also have a different objective. Each person thinks that they’re doing something good. The corresponding states don’t think that they’re doing something good. For example, if somebody in an intelligence agency in the UK is out to get some data from China, then the people in China will not think that is very good. Somebody in China out to get some data from an American database, the Americans won’t think Yeah, yeah, definitely.

Speaker I: So yeah, yes, yes. And there’s, there’s lots of use of machine learning basically for marketing purposes. So selling more things to people, you might argue that that’s kind of, of course, good for the company, selling more, getting more revenue. Is it always what the people need? Well, that might be a different question in some cases. So especially if we’re talking about something that’s not necessary, like I’m not talking about really like really bad things, but even promoting people to drink a lot more soft drinks, which are arguably not that really healthy for you, or kind of like— there’s many shades of grey in this, and it’s always relative to kind of your perspective. And then, yeah, hard questions.

Speaker B: Dr. Matti Aksela of EvSecur saying he doesn’t want a machine to be able to make the decision as to whether to press a button that could kill hundreds of people. A slightly paradoxical position, to say the least, because it suggests that that should be a task that falls to someone like Hitler or Stalin. And not a system that could be arguably more intelligent than both of them put together, that may make such a decision from a far more informed position. But the very existence of AI research in and of itself presents a very real threat of an AI arms race, as outlined by the famous Finnish hacker and technologist Mikko Hypponen. For him, it is as dangerous to win as it was to lose.

Speaker G: Let’s pick IBM. IBM makes a press release next week that they’ve made a breakthrough. They believe they are on the verge of superhuman intelligence and they’re ready to show demonstrations next month and ship it next year. Unimaginable right now, but let’s just assume that they’ve had a breakthrough and they could actually do something like this. What would happen? Think about players like President Xi or President Putin. What they would see from their eyes is that those guys, the Americans, are going to win this race. And if they win this race, this is the most important race, if they win this race, they will win everything. The Americans will be superior in everything forever. They will win every race in every area. They will do every innovation from now on. They will be the economic superpower forever, they will win every war. And if that’s the case, then the obvious thing to do is to steal that technology at any price. Or if you can’t steal that technology, then you must destroy that technology so your enemies don’t get it. To me, it would seem that an innovation at this scale would destabilize our global peace instead of bringing great benefits.

Speaker B: The deployment of AI weapon systems poses the very real risk of causing great damage to people. So why do we need this technology? Paradoxically, one of the areas that many see AI having the greatest impact is health, a point emphasized by Amir Butt, the former chief executive officer of the AI-based cancer screening company Tumor Trace.

Speaker E: It’s better in some diagnostic fields, and where it’s not better, it will become better, because the more data you gather, the more accurate your predictions become. So that’s a function of time. It’s not if, it’s just when. In those areas where it’s not better than humans, it quite simply will become so. As more data becomes available. One of the main reasons for it is that once an algorithm is trained to perform a certain task to the level that you require it, then it will always carry out that task. It doesn’t make mistakes. It does not get tired. It doesn’t get overwhelmed. It does not suffer from anxiety or stress because it had an argument with his partner before it started work. And its conclusions are not subject to interpretation. If you take those factors into account, as long as it’s been properly trained, it will keep giving you the correct outcome. In biology, most things have an underlying pattern. Put a certain number of chemicals together and they will form a particular shape of a protein. That’s an underlying pattern. If you have symptom A and symptom B and symptom C, the implication is that you have or have not got cancer. That’s an underlying pattern that is being discovered. Thus, in the area of medicine, anything that actually has an underlying pattern will benefit greatly from artificial intelligence, and anything which is truly random is unlikely to benefit. I can’t think of anything in biology which does not have a pattern that’s just waiting to be discovered. Our perception of free will, for example, it’s just biology we don’t understand. If you think about our behavior, our responses are so governed by our biology that I think we will soon be able to use AI to explain a lot of what we do in our biology. If AI uncovers a pattern that we do not understand, the cause for that is very exciting because that then highlights an area that must be researched.

Speaker B: This is a feature of the technology acknowledged by the US NSCAI.

Speaker C: AI is an inspiring technology. It will be the most powerful tool in generations for benefiting humanity. Scientists have already made astonishing progress in fields ranging from biology and medicine to astrophysics by leveraging AI. They are the kind of discoveries for which the label game-changing is not a cliché. AI is already ubiquitous in everyday life, and the pace of innovation is accelerating. Recognizing the pace of change is critical to understanding the power of AI.

Speaker B: The impact of AI on our lives will be so transformational that some have called it the Fourth Industrial Revolution, a title that is a significant understatement given the changes it is already wreaking upon our lives.? For just as with previous technological revolutions, such as the Agricultural Revolution and the three industrial revolutions that followed, one of the immediate consequences were job losses. The harnessing of water power, either as hydropower used in water mills or as steam for cotton mills, ended the careers of the handloom weavers, whilst the introduction of electricity further revolutionised industrial production in a similar way to the revolution brought about by the introduction of electronics, which has now ushered in the roboticisation of production lines that AI will manage. In a nod to the Industrial Revolution brought about by steam, Cambridge University’s DeepMind Professor of Machine Learning, Neil Lawrence, thinks there are important issues we need to confront about this potential human redundancy and how we will relate to AI-controlled systems in the future.

Speaker D: Well, that’s what’s so interesting. I think that what we’ve seen before is that the nature of the tasks has to change. So this happened in the past, but because of everything I said about the great AI fallacy, it’s the humans that need to adapt what they’re doing in order to accommodate the AI. If you’ve got a doctor, how is the AI going to help the doctor? Well, only in areas where the doctor was doing tasks which are being repeated multiple times, and we can now pass them over to an AI and they have to work with the doctor. So a lot of the themes we’re talking about are relevant there. How does the doctor maintain control, and how do we decompose the doctor’s day to take out those tedious things? Because the truth is the computer put more tedious things in our lives because it couldn’t handle the separation of tedious properly. So it’s sort of related to the Great AI Fallacy, and it’s also related to the intellectual debt, the control of not understanding. Both of those need to be overcome, and in jobs where they’re easy to overcome, those jobs will be change very, very quickly. And in jobs where they’re difficult to overcome, it will take more thought and some restructuring of the way we’re doing the job in order to help.

Speaker C: It would absolutely help, but it needs to now be empowered.

Speaker D: It needs to be turned into something portable and manageable and steerable by the user, which, which we’re a long way away from.

Speaker C: Because that’s the other interesting point about all of this, isn’t it? That AI and machine learning don’t have demand. That is what differentiates us, surely. That’s What’s the point?

Speaker D: Yeah, well, they don’t have common sense. People say, “Oh, well, just put it in.” Okay, where’s that come from?

Speaker B: As Professor Lawrence states, we like to think of ourselves as intelligent, so when something replaces us, we confer upon it intelligence, while in this case it is simply a machine. In agriculture, according to Professor Blackmore, the process of human redundancy has been ongoing. Farm mechanisation, enabled by the Industrial Revolution, which could make the farm machinery, began to reduce manpower. With AI, that reduction in farming is nearly complete. According to interviews carried out for Password, farmers are now swapping the tractor seat for a laptop at a desk. Up until now, the symbiotic nature of the impact of these various revolutions has softened their impact. The enclosure movement that brought about the large fields that mechanisation and modern farming methods could take place upon forced peasants from the land. They moved to the slums of the cities and found employment in the new mills and factories of the Industrial Revolution. The argument has frequently been made by economists that while the initial consequence of rapid economic change is unemployment, that the world of work also changes to create new jobs. With AI, that development is less clear. Thus, the societal ramifications of this pervasive AI involvement are huge. AI can quite simply do a lot of repetitive tasks that people currently do much better than us, as we have seen from the robot takeover of the manufacturing industry. In South Korea, mobile phone production lines for the company LG are totally automated. The only current role for human beings is to slide around upon cushions on their bottoms to clean and service the machines, a trend that has seen factory workers suffer a similar decline to that observed in farming. This process of human redundancy will only continue because, as Amir Butt observed, machines do not tire, do not sleep, do not make mistakes. Which leads us to the thorniest issue of the AI revolution: fear of AI. This fear is very real among ethnic minority communities. And with just cause. It is something that Ruha Benjamin, a sociologist and a professor in the Department of African American Studies at Princeton University, has consistently underlined. A point highlighted in AI on Trial. Here is Professor Benjamin talking to Passwort’s Jane Wyatt.

Speaker A: So one of the things I try to do is to draw a spectrum from the most obvious harms to the kind of harms that happen precisely because people are trying to use technology for good. So it could be, for many people, the obvious harms, the way that technologies are used to reinforce, explicitly reinforce different kinds of social hierarchies. So there’s certain kinds of citizen scoring systems and ways in which our existing inequalities get reinforced and amplified through technology. And so for some people, that could be seen as the worst of the worst. But for me, what the worst of the worst is, is when we unwittingly reinforce various kinds, forms of oppression and hierarchies in the context of trying to do good. And so I describe this in the book as techno-benevolence. And so it’s when it’s an acknowledgement that Humans are biased. We discriminate. We have all kinds of really institutionalized forms of inequalities that we take for granted. And so here you have technologists that say, well, we have a fix for that. If we just employ this software program or just download this app or just include this particular system in your institution or your company, we can go around that bias. So it’s that arena of technology for good that when we don’t really think through and include the people who are potentially most harmed by various kinds of tech adoption, that’s what concerns me most, not the obvious forms of harm.

Speaker B: Can you give an example of how that works in practice?

Speaker A: Sure. And so there’s a number of organizations, hundreds even, that are adopting hiring algorithms instead of having like human resource people go through thousands of resumes maze. They’re outsourcing this to automated AI-powered hiring employment decision-making. And one of the things we’re finding is that those decisions often are just as sexist and racist as the humans who often make hiring decisions. But it’s because the training data and the things that are being looked for in the applicant pool are very much the same as in the past. So although there are many more data points being considered, it’s actually reproducing the same demographics as we’ve seen in the past. More men are being hired, more white people are being hired, more people with higher education, et cetera. And so this is the kind of thing where the danger is that we think that it’s more objective and more neutral than a human sitting behind a desk, but it’s actually hiding the various forms of bias under the guise of neutrality. And so when we just think about workplace workplace discrimination, we know through mounds of social science audits and various rigorous analyses that even when you don’t ask or know the person’s race explicitly or their gender explicitly, there are all kinds of cues, whether it be the name of the person, where their zip code is, where they went to school, that inform that decision. And so what we have is the smarter that the algorithms are becoming in discerning the applicant pool, they’re becoming more sexist and racist as a result. So intelligence, or artificial intelligence as we like to call it, often goes hand in hand with being more biased. It doesn’t get us around the problem. And so I think we have to have, as a starting point for a solution to that, we have to understand that we don’t just need technical know-how and prowess in terms of developing these systems. We need people who understand the social history behind let’s say in this case, employment discrimination, or in another case, it might be healthcare discrimination or criminal justice discrimination. We need the people who’ve been studying this and who are most affected by it to inform whether we want to even employ automated tools. And then if we do, then how we’re gonna go about doing that in a way that understands the coded nature of discrimination, that it’s often not always explicit.

Speaker B: Now, you mentioned zip codes there, and I know a large part of the book is concerned with the racial profiling that can spring out of just geographical location or how your name sounds and so on. Can you explain how this becomes part of the policing and predictive policing system?

Speaker A: In some ways, zip codes are a kind of low-tech form of coded inequity, because what happened historically in the US context when we outlawed very explicit forms of discrimination, geography became a proxy for controlling different populations and containing them in certain areas. Banks would use zip codes to create maps of a city and then invest in certain areas and divest from certain areas. And so one of the things I’m trying to trace in the book is the connection between this kind of low-tech form of coded inequity in which it’s a zip code, and then look at the way that that becomes folded into more high-tech form of coded inequity, and geography continues to be a proxy. And so, one example comes out of Facebook and the targeted ads that people often see. When you’re on Facebook, you see like the ads on the side panel, something maybe you were searching for earlier in the day, pair of shoes or something, and then you see certain ads for that. Now, the ability to do that is because advertisers can target different populations, people in different areas. They can use your search history to target ads, but you can also exclude people from seeing your ads. And so say if you’re a real estate developer or you’re trying to sell housing or something that has to do with a kind of real estate good, you can exclude, let’s to say African Americans from seeing your ads. And until very recently, people didn’t even realize that was possible and then didn’t make the connection to our fair housing policies that say if a human was doing that, that would be against the law. But because it was an algorithm that was facilitating that exclusion so that certain racial groups wouldn’t get to see your ad, then we didn’t have a framework for really identifying that as discrimination and part of this longer history of trying to essentially reinforce segregation in our communities. And this thing that allows us to buy and sell goods online also facilitates this form of coded inequity.

Speaker B: Princeton’s Professor Ruha Benjamin, and she is not alone. Bias in data has now been identified as a huge issue by a significant number of other academics, Such technological distrust, then, has some very real foundations, not only because of the AI monitoring of our data, but also because of the AI monitoring of camera systems, which have also proved to be racist. And it is not only minorities that have begun to take exception with AI. Soon, according to Warren, the distrust of the machines will spread as work opportunities are lost to robots, which has led some experts to warn of a similar reaction to that seen in 1811 when a revolt against mechanisation started amongst rural handloom weavers in Nottingham, Yorkshire, and Lancashire who felt they were losing their livelihoods to the textile mills. The Luddites smashed machinery and fought the army. At one time, more troops were involved in suppressing the Luddites than fighting in Wellington’s Peninsular War in Spain against Napoleon. The revolts famously became known as the Luddite Rebellion, and was the origin of the use of the word Luddite for a person opposed to the introduction of new technology. This is a predicament that many are forecasting that AI will deliver right across the economy, hitting every single sector, and for the first time, cutting into the livelihoods and careers of the middle classes that until now have been the champions of technological innovation. Even the practice of law and the administration of justice will not be immune, a point identified by the barrister Sandeep Patel QC, who specialises in technology, who prosecuted the LulzSec hackers and has researched the ramifications of AI at Oxford University.

Speaker G: That in about 5 years’ time, if not before, 70% of a lawyer’s work, judicial work which was done by a lawyer, will be done by a machine. I think it’s going to be as high as 70%. And apart from anyone who is a ballet dancer or a footballer or something like that, in the arts, AI will inevitably compete or will encroach in those areas of life, whether it be work and play and entertainment. You know that from your understanding of it. Already AI machines are replicating scores which mimic Mozart.

Speaker B: Patel told Peter Warren in the interview for his book. This nightmare scenario has already been recognised by the EU. In February 2017, the European Parliament passed a resolution stressing those employment fears. So exactly why do we want to develop this technology? According to Sir James Pace, UK Agriculture Secretary in 2010 under David Cameron, we have no option but to use AI because of population growth and global warming.

Speaker C: Farming through technology, well, it’s going to have to use technology to meet the huge challenge of food supply in the future. I don’t think as a society And certainly, I don’t think the government of the country, which actually includes the whole of the governing body, has fully understood what’s going to happen in the next 30, 40 years with the growth in world population, the impact of climate change on many parts of the world, its ability to produce food. And we’re not helping it back at home with some of our energy policies and other land-use policies, where the whole thing means we’re going to be— there’s a serious risk we could be short of food in the next 30 or 40 years, and I don’t think people have fully woken up to that. We’re going to see a continued advance of technology, the use of robotics, the use of satellites in growing our crops, robotics in both crop production and in livestock production, where we’ve now got automatic milking parlours, so where there is no man involved in milking dairy cows, that the cow puts itself into a machine and the machine operates and milks the cow and the cow walks out again. I’ve seen that happening. We’re seeing a dramatic change like that happening. We’re going to see more. We’ve got to see advances in science.

Speaker B: The AI revolution and its impact on what is the oldest industry in the world is a good example of the power of the new technology. Farming is being revolutionized by computer-driven mechanization. According to Professor Simon Blackmore, whereas once farms used to employ 20 men and use an equal amount of horses, all of those have been replaced by massive tractors driven by one man. Since 1950, the number of agricultural workers in the UK has fallen from from 491,000 to 108,000 in 2020. Now even those remaining workers are set to be replaced by AI because soon robots will be tending our fields and only harvesting plants when they need to instead of bringing the harvest in all in one go and risk possibly losing 60% of it. Something backed up by Professor Blackmore of the world-leading agricultural research centre Harper Adams in 2016, who said that already farm machinery is being roboticised, with combine harvesters able to perform sophisticated crop analysis on data harvested from field sensors that is then stored in the cloud. If a crop like corn is not ready, the combine will decide, rather than harvesting it, to cut lanes through it to dry it. A cultivation system that will increasingly see farming, once one of the largest employers in the UK and the US, become almost completely roboticised. We will become accustomed, according to Professor Blackmore, to seeing robot tractors tending crops using data collected by satellites, robots looking after animals, and drones weeding in the fields. As an example of that process, in 2017, Harper Adams and York-based Precision Decisions harvested 4.5 tonnes of spring barley from a field that had been sown and tended entirely by robots and drones. In April 2021, in a nod to this new future, a farm in Nacton in Suffolk took delivery of the UK’s first robot tractor. It’s not easy, this ethics game. We want the brave new world that technology delivers, but we also want to have the jobs that allow us to buy into it. These are all questions that we have tried to puzzle out in AI on Trial. Did we succeed? Perhaps we got a little closer to the truth by interviewing some of the world’s top AI experts. Perhaps, as the barrister Sandeep Patel as Mattel points out, we should have interviewed an AI. To find out more on the subject, log on to www.futureintelligence.co.uk and listen to previous programmes, or even order a copy of the book. You’ve been listening to Password on Resonance FM, presented and produced by me, Blue Bufferee, and written by Peter Warren. Thanks for listening and goodbye.

Speaker A: This program has been brought to you by Resonance FM.

Speaker B: If you like what you heard, please support our work by making a donation at resonancefm.com/donate.