PassW0rd – 8th February 2017

Speaker A: This program is brought to you by Resonance 104.4 FM. If you like what you hear and want to support our work, please make a donation at fundraiser.resonance.fm.

Speaker B: Hello and welcome to the Password Technology program on Resonance 104.4 FM. This month, in a historic moment, the European Parliament is to begin the first ever debate on the position of robots in our society. And the legal status that they should enjoy. We are quite literally living in history. It might sound like science fiction, and the European Parliament were panned in the post-Brexit British press for a moment of madness, but with autonomous cars already working around the world, robot journalists writing articles for news agencies, and robot stockbrokers affecting our lives, the European Parliament is being hailed by some as a long-sighted organisation. Here is Maddy Dalvo Steyr, MEP, the rapporteur to the European Legal Affairs Committee, whose report provoked the debate in the European Parliament, to tell us why she wants robots to be regulated.

Speaker C: Yes, the report is on rights and rights for robots. It’s a report on how society will interact and what legislation we need to allow robots to develop and to deploy on the market.. So it’s a report on liability. Who is liable for the actions done by a robot? It’s on privacy, on dignity. It’s about ethical, it is about standardization, and it is about the societal implications of robots. We see that robots, the robotic industry is developing very fast, and so there will be, there are robots on the market. There will be more robots. To have a regulation about liability, for instance, in driverless cars. And the next step will be that probably robots will become more intelligent with deep learning and that they will be more autonomous. And so we should start a reflection on how will we interact with these more sophisticated robots. And we will propose different options. And say, well, let’s look at both of these different options. So it is not at all about giving rights to robots.

Speaker D: What are the societal implications?

Speaker C: I would say my first concern is to raise awareness that the development and deployment of robots and of artificial intelligence will have consequences on our societies, and that we need a debate not only between experts, policymakers, experts, but also the large public on how do we want to robots to change our lives. First of all, that robots should make life easier for human beings, that it is a very intelligent device which can bring benefits to humanity, but that we do not want robots to do harm. And that if robots— because you cannot prevent a robot from causing an accident or doing something wrong.. Then in this case we have to look how we can compensate this harm, this damage. And so we need to identify who is liable for the— for instance, if a robot causes an accident. And for me at present it would be the producer. The programmer of the robot is liable for the action.

Speaker D: But isn’t one of the issues there going to be that quite often what we see with technology is there are unintended consequences?

Speaker F: That nobody foresaw.

Speaker C: And that’s why we need to put into place a framework for at least to prevent the maximum of undesirable consequences. And this is very much about ethical principles. That’s why we promote the idea of creating charters, ethical charters, and to put into place ethical committees and to have a public expertise also on what is done in the industry and in the labs. That we can at least understand what is happening.

Speaker D: But a lot of people are saying that you cannot put controls on scientific development, that you’ve got to just let it develop and that the market will decide.

Speaker C: I hear this argument very frequently. I don’t want to put control on research, but I believe that policymakers have the right and the duty to regulate the market and that we can, we should at this think on what we will not allow to be on our market. And so this is a different political option. But at least myself, I fight for the regulation of the market. Everything which is feasible is not necessarily desirable. And I don’t want humans to lose the control of what is happening.

Speaker D: Many people will say, hang on, politicians have never ever been in control of technology. Why are they suddenly deciding now that they need to?

Speaker C: What is the threat that we’re I’m not quite sure that politics does not have the control. I agree that what has been discovered normally is deployed, but I believe that robots or the development or the perspective of artificial intelligence will have a very, very important influence on our societies, and that at least we should debate about it and try to understand what happens and try to influence, because For me, politics is about human beings, it’s about organizing the society, it’s about allowing harmonious development of our society. It’s not just about allowing technology to develop. Much more humanistic, or I don’t know how to say, human approach to all these issues.

Speaker D: Many people will say that’s the right approach, but a lot of the economists are now saying that a third of all jobs will go to robots and the robot can do just about anything that people can do. Surely we’re just now redundant and that’s what we’ve got to accept. There’s too many of us and the robots are going to take over.

Speaker C: Well, I have to say that I am not a scientist and I do not know what the future will bring. But precisely because I hear all these arguments, then I say, well, let’s develop different scenarios because some people, it will be fine for humans if we don’t need to work anymore. But then what, how will we pass our days? What will we do and how can we assure that human beings can have a decent life and that they have an income which allows them to live in dignity? So this is the other way around. And then I, when I look at the different studies, well you have half of the experts that say that jobs will be massively destroyed and then we have to look how exactly to finance our social system or public service. And the other experts say that robotics will create also new jobs and that a robot, at least the robots that are for the moment in the labs, that they will never be able to do all the creative things that humans can do.

Speaker B: Maddy Dalvaerstere, MEP, the rapporteur to the European Parliament’s Legal Affairs Committee, who says that it is essential that robots work in the interest of humanity and that there is a need to define what the position of robots will be in our world. So what is the problem, and what is it about this world of AI that is causing so much concern? Next month at CeBIT, the world’s largest computer show in Hanover, a former Cambridge academic, Professor Michael Kosinski, will talk about the intelligent analysis methods that can reveal your personality that many people claim allowed President Trump’s researchers to home in on key voters and win the US election. But according to Professor Neil Thurman, of Munich’s Institute of Communication Studies and Media Research.

Speaker D: He says not only can intelligent algorithms target you, they can even write directly to you.

Speaker A: So we’re looking at a variety of areas in which algorithms can impact journalism, and it’s the sort of whole of the news cycle, if you like. So from when journalists find information that might turn into a story, how they research that information, verify that information, how stories written and how they’re distributed to the audience.

Speaker D: Can an algorithm write a newspaper story? Do we need journalists any longer?

Speaker A: It certainly can write a news story, a text story, and actually it can also create a short video news story as well. But algorithms are fairly limited at the moment in what they can do, and so the sort of stories that we get produced automatically by computers are stories of company results, sports results, that sort of thing. They’re stories that provide information quickly that is useful for some people.

Speaker D: There are algorithms who are currently generating news stories now?

Speaker A: There are. One of the big news agencies in, in the US, AP, they’re producing several thousand stories every quarter on company earnings results. Using automation technology. So they have data that’s coming in, they have a program that takes that data and then turns it into natural language. Journalists have been involved quite heavily in programming that software, but now it operates autonomously, produces probably over 10,000 stories a year. And this is also rolling out in other news organizations. You know, a large number of other news organizations are also using this same technology.

Speaker D: Should we be worried about that? I mean, if there are algorithms that are providing us with a news feed,— surely we can be terribly controlled by this. We can just get the information that somebody wants us to see.

Speaker A: There’s not going to be very much choice, is there? That is an issue because, you know, this technology can greatly increase the amount of news information that’s out there. And we see this with some of the social media bots that have been implicated in, you know, in the last US presidential election, supporting one side or the other. Putting out a lot of information on social media. So there is a concern, I think, that technology will allow news organizations, will allow individuals to produce very large quantities of information that they can put out into the web or online for various purposes. It might be for gaming search engine results. It might be for gaming news feeds in social media. So we do have to be concerned about this, I think.

Speaker D: Obviously, what you can do, as has been claimed in the US presidential election, it was claimed that the president’s son-in-law, Mr. Kushner, was using social media to home in on particular groups and make sure that they were convinced of Mr. Trump’s views and that they should vote for him, wasn’t it?

Speaker A: Big data is certainly being used in political campaigning. I think big data and algorithms do also offer some opportunities for journalists and news organizations to check, verify information that they find on social media. So it can work both ways. So, you know, technology is also increasing potentially the amount of information that’s on social media, but it’s also providing journalists with a means of filtering that. So that’s another part of the project that I’m working on, and that’s looking at how build tools and how we can evaluate tools that allow journalists to find stories within social media and even to verify, you know, the contributors on social media. So, you know, is this person who’s produced this tweet trustworthy? You know, what information can a piece of software provide a journalist to make that kind of judgment?

Speaker D: Well, this is going to be a big issue, isn’t it? Because there also now exist algorithms which go out to combat heaps speech. So theoretically, what we’re doing is we’re going into an algorithm arms race.

Speaker A: I think that is a— I mean, that’s a very good analogy. So we’ve got this algorithmic arms race. On one side, we have people trying to game social media to promote their message, and then the other side, we have people like journalists who, you know, to sift this information, find out what’s trustworthy. And, you know, they need to also arm themselves with tools in order to do that, because if you think there are several billion tweets, you know, sent out every year, it’s impossible for a journalist to sift those manually. So, yeah, an arms race is a good analogy.

Speaker D: Do you think that there should be almost a declaration of interest by an algorithm that it is doing what it’s it is doing, that it says to you, look, I’m trying to affect what you’re thinking, I’m trying to go after your vote. Do you think that that’s something that we require?

Speaker A: I think there are ethical issues in terms of transparency with this technology, particularly as it can be used in news and journalism. So, for example, there are some studies that have shown that if you label a story as being written by a human, or if you label a story as written by a computer, then people have different reactions to that story. They might, you know, have more confidence in the human-written story. So as we see algorithms start to write more and more news, there may be a temptation for news organizations to label the stories as being much in the same way as we sometimes find newspapers putting a byline of a reporter on a story that’s actually been produced by a news agency. So I think we need to watch out for these ethical issues because it’s important that we know where information comes from.

Speaker B: Professor Neil Thurman on the robot journalists that may be after your vote. Last week, all of the leading scientists working on artificial intelligence sent out an open letter to world leaders saying that there was a need for controls on artificial intelligence. The signatories included Professor Stephen Hawking, Elon Musk, and Professor Murray Shanahan, an occasional contributor to this program. I also signed it on behalf of the listeners to the Password program, a courtesy of counting for something that our society affords me. But it seems that there are legal academics out there who say that because of the way our economy is developing, we need to do the same for artificial intelligence programs. Here’s Surrey University’s Professor Ryan Abbott on why he wants robots to be given the right to patent their own inventions.

Speaker G: Robots are taking over from people in a wide variety of fields and on an unprecedented basis. Computers are already working as doctors, lawyers, scientists. This requires us to rethink the sorts of laws and policies that apply to these computers, because computers taking over from people has really very substantial effects on the innovation we get, the harms people suffer, the economics of industries, and in potentially rendering humans technologically unemployed.

Speaker D: Okay, so if we are going to have rules, what will those rules be?

Speaker G: There isn’t a consensus yet on what those rules will be. Some people think that computers should have rights like people. Some people think that’s all ridiculous. Most people don’t seem to realize that computers are already doing these things and out on the streets. Self-driving vehicles are already being used in cities in the United States by Uber.. And right now we don’t have a consensus about what legal and policy frameworks we should apply to these machines. And that’s one of the reasons why the EU is very right to flag the importance of these issues.

Speaker D: You’ve suggested that robots should actually be given the right— patent rights to their inventions. It’s really— I mean, people say that’s ridiculous. How can you possibly give a robot rights?

Speaker G: Well, you wouldn’t give the robot rights in my proposal. I’m proposing that we acknowledge robots as patent inventors because That’s what’s actually happening in practice. We’ve had machines for the past 20 years that have been able to generate patentable subject matter essentially independently of people. And I’ve argued that giving— acknowledging computers as inventors would incentivize research and development because it would incentivize big companies like IBM and Google to create creative machines.

Speaker D: But theoretically though, if you’ve got that creative machine, anything that creative machine invents belongs to IBM or Google?

Speaker G: Well, yes. I mean, a computer couldn’t really own property. It wouldn’t be motivated by a patent the same way that a human owner of a computer would be. And so I think it makes a lot more sense if what you want to do is incentivize innovation to give the actual rights to the owner of a computer, but to acknowledge the computer as the inventor to prevent people claiming credit for work that they haven’t done and for making sure in research collaborations ownership rights are worked out appropriately.

Speaker D: You say that the EU is doing very well in considering these things now. A lot of people would find this really, really quite shocking that we’re even thinking about it. You say disruptive. Disruptive effects that you can see from robots and artificial intelligence?

Speaker G: Well, I think the disruptive effects are really going to be profound, and for people who don’t think this is something we should be talking about, I don’t think they’re aware of the state of the art of artificial intelligence. Right now, about 3% of the population is involved in providing transportation services, and within the next 5 or 10 years, we’re going to see a massive rollout of self-driving vehicles. That could render 3% of the working population unemployed right there, which would be a tremendous disruption.

Speaker D: It’s not just in transport, is it? We’re looking at profound changes in things like agriculture, for example.

Speaker G: Yes, and I think agriculture is an excellent example. If you look at the year 1900 in the United States, about 40% of Americans worked in agriculture, and in the year 2000, only about 2% of Americans worked in agriculture. But we haven’t had a 38% unemployment rate. You know, the introduction of machines has allowed workers to be far more productive, and in response, people no longer in agriculture have been able to contribute to other sectors of the economy. So I think in the short and medium term, it’s likely that automation won’t cause a net reduction in jobs, and that really our task as policymakers, people who are setting up legal frameworks, is going to be to ensure that We provide support to people who need new jobs. Though in the long term, it’s not clear what a computer can do that a person can’t. And yes, it’s not just in driving right now. Computers are already working as doctors. So IBM supercomputer Watson is now working with doctors to provide treatment regimens for cancer patients. And it’s not going to be long until Watson is outperforming human doctors. And why shouldn’t it? Can access every medical study that has ever been done. It can treat millions of patients directly and remember all of them perfectly. It never gets fatigued. And what’s going to happen when people have a choice of seeing a robot doctor or a human doctor, when, by the way, the human doctor has much worse outcomes?

Speaker D: Some people are talking about the, the ability for computers to commit crime.

Speaker G: How does that happen? Well, for example, autonomous computers are prevalent in the financial services industry where a lot of trading is done by software. And these software programs can be designed so that they’re self-learning and that they actually go out and engage in behavior that is unpredictable to their software developers. So, for example, a trading algorithm could come across some insider information, make illegal trades, and generate profit., and then you’d have the question, you know, who do we hold accountable for that? Well, you can’t lock a computer up. The computer isn’t worried about committing a crime. Would you go after the software’s owner, its developer, the manufacturer of the software?

Speaker D: These are unanswered questions. But in those terms, though, isn’t it going to be incredibly difficult to start putting ethics into machines? And I’ve spoken to a number of scientists about this and they just hold their hands up in horror at the mere thought of it.

Speaker G: Well, that is certainly going to be one of the very important problems to dealing with in creating the right frameworks for artificial intelligence.

Speaker B: So robots, it seems, may even be able to have patents on the things that they invent with intelligences that are faster better at things than us. In fact, it would appear there is not much that we can do that the robots can’t. Work is going on even now to come up with a team of robot football players which aims to beat their human counterparts in the Premier League. It’s developments like all of these that are concerning Mary Honeyball, this time a London MEP who wants a legal framework created for just what robots will do. Speaking from a Delhi conference about the rights of sex workers, she wants humans to take control of where the robots are going.

Speaker H: Well, I’m not sure we can stop it, but what we can do is develop an infrastructure for it itself. And what we were looking at in the report is some of the legal questions, because it was a report that went to the Legal Affairs Committee. Of the European Parliament. So that was its focus. And we looked at certain key things like the basis of which was that robots should work for us as human beings, not the other way around. So how you do that, how you make sure that robots don’t harm people. And so that was the basis of it. And what kind of legal structure, what sort of legal system you need to deal with these things. But the main thing I think, the basis of all of this was that we want to set up some kind of system and structure that allows member states of the EU to come together to cooperate because this is not something which will be confined to one nation state. This is an international phenomenon and it’s something which the EU needs to do as a whole because this will travel. This isn’t just, just Britain or Germany.

Speaker C: This is—

Speaker H: everyone will be involved.

Speaker D: So we therefore need international solutions. Do you not think though that if we put any controls on science that we’ll be hindering scientific development? Won’t the scientific community say you’re being Luddites here?

Speaker H: We’re not putting controls on science as such, and I would never want to do that because I think scientific innovation is very, very important for all of us in a whole lot of ways. And all the developments that we’ve been talking about that have taken place have happened through scientific innovation. But what we also need to do is make sure that that is used in a way that is constructive and positive and works for people. And that’s, I think, where we come in.

Speaker D: I don’t see those two things as mutually exclusive.

Speaker F: You’re saying no, can we actually take some control over where our lives are going?

Speaker H: Well, I think so. I don’t think we should in any way stop the the scientific advance. I would absolutely say we shouldn’t do that. But I think what we should do is, I mean, I think the idea of groups of people who look at this and make decisions about where it goes is a useful one, and I think that’s what politicians would see themselves as doing. I think others should be involved as well, and I think the scientific community should be involved in those kind of debates as well, so that it’s done for the best reasons and the outcomes are mutually beneficial and beneficial as much as possible to everyone as possible.

Speaker D: Quick final question. This isn’t just about robotics, is it? This is about the extension of technology.

Speaker F: This is about the smart city too.

Speaker H: It’s regrettable that in the past with some technological developments such as nuclear capability, these kinds of discussions never took place, and there wasn’t really public input. It was all done by a few people, not only scientists, but all done very secretly. I think that’s absolutely the wrong approach, and that we need to have a real public debate about this and take it at a pace which people feel comfortable in, the way that people find beneficial to them. I think that’s really important, and I do think that that is really what the ethical discussion and debate is all about.

Speaker B: So this is not just about the robots working in our factories. It’s about the intelligent computer programs that will control our cities and our lives and tell us just what we should and should not do. Two weeks ago, a female US journalist published a review of her experiences with a sex robot performing in a male role, a development that came as a shock to Mary Honeyball. Such potential intimacy with machines can only increase, and it is this that concerns Harvard University’s Professor Sheila Jasanoff, the author of The Ethics of invention. According to Jasanoff, the issue of hybridization—how far we allow ourselves to become part of the machines themselves—is one of the key areas. Fusing with machines to become cyborgs and having chips inserted into our brains, a developing technology that we heard about in last January’s Password program, are all things that need to be regulated as closely as work on the human embryo. In the US, rights are being given to mobile phones by the courts because the courts have decided that they are an extension of people’s minds. For Jasanoff, this is not an issue of rights for robots, but about rights for humans. People, she says, need to be at the heart of things.

Speaker E: One is deciding that that control therefore means wanting more autonomy to robots, and secondly, that the language that’s being picked up so quickly is the rights language. And we could talk about both of those things. That is the first thing, you know, why think of autonomy for robots instead of thinking about protections for us? If what’s being worried about is the production of entities that are going to take away our autonomy, then it seems to me that the thing that we should what we should worry about is our autonomy, not theirs. And secondly, since in law and ethics, there’s been for a long time a question about the appropriateness of the rights framework as itself the right package for— or the right envelope within which to talk about how to grant responsibilities and how to exercise control. And on the whole, European legal systems have have not been particularly rights-focused. The common law systems have been much more oriented to rights at the individual level. So even that, I mean, the picking up of the rights language without debating the pros and cons of rights strikes me as also a little bit odd.

Speaker D: There’s another interesting point too, which is that— and I’ve found this lady’s name, Maddy Delvaux-Sterres— also been saying that there is a need to actually put ethics into the programming of the systems, which is quite—

Speaker F: a lot of people—

Speaker E: I mean, I understand all of that. I just— the strange thing is that this is all being talked about as rights for robots. I mean, so where would I begin the analysis? First of all, I would say, you know, it’s sexy to talk about robots, but we’ve already been talking about The fact that human beings are being increasingly hybridized with various kinds of add-ons. I mean, it began maybe with physical prosthetics, but already we’ve moved over into a domain where, for instance, the US Supreme Court, the current Chief Justice is on record saying that we are essentially some kind of cyborgian entity together with our cell phones.. And cell phones are serving as adjunct memory banks in a lot of different ways. I mean, they store records of who we’ve been, who we’ve been with. They store our preferences in keeping track of books and movies and sites visited. And of course, they have a lot of information about our contacts and our calendars and so forth. And this case arose— I actually talk about it in the book— the case in which the Supreme Court held that you could not have a warrantless search of a cell phone. The reasoning went in the direction that these cell phones are essentially extensions of our own minds and brains, and therefore whatever protections we have against the government arbitrarily coming into our minds and brains are protections that should extend to this add-on entity. So robots are becoming increasingly capable of storing data about us and becoming partnered in a sense with us so that if they’re serving— I mean, if they’re service robots of different kinds, they take over certain kinds of human properties. And therefore, if we are worried about a system in the world where the kinds of walls of privacy, walls of protection we’ve created around the autonomous human being is suddenly penetrated into or intruded into because our robotic extensions do not have the same kinds of protections, I still think that the language of giving rights to robots misses the central point, which is that it’s the human-robot hybrid that is entitled to protections and not just the robot itself. Now, I mean, I can see another direction of development in which one might start to worry about the robots themselves, and that is when consciousness inside the robot begins to develop in such a way that, although it’s made out of artificial materials and not biological, nevertheless, it achieves a kind of consciousness. And this is something that you’re seeing cutting across new and emerging technologies, because in the sort of wet lab bio sector, as people doing synthetic biology get closer and closer to developing proto-organs in labs and in dishes, there’s a beginning of stirring of worry about what happens when these organoids, as some people are calling them, they’ve even been called mini-brains, which is very, very hypothetical at the moment, but if they start developing capacities that we associate with protected things in humans, I mean, like the capacity to feel pain or the capacity to have memory, then what do we owe to these entities for which there currently is no system of rights and obligations in place? So, if AI and robots cross over into that space, then I can see the autonomy language being triggered. But much more of the debate is around the fact that if you seize the robot, you’re seizing a piece of a person, just like the cell phone.

Speaker D: Right. Now, on all of that, which I do agree with, the point that I think is In terms of actually giving status, do you think then that there is an urgent need to actually come up with some new definitions? I mean, in a sense, yeah, we were talking about in the UK where we don’t even have a Bill of Rights here for people. You almost need to have a definition of, you know, what humanity is, and then you need to start working from there.

Speaker E: Is that the point that you’re making? Well, not what humanity is, and moreover, I’m quite against foundationalist definitions of that sort, because I think that as our imaginations change, those definitions also change, and I want to leave the freedom for human creativity to push at the margins of the human. I mean, you know, if you have just a teensy bit of history, you know that people that we today regard as as absolutely our co-equals, were not regarded as humans.

Speaker B: Harvard University’s Professor Sheila Jasanoff on why the European Parliament is right and why we need to start seriously looking at our relationship with the machine from a human point of view. Jasanoff is not alone. Indiana University’s Professor Fred Cate is another leading academic in the area of technology and ethics. For Cate, an advisor on ethics to Microsoft who’s also advised the US government and US intelligence agencies on privacy, there is a very real need for the technology industry itself to realize that people are beginning to become tired of the rate of change that the technology industry is forcing upon them in pursuit of profits. According to Kate, MEP Maddy Dalvaux, has got it right. The issue is the relationship of people to the technology companies, and we do not want to be enslaved to the companies that are selling our technology to us.

Speaker I: You know, in many ways I’m not an imaginative futurist. I very much, maybe it’s because I’m trained as a lawyer, I think in sort of the present. Right now, biggest challenge that we face as humans in the industrialized world is getting control of our technology. Not trying to empower our technology with independent legal rights. And so again, I would focus on the design, the purchasing, the deployment of technology, and the users of technology, not on the technology itself. I don’t want to have to give my cell phone days off every week because of its rights. It might be a good idea to give it days off every week because of my rights and the fact that I should be free profit.

Speaker D: I agree completely. And I think that’s something that we should all embrace. Do you think that given we’re having this rather worrying discussion about technology, do you think that with things like, say, the smart city, for example, that technology should become a political issue? Do you think that we should put technological innovation on manifestos when we vote in leaders?

Speaker I: Well, I don’t know about that. I mean, I would say in some ways I think technology has always been a political issue because it affects in pretty significant ways the balance of power between individuals and organizations, and between individuals and their governments. And so we’ve worried about technology used in surveillance. We’ve championed the role of technologies as a way of making governments more accessible or providing greater oversight. I don’t think the political aspect here is necessarily new. I guess I think this is an area where we might need some help from law and political processes. But frankly, more than anything is we need people, both as individuals and as, you know, people who make decisions for organizations, to be just a little bit less enamored of the technology. This sort of gee-whiz mentality, if it’s new it must be good, and instead to make more rational, sound judgments. And there’s some practical sort of mundane aspects of examples of this. I mean, in the United States for many, many years telemarketing was something that the public hated and industry loved. And the public hatred eventually spilled over into law that effectively prohibited it. Had industry exercised any independent judgment about the use of that technology, they would have said, look, we are really angering our customers and we should cut it out.

Speaker D: But they didn’t, and now they have no choice. That seems to me, though, that one of the things that is going to happen to technology is that there will be a move pushed by people to say, actually, we’ve had enough. There may almost be such a revolt in the way that you see people voting for things like Brexit or for a new president who will change the way of doing politics. If the industry doesn’t stay on top of what people want, because the industry does tend to try to convince people it wants technology rather than the other way around, doesn’t it?

Speaker I: It does, and I mean it’s been incredibly successful at it. I mean if you If you look at some of the most widely used technologies today, they are all things that when first debuted, certainly experts would have said, “No one’s going to care about that.” Facebook would be a perfect example. Why do I want instantaneous access to all of these people? And now, over a billion users. Obviously, the experts, which would have included me at the time, would have been wrong. On the other hand, it is a little bit of a slippery slope, which is as we become more dependent on technologies. And frankly, this is, you know, as they fight, you know, Amazon and Apple and others about what platform are we going to use, and can they hold us hostage to that platform so that we can’t get away from it. I think this is going to become a bigger and bigger challenge. So the provider of broadband service wants to control that relationship. The provider of the apps wants to control it. The provider of the platform wants to control it. And at some point we might find that some traditional legal concepts like antitrust or things like this might actually help sort of re-empower consumers to say, look, there can be benefits from technology, but I don’t want to be enslaved by my technology or by the companies selling it to me.

Speaker D: Okay, one thing that we haven’t covered is unintended consequences. People are now talking about autonomous cars. And one of the things is it’s become a joke now. I’ve heard it on a comedy program where, you know, the cars are discussing who they’re going to hit. And, and right, it depends on, you know, the robot that ultimately comes out top is the one that’s programmed according to Nietzschean principles. That thing about unintended consequences How does one deal with that in all of this mess?

Speaker I: Right, so I mean, first of all, we always have unintended consequences. I mean, with absolutely everything. With, you know, the moment that humankind rubbed two sticks together to get fire, we’ve had unintended consequences from technologies. And obviously those are going to become perhaps greater as we become more technology dependent, and also more unintended as we become less aware of how our technologies work. Part of the key is to stop having so many of them be unintended. In other words, let’s think through what’s going to happen. Of course cars are going to end up ultimately making choices. I mean, for years in driver’s education courses— I mean, when I took driver’s education many, many years ago in California, we discussed the challenge of making strategic choices as a driver. Do you hit the child in front of you, or do you drive the family of four off the cliff?. And that part of being a good driver is the ability to make very hard choices very quickly. Now, if we don’t think computers are going to have to deal with those same issues, we’re just crazy. So those should not be, at minimum, unexpected choices. What we then should be thinking about is how are these being programmed? What are we talking about as a society? What are regulators looking at in this consequence? I mean, for me, The best example of this is the question right now, we apparently can’t secure any technology. There’s no data or system that we can actually make foolproof. So before we put these technologies 100% in control of cars and airplanes and trains and other forms of transport, maybe we’d like to address that issue in a more frank and aboveboard way rather than wait till two trains are driven into each other by technology and then say, oh, that was an unanticipated consequence, that’s a fully anticipated consequence.

Speaker B: So now let’s figure out how to minimize the risk of it happening. So for Kate, we need to end our love affair with technology because our affections are not being reciprocated. Our technology may be flagging up flowers for Valentine’s Day, but there’s no thought behind them. Even more importantly, as Kate points out, given the computer security issues that are undermining confidence in the technology industry. We should make sure that the technology that can have such a profound impact on our future should be properly secured so that we can feel certain that it would do what we tell it to. For Mark Deem, a technology lawyer for the law firm Cooley, which represents some of the world’s top technology companies, that’s why we need to build some accountability into the system. We are now making. For Dean, this is all about responsibility, and the buck doesn’t stop with the robot, it stops with the manufacturer. A potential liability that could quite easily put the brakes on the pace of change that we are seeing. As Kate points out, fire is a good servant but a bad master, and being held liable for an unintended unintended consequence could be fatal for the manufacture of any artificial intelligence.

Speaker J: I think what’s really driving a lot of this is trying to make sure that we have some degree of accountability for what’s going on at the moment. It’s slightly unfortunate that when we hear the word personality, we immediately try to assume that they’re trying to give or afford robots precisely the same rights and protections that we might have as persons or as human beings. I don’t think that’s really at the core of what they’re doing here. I think what they’re really trying to do is, I think firstly, they’re trying to make sure there’s accountability for those that are developing the technology which is leading to artificial intelligence and the creation of these robots. And secondly, I think they’re paving the way for what happens when these super intelligent robots are created, to make sure that we don’t get in a situation where people can just abdicate responsibility for what has been created. And actually pass the buck and say, well, this isn’t for us, we’re not responsible, it’s the robots that should take responsibility. Because actually, if that’s where we end up, we’re in a very dangerous place indeed.

Speaker D: But I mean, that’s an interesting point, isn’t it? I mean, this call for robots to have responsibilities, there can’t be any legal precedent for that at all.

Speaker J: Well, you say that, but there are a number of different precedents we have for these sort of things. We have the concept of legal persons as a matter of law. So, you know, take a company for example. A company is able to enter into contracts in its own right. It’s able to assume obligations in its own right. This is the way that the law has developed to create what are called legal fictions, if you like, to ascribe a personality to something that obviously is not a person.

Speaker D: Is this why there is a discussion for making robots liable for their actions? And can that work?

Speaker J: How can a robot be liable for its actions? It isn’t aware. At the moment, we can certainly be in a situation where we’re dealing with narrow intelligence or general intelligence. To make those people that are responsible for manufacturing the product, coding the algorithms, possibly programming the robot, we can certainly come up with rules which would help make them assume some responsibility for their actions. The really tricky part comes from when you then try to place those responsibilities, those burdens onto the product that has been created. And that’s really where a lot of effort and attention is going to have to be paid in the years to come as how we achieve that.

Speaker D: Surely that just means that where we make place the liability is going to be on the manufacturer of the robot or the programmer of the robot. It must be the creator of the robot that has the liability because they’re the ones who told it what to do.

Speaker J: In a way, although I think that the answer is going to be that. I mean, certainly if you’re talking about a product that has come out of the box, then certainly the manufacturer probably has to take a little bit more responsibility than other parties. But as soon as that’s going to learn and process data that it is ingesting itself, then there are whole other factors that come into play. What is the quality of that data that has been received? Received by that robot, who is responsible for that data that has been ingested and dealt with and programmed and computed in a certain way. The whole position becomes very complicated and very nuanced. And so it’s not really as simple as just saying that the manufacturer, the designer, or the programmer automatically should assume strict liability for—

Speaker D: you go to this point of data in, data out. That feed for where the robots got the data from where the liability is. Does this possibly argue then for the need for robot lawyers and robot policemen? Because it can surely only be a robot who can understand where the information came from in the first place and where the mistake was made.

Speaker J: Undoubtedly there will be a whole swathe of individuals who are trying to become specialists in precisely this area of law. What is really quite important though is that we end up with a framework which enables there to be some sort of audit trail or accountability. For the various different actions and the various different ways in which algorithms are processing data. And that, I think, is precisely what the European Union Parliament is trying to do at this moment by the journey that it’s starting by debating some of these proposals.

Speaker D: Okay, one of the issues seems to be to make robots different from us.

Speaker J: Why do the politicians think there should be a discrimination? I don’t necessarily agree with that as a sort of conclusion. I think actually it is very difficult to, to try to treat robots and humans differently if they’re trying to perform precisely the same functions. We have to look at what is trying to be achieved as a matter of law, be it entering into a contract or conducting some form of business in which that has taken place. And it’s only then can we actually start to work out whether or not that there should be any discrimination as between the two.

Speaker D: But for my part, I don’t think there should be. But I’m sure what he’s saying is that this is— if you’re going to discriminate discriminate between us and the robots, you want to say, okay, a robot did that, then in a sense that does become as discriminatory as saying a person of a different race did that.

Speaker J: Well, it does, and it’s precisely that sort of blame game that we have to stop by putting in place framework as to who is accountable for the actions of a robot.

Speaker D: Exactly, because as you know, hitherto we always used to see printed in newspapers computer error caused.

Speaker J: It’s something that we like to do, is blame technology. It’s almost a knee-jerk reaction when anything goes wrong. It’s the, the leaves on the line in the technology world, isn’t it? It’s the way of just trying to offset liability and apportion it to someone else. And that’s precisely what cannot happen in circumstances where we have these very intelligent machines that are in operation and that are going to be very much part of our everyday lives, we have to have a way of being able to apportion responsibility, work out who has legal liability, and to be able to assess where the causal link is between an act of that robot and the harm that has been caused.

Speaker D: But that leads us very, very nicely to the next question, which is, yeah, any intelligent robot will just say, hang on a minute, you didn’t ask me about drawing up this framework. I reject it. I’m not having anything to do with it.

Speaker J: I’m not to blame, you are. And that is, I think, why we have a sort of call in some of the publications that I’ve seen that have been put out by the working group in the European Union for there to need to be a moral code, if you like, as to the way these things are developed. And the ability to put a kill switch into any device that is created like this, so that to the extent that they’re not playing it by our rules or by the rules that we have imposed on them, that we have an ability at that stage to stop precisely what’s going on.

Speaker B: So for Cooley lawyer Deem and the politicians in Europe, it’s going to be an essential requirement to be able to pull the plug on the machine if we need to and regain control. But doing that could in itself cause us problems, according to Manchester University philosophy professor John Harris. If the machines that we are building begin to remotely achieve something that could be seen as real intelligence, then that will accrue rights because that’s the way that our legal and philosophical system is set up. Harris sees this as a slippery slope. The European Parliament’s initiative to confront this thorny issue will inevitably lead to a discussion on human rights for robots, something the European Parliament’s MEP Maddy Dalvaux complained about, accusing the UK media of exaggerating the European Parliament initiative.

Speaker F: We’re having the debate about whether or not to give rights to machines because we are the sorts of creatures that have come to believe, for very good reasons, that creatures of a certain sort, like most human beings, have rights and interests worthy of respect. Now, at the point at which we have created machines that have interests and the rights that flow from those interests worthy of respect, then, as it were, consistency demands that we extend to them the same concern, respect, and protection that we extend to one another.

Speaker D: One of the reasons that the European Parliament seems to be wanting to do this is so that they can actually make these machines responsible. They want to find somebody who they can sue.

Speaker F: Do you think that’s the right way to go? It may be difficult to make the machines responsible. What the best we can hope for is that they will come to see what— I think many human beings are now really seeing that creatures of a certain sort just demand, and not because they ask for it, but because their nature demands that we respect it. That is to say, they have autonomy, they have their own decisions, and they want those decisions and interests respected. But if we think that we can hold them accountable at law, then we’re going to need to create a whole different way of relating to them. And if we want to be able to sue them, they’re going to have to have their own resources. And so it’s going to be a very complicated parallel society that we would be creating of non-human persons, mechanical machine persons. Persons who would have analogous rights and interests. I think it will be like, actually, because they will be so different from us, it will be like the first attempt to, as it were, develop enforceable treaties that nation-states could have, one with another, so that they would respect the rights of other nations to the freedom of the citizens of those other nations.

Speaker D: What are the problems that we’re facing from this development of all of these very, very intelligent systems in our society? And also, people don’t really realize that they’re here already, do they? They don’t realize that the car can often be listening to them, that other things can be listening to them, because it’s trying to find something that it thinks that we might want.

Speaker F: It can certainly be hearing us. And if it hears certain keywords, it may turn on or turn off various systems that the car might have. If it detects that we’re driving erratically, it might take over control of the car. At least that will come soon. Whether it’s listening in the sense that you’re listening to me and I’m listening to you is a very different question, and we haven’t got there yet with machines.

Speaker D: Well, it’s been suggested that machines and software has already started to do things that are criminal, that basically it just does what it’s told to do.

Speaker F: Do you think that that can actually be defined as criminal if it’s doing something that it’s been told to do? Well, it’s only criminal if it doesn’t have free will. The question is whether— and if machines just can’t help themselves. What we’re telling them to do, then they won’t be criminally responsible.

Speaker G: It will be the person who did the telling that will be criminally responsible.

Speaker D: You could have an unintended consequence. You can say to a piece of software, your job is to make me as much money as possible. The software won’t know that it’s actually doing something that is criminal. It could be, for example, engaged in insider trading with other machines. But, uh, um, it is actually doing something criminal.

Speaker F: You haven’t told it to do something criminal. Where’s the liability in that? Normally in law, an important element in any crime is the intention with which the individual acted. And so if they don’t have criminal intent, as the law calls it, the mens rea, the evil mind, to go along with the actus reus, the evil act, then they haven’t committed a criminal offense. And so the question will be whether these machines are capable of criminal responsibility.

Speaker D: Well, that’s an interesting point then, isn’t it? Because that almost makes a nonsense of whatever it is the European Parliament are trying to do, because what you seem to be suggesting is that we’re going to find it very difficult to use law to control robots.

Speaker F: We won’t be able to use the law to control them at all. What we may well be able to do is to use the law to control the people that control them, if they’re still in control of them. But the issue, of course, is whether the human beings that created these machines will be capable of remaining in control of them and criminally responsible for their actions.

Speaker D: Well, given that you’ve painted that rather worrying scenario, shouldn’t we just say stop this right now? Shouldn’t we just say, hey, we don’t need this, this is a lot riskier than we thought?

Speaker F: If there was no upside to the creation of intelligent machines, then yes, we should. But of course, there will be an upside. There is an upside. There are lots of tasks that human beings are either very slow at performing, which machines perform much faster, more efficiently, and we may be reluctant to give up that advantage., and there may be things that they literally can do for us, things that they can think, that if only we had the imagination, we would want to think those things through and benefit from the results. So they may be much better, for example, creating effective drugs against diseases, able to help us to create less polluting methods of heating our homes and powering our machinery and so on. So all of those things may be so advantageous to us that we’ll be very reluctant to get— not to risk the downside of creating intelligent machines in order to have the advantages of doing that.

Speaker B: Manchester University’s Emeritus Professor of Philosophy, John Harris, on the choice that we face. Continue to use technology technology to make our lives better, more fulfilled, and to allow us to do things that we could not do before, like fly to the stars, but risk being dependent on it and ultimately enslaved by it? It’s a debate that Future Intelligence, the makers of this program, intend to bring to the UK’s politicians. Technology is a greater threat to future employment prospects than the open borders that provoked Brexit. So Future Intelligence is organizing a one-day conference on the issues that we have discussed at the Institution of Engineering Technology on April 25th. You can find more details about that on our website, www.futureintelligence.co.uk. You can also find our survey and provide us with your thoughts on whether you want AI, robots, and the smart city to run your lives.

Speaker A: This program has been brought to you by Resonance 104.4 FM. If you liked what you heard and want to support our work, please make a donation at fundraiser.resonance.fm.