PassW0rd – 12th October 2016

Speaker A: This program is brought to you by Resonance 104.4 FM.

Speaker B: If you like what you hear and want to support our work, please make a donation at fundraiser.resonance.fm.

Speaker D: Hello and welcome to the Password Radio Show on Resonance FM, a monthly dive into the wheres and wherefores of the technological world with me, Peter Warren. In today’s program, we speak to the people who think that we’re getting too intimate with with our computers and the people who are worried that we’ve used them to reduce our world to an accounting exercise, a set of numbers that have left humanity behind. First, it’s been a busy summer for technology and artificial intelligence. The subject that has been simmering away since Professor Stephen Hawking warned that it threatened the future existence of our race has not gone away. The worries over the emergence of a super-intelligent machine have provoked a flurry of books that all question whether our high-tech future is really much of a place to aim for. Indeed, many people, such as Anthony Peake, author of the recently published Opening the Doors of Perception, think that our rational digital world is actually only a small part of a much larger world that we only dimly perceive.

Speaker C: The book Opening the Doors of Perception is effectively a review of altered states of consciousness, and it takes as its basis the experiences of Aldous Huxley in 1954, when he arranged to have sent or brought down from Canada by a guy called Humphrey Osmond some peyote mescaline, and he had an experience that effectively changed his life because it made him realize that there are areas of reality that literally are cut away from our areas of perception by the brain. And he concluded in the way that sort of people like William Blake had before and Henri Bergson, that The brain is an attenuator. It’s a reducing engine that gives us the information we need rather than gives us the wider informational field. And basically the book is about why certain individuals who have certain neurological issues seem to have doors of perception that are wider than the neurotypicals.

Speaker D: I mean, a lot of people will say, hang on, this is all this hippie mumbo-jumbo, this is all of this new age thinking. But you actually go into this in quite academic terms. Academic detail, don’t you? You, you, you really do go for the science.

Speaker C: Well, as far as I’m concerned, I very much, uh, of the similar opinion to Marcello Trulli when he said that extraordinary claims need extraordinary proofs. In fact, most people consider that to be by Carl Sagan, but it wasn’t Sagan, actually quoted truly on this. So in which case, what I do in my books is I go to as much detail as I possibly can as an enthusiastic layperson in the sense that I will go to research papers, I will speak with researchers, and I will give my readers the opportunity to read the academic papers themselves to come to their own decisions as to whether my conclusions are valid or not. It is a typical academic approach because effectively I’m not trying to prove anything. All I’m doing is making other people read and ask questions and carry things forward. So for instance, I go into great depths with regards to neurology in terms of neurochemistry, the latest research in neurochemistry, for instance, the role of the glial cells within the brain. Also, there’s various effects that have been discovered in the brain in terms of certain neurotransmitters that suggest that certain neurotransmitters do seem to be able to open up areas of perception that are generally denied to us under normal circumstances. And these things seem to be released during times of stress, which could in its way explain such things as the out-of-body experience, the near-death experience, and various other liminal experiences that we have throughout life.

Speaker D: I mean, it’s the interesting point that you make in your book, isn’t it, that our brains are actually a mixture of chemicals and electricity And to actually achieve the consistency that we have in the way that we think about our reality is quite remarkable in itself.

Speaker C: Well, effectively, one of the great mysteries of science, and it was what David Chalmers stated in a very famous speech he made at the University of Arizona about the mid-’90s, and he called it the hard problem of science. And the hard problem of science is how, in effect, inanimate matter— in the role of molecules, chemicals, and electricity, can create— I know electricity is not matter, but nevertheless— can create a sentient, conscious, self-referential human mind, human consciousness. You know, the argument I use in the book is, you know, how does this happen? Is there just one point, you know, the addition of one molecule, and suddenly we become self-referential consciousnesses? And indeed, we are in the position now that We have evolved to such an extent that we are a collection of atoms ruminating and trying to understand what atoms are. And this is a huge mystery. It’s almost like a bootstraps theory. And to me, this, this is the most intriguing thing, you know. What exactly— how does consciousness appear within electricity and chemicals within the brain? What, why, and how, you know? And for instance, in neurology, there’s the heart, the the problem of the binding problem in the brain. You know, how is it that, for instance, if I see a bouncing red ball going down the street, different parts of the brain are processing the movement, other parts of the brain are processing the red, and yet I get the feeling of simultaneity that I see a bouncing red ball? That is impossible unless the brain works non-locally, and I argue that the non-locality could be something to do with the glial cells.

Speaker D: Right. Now, one of the ways that we seem to be seeking reassurance at the moment is to create this digital world and to use that as a reference point for all of us. Um, but there seems to be some disaffection with that, doesn’t it? I mean, there’s a lot of interest in the ideas that you’re talking about at the moment, and it’s almost as though people are saying, no, actually, I don’t want that digital world.

Speaker C: The fear is that We feel secure in the reality that as it presents itself to us. So we sit in our rooms and we look around and we believe that there is a definite one-to-one relationship between what we see and what is out there. But there isn’t. And that is the mystery and that is the major issue of modern neurologic and consciousness studies. Because the people who believe there is a straightforward quick program between what I perceive and what is out there are called naive realists, because whatever is out there, it is not as we perceive it. For instance, we see a very, very small part of the electromagnetic spectrum that we consider to be colored light. I once used the analogy that if the electromagnetic spectrum was the Mississippi River running down from Minnesota to the Gulf of Mexico, the visual spectrum that we think is everything would be around 1.5 inches just south of Hannibal, Missouri. That is exactly how we should perceive things. It’s much wider. There are things we cannot see, things we cannot perceive, and even the machines we use cannot even perceive them. The reality out there is not as real as we think it is. We’re never in contact with anything, for instance. The brain inwardly generates a model of external reality, taken from the information that is given by the senses. You know, it doesn’t— visual, for instance, the visual field that we see is created by a postage stamp inverted image on the retina, which is converted into electrical signals, which is then reconverted again in the visual cortex of the brain. And we see this fantastic three-dimensional world that surrounds us. There clearly is something very magical going on here. It’s not magical. Wrong term, dangerous use of the word. But clearly there is something more to it than simply we just exist in this world. We are thrown into this world and then we’re thrown out of it when we die.

Speaker D: So in a sense you’re saying that there’s almost a very real danger that if you only look or only can see what you are going to measure, what you’re going to make, if you don’t make the rest of it, then you’re not going to have that. If you become part of this digital world that people are talking about, where they’re talking about potentially engaging with robots, even having sexual relationships with them.

Speaker C: Well, this again is the problem of the other, which has been discussed in philosophy for centuries. The idea that how can we ever tell that anybody else is conscious and thinking? You know, it’s the problem of the zombie, the problem of the Turing test, you know, the argument that you could create a robot that looked like a human being that could be programmed to answer your questions in a way that would make you believe that they were sentient. And as soon as you start to do that, you can then project onto that machine a belief that in some way it is alive.

Speaker E: Well, it could.

Speaker C: And of course, here’s the question of what is life. Life, probably, is argument, is movement, motivated movement, the élan vital. Now, if something can move itself, that’s on its way to being alive. And if it then shows an ability to think, then we’re starting to get into some interesting areas. But then it erodes away what it means to be human. And of course, in our postmodernist world, you know, we have people like Ray Kurzweil saying that we’re actually moving towards a posthumanist phase of existence whereby the man and the machine will become one.

Speaker D: That was Anthony Peake. Author of Opening the Doors of Perception, a nod to the title the author Aldous Huxley took for one of his books from the poet William Blake. In Brave New World, Huxley’s famous dystopian future, science is strictly controlled to prevent it from upsetting the social order, something that Huxley appears to be critical of. And for a long time, Huxley’s view that science should be unfettered was the popular orthodoxy. But recent technological moves towards a smart city and artificial intelligence have led to calls for a challenge to that view. Harvard University’s Professor Sheila Jasanoff, author of another recently published book, The Ethics of Invention: Technology and the Human Future, is one of those like Hawking and techno-billionaires Bill Gates and Elon Musk who is arguing for an examination of where we are heading with technology. Zhasinov says we are dependent on technology, but we have to work out our relationship. Beneficial. Do you think that there is a risk that trust is beginning to be impugned by the way that people are handling data about people, that there is something of a disenchantment going on with technology companies, not just about the use of their data, but also about the world that we’re being sucked into, this, this world of reductionism?

Speaker F: I think that one can see stirrings of that kind of discontent, but I doubt that we are seeing anything that would amount to a wholesale rejection. There’s frankly way too much value in the kinds of connections that the digital era has enabled, and if indeed it happens that much of the physical freedom that we gained in the 20th century to fly about here and there begins to get— or drive about here and there— begins to get curtailed because of the environmental implications. If anything, we’re going to rely on the digital more to be more efficient, certainly not to go to malls but instead have the goods brought to you. So I don’t see that even if people don’t like it, that they’re going to be easily able to detach from the technologically mediated digital worlds that we’ve all become citizens of, whether we like it or not.

Speaker D: I mean, that’s the issue though, isn’t it? We’re dependent on these worlds. One of the things that’s very interesting at the moment is there’s been a spate of books that have come out that have been suggesting that there may even be a need to regulate this world that we’re approaching, like yours. There have been other books that have been suggesting that maybe there is some other world that we could look to rather than this technological rationalist world, that maybe we’ve lost out on some innocent past. And there’s also been a rise in spirituality. There’s also been a number of films that have come out just recently which have been suggesting that we’re beginning to merge with the machines and that maybe that’s not a good thing. You know, there’s books on superintelligence, there’s books on, um, uh, sex with robots.

Speaker E: There’s—

Speaker D: in fact, there was a, um, a conference in, in Salford just recently which was done under the aegis of the European Union on whether we were getting a little too close to the machine.

Speaker F: I think that the, um— the kinds of phenomena you’re describing about disenchantment with the world of the machine, they have been with us for quite a long time. Maybe we’re seeing a kind of acceleration, and I think you’re seeing the world dividing into two camps: the ones that hype the post-human. In fact, it’s interesting to me that Britain is the site of both some of the more pessimistic responses that I know, that we need to withdraw and find other kinds of resources from within ourselves. I’m thinking of something like the Dark Mountain project of, of King’s North, and some of the more hyped-up versions of the post-human and how wonderful it will be to have minds connected to produce superhuman feats of everything— creativity, even cognitive immortality. So it’s not a one-way street. I think we’re seeing a reaction, counter-reaction, and my own view is that we want to insert into that a couple of different things. So one of the words that— or one of the things that you said that I particularly resonate with is kinds of worlds. I mean, so what kinds of worlds are we imagining? Does the technological world have to have the word rationalist coupled to it in the way that you said? My view is that obviously it does not, but that we willy-nilly are to some extent in the power of people who are at the frontiers and making the new things happen, and they of course come with very particular visions. So it’s not totally accidental that there is a Silicon Valley and that happens to be in California, and it has certain liberationist ideas about how to use technology to get out of some of the binds of the present. But it’s not obvious that if those capabilities are put into the hands of a billion Chinese or a billion Indians that views about how the future should unfold actually come from very different social and cultural contexts. In fact, my own work suggests that even Europeans will not want their technologies to create for them the same kinds of futures. So part of what I’m wondering about is how does one free up the possibility of imagining alternative futures so that It doesn’t get nailed down in advance by the people who have the technological wherewithal to enable people to think about these things. There’s no reason why the imagined future has to come hand in hand with the ability to click or point on a computer. And yet how to decouple those is not the easiest problem for the world to be grappling with.

Speaker D: Professor Sheila Jasanoff, on why she wants to start a debate about our high-tech future and why she thinks we are going to have to change our value system if we are to make that future work for us. Dr. David Levy is another academic who would like to see a debate about the future and also thinks that our values should change. Unlike Professor Jasanoff though, Dr. Levy thinks that we should get a lot closer to the machine than we are at the moment In his book, Sex and Love with Robots, Dr. Levy argues that it is inevitable that the machine develops even further as we explore the bounds of what we want from it.

Speaker A: One of the areas that I found very interesting was the question of people’s relationships with computer software. In the course of my research, I read a book by Sherry Turkle called The Second Self, which was originally published in 1984. ’84, republished fairly recently. In that book, she investigated how people at MIT in those days felt about their relationships with their computers. And she interviewed a lot of the computer people at MIT, programmers and so on, to find out what their feelings were about working with computers. And one of the people she interviewed was a student who she called Anthony. And Anthony said something that struck me like a sledgehammer. He said, “I’ve tried having girlfriends, but I prefer my relationship with my computer.” And it turned out that the reason why he preferred his relationship with his computer was because people and society in general are unreliable. You never know how someone or society is going to react to you. Whereas with your computer, you know how it’s going to react because it’s sort of deterministic. And so, he felt more comfortable in a kind of relationship with his computer than he did in relationships with girls. And I thought to myself, well, this was 1984, and that was 20 years before I read the book. I thought, I wonder what has happened in the intervening 20 years in the realm of relationships between humans and computers. And I started researching it, and that’s what led to the book Love and Sex with Robots.

Speaker D: Isn’t there a sort of autism in that though? Isn’t that an expression of yourself that shows that you’re not particularly comfortable with being able to deal with change?

Speaker A: To some extent, yes. But then of course, if you mix a lot with people who are techies, I mean, they do exhibit some signs that are akin to autism because they’re, you know, a lot of them are totally immersed in technology and they do find it difficult to create relationships with other people. So yes, I think that’s quite a reasonable point of view.

Speaker D: But the other thing is, you suggest determinism, that people like things to be as they want them. I mean, in a sense, people actually don’t like change, do they? We see that in our society all of the time when people are talking about job losses, when people are talking about economic change. People want stability, don’t they? So is the machine or the computer that we’re beginning to see develop, is that something that people are beginning to see as reassuring?

Speaker A: Well, I think there are two sides to this. One side is that advances in technology bring benefits to society in general. The other side is that they— that there’s some sort of threat for the reasons you mentioned. They can cause unemployment, for example. And many forms of technology have a downside. So one has to be careful when thinking about new technologies. There are the benefits and there are the detriments.

Speaker D: But there’s also an over-reliance, isn’t there? We’ve— several press releases have come across my desk in literally in the last 3 months showing that people are beginning to lose their memories to their mobile phones. People are beginning to lose their initiative to their mobile phones. I mean, that, that’s actually quite worrying. It’s almost as though the machine is taking us over and we’re being absorbed into it rather than the other way around.

Speaker A: Well, to some extent, yes. I mean, the first example I can think of that’s happening was in, you know, when my children were very young, the government in its infinite stupidity decided to bring in electronic calculators. And so you had a whole generation, or more than one generation of schoolchildren, who are unable to cope with mental arithmetic. And you see them, you know, nowadays I can add up quicker in my head than the girl on the checkout in Waitrose. Simply because I went to school in the days when we didn’t have electronic calculators. So what you’re saying is quite right, that to some extent, um, we’re becoming victims of the fact that computer software can outperform us in certain fields. Having said that, there are of course a lot of brilliant advantages in some of the achievements of computer software, and there will be even more advantages in the future.

Speaker D: Surely if you’re having sex with a robot then you’ll know that there will be this feeling of disaffection and dissatisfaction.

Speaker A: I believe that is going to be true for people who understand the technology because they will be thinking to themselves, oh, I know how that, how that’s done, and it will take away, you know, some of the magic and pleasure of the experience. But I think for the vast majority of society, that won’t be a problem. And I’ve always said, and I firmly believe, that the I think the greatest benefit of sex robots is what they can do for people who are lonely, people who have no one to love, people who have no one who loves them, and people who can’t form a good lasting relationship that satisfies them. For such people, I think these robots will be a huge boon to their lives and enhance their life quality enormously.

Speaker D: But I mean, just to follow through on that line, one of the things about sex is that actually part of its unpredictability, that there is something a little boring about having your fantasies enacted out, especially if you don’t have very much imagination.

Speaker A: Well, that’s true, but remember that it’s possible to program some randomness and variation into almost any kind of application. And so the robots that we are talking about will be programmed to, um, do things differently, slightly differently from time to time, to create some variety and thereby create some additional interest for their partners.

Speaker D: That was Dr. David Levy telling us why he thinks the robot sex workers will prove to be a boon to the lost and the lonely. But it’s a development that Dr. Kathleen Richardson deplores. Sex robots, says Dr. Richardson, are a symptom of an illness in our society, and that rather than solving problems of loneliness, they will only isolate us even further.

Speaker G: I don’t think inanimate objects can meet our needs as human beings. If we took away the word robot and we used the word property instead, right? And we took away the word cyborg and we used the word property, I think what is really going on are different people advocating for property relations. So, different kind of arguments today expressed through the cyborg as one example that, if you like, venerate property. And property is quite a, you know, property relations is a dominant way in which we understand the world because It’s great for people who have power and resources if they can convince the rest of us, the rest of humanity, that we’re also property, that our bodies can be property, that we can remove parts of our body. Because when you have this pesky thing called human rights, you know, getting in the way, there are kind of limits to how you can keep extending the market, particularly into a human body. For example. But if you can refine and reframe that narrative as some kind of evolutionary, progressive, idealistic development, then all of a sudden people think that they get on board with it, and they think that what they’re doing is they’re not actually turning their body into property, but they’re actually being individual. They’re expressing their individuality. The other camp, well, first of all, we’re not property. I believe in a radical distinction between persons and things, and I think we need that distinction because the confusion comes from slavery, as it happens. So slavery was a way of presenting people as property so that they could be exploited. So I think it’s important when we think about the cultures that we live in, that we don’t repeat the mistakes. We don’t keep endorsing the idea that people are property. Because the impact of that is really going to be on ordinary people. It’s not going to be on the power structures or the status quo. The status quo is always finding a new way to assert its ideology in the world.. And it just so happens that robots and digital culture is, and the cyborg manifesto is the way in which it’s doing it today.

Speaker D: Okay, I think that’s a very good point. Cynthia Breisel at MIT, who’s one of the professors of robotics there, she says that it’s very important that robots are made to be different from people so that we don’t actually start to have relationships with them because she says that that is a psychologically bad thing.

Speaker G: I’m quite happy for people to use robots, property, as instruments, to have some useful purpose out of them. The thing is, and here I would agree, you know, that instruments can’t play relationships. They can’t actually do it. And we have a lot of evidence from studies of animals, for example, primate studies, infant studies, that if you take away a person in a child’s life, or in an animal’s life, and you replace it with an object, right, or even another animal. So in my own research, I’ve looked at studies of children in the wild. So what, you know, people think that you can have anything as a child, that it wouldn’t make a difference. When they found children in the wild, they didn’t, they found these children to be very kind of, I don’t want to use the word dysfunctional, but I guess that describes it. You know, they had difficulties walking, difficulties talking, difficulties expressing language. Because actually human beings play a very important role in each other’s lives. And because we can’t see it and measure it, it doesn’t mean that it’s not going on, but it is actually going on all the time. And that goes on not just between adults and child, but it goes on between adults as well. And I think there’s an argument that people are saying you can just— a person can now have a relationship with a human, a person, or a thing, an inanimate object, and it won’t have any impact. What I’m saying, it will have an impact. It’s going to have a profound impact on human beings’ ability for even their own happiness, actually. I would say this increased world that we’re living in that’s pushing us more and more into these isolated technological groups, The impact of that is going to be depression, isolation, anger, aggression.

Speaker D: It’s a sci-fi world of the film Blade Runner: An Ex Machina that is coming faster than we think. In fact, it’s coming so fast that Dr. Richardson was debating it at a conference in Salford last month called Technology and Intimacy: Choice or Coercion, which was organised on behalf of the European Union by Salford University’s Dr. David Krebs. Kreps, author of another recently published book, ‘Bergson, Complexity and Creative Emergence.’ For Dr. Kreps, the sci-fi world of the robot that we are going to be engaging with won’t be able to look us in the eye. Indeed, that will be the test for it, because the moment it can make eye contact with us is when the machines will have come alive and we will have to start engaging with them in a way that Dr.

Speaker C: Richardson will find acceptable.

Speaker E: The issue here is that you can’t get eye contact with or have a conversation with a machine. A machine can only reproduce the responses that have already been programmed into it. Only a sentient, aware consciousness can respond to you with their own unpredictable response that shows to us that they also have this unverifiable awareness thing going on inside them. And the moment you get to just sort of discern that, that’s when you know that you’re engaging with another being. And that’s, I would say, how a relationship comes about. I don’t know that you can necessarily have a relationship with an inanimate object. You know, the technology is a tool.

Speaker D: The technology does our bidding.

Speaker E: We provide the desire, we provide the need. And a lot of people are saying that that is the essential thing, that, you know, yes, you can put in need for power into a robot. It needs to go and find a plug, but that’s not really need. Although Agency does reside in us and in our awareness.

Speaker B: We’re not omnipotent, far from it.

Speaker E: For starters, there’s many of us, and so there’s lots of conflicts between all the different desires and wishes and agencies. And secondly, we are corporeal. This awareness arises out of and is not independent of the biological brain and body.— that enables all of this to happen. And as that brain and body deteriorate, our awareness becomes thinner and thinner, and eventually we disappear. Goodness only knows where to. Continually within a set of, or network of, power relations, if you like, between us, and also between us and the objects around us. We cannot walk through walls, even if we want to. Though the interesting point about this is that that agency is one of the things that actually works against us, as you are saying, because, for example, automated cars should theoretically not get into traffic jams because they won’t rush down to the front of the queue and try to behave badly. One of the things about agency is that it tends to make you a bit chaotic. Whereas the robots would be better at doing certain things. I mean, isn’t that really the conclusion, that robots and automation and technology are better at doing certain things? There’s been a sudden spate, a sudden rash of books and of films about this, perhaps prompted by the revelations that AI is moving a little faster and the results are better than people thought. Yeah. Do you think that this is beginning a disenchantment with technology? Why do you think that there are lots of people who are suddenly just saying, hang on a moment, I don’t quite like this world? In terms of disenchantment, I would say that people are questioning and that’s a good thing. But I wouldn’t say it’s particularly new. I remember a lot of the cyberpunk novels in the 1990s were very critical. I think there’s always been a mixture of blind enthusiasm and sort of back-to-the-woods critique. I think that, you know, the Luddite approach of trying to ban things is never going to get anywhere because, of course, you know, we’re dealing in a globalized world and these things will be made somewhere. And be available to you in due course. Or— Oh, absolutely, yes. I think that, you know, they’re investing in AI and in robotics because they see that it’s going to be popular and sex will drive it like it drove— like pornography drove the internet. Do you think that this will mean that there is a need to legislate in the area of sex, or do you think that it will be a world like that of Westworld where you’ve described anything going? People seeing how far they can go with their fantasies? I think the latter, probably. You know, when we had a slave economy, right up until the 18th century, this is how people used to behave with human beings. So perhaps it would be an improvement if we had robots to treat like that instead. The robots achieve sentience and complain about how they’re being treated.

Speaker B: But this—

Speaker E: oh well, that’s a different matter. One other point though, isn’t there, which is of course those people who are involved in the sex industry will complain that they have been thrown out of work by machines. And that’s one of the other issues that has been exercising people quite a lot recently. Well, maybe this sort of thing has happened before where large numbers of people have lost their jobs to industrialization and automation. I think the real concern is the speed. In the past, when certain tasks have become automated, which to a certain extent, you know, even the people who used to do those jobs were quite happy because they were dangerous and unpleasant and dirty jobs. And new work came about, the economy refreshed itself, and they got nicer, cleaner, safer jobs. Exactly. For a long time there were people saying, I don’t want my son to go and work down the pit. Exactly. So, you know, in the past it hasn’t always been too bad a thing. Suddenly technology will become the bête noire. Technology will become the equivalent of the European immigrant, and rather than having a vote against Brexit, we’ll have people suddenly saying, we want to vote against technology. Well, that’s a nightmare scenario for another science fiction novel, I expect.

Speaker D: Dr. David Krebs on the new slave class of recreational machines that he thinks that we will develop to fill the leisure time that the worker robots will have given us by taking away our jobs. A very real prospect, according to studies from both Oxford University and the Bank of England. Some one-third of the jobs that we now do will be taken over by machines and algorithms within the next decade. Just when the robots will be able to engage us in eye contact is quite literally the burning question of the moment. Though, for Professor Nick Bostrom, director of Oxford University’s Future of Humanity Institute, and author of Superintelligence, it’s not a question of if, but when.

Speaker B: Well, it long seemed like one of the most important things that will ever happen to me, the eventual creation of machines that are smarter than us. It actually started out as a book about existential risks in general, and one chapter was going to be about possible risks from artificial intelligence. But then when I began writing that chapter, it kind of expanded and grew, and eventually it became clear that the book that really wanted to be written was a book about AI specifically. So I took that draft chapter out into its own document, and then yeah, eventually it became this book, Superintelligence. But how much of a risk is this? I don’t think we are necessarily at all close to being developed, able to develop human level or superhuman levels of machine intelligence. But it looks like we will eventually figure it out. And at that point, this transition to the machine intelligence era, I think, would be associated with significant existential risks.

Speaker D: So what you are saying is that really we will actually embark on the pursuit of a superintelligence and that this will be a very, very sort of something that we will have determined right from the word go. This is what we’re going to do and this is what we’re going to build.

Speaker B: And that’s how you see something happening. Well, not quite. I think the majority of the effort is driven not by this long-term goal of superintelligence, but driven rather by the desire to make incremental progress on various information systems that we have today. So if you can develop, say, a recommendation engine that could be used by Amazon to suggest books to its customers that they’re slightly more likely to find interesting, then there is a commercial payoff to that. Or if you can develop a microchip that is slightly more energy efficient, so your, your cell phone’s battery life is extended. There’s a commercial payoff for that. I think that those kinds of incentives are the main drivers. But if those incremental progress are stacked upon one another for a sufficiently long time, then I think we will eventually come within striking distance of achieving general intelligence in machine substrate. And then maybe at the end of that, some project will specifically aim to achieve human-level general intelligence or superintelligence, and maybe they will then be the ones crossing the finishing line.

Speaker D: Now, but there are two things that people have talked about for almost as long as this debate has gone on. One of those is that the two things that would differentiate humanity from robots or machine intelligence is consciousness. Or to creativity. Those are the differentiators. I would say that there’s actually probably a third factor, which is desire. We actually have something that impels us to want things, and machines don’t necessarily. Do you—

Speaker B: how are the machines going to cross those hurdles? It’s not clear that having consciousness would be necessary. In order for an AI to have superhuman powers of reasoning, you might have a superintelligence perhaps that had no internal subjective experience. I think that’s an open question. I don’t really take a stand on that in my book. As for creativity, it’s a matter of degree. We already have systems that can exhibit limited forms of creativity in particular domains. You could say that to play chess well, you have to be creative. You have to come up with some great move that, that wasn’t obvious, that nevertheless leads to strengthening of your position in the game. And certainly computers can do that. Computers have also composed music and done other things that we tend to think of as creative. So they would need more of that, but I don’t think there is a qualitative difference. As for desire, this is an interesting open question, whether we would want to try to engineer a hypothetical superintelligence such that it was an agent, that is a system that acted in pursuit of some specified goal, or whether there is some other kind of architecture that we might think is safer. But the ability to create an artificial intelligence such that it has something we can think of as a goal is not in itself a difficult thing. We could create simple agents already today, like a robot that looks for, tries to find an electricity outlet to plug itself in and replenish its battery. So things like that can be done and have been done. And but obviously you would need a much more advanced form of agency in order to achieve superintelligence through that.

Speaker D: Yeah, there was the example, wasn’t there, of a robot that went around gardens collecting slugs and snails and putting them into a container on its back that then broke down their tissue and turned them into electricity.

Speaker B: Yeah, so whether that that robot has a goal, I think, would depend on how this behavior was implemented. In, in general terms, I think if inside this agent’s head there is some process that searches for different plans and, and then compares each possible plan with some criterion and then acts according to the plan that meets this criterion best, then we can think of it as a goal-seeking agent. It kind of looks for a way to achieve its criterion, and that’s like a goal.

Speaker D: So the future that Professor Zazenov is suggesting that we should be considering is one that Professor Bostrom says is coming, though what we are going to have to do is to convince that superintelligence of sharing some goal with us that it will consider makes some sense.

Speaker B: We should be wary, I think, of importing intuitions that work well for thinking about human minds into artificial intelligences. And AI might not look anything like a human mind. It might not have deep-rooted human emotions. It might not have urges in the same way that we have. It might have a very different structure and nevertheless be able to achieve intelligent behavior.

Speaker D: Yeah, I mean, that was the point that both Professor Murray Shanahan made and Stephen Hawking’s, that why should we expect any artificial intelligence to be anything like us?

Speaker B: Yeah, and I don’t think that that would happen, certainly not by accident. Like, it’s something that if we wanted to, we could maybe try to achieve, and it would be difficult. But for most types of machine intelligence, it’s not the outcome you would expect by default.

Speaker D: But doesn’t that then, given the fact that, I mean, it is a science fiction myth, and it’s something that’s been going through science fiction forever, that the robots will take over, that the Daleks would take over, that the machine would take over. Given that everybody’s been talking about this, doesn’t it make some sense then— or sorry, that people have been talking about this for such a long time. Doesn’t it make some sense for people to build some sort of safeguards into the system before we actually even get to those sorts of stages?

Speaker B: Yes, that, that would make a lot of sense. So the question is exactly how you would do that. At the moment, we don’t know how you would solve this control problem. The problem of ensuring that the super intelligent agent that came out of your software would be safe and beneficial. It looks like a very difficult technical problem to achieve that. And one of the reasons why I said that there would seem to be significant existential risks associated with this transition to the machine intelligence era is that it’s not clear whether we will have solved this control problem in time. There are two problems here. One is how to make machines intelligent. The other is how to make intelligent machines safe. I think we need to solve both of these problems ultimately, but it’s crucial that we solve them in the correct order. We need to have a solution to the control problem before somebody figures out a solution to the problem of how to make machines intelligent.

Speaker E: Yes.

Speaker D: With that in mind, I mean, there’s two professors in the US who have been suggesting that there is a need to put ethics into computer code. And well, the computer programmers are throwing up their hands in horror at the mere suggestion of that. What they’re suggesting is that, you know, when computers hit a particular scenario, they then refer that scenario to ethicists, and the ethicists then say what the correct route for the computer to follow should be, and then that gets coded in, which sounds like quite an unwieldy way of doing it. I mean, is that the sort of thing that you would envisage occurring eventually, or?

Speaker B: Not really so much. There is a class of scenarios in which it will be key to have the very first system, the first version of the system work correctly. You have to set up the initial conditions in just the right way, because once you have a superintelligence that is not friendly to humans, then it might be too late. Like, it could then resist attempts that we might make to change its values. So in one class of scenarios, it would be crucial to get this right from the outset. And that then is part of why this is a difficult problem. Like, we are— humans are not very good at getting very difficult, complicated things right on the first attempt. We usually sort of see what works and what doesn’t work and then patch things up as we go along. But for some kinds of things, when there is an existential risk involved, we can’t really see what goes wrong and then learn from experience, we might only ever get one, one shot at it.

Speaker D: Yeah, I mean, it’s been suggested that Kant— A potentially difficult prospect, because as the internet thinker Mikko Hypponen has pointed out, the first thing that any superintelligence or sentient robot would do would be to carry out a web search on humanity and its relationship with machines, and it would discover very quickly that not only was humanity scared to death of it, but that it had spent huge amounts of time making films on how to destroy and enslave intelligent machines and robots. Interestingly, that’s something that the Warwick University economist Dr. Jonathan Cave hopes may not happen. Indeed, Dr. Cave hopes that, given the huge advantages that a superintelligence should have over us, that it may do a second internet search and attempt to find out what the stated objective of humanity is and come to the conclusion that we claim to be interested in humanity and that the superintelligence will decide that that should be its goal.

Speaker E: What is it that makes the human world human? Part of it is our interactions with each other and part of it is our limitations. You know, the fact that our minds are not so large that there is nothing to learn by talking to other people. That’s what makes us as individuals different from each other, because the world is the same. It’s just that we have different impressions of it. So maybe humanity is just that complexity of perspectives on a sort of single reality.

Speaker D: I mean, arguably, arguably, you could also say, though, that any superintelligence, if it took the mass of human knowledge available and what human aspirations were, would settle on humanity as being the be-all and the end-all. Or the objective of humanity as a whole, whereas in fact human beings aren’t very human. I mean, by and large, that may be their intellectual aspiration. By and large, they’re terribly, terribly cruel. So therefore, a superintelligence could then be a little like a Platonic ruler and start imposing upon humanity a superhumanity.

Speaker E: Yeah, no, I think that’s correct in either sense. I mean, it may be that our humanity, in the sense of being humane, perhaps rather than human, is simply a way of hedging against uncertainty. You know, “There but for the grace of God go I,” or “I could get it wrong,” you know, Pascal’s wager kind of thing, that I might get punished in the future if I get it wrong today. Or it may simply be that intelligence as a means and intelligence as an end are two very, very different things. Whether intelligence is simply an emergent characteristic, right? It’s just a pattern, a regularity, and it’s not knowledge in any particularly absolute sense because we know or think we know that there are limits to what we can know about reality, and we know from things like physics that what we know about reality sort of determines reality. When we talk about machine intelligence generally, we talk about their ability to perform certain computations that have been defined for them, and which we then wire up to things that we call solutions to problems. But it’s not necessarily self-motivating. So if we try to add to computational ability something else, that something else may be a form of self-motivation, or it could be a form of aesthetic sensibility, that we think about things because it is pleasant or pleasing to us to do so, in which case there’s nothing particularly intelligent about it at all. If we have these super servants who can anticipate and fulfill all of our needs, do they require us at all? Not do we require them, because we may become completely dependent upon them. But this was this thing about, you know, like the insulin pump or the automated avatar that served our interests and how it would respond if our interests were conflicted or unclear. And it might seek to anticipate that by studying us or performing experiments upon us and in the process programming us so that we were easy to serve, in which case we would become entirely, you know, nothing of our humanity or volition.

Speaker B: Would matter.

Speaker E: We would become kind of inessential, and there would be a closed loop serving and programming or serving and controlling, and the wisdom would just close back in on itself. But in that sense that it becomes entirely self-consistent, it has removed the singularity. If the thing that makes us human is the thing we’re trying to automate away, then The superintelligence probably negates our intelligence, or may negate our intelligence, because it reduces us to that package of needs, requirements, and objectives that it’s designed to serve.

Speaker D: It will also probably decide to placate Dr. Richardson and stop having sex with us because it will realize that there is biologically no point whatsoever in doing so, as it doesn’t make any rational sense. For Anthony Peake, though, rational sense is the problem, because while many are viewing the evolution of the superintelligence as the next level of attainment for humanity, others are seeing it as a victory of a reductive world of data that is actually eventually delimiting. For Yasinov, Richardson, and many of those contributing to this program, to work towards creating the necessary world of humanity for the superintelligence to understand, we have to stop producing a materialistic world of big data solely motivated by financial gain. Perhaps one of the biggest problems that we face in the near future, though, is not that machines will take over our world, but that increasingly we have become victims of the data that we generate that is already being used by the machines that we have made. A point made by the philosopher and author Geoff Croker.

Speaker A: Yes, well, I think that is a very real ethical danger. It’s a macro danger, and that’s why we urgently need to discuss the philosophy of technology, whether the machine is controlling us. At least we could say maybe this machine is taking us on a ride, and we don’t have any choice as to what the direction of that ride is. But as I say, whether we can resist technology or not, or have any choice about technology, is a huge question. There have always been people with a massively dystopian view of technology. I mean, Heidegger clearly, and Jonas and others of that sort. And then Auguste Comte and so on, with a massively beneficent view of technology. It’s going to make our lives easy. So, wherever we land on that spectrum of view, whether we think it’s dystopian or utopian, whether we think it’s deterministic or subject to our human agency.

Speaker D: Those are the issues that need this huge debate, and the debate’s not occurring anywhere, right? I mean, some people were saying that there was a need for politicians to become actively engaged in this, because, for example, if you look at the smart city The smart city is going to be imposed upon people. There doesn’t seem to be many places where people can say whether they do want or do not want some of the changes that are going to occur. They are going to be imposed on them on the basis of efficiencies.

Speaker B: I agree.

Speaker A: I think politicians should be massively involved in considering the philosophies of technology. I think academia should certainly take the lead. And I’m seeking to meet at the moment with academic leaders in leading universities in the UK to persuade them of the importance of the philosophy of technology and to persuade them to seek to develop courses, modules, and departments in that subject in their universities. So academia is terribly weak where it ought to be strong on the philosophy of technology. Politicians are completely unaware, frankly. And the public debate through the media and so on it needs to engage the subject as well.

Speaker D: It needs to be a multidimensional grasping of the issue of the philosophy of technology. What you’re saying is actually it’s not the technology that’s wrong, it’s the way we think, it’s the way we want to use it, it’s our view that we have to make money from it or that we have to reduce people to product, really.

Speaker A: Well, I think it’s a pathway towards opening up the debate, yes. I think the dystopian view, famously, as we’ve already said, Heidegger, Jonas, Rachel Carson in Silent Spring, for example, in other words, that we’re facing Armageddon with nuclear weapons, or we’re facing a pesticide pollution, or we’re facing a smog pollution, and so on, that all these dystopian views are relevant powerful, but they’re relatively easy to state because they’re one-sided. The other side is that technology has clearly brought benefits. So being at either end of that spectrum isn’t really very sensible. It’s a question of a much more nuanced view of what’s good, what’s bad, and how we do determine our future with technology, if indeed we can determine that future, which is the big intellectual question.

Speaker C: This program has been brought to you by Resonance 104.4 FM. If you liked what you heard and want to support our work, please make a donation at fundraiser.resonance.fm.