PassW0rd – 9th October 2019 (The Only Way Is Ethics)

Speaker A: This program is brought to you by Resonance 104.4 FM. If you like what you hear and want to support our work, please make a donation at fundraiser.resonance.fm.

Speaker C: Hello and welcome to Password at a turning point in the history of technology. I’m Peter Warren, and when I look back in years to come, I’ll be proud to tell my grandchildren that I was there when The moment came. Where? Well, Copenhagen, of course, because the Danish capital hosted its annual tech festival. But the Copenhagen Tech Festival is not like the other tech festival lovings where tech is lauded as the new god and the new good. This is a serious look at tech. The Danes have long given up being Vikings. And they take life very seriously these days. And they take tech even more seriously. And they even have special embassies in Silicon Valley and Beijing where the tech ambassadors practice something they call “techplomacy.” They are worried about where tech is taking us and whether we are in control of it. They’re not the only ones. In the US, 200 of the top companies in the country’s Business Roundtable have announced that profit should not be the main concern of business.

Speaker D: Business.

Speaker C: They should be concerned about their employees, where they are based, and the environment too. In the UK, the Financial Times carried a wrapper recently asking whether capitalism is dead, and The Economist magazine has belatedly revealed that the UK and US stock exchanges are now completely run by computers, something that scientists have been claiming for years. This, then, is the backdrop for Copenhagen, where tech experts and business owners hammered out a series of promises aiming to use their awesome superpowers not just for profit, but for the good of humanity, which they want to have center stage in any future relationship with technology. The result of all of that was something they called the Copenhagen Pledge.

Speaker E: The Tech Pledge. Tech has fundamentally changed our world. Where we go from here is up to us. I understand that for my good intentions to actually matter, I must make myself accountable to my community. I commit to take responsibility for what I create, to only help create things I would want my loved ones to use, to pause to consider all consequences of my work, intended as well as unintended, to invite and act on criticism, even when it’s painful, to ask for help when I’m uncertain if my work serves my community. To always put humans before business and to stand up against pressure to do otherwise, even at my own risk. To never tolerate design for addiction, deception, or control. To help others understand and discuss the power and challenges of technology. To participate in the democratic process of regulating technology, even though it is difficult. To fight for democracy and human rights and to improve the institutions that protect them. To work towards a more equal, inclusive, and sustainable future for us all, following the United Nations Global Goals. To always take care of what I start and to fix what we break. I take this pledge as an individual working to make tech a force for good in society, and I will challenge everyone I work with to do the same.

Speaker C: Of course, it’s not a new idea. I myself have been writing about putting ethics into technology for years. If you’re a regular listener to Password, you’ll recall that back in 2014 I published a major report called “Can We Make the Digital World Ethical?” and we presented it to the European Commission and the French Senate. What’s new is that now some major industry figures are signing up to actually make it happen. At the last count, the Copenhagen Pledge had around 1,350 signatures from them. I’ve been talking to some of those people who signed. Bjorn Jeffrey, for example, who founded the Toca Boca tech startup in 2010. It’s a children’s app that has become an accepted educational platform. But while Jeffrey was being very scrupulous, he found that other companies were less scrupulous.

Speaker E: It’s a starting point. I think it’s good to kind of move the conversation forward with something which is very concrete and specific. I think it would be naive and maybe too hopeful to think that the companies that are making their money off selling ads to small children in various devious ways, that they’re going to look at this and change their minds on the basis of it. That would be unrealistic. There could be sort of a more of a slow movement though, where there are more and more people in technology in general that, that are questioning, am I a net contributor to society by doing what I’m doing, or am I actually withdrawing something? Am I causing more trouble? And I think that is a growing concern. And hopefully, sort of over time, that can become a competitive factor.

Speaker C: Companies—

Speaker E: people want to work for companies that have a sort of a stronger voice in these things. You know, I live in Silicon Valley. Everyone thinks that they’re changing the world. Very, you know, even people that are working for a payment processor are apparently changing the world in some sense. At least according to themselves, but I think that is gradually changing. This is a slow thing, though. This is not going to change overnight, and it’s not going to solve the problems of children’s technology overnight either, but that’s not to say it’s not worthwhile doing.

Speaker C: The UK editor of TechCrunch, Mike Butcher, was recently awarded an MBE for services to technology and journalism, and he also runs campaigns to get people to register to vote and many other good causes, so we can assume that he is also on the side of the angels.

Speaker F: I think this Copenhagen Tech Festival event that we’ve been at has been an interesting exercise. It’s not your typical tech conference. It’s been sort of trying to address some of the existential problems that have been thrown up over the last few years about technology and its impact on society.

Speaker C: What, so this is a sort of Cambridge Analytica thing? The abuse of data, the— all of the things that we’re seeing fines leveled on the technology giants for.

Speaker F: Yes, I think that’s fair, fair assessment. And I think the old adage that has been going around for several years now about building technology to, quote, make the world a better place, unquote, has been satirized also by Silicon Valley, as well as coming from it. And that culture, you know, isn’t— the joke’s a little bit over now to some in the sense that there were some big issues that got thrown up over the last couple of years about how technology and platforms like Facebook are being used to affect democratic elections and have been actually for years as, as it started to emerge, which is really quite worrying. In fact, it looks like we’ve all been part of one big experiment for quite a number of years.

Speaker C: But I mean, that’s the point, isn’t it? Because people are saying, hang on a moment, The glitter’s come off the internet, the gloss has come off it. Now we see it revealed, we see it for what it is, and it’s, you know, it’s basically bleeding us dry, sucking out our information, sucking us up.

Speaker F: Yeah, I mean, let’s, let’s, you know, historically let’s acknowledge that initial hardware and software hackers from the old school of open source software which were really trying to build a much more connected world, came from the best intentions, and that still does continue to this day in the shape of movements like the so-called Indie Web, which still exists on channels like IRC and places like that, which are all open source platforms and not locked down in the same way that these huge platforms like Twitter, Facebook, etc., LinkedIn, in, etc., operate. But, but yeah, the, you know, the, the ideologues lost and the hyper-capitalists seem to be winning.

Speaker C: So is this a battle for the soul of the internet and it’s a battle that’s been lost? This is a battle where the big corporates came in, got the data, got the information, they’re now making all of the money from it, and the hippies who started the whole thing are all sort of running away crying.

Speaker F: Well, yeah, perhaps if that was the, you know, the movie pitch. But The Empire Strikes Back, perhaps to some extent, and perhaps not the Empire, but certainly the Jedi are returning. If you, if you think about it, there’s a lot of movement in, say, the blockchain space towards trying to remedy some of the problems. But I’m not, I’m not sure that they will necessarily all come up all with all the answers either. The blockchain, the nature of the blockchain or blockchain technologies are that things just get hardwired down. And you know, that kind of stupid comment you made when you were a teenager, but you know, nobody wants that to be around in 100 years’ time on a blockchain. And so I don’t think necessarily that’s necessarily the solution either. But you know, gradually people are becoming a lot more aware of of the problems associated with social networks. But I think, you know, I think we’re going to go through this cycle more than once. I mean, if you look at the growth of social platforms like TikTok, which is effectively Chinese-controlled, millions of young people across the planet, and also in Western countries, classic liberal democratic societies, liberal democracies, tons of data being sucked up there. The success of the the Face app recently, you know, sucking up huge amounts of imagery into Russian servers. This battle isn’t over any time soon.

Speaker C: Okay, so, and you say the Jedi, you know, the Jedi Knights are coming back. The Jedi Knights here in Copenhagen have been writing a tech pledge. In fact, you’ve been actively engaged in the writing of that tech pledge. Will that do any good?

Speaker F: I think it’s very nice to be aspiring towards higher, higher thoughts and higher, higher aspirations, obviously. I think that there has been this discussion here about whether or not technologists should take some form of the Hippocratic Oath which doctors take, you know, the main tenet of which is do no harm. When Oppenheimer created the atomic bomb, he famously said, “I am become death, destroyer of worlds.” And technologists for many years have not necessarily questioned the wider implications of what they’ve been creating. Many have, but many have not. And following the money has become the big deal. And the big aim, you know, and screw society, screw democracy, screw workers, screw individuals. That’s obviously massive generalization, but those are a lot of these bigger players. A case in point, today was revealed that one of the co-founders of Uber has bought a $74 million mansion while Uber drivers, you know, having to work all the hours to make ends meet. So it’s, it’s not an ideal scenario. It’s not a great picture that’s being painted, painted in it. And it’s, and it’s refreshing that people in the tech industry have come together here in Copenhagen to try and address some of those shortcomings of the industry and, and write down real pledges about how really should technology people behave in the future to mitigate some of the problems that have been created directly from the industry.

Speaker C: One of the industry giants that’s regularly accused of harvesting data for profit and unfair competition is Google. Sarah Drinkwater used to work for Google. Now though, she is the lead on tech and society solutions at the Omidyar Network, a charity founded by the eBay billionaire Pierre Omidyar to put technology to good use following its own set of values, which, if you’re interested, you can read on its website.

Speaker B: Tech only works as a force for good if it’s properly bounded, if it’s properly guard-railed. And I’m talking not just about regulation, but also about, you know, having very clear lines around who we do and don’t sell to, what we do and don’t do. So, you know, the Business Roundtable piece is part, I think, of an evolution of change. And Copenhagen too is an evolution of change. And there’s a really interesting side trend there around You know, if you look at the history of Europe, certainly the history of Germany, I think we are the canary in the mine on a lot of these topics. Certainly with things like GDPR, we’re far more sophisticated compared to the US. But what’s interesting being based in London, working for a Silicon Valley company is, if big tech is based still in Silicon Valley and Shenzhen, they’re only now in the last year looking to Europe and taking note of this conversation. And as of yet, we’re still not seeing enough action. And again, that’s what we’re kind of working we’re working as a team to accelerate is, you know, how do we invest better? How do we nudge product managers and engineers to devise frameworks? How do we convert this great talk to action? And this is where, you know, the pledge is really powerful, is in that it’s been co-designed by a group that work on tech. At the same time, I always have to— I think it’s always worth us interrogating. The pledge has to be partnered with real action on the side of companies. It’s not up just to the individual. That’s a very long and winding answer, Peter. I’m sorry.

Speaker C: That’s okay. That’s fine. So one of the things about this, though, is how on earth do you find out what damage you are going to do in the future? One of the points of the pledge was to say that they didn’t want to be working on technology that eventually did bad things. It’s going to be very difficult to find out what things are bad when you don’t know anything about how they could be developed in the future.

Speaker B: Yeah, I think that’s very true. I think also, you know, I’ve personally worked on things which were launched with the greatest of intentions that in time we’ve learned, you know, you look at the evolution of our thinking around gender binaries in the last 10 years, like, was evolving. And I think we’re very unkind to ourselves as a species too. I think we forget that we’re always in evolution, and that’s a really positive and exciting thing. You know, if I thought now that humans now are the best they can be, I’d be a bit depressed about it. I think we’re always changing. I think that’s where some of the tools we funded, Ethical OS, developed last year. You know, so, we’re a bit unusual in that we’re a fund that does for-profit funding, nonprofit funding, but increasingly work, you know, my background is around really in the community building space and the product space. So, last year, we were kind of looking at, you know, I personally worked on Google Maps, maybe like 10 years ago. And, the way that you build product, you know, you design a product roadmap as a group, and then you execute it. And, you know, often these are kind of 5-year cycles. And it struck me afterwards that, you know, there was nobody in the room asking us to kind of imagine, you know, a lot of the toolkits that you need to kind of imagine the future and possible consequences are not toolkits that tech companies naturally have. You know, and I’m thinking of, you know, my background as more on the creative side, A teammate that I work with, Amma Jah, is a futurist, which I never really understood, and also could be critiqued, but she’s so— her skill set is really helping people imagine possible outcomes. So, we funded and then ran loads of user sprints around, what would it look like if we inserted something into the room? If we were that, if we had a framework or toolkit that asks those hard questions? And the Ethical OS, I’ll send you a version of it afterwards, we’re in the middle of creating version 2 right now, is a toolkit for product managers to ask these questions when they’re either starting a new product or creating product changes. It’s basically consequence scanning. It’s not perfect. I don’t believe any tool is, but at the same time, I think there are 8 risk zones that you can kind of go through, whether it’s, you know, sort of misinformation or the kind of the impact that tech has on climate change that I think gets really not talked about enough at all. I think it’s about, you know, the pledge itself is necessarily imperfect as a start, but I think it’s about how do we put products and tools like this, not necessarily ours, but any tool, our tool is open source anyhow, like, how do we help teams either design what works for them or adopt something that can become industry standard?

Speaker C: Those values that Sarah Drinkwater talks about in relation to the Omidyar Network, work are similar to the concept behind another leading technologist think tank. Ezer Raskin used to be one of the brains behind the ethical search engine Mozilla. Now he’s decided he wants to do a little more.

Speaker D: I’m one of the co-founders for the Center for Humane Technology, and we are a group of tech insiders originally that were one of the first groups to stand up and point at the ways that the incentives and our business models and products are tilting the world towards societal incoherence, extremism, and threatening both our mental health and our democracies. So that’s why I’m here, because we’re talking about a clear diagnosis of the problem so that when we come up with solutions, we can move in the right direction. And that’s everywhere from thinking about it at the policy level down to the pixel level.

Speaker C: Okay, so when you talk about social incoherence, what do you mean?

Speaker D: What I mean is, like, imagine a world where every time, like, you went to Wikipedia to research whatever, you got a different version than when I went to Wikipedia, right? Like, Imagine the kind of incoherence you’d have when, like, basic fundamental facts are now tailored not to what’s true, but to the thing that maximally would grab your attention, right? Well, that’s not true for Wikipedia. It is true for pretty much everything else. When you go on Google and you do a search, you’re getting a set of results that are tailored to what Google thinks you’ll be interested in and what you’ll click on. When you scroll through your newsfeed, feed and discover what’s happening in the world via your friends or via an algorithm, that’s being tailored to what that algorithm thinks you in particular care about. If you click once on a far-right or far-left politician on YouTube, YouTube’s like, “Oh, I guess you like it,” and it starts showing up the very first in the search results because it turns out extreme things win. We are being pulled into deeper and deeper extremism by tools that were, you know, as recently as 2007-8 sort of heralded as like the burgeoning of democracy, Arab Spring. Now we actually see that the effect is driving us exactly in the wrong direction.

Speaker C: I mean, it’s interesting, isn’t it? Because one of the things that people used to say was wisdom of crowds. Wisdom of crowds is a great thing. But it seemed to be the wisdom of crowds was a great thing, so as long as it was middle-class kids doing it. Now you hand it over to the people at large, it becomes the wisdom of the mob, doesn’t it?

Speaker A: Mm-hmm.

Speaker D: I think it’s a little unfair to just jump from A to B like that, but the wisdom of the mob or mob mentality is something our technology is incentivizing. On Twitter, posts which get a lot of engagement are recommended. And so that means if you say something outrageous, or you don’t even say something outrageous and somebody takes issue and they start like denigrating or calling names, well, people get worked up. And the things that are enraging to us are engaging, and that drives directly back to the business models of these companies. And so the fundamental infrastructure of the platforms are pushing us towards our most extreme, most amygdala, most limbic version of ourselves. So it’s not just about like wisdom of a crowd, because that places all of the blame back on us, the users. But that’s a kind of cultural gaslighting, right? We’re not being shown what we want. We’re getting shown what we can’t help but look at. When we drive past a car crash, We can’t help but look at the car crash, and then that makes us much more likely to crash. That’s what’s happening at a society scale with the architecture of social media and the architecture of our news systems.

Speaker C: How much of a new thing is that, though? Because the internet tends to be an amplifier. Yeah, that technique that you’re talking about that exciting of people and polarizing. The media used to do that before. Speaking as a former tabloid journalist myself, you know, you would say somebody would say something to you in an interview, you’d immediately run off to their polar opposite and say, you’ll never guess what this person’s just said. And then you start to, you know, and then we would say a row has broken out, but we basically orchestrated it.

Speaker D: Started it.

Speaker F: Yeah.

Speaker C: So how much of that is a new thing? Is it just this amplification factor that occurs because of algorithms?

Speaker D: That’s exactly it. So you’re right, it’s— you could say, you know, there’s nothing new. We’ve always had propaganda. We’ve always had this sort of like, you know, he said, she said kind of journalism. But what’s different now is that our technology is with us 24/7. We check our phones 150 times a day. That means every, what, like 6.4 minutes, like, you’re taking a look at your phone. So it’s, it’s with you from the moment you wake up, generally before you get out of bed, to the last thing you look at before you sleep. So it’s just like the surface area of our lives is much more, which means the effect is going to be bigger. And then two, it’s not just coming from journalists or outside. This is now coming from our friends, right? There’s the Solomon Asch study on social conformity, which, I don’t know, it essentially says people conform to what they believe the people around them think, right? The study was there are 3 lines of different lengths, A, B, and C. You give a person a new line and you said, like, is this the same length as A, B, or C? It’s really trivial. You just measure with your fingers. You get it right 100% of the time. But when 3 people give the wrong answer first, 5% of people always conform to the wrong answer and believe not the evidence in front of them but what other people say. 20% of people never conform—all of the listeners, I’m sure—and 70% of people conform some or most of the time. So that means there are 3 times more people that conform, and that effect goes up when it’s people that you know and that you already trust. So that means it’s very different because it’s intermediating our, our social connections. And then the last one is that it’s— there’s an asymmetry of power and knowledge. That is, newspapers before couldn’t target specifically to your interests and your weaknesses. Now with the recommendation engines and AI, all these systems means that what’s coming is targeted for you in particular. So those three things make it a very different kind of beast.

Speaker C: As Mike Butcher said earlier, the Copenhagen Pledge is similar to the Hippocratic Oath, which all doctors, nurses, and medical professionals must be aware of before they are allowed to practice. It starts with a simple premise: first, do no harm. And one of the first scientists to try to adapt Hippocrates’ formula to the internet was Dr. Philip A. Laplante, as we heard in our Password show in 2014. He’s a professor of software and systems engineering at Penn State University in the US.

Speaker A: Well, you know, I have to admit I haven’t tried to craft a software version of it, but probably would begin with the words “First, do no harm.” It may echo in some ways the Nurse’s Nightingale Pledge, which talks about never knowingly administering a harmful drug. We could revise that wording to say, “I will never knowingly create a software that will cause injury or harm to another human being, nor cause destruction to infrastructure.” But I think it would be very, very difficult to do. And I think attempts have been made. Took to create a Hippocratic Oath-like code of conduct. You know, the ACM has one, the Association for Computing Machinery, as well as the IEEE, and both are international organizations. I’m sure IET and others do as well. It’s difficult to do and it’s controversial, but I guess you have to start with something. As I stated, at the time I was thinking very deeply about the issues and problems of licensing professional software engineers in the United States. I came across a book, oh, the title’s going to escape me, but it had to do with, I can send it to you later. It had to do with the evolution of licensing of medical professionals in the United States. And up until the turn of the last century, there were still many states that did not fully, that did not license physicians. And so you’d have on the one hand, physicians who were trained at Johns Hopkins University and Harvard, and on the other hand, You’d have country doctors who had never completed a high school education, but were trained in various forms of folk medicine, and they were practicing as physicians and calling themselves physicians. I thought likewise, we have a parallel in software engineering where we have many highly trained individuals with baccalaureate degrees, graduate degrees, doctorates, and so forth in computer science, software engineering. And related disciplines writing software for critical infrastructure, medical devices, and so on. And at the same time, we have individuals with no training in software engineering developing code, which may in fact work, be very good, but the disparity seemed great, and it was very, as I said, analogous to the evolution of licensure of medical professionals. And again, that drew me back in the long line of reasoning to the starting point, which was the Hippocratic Oath. Do no harm.

Speaker C: The motive behind the medical profession’s pledge is reflected in the title of the UK government’s new plan for restricting damaging content on the internet, the Online Harms White Paper. It was also parodied in the slogan adopted at an early stage by Google. Don’t be evil. At least we can assume it was a parody, since Google’s 2017 attempt to create an ethics board to guide its innovators collapsed after the first meeting in April 2019, some 5 months before the Copenhagen initiative. Still, Google continues to invest millions of dollars in projects such as the Google News Initiative that aims to mitigate the damage it causes. In that case, it’s the loss of revenue suffered by newspapers, which cannot compete with the tech giants and their vast offer of information. Facebook is going one step further and co-funding an Ethics in Artificial Intelligence Institute at the Technical University of Munich in Germany. Yeah, it’s a very complicated name, Netzwerkdurchsetzungsgesetz. I’m actually not sure how that would translate into English. But it’s true. I mean, there was already some—

Speaker E: so I wouldn’t call it the Facebook Lloyd.

Speaker C: It applies to others as well. In particular, Twitter. There’s a lot of discussions about Twitter here and Twitter blocking certain content and maybe overly blocking content which is not harmful at all. But it’s true that Certainly Facebook as well as some other companies are looking to the German scene with a particular eye because, yes, the reception here is maybe more critical than elsewhere. And I don’t think this is— this might be a good idea. This might also in some other regards be not such a good idea. But, but whatever, it’s certainly true that that there is some way. Well, on the one hand, we have strong engineering departments, strong computer science departments at my university, also at the Technical University of Munich. But we also do a lot of work on ethics. And so I think the challenge is there to move forward with this in a joint fashion. So not just to have the philosophers or social scientists work on one aspect and the engineering people or computer scientists work on a totally different aspect. That’s why we have here in Munich in particular, we have coined this term, or others use it as well, the human-centered engineering. And I think this is a good concept, human-centered engineering, that it must be clear that What engineers are doing, what they are designing, should have also, should serve some purpose in this way of ethics or sustainability and others. That was the University of Munich’s Professor Christoph Lüdtke. And now to another academic. Dr. Jonathan Cave is a top UK government expert on the economic impact of regulation on business. He sits on the UK’s Regulatory Policy Committee and has recently worked on the cost of the UK’s Data Retention and Investigatory Powers Act and as an advisor to the Department of Digital, Culture, Media and Sport on its internet harms legislation. As Dr. Cave points out, one of the big issues is not people saying that they will do no harm. It is machines doing harm like AI systems making decisions about personal buying behavior that we know nothing about because we don’t know what the machines are doing.

Speaker A: It may not be possible to place a duty of care on the people who use these algorithms or indeed the people who write the source code because especially if they are— well, reinforcement learning is sort of okay, but deep learning is much harder. Because you can’t actually tell what the algorithm is doing, or there’s no more concise description than the entire history on which it’s been trained up. And if you have algorithms that interact with other algorithms, even that’s not sufficient, and you get a kind of complex or emergent pattern of behavior that is— so even if you ask people to be responsible for what their algorithms did, It’s not clear that that’s an efficient way of placing responsibility. It won’t necessarily lead to anything good. It may lead to things that are extraordinarily bad.

Speaker C: For Professor Leutke, who sat on the German government’s ethics committee on autonomous vehicles, this too is a key point. For me, the key point is that the data, uh, when used gets anonymized to some extent. So probably not totally anonymized. And so the data should still be of value. I mean, we want big data applications and so we need to have big data, otherwise we will not have the information. So we will need to, to anonymize these data. And of course, there are many people working on that. So that’s in principle, this is not the problem. The problem is where do you draw the line? To what extent should you anonymize that in such a way that it’s still of value? But anonymization in the face of an AI may be a delusion, according to Dr. Cave.

Speaker A: There was a big fuss about the GPS system being used to target cruise missiles and things like that. And the response of the Americans who ran GPS, because that was pre-Galileo, was to lower the resolution of the GPS so that it would only provide locations accurate to within about 6 meters. And that was to prevent these sort of militarily dangerous uses. But people then rapidly found that a bit of line-of-sight calibration of that blurred signal— so in other words, the GPS signal plus, oh, here’s this building whose coordinates I know— is sufficient to allow you to use the GPS signal to navigate as precisely as you like. And so these sort of recombinant uses of data, some of which are protective, protected and others not may entirely circumvent the protection.

Speaker C: But would you care if your data were misused under GDPR by an autonomous vehicle if it saved your life because it knew who you were and crashed into someone else the car considered less valuable? Indeed, we may not even know how AI systems will interrelate with with each other. Dr. Cave again.

Speaker A: It may not be possible to place a duty of care on the people who use these algorithms or indeed the people who write the source code, because especially if they are— well, reinforcement learning is sort of okay, but deep learning is much harder because you can’t actually tell what the algorithm is doing. Where there’s no more concise description than the entire history on which it’s been trained up. And if you have algorithms that interact with other algorithms, even that’s not sufficient, and you get a kind of complex or emergent pattern of behavior that is— so even if you ask people to be responsible for what their algorithms did, it’s not clear that that’s an efficient way of placing responsibility. It won’t necessarily lead to anything good. It may lead to things that are extraordinarily bad.

Speaker C: So that’s one of the things when we spoke first about all of this that struck me, is that— and it’s one of the points about this big AI debate, because obviously if you have an AI that is owned and run by a very, very large company, you’re not going to stand hope against that, because as an individual, and even if you buy an off-the-shelf AI of your own to try to protect you, the bigger AI will just beat the hell out of the one that you’ve got.

Speaker A: Well, particularly because big in this case may mean faster. And of course, what we see with like, um, computer-based financial trading is that a slight improvement in speed translates to an enormous selective or competitive advantage. And it’s actually one of the paradoxes that among entities that are more or less close to each other, slightly stupider algorithms will outperform smarter ones, simply because they work that little bit faster. And in fact, really stupid algorithms like moving averages can work extraordinarily fast because you don’t use software for them. You just, you know, make the silicon to do just that one operation. But the other thing is that the selective advantage comes down at least as much. I have a graduate student who’s just handed in her thesis, which is on machine learning in auction behavior, bidding for places, well, Google words and things like that. And it may be that the advantage comes from access to large bodies of data or preprocessed data. And so it’s not necessarily the size of the firm, But if having more data or better models using those data is a comparative advantage, you then create a huge incentive for these firms to hoover up as much data as they can and to become what you might call data vores, right, like carnivores. They just eat data, not data that they have any purpose for eating. And so if you thought about it in terms of in terms of private data, you would immediately fail the kind of legitimate purpose tests that are built into DPA 2018 or GDPR. But nonetheless, there would be a correlation. The other thing is that these correlations don’t depend on the data all pertaining to the same subject, which might be, for example, in the private data sphere, it may be the data about you tell this algorithm everything they could learn from observing those data about me. But because they’re data about somebody else, they’re not protected in that way. And so it’s like, oh, years ago there was a big fuss about the GPS system being used to target cruise missiles and things like that. And the response of the Americans who ran GPS because that was pre-Galileo, was to lower the resolution of the GPS so that it would only provide locations accurate to within about 6 meters. And that was to prevent these sort of militarily dangerous uses. But people then rapidly found that a bit of line-of-sight calibration of that blurred signal So in other words, the GPS signal plus, oh, here’s this building whose coordinates I know, is sufficient to allow you to use the GPS signal to navigate as precisely as you like. And so these sort of recombinant uses of data, some of which are protected and others not, may entirely circumvent the protection. Well, like differential privacy, right? That I don’t know you, but I know people like you to an arbitrary level of precision. So there is this—

Speaker C: essentially the reversing of the anonymity because obviously, yeah.

Speaker A: Yeah, I mean, so part of it may be that you can identify people Part of it may be that you can do the things for which identification is useful in ways that don’t require you to identify the person. So if I collect data about you, maybe they’re useful for targeting you, but if it is something like, I don’t know, your purchase behavior regarding cars or refrigerators, things you’re not going to do very often, the value of those specific data may be pretty low, but the value of your data in understanding what the market looks like, you know, what, how people will respond to specific bits of information, well, like the Cambridge Analytica stuff.

Speaker C: But the transparency over how those decisions will be made in the future will become a legal battleground if the current lack of clarity in the courts is anything to go by. Germany’s Facebook law turns out to be less draconian than it appears, at least in the case of Green politician Renate Künast. Backed by a charity called HateAid, Künast demanded to know the identity of more than 20 people who had posted comments about her which are too vile to repeat here. The regional court in Berlin ruled that the comments were acceptable in the context of an online debate about sexual abuse of children. So she’s taking it to a higher court, and we’ll be watching with interest to find out whether it’s really possible to legislate against anonymous online haters. The idea that judges can decide what online comments are acceptable sticks in the throat of most internet libertarians. But if not judges, who should decide? The owners of the multi-billion-dollar tech giants, sitting on their beanbags or playing football in their brightly colored Silicon Valley co-working spaces? Politicians, many of whom are the targets of extreme verbal violence in social networks? The murder of British MP Jo Cox, which followed a campaign of hate speech, and the shooting at the pizza restaurant Comet Ping Pong in the US are both chilling reminders that words lead to deeds. So how should we legislate in order to enshrine ethical values and human rights in the online world? In the European Parliament, John Howarth, the Labour MEP for South East England, takes a robust approach.

Speaker E: You know, I’ve changed jobs and changed careers rather a lot in comparison to a lot of people. Personally, I find that exciting and invigorating. Not everyone does. It makes them feel insecure, it makes them worried, it makes them feel, particularly if their skills are thrown on the scrap heap, somewhat redundant. And I don’t think that’s a great world to want to live in. So I think that we need to start thinking seriously seriously now about the kind of solutions that we will need in 20 years. Because if we simply leave it to the big IT companies, nothing will change. And they will all claim, of course, that they’ll be able to educate everyone.

Speaker C: Isn’t there finally and eventually going to be this wonderful situation which will occur, the singularity will occur, suddenly the artificial intelligence system is so intelligent that we won’t need to make any decisions any longer? We don’t even need to do these things, we just need to breathe.

Speaker E: Well, I think you need to keep the humanity in this, in these things, and keeping the humanity in this and ensuring that people can make decisions and people can be thinking beings and make decisions that reflect their personal freedom as individuals and not have to ask my car where I’m particularly going to go there and make some decisions you know, in a spontaneous way, and not to necessarily to allow the state to know where I want to go or what I want to do or what I want to eat. And they’re all very worrying ethical aspects which need to be addressed. So I don’t know if I’ll see the singularity in my lifetime. I’m kind of thinking probably not, but it’s certainly something future policymakers are going to be thinking about.

Speaker C: It is a sort of battle for determinism, isn’t it? One of the other things I’d like to talk to you about is copyright directive. Jimmy Wales, people like that, a lot of the internet libertarians who started off on this technological dream that we’re now undertaking, they’ve been saying that the soul of the internet is being lost because of this move towards regulation. That because of this disruption that occurred, we’ve all got used to using free things. There’s no free.

Speaker E: There’s no free. This is, this is a huge, massive illusion. People pay for things. People provide value in exchange for services. Microsoft and others constructed the ludicrous notion of free services. And it’s a massive con. There are no free services. Nothing is free. Somebody is doing the work. Somebody’s providing value. Somebody’s objectifying what they do. There is no free internet. The free internet is a libertarian construction based on the ability to do anything we like to make as much money as we possibly can, and somebody else is paying. They’re either paying with their data or they’re paying in other ways. It was always, however, about data. Now, the Copyright Directive is— the simplest conversation I can illustrate that convinced me I was entirely right to back the Copyright Directive is one I had with someone who called up my office the day before the vote. And as I was going through the list of people who used to have valuable roles, who were reimbursed for their roles and their talent in the creative sector, like musicians and filmmakers and so on and so forth. And I pointed out that many musicians were simply not able to earn a living. I was told at the other end of the phone that musicians are people who will happily work for nothing because they’ve got a great job and it’s great fun to do and that they don’t need to be paid. Well, I actually, I play a little bit myself, but I could never be a professional musician. I do know how much work goes into preparing for that moment on stage or that recording, and it is entirely right that these people are properly rewarded for their work. And they have been comprehensively ripped off. And the notions of copyright that apply to people who created images, who create films, who create television programs and so on, need to have their interests protected too. So for me, The Copyright Directive was about fair work, fair pay, and rights over intellectual property that the people who created the so-called free internet are happy to exploit other people to make money from.

Speaker C: But they’re saying that essentially the act of copying is so easy to do on the internet that the act of restricting supply By—

Speaker E: No, no, it’s not restricting supply. It’s restricting theft. Stealing people’s work is not okay. And when you have your work stolen, as I have, you will come to the conclusion pretty rapidly that that is not okay. And that’s the situation we’ve been in. And also, we’ve been in a situation where journalists— and, you know, I agree with Rupert Murdoch. On this. Journalism isn’t free, you know, it requires people to create stories, create professional stories, and so on and so forth. Also, the notion of a free internet where anyone can say anything regardless, as positioned by environmental libertarians like the Pirate Party and some of the Greens, or economic libertarians, again such as the right in the United States, is all about being able to say, okay, we can say anything we like in any forum. Well, I actually don’t think that type of freedom of speech is freedom. I think that that is a particular form of freedom regardless of the harm it does. And the same people will frequently argue that people like the Christchurch assassin who went into those rooms and shot those people have the perfect right to deploy anything on the internet. Well, if that’s the free internet, I want to regulate it.

Speaker C: Strong words there from John Howarth on a call from Brussels. But as Brexit rumbles on, the views of British MEPs will of course be sidelined, and we must turn to Westminster for guidance on how to give the Copenhagen Pledge some legal teeth here in the UK. I’ve been part of deliberations in the House of Lords on the ethics we need to apply to artificial intelligence and the wider question of how to use technology for good, or at least to do no harm. So I’ve been able to observe what’s been going on firsthand. The politicians want a code of conduct that forces the tech companies to examine their behavior here, and tech companies are asking for laws that they say that they will work to. The cynics, they say, that those laws will simply mean that the tech companies will have something to avoid. Lord Tim Clement-Jones, a Liberal Democrat life peer and lawyer, chairs the Lord’s Artificial Intelligence Committee.

Speaker G: It’s really tricky, this, because if you’re not careful, we’ve got a whole bunch of stuff, everybody making pledges. I mean, we’ve got Google principles, we’ve We’ve got Microsoft doing pretty much the same. We’ve got the G20 expressing principles back in June. We’ve got the OECD, we’ve got the EU. There’s a whole bunch of ethical principles knocking around. And for my money, although, and I’m very keen to tie the development of AI to sustainable development goals, because I think if AI is going to be really useful, that’s what we need. We need AI to have public benefit, and the way, the best way in my view, to show that public benefit is to make sure it meets the UN Sustainable Development Goals, you know, in terms of conservation and wealth distribution and so on and so forth. Now, I don’t think anybody’s— not many people are getting that explicit, but I do think that we’re increasingly, in a sense, getting a marriage between some of the climate change ethics and some of the AI ethics. And I think that’s a really interesting development. But, and my big but, is that unless we have the means to make sure that people are compliant with that, you know, and this is all about ethically aligned design, which the IEEE is very keen on, those sorts of things, unless we get that right with audit, with impact assessment, real upfront design, ethical design. I just don’t think that’s going to really add up to very much. It’s going to be very pious and it’s not going to be practical.

Speaker C: Okay, so what would you like to see? Evidently, people are very, very concerned. It was very interesting that in Copenhagen, people were actually starting to talk about Hang on a moment, we’ve got to change the world so utterly that we’ve even got to change the financial system. Capitalism itself is part of the issue. The way that companies are constructed is part of the issue. What do you think about that?

Speaker G: Well, I think there’s, I think there’s some truth in that. I mean, certainly when we’re looking, when we’re looking at the fruits of, if you like, the AI and the fourth industrial revolution, I think we may well have to rethink our corporate structures in order to make sure that what we don’t have is a situation where those who own the AI and the algorithms and so on are the ones who make all the money out of it. And all the rest of us are sort of wage slaves basically in the cause of developing AI and in a sense are the subjects, are the objects rather than the subjects. This is what I’ve always we’ve talked about in terms of AI being our servant, not our master. But that means that the people who own the AI have to be very conscious that this isn’t just a question of using AI to substitute for jobs and all the kind of slightly dystopian things that we’re thinking about where AI does everything and we just put on the scrap heap. I don’t agree with that as a vision. I don’t think it’s likely at this stage, but I do think we need to guard against it. And the first step, and I don’t think we, we know that liberal capitalism is under threat in any event in many ways, but I don’t think we need to go so far and start radically altering what we need to do in the first place in terms of structures. But I do think we radically need to alter our corporate governance. So I still think, I still believe in competition. I still believe in, if you like, the profit motive for our commerce and trade. But I do think that the way in which we govern our capitalist structures does have to change. And it has to change pretty radically in terms of corporate governance and the way that we look at how we introduce new technology. And I think also the way that we think about how we share the fruits of the technological revolution.

Speaker C: Sipping water with lords and ladies in an oak-paneled committee room and swigging lager and wine with tech experts in a steel and glass Scandinavian conference hall, I’ve realized there’s a wide gulf between these two groups of people. They think differently. But the Copenhagen Pledge might just be a bridge that can span this gap in knowledge and understanding. It’s a fine set of words, though. I say so myself as someone who worked on it. You can find it online at www.techpledge.org. Here are some final words from Chris Messina, the inventor of the concept of the hashtag.

Speaker E: How do you look at the reduction of climate change as being something meaningfully measured and then create a sport around who’s going to reduce their impact on climate change the most in a way that has an index like the stock market and says, here are the people who are winning gaining and losing. So we haven’t done that sort of broader contextual work because we’re so early in this process. I mean, just remember the hashtag was created in 2007, so it’s been 12 years since that point, right? I don’t know how long it took to start the stock market or like any of these things, but we have gone through such a rapid technological change and we have scaled up so quickly because the Industrial Revolution gave us the ability to scale and propagate changes throughout society very quickly that we’re now having to confront an enormous amount of changes across an enormous number of people across an enormous geographic space in a way that we’ve never really had to deal with before. So we’ve made ourselves really, really efficient and really, really powerful, but we’re kind of like, oh, we didn’t really stop to think about or have any way to know how this could possibly go wrong. And so I’m trying to look at those things and not be too naive, but also think about, well, what was it about the culture that was in place in 2006, 2007 that was either undeveloped or immature or may have led to these things. And if we had listened better, if we had been more empathetic, if we’d slowed down to see— even recently Biz and Ev from Twitter have said, “Yeah, you know, in the beginning we really weren’t thinking about all the people complaining about abuse, you know, all the people of color who are, you know, being attacked.” Because they didn’t feel the pain.

Speaker C: They didn’t see the pain. What is that? So what’s the hashtag now? What is this? Yeah, what is that? Is that hashtag— this— is it hashtag technological debate? Is it hashtag technological crisis? What is it?

Speaker E: It’s hashtag better humans.

Speaker C: That’s what I’ve come down to. Download your own logos for printing on t-shirts and baseball caps if you’re a techie or you have the skills. If not, there’s mugs. You can buy for your tea. The pledge is not suitable for engraving on tablets of stone. Its supporters hope it will be lived out in practice, becoming as much a part of our everyday experience as ticking the box that says, “I’m not a robot,” “I accept cookies,” or “I understand that I’m clicking away my privacy here.” Whether or not the fine words will become fine deeds Well, that’s something I’ll be investigating in future editions of Password, here on Resonance FM on the second Wednesday of every month. This edition of Password was produced by Blue Buffering and written by Jane Wyatt, and you can download the podcast at our Mixcloud page. I’m Peter Warren. Thanks for listening and goodbye.

Speaker A: This program has been brought to you by Resonance 104.4 FM. If you liked what you heard and want to support our work, please make a donation at fundraiser.resonance.fm.