PassW0rd – 9 June 2021 (Feelings)

Speaker A: This program is brought to you by Resonance FM.

Speaker B: If you like what you hear, please support our work by making a donation at resonancefm.com/donate.

Speaker C: Hello and welcome to Password on Resonance FM with me, Peter Warren. How are you feeling? That’s a rhetorical question, of course, because I’m asking it. I don’t know you, and the only way for you to answer is to send me a direct message on Twitter @petewarren. Yet it’s a question that machines are asking you tens of thousands of times a day. They don’t know you either, not in a personal sense, but they know all about you. Your smartphone apps, your Siri or Alexa personal digital assistant, your car, your social networks, all are constantly interrogating your facial expressions, tone of voice, and your choice of emojis. If you’re one of our long-standing regular Password listeners, you’ll remember Brad the toaster. Brad was so smart that if he felt neglected and wasn’t getting enough bread, He would hook himself up to the Brad Toaster online network and find himself a new owner, then arrange for a dispatch company to collect him and take him to the new address. Notice I’m talking about Brad as if he is a person. It’s not. It’s a machine. And these days, machines are very good at gauging our feelings and using these insights to sell us the things that they know we want. Take tomato ketchup, for example. There’s a whole phalanx of 80,000 people ready and waiting to tell the machines what they think of the ketchup.

Speaker A: Elissa Moses of Emotion Metrics explains: We have been mostly lab-based, and we have built a panel, however, of 80,000 consumers across America, and we’re starting to put in neurometric equipment that measures heart rate, measures galvanic skin response, and these are two very reliable indicators of how people are reacting to all kinds of stimuli.

Speaker C: So is this a sort of internet of things? I mean, it sounds as though you need to wear some sort of kit if you’re going to measure heart rate and galvanic skin response.

Speaker A: That’s absolutely right, but it doesn’t need to be worn all day long. It’s worn by appointment when somebody is going to actually be viewing certain media, surfing the internet, trying a product in their kitchen. It has a finite beginning, middle, and end to the research program at this point in time. I mean, in the future we might think about having neurometrics as like a Fitbit where we can track people all day long, but that That’s a good idea, Peter, but it’s a little bit in the future. For right now, we’re very happy with having it be focused and finite, and it’s working quite beautifully.

Speaker C: You say you have 80,000 people. Presumably they’ve agreed to do this. You know, you’re not turning up on their doorstep, jumping on top of them and sticking things on their body, are you?

Speaker A: Now, our panel of 80,000 is a potential panel. It tells you the universe of people that we could go to based on different characteristics, but we have not placed nearly that much equipment, and we only place it— we’re starting to roll it out on an as-needed basis based on the kind of studies we’re doing. So for instance, we did a Super Bowl study recently testing advertising, and there we needed people that were going to be watching the Super Bowl anyway. And were fans. And so we recruited on that kind of basis. But if we’re looking for ketchup users, we would recruit ketchup users potentially of a certain brand, et cetera. We might observe how they use the product. And, and we also have an overlay with our StreamPulse Neuro product where we have another division that actually I had at Media Science called HeartConnect. And HeartConnect is a qualitative research streaming platform. And so we’re able to do very holistic in-home research where we can observe what people are doing, get their non-conscious measurement, understand how, how excited they get, how emotional they get about different elements of an experience. And then we can talk to them afterwards and get a really deep understanding of what their perceptions were. What, what were their conscious emotions? What did they like? What did they not like, etc.? And I really believe as a good researcher, you need to put all those pieces of the puzzle together. You need to understand both the visceral response as well as the cognitive response of how people process and what their opinions are. And then you really get it.

Speaker C: Just talk me through then how this works with, say, something like ketchup. There I am, I’ve got my heart measurement and I’ve got my galvanic skin response sensor. How does it work?

Speaker A: I might potentially— let’s say you’re testing a new kind of package where it pours differently, and I send you the product in advance. I give you instructions not to use it until we’re going to do the test. You’re ready for the test. And then we may do a very timed sequence where I may have you pick up the bottle and look at it. And we usually would probably maybe use a point-of-view camera or some way for to determine what you’re actually seeing. So we’re seeing how you’re looking at it and what kind of visceral reaction you’re having. We might ask you to turn it over and read the ingredients We might then ask you, and again it’s done in a very structured time way, to open the package because maybe we’re testing how difficult or easy it is to open and if people find it a struggle. And then to put the product on something. Maybe we’ve told you ahead of time to make some frozen fries or chips as you call them. And so everything is orchestrated and choreographed for a product test like that step by step. Essentially, the answer, Peter, is anything that you would do in a lab can now be done at home. It’s as simple as that. It’s that the technology has advanced and that the expertise and skill set so that we don’t have to think about artificial environments. We can think about the home as a lab and to get much better information because people are doing things naturally the way they would in a normal context.

Speaker C: That’s market research. It’s perfectly respectable and, as Elisa was at pains to point out, it’s strictly governed by ethical codes. But where are the ethics when it comes to mapping our facial expressions, logging our moods and counting our likes?

Speaker D: Non-existent.

Speaker C: This is market research in the wild. New York City computer scientist Noah Levinson describes it as stealing your feelings.

Speaker E: Introducing facial emotion recognition, a new kind of artificial intelligence that decides how you feel just by looking at your face. It’s already in the apps, devices, and websites you use every day, including this one you’re using right now. And it’s so fast and invisible, you don’t even notice that it’s learning all kinds of interesting things about you. Like, you clearly prefer men. Wow, isn’t this tech fun? It’s like an expert you never asked for.

Speaker C: That’s the name of the short film that Noah made that exposes how social networks have deliberately created software that collects data points in social media users’ faces, voices, and text messages and uses them to target adverts at exactly the right person in the right place at the right time to make a sale. Noah Levinson wrote and narrated it.

Speaker E: Stealing Your Feelings, well, it’s an interactive film that watches you back. The movie is about sort of the societal risks posed by emotion detection AI, which is a new kind of artificial intelligence that claims to be able to determine how a human being feels just by visually analyzing their face. And so Stealing Your Feelings watches you react to the content it shows you, and it makes a series of decisions about you in ways that you probably wouldn’t want a machine making decisions about you.

Speaker F: So why did you want to do it? What prompted you to want to do this?

Speaker E: It started as a hypothetical thought experiment. Like, I knew some things about emotion detection AI because I’m an engineer and a computer scientist, and so I had been working on computer vision problems related to the human face. And I thought, you know, there’s this whole class of applications like Snapchat and TikTok and Instagram that we’re giving unrestricted access to our faces all the time, right? We’re taking selfies and putting the silly filters on and doing dances. And so I thought, well, wouldn’t it be far out and quite sinister if every time we did that, these companies were secretly using emotion detection AI to figure out how we feel and then correlate that with other inputs from our devices like our geolocation, or what content we happen to be consuming at that time, then they would kind of know how you feel about wherever you’re at or whatever you’re looking at. And that’s very valuable information about consumer behavior that they could sell. But I thought, I liked it a lot. I thought it’s a really far out kind of dystopian science fiction idea. And I guess I started Googling it to see if anybody else had thought about it. And I quickly discovered it’s not really science fiction because Snapchat had filed patents indicating that they had been thinking about it for a few years.

Speaker F: Okay, so what you’re basically saying is that with that information about how you’re feeling coupled with other information, you could find out that some people had a particular liking for multi-story car parks because they’re happy when they’re in a multi-story car park. Just as a stupid example, but that sort of thing. Is that what you’re saying?

Speaker E: Precisely, yeah. And actually, to be specific about the examples in the Snapchat patent, They drew pictures of a parade, for example. How do people feel when they’re at this parade? For example, a Pride parade, or like a parade celebrating, well, anything. Another example was, what about a political rally? What if we get how everybody’s feeling when they’re attending a particular political rally? That sort of thing.

Speaker F: Okay, well, I mean, in the case of the political rally, that’s very dangerous, isn’t it? Because that basically means that you can find out what someone’s political allegiance is.

Speaker E: Yes, or you could, depending on whether you believe that emotion detection AI is settled science, you could either correctly extract that or incorrectly extract it, which might be just as dangerous.

Speaker F: So why are companies doing this? What’s in it for them? It’s a lot of effort to go to.

Speaker E: Oh, it’s just the money, isn’t it? I mean, this is how they make their money, is by surveilling us and extracting data about our behaviors and preferences and building personality inferences about us. To sell to advertisers or whoever else is interested, which is just about everyone who sells anything. The problem is that these big corporations are very, very adept at marketing their potentially bad data as truth to their advertising partners. And so we could already be in a world where all the companies that make the products and services you need, and your local government, and the schools you wanna send your kids to, are buying data about consumer behavior and making decisions about how to price things and what should be available to what people, and it’s based on garbage insights. We don’t know what happens to society, but I think most people agree it’s bound to have a profoundly destabilizing effect.

Speaker F: Do you think there should be more transparency on this?

Speaker E: I’m not certain that transparency would solve the problem, ’cause I’m not sure that there’s a way to present the truth in a way that laypeople could understand.

Speaker F: Does it worry you that in a sense this could be deterministic? Because if somebody’s finding out what sort of mood you’re in, apparently if you’re in a happy mood, you’re more inclined to buy things, for example. And people will start to push things at you based upon what you’re watching on Netflix. Or, you know, if you like this film, then you like this, this other people like you. You liked these things. If you’re eating this food, you might like this. I mean, it does sort of rob you of volition a little, doesn’t it?

Speaker C: Yeah.

Speaker E: I mean, the dystopian promise of these techniques is sort of like, now the ads can read your mind. That’s sort of my TL;DR on this, which is like, you know, you don’t have to anthropomorphize AI and think about it as a big scary monster that’s coming to get you. All this is, it’s about advertising in a world that’s increasingly controlled by corporate advertisers. So it’s just a very extreme version of advertising, and it’s the same financial incentives that are involved, but the techniques are far more sophisticated than ever before.

Speaker F: The interface that you’re involved in is, in a sense, betraying you all of the time. It’s very 1984.

Speaker E: It is, yeah. You know, I think in many ways you don’t have to intellectualize it too deeply. It’s just obviously not a world you want to live in. I think some of the solutions, I’m not so interested in these solutions about making it transparent to consumers or explanatory AI. I don’t think that’ll help. But I do think one thing is just laws that restrict how the information can be sold. So like kind of the canonical example is like none of your information can be sold to a health insurance company to determine your premiums in the States. That’s like clearly a law that should be passed. Right? Because maybe we can’t prevent companies from extracting data and collecting it and analyzing it, but we— it should be pretty simple to prevent companies from using it against your best interests. Sadly, I think there’s something— when I think about it at my most— there’s something it reveals about human beings that’s actually wonderful, which is most of the trouble we’ve gotten ourselves into is because We want to connect with each other. That’s what we’re using all these products for. It’s like we’re lonely and we want to be around other people and connect with people and fall in love with people and make friends. And that’s the part about it that’s really tragic, in my opinion. They exploited the thing about us that makes us human.

Speaker C: As Levinson’s highly acclaimed film points out, this data scraping is perfectly legal because Snapchat and other networks have patented their algorithms And you agreed to it because you accepted the terms and conditions, and there is no law against harvesting users’ reactions and emotions. If there is, by his own admission, Levinson would not have known about it since he called it his TL;DR. That’s too long, didn’t read. How I love acronyms. Yet perhaps there should be a law against emotion harvesting. That’s certainly the opinion of lawyer Romain Robert of the None of Your Business campaign based in Vienna, Austria. It was founded after privacy campaigner Max Schrems took Facebook Europe to court for hoarding his personal data and storing it on servers in the United States. He won, and his legal victory is known as Schrems I. Still, It wasn’t enforced, so Max came back for another go and won a further case at the Court of Justice of the European Union.

Speaker F: Shrems 2.

Speaker C: The None of Your Business campaign, or NOYB— if it’s tech, there has to be an acronym somewhere— aims to get it enforced. But as their lawyer Romain Robert told Password’s Jane Wyatt, it’s difficult because users don’t seem to mind losing their right to a private life to get online.

Speaker D: Do we really have to see a risk if I don’t want to protect my privacy? Do I really have to explain myself? That’s something that we see a lot from the industry. They used to ask, what is the risk? Guys, it’s not up to me to explain the risk. If I don’t want to give my data, it’s my decision. I don’t have to justify the fact that I don’t want to share my data. But should I have to share my data? The risk is quite clear because we had a nice example of what can be a risk. The risk is quite clear. If you don’t have a data protection regulation, that means that all providers and companies can share any kind of information with anyone. That means that your pictures on the cloud that you have of your holidays, that can be naked or not, because again, it’s not about intimacy or something. You are totally allowed to refuse to have your not naked picture revealed to the world. It’s like, that’s what I don’t really like, this kind of risk, this term risk. It’s not a matter of risk. It’s just for you to decide whether you want to reveal your information to the world or not. And I would say to go a little bit further, we have also quite concrete risks like the Cambridge Analytica data breach that you heard about, I guess, 5 years ago when there was this manipulation from this company of the data to influence and nudge people to vote a different direction. And that’s very concerning, of course, and that’s because of the power of this company to abuse the data, to manipulate the data. And that’s what I would like to reach, and it’s very difficult to explain, but The more you give data to these big companies, the more you just feed the giants, the more power you give them, the more power you give them, the more power they will have and the more power they will use. And that’s also something we know in competition. You abuse your dominance if you have a dominance. You abuse your power. And that’s the risk. Collectively, we will be subject to a huge power that we cannot control. The government, you can vote for who you vote for. And there is a real risk, of course. We’ve been, especially in EU history, we know that the information you give to the government may be really risky to you, but we don’t have the control that we have on the government when it comes to these big companies. We cannot vote for who is the CEO of Amazon or Facebook. So you don’t know what they do with the data. And this notion of choice and risk is at the core of privacy, of course, because with the big data, for example, all they want to do is to avoid accident. We have this idea of the data, you know, is like the same thing as what we live day to day. We can do with the data what we can do with you, human, because your data is you. But I’m not data. I’m less than that, and I’m more than that. I don’t want to be nudged by Amazon to know what I may like tomorrow. I want to be another person than the one that I’m today. And that’s the collective risk of privacy. I hope I made it clear, but I think it’s another level of understanding why it’s privacy so important.

Speaker G: Absolutely, that’s very clear indeed. May I ask, do you actually use social media networks for your own personal communications? A lot.

Speaker D: I use a lot Facebook, I use a lot LinkedIn. I must say, because I really have a free choice, I don’t use WhatsApp, I use Signal, for example. But it’s just, it’s not about that. I think we should be able to use these products, which are wonderful products. I don’t have anything against Facebook or Google or anything. It’s there’s nothing against that, but they have to comply with the law and they don’t. They don’t because they have the power to do so. And you see SMEs and small companies trying to struggle with a really strict regulation in the EU about data protection, and the big ones, which were obviously the first target of the EU legislature, are still not complying. We won, and Max won, Schrems I and Schrems II last July. It’s been 7 years that he’s struggling to make this judgment enforced by the Irish DPA. Because it’s Facebook, of course. So we have two judgments quite clear of the Court of Justice saying that transfer cannot happen to the US, and guess what? These transfers are still happening. So it’s not about how bad is those services and products, it’s how bad are they compliant with the law, and they are really bad in this moment.

Speaker C: Perhaps this rather blasé attitude to privacy is rooted in in our secret love for our devices and desire to entrust them with our intimacies. Angela Sasse is Professor of Human-Centered Technology at University College London. She tells me her research shows a general willingness to open up to computers that dates back to the 1960s.

Speaker B: I don’t know if you know Weizenbaum, who was one of the pioneers of basically of computer research, right? And he wrote this program called ELIZA. Which essentially just used this sort of reflection, this mirroring technology where Eliza would just repeat. And he was himself very surprised to find how readily people would engage with it and Bessie’s dad. And what came out there is that in contrast to confiding your problems or discussing your problems with another human being, people did it because they felt the computer wasn’t judging them.

Speaker F: Which is odd, isn’t it? Because we know that that information must be going to a human being. I mean, people are apparently prepared to confide some of their most intimate secrets, and they’re prepared to talk to people about their sex lives.

Speaker B: I think when you say we must be aware, I think in that moment, you know, and if you’re in a particular sort of like train of thought or flow, and you know, if you start confiding something or writing something, right? That in itself, there’s partly cathartic effect to doing that. I don’t think you really are aware at that moment what potentially could happen with it, who else could use it, and so on. That once, you know, we only have one focus of attention. And if your focus is on, you know, I’m burning yourself, I don’t think people really think that much about the consequences, you know. And I think they certainly don’t think back to the terms of service or whatever that they signed up to and in which they agreed that this data could be used and so on. Most people don’t read those because they’re far too long. And even if you had time to read them, you’d also need time to take a course in legal, in law, and to understand what you’re actually agreeing and signing to. And I think that’s one of the fundamental problems. So I think that awareness isn’t really there.

Speaker F: And this is in despite of all of the stories and the wealth of material that comes out that says that, of course, we are being analyzed. We know we’ve been analyzed. Lots of people notice in a strange way that suddenly adverts appear on their web searches because, say, for example, they’ve decided that they want to go on holiday to Portugal. It’s—

Speaker B: I think the part of the problem is that this kind of— I mean, we actually talk about a habit of disclosure. You have to give your information so many over and over again, and people are in the habit of disclosure that it’s actually with— this is one of my PhD students did a study on this where people were completing the non-mandatory fields even though they didn’t have to do it because having to decide whether you to give that information or not is more work than just executing this habit of, you know, if you’re so used to filling this information in, you just give it. Do it without thinking about it, because stopping and thinking about it is effort.

Speaker F: So what you’re saying is that we fill in information that we don’t need to give, that we’re not necessarily asked to, it’s voluntary to give it.

Speaker B: Because it’s a script, it runs like, you know, there is that cue, you’re familiar that somebody puts a form in front of you and you fill it in. That’s what we do hundreds of times in every week in our interactions. And I would say that actually some of these companies collect this They’re very aware of this and they’re using this mechanism. So much of our behavior, 80 to 90% of our behavior is automatic. And that means we don’t think, we just, you know, our brain recognizes a cue and executes a script that’s stored in long-term memory. And it’s very, to stop that, I mean, you sometimes see that also, for instance, when people click phishing links or so that suddenly maybe a few seconds later, they suddenly go like, ooh, maybe I shouldn’t have done it. It wasn’t in time to stop this automatic execution. And stopping, unlearning some of those habits is really, that requires effort. And it’s actually something that in our work now, we are really working, we have to work very hard to get companies to understand that they need to, if they want a new secure behavior, is that they really need to remove all the cues to this old behavior. And that’s also what you would need to do as an individual. I think that’s one, you know, you really, it’s a lot of effort to undo a behaviour that’s very deeply learned and embedded. You need to do that very consciously and very systematically.

Speaker F: So all of this information is being collected about us all of the time. So when we’re typing answers back in these little chat boxes that now pop up on websites, when we’re talking, all of this information is being gathered and it’s being gathered about what sort of emotional state we’re in. And you can infer all of that from this information.

Speaker B: I think if you really wanted to get a change of these habits, right, you must be familiar with some of those intervention programs they do when people spend too much money or when their habits are unhealthy, right? It’s that you basically have a big table and you put down basically big wads of £20 notes showing this, you’re spending this much in a month or in a year on coffee in Starbucks, right? And then when it’s there, really physically looking, that’s very different than this just habitually spending. And that can prompt a rethink and saying like, no, no, I don’t want to do this anymore. I want to change my habits. So if we basically had like a sit down, and there’ve been a couple of attempts to do this, to show people just how— who’s got what information about them and what’s being collected as a result, then suddenly people might go like, oh my God, that’s really— that’s really not— you know, this is some— something I want to— most people do self-presentation. They want to control what other people see about them and know about them. And that they start to realize that in the online domain, you need to do that as well, and that you’re just giving away lots of information all the time. I think it’s just a thousand little, little cuts.

Speaker F: You seem to be implying then that there is a need for almost regulation on this, or that we need to have some sort of messaging system very similar to that that you now see for adverts about betting, for example.

Speaker B: Yeah, that give you, that give you feedback on, that basically give you feedback.

Speaker F: A lot of people aren’t aware of this. So many people say, ‘Oh, if they are monitoring people, they’re not interested in me because I’m not important. What I say is unimportant.’ They don’t realize that what essentially people are doing is looking for patterns. That’s how they’re deriving the value.

Speaker B: I think some of the— what’s been more visceral is this, for instance, this image collecting thing that’s just been in the news, that you’re being, you know, and that people now realize that you’re being identified almost anywhere in the physical real world based— and people, governments make decisions about whether to give you a visa based on these things. I think that’s basically jolting. One intervention we tried, it’s quite a while ago now, but that we actually showed people, we mocked up some really targeted advertising, you know, and so if I was sitting down, you know, it would suddenly pop up a picture of me aged, you know, which we then electronically morph to age and say like, look, if you’re not using something, don’t start using face cream, you’re gonna look like this in 10 years’ time, you know. And we there realized that people were really quite shocked, right, about these kind of— so if you basically, if people feel it becomes too personal and if it becomes creepy, then a lot of people will sort of stop and think and withdraw. But the question will be, you know, can you really withdraw if all your kids’ school stuff is organized via Facebook? And so you can’t stay off. So I think ultimately the point you made about regulation is a good one. You know, people should be able to see what’s collected and they should be able to delete it.

Speaker C: At the start of our interview there, Angela Sasse mentioned Joseph Weitzman. He invented ELIZA, an early psychiatric chatbot. Weizmann has passed away, but ELIZA, being an inanimate computer program, continues to exist on YouTube, where AI researcher Karolina Sawinska met her. I mean it. For a therapy session.

Speaker H: Hi guys, today I’m going to chat with ELIZA, the legendary computer psychotherapist that fooled people. Quick background info: ELIZA was created in 1966 by an MIT professor, Joseph Weizenbaum. Interestingly, he created ELIZA to show superficiality of communication between machines and humans. What the guy clearly didn’t expect was that people would start treating ELIZA like a human being and get attached to her. Eliza fooled people that she can understand them and empathize with them. Even Weidenbaum’s own secretary was loving Eliza, and she famously said to him, “Would you mind leaving the room, please?” when she was chatting with her. Okay, so as you know, I often feel quite existential, and yeah, let’s see if Eliza can help me get out of this. Eliza? Hi? Hi, I’m Eliza. I feel quite existential. You say you feel quite existential? Yes, exactly. You seem to be quite positive. Positive? Why? Please go on. Well, I didn’t say anything positive. Can you elaborate on that? Stop promoting toxic positivity, Eliza. I’m here to talk about my problems, so suck it up. Your problems? Oh, you thought we’re gonna talk about yours? We were discussing you, not me. Perfect. I’m glad you got it. Next time I will pay for a psychotherapist.

Speaker C: Perhaps it’s not surprising that Eliza was so popular. Everyone likes to have someone listening to them and repeating or reflecting back what they say. Professor Chris Reid is an artificial intelligence expert and professor of philosophy at Dundee University. He’s been telling me how stealth marketing through facial recognition can help to boost sales of toothpaste or anything.

Speaker I: If you can identify what opinion people might hold, that’s really useful. So knowing that they really like your toothpaste is an important thing. Knowing that they really like your competitor’s toothpaste is also a really important thing. But what fascinates me is how you move from knowing what people think to why they hold the opinions they do, right? So when they give reasons for their opinions, And that I think is really revealing. And you can see there the commercial value as well. In fact, I think it’s crystal clear that if you, if you see people liking your competitor’s toothpaste, that’s one thing. If you can understand what it is that they like about your competitor’s toothpaste, then that gives you some real kind of market advantage. Sure.

Speaker C: But I mean, obviously it’s good if you’ve almost got a little symbol that comes up that says, This person’s in a good mood, this person’s in a bad mood. None of us like talking to bad-tempered people. Obviously, it’s a good clue, isn’t it?

Speaker I: It is, it is, yes. But that’s also— the stakes start to get a bit higher there, right? So, you know, if you try and sell someone a credit card at the wrong moment, then the risk there is that they don’t buy your credit card. Whereas if you’re in a meeting and you’ve got some kind of overlay on your screen that’s trying to demonstrate some emotional register of the person you’re talking to, then A, I think it’s rather worrying in terms of the reliability of the underlying science, which has been brought into question over quite a long period now. But also, there seem to me to be some fairly serious ethical issues with interpreting people’s on-the-fly discourse with an emotional context. That’s— there’s some challenge around the ethics of what’s reasonable there, I think.

Speaker C: Do you think that needs to be regulated in some sense? It does seem to be giving billionaire companies an unfair advantage if you’re using lots and lots of technology to take advantage of people, for want of a better phrase. I mean, I understand that AT&T started off this work a long, long time ago. Where they found that basically if someone was in a good mood, you could sell things to them, and that was to their advantage. That would put me in a bad mood if I knew that I was in a good mood and somebody was trying to sell stuff to me.

Speaker I: Yeah, I guess, I guess that’s true. So in principle, yes, perhaps regulation is, is going to improve things, but my, my instinct is that it’s going to be an extremely hard area police, and it’s part of a broader AI regulation piece, I think. I don’t think that any government has got a clear route to understanding how AI should be regulated yet, because apart from anything else, it’s such a fast-moving target. But clearly there are boundaries to these kinds of algorithms. There are boundary cases in their application. That, that maybe would warrant some kind of societal level control.

Speaker C: Toothpaste and ketchup, while they are important, are not going to change the world. What comes next in market research definitely will change it. The tech company Nvidia is building an alternative reality. They call it a metaverse where people or their avatars will be used to model data from the real world and virtually represent, well, virtually everything to see how a production line should be moved, a building or a city designed, or how something makes us feel. NVIDIA’s Richard Kerris is excited about the possibilities.

Speaker J: I think it’s going to be a mixture of a lot of things. If you look at how you find your way around the World Wide Web today, A few things that happened with the start of the web that kind of, you know, was the tipping point. When it first started off, you know, you had different chat groups and things like that. You didn’t really have an experience, but then HTML came along and became a foundation that everybody could rely on. That meant everything could be consistent. So whether you were sharing a document or sharing an image, all of a sudden the web became— it had a common highway, if you will. And we believe that USD, or Universal Scene Description, which was originally developed by Pixar, and it’s what we’ve used as our foundation for Omniverse, is the HTML of 3D. And by having that, you now have a common way with which to experience things. And so our goal is really to help connect these experiences together. What’s a metaverse for?

Speaker F: I mean, there must be some commercial reason for you to do this because it’s a massive exercise, isn’t it? Well, it is.

Speaker J: It’s a, it’s a huge undertaking. It’s something that we needed as well as our customers. So what is it? Well, it’s a number of different things. That example of the architect and the designer walking around a building, that’s incredibly important if you’re building some kind of new structure with lots of glass and mirror facades and things like that. If you don’t understand the sun and how it’s gonna react on these things, you may end up building something that’s just a big reflector in the middle of the city, creating havoc. So you need to understand these things from a simulation standpoint. But it also works for things like factories. We’ve showed a vision of the factory of the future where at our last GTC with BMW, where the entire factory was simulated in our Omniverse world. And why that’s important is when you’re producing 2.5 million cars a year as BMW does, every day matters. So when you’re changing the line for a new vehicle to be constructed, knowing where the machinery is going to go, knowing how the people are going to flow and how the parts are going to come in and go out and all of those things, become a very costly exercise if you’re doing it in the physical world. And by simulating that in the digital world, you can make all of these decisions and you can use things like artificial intelligence to help train the robots and train and understand how layout is gonna go and things like that. So that when you make the commitment in the physical world, you’ve actually done all of the testing and validation in the simulated world, which gives you a higher degree of cost savings than you would’ve ever had before. BMW says it’s saving them about 30% in their cost of just running the factory by having digital twins. It is going to be the next big thing for that. That’s another example of where a world in the metaverse is being used in an industrial-like thing. But then you have plenty of games and things like that, you know, on and on.

Speaker C: Richard Kerris, head of NVIDIA’s Omniverse project and developer relations at the company on the good things that tech can do for us and the flip side of our big data world. If you want to learn more about NVIDIA’s 3D copy of the world, go to our website, www.futureintelligence.co.uk, where you can listen to the whole interview with Richard and find links that will take you into the metaverse. You’re listening to Password on Resonance FM with me, Peter Warren. After this, you can hear DJ Ritu with A World in London. We’re musing on how the big tech companies behind our social media networks find out how we’re feeling and use those insights to sell us things and tell us things that they think are relevant to our frame of mind. Yet the very idea of a frame of mind means that we humans still have the upper hand here. It could be our antidote to AI. Oxford University’s Viktor Mayer-Schonberger and colleagues have just published a new book called Framers, which sets out this human advantage or superpower. You might remember Viktor from past editions of Password. He’s the professor who accidentally deleted all his digital files, including his PhD thesis, and had to start all over again. Anyway, the new book didn’t get deleted. It’s out now, and one of the co-authors, Professor Francis de Verecourt of Berlin’s ESMT University, has been explaining to me what they actually mean by framing in this context.

Speaker G: It’s a very new phenomenon that we start to learn. In fact, the machine, working in the machine, can influence the way you look at the world. And it’s not only the machine, but if you get into the fact that the machine is predictable, is very controlled, it elicits a mindset of predictability in you. And there is research that shows that if you put people in this mindset, so if you if you put them, having them to do controlled, predictable tasks, that is working with a machine, an AI, then their ability to innovate or to have creative ideas diminishes. So it is true that the use, if you do not retain, and we go back to the idea of framing and reframing, if you do not maintain and enrich your ability to construct your own mental model, you are going to be influenced, primed, by a mindset that is going to be— I do not want to say dictated, but nudged by working with computers. And there are a series of experiments on that, yes.

Speaker C: And when you say nudge, what does nudge mean?

Speaker G: Nudge means that you’re not aware, so there’s an unconscious process that because I put you in a certain environment, is going to elicit a certain mindset without you being aware of it. So it’s going to influence, I’m more likely to have this mindset.. There’s no determinism. It’s not because I’m working with a machine that I’m totally losing my creativity, but I’m going to start to, most likely, it’s a statistical result, think in a certain way, in a more predictable way. And I’m not going to try to find new alternatives, new ideas as easily.

Speaker C: And it has been said here in the UK, in fact a book came out about this just recently, saying that the government nudged people into lockdown during the pandemic, that they used messaging to achieve acceptance of that. Is that likely?

Speaker G: So, you know, I, I must confess, I, I do not know, and I certainly do not want to get into Great Britain’s politics because as a Frenchman living in Berlin, you know, I don’t think it’s my place. I, I can see how I’m going to get all the tweets and all that. So what is certainly true is— and I do not put a value judgment here— but when you’re facing a pandemic, the way you frame the situation is going to determine the type of actions you’re going to make and your chance of success. And so of course government— but it’s not only government, it’s leaders, it’s when you’re selling a product— is going to try to frame it, the situation, in a way that they hope that it’s going to be better for the population as a whole, you know, like assuming that people care about their population, they try to bring in a frame that in which hopefully they believe is going to help get rid of the pandemic and maybe find the best trade-off between the economy and the pandemic. So the way you’re going to frame and elicit the mindset, the right mindset into people, may also save the population. So I don’t want to put a value judgment in it. But it is true that the mindset, the mental models you are going to elicit in a population or in a government is going to determine your chance of success.

Speaker C: Okay, so sum some of this up then. Computers can’t frame, and framing is essentially putting together some hypothetical examples, some, or some scenarios to people and saying, okay, you know, from these scenarios, what is the thing that you would prefer? And if you want to frame something so that you wish to get the outcome that you want, you present a scenario that you think probably most people would accept.

Speaker G: So, so here you, you’re talking about framing as a communication device, and it’s true that you can think of it this way. But fundamentally, how a framework, how a mental model works is that it’s going to highlight some aspects of a situation and disregard others in a way that will help you hopefully achieve your goals. Now, you can— I can, when I talk to you, I can try to have you have a mindset where I’m highlighting some elements and obfuscating others. But even when I think as an individual, I do that constantly. I’m always highlighting, putting things in the foreground and putting other aspects in the background. And this is very powerful. This is not a bug. It has to be this way. Since you’re interested in pandemic, you know, a great, great example of that is not COVID. I mean, there is a lot of framing COVID, of course, but it’s— you go in 5 years ago in West Africa, you had the Ebola outbreak and you had 2 organizations on the ground. You had WHO, WHO and you had MSF, Doctors Without Borders. They had exactly the same data, very few data points. So a machine cannot help here because you have like, I don’t know, 30, 40 data points. And based on that, you need to make predictions about how you are going to deal with these few cases. WHO highlighted the historical cases. So basically they look at what happened in the past. That was very vivid to them. So their frame was more historical frame. And what they saw is it’s not too different. We’ll be able to manage that. MSF disregards, you know, they disregard the past, they just look at the geographical spread of those cases, and they saw something totally different. They saw something that is going to become a huge outbreak, something that is going to be hard to control and to mitigate. They had the same data, but they highlight different information based on the situation, and— that led them to make different decisions. So this is how a frame works individually, I mean, for you, or in an organization. But of course, if you want to convince someone, that’s what you’re talking about. You’re talking about the way you want to message or represent your message so that you highlight certain aspects and obfuscate others. But fundamentally, this is not a bug. You want your frames to work like that, because otherwise you have just a flow of data that doesn’t make any sense to you. Organizing data is really to highlight some pattern and to obfuscate other aspects of the situation. Otherwise, it’s just a random process or a black screen.

Speaker C: Can you use computers to nudge people? Can you analyze the data that you’ve got about people from AI systems or from big data analysis and then nudge people. Is that what’s occurring, for example, when you go onto the internet and you say, I want to buy some batteries for my torch, and then suddenly you keep on getting lots and lots of adverts for batteries?

Speaker G: Well, so first, this is not new to the machine. It’s, it’s marketing. So the idea of using marketing— if you— I’m not doing marketing at all, but if you talk to, to people who are marketers They will tell you they are not trying to nudge you. They are trying to cater to your needs. And so on the internet, what an AI algorithm is going to do is try to infer in what are you interested in. And so you visited, you know, let’s say Amazon, you look at a fantastic book called Framers, and then suddenly you go on others’ website and automatically the algorithms ‘Okay, this guy is interested in psychology, mental models,’ and then he’s going to try to propose other things. Now, of course, they want you to buy. The idea is to try to satisfy your taste. There is, of course, some nudging, because if you start to see again and again and again and again books on psychology, or, you know, the same shoes and all that, at some point, it may elicit a desire to have it. But to be honest, I don’t think this is new to the machine. I think this has always been the way business works, to try to find the customer who wants their product, basically.

Speaker C: I think, again, I mean, I think you’re right, but there was the old adage that 50% of advertising works. The problem is I don’t know which 50% of it is. Now, Fortunately, what the large companies are trying to do is find out which of the 50% works and then nudge you.

Speaker G: And that, I mean, nudge you perhaps. I think what is new is that they have an ability to differentiate among Peter and Francis, right? They can, they can, they try to find a way instead of having, I’m going to do one product for whole, they try to adjust, to customize, to cater to our needs. Now, of course, you can claim, and in some situations they will present the product in a way that nudges you to buy it, but the way the algorithm is trained is not really— it’s more the display on the website that does that. But the machine, what it does is to try to predict what you’re more likely to buy. Now, you could— you can call that nudging because maybe you don’t want to really buy it now, and so suddenly something is offered to you. But what is really new here is the fact that I’m trying to have something for Peter right after talking to Francis.

Speaker C: This is what Peter is likely to like. Okay, now the antidote or a solution to that that the large companies are thinking of is trying to capture as much data as they possibly can on us because they don’t want to, or they want that sort of certainty of the efficiency of the world, is essentially what you’re saying.

Speaker G: Yes, but I think it’s a bit of a delusion. I think big data is certainly playing a role, and you can do amazing things that you could not do before, there’s no doubt about it, but you cannot summarize reality with data. And I would even add, the data you collect reflects the way you look at the world. When I’m collecting on, let’s say, on Facebook, your friends, let’s say I want to see who are the friends of Peter, so I’m going to elicit a social network. This is a representation that people at Facebook have in their minds. It’s not a representation from the computer. They are forced to look at people as a massive social network. But life is not only a massive social network. So the data they acquired, billions and gazillions, is going to be on this social network with gender and age, and, you know, yes, and professions maybe. But Peter, Francis are much more than just this way of looking at the world. So just to finish, it can be a bit deceiving to believe that Big data is going to be enough. It’s not enough. You need to have the way to interpret it and the right friends.

Speaker C: So what you’re saying is that when people say that they’ve got 5,000 contacts on LinkedIn, that they’re not really all their friends.

Speaker G: That’s another way to put it. That’s right. That’s right.

Speaker C: Or friends, or on Facebook for that matter. Francis de Vere call. Not all thought leaders are so optimistic. Sherry Turkle is the Abby Rockefeller Moores Professor of the Social Studies of Science and Technology in the Program in Science, Technology, and Society at the Massachusetts Institute of Technology. And she’s just written another book. It’s called The Empathy Diaries, and it’s a memoir of her remarkable life and career.

Speaker H: There’s an experiment where you leave people alone in a room And you say, well, you just have to sit here for a while and without a phone, without a book, and just quietly and 15 for 15 minutes. And oh, by the way, if you need to give yourself electroshocks, here’s the machine. And the people are college students and they’re grossed out and they say, absolutely, electroshocks. That’s like crazy. Why would we want to give ourselves electroshocks? But after 6 minutes of total non-stimulation, they are giving themselves electroshocks because they cannot tolerate that just sitting. And we are so used to having a regime of constant stimulation that solitude becomes something we kind of can’t handle. It becomes intolerable.

Speaker C: One of the reasons why people are prepared to tell these computers things that they may not share with other people is because, as I understand it, it’s because they consider the computer or the mobile phone or the app less judgmental.

Speaker H: Well, that’s our, you know, that’s what I mean about we have failed each other and it’s our crisis in empathy. That’s another reason that I write the Empathy Diaries because I write about the engineering culture and its tremendous lack of empathy towards me as I came through it, towards, you know, what it was like to teach MIT freshmen. A freshman, a student commits suicide who’s in my class. And what the class talks about is whether or not he would have fell to the ground faster. He jumped off a building. And the class talks about whether or not he would have reached the ground faster if he hadn’t been holding on to something. He jumped out of the window holding a desk. And they say, no, he could have done without the desk. He would have reached the ground the same by Galileo’s laws. I mean, the way in which when engineers use engineering language to talk about people, how they leave empathy aside, and to the extent that our language has been filled with the language of engineering and science, but with the language of objects to talk about people. I think our language for talking about people have been demolished, undermined. And so I think that computers are good for many things and engineering culture is good for many things, but it’s not good for talking about human relationships and human understanding. And I think in the current situation, our current moment, you can see engineering language creeping more and more into our way of talking about everything. And that’s one of the reasons I wrote the book, to get us back to the great question you started with, why now? Is because I feel that the culture of engineering that caused me such troubles as I tell my story of a woman and a humanist at MIT, are now inhibiting our ability to seize our moment now and move forward with a greater and more empathic politics towards each other.

Speaker C: Is it too far to say that these machines are after our souls?

Speaker H: They’re not after anything. The machines do not have will. The machines do not have a personality. We’ve created an industry that does very well by scraping our data and selling it. That’s all that’s happened. The machines have not taken on a mind of their own. They’re not like chatting about this interview and saying, well, how can we get Peter and Sherry to buy some new product they’ve invented? Insofar as they’re interested in your soul, they’re analyzing what you look at on screen so they can send you more of it. That’s become an existential dilemma because once you’re being filtered only the people and the certain people, because your news is being filtered by them and what you see to buy is being filtered by them. But in particular, your news is being filtered by them. So it’s not the machines, it’s the companies and the policies behind the machines. So I think we have to get straight who the, you know, who we’re fighting. This really is a fight that consumers can win.

Speaker C: If you’re concerned about your devices and apps spying on you round the clock, listening to you talking in your sleep, and storing every grin and grimace that you make, you can contact Romain Robert at the None of Your Business consumer watch group in Vienna. You can read all the terms and conditions to find out how your feelings are being shared with advertisers, then simply untick the box. Or if that seems like too much trouble, TL;DR: switch the devices off. They’re just machines. Still, if you’re staying on Twitter and LinkedIn, you can connect with me, Pete Warren, and the Password Team— Blue Buffery and Jane Wyatt. We are all too human, and we’ll be back next month with another touchy-feely exploration of the world of tech. If you can’t wait until then, then go to our website, www.futureintelligence.co.uk, where you can find out more about the ramifications of technology and those links that allow you to enter the metaverse. Thanks for listening.

Speaker B: Goodbye.

Speaker A: This program has been brought to you by Resonance FM. If you like what you heard, please support our work by making a donation at resonancefm.com/donate.