PassW0rd – 13 March 2024 (Benevolent AGI)

Speaker A: This program is brought to you by Resonance FM. If you like what you hear, please support our work by making a donation at resonancefm.com/donate.

Speaker B: Hello and welcome to Password on Resonance FM with me, Peter Warren. In this program, Beneficial Artificial General Intelligence. It may be much smarter than you, but relax, it’s got your best interests at heart. We traveled to the BAGI conference in Panama to meet the great and the good of the AI world, to listen to their thoughts on where the technology was going and how to make it a force for good. Less a conference, more a meeting of minds, the items on the agenda ranged from eternal life to the emergence of the superintelligence, whether we should fuse with machines and live inside computer servers or link our brains together via the internet, how we would make a living when the machines had taken all of our jobs, to whether there was a need for a constitution for AI. All weighty issues. But then, so were many of those attending the conference at the Hilton Hotel. For alongside the AI scientists and politicians, senior political advisors and top economists rubbed shoulders with financial advisors, philosophers, ethicists, and science fiction writers. Outside on the horizon, container ships queued up to go through the Panama Canal, a potent symbol of the source of Panama’s wealth. Well, towering skyscrapers made a nod to the shadier side of Panama and the, shall we say, laissez-faire banking regime which was underlined by The Guardian in the famous investigation that became known as the Panama Papers. It was perhaps fitting that a number of black vultures, known in South America as omen birds, were taking advantage of the thermals generated by the buildings to soar effortlessly up the heights. They have a highly developed sense of smell and can detect decay from afar, leading to their local name. Being followed by an omen bird is not a good sign, so the numbers circling a hotel dedicated to saving humanity from itself could only be called ominous. The philosopher and futurist Jerome Glenn, the director of the Millennium Project, was one of the speakers at the event. Glenn, a former associate of the renowned thinker Alvin Toffler and one of the architects of the Strategic Arms Limitation Talks, which limited nuclear weapons in 1979, told Password why there was a need for a similar treaty to govern AI, and specifically artificial general intelligence, and not NAI, the narrow artificial intelligence we have all become aware of in the last 2 years.

Speaker C: Well, one of my concerns is that people are not aware of how soon AI could evolve beyond what they think today. So they’re not taking seriously coming up with what it might look like and what sort of rules there might be. There have been meetings of the UNESCO and there’s been OECD. The White House has got some rules. China’s got some rules. The EC’s got some rules. But they’re mostly focused on narrow intelligence that we have today. They’re not anticipating the future artificial general intelligence that we can have tomorrow. And the reason there’s concern with me is that several years ago we thought, oh, artificial general intelligence is 10, 20, 30, 40, 50 years from now. Now people aren’t saying that, it’s much sooner.

Speaker B: Sooner.

Speaker C: So to get national rules and UN rules would be a miracle if we got them done in a couple of years, but that’s what we’re going to have to do. Because if AGI could be released in, say, 3 to 5 years, and if we don’t have something in place that helps manage that, how it gets done and released, then unregulated AGI many of them, different ones, different countries, different corporations, is it seems to me that it’s not likely that they’ll all turn out to be okay. I figure there’s at least some not going to come out okay, and that could be a mess for the future civilization. So to prevent that, we’ve got to have serious conversations about future rules, about systems that don’t exist yet.

Speaker B: But at this conference, people have been talking about superintelligence. You can’t stop— they’ve been saying if you put the rules in place, they’ll just be circumvented.

Speaker C: Okay, there’s a distinction to be made here. I agree that you cannot control superintelligence, but that’s not to say you can’t influence it in the following way. We are now in a transition, or beginning a transition, from from narrow to general. We can have a say on how that transition occurs. That then will set the shape of how general gets evolved and then into super. So just like children, you can’t control what they are as an adult. You can try, but we know that’s not possible. But you can control how they are as a baby. What they eat, how they deal with friends and so forth. You have a little less control when they’re teenagers. That would be the general analogy. So right now we’re in a stage of going from little kids to teenagers, and that transition we can have a say in, just like parents can have a say in their children.

Speaker B: But you make the point the Russians aren’t playing pool. We’re in an arms race now.

Speaker C: So we will decide, we’ll go do an end run around Russia by going through the General Assembly. Any country can offer tomorrow morning a UN resolution to say we’ve got to create a convention on artificial intelligence and create a working group to put it together, the content. That could be done tomorrow morning. Now, I’m in conversation with various people, and we’re hoping to get eventually a U.S.-China resolution. Now, the initial simple resolution can be from anybody. You know, you got to start working on it, but then the working group ought to end up with U.S. and China as partners, and others can obviously join in. But that would give great hope to the world, not just because it’s a smart thing to do with AI, but it gives hope to the world that the US and China can collaborate to make a better future together rather than the old tribal nonsense in the past.

Speaker B: The philosopher Jerome Glenn, director of the Millennium Project, on the fears surrounding artificial general intelligence, the AGI that provoked Rishi Sunak to hold his Bletchley Park summit last November and his excruciating fireside chat with the world’s on-off richest man, Elon Musk. For the internationally renowned technology ethicist Wendell Wallach, author of A Dangerous Master and the senior fellow of the Carnegie Council for Ethics and International Affairs, the debate on the future is essential.

Speaker D: There’s the prospects of all kinds of things going bad, but all of these technologies are being pursued, including AI, because is this vast plethora of benefits that we can realize from them. My concerns at this stage are much more focused on the near-term impacts of AI, so the misuses of it by humans, the ways in which it’s going to exacerbate inequalities and build up the power of AI oligopolies, the deepfakes, the delusions it creates, that sometimes superficial thinking or surface research is the equivalent of intelligence. So those are the near-term things. That’s not what most people focus on. Most people who see AI as dangerous are very concerned about existential risks. I’m also concerned about tech unemployment, lethal autonomous weapons, all kinds of things that are being enabled by AI. Not that they cannot be managed, but so far it’s not clear that we’re managing them.

Speaker B: But that’s the issue, isn’t it? Everybody’s saying AI bad. What they don’t realize is it’s people that are bad. It’s the intent in the system that is the problem that we have to worry about.

Speaker D: Right now, it’s very much the intent in the system. Right now, it’s very much the uses of AI by humans, the illusions created by humans, and even, even the existential risk which people are seeing from 5 to 50 to 100 years off. Even that is actually being utilized in a much broader narrative that is lending power to those who pretend that they will control AI but actually are trying to reap profits from artificial intelligence. So you look at an example like the large language models that were released in 2022 by OpenAI and Microsoft. OpenAI was created as a nonprofit that was going to do safety research. Microsoft has advertised itself for years now as having ethics in its DNA, and yet they released this product that they both knew did not have sufficient guardrails in place and it would be problematic for education, for various jobs and so forth, for people who want to disseminate disinformation, and they did it for profit incentives. Let’s not pretend it’s anything else. They want to say we want to warn people, you know, get a conversation going. No, they knew that they needed to be first in the deployment of these technologies to have the leading position.

Speaker B: I mean, that’s the real point, isn’t it? Because whenever people criticize what’s been happening in the development of the internet and they say, oh, there’s these pornographers, there’s these people who are affecting the safety of children, What they don’t accept is that basically the internet is just a mirror to humanity itself. It’s just showing you the good and the bad.

Speaker D: It is, but I don’t think the old adage that AI or biotech or any of these things are just tools captures what’s going on either, because it’s not just that they enable this misuse they can actually drive the misuse. So you look at things where AI modules are being put in control systems because ostensibly they’re going to respond very quickly, but the problem is these are statistical tools and at times they’re going to act in ways that you had not anticipated, so at times they’re going to create crises that nobody had even considered might take place because of the way in which they respond to the inputs. So there are definitely situations in which, again, the failure is the humans who are putting them, who are deploying them, because they aren’t looking adequately at what the implications of those deployments are. But once deployed, the systems can act in unpredictable unpredictable or uncertain ways that on occasion will cause serious harms.

Speaker B: Wendell Wallach, co-author of the book Moral Machines: Teaching Robots Right from Wrong, whose latest book, A Dangerous Master, examines how current technology might not have our best interests at heart. It is a debate on the limits of AI that may be badly needed because, according to many of those at the conference, the promise for AGI is quite literally off the scale because of its potential, with many maintaining it should be able to banish both disease and aging. A potential that the economist and MIT-educated engineer José Luis Cordera passionately champions. It is time, says the transhumanist Cordera, author of the book The Death of Death, to use AI to banish dying.

Speaker E: Well, it is my personal mission today. I want to kill death before death kills me. And actually, science is advancing very quickly, and I do think that in about 2 decades we will be able to stop aging, and even better, reverse aging. And that’s the goal—reverse aging. So to live indefinitely young, not to live indefinitely old. No one wants to be old forever, but we might enjoy being young forever, and that’s my goal.

Speaker B: But why do you want to do that? Isn’t death part of life? I mean, what about all of these celebrations that you have where you have death-like figures moving around and these carnivals that are going on? What’s going to happen then? There’s a cultural significance to killing death.

Speaker E: Yes, which is fantastic. I don’t want to kill death because we need to avoid death. Death, aging and death cause a lot of suffering in advanced economies in Europe, in North America, in Japan, 90% of the people die of age-related diseases. 90%. That means that all other causes, causes of, uh, dying, including accidents, suicides, homicides, AIDS, malaria, COVID, you add all of those and it is only 10% of the people dying. The Russians also climate change. All of those, those don’t kill more than 10% of the people in advanced countries. So if we want to solve the biggest issue for humanity, we need to kill death. And we are very close also. This will be optional, by the way. You know, if you don’t want to be indefinitely young, you might die anytime, which is what we did until now. We had no choice. So we have actually made peace with death. Because we could not stop death until now. So we have been hijacked by death, and we have this Stockholm syndrome with death. We have accepted death because there was no way out. But I am not at peace with death. I am at total war with death, because now we are close to stopping aging and death forever, at least optionally. And that is my goal in life, because it causes so much suffering. For the person, for the family, for the society, for the country, for the world.

Speaker B: You’re at an AI conference. What’s artificial intelligence got to do with killing death?

Speaker E: Well, it is very important because artificial intelligence can analyze, can discover things that we humans with simple minds cannot do. For example, Google’s DeepMind company, they developed a program called AlphaFold, AlphaFold that was able to solve the protein folding problem. This would have taken centuries for humans in the past to solve that protein folding. But now AlphaFold can create any protein. And that is something that we could not do before. The same with some new drug discoveries. In the past, they were actually by random luck. Even Viagra. Viagra was discovered not for the purpose used now. It was discovered by chance. So now we can actually discover new drugs, new therapies by design. And artificial intelligence can do that. We cannot. We don’t have all the possibilities and the analysis of big data that artificial intelligence does. So it is fundamental. But there is another totally different reason, because I think we are going to be immortal twice, not once but twice. We are going to be biologically immortal, and also we are going to be computationally immortal. We will be able to upload our minds into another substrate, and for that we also need artificial intelligence.

Speaker B: The economist José Luis Cordero, author of The Death of Death, with a message that may not go down too well with the vultures known as omen birds circling the hotel as we talk. Yet it’s a world that The technology philosopher and transhumanist Max More sees as more of a human evolution than a revolutionary change. More, who is also evidently a bodybuilder, sees AI as offering humanity enhancement. According to More, we should aim to merge with the machine, becoming not only part of it but also enjoying superhuman powers. Yeah, to actually reverse the aging process, to stop it and reverse it at some point so we can just fix it at whatever, biologically 30 or whatever we feel like. And beyond that, of course, being a transhumanist, I want to see more than just good health.

Speaker A: I want to actually see us improve our capabilities, to expand our senses.

Speaker B: If you think about the animal kingdom, our senses are very poor in many ways. Like, we have two dogs, and obviously their sense of smell is vastly better than a human. Some insects can sense distant earthquakes. I mean, some can see in infrared and ultraviolet. We have this very narrow spectrum, and we’re very used to that, but imagine being able to expand that. And we can kind of do that crudely now with a new headset, the VR headset, you can actually see in infrared apparently. So it’s not just living longer and healthily, it’s actually being able to expand human capabilities. So if you’re talking about this transhumanism, you’re not talking about fusing with the machine, you’re not talking about, say, achieving eternal life because you’ve gone into a Well, that’s a possibility. Transhumanism itself doesn’t commit to any particular pathway. It’s really an extension of the Enlightenment project of humanism. The humanists basically said that whether or not there’s a divine being, it’s up to us here to use our intelligence, whether it’s God-given or evolved, to make our lives better. Humanists, though, think that there’s a limit, you know, we have to deal with human nature, just make the best of being human. Well, transhumanists say, well, why? Why can’t we actually challenge human nature itself, which includes aging and death and these limited 3-pound pieces of meat that we call brains that we think with. So I think there’s a lot we can do biologically. For instance, we are very bad at understanding our emotions. We do things and we kind of make up a story of why we do things, and we’re not really sure why we feel depressed or anxious, because the brain didn’t really evolve to do that. The brain evolved to deal with very immediate threats. So, you know, see the saber-toothed tiger leaping at you, feel fear, run. It didn’t really develop to deal with complicated technological civilizations. The transhumanist Max More on the world that he maintains may be only an algorithm away, and one that many of those at the summit in Panama were extolling. Rapid developments in AI, they claim, not only mean that we can banish death, but we can usher in a completely new existence and possibly a godlike superbeing. It’s a world that George Gilder, an influential economist and former speechwriter for US President Richard Nixon, urges caution over. AI, says Gilder, is not an exact science.

Speaker E: A machine learning and artificial intelligence system just produced its best outcome. It’s unacceptable. That, that would be temperature zero, where the machine just uses its algorithms and its learning scheme to produce the best answers, the most accurate answers. And that’s called zero temperature, and it’s unacceptable. The outcomes are just boring.

Speaker B: So what you’re saying is that if we don’t ask the questions, then, and we leave the machines to themselves, they actually come up with something that, you know, no, I’m not talking about the questions we ask.

Speaker E: I’m talking about the degree of randomness that’s getting injected into the machine in order to produce an outcome that we regard as interesting. And it’s just ChatGPT’s experiences that if they just trust the computer to give the answers that the program generates, the best, most accurate, true answers that the program can generate, that’s zero temperature, and the output is so boring that nobody will really use it. It’s in order to make it, to give ChatGPT the kind of animation and ostensible creativity and unexpected outcomes, they attach a random number generator, which is defined by a temperature. And 1 would be a very high temperature and 0 is is pure accuracy and nothing else. And the temperature that they find is optimal for people actually using it to produce usable material is 0.8. In other words, they got to inject a lot of randomness. And the assumption of AI is that randomness is tantamount to creativity, that creativity can be translated into a random number generator. And in other words, entropy gets information, entropy, unexpected outcomes get injected into the machine by just a pure randomness factor. And that is a false notion, the idea that human creativity is simulated by a random number generator. It looks— creativity always comes as a surprise to us. If it didn’t, we wouldn’t need it. It comes as— it’s unexpected, it’s improbable. The more improbable it is, the more possible value it achieves. And AI tries to generate the most probable material, and that is boring, and so it doesn’t work. So they got to simulate creativity, and all these stupid creative people imagine that the that these simulations actually are creative rather than just the effect of reducing the amount of information so there’s more unexpected outcomes.

Speaker B: The respected economist and prolific author George Gilder, whose latest book, Life After Capitalism: The Meaning of Wealth, the Future of the Economy, and the Time Theory of Money, was published last year. And yet the world painted by technologist, transhumanist, and philosopher Max More is strangely compelling to the AI scientists. Though this vision of endless potential has an uncanny echo of Aldous Huxley’s Brave New World, a place of endless leisure and sex where aging has been partially solved. Though speculation and development is only left to the very top of society, and those below them are genetically limited so that they can happily perform menial tasks. Though according to the philosopher, businessman, and author of Surviving AI, Callum Chase, paid work may be something in the post-AGI world that we fondly look back on as a luxury. It’s plausible.

Speaker A: In fact, I think A very, very glorious future is my most expected outcome. I don’t think we should shy away from the fact that there are massive risks, including the possible extermination of the human race. But yeah, I think a very, very glorious future is possible. I just— to pick up one thing, I don’t think universal basic income is the answer. The little word in the middle gives it away. We want much more than a basic income. We want a an access to excellent resources for everybody.

Speaker B: Okay, well, how’s that going to happen? We haven’t managed to do that for the entire history of the human race. We’ve never been fair to each other. We’ve been greedy. We’ve been wanting to make sure that some of us have more resources than others.

Speaker A: Oh, I don’t think that’s true. I think, I think although there is no such thing as a country where there is full equality, We are pretty generous to each other. I mean, there’s no other carnivorous animal— possibly dolphins, I’m told, might be an exception— which can assemble more than members of their own family or small tribe without killing each other. And we humans gather millions in cities and we don’t, you know, mostly kill each other. There are murders, particularly in America, where they’ve got the bizarre— made the bizarre decisions to have lots and lots of guns available, but we actually by and large get on and we are pretty nice to each other. And over the last 200 years particularly, to a much smaller extent over the last 1,000 years, we’ve improved the living standards of people unrecognizably. You know, you bring somebody from the Roman Empire to today’s world, they would not believe the wealth that ordinary people have now. We’ve still got a very, very long way to go. We’ve got totally unacceptable inequality in most places, and we’ve got a lot of room for improvement. And I think that we will make an improvement, both in terms of absolute improvement of living standards and probably improvements in quality, because we’re going to hit something which I call the economic singularity, which is the point when machines are capable enough that they can do pretty much everything that we can do for money cheaper, better, and faster than we can. And I think this will happen suddenly. I don’t think this is going to be a gradual encroachment, and first the translators will be put out of business, and then the junior accountants, and then the people working in car factories. I think everybody’s going to have jobs. We’re going to have to keep changing jobs faster and faster, but everybody’s going to have jobs. Everybody who wants a job or needs a job is going to have a job, pretty much. Until there’ll be a phase change when, over a period of weeks or months, it’ll become apparent that suddenly machines are capable of doing pretty much everything we can do, and then there won’t be jobs for humans. What we’ll need then is a different kind of economy, and I call it the economy of abundance. Now, I think this will happen. The economy of abundance is when the price of everything you need for a really good standard of living— American middle class or above— is very, very low, not free, because there’ll always be shortages, but— or scarcity, but very, very cheap. And this is going to happen for 3 reasons. One is because we’re going to take the humans out of the production process, and the humans are the most expensive part of most production processes. We’re going to have virtually free energy, thanks to massive amount more solar energy and wind energy, and eventually fusion, and there’ll be some other more esoteric forms. So energy will be too cheap to meter. And thirdly, AI will make all our production processes incredibly efficient. So because of these three reasons, things will get very, very cheap. And that, I think, will enable the economy of abundance in which we all have access to fantastic living standards. Of course, the transition from here to there could be very bumpy. And I’m not clear how we do it. And I don’t think anybody is. And the main reason why we’re not clear is that very few people are thinking about it. And that is a major mistake we’re making. We’re not taking this seriously.

Speaker B: Surely we are. I mean, here, Rishi Sunak had his AI conference. He was, you know, sort of trumpeting to the great and the good that the UK had suddenly woken up to the existential risks of artificial intelligence.

Speaker A: Well, that’s a different thing. I give Sunak credit for taking seriously and actually speaking in public about the existential risk from AI. That’s different from the economic singularity. The economic singularity is about the future of jobs and a time when there are no jobs for humans, pretty much no jobs for humans. What he was talking about is the risk of superintelligence, which might possibly wipe us out. He was explicitly denying or doubting the possibility of a jobless future. When he sat down and did his squirmingly embarrassing interview with Elon Musk where he applied for a job with Musk, he said, well, obviously, you know, there’s no such thing as a jobless future. And Musk said, well, you’re wrong about that. And for a change, Musk was right. But yeah, as I say, credit to Sunak for being willing to speak in public about the existential risk from superintelligence. It’s probably about the only decent thing he’s done in his career, but credit to him for that.

Speaker B: You’ve been talking about jobs and you’ve been talking about work. I mean, the two terms have almost being interchangeable. I mean, the problem here is going to be paid work. I mean, I mean, I get that you say that universal basic income or a small amount of money in a world of superabundance would go a long way, but paid work is going to go.

Speaker A: You’re quite right, and it’s very important to distinguish between jobs and work. So jobs are work which you get paid for, and there’s going to come a time when we won’t be able to get paid for the work we do because machines will be able to do it cheaper, better, and faster. We are wired by hundreds of thousands of years of evolution to work, and I don’t see that changing overnight. We’re going to have to become post-human to not work. And in fact, even then, I suspect work will be part of our futures, but it’ll be work that we want to do rather than work that we do rather resentfully in order to put food on the table. Work is good, jobs not necessarily good.

Speaker B: Callum Chase on the rocky road to a future world that many may not only find unrecognizable from reality, but one they may not actually want to be a part of. A world they may not think of as the world, but it’s a ripped-up world where the ultra-rich and the large corporations will also be required to change, according to the roboticist and crypto chain expert Dr. Mihaila Ileru. The president of the Impact Institute for the Digital Economy, who was awarded the research chair and professor of artificial intelligence by the Canadian Prime Minister Justin Trudeau. Describing the economic system as Moloch, a reference to a Hebrew entity said to be a child-devouring bull-headed creature, Uleru says that Moloch is the state of mind that our current society finds itself in. A belief that wealth is the be-all and end-all of technological development, where giant corporations take over our lives in pursuit of money and power.

Speaker G: Moloch is a trap in which we find ourselves. So it is a world in which we ended up, and I do not want to say that we don’t have any accountability for it, but it’s a game of of bad incentives that it’s pitting us against each other. And it’s like the arms race. So it’s like, okay, let’s say I don’t want to put advertisements on a social network. I want to play it nicely. But then if that niche is open, then you will take it and you will make a lot of money and I remain. It’s like natural selection, right? It’s a cruel game and we seem to be captured by it. So it’s like we cannot get out of it. Why? For many reasons, including our own guilt. Why? Because, I mean, because we are unwise, I would say. But one example is, well, okay, so the price of energy decreases. Oh, then why should I switch off the lights? Why? I mean, who cares? Yes. Then I will not save energy, and what does that do to decarbonization? And then here we are, right? So society seems to act like a superorganism that is driven by these bad incentives to its own extinction. And we are all in it and playing each other out based on these incentives, like in a machinery. What is worse is that even education is kind of teaching us in the same way, you know, I mean, like profits and maximizing and all, you know, how the Harvard graduates are the best at this game. And so how to get out of it? So life after Moloch is based on completely different incentives, like incentivize collective flourishing. Incentivize less consumption, incentivize quality content rather than content which is making me more angry against you on social networks. It’s just changing the game, but not just on better incentives, but as well as, when I say wisdom, more aligned with fundamental values of treat everyone as sacred, you know, more like the moral philosophy. Philosophy, like everyone and everything on the planet, treat them as sacred, respect them, and nurture them. So it’s a game of nurturing versus a game of competition and a game of obviously cooperation. So far, besides indigenous tribes, I do not know many examples of that. Yes, And so it’s an aspirational—

Speaker B: But that’s where we need to go. It’s one of the interesting things about the discussion that is starting up about AI is that in a sense what we’ve been saying is that humanity is a failure and we need AI to solve ourselves. So it’s almost a sort of abrogation of responsibility, isn’t it?

Speaker G: Yes. And here what I have to say is that Indeed, I also have the hope that the artificial general intelligence will help us find the wise rules, let’s call it this, rules of collective flourishing. And, you know, so far attempts are being done and some of them successful. For example, if you combine blockchain and artificial intelligence and then you can track corruption if it’s happening and eliminate it from the system. And you can also do, you know, the social contract enforcement on blockchain and through smart contracts. So you can do a different game, yes, and through technology. And with artificial general intelligence, hopefully we can also discover more ways of doing this because So far, man-made rules. There’s something in us, and I’m just, you know, looking at myself, yes, and I’m like, okay, I of course would enjoy and enjoy having the penthouse at the Hilton here versus, you know, even an ordinary room or a room in a worse hotel. And then I’m thinking, okay, so what would it take? Yes, so imagine the same manner. Of course, every— it seems like most people aspire for more money, more power, more having than being, right?

Speaker B: The AI scientist, roboticist, and cryptocurrency expert Dr. Mihaila Uleru on the need to evolve a different state of mind to develop AGI into an inclusive system that will work for humanity. That its proponents are dubbing Beneficial Artificial General Intelligence, or BAGI, one of the baffling list of acronyms that the AI revolution is spawning. And it’s not only the acronyms that are new, so are some of the terms in this brave new world of the future where nothing is but what is not, to quote Shakespeare’s Macbeth. For in Panama, We are in an unconference put on by SingularityNet. Singularity is a grouping of thinkers and AI companies that is aiming to bring about this change, and along with her other roles, Uleru is its chief alchemist. The organization is led by the renowned AI guru Ben Goertzel, a man who has taken the adage, if you want to get ahead, get a hat completely to heart and spends much of his time wearing a trademark leopard-patterned wide-brimmed hat. And he’s not alone. Half of the conference splits along hats, flowing locks, ponytails, and topknot lines, with a sprinkling of the more conservatively dressed like Glenn Wallach, Chase Gilder-Cordero, and a few others. Though while exotically hatted, Götzl is remarkably pragmatic when it comes to setting out his ideas on BAGI.

Speaker H: Beneficial AI has the obvious meaning of artificial intelligence which is oriented toward good, and of course that raises the question, who’s good? We’re thinking in this case about human good overall and the good of other living and sentient beings, which can include the ecosystem on our planet, but also the good of AI minds that we may create and that may create each other. So, you know, there’s not many people out there looking to create malevolent AI evil scientists sitting around thinking, like, how do I create AI to annihilate everyone or just make everyone’s life more difficult. On the other hand, nor is it the case that most entities creating AI now are doing it primarily for the good of humans and other sentient beings, right? Most AIs being created in the world now are being created either to make money for some company or to enable some country to sort of boost its status relative to other, to other countries, right?

Speaker B: So how do we arrive at good then? I mean, as you’re making the point, it’s very, very difficult. What’s good for Russia might not necessarily be good for America, might not necessarily be good for the UK. What’s good for the stock exchanges may not be very good for the people on the street. How are we going to get that line?

Speaker H: Well, as someone who grew up as a programmer in the open source world, I’m somewhat inclined toward the idea you do it by at least creating an environment that doesn’t explicitly block you from building things for human benefit. So the internet And the Linux operating system are two examples of reasonably open decentralized playing fields on which people can build benevolent or malevolent things and we see cases of both obviously. But I think this sort of open and decentralized infrastructure to my mind is a good start, that, that at least means if you want to, you have an opportunity to engineer benevolent and beneficial AI systems. And then beyond that, beyond having an infrastructure that doesn’t stand in your way by being owned by some party with very special interests, beyond having an infrastructure that doesn’t stand in your way, I mean, to me it’s about what are the goals you’re giving your AI system? What are you actually directing it to do? What is it doing with its time? What are you doing collaboratively together with it? I mean, are you plotting an invasion of enemy territory? Or are you figuring out how to cure diseases that help children and make elderly people’s lives better, right? Are you working with it to create new creative art that’s never been seen before, or are you working with it to plagiarize other people’s art without compensating them, right? So I mean, that’s it. There’s not, you know, there’s not that much of a mystery about how to achieve beneficial things. In my view, doing it perfectly is hard, and there are always trade-offs in life, but there’s not that much mystery about how to do broadly beneficial things. I think the challenge is more that people have a divided motivation between doing things that are for their own self-interest and things that are broadly beneficial. It’s not like— if you look at the major problems in the world today, you know, 60% of Ethiopian kids are brainstunted due to malnutrition. But it’s not that big a mystery how to stop that. I mean, there’s some war zones in some parts of Ethiopia now, but that’s quite recent. And you can still fly to every major city, right? I mean, we could actually fly and truck food to these kids every day. It’s not that hard to imagine how to do it. We’re not doing it for complex reasons. And I think the same is true with AI. Like, it’s not that hard really to imagine how to raise an AI and teach it to be compassionate to people and have it help people. The issue is that then these controlling resources have other things on their mind. So then the dialogue always comes down to, well, how can we do good? Good as a side effect while at the same time making ourselves a lot of money here, but at the same time making sure we conquer the enemy, and that becomes a hard problem, right?

Speaker B: The AI guru Ben Goertzel, leader of Singularity.net, a reference to the singularity, the event that the former Google chief scientist Ray Kurzweil has coined for when the machines achieve consciousness. So how do we protect ourselves from, as Wallach has said, keeping technology from slipping beyond our control? Do we regulate the technology, fuse with it, or try to change its intent? Do we take on our terrible tendency to worship Moloch and sacrifice everything to our relentless and corrupting drive for money and power? According To many of those at the conference, it is the only solution to evolve into a future that changes the intent in our society. It’s something that Toufiq Saleeba, an AI scientist and the head of Hypercycle, says that can be achieved by changing the structure of the machines of the huge tech corporations. Instead of allowing the consolidation of the processing power of AI and our data centrally, Saliba and others argue it must be decentralized. We must make the processing of data democratic, effectively giving the power back to the people. A technique, those at the conference say, means that people will be able to make money from the processing of their data by storing it locally, i.e., on their computers or their phones. Every time it is used by another machine, or in association with other data, a fee will be generated that will allow people to share in the economics of the AI world, and one that light-speed processing and infrastructure will enable.

Speaker F: Saliba is adamant that the system is a solution. It’s enabling the artificial intelligence to run in a fully decentralized setting. What that means is that the alternative would be centralized, which is the current state. A centralized AI that is running by Microsoft is subject to what Microsoft decides that this AI can or cannot do, who or who it should not provide services to, and for how long, and what would it do with whatever intelligence that it collects from somebody. In full control of it when you have it centralized. And the intent of having an AI decentralized is the enablement of that AI agents— AI is a bunch of tiny little agents that they’re sitting inside themselves— to free them into the world. If you were to free them into the world, they need to have some methods to be freed. And the AI decentralized is just— it was a— at the time it was just like an idea that seemed to appeal to a lot of folks because Freeing those tiny little modules in a decentralized setting so that not a single entity can control them seems to appeal to a lot.

Speaker B: Is what you’re saying that you’re going to share out the AI, everybody’s going to have a piece of it, and the AI is going to be the combination of all of those things? Correct.

Speaker F: We will have a global brain that’s been a dream for AI scientists exist from the beginning of the computer science days, and a lot of folks talked about it, even wrote the code about it, like Marvin Minsky in the ’70s. In ’83, wrote a huge thing about it. In ’86, wrote a book about it. So it’s been a dream of AI scientists to get to that and achieve that in a proper way.

Speaker B: Is it the sum of all of the processing power of everybody’s phone.

Speaker F: Or not only just phones, all the computation that is in the world will have the capability to coordinate, cooperate, and compete. So that’s analogous to Homo sapiens coordination was amplified when the internet existed. Now when internet existed, the certain companies, they did not want to put their employees into the internet because they were afraid of the unknown. So they locked them into this thing that is called intranet. The intranet, for those that don’t know, large companies still have intranet, which looks like the internet but just for those people inside that company. And when they are confined to that intranet, their collective intelligence is confined to those walls. They started looking into opening those walls to the world, they were concerned that they’re going to have a lot of intelligence leaked out. But it turns out that yes, some intelligence leaked out, but more intelligence actually coming in. And those that didn’t open their doors, they didn’t really benefit as much as they did. So fast forward to 2 and a half years ago when I was at the AGI Summit at the Computer History Museum in California. California. I basically posed that question again and seemed to appeal to a lot of AI scientists from around the world, but they knew that there’s a missing ingredient, and they know their stuff really well. And they said, like, it’s been tried. In order to enable AI agents to be free, they need to have a certain thing that protects them the same way we have firewall on the internet, we have the internet open all the time and whenever we want. How can this open it whenever it wants? How can it have a spam filter and all of these things? So the communication layer is missing to enable the peer-to-peer because up until 2.5 years ago, if one AI agent is going to be freed— so let’s say that this This is the AI agent, it’s part of 100,000 other AI agents inside, let’s say, Microsoft. It’s going to be free and it’s going to reach out to some other AI agent to ask it for some help. The other agent is going to be like, why would I help you? And this one will be like, I will pay you later. And it’s going to be like, you’re right, I’m going to do the work and you’re just going to run away, you’re never going to pay me. And that dilemma exists for us Homo sapiens, because if this guy says, I’ll pay you the penny, do the work, this guy may take the penny and run away. We Homo sapiens solve this dilemma by having those things in the middle that they’re called banks or ledger-based blockchain. They’re slow, but we are slow, so we can tolerate that slowness.

Speaker B: The AI scientist and head of Hypercycle, Tufi Saliba. Though cynics would say it’s a move that sounds disturbingly like the Russian post-communist initiative to give shares in former state-run run industries to people on the streets who had no idea what they had been given. Something that allowed billionaires like Roman Abramovich to buy them up for cash for a fraction of their value from people desperate for currency. Yet it’s a system that the science fiction writer David Brin, whose apocalyptic book The Postman was made into a film starring Kevin Costner, another delegate to the conference, thinks will work if we can give AI systems entity, something he describes as a soul. Do this, argues Bryn, and AI systems can become extensions of ourselves in the system and look after our interests in cyberspace.

Speaker I: Well, for one thing, we tend to respect things that have souls, but You know, that’s a little too mystical for a modern scientist like me. Soul is a shortcut term for having an ego or having a sense of individuality. And most of the people in the AI field, some of them are geniuses bringing us this crisis at high velocity, like a truck heading for a wall. Most of them don’t think in terms of AI that can be looked upon as individual beings. There are three formats of AI that are clichés that they all seem to subscribe to. One is that these new entities will be controlled by 20 or 30 major groups— Google, Microsoft, the Defense Department, Beijing, Wall Street. And the historical parallel for this is feudalism, castles, lords with castles on their hill, and the AIs being viewed as their men-at-arms. So there’s a historical parallel to that. And it ruled our ancestors for 6,000 years, most of it. And it was hell. The second format that they often use is chaos. And that is that we’re creating this new ecosystem, just like the 3-billion-year-old ecosystem on Earth. Consisted of sunlight hitting plants. Plants turn it into carbohydrates. Herbivores eat the plants, turn it into protein. Carnivores eat the herbivores, and there’s parasites all along the way. We’re creating a new ecosystem that consists of electricity, computer chips, memories, and all the information information in the internet. And already there are free-floating or cyberorganisms floating across this new ecosystem. There are free-floating cyberprograms that are replicating in borrowed memory space all across the internet. So it’s a prodigious achievement. God made the Earth’s ecosystem We’re making a new one for new beings as we speak. But the assumption is that the highest order creatures in this ecosystem will either be, as I said, controlled by 20 or 40 macro corporations and entities and nations, or else number 2 is that there’ll be blobs. They’ll float like invasive species throughout this new ecosystem. And the closest parallel there is historical chaos or a 1958 Steve McQueen movie called The Blob. It can seep through any cracks and so on. The third assumption people make is that these new entities will coalesce into a macro superorganism being, as like the Skynet godlike AI in Terminator or the Master Control program in Tron. There have been many sci-fi films about all three of these, and there are lots of historical parallels because that one has the history historical parallel of absolute monarchy or despotism. All three of these models for AI, these formats for AI, are clichés that come out of human history or out of science fiction or both. And if AI takes any of those three forms, we are totally screwed. But as it turns out, There is another possibility.

Speaker B: Scientist and science fiction writer David Brin, author of The Postman, on why the singularity might just be closer than anyone thinks, and on why it might not even be the singularity that we have been thinking about. If Brin is right, it might be very good news for us and very bad news for the omen birds circling around the conference unless they can find a way to input some decay into the AI models that Cordero and Moore hope will give them eternal life. It’s whole worlds away from the AI systems being debated by Rishi Sunak and Elon Musk and may still be science fiction, but for how long? You’ve been listening to Password on Resonance FM, written and presented by me, Pete Warren, and produced by Blue Buffery. Thanks for listening and goodbye.

Speaker A: This program has been brought to you by Resonance FM. If you like what you heard, Please support our work by making a donation at resonancefm.com/donate.