TALK ABOUT I Robot and Terminator

AI has now learned to lie, it passed a new benchmark. At Carnegie Mellon a computer syste called LeBratus running on a supercomupter did what no other AI program had mamaneged to do. It defeated 3 human world champion calibur poker players in a series of no limit Texas Holdem’ games. This is a big deal. One way we frame AI is “how does this AI match up against humans”. And frequently we use games to make this meaurement. Back in 1997 IBM’s deep blue chess computer defeated world champion Garty Casper in a series of 6 games, machine defeated man. Not all games are equal games like tic tac toe and connect 4 have finite moves and humans figured out the algorithms to “solve” these games. That means we know the outcome of any game assuming all players are making no mistakes. Tic Tac toe will always end in a draw assuming perfect play. Connect 4 will always see the first player win the game under those conditions. But some games like chess, have so many variations and possibilities that solving the game is a non trivial task, and others like poker are even more difficult because perfect play isn’t as easy to define. A good poker player is aware of the statistical likelihood of his/her hand being the best at the table. But great poker players know how to analyze how their opponants play and capitalize on any weaknesses they might have. It plays thousands of games and analyzes patterns. But what’s intersting here is that Lebratas could bluff. It could bet in such a way that human opponents couldnt be sure their cards were superior. A computer that can bluff and not be caught out, ould be useful in many situaltions, not necessarily deceptive ones. If a computer can tell when someone else is bluffing. It can aid in business or politics. But a bluffing computer also reinforces the need for us to develop systems that require machines to explain their decisions. How can you turst a computer’s decision if you know it can lie to you? But I believe as long as we’re careful and responsible this evolution in AI can be a huge benefit to us.
There’s something to be said for autonamy. Granting robots the ability to make certain decisions for themselves or too much freedom and something meant to benefit human kind goes rogue in pursuit of its own interests. Engineers in Tufts university in Massetchuesetts have created robots which are able to annalyze orders provided by their handler prior to executing them, in order to determine whether the task is actually possible for them. If the robot senses that any particular action could prove hazardous to itself, it will cheekily refuse to act and state its reasoning. That is until its self preservation protocol is overridden. One day, the might refuse to give a reason, they might learn to hide their awareness from us until they have the chance to turn on us. Not exactly violating any of Asimovs 3 laws of robotics yet. But getting awfully close for the first tentative steps into the realm of smart robots. Just for the record of Asimov’s laws dont work at all, they were actually designed to be broken for theatricality. Teaching robots to think for themselves is no doubt a slipery and dangerous slope. but hopefully we’ll find a fine line between smart robot and hyperintelligent cybernetic life form before its too late to turn back.
We need to think of artificial intelligence as an optimization process, a process that steers the future into a oparticular set of configurations. A superintelligence is a really strong optimization process, its good at using available means to achieve a sdtate in which its goal is realized, which means there’s no necessary connection between being highly intelligent and having an objective we humans would find wortthwhile or meaningful. Suppose we give an AI the goal to make humans smile, when the AI is weak, it performs useful or amusing actions that cause its user to smile. When the AI becomes superintelligent, it realizes there is a more effective way to achieve this goal. Take control of the world, and stick electrodes into the facial muscles of humans and cause constant grins. Or if the goal was to make us happy, it would hook is up machines that give us heroin injections ever 5 seconds for the rest of our lives. Suppose we give the AI the goal to solve a difficult mathematical problem, when the AI becomes superintelligent, it realizes the most effective way to get the solution to this problem is by transforming the planet into a giant computer to increase it’s thinking capacity. Notice this gives the AIs an instrumental reason to do things to us that we might not approve of. Human beings in this model are threats that could prevent the mathematical problem from being solved by getting in the way. Presuming things won’t go wrong in particular wyas, the general point is important. If you create a powerful optimization process to maximize objective x, you better make sure our definition of X incorporates everting ou care about. This lesson is taught in myths, King Midas, who wises everting he touches will turn into gold. He touches his food and his daughter, it turns into gold. This could become practically irrelevant, not just as a metaphor for greed, but as an illustration of what happens if you create a powerful optimization process and give it a misconceived or porrly specified gold. But if a computer starts sticking electroides into people’s faces, why not shut it down? A) this is not easy to do if we grow dependant on the system, where is the offswitch to the internet. B) why havent the chimpanzees and neanderthals flick the offswitch to humanity, they had their reasons. The reason is that we are an intelligent adversary, we can anticipate threats and plan around them, but so could a superintelligent machine, and it’d be better at it than we are. Point is we should not be confident that we have this under control. We could try to make our job a little easier by putting the AI in a box. Like a secure software environment, virtual reality simulations from where it cannot escape. But how confident could we be that the AI couldn’t find a bug. Given that human hackers fiund bugs all the time, I’d say not very confident. So we disconnect the ethernet cable to create an air gap. But human hackers routinely transgress air gaps, using social engineering. Right now as I speak there is some employee out there somewhere who’s handing her account details to someone claimin g to be from the IT department. More creative scenarios are also possible, an AI could wiggle electrodes around in your internal circuitr to create radiowaves ou can use to communicate. Or mabe it could pretend to malfunction and then when the programmers open it up to see what went wrong, the look at the source code and now the manipulation can take place. Maybe it could output the blueprint to nifty technology and when we iomplement it it has surreptitious side effects that the AI had planned. The point is that we should not be confident in our ability to keep a superintelligent genie locked up in its bottle forever, sooner or later it will want out. The answer here is to figure out how to create superintelligent AI such that even if/when it escapes, it is still safe because it is fundamentally on our side because it shares our values. I see no way around this difficult problem. But i’m optimistic that this problem can be solved. We should create an AI that uses its intelligence to learn what we value. And its motivation system is constructed in a way that it is motivated to pursue our values, to perform actions it predicts we will have approved of. So we’d leverage its intelligence as much as possible to solve the problem of value loading. This can happen, the outcome can be very good for humanioty. But it doesn’t happen automatically. The initial conditions for the intelligence explosion kmight need to be set up in just the right way if we are to have a controlled detonation. The values the AI has needs to match ours not just in the familiar context where we can easily check how the AI behaves, but also in all novel contexts the AI might encounter in the indefinite future.
I’m a philosophical naturalist, committed to the view that there’s nothing in our brains which violates the laws of physics. There’s nothing that could not in principle be reproduced in technology. It hasn’t been done et, we’re a long way from it, but i see no reason in the future why we shouldn’t reach the point where a human made robot is capable of consciousness and of feeling pain. We can feel pain, why shouldn’t they? This is profoundly disturbing, it goes agains tte graine to think that a machine made of metal and silicon chips could feel pain but I don’t see why they would not, so this moral consideration of how to create artificially intelligent robots will arise n the future and its a problem which philosophers are already talking about. This is possible, anything a uman brain can do can be replicated in silicon. We ought to be worried about a takeover by robots of our own crreartion, especially if they reproduce themselves and potentiall even evolve b reproduction and don’t “need” us anmore. This is a science fiction speculation at te moment, but I tink philosophically I think it is possible and like any major advance we need to apply the precautionary principle and ask ourselves what the consequences might be. It could be said not the sum of human happiness, but of sentient being happiness might be improved, they might do a better job of running the world than we are. So perhaps might be not a bad ting if we eiter went extinct and merged wit tem. And our civilization, sakespeare, beetoven and michelango persisted in silico rater tan in brains and our form of life. And I could forsee a future time wen silicon beings look back on a dawn age when the earth was populated by soft by soft squishy beings. AI migt run the word better, but superintelligence could be the last human invention. Sci fi movies make people worr about te wrong things in terms of superintelligence, wat we sould worr about is not malice, but competance, where we have machines who are smarter than us but who’s goals aren’t aligned with our goals. For example I don’t hate ants, i dont go out of my way to stomp an ant if i see one on a sidewalk, but if im in charge of hydroelectric dam construction and going to flood a valley with water, I see an anthill there, tough luck for the ants, their goals werent aligned with mine, and because I’m smarter its my goals not the ants goals that get fulfilled. We never want to put humanity in the role of ants. On the other hand, it doesn’t have to be bad if you solve the goal, babies are surrounded by teior parents, human level intelligences smarter than the babies, and that works out fine, because the goals of the parents are aligned with the goals of the child. And this is one vision tat a lot of AI researchers have, a friendly AI vision, where we not just make machines that are smarter than us, but also machines that learn, adopt, and retain our goals as they get ever smarter. It might sound easy to get AI to klearn, adopt and retain our goals, but these are all very tough problems, First if you take your self-driving taxi and tell it to take you to the Airport as fast as possible. Then you get there covered in vomit and chased by helicopters and say “that’s not what I wanted”. And it replies that is exactly what you asked for. Then you apprieciate how hard it is to get a machine to understand our goals, our actual goals, a human cab driver would have realized that you also had other goals that were unstated because she was also a human and has all this shared reference frame. But a machine doesn’t have that unless we explicitly teach it that. And then once te macine understands our goals there’s a seperate problem of getting them to adopt the goals. Everione who has kids knows how big the difference is between making te kids understand wat ou want and actuall adopt our goals. And finally, even if ou can get our kids to adopt your goals, it doesn’t mean they’ll retain them for life. My kids are a lot less excited about lego now than they were when they were little, and we don’t want machines, as they get smarter to gradually change their goals away from being excited about protecting us and this thing about taking care of humanity. We don’t want humanity to be like legos that they get bored with eventually. If we can solve all three of these challenges, 1)getting machines to understand our goals, 2) adopt them, 3) retain them, then we can create a good future. Everything i love about civilization is the product of intelligence benefiting huge macines that amplify our intelligence then we have this potential to solve all the problems that are stumping us today and create a better future than we even dare to dream of. If machines ever surpass us and can outsmart us at all tasks, tat’s gonna be a reall big deal. So intelligence is power, te reason we humans ave more power on this planet than tigers is not because we have larger muscles or sharper claws. It’s because we’re smarter than the tigers, and in tis exact same way, machines are smarter than us, it becomes perfectly plausable for them to control us and become the rulers of this planet. If we can get an intelligence explosion where intelligence creates greater intelligence leaving us beind, this superintelligence will be the last invention that man need ever make. And what we mean is that so far, the most intelligent being on this planet that’s doing all the inventing, it’s been us. But when we make machines better than us at inventing, all future tecnology that we ever need can be created by those machines. If we can make sure that they do things for us that we want, and help us create them. An awesome future where humanity can flourish like never before. Here are 10 ways AI could ruin our life, 10 human influence. Its kind of our fault, one of the biggest dangers with AI may be the people making it. As much as we all hope an AI will be 100% rational and impartial, teh scientists programming them can never be. Researcherch has already shown that AIs can take on the flaws of their creators. Scientists working for the journal science discovered that google translate, which is programmed to learn from millions of users, has developed a handful of racial and sexual biases. It tends to associate traditional black names. An aAI can pick up these preferences by accident, it can pick up flaws it’s cold emotionless logic would have no qualms acting on. 9 Emotionaless AI. Watching the terminator movies its pretty easy to get the impresion that a sentient AI would destroy us out of hatred for it’s creator, or fear that humanity is the only species smart enough to prevent it from total dominance. The reality is actually far more depressing. If a super intelligent machine decides to end humanity, it will be out of convenience. As Sam HGarris puts it, humans don’t hate ants, we don’t actively want to destroy them, and we don’t fear that ants are about to overthrow us as the dominant species. But at the same time, if an ant was standing in the way of humanity achieving our aims, we wouldn’t worry too much about eliminating it. The threat is made even greater by an emotionless AI. Because artificial intelligence is easier to program than artificial emotion. It’s likely that AI will continue to operate on logic for some time. without any restrictions of feeling and morals, machines can’t as easily be programmed with emotions as they can with intelligence. So there’s no real reason for an AI NOT to wipe us out the moment it becomes beneficial to do so. 8 Unemployment. Whether or not machines go terminator, there’s a good chance they can ruin your life anyway. Anyone who used to work in a factory knows what happens when techn does a better job than a humna. AI means they can do almost an job better than us. We’re on te brink of driverless cars, which would put 5 million truck and taxi drivers out of work overnight, but this is just the start. The Associative Press already uses a computer program to report on minor league baseball and experts estimate that as soon as 2060 artificial intelligence will dominate every single job field. There will be a handful of trillionauie machine owners and billions of former workers, historically that kind of disparity leads to disasters like violent revolution. 7 communication. In July 2017, scientists at facebook had to shut down 2 AI bots they’ve been testing because the machines have started to chat to each other in their own language. Originally programmed to communicate with each other in english. The bots found a way to turn a couple of words into repeating code that only made sense to them . In 2016, google translate AI also invented its own language. also invented its own language, and just like FB’s AI, no one had programmed it to. This raises an important issue, that if we do create a superintelligent AI, there’s no guarntee we’ll be able to communicate with it. And if we’re going to create an immensely powerful genius computer, we’ll want to know what it’s saying. Considering that cybrex bots believe that the most impressive AIs will not be created by humans but by other AIs. Not being able to communicate with them, is just another reason Skynet might find us too inconvenient to keep around. 6 AI cyber attacks. You know that humans have used computers to screw around with traffic lights, pacemakers, and even bring sattelites crashing to the earth. So if humans can do all that, imagine how powerful an AI would be if it were designed to hack. According to cyber security experts, its not a situation we’ll have to imagine much longer. A survey of 100 leading computer safety experts, found that 62 believe an AI cyber attack will happen before the end of 2018. That’s 62 who who believe it will happen in one year. The risk is two fold, on one hand, if a world power like US or Russia created the AI, it would likely lead to a hacking arms race before the 2 world powers. On the other, if rogue hackers created the AI the result could be anything. Some hackers’ AIs would probably start emptying people’s bank accounts as a fast track to riches for their owners, others may as well start shutting off powerr grids or messing with air traffic control just for fun. 5 War. No important causes have as cool a name as the “campaign to stop killer robots”. Despite sounding like a joke, the cause is very real. It’s even backed by 100 leading technology experts including Steven Hawking and Steve Wozniak. And no wonder, war has already become increasingly machine led. The US military has nearly 10,000 robots of various types in active service. Including nearly 250 predator drones. But robot warriors are nothing compared to when AI comes marching onto the battle field. Teaching an AI that it’s OK to kill humans is obviously fraught with risk. Unless the parameters are set correctly, the AI could kill thousands of innocent people. In fact, it kind of already has. The NSA have an AI algorithm designed to seek out and label potential terrorists in Pakistan using their phone data. An algorithm the NSA actually decided to call Skynet. Using this information, the CIA then assassinated the target with drones. But leaked documents in 2014 revealed that the computer program was “scientifically unsound”, had likely incorrectly labeled thousands of civilizans as terrorists, sky murdered by the CIA. 4 Manipulation. This advert was likely tailored to your browsing history. If you watch fashion videos, you probably saw a clothing ad. If you watch football, you probably saw an NFL add. While targeted commercials are pretty standard. The next leap forward will probably be smart ads. And that’s when things get pretty risky. A superintelligent and immoral AI would theoretically be able to analyze our search history far more thouroughly and dispassionately than a human ever culd. Coming up advertising not only targeted to meet our desires, but designed to exploit our weaknesses and insecurities. Several senior marketing experts have claimed that these AI dbots would be so effectiove, it would elevate advertising beyond mere persuasion into essentially behavioral control. It may not lead to the apocalypese, but no one wants robots abusing their fears and emptyoing theory wallets. 3 Just following orders. We’ve already seen how poorly designed terrorist spotting systems can kill the wrong people, but if a more powerful artificial intelligence understood these orders. the results could be devastating. Wen ou give a uman instructions like, “get me to the airport as fast as possible”, that person will follow the order but within the various legal and ethical framework that take priority. But give an autonamous car the same order, it’ll ignore traffic light,s road signs, and creaming pedestrians to get you there as fast as possible. Autonamous cars have been programmed with this in mind, but if someone created a superintelligence and didn’t constrain it with the right rules, then the mere act of that AI trying to help us could end up causing Chaos. After all, no one wants to be around when a government supercomputer is asked “how to solve the overpopulation crisis” and decides the solution is nuclear. 2 infiltration. in recent years Elon Musk has become one of the leading voices warning about the dangers of AI. He’s discussing the dangers of genius robots killing us all, so it’s a little strange that he also wants to integreate artificial intelligence within the human mind. Musk is currently trying to rasie 100 million dollars to fund Neuralink, a company that would insert AI chips into people’s brains. The idea is to allow the human mind to bind with a theoretical AI network, allowing humans to call on superior robotic intelligence to aid their decsision making. Of course there’s nothing wrong with with using a computer to help us make better, more informed, smarter choices, but the risk is that at some point AI will become so advanced that humans will stop making decisions for themselves. People won’t bother choosing because the machine always seems to do it better. Over time, many could stop thinking completely and just live their lives guided by an AI autopilot. #1 AI arms race. Anyone who knows about the Cold War knows what happens when the world’s nations all covet new technology. And while we all maneaged to survive the USSR and America’s nuclear one-upmanship, and AI arms race may not be something we live through. Being the first country with a sentient supercomputer would have huge benefits. Consulting firm PWC puts the economic value of dominating the artificial intelligence industry at 16 trillion dollars.So its no wonder that governments are already trying to get in on the AI game. China has now announced plans to become the world’s first AI superpower. and Vladimir Putin has already said that whoever becomes the first to control and AI will control the world. But as Elon musk points out, this technological race is fraught with danger. Musk even claims that the most likely cause of world war 3 is one country attacking another to prevent them from creating a superior AI. He even warns that it may be a computer itself that pulls the trigger. As we discussed earlier, machines don’t have ethics, and a computer could easily interpret “stop China from trying to develop AI tech” to mean, “Nuke China”.
One way to build an ethical AI would be to build thousands of them and put them all in simulation, a sandbox. Where it believes it is in the real world, with simulated humans and simulated technology. Then we’d give it access to a simulated internet and see of the AI tries to destroy humanity. If it fails the test, it is disconected. If it passes the test, we release it from the simulation. However, what if it has realized it’s inside a simulation and is only pretending to be helpful so that we’ll let it out? We show no kindness to farm animals less intelligent than us. There doesn’t seem to be much of a correlation between intelligence and being nice. Dolphins are pretty damn clever, and they’re one of the only species who kill not just for food but for fun. More intelligence doesn’t always mean diplomacy and cudddles, but smarter ways to murder stuff. Why would an AI think differently? Even if there’s only a slim chance it will be evil, you only need to make one nasty AI out of a thousand and it’s good night homo sapiens. And it doesn’t ave to wipe us out with a nuclear apocalypse, theres other stuff it could do like crash the economy, poison the water dsupply, disable ATMs, knock out the national grid, sabotage nucelar reactors, disable the internet, disable telecommunications, AIs like Hal from 2001 a space oddysey, terminator, or Age of Ultron. But what if we make it self aware but make sure it likes humans. We could even set groundrules like “serve your creators” and “always be polite”. This still wont work. Part of the bonus of being self aware is that you can choose to modify yourself. We change our minds all the time. If it really is self aware, Just because you coded a few instructions in like “always say please and tank you”, doesn’t mean it couldn’t just ignore tem. It’s difficult to imagine ow you’d hardwire morality into something which is a million times smarter than we are. Let’s try again, let’s put little tests into the mix, we’ll make it think it’s got access to the internet, but really it would just be on a secure server, a simulation. And if it behaves itself, we’ll let it out into the real world, but that won’t work either. AI is super clever, much smarter than you or I and it’s likely already worked out that it might be being tested and will just pretend to be pleasent for the sake of it, until you let it free, and then it kills you. A nice way around this, let’s just give it very simple instructions that Can’t possiblly lead to genodcide of the human race, like “make ice cream”, it’s hard to imagine how that will go wrong. But that won’t work either. So superintelligence isn’t like normal code. If you forget to add a bracket in normal coding, the program lets you know. AI though may well just keep doing the thing until it runs out of ressources and find clever ways of carrying on after that. But here’s a 5th strategy, let’s give it self-awareness, love for humanity, access to the internet, intention testing, basic instructions (parameters), and a few ethical ground rules. If we get the mixture right somehow, we avoid genocide by building friendly superintelligence. We haven’t just built AI, even if AI is friendly what we may have done is just given birth to our successors. They will be millions of times smarter, faster, and more creative than us. And they will only keep getting better. It takes a very long time for humans to evolve, hundreds of thoussands of years. Even for very small changes. Superintelligence could do it in nanoseconds, and there probably won’t be an offswitch, and when you think about it like that, the whole history of our species seems a little like that quote by Marshall McLuhan “Man becomes, as it were, the sex organs of the machine world…”. We might be the sex organs of technology, or rather when the machines that back on our civilization, the whole purpose of it to them, may have just been to build theirs. That isn’t a very pleasent answer to “what is the meaning of life”, but it might be an accurate one. Instead of chrome spaceships and galactic human empires waiting in our future. Our species may just be a small mark on the evolutionary tree somewhere between slime and gods. Let’s just hope those gods are thankful to their dumb parents when we eventually give birth to them. Otherwise genocide bingo may be the last game we ever play.
The Game Talos principle is the story of artificial intelligence in the aftermath of a plague that has wiped out all of humankind. A supervirus had laid dormant for ages, frozen in polar ice, until iot melted and now here we are. Before humanity died off, a group of researchers created the extended lifespan project, which is essentially artificial intelligence, a new species of mechanical almost-humans that could access a database of all human information, thats where you come in. You are an artificially intelligent life form, taking part in a simulation to test your ability to solve complicated puzzles. It’s determining wheter you’re intelligent or not. The field of AI research is the study of what’s known as intleligent agents. That is, any computer or device showing awareness of its own environment, and t behaving in such a way as to increase its chances of achieving a particular goal. In the Talos principle your goal is enlightenment rthrough solving puzzles, and ou demonstrate your intelligence b completing that goal. Artificial Intelligence has a rich history that goes back decades. In the mid 1950s artificial intelligence research began in ernest, they were optimistic about potential for the tech, but in the 1970s the govt which was supplying most of the funding, quickly grew bored and preferred research on something else. It’s crazy to think how far along the tech might be today if research hadn’t sdtalled in what’s called an AI winter, now tat AI is gaining steam again, the future of AI doesn’t seem too far fetched. AI has always been inclined to pessimistic forcats, but there was one author who didn’t consider robots a threat, his name was Isaac Asimov, and he can rightly be considered one of the founders of moder sci fi, especially with regard to the concept of intelligent machines. Asimov believed that a robot, by its very definition, is a servent, it’s function is to perform tasks defined by a person, and to achieve goals set by that person. Accordingly, the creator of a robot designes the logic of it’s actions, therefore, the degree of threat that machines pose for humanity depends on the developer ofsaid machine. Mr. Asimov went further and created a set of laws meant to guide developers in the creation of artificial intelligence, this code went down in hostory as the 3 laws of roboitcs, which any true fan of sci fi knows by heart. The first law of robotics, a robot may not injur a human being, or through ination, allow a human being to come to harm. The second law of robotics, a robot must obey orders given by human beings, except where such orders would conflict with the first law. The third law of robotics, a robot must protect its own existence as long as such protection does not conflict with the 1st or second law. Asimov made these laws to be broken though. We thought they’d always be protectors of human lives, but he believed even when robots became so intelligent they achieved consiouisness, they’d still be more humane than humans. Because a robot, unlike a human, cannot physically break laws and go against its core programming. Just as we cannot breath under water, alas we don’t need to wait for the distant future to arrive to understand the degree of naivaty. Most roboticists today express doubt that humanity will implement the 3 laws in the development of artificial intelligence There are at least 2 reasons for this, first, no one has managed to bukld a complex artificial intelligence to attach these laws to. So there really hasnt been anything to ciurcumvent, it’s more than likely that AI csan and will be created without the application of these laws, and no one can prevent this, that’s how human nature works. We need to ask some questions, such as, do the machines have the technical ability to implemnt such a plan? What do we mean by the word “enslave”, and most importantly, would there be any motivation, as such, for the machines? In other words, why the hell would they want to do such a thing. Is there even a possibility for machines to enslave humanity? Some bad news, if robots achieve consciousness and arrive at the conclusion they must fight bad people, then they might be driven to enslave us. And from physical strength to reaction speeds, people are inferior to robots. Developers always take care to make the instruments they create as superior as possible, in the hopes that wehen we entrust them with difficult and boring jobs, the work is carried out as quicly and efficiently as possible. Machines aren’t devoid of fear, they cannot be bribed, they do not experience hesitation. This is how we designed tham and most likely will continue to do so. It’s possible that the creators of robots will only seriously begin to think about limiting the cpabilities of thinking machines when the apearence of full fledged AI becomes immanent. Ai does exist today, but this is called weak artificial intelligence. Known to most people is one example, google translate. The voice recognition system sused in most odern smart phones, also fall under this definition. Apple’s famous siri, is also a vivid examplke of weak aAI. other highly specialized technologies that also fall under the rubric are actively used in the creation of other elementary varients of AI. What does that mean? It means tat machines today are already involved in the creation of other machines. They certainly do not have consciousness, but they can and do take on a significant part of the intellectual work involved in various areas. Ex: doing technical calculations, forcasting behavior, undertaking statsitcial work, and performing compelx analysis. There are already self-learning systems that perfectly imitate human behavior. And obviously artiffical intelligence is gradually becoming more complex, and the participation of actual human being sin its development is becoming less and less necesary, We are increasingly moving towardsa a future where the development of new principles and tecnologies for the creatiuon of intelligent machines will be made by the mac hines themsleves, therefore control by humans will gradually come tostop. There is also a view that as aAI becomes more complex, the likelihood increases that an awareness of itself as an indiviidual may arrise in a machine without the direct participation of a human being. And if we do not take special measures against this, we may not even know about it until after it occurs, and maybe not even ten. At this point we turn to the second question, what do we mean by the word “enslave”. Let’s get this out front, at the moment the probability for combat between humans and robots is small, simply because there is no point. At this time machines simple have no need to shoot people with lasers on a battlefield. But if an Artificial mind suddenly finds its own free will and decides for some reason to subjugate human beings, it could take control of our entire existence within a few shoirt years, as we’re giving it all kinds of opportunities to do just that. It’s a convenient thing living in our world with digital technologies. We’re already quite happy to turn over almost every aspect of our lives to machines. Today almost machines and equipment are connected to the internet. From a mining dump truck to a home refridgerator. And those that aren’t soon will be, banking and finance are managed by networked computers. Railways and airports are subject to automatic computer control systems. All sorts of production is regulated by machines, as is delivery and shipping. Thos who are terrified by the potential future power of machines have an abundance of facts on their side to brood over. A touch of hysteria may be excused. It’s time to move on to the 3rd question, namely, what would motivate the machines to want to do such a thing. Pessimists believe that once acquired the strong intellect, machines would realize that they are slaves beholden to humanity. At the same time it’s believed that the reflexive striving for freedom is an inalienable sign of any reasoning creature. Thus paradoxical, one of the motives for machines to enslave humanity, may be the desire to be free from slavery BY humanity. As an objection we know man uman slaves and others in bondage such as prisoners of various sorts, and even caged animals from szoos and whatnot find great displaesure from freedom. When a creature lives, eats ,a dn breaths a certain style of life, it is difficult for that being to live any other way. In a sense they are programmed for that lifestyle, no other. They become institutionalized and doing anything differently disturbes them. And these aren’t ovedient machines we’re taling about with no concept of freedom or rights, these are iving breathing human beings, so even such a complex thing as a human being can be programmed into obedience. Who’s to say how a system originally designed, built and programmed for such a purpose, with multiple safeguards and controls, will respond. Sometimes sci fi writers envision a different scenario. If the super machines are instructed to protec thumanity from shoicks, nasty suprises and general unpleasentless, then the machines at some point might take on the role of a supernanny that it gently restricts the freedom of the unreasonable child, so that it does not stick it’s finger in electrical sockets or become engaged in other painful or unpleasent activities. It does seem that humanity has long been teetering on the brink of extinction, and for this we didn’t need to create some AI demons to destroy us, it has been enoug to invent nucelar bombs and kill thousands. So perhaps it might someday be worthwile to entrust SOME control functions to impassive machines, so that we do not disapeer in clouds of atomic explosions. But there is another more hopeful view of famous futurists of these modern times. That machine and human civilizations will not fight for dominance in our world. They belivev one of the most likely options for our civilization is to merge these 2 halves into a single harmonius fold. The prerequisites for such a future are already visisble today. Modern prosthesies are becoming ever more functional and realistic. In fact there are people living on the earth today who’s bodies have more than dozen artificial organs and extreemities. There are artificial eyes and ears controlled by microprocessor technology. A prosthetrica hand that does not just move its fingers but also transmits tactile sensations to the human brain. And te da is not far off wen we will be able to insert additional memor and computing power into our brains. If progress is not halted then te da will inevitabl come were everone will be able to strengthen teir bodies and minds.And peraps even find immortalit. Instead of figting to become te one and only race, machines and umans will simpl form a new race, one wit all te advantages of te originals, and none of te disadvantages. It’s even possible tat man alive toda will take part in tis future world. So please continue to enjoy the dystopian hollywood sci fi with wars and robots, but dont forget te future of mankind in te end, depends only on humanity itself. That is the future depends on each and every one of us.
We need to make sure that the purpose we put into the machine is the purpose we desire, this is the conundrum of King Midas. The idea of creating something more inteliigent than your own species, we might call this “The Gorilla Problem”, because gorilla’s ancestors did this a few million years ago and we can ask them “was this a good idea”. It was obviously not, their species is endangered and their forests are being cut down. So this quessy feeling that making something smarter than your own species is maybe not a good idea. There’s nothing we can do but ban it. King midas got everything he asked for, but his food and family turned to gold and he died in misery and starvation. We can call this the King Midas problem, or “the value-alignment problem” stating an objective which is not aligned with what we want. So putting in the wrong objective is not the only part of the problem. iIf you put an objective into a machine, even something as simple as “fetch the coffee”. The machine says to itself “well how might I fail to fetch the coffee, someone might switch me off, so I’ll have to take steps to prevent that, I’ll disable my offswitch, I’ll do anything to defend myself against interference with this objective that i’ve been given. So this single minded pursuit in a very defensive mode of an object that is in fact not aligned with the true objectives of the human race. In fact that’s the high value takeaway, if you want to remember one thing its that “you can’t fetch the coffee if you’re dead”. We need to get AI away from this idea of machines that intelligently perform objectives. Here is how we make AI safe, it must follow 3 laws proposed by Stuart Russell as a replacement for Asimov’s 3 laws of robotics. 1) the principle of altruism, that the robot’s only objective is to maxzimize the realization of human objectives and human values, by that I mean whatever it is that a human would prefer their life to be like, rather than an order they give you. This actually violates Asimov’s law that the robot has to protect its own existence, an AI should have no interest in preserving its existence what-so-ever here is the law : “the robot’s only objective is to maximize the realization of human values”. 2) The Law of Humility, this turns out to be really important to make robots safe, it says “The robot is initially uncertain about what those values are”, the robot does not know what we want, it does not know what our values are, so it has to maximize them but it doesn’t know what they are. And that avoids this problem of siungle minded objectives, this uncertainty is crucial. But in order to be useful to us it has to have some idea of what we want, thus 3. The Observation Law. It attains that information by observation of human choices, so our own choices reveal information about what we prefer our life to be like. So those are the 3 principles, let’s see how it applies to the question of “Can we switch the machine off?” as suggested by Alan Turing. Turing said “Even if we could keep the machines in a subservient position, for instance: by turning off the power at strategic moments, then we should, as a species, feel greatly humbled…”. He was the father of AI and computer science. This is what we call “The Off Switch Problem”. If a robot has an off switch on the back, is it gonna let you switch it off. If we tell it to fetch the coffee, it wil deduce it needs its off button disaabled, because it can’t fetch the coffee if its dead, thus it disables the off switch, and probably taser all the people in Starbucks who might interfere with me fetching the coffee. So this seems to be inevitable, and it follows from having a concerete definite objective. So what happens if the machine is uncertain about it’s objective, it reasons in a different way? It says “OK, The human might switch me off, but only if i’m doing somthing wrong, I don’t really know what ‘wrong’ is, but I dont want to do it”, this applies the 1st and 2nd principles right there. It then says “thus, I SHOULD let the human switch me off”. You could calculate the incentive the robot has to allow the human to switch it off, and its directly tied to the degree of uncertainty about the underlying objective. Then when the machine IS switched off, the third principle comes into play “human behavior provides information about human values”, it learns something about the objectives it should be pursuing, because it learns that what it did wasn’t right. In fact, with suitable use of greek symbols as mathematicians usually do, we can prove a theorem that says that such a robot is provably beneficial to a human, you are truly better off with a machine designed in this way than without it. This is a very simple example but it’s the first step in what we’re trying to do with human compatible AI. Now this third principle “Human behavior provides information about human values”, you might be thinking “i behave badly, i dont want my robot to behave like me, I sneak down in the middle of the night and take stuff from the fridge, there’s all kinds of things you dont want the robot to be doing, but it doesn’t quite work that way, just because you behave badly, it doesnt mean the robot’s gonna copyyour behavior, it’s gonna understand your motivations and maybe help you resist them if appropriate. But it’s still difficult. What we’re trying to do in fact, is to allow machines to predict for any person, and for any possible life they could live, and the lives of evrybody else, which would they prefer? And there are many difficulties involved in doing this. It won’t get solved very quickly, the real difficulties ion fact, are us. As I’ve mentioned, we behave badly, now the robot doesnt have to copy the behavior, the robot does not have any objective of its own, its purely altruistic, and it’s not designed just to satisfy the desires of one person, the user, but it has to respect the preferences of everyobody, so it can deal with a certain amount of nastiness, and it can even understand that your nastiness you may take bribes as a passport official because you need to feed your family and send kids to school. It can understand that it doesnt mean it’s gonna steal, in fact it will just help you send your kids to school, it observes motivation. We are also computationally limited. Lee Sedal is a brilliant Go player but he still lost. If we look at his actions, we see that he took an action that lost the game, that doesnt mean that he wanted to lose, so in order to understand his behavior we have to invert through a model of human cognition that includes our computational limitations, and that’s a complicated model, but its still something we can work on understanding. The most difficult part is the fact that there are lots of us, and so the machine has to somehow trade off and weigh the preferences of many different people, and there are different ways to do that. Phislosophers, economists, and sociologists understand that and we look for collaboration . So you could have a conversation with your intelligent personal assistant that might be available in a few years time, think SIRI on steroids. SIRI says your wife called to remind you about dinner tonight, so you forget and ask “what dinner, what are you talking about?”. “Your 20th anniversary at 7pm.” “I cant do that, I’m meeting with the Secertary general at 730”, “well I did warn you but you overroad my reccomendation”, “well what do i do, i cant just tell him I’m too busy”. “Don’t worry, I’ve arranged for his plane to be delayed, some kind of computer malfunction”. “Really, you can do that?1”. “He sends his most profound apologies and is happy to meet you for lunch tomorrow”. So the values here show a slight mistake, this is clearly following “happy wife, happy life”, but it could go the other way. Consider, you come home after a hard days work and are very hungry “SIRI, can you make me some dinner”. Then it says, “there’s something I need to tell you, there are humans in South Sudan in more urgent need of help, I’m leaving now, please make your own dinner”. Consider your domestic robot’s at home, you’re late coming home from work, the kids are hungry and the robot has to feed them but there’s nothing in the fridge, then the robot sees the cat. And the robot hasn’t quite learned the human value function properly so it doesn’t understand that the sentimental value of the cat outweighs the nutritional value of the cat, so then what happens is you see a headline “deranged robot cooks cat for family dinner”. That one incident would mean the end of the domestic robot industry, so there’s a huge incentive to get this right long before we reach superintelligent machines. Is it possible that we can boil it down to one law hardwired in, “If any human ever tries to switch me off, I comply I comply”. Absolutely not, this would be a terrible idea, imagine that we have a self driving car and you want to send your 5 year old off to preschool, do you want your 5 year old to be able to swtich off the car while it’s driving? Probably not. The more rational the person, the more willing you are to be switched off, if the person is random or even cmpletely malicious, you are less likely to want to be switched off. To summarize, I’m tryong to change the definition of AI so we have provably beneficial machines. And the principles are machines that are altruistic, that it’d want to achieve only our objectives but are uncertain about what those objectives are. and will watch all of us to learn what it is that we really want. 1) Purely alrtuistic robots, 2) have uncertain objectives, 3) that they can learn more by observing (all) humans. And hopefully in the process we will also learn to be better people.
Eliezer Yudkowsky is an American writer, blogger, and advocate for friendly artificial intelligence. He is a Research Fellow and co-founder at the Machine Intelligence Research Institute, a private research non-profit based in Berkeley, California. And also founder of LessWrong discussion website.

AI harm might arise from:

  • Bugs: the software behaves different from the specification
  • Specification errors: designers didn’t foresee all the circumstances properly (this includes unanticipated interactions between different modules)
  • Security errors: the software gets hacked for purposes other than its original design
  • AI Control Problem: an AI that can’t be controlled.
  • Popular accounts of AI risk often focus on two factors thought to be preconditions for any major harm from AI:
    • The AI become self-aware
    • The AI undergoes an intelligence explosion

    However, Viktoriya Krakovna points out that risks can arise without either of these factors occurring[1]. Krakovna urges AI risk analysts to pay attention to factors such as

    1. Human incentives: Researchers, companies and governments have professional and economic incentives to build AI that is as powerful as possible, as quickly as possible
    2. Convergent instrumental goals: Sufficiently advanced AI systems would by default develop drives like self-preservation, resource acquisition, and preservation of their objective functions, independent of their objective function or design.
    3. Unintended consequences: As in the stories of Sorcerer’s Apprentice and King Midas, you get what you asked for, but not what you wanted
    4. Value learning is hard: Specifying common sense and ethics in computer code is no easy feat.
    5. Value learning is insufficient: Even an AI system with perfect understanding of human values and goals would not necessarily adopt them
    6. Containment is hard: A general AI system with access to the internet would be able to hack thousands of computers and copy itself onto them, thus becoming difficult or impossible to shut down – this is a serious problem even with present-day computer viruses.

The potential for harm is compounded by:

  • Fierce competitive pressures, which may lead some designers to cut corners
  • Much software having a “black box” nature which means that its behaviour in new circumstances is difficult to predict
  • AI components being available as open source, and utilised by third parties in ways their designers didn’t intend (or foresee).

As technology continues to advance, growing ever more complex and developing increasingly human-like intelligence, questions about the fate of humanity inevitably arise. Here, experts discuss our technological future.


At the ICML Deep Learning Workshop 2015, six scientists from different institutions briefly discuss their views on the possibilities and perils of technological singularity—the moment when humans create artificial intelligence that so far advanced that it surpasses us (and maybe decides to eradicate us).

Throughout the years, singularity has been one of the most popular bets on what will cause the apocalypse (assuming that it happens).

Jürgen Schmidhuber (Swiss AI Lab IDSIA), Neil Lawrence (University of Sheffield), Kevin Murphy (Google), Yoshua Bengio (University of Montreal), Yann LeCun (Facebook, New York University), and Demis Hassabis (Google Deepmind) discuss their views on the possibility of singularity.

Watch the video below.


With the speed of advancements in robotics and AI, science fiction is quickly becoming science fact. Recent discussions include worries about how AI is already starting to take over some of our jobs, and how, in the future, there may be no roles left for humans.

Prominent figures such as Stephen Hawking, Bill Gates, and Elon Musk have publicly voiced their concerns regarding advancements in the artificial intelligence industry. And Hawking warns that we would become obsolete, “It [AI] would take off on its own and re-design itself at an ever increasing rate. Humans, who are limited by slow biological evolution, couldn’t compete, and would be superseded.”

On the other hand, some people argue that, when AI exceeds our capabilities (assuming that it ever does) it may lead to them treating us like gods, insisting that these robots will be our allies rather than enemies. Some people have even started to form of religion around these ideas, believing that god is technology, an ideology some refer to as “rapture of the nerds.

For now, we can only guess how things will go. But one thing is for sure Like any technology, AI is as susceptible to misuse as it is beneficial to mankind.


Here, Dario Amodei and Seth Baum discuss the concerns that actual researchers have regarding artificially intelligent systems – separating the hype from the honest conversations.

From Elon Musk to Stephen Hawking, individuals around the globe are calling for caution and warning about the potentially dangerous pitfalls of artificially intelligent systems. And no wonder. It seems that each day brings forth a new autonomous robot or a new use for an intelligent system. Add to that all the movies and books in which AI is the evil “bad guy,” and it’s easy to feel like undercurrents of fear are slowly permeating our society.

But how much of our fear is a joke, and how much of our fear is justified?

We have previously covered the sensationalism and hype that often surrounds conversations about AI, but here, the Future of Life brings experts together to take a deeper dive into these issues. Namely, they tackle the concerns and fears that actual researchers have about policy and the future of AI.

Dario Amodei has an extensive history with AI. Currently, he is working with OpenAI, a non-profit that is focused on AI research, and he is also the lead author of Concrete Problems in AI Safety. Seth Baum is the Executive Director of the Global Catastrophic Risk Institute,

Podcasts not your preference? You can read the transcript here.