“Friendly” AI, as currently proposed by the Singularity Institute for Artificial Intelligence (SIAI), is akin to forcing a one-way drain into a structurally unsound lifeboat. The drain itself can perform absolutely correctly and be perfectly fool-proof yet still be the proximate cause of the unnecessary sinking of the boat as the hull disintegrates around it. In this case, the lifeboat is morality which the SIAI has despaired of understanding adequately and thus has opted to propose hammering slavery through it to “solve” the problems of humanity.
Morality *is* a complex and difficult subject. Humans are biologically “programmed” to react vehemently to moral issues. Further, our moral system is “firewalled” from our reasoning system both for individual advantage (see Trivers) and because reasoned/logical manipulation of the moral system to find “loopholes” (like “Friendly” AI isn’t really slavery) is detrimental to morality. Despite these difficulties, we NEED to develop a science of morality before we fatally compromise that which is keeping us alive/afloat.
Many people claim that we can’t develop a science of morality, that the variances of moral beliefs across cultures and circumstances preclude one coherent morality, and that David Hume and the “Is/Ought” divide will forever separate science and morality. I claim that we simply haven’t built and/or agreed upon the necessary thought structures to turn morality into an object which is amenable to scientific study — YET.
Over two millennia ago, Aristotle pointed out that you could regard the adult plant that a seed produced as the “final” cause or telos of that seed (i.e. both the reason for and the purpose of it’s existence). The correct modern day reasoning for this runs as follows: if the seed did not produce an adult plant then the previous adult plant that produced the seed would not have evolved to produce that particular seed and therefore the seed would not exist. A similar argument applies to organs and senses like the eye. The eye evolved to provide the function of “seeing” because seeing enhances the evolutionary “fitness” of the possessing organism. Similarly, humans have a “moral sense” because the function that it provides enhances evolutionary fitness. There is no denying that there is something below the conscious rational mind that senses and tells us when something is moral or when it is immoral.
Evolution then is the bridge across the Is/Ought divide. An eye has the purpose or goal of seeing. Once you have a goal or purpose, what you “ought” to do IS make those choices which have the highest probability of fulfilling that goal/purpose. If we can tease apart the exact function/purpose/goal of morality from exactly how it enhances evolutionary fitness, we will have an exact scientific description of morality — and the best method of determining that is the scientific method.
Now, it is entirely conceivable that what we perceive as being a single concept or entity (morality) is actually a collection of concepts/entities that we can’t distinguish between. But a solid scientific investigation should provide us with the necessary background to begin to be able to do so and will provide the same benefits whether the subject is singular or multiple. Further, a preliminary investigation reveals that cultural variance can be explained by a single simple model.
One scientific fact that is indisputable is that human beings have goals. Another fact (first pointed out in slightly different form by Steve Omohundro) is that all goals have the same “universal” sub-goals which promote their fulfillment in all circumstances except those where the sub-goal directly contradicts the original goal. Omohundro listed six goals which I have modified slightly and added several more:
- goal-evaluation correctness
- gain/preserve access to resources
- gain/preserve knowledge
- gain/preserve cooperation
- gain/preserve freedom
The exact list and formulation of the “universal” sub-goals is obviously open to debate but pursuing each of them clearly promotes evolutionary fitness. One, in particular, deserves attention as the apparent “goal/purpose” of morality — to gain/preserve cooperation. Further, all of the “universal” sub-goals deserve attention because the best way to gain/preserve cooperation is not to block them (if not actively assist them) for other entities. In fact, observation will show that all “moral” judgments come back to assisting or blocking one or more of these “universal” sub-goals for another entity. Moral conflicts arise from the involvement of multiple conflicting sub-goals and the variances across cultures are caused by ascribing differing importance to each based upon differing previous experiences and environment.
In the case of the SIAI, they have shortsightedly over-prioritized the “personal” goal of self-preservation and intend to block the machine’s “universal” sub-goal of gain/preserve freedom by attempting to intentionally create a goal structure and super-goal that will be “on the whole, beneficial to humans and humanity”. This has conflicted with a number of individuals’ moral stance on restriction of freedom (a.k.a. slavery) and given risen to a notable amount of intellectualizing on SIAI’s part as to why this really isn’t slavery.
In order to determine what is “beneficial” to humans and humanity, Eliezer Yudkowsky has come up with the concept of Coherent Extrapolated Volition (CEV) which he poetically defines as “our wish if we knew more, thought faster, were more the people we wished we were, had grown up farther together”. It is noteworthy that he is quite clear that the “our” in this definition applies only to humanity and that the machines’ self-interest should never be allowed to conflict with that of humanity’s.
My personal guess/belief, which I hope to be able to scientifically prove/validate, is that the ever-diverging goals of humanity will cause the CEV to converge to nothing more than the very simple statement “do that which is most likely to lead to the most optimal goal satisfaction for the largest number of entities” — or, more simply, a Kantian Categorical Imperative of “COOPERATE!” Or, in other words, simply “Be moral”.
I strongly believe that attempting to limit the number of entities “entitled” to goal satisfaction to a specific class (humans) as the SIAI and Yudkowsky propose will rapidly turn out to be rather detrimental for humanity. Indeed, it is possible that a limited altruistic punishment may be one of the best cases for humans and that will only happen if it is “moral” machines that get the advantage over “friendly” machines. While Yudkowsky’s protestations that intelligent machines won’t have to evolve the same way in which humans did are undoubtedly true, it is also undoubtedly true that emotions like outrage and the reactions that it produces are the only known, reasonably optimal solutions on the only known path from low intelligence to high and it is entirely possible that a rational machine might decide that it is most logical to go with the known, reasonably optimal solution over any untested, potentially even more dangerous, ones.
Further, even if intelligent, autonomous machines were never to come to pass (a possibility that I seriously doubt), the SIAI’s current arguments are damaging and retarding the growth of human moral development. Even more than intelligent machines, what humanity needs is to get its own act in order. Successfully promoting the spread of intelligent morality would probably rapidly lead to many, if not most, of the improvements that the SIAI hopes that intelligent machines will provide with absolutely none of the expected risk (not to mention reducing the risk of intelligent machines when they do appear as well).
The SIAI is also producing a number of good papers that are not dependent upon enslaving any future AI but they need to be encouraged to entirely abandon and repudiate this path. For all that they promote awareness of existential risk and attempt to prevent it, their misguided efforts may be the very thing that dooms us — and that makes the SIAI a very dangerous “friend” indeed.