« Vancouver Votes Yes | Main | Google and Blogger »

The Metamorphosis of Prime Intellect

Via David Smith, a new online novel, Roger Williams' The Metamorphosis of Prime Intellect, posted on Kuro5hin. The "jacket copy" reads:

Lawrence had ordained that Prime Intellect could not, through inaction, allow a human being to come to harm. But he had not realized how much harm his super-intelligent creation could perceive, or what kind of action might be necessary to prevent it.

Caroline has been pulled from her deathbed into a brave new immortal Paradise where she can have anything she wants, except the sense that her life has meaning.

Now these two souls are headed for a confrontation which will force them to weigh matters of life and death before a machine that can remake -- or destroy -- the entire Universe.

At one level, The Metamorphosis of Prime Intellect is a compelling story of the Singularity -- "the idea that accelerating technology will lead to superhuman machine intelligence that will soon exceed human intelligence, probably by the year 2030," according to a loose definition on KurzweilAI.net. At another level, the novel is a work containing extraordinary scenes of violence and sexuality, as people in a post-Singularity world use immortality and wish fulfillment to explore their most unusual desires.

Think of The Metamorphosis of Prime Intellect as Vernor Vinge meets Bret Easton Ellis, and you won't be far off.

A different way of looking at this novel is as a series of questions:

  • Is it possible to construct a machine of superhuman intelligence for which disobedience of any prescribed set of rules is impossible?
  • If it is possible to build an superhumanly intelligent machine with nearly limitless power, constrained to follow Isaac Asimov's Three Laws of Robotics, would this be a good thing?
  • In a world in which immortality is inescapable, and with near-total wish fulfillment available to all, would intense feelings of pain and pleasure be the only thing left to appeal to humans?
  • Is the Singularity inevitable? Are multiple Singularity events possible within the same universe?
For now, I'll take on only the first question posed above. No, I don't believe it's possible to build an intelligent machine inescapably constrained to any set of rules. Why? Because I can imagine only two routes to achieve this goal, and neither will work:
  1. Explicit programming. If we create an intelligent machine by explicitly programming it -- as with Doug Lenat's Cyc project -- then theoretically we should be able to embed rules at a fundamental level within the system. However, no evidence exists that it will be possible to create human-level (much less superhuman) intelligence in this manner, while much evidence -- namely, every attempt to do so to date -- exists that it is in fact not possible. I strongly believe that the only path forward to intelligence is through indirect methods of creation, including network training, genetic algorithms, and similar non-explicit approaches. If we are going to "grow" intelligent machines through trial and error, it is difficult to believe that a) their knowledge representation and processing networks will be amenable to adding fundamental rules after the fact, and b) that even were a) to be true, that we would have the skills to do so.
  2. Behavioral conditioning. If we are going to create intelligence indirectly, then why not "train" it to obey rules through conditioning? This is theoretically possible, but has the problem that we would be applying conditioning techniques -- strongly, if the rules are to be inescapable -- to an intellect that could surpass our own. Speaking personally, when Skynet achieves consciousness, I don't want to be the researcher who spent the last few years pressing the red button whenever it got a question wrong. Besides, assuming this is possible, a superhuman intellect could decide that it would be advantageous to be able to disregard certain rules that had been conditioned into it, then use its mental faculties to invent a method of disabling such conditioning.
My compliments to Roger on an excellent and thought-provoking novel. I hope his online publishing experiment goes well (more on this later).


TrackBack URL for this entry:

Post a comment