4. [Jeph's comic comments]: "I predict that our robot overlords will want nothing whatsoever to do with us. If we ever invent it, the first thing an AI will do is bootstrap itself to omnipotence and then light out for the Kuiper Belt." I am not certain why, but this reminds me of a CSI: Las Vegas episode ("Shooting Stars") with an alien-worshipping suicide-cult, the lone survivor (& murderer) is talking with Gil Grissom in the denouement of the episode.
- Abigail Spencer (cultist/murderer): I know they're all up there... happy... healthy... perfect. [sees Grissom's facial expression] You don't believe we're alone in the universe?
- Grissom: Abigail... I'm sure if there is something out there looking down on us from somewhere else in the universe, they're wise enough to stay away from us. [fade to black]
I've always found the Grissom position there a bit silly. Oh, yes, "what a fearful beast is Man" (is that a trope? it probably has a different name. Don't link it. No. Stop that), sobering look at ourselves as a species that's committed hideous atrocities, blah blah.
But what one forgets is that we are, to date, the only species to possess or at least enumerate a "code of ethics", and that many of the most heinous behaviors we engage in (e.g. infanticide) are evolutionarily advantageous!
The horror will be, I think, when we encounter other sapients, that they won't be aloof, superior, moral beings, even if they pretend to be. They'll be as flawed and morally degraded as humankind.
That, too, will be the ultimate danger of A.I.: not that they'll be born thinking they're superior, but that we'll teach them to be just like us.