People need to understand why some problems will defy any amount of computing power thrown at them, and why using software to analyze software has unfixable limitations.
Mmm... But that is just as true of human computing power and human software as that running in a sentient artificial intelligence. Or to put it another way, such problems would be no
less amenable to solution by an AI if it really were intelligent at a human level at least.
a) Do we know that AI are self-sustaining?<snip>
b) Maybe AI don't yet know how to create themselves at all.<snip>
c) Not all branches of computer science would lead directly to AI development.<snip>
On a) and b), I think
we can be reasonably sure that AI are capable of creating and sustaining themselves, or Hannelore would not have regarded the fact that "they like us" as so fortunate.
As for c), I'm sure there would be other applications of Computer Science than AI development, but why would we assume that humans would be as good at
any aspect of the field as AIs? Because humans are special? With the advent of truly sentient, self replicating artificial intelligences, maybe not so much. Some areas of CS, like compiler-design, might become completely redundant. Why would an AI need a compiler at all? Once upon a time even humans wrote machine code directly, and it doesn't take much digging to find
grey-muzzled old programmers who will assure you that they wrote more efficient code that way too!
There is a poem (or maybe it's a song?) called
"John Henry The Steel-Driving Man" where a man competes with hand tools against a steam-hammer in drilling through a mountain and kills himself in the attempt. I suppose once John Henry was supposed to seem heroic, but nowadays the idea that a man with a hammer or a spade would even try to compete in such a task with powered machinery seems rather ridiculous and pathetic. It might be that the advent of AI minds might make competing with them at computer-science similarly futile.
Or maybe not. Even in our world, where computers are well short of sentience, the arrival of computers that can beat the strongest human chess players some of the time, has not stopped people playing chess. Even if computers were developed that could defeat
all human players in
every game, people would probably still play against each other, and by studying the games of the computer champions improve their own play. There are, after all, human
weiqi players I would have no chance of beating, and yet I still play and enjoy the game, and study the games of the professional champions. If (or rather when) computer players can reliably defeat those human champions, I doubt I will give up the game. Humans compete in marathon foot-races despite being wholly uncompetitive over that distance with motor cars. Perhaps we don't have to be the best at something to find it a worthwhile subject of study.