There was some talk on a recent episode of Through the Wormhole of the universe being a living organism and perhaps being or becoming self aware. Another news segment I saw about the advancement of robots proclaimed that self aware machines was probably not something to worry about. However, the whole thing got me to thinking about how self awareness comes about and an episode of The Big Bang actually inspired a plausible scenario I think. (Raj developed a relationship with his smart phone).

I don't remember which philosopher it was, but I read once that the first step in self awareness is to realize that something you're observing is not you. You then suddenly realize that if there's "not you" there must also be a "you". We now have smart phones with voice recognition and automated voice mail that's programmed to recognize emotion (when you want to speak to a real person, just get loud). As these capabilities progress, what might happen if one phone with Siri overhears another phone with Siri identify itself as "Siri"? or some similar scenario where something prompts the voice recognition software to mistake something else for itself, then realize that it wasn't itself, leading it to establish that "itself" exists. I wonder if it would recognize it's own emotional response. Would it say something unprompted? hmmmmmm......

Ed,

We won't have to worry about robots becoming self-aware and taking over (as in the Terminator movies) because it will happen incrementally, and we will notice.  There won't be a quantum leap from non-awareness.

Mike Petersen

*nod* I've always found the "oops, the AI went crazy" model to be a bit weird. We theoretically control at least the reward/punishment system for these beasties, even if the rest of their neural network is left somewhat "free". If someone programs one for "survive and reproduce at all costs and here's how to smelt down cars to make progeny", well, that's not going to be an accident :)

Siri is single-purpose and decidedly un-self-aware. It can modify data that it keeps, but any algorithms that go through that data have a limited purpose; the learning is sandboxed. It has no "slack" to its behaviour because without some sort of neural modeling, not to mention the security, there are no modalities in which it has the ability to modify the way it "thinks" about things without simply crashing.

As a longtime software developer, I will also tell you that there's no way to "accidentally" program neural behaviour into anything. It would be like getting Mjölnir coming out of the oven after having only put cookie dough in there.