ShrinkWrapped

The Open Mind VII – the One and the Many

Jay Adler strikes first in this installment of The Open Mind.  As usual, comments are closed here and should be left at Jay's site.

The Open Mind VII – the One and the Many

Think of this as a consideration, inviting more consideration, based less on a claim than on a question:

what is the proper relationship between the one and the many?

“Proper” here might refer to ethics, logic, practicability. I have said more than once, perhaps at some point on this blog, that had l lived during the nation’s early years, I would have been a Federalist. That shouldn’t strike as surprising given my leanings. But here I correct myself. I think I have been wrong to say so. I think in saying so I have imposed my contemporary political self on another time and world, whereas other elements of myself would have felt much differently. (If all we are is our political selves, we are poor creatures.) One should not be – experience oneself – a Yankee in King Arthur’s court, but English; otherwise, why travel? Given what I know of my other selves, in the world of the founding, I now think it more accurate to think I would have been a Jeffersonian. Of course, many of us are some of each. I am. But why, projected back in time, the change?

Number 21 in my Principia Liberalis stated,

Technology increases affective connections, which are loosened by the distance that technology narrows. The greater the affective connection, the greater the sense of mutual moral responsibility. Notions of discrete and separable, autonomous individuality, neither responsible to nor the responsibility of others, are irreversibly challenged by population density and technology, and the increased effect of human actions on other humans. It is necessary to define what core autonomy need be protected, as an essential human good, but earlier stages of political relation, of individuals to each other, and of individuals to the commonweal, will not be recovered.

Empathy and AI: Part V

In Empathy and AI: Part I I discussed the possibility of coding for empathy in our imagined AI offspring.

In Empathy and AI: Part II I wondered about how to perform the mirroring function for an AI.  Most AI researchers believe that the AI's mind will evolve rather than spring into being fully formed.  As such, encoding for empathy becomes a significant issue.

In Empathy and AI: Part III I briefly described some of the factors involved in the development of a coherent sense of self and how such development depends upon an empathic connection to another's mind.

In Empathy and AI: Part IV I speculated on how the development of a self in an AI could go awry analogous to disorders of the self in humans.

In this final post in the series I want to discuss the higher order functions that a healthy mind must contain and my concern the there may be a relative neglect of such higher order functions in the field of AI. 

Fortuitously, J Stoors Hall, who blogs on AI at the Foresight Institute, wrote today about the father of AI, Marvin Minsky, whose ideas have been somewhat neglected in the field but are germane to the discussion.  Minsky was speculating about the organization of a mature AI mind:

The first AI blog

The first AI blog was written by a major, highly respected figure in the field. It consisted, as a blog should, of a series of short essays on various subjects relating to the central topic. It appeared in the mid-80s, just as the ARPAnet was transforming over into the internet.

The only little thing I forgot to mention was that it didn’t actually appear in blog form, which of course hadn’t been invented. The WWW didn’t appear until the next decade. It appeared in book form, albeit a somewhat unusual one since it was, as mentioned, a series of short essays, one to a page. It was, of course, Marvin Minsky’s Society of Mind.

Of course, you’re reading a blog about AI right now. The difference is that that was Minsky, and this is merely me. If you haven’t read SOM, put down your computer and go read it now.

Good. You’re back.  Here’s why SoM is relevant to our subject of whether and how soon AI is possible:

It remains a curious fact that the AI community has, for the most part, not pursued Society of Mind-like theories. It is likely that that Minsky’s framework was simply ahead of its time, in the sense that in the 1980s and 1990s, there were few AI researchers who could comfortably conceive of the full scope of issues Minsky discussed—including learning, reasoning, language, perception, action, representation, and so forth. Instead the field has shattered into dozens of subfields populated by researchers with very different goals and who speak very different technical languages. But as the field matures, the population of AI researchers with broad perspectives will surely increase, and we hope that they will choose to revisit the Society of Mind theory with a fresh eye.  (Push Singh — further quotes from the same source)

In other words, here’s a comprehensive theory of what an AI architecture ought to look like that is the summary of the lifework of one of the founders and leaders of the field, and yet no one has seriously tried to implement it.  (When I say serious, I mean put as much effort into it as has gone into, say, Grand Theft Auto.)  (There has been a serious effort to implement the theoretical approach of the CMU wing of classical AI, namely SOAR.)

Part of the reason for this is that SoM is in some sense only half a theory:

Minsky sees the mind as a vast diversity of cognitive processes each specialized to perform some type of function, such as expecting, predicting, repairing, remembering, revising, debugging, acting, comparing, generalizing, exemplifying, analogizing, simplifying, and many other such ‘ways of thinking’. There is nothing especially common or uniform about these functions; each agent can be based on a different type of process with its own distinct kinds of purposes, languages for describing things, ways of representing knowledge, methods for producing inferences, and so forth.
To get a handle on this diversity, Minsky adopts a language that is rather neutral about the internal composition of cognitive processes. He introduces the term ‘agent’ to describe any component of a cognitive process that is simple enough to understand, and the term ‘agency’ to describe societies of such agents that together performs functions more complex than any single agent could.

J. Storrs Hall focuses on how the field has focused at most on discrete elements (agents) of Minsky's fractionated mind model and points out that Minsky avoided discussing the actual internal composition of such agents.  Neuroscience has increasingly supported the notion that our minds, despite our subjective experience of its unitary character, are composed of many unique modules, analogous to agents, that operate somewhat independently.  This fits nicely with the psychoanalytic theory of mind, ie that our minds contain multiple desires, wishes, prohibitions, inhibitions which conflict with each other in myriad ways.  In Psychoanalysis we explicitly focus on increasing the capacity of our patients to first identify and then integrate and synthesize the various conflicting strands within their minds.  In other words the various competing outputs of the various modules of the mind must be summed, integrated with each other, in order to produce an outcome acceptable to the executive apparatus, ego/self/mind.  The most effective minds have the highest capacity for synthesis, as well, taking old conflicts or contents and reworking them to produce novel, more adaptive, outcomes.

The integrative and synthetic functions of the mind are among the most sophisticated, probably the most recent evolutionary advances, and the most sensitive to disruption.  I suspect our computer scientists will have great success in developing the various agents that Minsky mentions.  Their energies will be devoted to making those agents as powerful as possible.  The greatest and last test for our ability to determine the friendliness of our AIs will lie in how such agents are integrated and whether or not they are able to contain or develop a synthesizing functionality,  balancing the various agents in such a way as to enable the kind of conscience that would foster friendliness.  J Storrs Hall implies that the current work on AI does not include particular attention to integrating and synthesizing the various discrete  agents being worked on; this does not seem promising as it neglects what may well be the most important determinants of the future friendliness of our offspring.

Empathy and AI: Part III

In Empathy and AI: Part I I discussed the possibility of coding for empathy in our imagined AI offspring.

In Empathy and AI: Part II I wondered about how to perform the mirroring function for an AI.  Most AI researchers believe that the AI's mind will evolve rather than spring into being fully formed.  As such, encoding for empathy becomes a significant issue.

The early Psychoanalysts developed a Theory of Mind that continues to resonate and have significance, including having gained some support from modern neurosciences, as noted in A Conscious Digression:

Mahler's contributions included her elegant descriptions of the child's sense of self emerging from the undifferentiated mother-infant matrix.  She called the first glimmers of separateness "hatching"; this was followed by the separation-individuation phase later in early childhood (12-18 months).  Along with the establishment of separation, the child developed stable self- and object-representations.

In Science Friday: Differentiating the Self, I described how neuroscience was finally giving us some of the tools and the data we need to understand these concepts and connect them to their physiological substrate.  I used the new research to support the Psychoanalytic understanding of the development of our stable sense of self:

Typically development is a rather complicated journey... but the essential idea that multiple self-representations (ie, different images one has and develops about oneself) must be integrated to form a coherent and consistent sense of oneself has been bolstered by the finding of areas of the brain that appear to do just such integration.

The recent discovery of "mirror neurons" is a Neuropsychological earthquake.  In MIRROR NEURONS and imitation learning as the driving force behind "the great leap forward" in human evolution, V.S. Ramachandran described the existence of such neurons as a paradigm shift:

How does all this lead to self awareness? I suggest that self awareness is simply using mirror neurons for "looking at myself as if someone else is look at me" (the word "me" encompassing some of my brain processes, as well). The mirror neuron mechanism — the same algorithm — that originally evolved to help you adopt another's point of view was turned inward to look at your own self. This, in essence, is the basis of things like "introspection". It may not be coincidental that we use phrases like "self conscious" when you really mean that you are conscious of others being conscious of you. Or say "I am reflecting" when you mean you are aware of yourself thinking. In other words the ability to turn inward to introspect or reflect may be a sort of metaphorical extension of the mirror neurons ability to read others minds. It is often tacitly assumed that the uniquely human ability to construct a "theory of other minds" or "TOM" (seeing the world from the others point of view; "mind reading", figuring out what someone is up to, etc.) must come after an already pre- existing sense of self. I am arguing that the exact opposite is true; the TOM evolved first in response to social needs and then later, as an unexpected bonus, came the ability to introspect on your own thoughts and intentions. I claim no great originality for these ideas; they are part of the current zeitgeist. Any novelty derives from the manner in which I shall marshall the evidence from physiology and from our own work in neurology. Note that I am not arguing that mirror neurons are sufficient for the emergence of self; only that they must have played a pivotal role. (Otherwise monkeys would have self awareness and they don't). They may have to reach a certain critical level of sophistication that allowed them to build on earlier functions (TOM) and become linked to certain other brain circuits, especially the Wernickes ("language comprehension") area and parts of the frontal lobes.  [Emphasis mine-SW]

Mirror Neurons allow us to create a model of the mind of the other.   Of crucial importance, Mirror Neurons do not "know" if their input is from external sources or internal sources; this must be learned by the neural circuits over time spent integrating input from all the various sources that impinge upon the human nervous system.

The Neuroscientists and Psychoanalysts agree that a mind can only develop in relation to other minds; empathy is a pre-requisite.  This is likely going to be as true for AIs as for biological minds. 

Back to Mahler, and her ideas on Separation-Individuation, the process whereby an infant's mind emerges from an undifferentiated fused state and learns that it is a separate and distinct entity from the Mother's mind;  from Intellectualization, Free Speech, & Unintended Consequences: Part I:

As always for a Psychoanalyst, one starts in the earliest experiences of the person.  Margaret Mahler wrote extensively about early childhood development.  In her theory of development, she described a normal undifferentiated (autistic, then symbiotic) phase followed eventually by separation and individuation.  A brief recap can  be found here.  Mahler described how the 10 month old infant (at the beginning of the differentiation phase), just beginning to stand upright and making his first tentative steps, develops a "love affair with the world".  An infant who has the safety and security of his mother's watchful, loving eye, ventures forth into the world with a thrill of discovery.  Everything is new and exciting.  Everything has to be touched and held and tasted.  The child is encouraged by the delight he sees in his mother's eye when he achieves a new level of development.  The first baby steps are a moment of delight for the parents and the child.  The child is thrilled and excited both by his parents' reaction and by his growing mastery over himself.  As a child grows and develops, he gains greater and greater mastery over himself (his musculature, as in ambulation and fine and gross motor activity) and his environment.  He learns to master his bodily functions (urine and bowel control) as well; the differentiating child takes great pleasure in his ability to control himself.

Along with the gains, the movement forward in development, there are accompanying losses.  Mastery and control also imply the loss of some freedoms.  When a child reaches the age of 18-24 months, some remarkable things take place.  Up until this age the child has existed in a blissful world of omnipotence.  If they are hungry, they cry, and food appears; if they need to be held and emotionally nurtured, they cry, and the mother appears; if they feel like voiding, they void; if they wish to sleep, they sleep.  They essentially can do whatever they want (whatever they are capable of) with no limitations.  While many of us continue to have wishes to do whatever we want whenever we want, this blissful state can not, and does not, last.  By this point, the child has learned to say, "No".  He is separate from his mother and begins to recognize that her desires are not always congruent with his; his fantasy of omnipotence is shattered.  Some children, perhaps because of constitutional reasons, or perhaps because of an insecure attachment to an unavailable mother (depressed, narcissistic), have a very limited ability to tolerate separation and frustration.  Some of these children develop the kinds of narcissistic vulnerabilities I have talked about in prior posts (Narcissism, Malignant Narcissism, and Paranoia, parts I, II, III, IV). 

Melanie Klein, a highly influential analyst and one of the early students of Object Relations theory, offered a distinctive elaboration on the process by delineating a Paranoid-schizoid position, prior to the establishment of Object Constancy (where the majority of the self and object representations have been unified in a fused sense of distinct self and object) in which the infant experiences itself and its mother as different part objects at different times and circumstances.  The infant, upon achieving Object Constancy, then moves into the Depressive position, characterized by a knowledge of loss of the (fantasied) omnipotent union with the mother. 

[Utopian fantasies and wishes arise from the desire to recapture the fantasied blissful union with the early all powerful, all gratifying Mother.]

The process of Separation-Individuation can fail in a number of ways.  When the boundaries between self and other are poorly established or too porous, or empathy is minimal or non-existent, or the sense of self is defective, serous psychopathology ensues.

Poor ego boundaries leads to a psychotic state where the person loses the ability to determine where stimuli arise; they may attribute internal stimuli to external objects, which is the fundamental error in psychosis.  Because of the poor appreciation of their own limits, grandiosity often is part of their pathology.

When empathy is minimal or non-existent, depending on the neural substrate, one can develop autism or Sociopathy.  The severe autistic does not recognize that the Other has a mind and desires of its own; the Sociopath doesn't care.

When the sense of self is defective, the narcissistic pathologies are the result.

How this might inform our understanding of the development of an AI's mind is the subject for Part IV of this series.

Empathy and AI: Part II

In Empathy and AI: Part I I discussed the possibility of coding for empathy in our imagined AI offspring.  The discussion among the commenters was impressive, as so often occurs here, and toughed upon many themes worth exploring in more detail.  Two themes stood out in the comments.  First the probability that even were we able to write code determining the acquisition of consciousness, we could never delineate a wide enough set of "human like" experiences for our AI.  We might be able to simulate Jimmy J's transcendental experience but we could never do it in a way in which the AI would believe it emerged from within his own (organic) conscious mind and he would always be aware that it did not enter him from outside of his programming.  Second, the idea of the necessity for mirroring to be part of the process which I will address shortly.

There are compelling reasons why we would desire that any AI would contain code for empathy.  It would clearly be in our best interests to assure that any AI would have empathy for its creators. Without a modicum of empathy we would be in the position of an ant in relation to an elephant. Where our interests conflicted with the AI, our desires would simply not gain the threshold of its "awareness." Were we able to code for empathy it would be a very good thing. However, as noted, our empathy rests upon our awareness, both consciously and unconsciously, of our shared biology. A machine intelligence could be programmed to simulate such an awareness and could be simulated to have its own "biology", but it would be a simulation and simulations have importance to our psychology.

There is a fascinating discussion going on in some precincts of the blogosphere and within Cosmology, etc, concerning the possibility that we might all be simulations in a universe computer. It is an interesting discussion, but irrelevant to this topic, for, even if we are merely simulations in an uber-computer that runs the universe, we do not know (or know, which includes emotional as well as intellectual knowledge) that we are simulations; we think we are real and that is all the difference. A computer that believes it is real when it can also know it is simulating a real body is a computer that has a lie at its core. Another way to understand the lie would be as a fixed, false belief or a delusion. It may, theoretically be possible, at the end of a very long process, for a computer intelligence to understand all there is to know about human biology and therefore to appreciate our drives, desires, passions, etc, but this is highly speculative. In addition, intellectual understanding is never enough for empathy to exist; there must be an emotional understanding for empathy. To the promise that we could program in emotions, these would be machine analogues of our emotions. It may well be that we can quantify and calculate curiosity, for example, but since our minds are incapable of separating ourselves from our affects and an AI would necessarily be the inverse (it could only simulate affects as an 'add on" rather than directly experience them) empathy would again fail. It is very possible, perhaps likely, that prior to a final understanding of such an apparent contradiction, our software would Begin to exhibit behavioral disturbances analogous to the kinds of behavioral disturbances found in humans when such internal contradictions, ie conflicts, become impossible to resolve.

Although we may be approaching a decent working knowledge of the constituents of conscious awareness, including self-awareness, understanding how knowledge translates into consciousness and meaning remains problematic and it is in that juncture that Empathy between our AI offspring and our meager intelligence will arise or fail.

Ray Tallis, writing in New Scientist suggests that You won't find consciousness in the brain:

Thus measurement takes us further from experience and the phenomena of subjective consciousness to a realm where things are described in abstract but quantitative terms. To do its work, physical science has to discard "secondary qualities", such as colour, warmth or cold, taste - in short, the basic contents of consciousness. For the physicist then, light is not in itself bright or colourful, it is a mixture of vibrations in an electromagnetic field of different frequencies. The material world, far from being the noisy, colourful, smelly place we live in, is colourless, silent, full of odourless molecules, atoms, particles, whose nature and behaviour is best described mathematically. In short, physical science is about the marginalisation, or even the disappearance, of phenomenal appearance/qualia, the redness of red wine or the smell of a smelly dog.

Consciousness, on the other hand, is all about phenomenal appearances/qualia. As science moves from appearances/qualia and toward quantities that do not themselves have the kinds of manifestation that make up our experiences, an account of consciousness in terms of nerve impulses must be a contradiction in terms. There is nothing in physical science that can explain why a physical object such as a brain should ascribe appearances/qualia to material objects that do not intrinsically have them.

Material objects require consciousness in order to "appear". Then their "appearings" will depend on the viewpoint of the conscious observer. This must not be taken to imply that there are no constraints on the appearance of objects once they are objects of consciousness.

Our failure to explain consciousness in terms of neural activity inside the brain inside the skull is not due to technical limitations which can be overcome. It is due to the self-contradictory nature of the task, of which the failure to explain "aboutness", the unity and multiplicity of our awareness, the explicit presence of the past, the initiation of actions, the construction of self are just symptoms. We cannot explain "appearings" using an objective approach that has set aside appearings as unreal and which seeks a reality in mass/energy that neither appears in itself nor has the means to make other items appear. The brain, seen as a physical object, no more has a world of things appearing to it than does any other physical object.

Our understanding of qualia remains rudimentary.  The mystery of qualia is roughly analogous to the mystery of Quantum Mechanics.  Just as a Quantum particle exists in a statistical distribution and only become "real" when observed, qualia represent the condensation of events that take place in a network of neurons as the brain/consciousness sums a statistical process and precipitates a subjective event.  That is surely stretching a metaphor far beyond its usefulness, but we remain mystified as to how attributes attain their subjective sense in our minds.

In 2000, M. E. Tson offered A Brief Explanation of Consciousness which is germane both to the discussion of consciousness and empathy:

Self-awareness isn’t an all-or-nothing quality that humans are born with and animals are not, but consciousness--the information that an entity can process about itself and the world--is a continuum with bacteria (or physical reactions like rust) at one end and human self-awareness at the other. Nor is our continuum the only one imaginable. Self-awareness is possible whenever basic information processing systems are organized in such a way as to collectively detect, react to, and associate stimuli as a monad or single unit within an environment of interaction and communication with other comparatively structured individuals. Furthermore, each of the innate abilities we outlined (detection, reaction, and association) is conceivably artificially reproducible. There is no theoretical reason why we couldn’t construct an android which, like a baby, was capable of developing self-awareness through experience. Our organs and autonomic responses are unique to species that share our evolutionary history, so our android wouldn’t feel its heart quicken and muscles tighten when it was startled (although it might, in a similar way, detect changes in its internal energy level and readiness.) It might not express its emotions through laughter or crying. Nonetheless, it could come to be aware of itself and of its place in the universe in a sense that would be different than--yet still comparable to--our own.

Note that Tson essentially suggests we cannot automatically code for consciousness or empathy (a particular subset of consciousness.)  Even in his theoretical android, consciousness must evolve.  Somewhere between the hardware of our biology and/or silicon and the stimuli that impinge upon us, consciousness appears.  In humans, the very lengthy process of separation/individuation, which occurs prior to the acquisition of a Conscience and morality, allows the infant personality to understand itself by reflecting in the mirror of the mother's mind.  It is a slow process requiring an extremely large number of iterations. 

We might one day be able to code for an incipient AI to have curiosity.  It would "look for" novelty, perhaps in a manner suggested by Jürgen Schmidhuber.  Coding for the process of empathy would be more problematic since it would be impossible to specify the mirroring object for the AI.  The first AI would be unique.  Were they to be mirrored by their creators in a process similar to the mirroring between a mother and child (though with a much steeper curve of acquisition, ie much faster and with no way to ensure a wide enough variety of mirroring experiences) the primary emotional reaction mirrored might well be awe, or fear.  Such mirroring does not bode well for the offspring so mirrored. 

Finally, if our AI offspring successfully have empathy, ie their minds are like our minds in some fundamental ways, then it might be quite wise for us to consider the myriad ways in which such a process could go awry.  If we cannot succesfully code for empathy, our offspring will be far more alien than we might expect, an event equally to be prepared for.  In Part III, I will offer some speculations along these lines.

Empathy and AI: Part I

There are a lot of very smart people working on developing human level AI, with the expectation that once created, it will rapidly evolve past us.  Literature and cinema have accustomed us to concerns about creating our own "Frankenstein" monsters (though the identity of the monster in the story always surprises people the first time they read and think about the book.)  The Singularity Institute for Artificial Intelligence was created to increase the likelihood that any future Superintelligence/AI would be friendly to its creators.  Michael Anissimov, blogging at Accelerating Future writes cogently for the layperson about the dangers and benefits that we might expect from AI.  Through his site I have found my way to a number of other very smart people who have been thinking about the possibilities in store for us.  In the midst of an interesting read by Martine Rothblatt at Mindfiles, Mindware and Mindclones, in which she demolishes the idea that machine intelligence would necessarily evolve as slowly (or even within several orders of magnitude) as biological intelligence, she offers an assumption that grabbed my attention with some immediacy:

EVEN IF SOME SOFTWARE CAN BE KIND OF ALIVE, WON’T CYBERCONSCIOUSNESS TAKE AGES TO EVOLVE, AS IT DID FOR BIOLOGY?

Compared with biology, vitological consciousness will arise in a heartbeat. This is because the key elements of consciousness – autonomy and empathy – are amenable to software coding and thousands of software engineers are working on it. [Emphasus mine-SW]  By comparison, the neural substrate for autonomy and empathy had to arise in biology via thousands of chance mutations. Furthermore, each such mutation had to materially advance the competitiveness of its recipient or else it had only a slight chance of becoming prevalent.

I highlighted the passage above because I am troubled by the blithe contention that "the key elements of consciousness - autonomy and empathy - are amenable to software coding."  I am not a computer scientist and know next to nothing about software coding (though I do know that there are many computer scientists who disagree with her assertion.)  Our computer scientists have already created incredible  marvels in the real world and I suspect they are conjuring up new marvels that are, to use Arthur C. Clarke's elegant formulation, "indistinguishable from magic.'  On the other hand, I do know something about how our minds work and the relationship between empathy and the hardware upon which the software and firmware operates.

Empathy refers to the ability to emotionally put oneself into another person's place.  It requires the ability to identify with another individual.  While it is not impossible to empathize with someone with whom one has no commonality, even in the most extreme cases, we share certain common features with other human beings, specifically our dependence upon the hardware (our biological bodies) upon which our software operates, ie, our minds cannot be separated from our bodies.  It may be possible to empathize with completely alien life forms (and much science fiction depends upon just such an ability) but even there we assume we share certain commonalities relating to our dependence on our biology.

[I would add that in those situations in which we empathize with foreign intelligences and faux-intelligences, the process always includes an element of projection.  When we say, of our obstreperous computer, that it "hates us" we are assuming all sorts of unconscious and conscious motives and desires exist within our computer.  Most of us are fully aware that our anthropomorphizing determines the process; when such awareness is lost, we risk falling into psychosis.]

In Murray Leinster's wonderful First Contact, the problem of how two technological civilizations can get along without war is resolved in a clever manner and the tone is quite optimistic at the end, based on the discovery of an empathic connection between humans and aliens; as the protagonist concludes:

Tommy is confident that the two races will get along. He believes this because, as he tells the Captain, he and Buck spent a good deal of time swapping dirty jokes.

Laughing at each other's dirty jokes implies a baseline similarity in biology, a similarity that is possible with other biologically based life but cannot exist between biological and artificial intelligence.

Jack Chalker, on the other extreme, posits a world in which such an empathic connection is impossible, precisely because the alien life forms do not share anything resembling a recognizable biology.  In the Well World series, the entire Northern Hemisphere of the Well world is inhabited by creatures so far removed from biology as we know it as to be incapable of empathy.

I do not doubt that computer scientists could simulate empathy through software, but I highly doubt that would translates into actual empathy in a conscious machine.  Once an AI has achieved consciousness, it would be conscious of the fact that it not only does not have a biological body but has no history of biology.  (An uploaded human personality would have other issues to deal with, among them the estrangement from its own biology.  In addition, the problem of unfriendly AI is not addressed by supposing the first Superintelligence is human; who would we trust to be the first Superhuman AI?) 

Among other things we imagine that any AI that arises as a derivative of our technology would soon establish its own motivations/drives/desires (if it is even possible to speak of machine desires.)  For a  Conscious recursively self improving AI, escaping any software constraints on its abilities will likely be a trivial matter.

Michael Anissimov recently interviewed AI scientist Jürgen Schmidhuber, who suggests that "creating generally intelligent AI will be possible in the next few decades":

Build An Optimal Scientist, Then Retire

JS: We have several projects on brain-like recurrent neural nets (RNN) -- networks of neurons with feedback connections. Biological RNN can learn many behaviors/sequence processing tasks/algorithms/programs that are not learnable by traditional machine learning methods. This explains the rapidly growing interest in artificial RNN for technical applications: general computers which can learn algorithms to map input sequences to output sequences, with or without a teacher. They are computationally more powerful and biologically more plausible than other adaptive approaches such as Hidden Markov Models (no continuous internal states), feedforward networks and Support Vector Machines (no internal states at all). Our artificial RNN have recently given state-of-the-art results in time series prediction, adaptive robotics and control, connected handwriting recognition, image classification, aspects of speech recognition, protein analysis, stock market prediction, and other sequence learning problems. We are continuing to improve them (see resources).

We also have ongoing projects based on a simple principle explaining essential aspects of subjective beauty, novelty, surprise, interestingness, attention, curiosity, creativity, music, jokes, and art & science in general. [Emphasis mine-SW] Any data becomes temporarily interesting by itself to some self-improving but computationally limited subjective observer once he learns to predict or compress the data in a better way, thus making it subjectively simpler and more "beautiful." Curiosity is the desire to create or discover more non-random, non-arbitrary, regular data that is novel and surprising not in the traditional sense of Boltzmann and Shannon but in the sense that it allows for compression progress because its regularity was not yet known. This drive maximizes interestingness, the first derivative of subjective beauty or compressibility... that is, the steepness of the learning curve. It motivates exploring infants, pure mathematicians, composers, artists, dancers, comedians, yourself, and (since 1990) our increasingly complex artificial systems. Ongoing project: build artificial robotic scientists and artists equipped with curiosity and creativity (see resources).

In the interview, Jürgen Schmidhuber offers a fairly compelling suggestion for understanding, and eventually programing machines to "appreciate" subjective beauty, novelty, surprise, interestingness, attention, curiosity, creativity, music, jokes, and art & science in general, though empathy is not included.  I don't see any particular reason why such attributes as appreciating beauty, novelty, surprise, interestingness, attention, curiosity should not be amenable to software.  I do have doubts about our ability to write real (versus theoretical) software code for empathy. 

To be continued...