Krach et al (2008)

Krach, S., Hegel, F., Sagerer, G, Binkofski, F., & Kircher, T (2008)

Can machines think? Interaction and Perspective taking with robots investigated via fMRI

PloS ONE 3, 7, e2597

A key question for robotic dialogue design is ' how to communicate the internal system state of a robot in a way that is understandable to the human user'

 Research question

do we attribute human like properties to machines? Even those that look and/or behave like humans?

Specifically

Activity of right TPC and medial Pre frontal cortex is hypothesised to linearly increase with the perceived grade of human likeness of the interactants

Studies investigating TOM with fMRI 

•   usually asked participants to take the perspective of various stimuli types, cartoon characters, persons on a photograph, ie ps asked to explicitly evaluate TOM in a highly controlled context

•   More recently used reciprocal interactive games between human participants  in order to access a more implicit perspective

Design

Highly interactive game scenario

4 opponents, ' all hypothetically differing liberally in the perceived grade of human likeness. 'Human likeness was operationalised by increasing the degree of anthropomorphism and embodiment'. (  Embodiment refers to the need for physicality in attribution processes, anthropomorphism as a way of explaining things in a way that we understand). It might be that a robot gives a greater sense of presence especially if it engages with the shared environment. Robots anthropomorphise more easily when they more like humans they are interacting with

•   A computer CP no human shape, no perceivable button pressing

•   A functionally designed Lego robot FR no human shape, button pressing with artificial hands

•   An anthropomorphic model. Human like shape, button pressing with human like hands

•   Human partner HP human shape, button pressing with human like hands

Video of the image beamed into the scanner. P was in fact always playing with the same confederate but did not know that assumed opponent was as per video image

Findings

' as a prerequisite to derive meaningful interpretations of the behavioural and functional imaging data on-line response behaviour and questionnaires indicated that all 20 participants

•   believed in the setting I.e they believed to really interact with the partners online'

•   ' neither reaction times nor button passing differed significantly between conditions'

•   'Overall, participants played rather competitive with a ratio of around 60/40 (competitive/cooperative) decisions, irrespective of the partner being played'

•   Debriefing questionnaire

•   Fun Intelligence CP

•   Competiveness CP

•   Human likeness and sympathy  rated only for AR, FR

fMRI findings

' participants increasingly engaged cortical regions corresponding to the classical TOM network the more respective games partners exhibited human like features'

TPJ - each comparison

MPFC pro innately dorsal - AP & HP only

Implications and conclusions

' To summarise the present study provides first evidence that the degree of human-likeness of a counterpart modulates its perception, influences the communication and behaviour, biases mental state attribution, and, finally , affects cortical activity during such interactions'

Meltzoff et al (2009) notes

Meltzoff, A.N., Kuhl, P.K., Movellan, J., and Sejnowski., T.J. (2009)

Foundations for a new science of learning

Science, 325, 284-288

p284 'Human learning and cultural evolution are supported by a paradoxical adaptation. We are born immature. During the first year of life , the brain of an infant is teeming with structural activity' with sensory processes developing before higher activity'

'Three principles are emerging from cross-disciplinary work in psychology, neuroscience, machine learning, and education, contributing to a new science of learning'  and, in particular, are useful for explaining,, language and social understanding.

1.    Learning is computational, implicit

2.    Learning is social, implicit

3.    Learning  is supported by brain circuits linking perception and action

1. Learning is computational

' infants and young children possess powerful computational skills that allow them to automatically infer structural models of their environment from the statistical patterns they experience' eg 'before they are three, children use frequency distributions to learn which phonetic units distinguish words in their native language' p 285 ' Statistical regularities and co variations in the world thus provide a richer source of information than previously thought' and the learning    running around these regularities is implicit. ' Learning from probabilistic input provides an alternative to Skinnerian reinforcement learning and Chomskian nativist accounts' of learning

2. Learning is social

p285 'Children do not compute statistics indiscriminately. Social cues highlight what and when to learn'  young infants 'more readily learn and enact an event when it is produced by a person than be an inanimate device. Machine learning studies show that systematically increasing a robot's social-like behaviours and contingent responsivity elevates young children's willingness to connect with it and learn from it'

3. Learning is supported by brain circuits linking perception and action

' Human social and language learning are supported by neural-cognitive systems that link the actions of self and other.'  The brain areas responsible for initiation of movement and its action overlap. ' Social learning, imitation, and sensorimotor experience may initially generate, as well as modify and refine, shared neural circuitry for perception and action'.  KRO to what extent and what is the nature of 'the close coupling and attunement between self and other, which is the hallmark of seamless social communication and interaction'

Social learning and understanding

Three social skills are foundational

1.    Imitation

2.    Shared attention

3.    Empathy and social emotions

 Imitation

'Learning by observing and imitating experts in the culture is a powerful social learning mechanism' ' Imitation if faster than individual discovery and safer than trial and error learning' ' Children can use third person information ( observation of others) to create first person knowledge. This is an accelerator for learning: Instead of having to work out causal relationships themselves children can learn from watching experts' ' Imitative learning is valuable because the behavioural actions of others "like me" serve as a proxy for one's own' ' Children do not slavishly duplicate what they see but reenact a person's goals and intentions' ie ' they produce the goal that the adult was striving to achieve, not the unsuccessful attempts. Children choose whom, when, and what to imitate and seamlessly mix imitation and self discovery to solve novel problems'  attempts in robotics to emulate infant imitation include direct (input-action) and more recently goal based approaches .

 Shared attention

'Social learning is facilitated when people share attention. Shared attention to the same object or event provides a common ground for communication and teaching. An early component of shared attention is gaze following' experimental evidence to show that ' we project our own experience onto others'. P286  ' The ability to interpret the behaviour the behaviour and experience of others by using oneself as a model is a highly effective learning strategy that may be unique to human........It would be useful if this could be exploited in machine  learning'

Empathy and social emotions

' The capacity to feel and regulate emotions is critical '  ' In humans, many affective       processes are uniquely social'. Children will even help and comfort a social robot that was crying Tanaka,Cicourel,Movellan, 2007) 'Brain imaging studies in adults show an overlap in the neural systems activated when people  receive a painful stimulus themselves or perceive that another person is in pain  Hein & Singer (2008) These neural reactions are modulated by cultural experience, training, and perceived similarity between self and other Hein & Singer (2008)

Language Learning  - as shedding light on the interaction between computational learning, social facilitation of learning, and shared neural circuitry for perception and production.

Evidence to show that developing infants pick up the statistical regularities of a language leading to neural commitment. ' However, experiments also show that the computations involved in language learning are "gated" by social processes (Kuhl, 2007). In foreign language learning experiments, social interaction strongly influenced infants' statistical learning. Infants exposed to a foreign language at 9 months learn rapidly, but only when experiencing the new language during social interchanges with other humans. 'Temporal contingencies may be critical'.

Idea of neural commitment

A similar pattern , ' passerine  birds learn conspecific song by listening to and imitating adult birds' ' In birds, as in humans, a social context enhances vocal learning'.

Sanger et al (2011) notes

Sanger, Lindenberger, Muller (2011)

Interactive brains, social minds

Communicative & Integrative Biology

P 655  difficult ' studying the complexities of social interaction in tightly controlled experimental settings' p 661 'real-life social interactions are spontaneous, reciprocal , and multimodal, and thereby pose great challenges to experimental design and the ability to draw causal inferences'

Definitions

Social cognition ' the mechanism that allows us to understand others '

Mentalizing , theory of mind ' the ability to represent other people's mental states (Frith & Frith (2002) as well as the knowledge needed for interaction and formation of social relationships

Social interaction is more narrowly defined 'turn taking among active, autonomous agents who follow social rules and control their action and reactions according to. Their perceptions'

Joint action ' any form of coordinated action bringing about change'

Coordination ' non accidental correlation between the behaviours of two or more systems that are in sustained coupling, or have been coupled in the past, or have been coupled to another, common system'

Interpersonal action coordination  occurs in ' the context of joint actions and coordination' note synchronisation of speech and movements does not qualify. '

Interpersonal action coordination

Discussed in terms of musicians and dancers but could apply to collaborative learning, especially face-to-face.

'interpersonal action coordination requires the perception,representation and anticipation of one's own and  partner actions'

Joint goal, (task) required. The task determines individual intentions (which may be very different especially in learning contexts)

Investigating the neural basis of social interaction

P 656 'Currently little is known about the brain areas that are involved and the neural mechanisms that implements interpersonally coordinated behaviour'

Designs and methods

Collectively the following  implicate fronto parietal areas

Focus

•    Agency

•   Cooperation &competition

•   Intentional stance

•   Self relevance and interpretation of relational stance

Single subjects intact interacting with

•   Computers

•   Virtual counterparts

•   Real counterparts

Methods and techniques involving  EEG

•   Formation of shared action representations' (40)

•   Movement coordination (41)

•   Different forms of action coordination (50,51)

Conclusions and outlook

P 661

' reconcile the dynamics of e phenomenon with the requirements of experimental control'

' there is a need for studies that assess the target behaviour as well as the behavioural cues exchanged between  interaction partners in real time, and relate these measures to neural synchronisation within and between brains' ....  ' Interbrain synchronisation during interpersonal interaction (KRO how important is this online?) coordination clearly depends on multimodal perceptual cues ( e.g. Gestures, facial expressions, movements), but the relation between these cues and Interbrain synchronisation is rarely assessed or analysed' 

Saxe (2006) notes

Rebecca Saxe (2006)

Uniquely human social cognition

Current Opinion in Neurobiology, 16, 235-239

Foundational capacities are the only aspects  of human social cognition  that are not uniquely human they are shared by preverbal infants , apes and monkeys

•   Recognise co specifics

•   Monitor others' actions

•   Engage in contingent interactions

•   Understand  basic mental states such as goals and actions (apes and preverbal children (see also  Meltzoff & Decety, 2003) for preverbal children)

What aspects of social cognition are uniquely human?

However, apes and monkeys  and very young infants do not have the following two social cognition  competences

 1. Theory of Mind ( Temporo-parietal junction TPJ)

P235 being able to ' distinguish between the object of a mental state

 what a person's mental state is about , the state of affairs to which the belief or perceptions refers) and the content. (How that state of affairs is represented, what the person believes or perceives to be true of it). KRO for project work the object, state of affairs would be  the task and the understanding that group cohesion needs to be maintained in order to make progress with the task. 'Command of this distinction enables older children to understand how people's mental representations of the world might differ from the way the world really is' KRO or that it differs from their own understanding of it   I.e. Saxe ( shown as italics) would extend the 2003 definition by Meltzoff & Decety ' To become a sophisticated mentalist one needs to analyse both the similarities and differences between one's own states and those of others' as they refer to an object,state of affairs

2.  Joint attention (medial prefrontal cortex MPFC) - mental representations with a three place (triadic) structure ie triadic social relations

'This second unique component of human social cognition requires an individual to represent  triadic relations 'You, and Me, collaboratively looking at, working on or talking about This'

Saxe is discussing these theories  in terms of the physical presence of an intentional actor.

Foundational stages are the first steps when reasoning about others' actions.

1.  Detecting the presence of an intentional actor

 (Extrastriate body area (EBA)). A region in bilateral occipito-temporal cortex that shows a selective response to human bodies and body parts, relative to other familiar objects. Right specialised for perceiving others). Verbal stories about the human body do not suffice.

2. Interpreting the motions of a human body in terms of the person's goals

Posterior Superior temporal sulcus (pSTS), usually right lateralised, recruited both during direct observation and indirect observation of the results of the action.  I.e. it represents the relationship between a movement and its context.  For simple goal directed actions, the response in r. pSTS is increased when there is a mismatch between the action and the target an an action

Representing the specific (representational) contents of mental states such as beliefs

Temporo-parietal junction , adjacent to but distinct from the pSTS

EP236 ' the BOLD response in this region is high when subjects read stories that describe a character's true or false beliefs but low during stories containing other information about a character, including appearance, cultural background, or even internal, subjective sensations .....that have no representational content'

 this region is also recruited 'for determining how the spatial relations between two objects would appear from a character's point of view versus from the subjects's own position'

3.  Reasoning (the sophisticated end of social cognition) about mental states

Recent imaging work has reconstructed the knowledge base on this ie MPFC not the unique neural substrate of reasoning about mental states

 p 236 ' No part of the MPFC is specifically recruited for reasoning about representational mental states' ( ie beliefs) 'instead subregions are implicated in distinct components of social cognition'  Two areas involved ventral and dorsal, distinctiveness supported by double dissociations (neuropsychological evidence)

Ventral MPFC affective empathy and sympathy (supported by evidence collected using a variety of method)

Saxe  p 237 definition of emotional empathy  based on Blair 'the cognitive and neural processes that produce a congruent emotion in the observer in response to others' directly perceived emotional displays or to descriptions of others' emotion-laden experiences'

Dorsal MPFC  implicated in ' shared or collaborative attention and goals, that is triadic relations between Me, You and This

Oxford Saljo

Saljo (2007).

Lecture on the occasion of the opening of The Oxford Centre for Sociocultural activity theorey Research, Deprtment of Education, University of Oxford, March 14,.

Studying learning and knowing in social practices. Units of analysis and tensions in theorizing.

Congruence ( or lack of ) between object of the enquiry and the unit of analysis cites

• nonsense syllables in the context of memory research i.e. nonsense syllable designed to be neutral for previous learning.

• Piagtian designed tasks p1 ‘When considering what the children were engaged in from what Wittgenstein refers to as a first-person perspective, the difficulties children experienced seemed to have had much to do with the extent to which they were able to share the situation definition and establish some kind of intersubjectivity with the experimenter.’ Rather than be indicative of a stage of cognitive competence .

From a socio-cultural perspective, the object of analysis deviates from other perspectives.

take one example, the notion that learning and knowing are “situated in human practices”

but has it being misinterpreted?

a claim that there are no regularities in human behaviour or that the individual and his or her background when engaging in an activity play no role. Thus, a claim of this kind is often understood as saying that all human action is relative to context, and that people in this sense are merely responding to whatever they encounter. This brings the notion of situatedness quite close to a behaviourist epistemology, as some have pointed out. People are seen as behaving rather that acting. The reason for this interpretation is at some level easy to understand. Scholars in the behavioural sciences are so habituated to the individual as the unit of analysis that it is difficult to see how objectives of research such as generalization of knowledge and testing of hypothesis could be achieved otherwise.

Important insights and years of empirical research risk being silenced by the way in which representatives of the two perspectives construe each others’ positions with respect to the definitions of the object of inquiry and a relevant unit of analysis.

‘the fact that our interpretations of how to conceive of learning and other cognitive activities differ is precisely what we should be able to profit from when looking at the relationships between objects of inquiry and units of analysis. Thus, we should learn that different traditions could develop increasingly sophisticated understandings, but that they do this within a particular framework with certain premises. Neither is studying the privileged, most real, version of whatever we are attending to.’

‘In rationalist and idealist traditions, for instance in mainstream cognitivism and in differential psychology, the notion of the correspondence between what is in fact studied and what is conceptualised ( according to S a dualist perspective) has always been deeply problematic, although it may not have appeared so.’

Furthermore, quite often such general objects of inquiry as learning and memory are reified and come to be grounded in an object-like and biological or pseudo-biological conception of what is studied (Säljö, 2002). The schemas, the mental models and the various kinds of memory systems soon appear as reifications with specific characteristics that can be measured in objectivist manners.

This tendency to reify psychological processes represents an attempt to gain clarity, and maybe even respectability, by connecting learning and other cognitive processes with the biological substrate of the human mind, or, nowadays, even the brain. This implies that what is being observed and attended to as the unit of analysis is taken to be indicative of how the brain works in some specific cognitive sense. This attempt to ground cognitive phenomena in biological structures is evident in some of Piaget’s work. For instance, his argumentation in relation to the concept of “structure” attempts to provide such links between meaning making and biological structures. In his discussion of “logicomathematical structures”, he argues that these “structures essentially involve relations of inclusion, order, and correspondence. Such relations are certainly of biological origin, for they already exist in the genetic (DNA) programming of embryological development as well as in the physiological organization of the organism” (Piaget, 1970, p. 706). Thus, the “origin of these logicomathematical structures should be sought in the activities of the subject, that is, in the most general forms of coordinations of his actions, and, finally, in his organic structures.” (loc. cit.). Another sign of this reification of the object of inquiry is to try to measure it as in the case when the size of long-term memory has been assessed or when one asks how many memory systems or intelligences there are (Gardner, 1983; Landauer, 1986; Tulving, 1984). In our time, in the context of the growing success and influence of the neurosciences, this interpretation of the link between the object of inquiry and the relevant unit of analysis is very evident. What we see is to some extent a return to some of the features of localization theories of human functioning that have been strong earlier in history.

The socio-cultural alternative

As we all know this is what Vygotsky struggled with in many of his writings. One of his consistently explicit and implicit questions was: How can learning and development be understood as genuinely human, i.e. sociocultural, phenomena? In what sense is the sociocultural line of development of the child different from the biological one? (Vygotsky, 1978, 1986). What are the intellectual tools we need to document and analyse such matters on a level so that the specific features of the mediated nature of human language, thinking, reasoning, interaction and aesthetic experiences are preserved. An important part of the problem of achieving this lies in the choice of units of analysis that correspond to the objects of inquiry as understood in this tradition.

Already in his theorizing Vygotsky pointed to many observations that indirectly tell us that there is something deeply problematic with the notion of the unit of analysis that was used in testing children’s competences. For instance, the much discussed concept of Zone of Proximal Development immediately implies that the unit of analysis for understanding human knowing cannot be the restricted to individual performance. The child, when engaged in co-operation with ‘a more capable peer’ is able to solve more difficult problems than when working alone, as the definition of the ZPD goes. This implies that the object of inquiry requires a different unit of analysis in order to be interesting from a sociocultural perspective. This unit of analysis must be incorporate interaction and joint meaning-making between people.

as Linell (1992) has demonstrated, our actions are contextualized in a double manner, both in the context of the local activity we are engaged in, and in the on-going institutionalized activities of which this local practice is an established constituent. This double contextualization is also the reason why we need a concept of culture and a cultural-historical understanding of human learning.

On externalization of cognition

‘So, what we are experiencing at present is how technologies transform learning practices and the division of labour between people and tools. Why is this interesting in the context of the issue of the relationship between objects of inquiry and units of analysis? One reason for addressing this issue is that it is becoming increasingly interesting to ask the question: Where is the knowing? As I have argued, it is increasingly in the coordination between people and tools, and through the externalization of what was previously human thought processes, carried out step by step and often on the basis of learned algorithms, we engage in activities without anything near full mastery of the procedural work carried out by the tool. When appropriating such tools we can often learn to use them for practical purposes in local practices, while at the same time be more or less ignorant of how they operate. In sociocultural terms this can be understood as an appropriation of a range of representational systems and technologies that come together in one piece of technology which has an interface which is relatively easy to learn. However, my insight into each of these may be very limited, in fact I may be more or less ignorant. But can I navigate? Yes and no! With the GPS, I can do it, with earlier technologies such as compasses my skills may not be sufficient. But with Wittgenstein’s criterion of understanding and knowing where he argues that “understanding is like knowing how to go on” (§ 875, 1980), I certainly know how to go on with the activity of navigating if I have the GPS navigator around.’

And therefore his argument against units of analysis and a positivist approach

‘Rather than repeating what is already there, learning has become the ability to put previous insights and experiences to use in relevant manners; it has become future oriented.’ We learn, but we learn differently from previous generations. A consequence of this is that in research we can subsume less and less of our understanding on such traditionally very general notions as learning and remembering, and more and more will have to be an understanding that is relevant to activities and activity systems.

Säljö, R. (2002). My brain's running slow today-the preference for "things ontologies" in research and everyday discourse on human cognition. Studies in Philosophy and Education, 21(389-405).