agent design echoes

Echoes - design of the agent

Building autonomous agents for children with autism

Aims for the agent -

p 46 ' to be able to act credibly both as a peer and as a tutor'

 achieve appropriately designed transactional support. 

Autonomy p 47 ' an agent that is able to decide independently how to act best in order to achieve a set if high level goals that have been delegated to it'

Be pro-active - engender motivation & attention

Reactive - adaptive

Have social ability - so that it can maximise the chances of the child experiencing a sense of self-efficacy

An ideal social agent with socio-emotional competence

ECHOES environment

12 learning activities that focus on two sub components that are challenging for children on the autistic spectrum -

(i) joint attention ( the ability to coordinate and share attention and emotions

(ii) symbol use -p48 ' understanding of meaning expressed through conventional gestures and words and ability to use non verbal means to share intentions'

Design for the agent

Derived from OCC and appraisal theory. OCC identifies 22 emotions. Agents provided with an affective system composed of emotional reaction rules, action tendencies, emotional thresholds, emotion decay rates

P 499 ' The agent experiences one or more of the 22 emotions of the OCC model based on its appraisal of the current external events and IRS subjective tendencies to experience certain emotions instead of others. The agents deal with these emotions by applying problem-focused or emotion-focused coping strategies

The agent

Advantages of an agent - tireless, persistent, consistent & positive

Positive, motivating, and supportive. Tends to be happy, does not get frustrated easily.

Actions - verbal (using simple language or key words), non-verbal (eye gaze and gesture)

Facial expression, a range of positive facial expressions ( involving lips, eyes, & eyebrows)

Evaluation

Response to bids for interaction ( post intervention cf Pre intention)

From the agent -  slight increase but not SS

From the human practitioner- did increase and almost reached SS

Initiation of bids

To the agent - numerical increase but not SS

To the human participant - very low and remained low

multidiscplinary - echoes

Porayanska -Pomsta et al

Key features, benefits challenges of a multi-disciplinary approach

Journal of Personal and ubiquitous Computing

Literature

22. - emotion recognition

25, 40 - understanding the mental states of others

6,18 - LFA engage with robots more than humans (shared attention, turn taking), fail to generalise to a real world context

37, 38,29 - wearables

42 - abilities of autism

46 importance of reciprocity

ECHOES a socio- cognitive intervention

Multidisciplinary - theories, practices, methods, scientific tradition. Establish common ground and draw on strengths p2 Novelty of the approach lies in the way in which different methods and techniques are combined and applied.

Goal- enable social interaction skills.

Aim - develop tools for research in this area

Affective system for an agent - emotion regulation, recognise emotions, categorise emotions, express emotions

Objects in a garden as the focus of joint attention

Monitors - head posture, eye gaze, facial expression, screen touch data

Retain the development of resources within the users community of practice

Pilot - as a small scale intervention. Subjective also contributes to the design of the resource

Theory of mind - impute others mental state

Joint attention - there is a strong visual component , both the object and the other

How are objects in ECHOES linked into a narrative?

Two challenges p122 SCERTS, on which ECHOES as an intervention is based was developed for a human-human intervention context, in which practitioners use their long term experiences. Multiple data sources on which to base the decision. ' Another challenge relates to whether the child perceives the agent as an intentional being or merely an inanimate object' reciprocity is important 

f & F 2012 extracts

Learning & cooperation :

Taking a comparative approach

This comparative approach to social cognition can identify processes in common

across species. It can also help identifying the nature of those processes that are

unique or at least dramatically more highly developed in humans.

We passionately believe that social cognitive neuroscience needs to break away from

a restrictive phrenology that links circumscribed brain regions to underspecified

social processes.

 a) Implicit 

 ( observation, imitation) place, objects, actions:

gaze is important

perception of biological motion

animacy (is subserved by a circumscribed brain region in the superiortemporal sulcus (pSTS Puce & Perrett 2003) and the ability to distinguish this motion from other kinds of motion is exquisitely tuned.)

 Agents also typically have faces and their eyes give cues to what they are interested in. (A posterior region of STS is specialised for analysis of facial movements, while invariant aspects of faces are analysed in the fusiform gyrus (FFA Haxby et al 2000,)

Another important cue to agency is contingent responding. Human infants and adultsalike are likely to treat a shapeless object as being animate if the object moves, ormakes noises that are contingent on their own actions (Johnson 2003).

Alignment.

A necessary consequence of learning by observation is the formation

of behavioural similarity across a population. This is most obviously the case when

learning about actions. When we interact with others we often automatically imitate

their behaviour (Chartrand & Bargh 1999). In the case of verbal interactions this

alignment can occur at many levels. During a productive discourse, speakers will

automatically tend to align their posture, their speech rate, their choice of words and

their syntactic forms (Garrod & Pickering 2009).

Emotional contagion

Perceiving an emotional response of another person elicits the same emotional

response in ourselves. This is also called emotional contagion and allows us to share

the emotion of the person we are observing (de Vignemont & Singer 2006), a

prerequisite of empathy. Emotional contagion supplies a basic conditioning

mechanism through which we can learn from others on the basis of their emotional

expressions. This process, even though likely underwritten by a general mechanism of

association learning, has some claim to be a specifically social process, having social

content and being solely in the service of social cognition.

Costs and benefits of observational learning

1because demonstrators will selectively perform the actions that

they have found to be most beneficial for themselves, they effectively and

inadvertently act as a filter to provide the information that is most useful for an

observer. Copying is a highly adaptive means of gaining knowledge (Rendell et al

2010)

2. Prosocial effects of copying. While learning from observation can serve purely

short-term self-interest. Contagion and copying can also bias us towards the long-term

interests of our group (which also serves self-interest). This effect is seen in

experiments, which reveal subtle effects of copying. If we are covertly mimicked, we

tend to like that person. Furthermore, we become more helpful to people in general

(van Baaren et al 2004). Similar effects have been demonstrated in monkeys, who are

more likely to approach and share food with an imitator (Paukner et al 2009). These

effects are likely to be unconscious. In contrast, when people are aware that they are

being imitated (see Bailenson et al 2008), they experience high levels of discomfort

and thus the prosocial effects do not occur. At the neural level there is evidence that

mimicry is rewarding. When we observe someone else being imitated activity

increases in reward-related regions such as vmPFC (Kühn et al 2010). When others

make the same choice that we have just made merely on the basis of ‘liking’, then

reward areas of the brain are activated just as when we received the desired object

ourselves (Campbell-Meiklejohn et al 2010). Thus, the reward in being endorsed by

others may result in reinforcing group oriented behaviour and conformity.

Explicit

 deliberate communication, processes that enable individuals to understand one another with a high degree of precision. ( ? uniquely human) (see Apperly & Butterfill 2009 fora discussion of the two forms of mentalizing). Explicit mentalizing is closely linked to meta-cognition: the ability to reflect on one’s action

and to think about one’s own thoughts.

Conclusions

In this essay we have emphasised the importance of comparisons of different species

and the use of an evolutionary framework for understanding social interaction. Much

of human social behaviour derives from the same range of cognitive processes that

can also be seen in other social animals.

What then in social cognition is specific to human beings? First, through language

humans have the means of creating processes that are explicit. Second, humans, in

comparison to other species, have a much greater ability to exert top-down control

over automatic processes. This is particularly important when there is competition

between different components of social cognition. Third, humans have the

extraordinary ability to reflect upon their own mental states. This is a prime example

of meta-cognition, which may well lie at the heart of conscious awareness

Krach et al (2008)

Krach, S., Hegel, F., Sagerer, G, Binkofski, F., & Kircher, T (2008)

Can machines think? Interaction and Perspective taking with robots investigated via fMRI

PloS ONE 3, 7, e2597

A key question for robotic dialogue design is ' how to communicate the internal system state of a robot in a way that is understandable to the human user'

 Research question

do we attribute human like properties to machines? Even those that look and/or behave like humans?

Specifically

Activity of right TPC and medial Pre frontal cortex is hypothesised to linearly increase with the perceived grade of human likeness of the interactants

Studies investigating TOM with fMRI 

•   usually asked participants to take the perspective of various stimuli types, cartoon characters, persons on a photograph, ie ps asked to explicitly evaluate TOM in a highly controlled context

•   More recently used reciprocal interactive games between human participants  in order to access a more implicit perspective

Design

Highly interactive game scenario

4 opponents, ' all hypothetically differing liberally in the perceived grade of human likeness. 'Human likeness was operationalised by increasing the degree of anthropomorphism and embodiment'. (  Embodiment refers to the need for physicality in attribution processes, anthropomorphism as a way of explaining things in a way that we understand). It might be that a robot gives a greater sense of presence especially if it engages with the shared environment. Robots anthropomorphise more easily when they more like humans they are interacting with

•   A computer CP no human shape, no perceivable button pressing

•   A functionally designed Lego robot FR no human shape, button pressing with artificial hands

•   An anthropomorphic model. Human like shape, button pressing with human like hands

•   Human partner HP human shape, button pressing with human like hands

Video of the image beamed into the scanner. P was in fact always playing with the same confederate but did not know that assumed opponent was as per video image

Findings

' as a prerequisite to derive meaningful interpretations of the behavioural and functional imaging data on-line response behaviour and questionnaires indicated that all 20 participants

•   believed in the setting I.e they believed to really interact with the partners online'

•   ' neither reaction times nor button passing differed significantly between conditions'

•   'Overall, participants played rather competitive with a ratio of around 60/40 (competitive/cooperative) decisions, irrespective of the partner being played'

•   Debriefing questionnaire

•   Fun Intelligence CP

•   Competiveness CP

•   Human likeness and sympathy  rated only for AR, FR

fMRI findings

' participants increasingly engaged cortical regions corresponding to the classical TOM network the more respective games partners exhibited human like features'

TPJ - each comparison

MPFC pro innately dorsal - AP & HP only

Implications and conclusions

' To summarise the present study provides first evidence that the degree of human-likeness of a counterpart modulates its perception, influences the communication and behaviour, biases mental state attribution, and, finally , affects cortical activity during such interactions'

Sanger et al (2011) notes

Sanger, Lindenberger, Muller (2011)

Interactive brains, social minds

Communicative & Integrative Biology

P 655  difficult ' studying the complexities of social interaction in tightly controlled experimental settings' p 661 'real-life social interactions are spontaneous, reciprocal , and multimodal, and thereby pose great challenges to experimental design and the ability to draw causal inferences'

Definitions

Social cognition ' the mechanism that allows us to understand others '

Mentalizing , theory of mind ' the ability to represent other people's mental states (Frith & Frith (2002) as well as the knowledge needed for interaction and formation of social relationships

Social interaction is more narrowly defined 'turn taking among active, autonomous agents who follow social rules and control their action and reactions according to. Their perceptions'

Joint action ' any form of coordinated action bringing about change'

Coordination ' non accidental correlation between the behaviours of two or more systems that are in sustained coupling, or have been coupled in the past, or have been coupled to another, common system'

Interpersonal action coordination  occurs in ' the context of joint actions and coordination' note synchronisation of speech and movements does not qualify. '

Interpersonal action coordination

Discussed in terms of musicians and dancers but could apply to collaborative learning, especially face-to-face.

'interpersonal action coordination requires the perception,representation and anticipation of one's own and  partner actions'

Joint goal, (task) required. The task determines individual intentions (which may be very different especially in learning contexts)

Investigating the neural basis of social interaction

P 656 'Currently little is known about the brain areas that are involved and the neural mechanisms that implements interpersonally coordinated behaviour'

Designs and methods

Collectively the following  implicate fronto parietal areas

Focus

•    Agency

•   Cooperation &competition

•   Intentional stance

•   Self relevance and interpretation of relational stance

Single subjects intact interacting with

•   Computers

•   Virtual counterparts

•   Real counterparts

Methods and techniques involving  EEG

•   Formation of shared action representations' (40)

•   Movement coordination (41)

•   Different forms of action coordination (50,51)

Conclusions and outlook

P 661

' reconcile the dynamics of e phenomenon with the requirements of experimental control'

' there is a need for studies that assess the target behaviour as well as the behavioural cues exchanged between  interaction partners in real time, and relate these measures to neural synchronisation within and between brains' ....  ' Interbrain synchronisation during interpersonal interaction (KRO how important is this online?) coordination clearly depends on multimodal perceptual cues ( e.g. Gestures, facial expressions, movements), but the relation between these cues and Interbrain synchronisation is rarely assessed or analysed' 

Saxe (2006) notes

Rebecca Saxe (2006)

Uniquely human social cognition

Current Opinion in Neurobiology, 16, 235-239

Foundational capacities are the only aspects  of human social cognition  that are not uniquely human they are shared by preverbal infants , apes and monkeys

•   Recognise co specifics

•   Monitor others' actions

•   Engage in contingent interactions

•   Understand  basic mental states such as goals and actions (apes and preverbal children (see also  Meltzoff & Decety, 2003) for preverbal children)

What aspects of social cognition are uniquely human?

However, apes and monkeys  and very young infants do not have the following two social cognition  competences

 1. Theory of Mind ( Temporo-parietal junction TPJ)

P235 being able to ' distinguish between the object of a mental state

 what a person's mental state is about , the state of affairs to which the belief or perceptions refers) and the content. (How that state of affairs is represented, what the person believes or perceives to be true of it). KRO for project work the object, state of affairs would be  the task and the understanding that group cohesion needs to be maintained in order to make progress with the task. 'Command of this distinction enables older children to understand how people's mental representations of the world might differ from the way the world really is' KRO or that it differs from their own understanding of it   I.e. Saxe ( shown as italics) would extend the 2003 definition by Meltzoff & Decety ' To become a sophisticated mentalist one needs to analyse both the similarities and differences between one's own states and those of others' as they refer to an object,state of affairs

2.  Joint attention (medial prefrontal cortex MPFC) - mental representations with a three place (triadic) structure ie triadic social relations

'This second unique component of human social cognition requires an individual to represent  triadic relations 'You, and Me, collaboratively looking at, working on or talking about This'

Saxe is discussing these theories  in terms of the physical presence of an intentional actor.

Foundational stages are the first steps when reasoning about others' actions.

1.  Detecting the presence of an intentional actor

 (Extrastriate body area (EBA)). A region in bilateral occipito-temporal cortex that shows a selective response to human bodies and body parts, relative to other familiar objects. Right specialised for perceiving others). Verbal stories about the human body do not suffice.

2. Interpreting the motions of a human body in terms of the person's goals

Posterior Superior temporal sulcus (pSTS), usually right lateralised, recruited both during direct observation and indirect observation of the results of the action.  I.e. it represents the relationship between a movement and its context.  For simple goal directed actions, the response in r. pSTS is increased when there is a mismatch between the action and the target an an action

Representing the specific (representational) contents of mental states such as beliefs

Temporo-parietal junction , adjacent to but distinct from the pSTS

EP236 ' the BOLD response in this region is high when subjects read stories that describe a character's true or false beliefs but low during stories containing other information about a character, including appearance, cultural background, or even internal, subjective sensations .....that have no representational content'

 this region is also recruited 'for determining how the spatial relations between two objects would appear from a character's point of view versus from the subjects's own position'

3.  Reasoning (the sophisticated end of social cognition) about mental states

Recent imaging work has reconstructed the knowledge base on this ie MPFC not the unique neural substrate of reasoning about mental states

 p 236 ' No part of the MPFC is specifically recruited for reasoning about representational mental states' ( ie beliefs) 'instead subregions are implicated in distinct components of social cognition'  Two areas involved ventral and dorsal, distinctiveness supported by double dissociations (neuropsychological evidence)

Ventral MPFC affective empathy and sympathy (supported by evidence collected using a variety of method)

Saxe  p 237 definition of emotional empathy  based on Blair 'the cognitive and neural processes that produce a congruent emotion in the observer in response to others' directly perceived emotional displays or to descriptions of others' emotion-laden experiences'

Dorsal MPFC  implicated in ' shared or collaborative attention and goals, that is triadic relations between Me, You and This

M & D (2003)

Andrew N. Meltzoff and Jean Decety (2003)

What imitation tells us about social cognition: a reapproachment between developmental psychology and cognitive neuroscience

Phil. Trans. R. Soc. Lond. B , 358, 491-500

P491 ‘Through imitating others, the human young come to understand that others not only share behavioural states, but are ‘like me’ in deeper ways as well’.

P494 ‘Imitation seems to be intrinsically coupled with empathy for others, broadly construed’

‘The holy grail for cognitive and neuroscience theories of imitation is to elucidate the mechanism by which infants connect the felt but unseen movement of the self with the seen but unfelt movement of the other’.

Developmental psychology approach and evidence

Imitation as innate? Newborns imitate

P492  imitation is ‘part of innate endowment of humans’ imitation in the newborn has been shown in 14 independent labs. P493 ‘There is an innate link between the perception and production of human acts’

12-21 day infants could imitate four gestures ‘ infants confused neither actions nor body parts’ also p 492 ‘It is as if young infants isolate what part of their body to move before how to move it’ ie there is ‘organ identification’

‘infants can store a model and imitate from memory …… which requires more than a simple visual-motor resonance’

Self-other relations. Awareness of self

Knowing you are being imitated requires being aware of self.

p494 ‘ The infancy work shows that young babies correct their imitative behaviour which suggests active comparison between self and other ( Meltzoff and Moore, 1997)

‘a listener often shows interpersonal connectedness with a speaker by adopting the postural configuration of the speaker’

Can infants recognise when another acts ‘like me’?

What is the emotional value of the experience?

Exp:

C1. One adult imitated the baby 

C2. Another adult imitated the previous baby ( therefore each adult acted like a perfect baby)

Results:  For C1 cf C2.  baby smiled more, looked at adult for longer, directed testing behaviour at the adult.

Older infants and sharedness  p494-495 ‘ the relationship is being abstractly considered’ ….. ‘ the abstract notion that the other is doing the same as me’

Goals and intentions

(KR0 sharing attention etc hinges on being able to determine intention and hence why realness might be important)

P495 “ In the mature adult notion, persons have internal mental states – such as beliefs, goals, and intentions – that predict and explain human actions’

P493 humans do not simply resonate, however. Our goals affect how we process stimuli in the world’ i.e. intention determines pattern of neural representation

Exp:

C1: show infants an unsuccessful act – at 18 months ‘ can infer the unseen goals implied by unsuccessful atempts’ ….. ‘ they choose to imitate what we meant to do rather than what we mistakenly did do’

Human v.  non human

This was repeated using a mechanical device that traced the same spatio-temporal path ‘infants did not attribute goals or intentions in this case’

P497 “ This developmental research shows that infants distinguished between what the adult meant to do and what he actually did they ascribed goals to human acts, indeed, they inferred the goal even when it was not attained. The differentiation between behaviour versus goals and intentions lies at the core of our mentalizing, and it underlies our moral judgements’

Neuroscience approach and evidence

P 491 ‘monkeys do not imitate’ but they do have mirror units. But whether mirror units are innately present  or the result of associative learning is not known ( as of 2003)

Kinds of questions to ask p 495 “ What is the neural basis for distinguishing the self’s imitation of the other from the other’s imitation of the self?’ – the situation in the physical world is the same – there are two bodies in correspondence with one another.

Missing p491 ‘ how a neural mirror system begets theory of mind’

Experiments and experimental manipulations

1.  Observing the actions of others

2. Future action v future recognition

·       Remembering for future action

·       Remember for future recognition

3. Animate v. inanimate

·       Observe real person

·       Observe in animate

4. Self – other

4a. Imaging actions

·       Own

·       Others

4b doing actions

·       Imitate others actions

·       See others imitate own actions

5. Means and goals ( LEGO block exp)

·       Goal achived condition  only

·       Means only

·       Whole action

·       & control conditions

Ventral premotor

(Mirror units monkeys)

STS

(Mirror units monkeys)

premotor – somotropic

observing actions of others (1)

remember for future action (Decety work) (2)

left premotor

observe an achieved goal (5)

parietal (right and left)

Observing actions of others  (1)

remember for future action (Decety) (2)

left parietal

Imaging actions ( own) ie self (4a)

intention ( human actions only -  not inanimate from exp comparing animate and inanimate) (3)

imitate others actions  (4b)

right parietal

imagining others actions (4b)

see others imitate own actions (4b)

posterior cingulated

imagining others actions (4a)

fronto polar cortex

imagining others actions (4a)

medial prefrontal cortex

imitate  others actions (4b)

see others imitate own actions (4b)

differentiated means only from goal only condition of  Exp 5.  nb from other research areas plays a critical role in inferencing ie need to be able to observe the means in order to make inferences.

Right dorso lateral prefrontal

Both means and goals condition – exp 5

left somatosensory cortex

Imaging actions ( own) ie self (4a)

left middle temporal gyrus

observing actions of others (1)

left inferior frontal gyrus

observing actions of others (1)

SMA

remember for future action (Decety) (2)

Middle frontal gyrus

remember for future action (Decety) (2)

parahippocampal gyrus in the temporal lobe

remember for future recognition (Decety) (2)

right superior temporal gyrus

intention ( human actions only -  not inanimate from exp comparing animate and inanimate) (3)

visual analysis of others actions (4)

left superior temporal gyrus

visual analysis of other’s actions in relation to actions performed by the self. (4)

cerebellum

Both means and goals condition – exp 5

intention determines pattern of neural activity

p493 ‘ these result support the notion of shared representations of self and other.  The results also suggest a crucial role of the inferior parietal cortex in distinguishing the perspective of self from other’ and the medial prefrontal for inferring the actions of others ie need to see the means of a task as well as the goals and this is consistent, including in terms of source location,  with the work on theory of mind

Theoretical speculation from combining developmental Psychology and Neurocience: from imitation to social cognition.

Theoretical speculation

Proposed : a three step developmental approach

(1)  Innate equipment.  Newborns can recognise equivalences between perceived and executed acts.  This is that a starting state

(2)  Constructing first person experience. Through everyday experience infants map the relation between their own bodily acts and their mental experiences. For example, there is an intimate relation between ‘striving to achieve a goal’ and the concomitant facial expression and effortful bodily acts.  Infants experience their own inner feelings and outward facial expressions and construct a detailed bidirectional map linking mental experience and behaviour.

(3)  Inferences about the experience of others. When infants see others acting ‘like me’ they project that others have the same mental experience that is mapped onto those behavioural states in the self.

Neuroscience -What is common and what is distinct between self and other at a neural level?

Temporal and frontal

There is neural activation in the posterior part of the temporal cortex and the medial prefrontal whatever the imitation task but must be human. Note medial prefrontal is also involved in mentalizing

Parietal

There is differential activity right versus left for inferior parietal lobe

left  p498 ‘ left inferior parietal lobe computes sensory-motor associations necessary to imitate ‘( consistent with lit. on apraxia)

right  self-other p498 ‘ is involved in recognising or detecting that actions performed by others are similar to those initiated by the self and detecting that actions performed by others are similar to those initiated by the self and determining the locus of agency for matching bodily acts’  implies that we have a body scheme and this idea is consistent with neuropsychological evidence ( disorders of bodily representation)

Summary

P498 ‘ in light of our neuroimaging experiments, we suggest that the right inferior parietal lobule plays a key role in the uniquely human capacity to identify with others and appreciate the subjective states of conspecifics as both similar and differentiated from one’s own.  ……in other words, the adult human framework is not simply one of resonance.  We are able to recognise that everyone does not share our own desires, emotions, intentions and beliefs.  To become a sophisticated mentaliser one needs to analyse both the similarities and differences between one’s own states and those of others.  That is what makes us human’.

Diana 2012 book: on imitation

 P 163 ‘Learning through practice is different because it goes beyond the realm of language and representation.  In terms of human evolution, learning through experience long predates learning through language. Learning through language and communication is, of course, a vastly more efficient way of passing on accumulated knowledge and skills , so the teaching professions from earliest times naturally made use of ‘teaching through telling’. Learning through practice in the form of learning through imitation has always been part of human, and indeed some animal , society; and learning through apprenticeship, where the imitation is accompanied by communication, is inevitably more efficient. When you learn flint-knapping and find you broke off too large a piece of slate, it saves time to have someone tell you to hit a different angle when you might have thought you were hitting too hard’

KRO learning through imitation is different from learning through practice ( which may mean learning through repeated association) or learning through doing when actions and consequences can be related one to the other i.e. embodied in some way.

Margie on Meltzoff, p148-149 (Chapter 7)

‘In Chapter  4 I noted Daniel Stern’s (2004, p. 76) claim that our nervous systems are designed to be ‘captured by the nervous systems of others’ as we observe their gestures, facial expressions, their rising and dampening affect and then model, intuit and re-run their intentions and psychological states.  In recent years, a much clearer sense has emerged of the design features involved, how affective inter-subjectivity becomes established in infancy, and then shaped and ‘personilised’ within the relational patterns and interactional routines of childhhood ( Fonagy et al, 2004)

based on Meltzoff  ‘ In other words, what seems to be present at birth is an embryonic capacity to represent one’s own body and the other’s body and coordinate the two together