Balaam et al (2011)

Balaam, M., Fitzpatrick, G., Good, J., &Harris, E. (2011)

Enhancing Interactional Synchrony with a ambient display

CHI 2011, May7-12, Vancouver, BC, Canada

Paper brings together 'research from social psychology with advances in HCI to explore a new role for pervasive and ambient technologies'  p 1

Pervasive and ambient technologies could provide  support both for encoding and decoding in authentic environments

Most of the technologies that have been developed to support social interaction (face-to-face) are informational  or support the verbal aspects e.g. RFID badges, iBand, GroupWear with mixed reactions from those involved.

Other approaches use para linguistics information to provide a view of an  interactional context.

Non verbal communication ' Research suggests that it communicates information about emotion, interpersonal relationships, and is an essential aspect of social competence' p 2 (2,7,20,22,37)

Rapport

'Rapport is one element of social interaction that is strongly encoded through nonverbal communication, and additionally theorised as key to the development of interpersonal relationships, and the success of an interaction. Rapport is believed to be crucial to develop trust, to create closeness, and be included in a  group(22) as well as help interactions feel smooth (36) .' P 2

Has three components ( see ref 37 )

1.    Mutual attentiveness

2.    Positivity

3.    Coordination ( sometimes referred to as interactional synchrony) - postural and movement. Much of this work done by Bernieri who foregrounds the importance of interactant proximity. Also that physical aspects (physical artefacts) of the task are important.

Each of which is encoded through non verbal communication with p7 ' with positivity most important at the outset  whilst coordination becomes more important later.

'there can be a disconnect between how rapport is encode and how observers decode rapport from NV signals' p 2. That  it is ' expressivity that drives social judgements' p 3

Ethics of using persuasive ambient technology is discussed on p 8.

the implication he is that more subtle NV communication can be missed  and led to the research aim - to use ambient technology to amplify (make more salient) the the more subtle NV that is sometimes missed by the observer.

'There are very few guidelines available' p 3 technologically speaking that is.

Scenario

Holiday planning in duos . Map display surrounded by representation of water that could be animated according to a wizard of Oz observer who took the role of the ambient technology of the future.

Ripples activated by

•   Simultaneous movement

•   Tempo similarity

•   Coordination

•   Posture matching

•   Back channel responses ( e.g. Head nodes, utterances showing agreement,/listening such as 'uh ha'

•   Eye contact

Explored ' the relationship between self reports of rapport, observer reports of rapport, and the body language of the participants in the holiday planning.' P 3 did the task twice (for personal use, for an older pair), once with the animation and once without. Untrained observers rated a video of the exchange , the video record was subject to a mosaic procedure that removed some of the expressive codes ( the  ones that are attended too at the expense of more subtle NVs)

Rapport scale by Bernieri (

ref 7) based on theory of 37.

'The data presented in the Results section indicates that our amplification of rapport through an ambient display increased the amount of interactional synchrony, and more specifically, the amount of coordination and posture similarity seen within an interaction by untrained observers.  At the same time, interact ants themselves were seemingly unaware of these increased behavioural manifestations of rapport, with no significant differences in their self reports of rapport between conditions'

Barron and mutual attention p 9 ' To aid this mutual attention, groups coordinated themselves and their work through the use of physical artefacts in the environment, conversational devices, and coordinated physical movement'   Overlaps with Salt and Bernieri

Discussion section contains more detailed account of HCI considerations

Refs

23, 34 spontaneous versus impression management

28 empathy

37, 5, 22, 7rapport

12, 18 Pervasive technology to support Group work

Meltz paper at Stllar

Meltzoff, A.N., Kuhl, P.K., Movellan, J., and Sejnowski., T.J. (2009)

Foundations for a new science of learning

Science, 325, 284-288

p284 'Human learning and cultural evolution are supported by a paradoxical adaptation. We are born immature. During the first year if life , the brain of an infant is teeming with structural activity' with sensory processes developing before higher activity'

'Three principles are emerging from cross-disciplinary work in psychology, neuroscience, machine learning, and education, contributing to a new science of learning'  and, in particular, are useful for explaining,, language and social understanding.

1.    Learning is computational, implicit

2.    Learning is social, implicit

3.    Learning  is supported by brain circuits linking perception and action

1. Learning is computational

' infants and young children possess powerful computational skills that allow them to automatically infer structural models of their environment from the statistical patterns they experience' eg 'before they are three, children use frequency distributions to learn which phonetic units distinguish words in their native language' p 285 ' Statistical regularities and co variations in the world thus provide a richer source of information than previously thought' and the learning    running around these regularities is implicit. ' Learning from probabilistic input provides an alternative to Skinnerian reinforcement learning and Chomskian nativist accounts' of learning

2. Learning is social

p285 'Children do not compute statistics indiscriminately. Social cues highlight what and when to learn'  young infants 'more readily learn and enact an event when it is produced by a person than be an inanimate device. Machine learning studies show that systematically increasing a robot's social-like behaviours and contingent responsivity elevates young children's willingness to connect with it and learn from it'

3. Learning is supported by brain circuits linking perception and action

' Human social and language learning are supported by neural-cognitive systems that link the actions of self and other.'  The brain areas responsible for initiation of movement and its action overlap. ' Social learning, imitation, and sensorimotor experience may initially generate, as well as modify and refine, shared neural circuitry for perception and action'.  KRO to what extent and what is the nature of 'the close coupling and attunement between self and other, which is the hallmark of seamless social communication and interaction'

Social learning and understanding

Three social skills are foundational

1.    Imitation

2.    Shared attention

3.    Empathy and social emotions

 Imitation

'Learning by observing and imitating experts in the culture is a powerful social learning mechanism' ' Imitation if faster than individual discovery and safer than trial and error learning' ' Children can use third person information ( observation of others) to create first person knowledge. This is an accelerator for learning: Instead of having to work out causal relationships themselves children can learn from watching experts' ' Imitative learning is valuable because the behavioural actions of others "like me" serve as a proxy for one's own' ' Children do not slavishly duplicate what they see but reenact a person's goals and intentions' ie ' they produce the goal that the adult was striving to achieve, not the unsuccessful attempts. Children choose whom, when, and what to imitate and seamlessly mix imitation and self discovery to solve novel problems'  attempts in robotics to emulate infant imitation include direct (input-action) and more recently goal based approaches .

 Shared attention

'Social learning is facilitated when people share attention. Shared attention to the same object or event provides a common ground for communication and teaching. An early component of shared attention is gaze following' experimental evidence to show that ' we project our own experience onto others'. P286  ' The ability to interpret the behaviour the behaviour and experience of others by using oneself as a model is a highly effective learning strategy that may be unique to human........It would be useful if this could be exploited in machine  learning'

Empathy and social emotions

' The capacity to feel and regulate emotions is critical '  ' In humans, many affective       processes are uniquely social'. Children will even help and comfort a social robot that was crying Tanaka,Cicourel,Movellan, 2007) 'Brain imaging studies in adults show an overlap in the neural systems activated when people  receive a painful stimulus themselves or perceive that another person is in pain  Hein & Singer (2008) These neural reactions are modulated by cultural experience, training, and perceived similarity between self and other Hein & Singer (2008)

Language Learning  - as shedding light on the interaction between computational learning, social facilitation of learning, and shared neural circuitry for perception and production.

Evidence to show that developing infants pick up the statistical regularities of a language leading to neural commitment. ' However, experiments also show that the computations involved in language learning are "gated" by social processes (Kuhl, 2007). In foreign language learning experiments, social interaction strongly influenced infants' statistical learning. Infants exposed to a foreign language at 9 months learn rapidly, but only when experiencing the new language during social interchanges with other humans. 'Temporal contingencies may be critical'.

Idea of neural commitment

A similar pattern , ' passerine  birds learn conspecific song by listening to and imitating adult birds' ' In birds, as in humans, a social context enhances vocal learning'.

Barron (2003)

Bridget Barron (2003)

When Smart Groups Fail

The Journal of Learning Science,12(3), 307-359

no tutor, 6th graders, face to face, all triads

Value of collaboration and the problems that can occur

P308 ‘intense interaction between partners, nurtures achievements when partners share interests, knowledge, personal history, and a commitment to work ‘ quotes John-Steiner, 2000.

but

‘groups are source of aggravation, feelings of wasted time and discouragement’ ( Salomon & Globerson, 1989)

outcomes of collaborative groups can be very different, i.e. there is variance between groups

p309 ‘ there is a need for better articulation of the characteristics of interactions that lead to differentially productive joint efforts’ p301 ‘ even when the knowledge or insights that individual members generate does not differ between groups’

units of analysis – individual or group

Therefore

‘research in this article is concerned with advancing the understanding of how the microinteractional processes between collaborators ‘ influence collective achievements and what individuals learn from their interactions’

Three Main ( general) ideas

Attention ‘ management of attention is a fundamental aspect of interactional work during collaborative problem solving’, quotes Roschelle, Teasley

In Face to face there are non verbal strategies for joint attention eg pointing, tapping, moving to share perspective of the workgroup

Metacognitive comments eg need to keep pace

‘both speakers and listeners have consequential roles to play in establishing joint attention’

there are ‘barriers that some participants face having their ideas heard and how this is particularly challenging if partners are self-focused’ KRO seems that it applies to 6^th graders also. On the other hand ‘persistence coupled with increasing strength of presentation can pay off’ but might be offputting or discouraging, particularly for more passive members.

think in terms of a ‘cognitive space’ plus ‘a relational space’ ‘which are negotiated simultaneously and can compete for limited attention’ ‘ information made available in the space from the self and from others’ activities must be integrated’ ie relational aspects of the interpersonal context.

There is sometimes an unwillingness to negotiate a shared space.

Crook 1996, p116 describes ‘intersubjective attitude’

Fogel (1993) p337 ‘ true communication takes co-regulation , a willingness and openness to be influenced by others’

P331

“Maintaining the group as a unit of analysis focuses attention on the emergent patterns of interaction and allows for the identification of individual conversational moves that shift patterns’ eg persistence

‘an important property of language in interaction is its flexibility and generativeness. Silence, repetition of ideas, eye gaze, gestures, physical synchrony, laughter, pauses, interruptions and overlaps in turn taking do not have single meanings but have productive ambiguity, thus depending on the context they signal different things to different people’.

RQs

1. ‘What interactional processes are associated with better group problem solving’

2. ‘How does the quality of group problem solving relate to individual learning as indicated by subsequent independent performance on the same and a related problem’

3. ‘What social and cognitive factors contribute to the emergence of more and less productive interactional patterns’

p312 ‘ Although Rochelle’s analysis focused mostly on the cognitive aspects of creating a joint problem space other studies suggest that to deeply understand the nature of productive collaboration, attention must be paid to the ecology of relations that develops within interactions that allow group members to access and functionally express knowledge and other cognitive resources’

e.g.

Engle & Conant (2002) describe a student scenario with sustained debate over several weeks p312 ‘ students passionate engagement was reflected in intensive emotional displays, persistence in having their ideas heard, additional research, and continued attention over weeks’ . A key aspect of their discourse that allowed for productive learning conversation rather than developing into argumentation shouting matches was the appropriation of scholarly moves such as various kinds of evidence to justify their claims’ KRO what is the equivalent, appropriation of moves, for social ?

Design

Problem

How to get back from Cedar Creek before nightfall – based on mathematical calculations. 6^th graders, similar ability > 75%, same gender triads.

Students required to complete a workbook that posed 8 questions ( 2 planning, 3 subproblems planning questions, 3 solutions). Analysis based on the first subproblem.

Groups were videotaped

Group as the unit of analysis

IV Level of group performance on a complex problem

DV uptake of a correct proposal and response sequences ; based on the observation that correct proposals were not always taken up.

Scoring

Three calculations required for the first subproblem. 0,1, or 2 for each i.e. 1 for partial , 2 for a total solution.

Individual as unit of analysis

Ability of individuals to solve similar problems in the future.

Group as the unit of analysis

Time sequence of proposals and responses

Scoring

Transcripts parsed into turns. Backchannel e.g. umhh, yeah, aldo considered as turns

Analysis

6 staged approach. 1,2,3,4,& 6 based on group as the unit of analysis

Quant (1-5), qual (6)

1. Group problem (sub) solving compared

7 gained total solution, 1 a partial solution, 4 below 50% correct

2. Groups compared for other variables

prior achievement

number of turns

difference beteen partners in turns taken

correct proposals made

3. apply a coding scheme for the way in which correct proposals are responded too.

Three categories, accept, discuss, reject or ignore.

Chi square analysis- significant association between success of groups and pattern of response to proposals.

4. Links between a correct proposal and the relatedness of the prior discussion.

Used the video data

Using Chi square analysis , two associations are confirmed.

Success and relatedness to previous discussion

Relatededness to the prior discussion and acceptance of a correct proposal

5. Individual student subsequent performance

Students in the success groups performed better than student in the unsuccessful groups

6. Sequential view of the interrelation between cognitive and social based on four triads. p331 ‘maintaining the group as a unit of analysis focuses attention on the emergent pattern of interaction and allows for the identification of individual conversational moves that can shift patterns’

Four groups, 2 successful (c&d), 2 unsuccessful (a&b) included in this analysis

Group a

Competitive nature of exchanges.

Claims of competence

Failure to acknowledge the contribution of others

Domination of the workbook

3^rd member used mainly as a scribe

Group b

Parallel efforts on different parts of the problem

3^rd member mostly used as a scribe but makes timely and appropriate suggestions albeit in a hesitant way. Although one other member begins to take notice, other member jumps in with own ideas. Example of a damaged turn Erikson 1996, p37-38 due to hesitancy

Group c

Triad who confronted some of the same issues as group b but who had members that demanded the joint attention be preserved. Third member used physical contact when necessary. There was also evidence of metacommunication ‘wait until the first parties written down’

During initial brainstorming all three were visibly excited and engaged. Third member took on observer role to some extent for part of the time whilst other two put forward parallel ideas

Group 4

‘exchange of conversational turns is rapid ‘ with all three participating. Joint attention and joint problem space maintained throughout.’ P347 the workbook served as a centre of co-ordination ( suchman, 1997, p42) for group work

‘In the two most successful groups, we see an interweaving of members own content space work with an awareness of others’ progress’

Portraits of interactional contexts

‘In less successful cases, relational issues arose that prevented the group from capitalizing on the insights that fellow members had generated. These included competitive interactions, differential efforts to collaborate, self-focused problem solving trajectories.

Behaviorally these manifested as violation of turn taking norms, difficulties in gaining the floor, domination of the workbook, competing claims of competence’

‘Persistence and resistance to dominating efforts were effective’

p349 ‘successful achievement of a joint problem-solving space was especially reflected in high rates of huddling around workbooks and mutual gaze’…….’it was not that more successful groups were immune to problems of coordination but rather that members used strategies that recruited or evoked a joint focus of attention’

lead to

idea of ‘a between person state of engagement’

Finally author broadly discusses three issues

Ethos/assessment approaches that value competition over collaboration

Previous practice and experience eg norms, appropriation of practice

Friendship and familiarity including familiarity with practice