The Neuropsychology of Facial Expression:  A Review of the Neurological and Psychological Mechanisms for Producing Facial Expression

Rinn, W.E. (1984)

Psychological Bulletin 95 (1) 52-77

face movement as another communication channel p52 ' so powerful is the communicative impact of the face that it is difficult to separate the message from the medium.  We tend to describe facial behaviors not in anatomical terms but in terms of the emotions portrayed'

Measurement 

subjective judgements - which of two faces looks happier/ sadder?

-

• constructs such as happiness/sadness refer to internal states with an implied statement about emotional meaning
  
• agreeing criteria for the judgement
  
• can be influenced by stereotype bias
  

objective 

descriptive method

1.  descriptive in terms of movement of the skin.  Blurton-Jones 1971

identifies 9 anatomical components , for example involving brow, mouth, eye and uses these to provide a list of descripters e.g. for the eye wide, bit wide, normal, bit narrow, very narrow, upper lid down.

2. description in terms of the facial muscles involved.  Darwin(1872/1965, Ekman & Freisen, 1978). The Facial Action Coding System (FACS).

+

more detailed 

more direct  reflection of the action of the nervous system

can provide information about the modulation of spontaneous expression  by voluntary control

p54 ' The FACS is a catalog of all perceptible 'action units' (AUs) that the face is capable of producing and the muscular basis of these AUs.  Action units are not facial gestalts but discrete movements of some part of the face'  Some AUs involve more than one muscle & some muscles are involved in more than one AU.  ' Therefore AUs do not correspond to specific facial muscles but are defined in terms of movement of the facial skin'  ' FACS manual lists the most frequently observed combinations of AUs for each of six common emotional expressions.

3.   Maximally Discriminative Facial Movement Coding System (Max). Izard(1979)

Based on  muscle movement but not comprehensive; ' appearance changes that do not discriminate between different emotions are not dealt with' 'describes nine fundamental emotional expressions ( those posited by Izard's differential emotion theory) .  Max is easier to use and may be preferred when interested in emotion-based facial movement (KRO but assumes that theoretical basis is acceptable)  developed affectogram ( Izard & Dougherty) to represent change of emotion on a time line, and be used for comparison between  and within individuals

EMG measurement of the electrical potentials in muscles

p55 Schwartz et al (1976a& 1976b)   have reliably demonstrated 'that various patterns of facial muscle activity reliably accompany the experience ( KRo imagination actually)  of different emotional states'  These authors used the method with groups of depressed patients and showed that 'in clinically depressed patients , the facial muscle response to happy imagery showed the same patterns as for non depressed subjects but in markedly attenuated form ( KRO ? finding highly relevant for  supporting the rationale for exploring online experience) 

Video

can be used to make subjective judgement as well as the descriptive analyses described above

Recommended even for face-to-face encounters in order to capture the richness of facial expression and other non verbal communication, also some very short others sustained both are reasons to be missed when just part of the conversation. 

important to control for viewing angles  when making perceptual (gestalt)  judgements.  This caution is based on the evidence that the right hemisphere is superior to the left when processing faces and emotional expression.

Facial muscles

Two main groups. Mimitic & mastication 

Each group uses a different nerve tract and has somewhat different evolutionary origins

(i)  mimitic muscles

p57  'Most of the facial muscles attach to the facial skin and manipulate the facial features into meaningful expression'  

(ii) mastication

' on each side of the face, there are four muscles

temporalis

masseter

internal & external pterygoid

 that attach to bone and ligament and move these skeletal structures around) ' 'These muscles do respond to emotion, (KRO - could be impotant control mechanism for tesaing our production from response) as do the muscles of the neck and back, they may even have a minor effect on facial expression'

Ontogeny  & Phylogeny of facial muscles

p57 'the muscles of facial expression have their evolutionary origin  in the muscles of the breathing apparatus ( the gill arches) of vertebrate fish.'  (KRO  ???? automacity influence ) ' some of the autonomic connections to these muscles ( through the facial nerve) remain intact in humans)' 'in the human embryo, vestigial gill arches ( called the branchial arches) are present in the first few weeks of life.  The muscles of facial expression arise from the second of the five branchial arches'  'the muscles of mastication arise from the first branchial branch'  ' Gradually the arches develop into other structures, and the muscles migrate to new locations, carrying their nerve tracts with them, so that by the 8th week they have arrived at their final position.  (Crelin, 1981)

Functions & behavior of facial muscles

Ekman & Friesen (1975)  divide the face into 3 regions

1. lower face including the cheeks, mouth, lower nose and chin ( participates in language by articulating the lips).  Manipulated by many small muscles and can be moved in almost any direction  with unilateral independance of action.
   
2. eyes, lids and root of the nose (punctuating speech)
3. brows and forehead ( Darwin (1872/1965) involved in attention/concentration and mental effort .  manipulated by fewer muscles and can only move up or down together.  Few people can move brow or forehead unilaterally.
   

p57 'division based on the fact that these regions are largely motorically independent of each other and make somewhat independent contributions to the facial message'

Neural basis

Neural innervation  UMN & LMN

motor neurons

upper motor neurons (UMNs) carry motor impulses from motor centres in the brain to the brain stem or spinal cord.

lower motor neurons (LMN) carry the impulses from the brain stem or cord to the muscle itself.

p58 ' the LMN tract that innervates the muscles of facial expression (mimetic muscles) is called the seventh cranial nerve ( facial nerve)'  function of the facial nerve is to 'arrange the facial features into meaningful configurations.  That is it is specialised for communication'

LMN tract of the left face is completely independent of the right face and their respective nuclei in the pons are symmetrically paired and likewise independent.  'Thus, when the left and right side of the face behave more or less indentically , it is because more or less identical signals have been sent to both LMN nuclei; that is the intergration is accomplised by UMN circuits

another LMN tract ( trigeminal nerve, cranial nerve V) innervates the facial muscles of mastication 

brainstem nucleii

p58 ' 3 brainstem nuclei contribute fibres to the facial nerve tract'. 'Two of these are served by a particular group of fibres within the facial nerve , the nervus intermidus , ...... their functions are essentially independent' (visceral & sensory) although they serve the face region  'The fibres innervating the muscles of expression all begin in a small cluster of cell bodies ( the third of the 3 relevant brainstem nuclei), the motor nucleus of the facial nerve, located in the brain stem at the level of the pons.' 

LMN tract ( facial nerve)   pathway

• Fibres leaving the facial nerve nucleus  loop around the nucleus of the 6th cranial nerve ( involved in eye movements) before leaving the pons.
  
• First major branch given off by the facial nerve is the stapedius branch which innervates the stapedius muscle in the inner ear ( dampens vibrations of the ossicles)
  
• p58 'The organisation of the peripheral portions of the facial nerve show considerable variablility between individuals, both in the course of its branches and in the specific muscles innervated by each branch' (KRO - this could be important when setting up EMG)
  
• 'Five major branches (temporal, zygomatic, buccal, mandibular & cervical) are usually present, but the adjacent branches communicate with each other through a network called the pes anserinus or parotid plexus.  Thus the precise innervation of any given muscle in any individual cannot be stated with certainty ( Harker & McCabe, 1977).' 
  
• Temporal & to an extent the zygomatic branch carries impulses originating in either hemisphere.  Only about 75% of fibres that innervate the nuclear representation for orbicularic oculi are contralateral in origin.  This trend towards bilateral innervation is even more pronounced for the upper face.
  
• Buccal, mandibular & cervical branch carry impulses only from the contralateral hemisphere KRO alert here ? pure motor cf other
• p62 ' An important 'difference between contralaterally and bilaterally innervated muscles concerns the degree of voluntary control
  

Facial nerve nucleus

• The motor nucleus of the facial nerve is the largest of the cranial nerve nuclei and contains 7,000 to 10,000 nerve cells plus numerous glial bodies .
  
• Cell groups map to specific peripheral branches of the facial nerve.
  
• One group ( the dorsolateral) supplies the muscles of the upper face, including some of the auricular muscles of the external ear.'  Please note that these also map to the frontalis , see section on raised brows.  Other groups innervate the stapedius muscle and the rest of the auricular muscles.
  

Animal ( rabbit, dog, cat) and human  tography share some similarities however there are also some differences that may relate to differences in function

• more differentiation for groups concerned with movements of the mouth and lower face.
  
• large cells bodies innervate the upper face and auricular muscles in animals and are insignificant in humans ie auricular less vital for humans.
  

cerebral cortex

The frontal lobes of the brain are separated from the parietal lobes by the central sulcus.  The anterior lip of the sulcus is known as the motor strip and is topographically organised in relation to the muscles of the body.

neural fibres from each region of the motor strip follow a systematic course through the internal capsule and brainstem and form synapses with brainstem and spinal cord nuclei

Corticobulbar connections

The efferent fibres from the motor strip are generally referred to as the pyramidal tract.

Fibres going to the brain stem are more specifically labeled corticobulbar tracts.

Coricobulbar pathways to the facial nucleus are of two types

1. direct - synapse directly onto the facial nucleus and topographically map the motor strip.
   
2. indirect - convey cortical influences first to interneurones in the brain stem reticular formation.  From these , relay neurons deliver the impulses to the facial nerve nucleus.  p63 ' These interneuron connections presumably do more than relay cortical influences.  Probably ( KRO- possibly)  they provide for subcortical moduation of cortical influences and carry direct motor impulses from subcortical motor areas of the brain.  Unlike the direct corticobulbar pathways, these interneurons carry impulses to both left and right facial nuclei.  ...... These indirect, bilateral paths are more common for the nuclear representation of the upper than lower face.
   

Red nucleus connection

p63 ' In addition to being bilaterally rather than contralaterally innervated, the upper face has another neurological feature that distinguishes it from the lower face.  The cell bodies representing the upper face ( and only the upper face) in the facial nerve nucleus recieve direct fibres from the contralateral red nucleus.  Although it receieves fibres from the ipsilateral frontal cortex the red nucleus is not simply a cluster of interneurons from the indirect cortico bulbar pathways.  It is a large well defined structure in the brain stem that gets much of its input from the contralateral cerebellum.' ' Courville (1996a) has suggested that the red nucleus conveys cerebellar influences to help the ears motoroically locate targets The cerebellum is believed to play a similar role ( albeit without red nucleus) in regulating targeted saccadic eye movements.'

Volitional and Emotional (spontaneous) innervation of the face

p63-p64

Volitionally induced movements of the face use different UMN pathways than those used for emotionally induced movements.

Neural basis

Impulses for volitionally induced movements emanate from the cortical motor strip - pyramidal(corticobulbar projections) tract.

Impulses for emotional facial movements arise in the phylogenically older motor system known as the extra pyramidal motor system.  The extrapyramidal system is not a unitary system but a group of highly interactive neural circuits, each of which contributes its own specialised influences to the final motor response.  ..... The system involves mostly subcortical nuclei, and its influences are conveyed to the facial nucleus through pathways other than the pyramidal tract.

SEE REFS ON PAGE 64 AND GET PR TO CHECK THESE FOR ME.

Evidence ( neurological)

1.  lesions of the cortical motor strip.

'Typically such  patients cannot retract the corners of the mouth to command on the side contralateral to the side of the lesion.  However, the same patients are commonly seen to smile bilaterally when something strikes them as amusing. The emotional smile uses the same muscles that are paralysed for voluntary control'

2.  lesions of various nuclei in the extrapyramidal motor system, especially the basal ganglia.

'Such patients commonly display a mimetic facial paralysis - a condition in which the patient retains the ability to move the facial muscles to verbal command but loses all spontaeous emotional movement.  e.g. masked face of  Parkinsons- disordered neurotransmitter system of the basal ganglia. '  Although some Parkinsons patients exhibit depression it is not causal for the the expression deficit . 

3.  facial nerve anastomosis : surgical procedure to reanimate a face paralysed by a lesion of the facial nerve

motor root of facial nerve is surgically severed.

the central stump is  avulsed  to prevent regrowth while the distal portion remains raw.

a few fibres are then teased from another cranial motor nerve , usually the spinal accessory nerve which supplies the muscles that move the shoulder so that impulses coming to this cranial nerve (spinal accessory)  will innervate the facial muscle as well as the shoulder.  Following operation stimulation of the motor strip representation of the face yields no responses.  However stimulation of the representation of the shoulder will yield movement in both shoulder and face.  Gradually the patient learns to move the face muscles volitionally, initially by attempting to move the shoulder.  In time differential control ( face v shoulder) can be achieved. Genuine emotional ( spontaneous) movement is never achieved on the affected side.

4.  pseudobulbar palsy - lesions of the cortico bulbar pathway involving the internal capsule ( MS, amyotrophic lateral sclerosis, anoxia, strokes) Hornstein 1977).

involuntary laughter or weeping which are generally indistinguishable from normal laughing and weeping but the patients are unable to report any emotional experience during these episodes. These patients typically have some degree of voluntary facial paralysis.  'It seems that the involuntary expression stem from an inability to voluntarily inhibit these motor release phenomena through normal cortical influences'

Evolutionary influences ( for a cortical & subcortical pathway for emotional expression).

Relationship between emotional expression & experience of man & drive related ,stereotypical communication patterns of non human animals.

Simple reflex circuits

In simple organisms drive-related responses are handled by simple reflex circuits which remain when more sophisticated systems evolve. p66 ' They are well suited for managing behaviours that are directly and immediately in the service of basic drives ' but they lack plasticity. For examples in humans simple reflexes are retained as brainstem circuits regulating hear rate, respiration & arousal.  Gradually from fish-birds-man a motor system has evolved to the basal ganglia & cerebral cortex of man.  Although all mammals have a cerebral cortex , its destruction eliminates only certain discrete motor functions whilst destruction of the basal ganglia has a significant effect.

More complex behaviour and the cerebral cortex

p66 ' cerebral cortex allows learning  to influence motor behaviour in a substantial way '  " the cortical (i.e. pyramidal system ) in humans is extremely plastic and versatile and is capable of executing fine, highly complex, highly controlled movements'

Frontal lobes highly developed in man, even in higher primates they constitute a strinkingly smaller proportion of the brain.  Destruction of the cortex in humans  has a hugely disparate effect when compared to non humans eg decorticate cats & dogs can perform most motor behaviours.

Discriminating cortical from subcortical features of behavior.

in practice the distinction is difficult to make especially using observational techniques and therefore it is important that the subject is unaware or not paying attention to the fact that their facial expression is being observed.

Take language as an example of a 'cortical' behavior.  Not present in infancy, needs to be learnt , highly flexible and readily changeable, may also show cultural variability.  Typically we have conscious awareness and they can easily be produced or inhibited on command.

Display rules ( Ekman & Friesen(1975) are another example of cortically managed behavior.  p68 ' The social regulation of the face does not appear to begin until late infancy and is not well developed until at least middle childhood, when frontal lobe development is complete.  Learning through reinforcement, punishment, and modeling, clearly plays a prominent role in its development ( Ekman & Oster, 1979).  Ekman (1972), Friesen (1972) have demonstrated cultural variability in display rules.  Learning through modelling ( Ekman & Oster, 1979).

'As with most cortically mediated events, normal adults have relatively good awareness of what their faces are doing when they implement display rules.

In contrast genuine emotional movements are present from early infancy, ie muscle contractions of distress and disgust present at birth, social smiles emerge from 4 weeks, interest at 3 weeks, anger & contempt at 6 months, meaningful surprise & fear in year 2.

also

p. 69 anecephalic infants (no cortex, basal ganglia, or other structures higher than the midbrain) show at least some normal facial expressions'

'The observation that congenitally blind children display a full range of spontaeous expressions demonstrates that learning through imitation is not required ( Freedman, 1964; Gooenough, 1932; Thompson, 1941)

'Timing and coordination of the various regions of the face are usually conspicuously off in posed expressions ( Ekman & Friesen, 1975) of the basic emotions'. We have difficulty with voluntarily inhibiting genuine expressions of basic emotion.

Upper face expressive behaviors

note it is important to study the upper face independently of lower face when studying facial expression.  Some idea of why that might be is available in this section.

3 expressive behaviors of the upper face are distinctive when compared to the lower face.

1.  knit brow:  contraction of corrugator during a particular  kind of mental effort as  observed by Darwin 1872/1965. p221 ' a man may be absorbed in the deepest thought and his brow will remain smooth until he encounters some obstacle in his train of reasoning or is interrupted by some disturbance, and then a frown passes like a shadow over his brow'

evidence for spontaneous - Oster(1978) infants of 3 weeks usually preceeding a smile as if trying to make sense of something new, adults are commonly unaware, Darwin commented on its universality in all cultures (KRO only the Darwin evidence cuts the mustard for me).

2.  raised brow ( check back to Chris Frith who implied this indicated trust) tendency to contract frontalis during attentive listening.  Most people can produce this voluntary but at the same time most are unaware of doing so in everyday contexts.

There is a question as to whether this is a vestige of ear perking movements used by some lower mammals.  ie in lower mammals, including chimps, frontalis is continuous with the muscles that move the ears.  With the development of the frontal lobes in man , the front part of the skull is pushed forward and the frontalis has become separated from the auricular muscles. 

3.  punctuation movements ( Ekman & Friesen) usually very brief ( 50ms) contractions of the muscles of the upper face particularly frontalis.

Although commonly bilateral most show an observable degree of asymmetry usually systematically.  p70 ' although occurring in the context of speech, a manifestly cortical and volitional behavior, theses punctuation movements have features more typical of those mediated by extrapyramidal motor centres.

Most people are unawre of these movement and most can only produce a crude approximation volitionally.

other eveidence for them being emotional expressive:

Rinn observed that they are absent in Parkinsons therefore implies that basal ganglia play a role.  ? KRO any systematic studies of this?

Congenitally blind subjects produce knit brow during puzzlement and raised brows during attentive listening and punctuation movements during speech despite the lack of opportunity to learn these through imitation.

Trigeminal nerve links

sensory fibres of the trigeminal terminate in any of three different sensory nuclei in the brain stem. One of these , the mesencephalic trigeminal nucleus recieves input from muscle stretch and conveys sensation from the facial skin.  It sends output to both the trigeminal and facial motor nucleii and therefore appears to play a key role in reflexes involving the chewing muscles. as well as in reflexes involving the mimetic muscles.

also relevant in this context.  No evidence for muscle spindles in the muscles of facial expression.  Some believe that all muscle is insentient?' although Merton 1972 has a different view.  

Raises the question  p72 ' how is it that we are aware of the patterns of our facial muscles contractions if the muscles are insentient.

Does the trigeminal nerve carry the impulse for awareness? -  the fact that patients with complete trigeminal tractotomy , generally do not describe any loss of facial expression.

Facial Asymmetry & Hemispheric differences in emotion

Tucker, D. M. (1981). Lateral brain function, emotion,

and conceptualization. Psychological Bulletin, 89, 19-46.

for a 1981 review. By asymmetry it is claimed that the left side of the face dominates facial expression , more movement  and more intense.  

Moscovitch, M., & Olds, J. (1982). Asymmetries in spontaneous

facial expressions and their possible relation to

hemispheric specialization. Neuropsychologia, 20, 71-

81.

Most authors therefore claim a right hemisphere specialisation for emotions, and in particular negative emotions.p72  'Although this explanation has gained wide currency, it is not the only one possible.  The chief problem is that it is inconsistent with the widely accepted dictum that only higher cortical functions such as language, praxis or visual-spatial reasoning are lateralised to a hemisphere. ' 'The emotional tone of one's experience or behaviour is generally believed to be determined by subcortical systems .  The main role of higher cortical processes in emotion is in its instrumental regulation - the volitional control of emotions or in the social regulation of the face ( the implementation of display rules etc)'.

Alternative explanations

1.   Apparent right hemisphere lateralization due to left hemisphere superiority for the inhibition of emotion.  Luria (1973) & Vgotsky(1934-1962) have both identified language as a regulator of behaviour and Rinn suggests that the inhibition is greater for negative  than positive emotions.

2.  p73  'If only cortical functions are lateralized, and only volitionally induced expressions are cortical, one might expect only volitionally induced expressions to be lateralized and consequently, asymmetrically displayed on the face'  most studies have used posed expression.  Ekman, Hager & Frieson (1981) have shown that when subjects do not know they are being observed, their spontaneous expressions are essentially symmetrical