Michelangleo; Wikimedia commons
More than handwaving
What gesture analysis reveals
about acquisition and bilingualism
Marianne Gullberg
Centre for Languages and Literature &
The Humanities Lab
Lund University
an object of acquisition
© Marianne Gullberg, 2014
(e.g., Efron, 1941; Jungheim, 1991; Mohan & Helmer, 1988; v. Raffler-Engel, 1980; Woll, 2012)
a medium of acquisition and language use
(e.g., Allen, 1995; Gullberg, 1998; 2011; Kelly & Lee, 2012; Kelly et al., 2009; Lazaraton, 2004; McCafferty, 2006; McCafferty & Rosborough,
2013; Nakatsukasa, 2013; Smotrova & Lantolf, 2013; Sueyoshi & Hardison, 2005; Stam et al., 2012; Tellier, 2008)
a window on acquisition and bilingualism
(e.g., Brown & Gullberg, 2008; Choi & Lantolf, 2008; Gullberg, 2006a; 2006b; 2009; 2010; Neguerela et al., 2004; Özyürek, 2002; Stam, 2008)
© Marianne Gullberg, 2014
gestures in L2 acquisition and L2 use
crosslinguistic influence and the nature of L2 meaning
representations
the properties of adult learner varieties - discourse
cohesion
© Marianne Gullberg, 2014
Goldin-Meadow, 2003; Kendon, 1972; 2004; McNeill, 1992; 2005; Volterra et al.,1990
“actions that have the features of manifest deliberate
expressiveness" (Kendon, 2004: 15)
movements related to ongoing talk,
recognized as communicatively relevant
© Marianne Gullberg, 2014
conventionalised
gestures,
'emblems’
non-conventionalised
gestures
'gestures'
'speech-associated'
co-speech
we gesture
– for ourselves
© Marianne Gullberg, 2014
– for others
e.g., Bavelas et al., 2008; Kendon, 1994; 2004; Holler & Beattie, 2003;
Krauss, et al., 1996; McNeill, 1992; 2005; Melinger & Levelt, 2004
linked to speech and language
gestures vehicles of ‘linguistic’ meaning
© Marianne Gullberg, 2014
an ‘integrated mode of expression’
parallel development, break-down, processing semantic, temporal and discursive coordination
the ‘same’ meaning at the same time
what is focused and/or new
Feyereisen, 1999; Gentilucci & Volta, 2008; Kendon, 2004; Kita & Özyürek 2003; Kita et al., 2007; Krauss, et al,. 2000; Levy & McNeill, 1992;
Mayberry et al., 1998; McNeill, 1985;1992; 2005; Özyürek et al. 2007; Seyfeddinipur, 2006; Volterra et al., 2004…
© Marianne Gullberg, 2014
the co-ordination
en daar heeft ze die pingpong bal ingedrukt
and there has she the pingpong ball inpushed
© Marianne Gullberg, 2014
gesture as structured movement:
gesture phases
preparation
stroke
retraction
gestures vehicles of language-specific meaning
© Marianne Gullberg, 2014
speech and gesture express ‘the same’ information,
and spoken languages express different information
->> cross-linguistic differences in gestures
form (separate from function!)
timing
(frequency)
(e.g., Brown & Gullberg, 2008; Duncan, 1996; 2002; 2005; Gullberg et al., 2008; Kita & Özyürek 2003; Kita et al., 2007;
McNeill & Duncan, 2000, Özyürek & Kita, 1999; Özyürek et al., 2008…)
© Marianne Gullberg, 2014
A bimodal perspective on
crosslinguistic influence and meaning
Two assumptions:
new (L2) form onto old (L1) meanings
target-like form = target-like meaning
But ‘errors’ in production and comprehension
L2 meanings are different, but how?
Carroll et al., 2000; Carroll & Lambert, 2003; Coppieters, 1987; Costa, 2005; Elllis, N., 1994; Ervin, 1961; Graham & Belknap, 1986,
Ijaz, 1986; Jiang, 2000; Kellerman, 1995; Malt & Sloman, 2003; Odlin, 2005; Singleton, 1999; Van Hell & Dijkstra, 2002, inter multa alia
caused motion: placement
to put something somewhere
© Marianne Gullberg, 2014
English
French
Dutch
German
leggen
legen
‘lay’
put
mettre
zetten
stellen
‘set/
stand’
(Bowerman et al., 2004; David, 2003; Hickmann, 2006; Kopecka & Narasimhan, 2012;
Kutscher & Schultze-Berndt, 2008; Lemmens, 2006; Pauwels, 2000; van Staden et al., 2006, inter al.)
© Marianne Gullberg, 2014
do placement gestures differ crosslinguistically and
reflect language-specific event representations?
language-neutral
Dutch = French = German (= action)
language-specific
Dutch ≠ French ≠ German
cf., Glenberg & Kaschak, 2002; Pulvermüller, 2005; Zwaan, 2008
© Marianne Gullberg, 2014
Getting people to talk and gesture about the same things
event description tasks
– Describer - watch 1 clip
describe from memory
What did the woman do?
– Drawer - picture of empty room
draw the objects
© Marianne Gullberg, 2014
Coding
© Marianne Gullberg, 2014
Speech
– first spontaneous transitive description of target event
– placement verb
© Marianne Gullberg, 2014
first spontaneous description
(no hesitations, no probes, no elaborations...)
E
F
D
G
she takes the wastepaper basket she puts it to the right of the desk
elle prend la poubelle elle la met à droite du bureau
she takes the wastepaper basket she puts it to the right of the desk
de prullenbak is omgevallen die zet ze rechtop naast het bureau
the wastepaper basket has fallen over this she sets straight next to the desk
zuerst nimmt sie den Mülleimer und stellt 'n rechts neben den Schreibtisch
first she takes the wastepaper basket and stands it to the right next to the desk
Coding
© Marianne Gullberg, 2014
Speech
– first spontaneous transitive description of target event
– placement verb
Gesture (strokes+post-stroke holds)
FORM (no sound):
object info — handshape
(simple) path — spatial excursion, pointing/lax hand
(Kendon, 2004)
TIMING (sound; from gesture to speech):
stroke onset-offset
(interrater reliability scores)
© Marianne Gullberg, 2014
© Marianne Gullberg, 2014
native English
and she puts that in the other rear corner of the desk
in other words the left rear corner
© Marianne Gullberg, 2014
native French
elle met elle met le tout sur le dans le coin […]
‘she puts she puts all in the corner [ …]
© Marianne Gullberg, 2014
native Dutch
dat zet ze helemaal rechts achter op haar bureau
‘that she puts all the way right to the back on her desk’
© Marianne Gullberg, 2014
native German
und uhm rechts oben in die Ecke auf den Schreibtisch gestellt
‘and uh right up in the corner on the desk stood’
Gesture FORM - obj incorporation
("
**
© Marianne Gullberg, 2014
!#'"
!#&"
!#%"
!#$"
!"
)*+"
,-.*/0"
123/0"
4.-56*"
-> Dutch speakers incorporate objects;
others mainly gesture about direction of mvmt
Gullberg, ARCL, 2009; AILE, 2009
Gesture TIMING
("
© Marianne Gullberg, 2014
!#'"
730.-"
!#&"
Loc
!#%"
Verb
Verb
Loc
789"
:;/"
<"
!#$"
!"
)*+"
,-.*/0"
123/0"
4.-56*"
-> Eng and German align with Locatives;
French and Dutch align with Verbs
Gullberg, ARCL, 2009; AILE, 2009
language-specific meanings/event representations
reflected in speech and gesture
© Marianne Gullberg, 2014
English ≠ French ≠ Dutch ≠ German (≠ action)
G-form
G-timing
Path
Loc
move-to-goal
Path
Verb
move
Obj (and path)
Verb
move-object
Path
Loc
move-to-goal
© Marianne Gullberg, 2014
what do adult L2 speakers mean?
crosslinguistic influence?
ze zet de fles op tafel
ze legt de fles op tafel
she sets the bottle on table
she lays the bottle on table
many-to-one
she puts the bottle on the table
elle met la bouteille sur la table
(e.g., Ellis, 1994; Geeslin, 2003; Jiang, 2000; Pavlenko & Driagina, 2007; Schwartz & Sprouse, 1996; Viberg, 1998)
© Marianne Gullberg, 2014
Dutch1 English2
and then she puts it on the right side of the desk
© Marianne Gullberg, 2014
Dutch1 French2
donc elle a placé le bol eh en face
‘so she placed the bowl uh in front’
© Marianne Gullberg, 2014
Dutch1 French2
et elle met le bol au coin arrière droite
‘and she puts the bowl in the corner at the back to the right’
© Marianne Gullberg, 2014
Gesture FORM in L2 -‐ object incorpora7on
("
("
!#'"
!#'"
!#&"
!#&"
**
!#%"
!#%"
!#$"
!#$"
!"
!"
)*+,-("
.$/01"
/01("
)*+,-("
./01,-$"
-> Dutch L2 speakers of English and French
focus on moving objects (as groups);
individuals show shifts to path
Gullberg, AILE, 2009; ARCL, 2009
./01,-("
© Marianne Gullberg, 2014
German1 French 2
elle met sur la table
‘she puts on the table’
Gesture TIMING in L2
("
Proportion of gestures aligned with X
© Marianne Gullberg, 2014
!#'"
Loc
Loc
**
!#&"
Verb
***
!#%"
231*+"
245"
670"
Verb
!#$"
!"
Loc
8"
Verb
)*+,-.("
/+*.01$"
/+*.01("
-> German L2 speakers of French shift focus to verbs/actions
Gullberg, AILE, 2009
© Marianne Gullberg, 2014
L2 speech and L2 gestures
G-form
G-timing
Object (and path)
Verb
move-object
'transfer'
Path
Verb
move
'learning'
Path
Verb
move
Path
Loc
move-to-goal
'work in
progress'
© Marianne Gullberg, 2014
A bimodal perspective on
L2 discourse coherence and learner varieties
the woman – the woman – the woman
the woman – she – Ø
->> over-explicit and ambiguous maintained reference
hyper-clarity? - communicative account
double planning load? - psycholinguistic account
e.g., Carroll & Lambert, 2003; Chini, 2005; Extra et al., 1988; Givón, 1984; Hendriks, 2003;
Klein & Perdue, 1997; Perdue, 2000; Prodeau, 1998; Strömqvist & Day, 1993; Williams, 1988; Yoshioka, 2005
© Marianne Gullberg, 2014
Getting people to talk and gesture about the same things
narrative task
– Narrator - read 1 story
describe from memory
What happened?
Listener retell story in their turn
Coding
© Marianne Gullberg, 2014
Speech
referential expressions of person coded for
discourse status (introduced, maintained, reintroduced)
form (NP, pron)
Gesture (strokes+post-stroke holds)
TIMING (from speech to gesture):
onset stroke to offset
(interrater reliability scores)
© Marianne Gullberg, 2014
native L1 Swedish
uh en tjej i disken
som tar detta då
och Ø tycker att det ser mycke mystiskt ut
varefter hon tar det till typ föreståndarn
som också tycker det ser väldigt mystiskt ut
‘uh a girl at the counter
who takes this then
and Ø thinks it looks very strange
whereupon she takes it to the sort
of manager
who also thinks it looks very
strange’
the manager
Gullberg, 1998, 2003
the girl at the counter
© Marianne Gullberg, 2014
non-native L2 French (Swedish L1)
à la femme dans la réception
‘to the woman in the reception
et la femme comprend pas
et la femme dans la réception
donnE la script de le supervisé de
la pharmacie
and the woman doesn't understand
and the woman in the reception
give the prescription to the supervisor
in the pharmacy’
L2 discourse over-explicit
in speech and gesture
-> bi-modal learner variety
the reception
supervisor
the woman
Gullberg, 1998, 2003
Why?
© Marianne Gullberg, 2014
A communication strategy to disambiguate?
L1DutchL2 French
‘the two dwarfs make
for the third
- dwarf
house the third dwarf
looks at the two’
Gullberg, 2006a, 2008a
© Marianne Gullberg, 2014
No, not exclusively — lex NPs and gestures persist
L1DutchL2 French
‘and the secretary
- his secretary
yes she
- she rewrites the prescription
eh no ehm for the doctor the
precription for the doctor
-yes
ehm the doctor’
Gullberg, 2006a, 2008a
© Marianne Gullberg, 2014
Not fewer gestures, but different
disambiguation
wherever possible
Gullberg, 2006a; 2008a
externalise to reduce
cognitive load?
(Goldin-Meadow et al., 2001; Wagner
et al., 2004; cf. Carroll & Lambert, 2003)
© Marianne Gullberg, 2014
- presence of gestures – developmental aspects,
possibly cognitive load, processing and planning
capacity
- form and articulation of gestures – communicative
concerns for comprehension in real-time interaction
© Marianne Gullberg, 2014
gestures
reveal details about current representations
potentially masked in speech (gradual shifts,
transitions)
shed light on the interplay between communicative
and psycholinguistic factors that shape L2
acquisition and use
bridge the gap between interactional and
psycholinguistic approaches to SLA
Michelangleo; Wikimedia commons
Thanks to
The Max Planck Institute for Psycholinguistics
Nederlandse Organisatie voor
Wetenschappelijk Onderzoek, MPI 56-384
Vetenskapsrådet, A0667401
Melissa Bowerman,
Amanda Brown, Maria Graziano
Henriette Hendriks, Maya Hickmann, Marieke Hoetjes,
Peter Indefrey, Eric Kellerman, Adam Kendon,
Wolfgang Klein, Bhuvana Narasimhan, Clive Perdue
[email protected]
© Copyright 2024 ExpyDoc
