Assessing Language Pragmatics: Who, What and How Peter A. de Villiers Jill G. de Villiers Smith College *supported by NIH grant N01-DC-8-2104 to the University of Massachusetts and Smith College *webpage: www.umass.edu/aae Pragmatics • The functional use of language between speaker(s) and listener(s) for effective communication. • Syntax and semantics are always intertwined with their functional use in communication so cannot be totally separated from pragmatics skills in language acquisition. • Looking at the development or impairment of “communicative competence” (Hymes, 1972). • We will focus on pragmatic language (not gesture or other non-verbal communication) between the ages of 3 and 9. Who needs to be assessed? • Some children show differential impairment of the functional and social-interactional components of language – Children with autistic spectrum disorders – Children with Asperger syndrome – Children with specific Pragmatic Language Impairment (Bishop, 1998) • But most children with substantial language impairment will also show pragmatic problems – Especially if they have both receptive and expressive language impairment (Craig & Evans, 1993) What to assess? Five Components of Pragmatics 1. Conversational skills in social interaction 2. Speech Acts -- doing things with words and utterances 3. Reference and Presuppositions taking into account the perspective or knowledge state of the listener 4. Extended discourse genres: e.g., Narrative, Exposition 5. Styles or “registers”of speaking 1. Conversational Skills • Turn taking – Sensitivity to non-linguistic, paralinguistic, and linguistic cues • Topics – Initiation – Maintenance – Elaboration – Ending – Change Grice’s Conversational Maxims Cooperative Principles • Sincerity – Assumption that speaker means what she says • Relevance – Keeping on topic • Quantity – Providing enough but not too much redundant information for the listener 2. Speech Acts • Doing things with words and utterances – – – – – – – Informing or reporting Requesting information Requesting action (indirect questions) Commanding or demanding Denying statements Rejecting or prohibiting actions Greetings 3. Referential Specificity, Continuity, and Presuppositions • Assessing what your listener knows or needs to know – Shared information from the referential context or world knowledge – Shared information already introduced into the discourse • Coherence and Cohesion in discourse – Linking utterances together thematically, temporally and referentially Conversational Breakdowns and Repairs • Detecting communication breakdown and its source • Contingent queries and requests for clarification • Spontaneous and prompted repairs – – – – Repetition Further specification Elaboration Confirmation 4. Extended Discourse Genres Narratives • Coherence in narratives – Plot and episode structure – Story grammars • Cohesion in narratives – Reference • Introducing, maintaining, and specifying characters – Linking events • Causal links • Temporal links • Mental states and the “Landscape of Consciousness” (Bruner, 1986) 5. Speaking with Style Speech Registers • Tailoring one’s speech to the audience and social situation e.g. politeness, baby talk etc. – – – – Age of listener Social status of listener Formality of the situation Cultural expectations How to Assess Some existing procedures How to assess 1. Conversational Skills a) Language sampling – – – – Closer to authentic communication But needs to be structured to get some features Time consuming and labor intensive Often lack measures of reliability, validity, and norms b) Conversational Analysis – (Craig & Evans, 1993) c) Pragmatic Profiles – (Prutting & Kirchner, 1987). d) Checklists by teachers, clinicians, or parents – Children’s Communication Checklist (Bishop, 1998). b) Conversational Analysis • Craig & Evans (1993) – Analyzed children’s adjacent responses to adult utterances as contingent vs noncontingent – Cohesion ties with prior adult utterance as complete vs incomplete. – Complete ties of various sorts (e.g., lexical, referential (e.g. pronouns), ellipsis, connectives) • SLI children with receptive as well as expressive problems were: – Lacking in strategies for conversational interruption and access – Made fewer connective ties to previous adult utterance – Made more incomplete ties to adult’s utterances c) Pragmatic Profile (Prutting & Kirchner, 1987) • Age 5 and up. • Based on 15 minute spontaneous, unstructured conversation sample with a familiar partner. • 30 pragmatic features rated as appropriate (facilitate or neutral with respect to communication) or inappropriate (get in the way of communicative exchange) [or no opportunity to observe] • 18 Verbal aspects of communication: – Speech acts (pairs and variety) – Topic (selection, introduction, maintenance, change) – Turn taking (initiation, response, repair, adjacency, contingency, quantity) – Lexical selection (specificity/accuracy, cohesion) – Stylistic variation • 5 Paralinguistic aspects (intelligibility and prosodics) • 7 Nonverbal aspects (e.g, physical proximity, gestures, eyegaze) Pragmatic Profile (Prutting & Kirchner, 1987) • Compared 42 typically developing with 42 language disordered and 42 articulation disordered children between the ages of 7 and 10. • Language disordered children were most likely to be flagged as inappropriate in turn-taking repair, turntaking quantity, specificity/accuracy of lexical selection, and cohesion ties to the prior utterance. • Articulation disordered children did not show these problems but were more likely to be flagged for paralinguistic aspects. d)Children’s Conversational Checklist (Bishop,1998) • 70 items scored as “does not apply” (0), “applies somewhat” (1), and “definitely applies” (2). • Some items scored as positive (+), most as negative (-). • Items fall into 9 categories: – A. Speech intelligibility and fluency – B. Syntax – C. Inappropriate initiation – D. Coherence – E. Stereotyped conversation – F. Use of conversational context – G. Conversational rapport – H. Social relationships – I. Interests • So 5 categories involve Pragmatic Language (38 items) Children’s Conversational Checklist Some example items: • C. Inappropriate initiation • Talks too much • Keeps telling people things they know already • D. Coherence • Has difficulty telling a story, or describing what he has done, in an orderly sequence of events • Uses terms like “he” and “it” without making clear what he is talking about Children’s Conversational Checklist (cont) • E. Stereotyped conversation • Will suddenly change the topic of conversation • Has favorite phrases and sentences which he will use a great deal, sometimes inappropriately • F. Use of conversational context • Tends to be over literal (e.g., “watch your hands”) • May say things which are tactless or socially inappropriate • G. Conversational rapport • Ignores conversational overtures from others • Seldom or never looks at the person he is talking to Children’s Conversational Checklist • Fairly good clinician inter-rater reliability and internal consistency -- in the .80 range. • Pragmatic composite score (the 38 items) discriminates significantly between autistic, pragmatic language impaired (without autism), other SLI, and typically developing children. • Low correlations between teacher/clinician ratings and parent ratings (+.48 on pragmatic composite score) Summary of Conversational Assessment • The pragmatic problems of autistic spectrum and pragmatic language impaired children seem to lie fundamentally in the Gricean conversational principles that depend on mutuality of communication (and theory of mind) -- turn taking, topic and referential continuity and relevance, assessing listeners state of knowledge or ignorance etc. • The pragmatic problems of SLI children without any socialcognitive impairments seem to lie at the interface of syntax, semantics and pragmatics. Limitations in syntactic and lexical understanding and expression impair the linguistic aspects of conversational coherence and cohesion, referential specificity, and repair strategies. How to Assess a) 2) Speech Acts Language Sample Analysis – Structuring the interaction to get the child to produce different speech acts -- especially requests (Roth & Spekman, 1984) – e.g., where child has to ask a “shy puppet” to do things – e.g., situations where the child has to solve a problem: • Pencils with broken points, puzzle with a piece missing, paints without brushes etc. b) Controlled Elicited Production Procedures – Question Asking (DELV-CR, 2003; de Villiers, 2004) – asking the right Wh-question to find out specific information – Communicative Role Taking (DELV-CR; de Villiers, 2004; TOPL, 1992) – understanding the communicative perspective of others and knowing what speech acts they are producing. Key Features of Elicited Production Materials and Procedures • They provide referential support and pragmatic motivation for the language forms and functions to be produced, so they increase the likelihood that those forms and functions will be sampled in the assessment. • They constrain the range of appropriate utterances, so they are more easily scored than a more open-ended language sample. • They retain a considerable degree of communicative naturalness in the elicitation procedure rather than resorting to modeling and imitation to elicit these forms. The Subject Sample • Data will be shown from 1014 four to nine year olds, most of them from working class backgrounds and from all regions of the USA (taken from the DELV-CR field testing). • There were 217 four-year-olds, 266 five-year-olds, 300 sixyear-olds, 56 seven-year-olds, 101 eight-year-olds and 74 nine-year-olds. • Approximately 60% of the children were characterized by the testing clinicians as speakers of African American English (AAE), the other 40% as speakers of Mainstream American English (MAE). • Approximately 1/3rd of the children at each age and in each dialect group were identified by the participating clinicians and schools as having a specific language impairment and were receiving language services. Question Asking • The child is shown a picture with something missing from it. • They have to ask the right question to find out what the event is about. • The missing elements of the pictures include objects, people, locations, tools, and causes of emotions -- so what, who, where, how, and why questions are motivated. • Different levels of prompting are given for each trial if the child does not spontaneously ask an appropriate question -varying from the semantic domain of the question to ask, to the specific wh-word to begin the question with. • If the child asks an appropriate question they are shown the complete picture. The girl is painting something. Ask me the right question and I’ll show you the answer. QuickTime™ and a Photo - JPEG decompressor are needed to see this picture. c. The Psychological Corporation What? QuickTime™ and a Photo - JPEG decompressor are needed to see this picture. c. The Psychological Corporation who where why how what who where why who eats what 5 year old TypicalAAE WHAT IS THE NURSE FEEDING? WHERE DID SHE GO SWIMMING? WHAT IS THE GIRL MAD ABOUT? HOW IS THE GIRL FIXIN' THAT? WHAT IS THE WOMAN EATING? WHO IS RIDING THE BIKE? WHERE IS THAT BOY GOING? WHAT HAPPENED? WHAT IS THEY EATING? 6 year old TypicalAAE who WHO IS THE NURSE FEEDING? where WHERE DID THE GIRL SWIM? why WHAT IS THE GIRL MAD FOR? how WHAT IS THE GIRL FIXING? what WHAT IS THE GIRL EATING? who WHO IS RIDING THE BIKE? where WHERE IS THE BOY RUNNING? why WHY IS THE BOY CRYING? who eats what WHAT ARE THE PEOPLE EATING? who where why how what who where why who eats what 8 year-old TypicalAAE WHO IS THE NURSE FEEDING? WHERE DID THE GIRL GO SWIMMING? WHY IS THE GIRL MAD? HOW IS THE GIRL FIXING THE TOY? WHAT IS THE WOMAN EATING? WHO IS RIDING THE BIKE? WHAT IS THE BOY RUNNING TO? WHY IS THE BOY CRYING? WHO IS EATING WHAT FOOD? 5 year old DISAAE NR SHE MAKING A POOL. WHAT THE GIRL SHE'S FIXING HIS BIKE WHAT SOME MEAT WHAT A BOY THE BOY IS RUNNING TO THE ICE CREAM WHAT? NR 6 year old DISAAE WHO IS THAT FEEDING HIM? SHE JUMPED IN THE WATER. SHE MAD AT THE TABLE. SHE IS FIXIN THE TOY. WHO'S EATIN? A BOY RIDIN ON THE BIKE. WHO'S RUNNING? HE DROPPED HIS ICE CREAM. WHO'S EATIN? 8 year old DISAAE WHO IS SHE FEEDING? WHAT SOMETHING SHE SWIM IN? WHO IS SHE MAD AT? WHAT'S SHE HOLDING ON HER HAND? WHAT HER MOM EATING FROM HER TWO FINGERS? SOMETHING RIDING A BICYCLE. WHERE IS HIS HOUSE? WAS HE CRYING? HOW WAS THEY WAS EATING? Wh-Question production in MAE and AAE speaking children following all prompts. Wh-Question Production 9 8 Average Score /9 7 6 5 AAE MAE 4 3 2 1 0 4 5 6 7 Age 8 9 Wh-Question production in typically developing and language impaired children following all prompts. Wh-Question Production 9 8 Average Score /9 7 6 5 IMPAIRED TYPICAL 4 3 2 1 0 4 5 6 7 Age 8 9 Appropriate why-question production in MAE and AAE speaking children following semantic prompt. Appropriate why-question production in typically-developing and language-impaired children following semantic prompt. Communicative Role Taking and Understanding Speech Acts • Children not only need to produce different kind of speech acts at appropriate times (e.g., asking for information, requesting action, rejecting or denying, prohibiting etc.); they also need to understand the circumstances and force of those utterances in other people. • The children were shown pictures in which a person was communicating to another about some object or event that was clearly depicted. They were asked what the characters were telling (reporting an observed event), asking, or saying (prohibiting an action), depending on the scenario. Picture context for “asking” (requesting an object or action). QuickTime™ and a Photo - JPEG decompressor are needed to see this picture. c. The Psychological Corporation Picture context for “asking” (requesting an object or action). QuickTime™ and a Photo - JPEG decompressor are needed to see this picture. c. The Psychological Corporation Communicative Role Taking in two AAE speaking fouryear-olds. TYPICAL AAE TELL HIS BIG SISTER FELLED OFF HER BIKE ASK CAN I PLAY BASEBALL? ASK CAN I HAVE A PIECE OF CAKE? SAY YOU DON'T FEED THE DOG. THAT'S HIS OWN FOOD IMPAIRED AAE TELL HER BLEEDIN' ASK HIM CARRY SOMETHING ASK HER SAY LOOK AT THE CAKE SAY HE FEEDIN THE DOG Development of appropriate speech act production in a communicative role taking context (MAE versus AAE speaking children) Communicative Role Taking Average Score /4 4 3 AAE MAE 2 1 0 4 5 6 7 Age 8 9 Development of appropriate speech act production in a communicative role taking context (typically developing versus language impaired children). Communicative Role Taking 4 Average Score /4 3 IMPAIRED TYPICAL 2 1 0 4 5 6 7 Age 8 9 How to Assess? 3) Referential Communication and Specification • Telling my listener(s) who and what I am referring to. • Several linguistic devices in English serve to identify one object or person out of a possible set. • These include adjectives that specify a distinctive property of the object or person, prepositional phrases that specify their location, and relative clauses that refer to either a distinctive property, location, or action. • Eliciting these in a referential communication task – (P. de Villiers, 1988) Reference Specification -- Testing Procedure • Examiner and child play a referential communication game. • The child sees a picture that the examiner cannot see. S/he has to describe an event that is happening in part of the picture so that the examiner can pick out the person or object involved in the event from a set of similar people or objects. • In some trials there is a distinctive property that distinguishes the referent. In others it is the location or action of the referent that must be mentioned. • In this way the linguistic form that must be produced increases in complexity from adjectives to prepositional phrases and finally to relative clauses. Here are two horses. c. The Psychological Corporation Tell me what is happening in the red box. I need to know which horse it is. c. The Psychological Corporation Here are two policemen. c. The Psychological Corporation Tell me what is happening in the red box. I need to know which policeman it is. c. The Psychological Corporation Here are two boys. c. The Psychological Corporation Tell me what is happening in the red box. I need to know which boy it is. c. The Psychological Corporation Development of reference specification in MAE and AAE speaking children. Production of form and function following all prompts. Development of reference specification in typicallydeveloping and language-impaired children. Spontaneous production of form and function before any prompts. Development of reference specification in typicallydeveloping and language-impaired children. Production of form and function following all prompts. How to Assess? 4) Narration • Narrative Elicitation Techniques: – – – – – Personal narratives -- anecdotes Story retelling Wordless picture books (e.g. the Frog Story) Shorter picture sequences Fantasy, creative stories • Scoring Coherence • Scoring Cohesion • Mental States of the Characters -- the “landscape of consciousness” (Bruner, 1986). Scoring Narrative Coherence • Story Grammars (e.g., Stein & Glenn, 1979; Johnston, 1982) – Setting information • Conventional (“once upon a time”) • Characters, place and time • Purpose and motivation – Episode structure • Initiating events • Actions and behavioral reactions • Outcomes/results – Internal reactions, plans and goals • Emotions – simple (“happy”, “sad”, “angry”, “afraid”) – complex (“worried”, “jealous”, “surprised”, “guilty”) • Desires – From wanting objects and actions to wanting events • Cognitions – Thoughts, beliefs and states of knowledge Scoring Narrative Cohesion • Referential Cohesion – Introducing characters – Maintaining reference to characters (e.g. article and pronoun use) – Contrasting characters (e.g., names, adjectives, relative clauses) • Causal Cohesion • Temporal Cohesion – Sequencers (e.g., then, next) – Foregrounding and backgrounding temporal clauses (e.g., while, when, after) Ideal Properties of Picture Sequence Materials • Two similar same sex protagonists so the child narrator has to introduce them and maintain reference to them while keeping them contrasted for the listener. • A series of events that have to be explicitly related to each other by means of temporal and causal links. • Explicit depiction and implicit involvement of the mental states of the characters so the child is motivated to make reference to the landscape of consciousness of the story. The Candy Stealing Story The Balloon Popping Story Narrative Assessments • Comprehensive analysis of personal narratives (e.g., Bliss, McCabe & Miranda, 1998) or other elicited narratives (e.g., Hedberg & Westby, 1993). • Dynamic assessment of narrative skills based on picture books (Miller, Gillam, & Pena, 2003) • Cohesion and theory of mind in picture sequence narratives (DELV-CR, 2004; P. de Villiers, 2004) A short, wordless picture-sequence narrative to elicit reference specification, temporal cohesion, and mental state references (DELV-CR, 2004) QuickTime™ and a Photo - JPEG decompressor are needed to see this picture. c. The Psychological Corporation Narrative Samples 1 • I want my train. I’m gonna hide the train from him. I’m gonna play out of the toy box. I’m gonna find that train. Bring that train. (C: 4;2) • He was looking for the choo choo train because the other boy was playin’. And then… and then he said, “I want that choo choo train back”, and umm… he put it in his toy box. And then he came back to find it and he looked under the bed and it wasn’t there. (SC: 4;9) Narrative Samples 2 • The big boy came into the little boy’s room and took away the little boy’s train. Then he hid it under the boy’s bed where he couldn’t get it. Then the little boy… when he left… he got out his train and put it in the toy box while the big boy was eating. Then the big boy thought about the train and he went under the bed to go see it but it wasn’t there. (A: 6;4) • The little brother was trying to get his toy from the big brother. And the big brother hiding his toy under the bed. When he is eating his sandwich, the little boy go and get it and put it inside of his toy box. When his big brother walk in, he think about the train and he look under his bed for it. (J: 6;3) Development of reference contrast in narratives (contrasting the two main characters) in typically developing MAE and AAE speaking children. Reference Contrast in Spoken Narrative Proportion of Group 1 0.8 0.6 AAE MAE 0.4 0.2 0 4.5 5.5 6.5 8 Age 10 12 Development of reference contrast in narratives (contrasting the two main characters) in typically developing and language impaired children. Reference Contrast in Spoken Narrative Proportion of the Group 1 0.8 0.6 Impaired Typical 0.4 0.2 0 4.5 5.5 6.5 8 Age 10 12 Development of the expression of temporal links between events in the narratives of typically developing MAE and AAE speaking children. Temporal Links in Spoken Narrative Average Score /2 2 1.5 AAE MAE 1 0.5 0 4.5 5.5 6.5 8 Age 10 12 Development of the expression of temporal links between events in the narratives of typically developing and language impaired children. Temporal Links in Spoken Narrative Average Score /2 2 1.5 Impaired Typical 1 0.5 0 4.5 5.5 6.5 8 Age 10 12 Following their spontaneous narrative the children were asked two followup questions to probe for their theory of mind understanding: -- Tell me again what is happening in this picture (picture 5) --The big boy is looking for the train under the bed. Why is he looking there? QuickTime™ and a Photo - JPEG decompressor are needed to see this picture. c. The Psychological Corporation Development of mental state references to describe the “thought balloon” picture (typically developing MAE versus AAE speaking children) Mental State Descriptions of Picture 5 Average Score /2 2 1.5 AAE MAE 1 0.5 0 4.5 5.5 6.5 8 Age 10 12 Development of mental state references to describe the “thought balloon” picture (typically developing versus language impaired children). Mental State Descriptions of Picture 5 Average Score /2 2 1.5 Impaired Typical 1 0.5 0 4.5 5.5 6.5 8 Age 10 12 Development of “theory of mind” explanations for the character’s mistaken action in the picture narrative (typically developing MAE versus AAE speaking children). Mental State Explanations of Action Average Score /2 2 1.5 AAE MAE 1 0.5 0 4.5 5.5 6.5 8 Age 10 12 Development of “theory of mind” explanations for the character’s mistaken action in the picture narrative (typically developing versus language impaired children). Mental State Explanations of Action Average Score /2 2 1.5 Impaired Typical 1 0.5 0 4.5 5.5 6.5 8 Age 10 12 Development of combined narrative skills in MAE and AAE speaking children aged 4 to 12. Spoken Narrative Score 7 Average Score /7 6 5 4 AAE MAE 3 2 1 0 4.5 5.5 6.5 8 Age 10 12 Overall narrative scores in typically developing and language impaired children aged 4 though 12. Spoken Narrative Score 7 Average Score /7 6 5 4 Impaired Typical 3 2 1 0 4.5 5.5 6.5 8 Age 10 12 Conclusion • Assessment of language pragmatics is both important and doable (though not yet available all in one place). • A core tool-kit would include: – A checklist of key pragmatic features that are both diagnostic of disorder and informative about targets for intervention (and that can be reliably filled out by a professional who has observed the child in conversation with familiar adults and peers). – A conversational protocol to elicit a range of conversational skills from a child in interaction with a familiar adult, and focused categories of analysis for that conversational sample (preferably without having to transcribe it). – A set of controlled elicited production and comprehension procedures to assess mastery of important component communication skills (with adequate reliability, concept validity, and normative information). References for Pragmatic Assessment • • • • • • • • General: Adams, C. (2002). Practitioner review: The assessment of language pragmatics. Journal of Child Psychology and Psychiatry, 43, 973-987. Creaghead, N. (1984). Strategies for evaluating and targeting pragmatic behaviors in young children. Seminars in Speech and Language, 5, 241-252. Roth, F. & Spekman, N, (1984). Assessing the pragmatic abilities of children: Part 1. Organizational framework and assessment parameters. Journal of Speech and Hearing Disorders, 49, 2-11. Roth, F. & Spekman, N, (1984). Assessing the pragmatic abilities of children: Part 2. Guidelines, considerations, and specific evaluation procedures. Journal of Speech and Hearing Disorders, 49, 12-17. Conversational Analysis: Craig, H. & Evans, J. (1993). Pragmatics and SLI: within-group variations in discourse behaviors. Journal of Speech and Hearing Research, 36, 777-789. Ervin-Tripp, S. (2000). Studying conversation: How to get natural peer interaction. In L. Menn & N.B. Ratner (Eds.), Methods for studying language production. Mahwah, NJ: Erlbaum. References for Pragmatic Assessment • • • • • • • • • • Checklists and Profiles: Bishop, D. (1998). Development of the Children’s Communication Checklist (CCC): A method for assessing qualitative aspects of communication impairment in children. Journal of Child Psychology and Psychiatry, 39, 879-891. Bishop, D. & Baird, G. (2001). Parent and teacher report of pragmatic aspects of communication: Use of the Children’s Communication Checklist in a clinical setting. Developmental Medicine and Child Neurology, 43, 809-818. Prutting, C. & Kirchner, D. (1987). A clinical appraisal of the pragmatic aspects of language. Journal of Speech and Language Disorders, 52, 105-119. Standardized Tests of Pragmatics: Adams, C., Cook, R., Crutchley, A., Hesketh, A., & Reeves, D. (2001). Assessment of Comprehension and Expression 6-11 (ACE 6-11). Windsor, UK: NFER-Nelson. de Villiers, P. A. (2004). Assessing pragmatic skills in elicited production. Seminars in Speech and Language, 25, 57-72. Seymour, H., Roeper, T., & de Villiers, J. (2004). The Diagnostic Evaluation of Language Variation (DELV). The Psychological Corporation. Phelps-Terasaki, D. & Phelps-Gunn, T. (1992). Test of Pragmatic Language. The Psychological Corporation. Lloyd, P., Peers, I., & Foster, C. (2001). The Listening Skills Test. The Psychological Corporation. References for Pragmatic Assessment • Narrative: • Bliss, L., McCabe, A., & Miranda, A.E. (1998). Narrative assessment profile: Discourse analysis for school-age children. Journal of Communication Disorders, 31, 347-363. • Hedberg, N. & Westby, C. (1993). Analyzing story-telling skills: Theory to practice. Tucson, AZ: Communication Skill Builders. • Miller, L., Gillam, R., & Pena, E. (2003). Dynamic assessment and intervention: Improving children’s narrative abilities. AGS Publishing. • Seymour, H., Roeper, T., & de Villiers, J. (2004). The Diagnostic Evaluation of Language Variation (DELV). The Psychological Corporation.
© Copyright 2024 ExpyDoc