Metrics that Matter: Student Expectations and Effective Educational Practices Surveys for Enhancement Conference Birmingham 4 June 2014 Dr Camille B. Kandiko Howson King’s College London @cbkandiko The Student Experience What do you measure? Why? How? Surface, deep and strategic approaches 2 1990s • Values • Attitudes • Satisfaction Move on to: • Time on Task 3 Contemporary view • Aspects of teaching • The broader student experience • Learners’ lives beyond university • Institutional support 4 Background: Student engagement (US) the time and effort students devote to activities that are empirically linked to desired outcomes of college and what institutions do to induce students to participate in these activities (Kuh, 2009: 683) 5 NSSE Benchmarks • Academic Challenge • Learning with Peers • Experiences with Faculty [Academics] • The Learning Environment • High-Impact Practices 6 Background: Student engagement (UK) The participation of students in quality enhancement and quality assurance processes, resulting in the improvement of their educational experience (QAA Quality Code, Chapter B5) 7 NSS: Purposes 10 years on • Public information for prospective students • QA • QE 8 Student Expectations Quality Assurance Agency (QAA)-funded research project explored the views of students in higher education across the UK in 2012-13, to investigate their perceptions and expectations of the quality of their learning experience and the academic standards of their chosen programmes of study 9 Findings • How students frame higher education • Ideology • Practices • Purpose • Students and their course • Students and the institution 10 11 ED1 12 13 a.not in contact with an actual person, you know, you’re just filling out a survey, that’s really not engaging whatsoever (KCL12) You are not in contact with an actual person, you know, you’re just filling out a survey, that’s really not engaging whatsoever (International politics, female, researchintensive) 14 UK Review of Engagement • How do students understand the individual questions? • What do students mean by their response to survey questions? • How are the questions answered both with and without prompted response categories? • How do students interpret the questions as cohesive benchmarks? • Do students think these are important questions? • Do students have suggestions for changes or additional questions? • How do students respond to similar questions? 15 Methodology: cognitive review • ‘Think-aloud’ method (Willis et al 1999) • Verbal prompts • Scripted probes • Different versions of the survey (question design, format, response categories) 16 Key findings • Questions rigorous and thought they reflected important aspects of their academic student experience • Positive response to the survey 17 Institutions’ feedback to and from students • Students were very positive about institutions exploring the various dimensions of student engagement with different elements of student life • Voiced discontent that institutions did not seem to do much, if anything, with the various forms of feedback students provide 18 External comparisons Marketing 19 Pedagogical enhancement Institutional improvement Metrics Not only what you measure, but also the response to the data 20 Enhancement context • Academic-led • Identify best practice and support local improvements • Management-led • Monitoring and maximising institutional performance and reputation • Metrics used in league tables • Targeting ‘under-performing’ areas • Student-led (?) • Relationship-based 21 Data for enhancement • Who is involved? • What is the process? • Who does the thinking? Analysing? 22 Values What you measure is a reflection of what you value; what you value is reflected in the process 23 Teachable moment Institutions working to make scores go up on surveys is akin to students studying for what is on exams 24 Engage Students! • Challenge students • Support students • Inform students • Seek, ask and report on feedback • Provide opportunities for students • Hold students responsible • Work WITH not FOR students 25 References Kuh, G. D. (2009). What student affairs professionals need to know about student engagement. Journal of College Student Development, 50(6), 683– 706. Quality Assurance Agency (2012). UK Quality Code for Higher Education. Part B: Ensuring and Enhancing Academy. Chapter B5: Student Engagement. Gloucester: QAA. 26 Questions? Dr Camille B. Kandiko Howson King’s College London [email protected] @cbkandiko Thank you! Research Assistant: Dr Matthew Mawer 27