Effects of Course Length in a MOOC Dr. José J. Vázquez-‐Cognet & Jason Mock, M.Ed. -‐ April 9, 2014 Presenta(on URL: h0p://go.illinois.edu/MOOCLength 1 Poll: Who Are You? Are you: • Faculty/instructor • Instruc(onal designer or ed tech support? • Administra(on • Other Presenta(on URL: h0p://go.illinois.edu/MOOCLength 2 Poll: Experience with MOOCs? Regarding MOOCs, have you: • Never heard of them • Heard of it, but never taken or developed one • Taken one but not finished • Taken one and finished! • Developed one • Developed more than one Presenta(on URL: h0p://go.illinois.edu/MOOCLength 3 Presenta(on URL: h0p://go.illinois.edu/MOOCLength 4 Who took the MOOC? Presenta(on URL: h0p://go.illinois.edu/MOOCLength 5 127 countries 65,000 students 1 million quizzes 2 million video views Presenta(on URL: h0p://go.illinois.edu/MOOCLength 6 Presenta(on URL: h0p://go.illinois.edu/MOOCLength 7 “ My name is Richard! I am 12 years old, and my dream is to become a successful business entrepreneur and philanthropist in the video game industry. --Richard X ” Presenta(on URL: h0p://go.illinois.edu/MOOCLength 8 MoSvaSon for Research Presenta(on URL: h0p://go.illinois.edu/MOOCLength 9 Study QuesSon: Two Hypotheses Course dura(on does have a significant effect on student performance. Course dura(on does have a significant effect on student moSvaSon. Presenta(on URL: h0p://go.illinois.edu/MOOCLength 10 Study Design – Set up • 3 nearly-‐iden(cal sec(ons • Modified dura(on and deadlines • Common start date • Self-‐selected enrollments Presenta(on URL: h0p://go.illinois.edu/MOOCLength 11 Study Design -‐ Data • Pre-‐course & Post-‐course surveys • Coursera demographic data • All course ac(vity data Presenta(on URL: h0p://go.illinois.edu/MOOCLength 12 Questions 姒儿喵喵 / Flickr Presenta(on URL: h0p://go.illinois.edu/MOOCLength 13 Video Lectures Quizzes Projects Discussion Presenta(on URL: h0p://go.illinois.edu/MOOCLength 14 Video Lectures Quizzes Projects Discussion Presenta(on URL: h0p://go.illinois.edu/MOOCLength 15 Video Lectures Quizzes Projects Discussion Presenta(on URL: h0p://go.illinois.edu/MOOCLength 16 Video Lectures Quizzes Projects Discussion Presenta(on URL: h0p://go.illinois.edu/MOOCLength 17 Results • Note: Denominator in most cases is par(cipants who were ac(ve for >1 day • In some cases, denominator is survey respondents Presenta(on URL: h0p://go.illinois.edu/MOOCLength 18 Demographics: RegistraSons 20000 RegistraSons 15000 10900 10000 Not Ac(ve > 1 day 9161 Ac(ve > 1 day 6297 5000 4575 4327 4 week course 8 week course 7344 0 16 week course Presenta(on URL: h0p://go.illinois.edu/MOOCLength 19 Demographics: AcSve >1 Day 8,000 7344 7,000 RegistraSons 6,000 5,000 4575 4327 4,000 3,000 2,000 1,000 0 4 week course 8 week course 16 week course Total: 16,246 Presenta(on URL: h0p://go.illinois.edu/MOOCLength 20 Demographics: Age-‐Sex 702 South Wright Street 2040 Lincoln Hall Urbana, Illinois 61801 Phone: (217) 333-‐9776 www.atlas.illinois.edu [email protected] Presenta(on URL: h0p://go.illinois.edu/MOOCLength 21 Employment Status Demographics: Employment 16 week course 8 week course 4 week course 9 Unable to work, disabled, or convalescing 8 Re(red 7 A student 6 A homemaker, taking care of a family member, or on maternity leave 5 Not working and not looking for a job 4 Out of work and looking for a job 3 Working at an NGO, volunteer, unpaid intern, or religious worker 2 Self-‐employed/business owner, free-‐lancer, or farmer 1 Employed for salary or wages (either full or part-‐(me) 0% 20% Presenta(on URL: h0p://go.illinois.edu/MOOCLength 40% 22 60% Demographics: EducaSon Level 90% Less than high school 60% High school graduate Some college Bachelor's degree 30% Master's degree Doctoral degree 0% 4 8 Length of course in weeks 16 Total Presenta(on URL: h0p://go.illinois.edu/MOOCLength 23 Demographics: Mean Years of EducaSon 15.80 4 week course 16.13 8 week course 16.22 16 week course 16.08 Total Presenta(on URL: h0p://go.illinois.edu/MOOCLength 24 Questions 姒儿喵喵 / Flickr Presenta(on URL: h0p://go.illinois.edu/MOOCLength 25 Performance: % of People Who Passed the Class 20% 18.3% 18% 17.4% 16% 13.6% 14% 12% 10% 8.6% 8% 6% 4% 2% 0% 4 week course 8 week course 16 week course Total Presenta(on URL: h0p://go.illinois.edu/MOOCLength 26 Performance: % of Students Who Aaempted Quizzes >1 Times 100 80 60 4 week course 8 week course 40 16 week course 20 0 Quiz 1 Quiz 2 Quiz 3 Quiz 4 Quiz 5 Quiz 6 Quiz 7 Quiz 8 Presenta(on URL: h0p://go.illinois.edu/MOOCLength 27 Performance: Mean First Aaempt Quiz Scores 90 60 4 week course 8 week course 16 week course 30 0 Quiz 1 Quiz 2 Quiz 3 Quiz 4 Quiz 5 Quiz 6 Quiz 7 Quiz 8 Presenta(on URL: h0p://go.illinois.edu/MOOCLength 28 Performance: Mean Max Quiz Scores 90 60 4 week course 8 week course 16 week course 30 0 Quiz 1 Quiz 2 Quiz 3 Quiz 4 Quiz 5 Quiz 6 Quiz 7 Quiz 8 Presenta(on URL: h0p://go.illinois.edu/MOOCLength 29 Performance: Mean Score Improvement: First to Highest 90 60 4 week course 8 week course 16 week course 30 0 Quiz 1 Quiz 2 Quiz 3 Quiz 4 Quiz 5 Quiz 6 Quiz 7 Quiz 8 Presenta(on URL: h0p://go.illinois.edu/MOOCLength 30 4-Weeks 6000 # of Users on site 5000 4000 3000 2000 1000 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 Timeline Presenta(on URL: h0p://go.illinois.edu/MOOCLength 31 8-Weeks 6000 # of Users on site 5000 4000 3000 2000 1000 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 Timeline Presenta(on URL: h0p://go.illinois.edu/MOOCLength 32 16-Weeks 9000 8000 # of Users on site 7000 6000 5000 4000 3000 2000 1000 0 1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58 61 64 67 70 73 76 79 82 85 88 91 94 97 100 103 106 109 112 Timeline Presenta(on URL: h0p://go.illinois.edu/MOOCLength 33 8 7 5 4 3 2 1 Number of quizzes with any aaempts 6 Total number of quiz aaempts 700-‐800 600-‐700 500-‐600 400-‐500 300-‐400 200-‐300 100-‐200 0-‐100 0 0 1 2 3 4 5 6 7 Number of modules where any lecture was viewed 8 Presenta(on URL: h0p://go.illinois.edu/MOOCLength 4 week course 34 8 7 5 4 3 2 1 Number of quizzes with any aaempts 6 Total number of quiz aaempts 700-‐800 600-‐700 500-‐600 400-‐500 300-‐400 200-‐300 100-‐200 0-‐100 0 0 1 2 3 4 5 6 7 Number of modules where any lecture was viewed 8 Presenta(on URL: h0p://go.illinois.edu/MOOCLength 8 week course 35 8 7 5 4 3 2 1 Number of quizzes with any aaempts 6 Total number of quiz aaempts 1400-‐1600 1200-‐1400 1000-‐1200 800-‐1000 600-‐800 400-‐600 200-‐400 0-‐200 0 0 1 2 3 4 5 6 7 Number of modules where any lecture was viewed 8 16 week course Presenta(on URL: h0p://go.illinois.edu/MOOCLength 36 100 95 90 85 80 75 70 65 60 55 50 45 40 35 30 25 20 15 10 5 0 1% 6% 11% 16% 21% 26% 31% 36% 41% 46% 51% 56% 61% 66% 71% 76% 81% 86% 91% 96% # Students 4 Week % Quiz Points Earned 91% 81% 71% 61% 51% 41% 31% 21% 11% % Videos Watched 1% Presenta(on URL: h0p://go.illinois.edu/MOOCLength 37 100 95 90 85 80 75 70 65 60 55 50 45 40 35 30 25 20 15 10 5 0 1% 6% 11% 16% 21% 26% 31% 36% 41% 46% 51% 56% 61% 66% 71% 76% 81% 86% 91% 96% # Students 8 Week % Quiz Points Earned 91% 81% 71% 61% 51% 41% 31% 21% 11% % Videos Watched 1% Presenta(on URL: h0p://go.illinois.edu/MOOCLength 38 100 95 90 85 80 75 70 65 60 55 50 45 40 35 30 25 20 15 10 5 0 1% 6% 11% 16% 21% 26% 31% 36% 41% 46% 51% 56% 61% 66% 71% 76% 81% 86% 91% 96% # Students 16 Week % Quiz Points Earned 91% 81% 71% 61% 51% 41% 31% 21% 11% % Videos Watched 1% Presenta(on URL: h0p://go.illinois.edu/MOOCLength 39 97% 93% 89% 85% 81% 77% 73% 69% 65% 61% 57% 53% 49% 45% 41% 37% 29% 25% 21% 17% 13% 9% 5% 1% 33% % Quiz Points Earned 100% 97% 94% 91% 88% 85% 82% 79% 76% 73% 70% 67% 64% 61% 58% 55% 52% 49% 46% 43% 40% 37% 34% 31% 28% 25% 22% 19% 16% 13% 10% 7% 4% 1% # Students 95-‐100 90-‐95 85-‐90 80-‐85 75-‐80 70-‐75 65-‐70 % Videos Watched 60-‐65 55-‐60 50-‐55 45-‐50 40-‐45 35-‐40 30-‐35 25-‐30 20-‐25 15-‐20 10-‐15 5-‐10 0-‐5 Presenta(on URL: h0p://go.illinois.edu/MOOCLength 40 Questions 姒儿喵喵 / Flickr Presenta(on URL: h0p://go.illinois.edu/MOOCLength 41 MoSvaSon: Student Models MoSvaSon: Student Models 4 week course 8 week course 16 week course Total Submi0ed quizzes (with or without other ac(vity) Ac(ve in forum only Lectures and forum, no quizzes Watched lectures only Ac(ve, but no lectures, quizzes or forum ac(vity 0% 30% 60% 90% Presenta(on URL: h0p://go.illinois.edu/MOOCLength 42 MoSvaSon: Forum ParScipaSon 90% 60% 4 week course 8 week course 16 week course 30% Total 0% Not ac(ve in Viewed forum, Low volume High volume forum did not post poster < 2900 poster chars Presenta(on URL: h0p://go.illinois.edu/MOOCLength 43 MoSvaSon: Mean Days AcSve 8.8 10.0 9.1 6.8 Total 16 week course 8 week course 4 week course Presenta(on URL: h0p://go.illinois.edu/MOOCLength 44 MoSvaSon: Mean % of Weeks With AcSvity 62.7 47.6 43.6 29.4 4 week course 8 week course 16 week course Total Presenta(on URL: h0p://go.illinois.edu/MOOCLength 45 MoSvaSon: Mean Total Clicks 393.9 324.5 4 week course 305.4 8 week course 334.4 16 week course Total Presenta(on URL: h0p://go.illinois.edu/MOOCLength 46 MoSvaSon: Mean Clicks Per AcSve Day in the Course 37.9 34.6 32.1 26.9 4 week course 8 week course 16 week course Total Presenta(on URL: h0p://go.illinois.edu/MOOCLength 47 MoSvaSon: Mean # of Quiz Aaempts (>1 Aaempt) 4.0 3.5 3.0 2.5 4 week course 2.0 8 week course 1.5 16 week course 1.0 0.5 0.0 Quiz 1 Quiz 2 Quiz 3 Quiz 4 Quiz 5 Quiz 6 Quiz 7 Quiz 8 Presenta(on URL: h0p://go.illinois.edu/MOOCLength 48 MoSvaSon: Mean % of Videos Watched 90.0 60.0 4 week course 8 week course 30.0 16 week course 0.0 1 2 3 4 5 Module 6 7 8 Presenta(on URL: h0p://go.illinois.edu/MOOCLength 49 MoSvaSon: % of RegistraSons By SecSon And “Jumpers” Percent 43.9 26.1 24.0 5.9 4 Weeks 8 Weeks 16 Weeks Ac(ve in More Than 1 Course Presenta(on URL: h0p://go.illinois.edu/MOOCLength 50 MoSvaSon: % Who Pass the Class 25.0% 19.2% 24.8% 18.8% 14.4% Total: 14.4% 13.0% 8.7% 4 week course 8 week course 16 week course 4 & 8 week 4 & 16 week 8 & 16 week All 3 courses Presenta(on URL: h0p://go.illinois.edu/MOOCLength 51 MoSvaSon: Took Advantage of This Learning Opportunity 90% 60% 4 week course 8 week course 16 week course 30% Total 0% Not at all To a small extent To a moderate extent To a large/ To the fullest great extent extent Presenta(on URL: h0p://go.illinois.edu/MOOCLength 52 Regression Analysis # Modules # Modules # Quizzes % Any All aaempted Lectures % of Quiz Lecture Lectures (w/ score > Variable # Clicks Viewed Points Viewed Viewed 0) (Constant) 899.727 38.731 40.702 3.523 2.806 3.649 4-‐week 100.189 15.959** 27.344** 1.422** .631** 2.361** 8-‐week 202.469** 17.750** 20.079** 1.381** 1.371** 1.655** Age 13.472** .234** -‐.058 .018** .019** -‐.001 Male -‐115.833* 6.087** 4.536* .371* .594** .243 Current Student -‐67.606 -‐5.452* -‐2.565 -‐.337 -‐.491* -‐.139 English Index -‐39.956 -‐.035 .511 .026 -‐.118 .023 New to Subject 152.385* 4.353* -‐1.983 .237 .300 -‐.148 Years of Ed -‐18.645 .126 .429 .012 .012 .031 R Squared 0.043 0.082 0.121 0.089 0.066 0.129 * = significant to < 0.05 ** = significant to < 0.01 Presenta(on URL: h0p://go.illinois.edu/MOOCLength 53 Regression Analysis # Modules # Modules # Quizzes % Any All aaempted Lectures % of Quiz Lecture Lectures (w/ score > Variable # Clicks Viewed Points Viewed Viewed 0) (Constant) 899.727 38.731 40.702 3.523 2.806 3.649 4-‐week 100.189 15.959** 27.344** 1.422** .631** 2.361** 8-‐week 202.469** 17.750** 20.079** 1.381** 1.371** 1.655** Age 13.472** .234** -‐.058 .018** .019** -‐.001 Male -‐115.833* 6.087** 4.536* .371* .594** .243 Current Student -‐67.606 -‐5.452* -‐2.565 -‐.337 -‐.491* -‐.139 English Index -‐39.956 -‐.035 .511 .026 -‐.118 .023 New to Subject 152.385* 4.353* -‐1.983 .237 .300 -‐.148 Years of Ed -‐18.645 .126 .429 .012 .012 .031 R Squared 0.043 0.082 0.121 0.089 0.066 0.129 * = significant to < 0.05 ** = significant to < 0.01 Presenta(on URL: h0p://go.illinois.edu/MOOCLength 54 Regression Analysis # Modules # Modules # Quizzes % Any All aaempted Lectures % of Quiz Lecture Lectures (w/ score > Variable # Clicks Viewed Points Viewed Viewed 0) (Constant) 899.727 38.731 40.702 3.523 2.806 3.649 4-‐week 100.189 15.959** 27.344** 1.422** .631** 2.361** 8-‐week 202.469** 17.750** 20.079** 1.381** 1.371** 1.655** Age 13.472** .234** -‐.058 .018** .019** -‐.001 Male -‐115.833* 6.087** 4.536* .371* .594** .243 Current Student -‐67.606 -‐5.452* -‐2.565 -‐.337 -‐.491* -‐.139 English Index -‐39.956 -‐.035 .511 .026 -‐.118 .023 New to Subject 152.385* 4.353* -‐1.983 .237 .300 -‐.148 Years of Ed -‐18.645 .126 .429 .012 .012 .031 R Squared 0.043 0.082 0.121 0.089 0.066 0.129 * = significant to < 0.05 ** = significant to < 0.01 Presenta(on URL: h0p://go.illinois.edu/MOOCLength 55 Regression Analysis # Modules # Modules # Quizzes % Any All aaempted Lectures % of Quiz Lecture Lectures (w/ score > Variable # Clicks Viewed Points Viewed Viewed 0) (Constant) 899.727 38.731 40.702 3.523 2.806 3.649 4-‐week 100.189 15.959** 27.344** 1.422** .631** 2.361** 8-‐week 202.469** 17.750** 20.079** 1.381** 1.371** 1.655** Age 13.472** .234** -‐.058 .018** .019** -‐.001 Male -‐115.833* 6.087** 4.536* .371* .594** .243 Current Student -‐67.606 -‐5.452* -‐2.565 -‐.337 -‐.491* -‐.139 English Index -‐39.956 -‐.035 .511 .026 -‐.118 .023 New to Subject 152.385* 4.353* -‐1.983 .237 .300 -‐.148 Years of Ed -‐18.645 .126 .429 .012 .012 .031 R Squared 0.043 0.082 0.121 0.089 0.066 0.129 * = significant to < 0.05 ** = significant to < 0.01 Presenta(on URL: h0p://go.illinois.edu/MOOCLength 56 Regression Analysis # Modules # Modules # Quizzes % Any All aaempted Lectures % of Quiz Lecture Lectures (w/ score > Variable # Clicks Viewed Points Viewed Viewed 0) (Constant) 899.727 38.731 40.702 3.523 2.806 3.649 4-‐week 100.189 15.959** 27.344** 1.422** .631** 2.361** 8-‐week 202.469** 17.750** 20.079** 1.381** 1.371** 1.655** Age 13.472** .234** -‐.058 .018** .019** -‐.001 Male -‐115.833* 6.087** 4.536* .371* .594** .243 Current Student -‐67.606 -‐5.452* -‐2.565 -‐.337 -‐.491* -‐.139 English Index -‐39.956 -‐.035 .511 .026 -‐.118 .023 New to Subject 152.385* 4.353* -‐1.983 .237 .300 -‐.148 Years of Ed -‐18.645 .126 .429 .012 .012 .031 R Squared 0.043 0.082 0.121 0.089 0.066 0.129 * = significant to < 0.05 ** = significant to < 0.01 Presenta(on URL: h0p://go.illinois.edu/MOOCLength 57 Regression Analysis # Modules # Modules # Quizzes % Any All aaempted Lectures % of Quiz Lecture Lectures (w/ score > Variable # Clicks Viewed Points Viewed Viewed 0) (Constant) 899.727 38.731 40.702 3.523 2.806 3.649 4-‐week 100.189 15.959** 27.344** 1.422** .631** 2.361** 8-‐week 202.469** 17.750** 20.079** 1.381** 1.371** 1.655** Age 13.472** .234** -‐.058 .018** .019** -‐.001 Male -‐115.833* 6.087** 4.536* .371* .594** .243 Current Student -‐67.606 -‐5.452* -‐2.565 -‐.337 -‐.491* -‐.139 English Index -‐39.956 -‐.035 .511 .026 -‐.118 .023 New to Subject 152.385* 4.353* -‐1.983 .237 .300 -‐.148 Years of Ed -‐18.645 .126 .429 .012 .012 .031 R Squared 0.043 0.082 0.121 0.089 0.066 0.129 * = significant to < 0.05 ** = significant to < 0.01 Presenta(on URL: h0p://go.illinois.edu/MOOCLength 58 Regression Analysis # Modules # Modules # Quizzes % Any All aaempted Lectures % of Quiz Lecture Lectures (w/ score > Variable # Clicks Viewed Points Viewed Viewed 0) (Constant) 899.727 38.731 40.702 3.523 2.806 3.649 4-‐week 100.189 15.959** 27.344** 1.422** .631** 2.361** 8-‐week 202.469** 17.750** 20.079** 1.381** 1.371** 1.655** Age 13.472** .234** -‐.058 .018** .019** -‐.001 Male -‐115.833* 6.087** 4.536* .371* .594** .243 Current Student -‐67.606 -‐5.452* -‐2.565 -‐.337 -‐.491* -‐.139 English Index -‐39.956 -‐.035 .511 .026 -‐.118 .023 New to Subject 152.385* 4.353* -‐1.983 .237 .300 -‐.148 Years of Ed -‐18.645 .126 .429 .012 .012 .031 R Squared 0.043 0.082 0.121 0.089 0.066 0.129 * = significant to < 0.05 ** = significant to < 0.01 Presenta(on URL: h0p://go.illinois.edu/MOOCLength 59 Conclusions Learners in 4-‐ and 8-‐week courses: • Scored be0er • Had more clicks • Watched more videos • A0empted quizzes more oqen • Par(cipated in the forums more oqen • Were more likely to pass the class …even though ~1.5x more registered for 16 wk Presenta(on URL: h0p://go.illinois.edu/MOOCLength 60 Conclusions: Hypotheses Stand? Course dura(on does have a significant effect on student performance. Course dura(on does have a significant effect on student moSvaSon. Presenta(on URL: h0p://go.illinois.edu/MOOCLength 61 Conclusions: ImplicaSons • Course dura(on DOES ma0er—4 or 8 weeks are good op(ons; 16 weeks is not • Facilita(ng change (allowing jumpers) serves those popula(ons well • Explore also tailoring course design around video-‐only learners Presenta(on URL: h0p://go.illinois.edu/MOOCLength 62 Conclusions: Future Avenues • Is 8 weeks “ideal” or did it perform well merely be being the middle op(on? • How does this relate to self-‐paced or “on demand” models? • Why did some learners switch sec(ons? Presenta(on URL: h0p://go.illinois.edu/MOOCLength 63 Questions 姒儿喵喵 / Flickr Presenta(on URL: h0p://go.illinois.edu/MOOCLength 64 h0p://sloanconsor(um.org/conference/2014/et4online/effects-‐course-‐length-‐mooc [email protected] [email protected] Thanks for listening! Dr. José J. Vázquez-‐Cognet & Jason Mock, M.Ed. -‐ April 9, 2014 Special thanks to: Dawn Owens-‐Nicholson, Katherine Woodruff, and Jane Blanken-‐Webb Presenta(on URL: h0p://go.illinois.edu/MOOCLength 65
© Copyright 2024 ExpyDoc
