Documents

Case Study

Summary

This online course was built within the Open Learning Initiative (OLI) platform. The design process started by articulating student-centered, measurable learning objectives and then developing practice opportunities for all of the component skills students need to achieve those objectives. Many features (for example, targeted feedback, worked examples, and multimedia resources) were included based on the best evidence from learning science. The learning activities were instrumented to collect data on students’ interactions, and these data were continually analyzed to provide meaningful feedback to individual students and the instructor on students’ learning.

Evidence for the Design

The design features of the OLI-Statistics course are based on a large body of past empirical and theoretical work, such as rich and targeted feedback (Hattie & Timperley, 2007), worked examples (add ref), and multimedia learning (Clark & Mayer, 2011), to name a few.

Context of Application

This course has been used in a variety of teaching contexts. An Advanced or AP Statistics version has been used in high schools, while undergraduate and graduate versions have been used in community college and university settings. The course has been used in both online and hybrid formats.

Study Design

Across three studies conducted at Carnegie Mellon University, students enrolled in “Introductory Statistics” were invited to participate in a study. Volunteers were randomly assigned to stay in the traditional course or participate in an adaptive, accelerated hybrid-mode statistics course that:

• used OLI-Statistics for homework,
• met face-to-face half as often each week, and
• met approximately half as many weeks.

Both groups of students took a pre/post test and matched in-class exams.

In another multi-site, randomized, controlled clinical trial, course sections at seven different institutions were randomly assigned to be taught in the traditional mode or in an adaptive, hybrid-mode using the OLI-Statistics course. Again, both groups of students took a pre/post test and matched in-class exams.

Datasets

Data collected included re/post test performance data on the CAOS test, a comprehensive assessment of introductory statistics (Ziffler, Garfield ref), self-reported activity data from students in both groups, online activity log data from students using the OLI Statistics course, and exam scores.

Results

Students using the OLI-Statistics course in adaptive, hybrid mode showed an 18 percent gain in scores from pretest to posttest, whereas students in the traditional course showed a 3 percent gain in pre/posttest scores. (Lovett et al., 2010).

Broader Applications

This approach would arguably be helpful across a variety of other educational contexts – in higher education or even K-12 education – where the online learning environment allows for interactive activities with feedback and data collection of learner interactions.

Lessons and Considerations

Even though the course was well designed from the beginning, the online learning data collected automatically through the system revealed needs for further refinement. For example, some of the skills that were identified as critical to the targeted learning objectives were already well learned and did not require the amount of practice provided. At the same time, other skills that seemed similar in difficulty were not well learned even by the end of the course, requiring more targeted practice to be added.

Find Out More

Bowen, W. G., Chingos, M. M., Lack, K. A., Nygren, T. I. (2014). Interactive learning online at public universities: Evidence from a six-campus randomized trial. Journal of Policy Analysis and Management, 33, 94-111.

Lovett, M. C., Meyer, O., & Thille, C. (2008). Open Learning Initiative: Testing the accelerated learning hypothesis in Statistics. Journal of Interactive Media in Education.

Lovett, M. C., Meyer, O., & Thille, C. (2010). In search of the perfect blend between an instructor and an online course for teaching introductory statistics. Proceedings of the Eighth International Conference on the Teaching of Statistics. Auckland, New Zealand: International Association for Statistical Education.