The NYTimes reports on a study that finds that online education beats face-to-face, classroom education. Since this story and the associated report risks being cited as support for online initiatives, it is worth taking a look at the underpinning to see what kind of foundation the conclusions rest on.
The report was carried out for the US Department of Education by SRI International, and was conducted as a meta-analysis of existing literature on the research into the effectiveness of online education. The short version of this is that they look at the findings of related research and use stats to pull them together say something useful about a general hypothesis, in this case, the effectiveness of online learning.
Putting aside any concerns about the validity of the approach (which I don’t have specifically, although I am naturally weary of the insights gained from observations buried under two levels of statistics and therefore abstraction…), the quality of the results is surely dependent on the quality of the studies in sampled. Here is some more information on what SRI based their analysis on:
- They conducted a literature search from 1996-2008 and found only five studies “that both compared the learning effectiveness of online and face-to-face instruction for K–12 students and provided sufficient data for inclusion in a meta-analysis.” These five studies were all published in the 2006-2008 timeframe.
- They broadened their search to look at distance education in general, and found a total of 99 papers that included some comparison on face-to-face and online learning. Of these 99 papers, only 9 related to K-12 students, the others dealing with higher education, continuing education, etc.
They conclude that blended learning (a combination of online and face-to-face) provides the best results. However, they also note that their findings do not point to the online nature of learning being more effective.
Despite what appears to be strong support for online learning applications, the studies in this meta-analysis do not demonstrate that online learning is superior as a medium, In many of the studies showing an advantage for online learning, the online and classroom conditions differed in terms of time spent, curriculum and pedagogy. It was the combination of elements in the treatment conditions (which was likely to have included additional learning time and materials as well as additional opportunities for collaboration) that produced the observed learning advantages. At the same time, one should note that online learning is much more conducive to the expansion of learning time than is face-to-face instruction. [Emphasis theirs]
They also note several other weaknesses with the source studies: “[Many] of the studies suffered from weaknesses such as small sample sizes; failure to report retention rates for students in the conditions being contrasted; and, in many cases, potential bias stemming from the authors’ dual roles as experimenters and instructors.”
My take-away from this report is that there is value to looking at traditional face-to-face learning and seeing how best to incorporate online resources and interactions, while at the same time taking this opportunity to re-evaluate the basic course structure, content, planning, and assessment methods.