Over the last couple of years, I’ve been researching how real-time behavioral data, collected unobtrusively through technology, can predict learning outcomes. As part of this line of research, I’ve recently published the paper Predicting course outcomes with digital textbook usage data in The Internet and Higher Education.
The study used data collected from student engagement with digital textbooks in order to predict course grades. Two measures of student engagement with the texts were analyzed: an engagement index that was calculated through a linear combination of the number of pages read, number of times a student opened their textbook, number of days the student used their textbook, time spent reading, number of highlights, number of bookmarks, and number of notes. The second analysis included the individual components of the engagement index.
Major Findings: The engagement index was significantly predictive of final course grades and was a stronger predictor of course outcomes than previous academic achievement. However, time spent reading, one of the variables that make up the engagement index was more strongly predictive of course grades than the entire engagement index.
The effects of course level and instructor were controlled as well as the effects of gender, race/ethnicity, and previous academic achievement. In other words, data collected unobtrusively through the use of digital textbooks can be used to predict student outcomes even when using it for courses with different levels of reading, with textbooks that require varying levels of reading comprehension, with instructors who have different teaching styles, and for subject areas that vary in their technical nature.
Other Findings: Interestingly enough, students did not read a great deal with a median reading time of only 169 minutes over the course of a 16-week semester.
Also, students who were in the top 10th percentile in number of highlights had significantly higher course grades than those in the lower 90th percentile. This along with previous research on highlighting behaviors suggests that highlighting might be used as a proxy for level of reading comprehension.
Conclusion: Students who read more, do better in their courses. While that might be a “no duh” kind of conclusion, what is noteworthy from this research is the knowledge that behavioral academic data collected unobtrusively can predict how well a student will do in a course better than previous academic performance (which is typically the single best predictor of course outcomes).
Commonly, students are categorized as “at risk” based on previous academic performance (using high school GPA or SAT scores, for instance); however, this method of focusing interventions casts an overly broad net and misses students who might be struggling for other reasons. Prediction based on digital textbook analytics can help identify students at risk of poor performance in real time, even before a student submits any gradable material to the faculty member. In the future, digital textbook data can be added to other data sources (like learning analytics from learning and course management systems) in order to provide even more precise prediction of student success and to be better able to target interventions for those students most at need.