Over the last couple of years, I’ve been researching how real-time behavioral data, collected unobtrusively through technology, can predict learning outcomes. As part of this line of research, I’ve recently published the paper Predicting course outcomes with digital textbook usage data in The Internet and Higher Education.
The study used data collected from student engagement with digital textbooks in order to predict course grades. Two measures of student engagement with the texts were analyzed: an engagement index that was calculated through a linear combination of the number of pages read, number of times a student opened their textbook, number of days the student used their textbook, time spent reading, number of highlights, number of bookmarks, and number of notes. The second analysis included the individual components of the engagement index.
Major Findings: The engagement index was significantly predictive of final course grades and was a stronger predictor of course outcomes than previous academic achievement. However, time spent reading, one of the variables that make up the engagement index was more strongly predictive of course grades than the entire engagement index.
The effects of course level and instructor were controlled as well as the effects of gender, race/ethnicity, and previous academic achievement. In other words, data collected unobtrusively through the use of digital textbooks can be used to predict student outcomes even when using it for courses with different levels of reading, with textbooks that require varying levels of reading comprehension, with instructors who have different teaching styles, and for subject areas that vary in their technical nature.
Other Findings: Interestingly enough, students did not read a great deal with a median reading time of only 169 minutes over the course of a 16-week semester.
Also, students who were in the top 10th percentile in number of highlights had significantly higher course grades than those in the lower 90th percentile. This along with previous research on highlighting behaviors suggests that highlighting might be used as a proxy for level of reading comprehension.
Conclusion: Students who read more, do better in their courses. While that might be a “no duh” kind of conclusion, what is noteworthy from this research is the knowledge that behavioral academic data collected unobtrusively can predict how well a student will do in a course better than previous academic performance (which is typically the single best predictor of course outcomes).
Commonly, students are categorized as “at risk” based on previous academic performance (using high school GPA or SAT scores, for instance); however, this method of focusing interventions casts an overly broad net and misses students who might be struggling for other reasons. Prediction based on digital textbook analytics can help identify students at risk of poor performance in real time, even before a student submits any gradable material to the faculty member. In the future, digital textbook data can be added to other data sources (like learning analytics from learning and course management systems) in order to provide even more precise prediction of student success and to be better able to target interventions for those students most at need.
My newest paper, ￼Student class standing, Facebook use, and academic performance was just published in the Journal of Applied Developmental Psychology.
In previous work, I’ve discovered that social media use is related to a host of academic and psychosocial outcomes. Most notably, there is a relationship between Facebook use and academic performance and Facebook use and student engagement. When looking at time spent on the site, there is a negative relationship between Facebook use and outcome variables; however, when we parse out different ways of using Facebook, then the relationships become more complex. For instance, what students do on Facebook is more positively predictive of academic and engagement outcomes. My previous research has suggested that using Facebook in certain ways might be driving the negative relationship seen between time spent on Facebook and academic performance. Most notably, using Facebook during class or while studying seemed to explain these negative relationships.
In the current study, I surveyed over 1,600 college students and examined the time they spent on Facebook by splitting that time into two categories: 1) Time spent multitasking (i.e., task switching) with Facebook while studying and 2) “Regular” time spent on Facebook. Based on previous research, my hypothesis was that multitasking would drive the negative relationships seen between Facebook use and grades but that “regular” Facebook use would not. I also examined students at different class ranks (freshmen, sophomores, juniors, and seniors) to see if there were any differences that might be attributed to academic maturation.
Here are a few highlights of my findings:
- Seniors spent less time on Facebook than students at other class ranks
- Seniors also spent less time multitasking with Facebook than students at other class ranks
- Regular time spent on Facebook (not multitasking) was negatively related to actual GPA for freshmen but not for students at other class ranks
- Time spent on Facebook multitasking was negatively related to actual GPA for students at all class ranks except for seniors
What does it all mean?
Freshmen need to feel socially integrated into their college or university, for if they don’t, they’re at risk of dropping out. One of the ways that freshmen maintain a connection to previous friends and reach out, engage with, and learn about new friends is through Facebook. Therefore, Facebook plays an important role in helping freshmen adjust to college. However, the ways in which Facebook use are negatively related to grades suggests that freshmen have difficulty regulating their Facebook use in the service of academics. I hypothesize that this isn’t an issue related to Facebook per se, but the relationship between Facebook and grades provides a way of capturing self-regulation skills in freshmen. In other words, the pattern of Facebook use helps us see something about self-regulation we might not otherwise be able to measure. This is also evidenced by how regular use of Facebook for students at other class ranks is not related to academic performance.
Another interesting finding was that seniors did not exhibit a negative relationship between multitasking with Facebook and grades. While this is unexpected given the cognitive science literature on task switching, there have been other studies (including some of my own) that have found that use of certain technologies and use of them in specific ways while engaged in learning tasks do not impact outcomes. This area is ripe for further research and I expect to see more in the coming years elucidating what characteristics of social technologies and of their uses mitigates task-switching detriments in cognitive outcomes.
You can read the full paper here.
I’m looking for Ph.D. students who want to come work with me at Iowa State. If you are interested in social technologies and how they impact youth, please apply! This would be a funded position and you would work in my emerging research group that will be composed of Ph.D. and Masters students (and eventually advanced undergraduate students). I’m looking for students who are passionate about this research area. The ideal student will be creative in thinking about new research studies they could (eventually) run themselves and/or in thinking about how my existing data can be analyzed. Bonus points for coding and/or statistics skills.
I’ve got a number of projects going right now:
1. Apps and Educational Success. This is a grant-funded project being conducted in collaboration with my colleagues at University of Michigan. We are evaluating a number of apps designed to help middle and high schoolers get to college as well as apps developed to support students already in college.
2. Big Data/Predictive Analytics. This is a large-scale project where over 400 students allowed me to monitor everything they did on their computers for a month. I also have personality data on the students, survey data, as well as institutional data. One of the goals is to discover predictive models that can help identify students at risk by only using trace data.
3. A project on online safety that’s currently in the works.
4. A project on digital technologies to improve self-regulation skills that’s also currently in the works.
You can either apply as a Ph.D. student in Higher Education in the School of Education [Application link – Deadline December 1, 2014]. Or you can apply as a Ph.D. student in Human Computer Interaction [Follow The PhD Graduate Program Application Process steps at the bottom of the page – Deadline January 15, 2015]. Let me know if you have any questions!
I’m incredibly excited to announce that I’ve accepted a position as an associate professor in the School of Education at Iowa State University starting this summer. I’ll be teaching and advising students in the Student Affairs graduate program.
Many of you who follow this blog know that I started a new position in the Purdue University Libraries this past year. I have enjoyed my time at Purdue– the Libraries faculty are a dynamic and interesting bunch. My explorations of information literacy from an information science perspective will forever influence my research. Not to mention that Purdue is a great institution (and a really cool college town).
I have often heard from student affairs professionals who want to begin a Ph.D. to focus on social media/emerging technologies but have hesitated because no programs focus on such issues. Now is your chance to come work with me! Not only will I be continuing my research on how new technologies influence student development, but I’m joining an already impressive and vibrant community of scholars at Iowa State. We’ve also got two new assistant professors joining us in the fall whose research focuses greatly on social justice issues.
Drop me a line to learn more or find me at #ACPA14
I’m looking for examples of how you are using social media in your functional areas. Successes and challenges are both welcome! Feel free to post your story in the comments section or send it privately to me via email by clicking here. Please indicate whether you would like to be identified or whether you would like for your contribution to remain anonymous.
Here are some questions to help frame what I’m looking for (note that I’m not looking for you to answer every question – they are just food for thought):
- What did you do? Which social media tool did you use? How did you use it? How did you get students to use it with you? How did you overcome departmental/division/institutional resistance, if any?
- What worked? How did students respond to the intervention? What did you do (if anything) to measure what worked?
- What didn’t work? What were the challenges you faced? Were there challenges you didn’t expect?
- What were the major takeaways?
- What advice would you give to others?
If I use your example and you choose to be identified, you’ll get credit in the chapter where the example appears and I’ll also list you in the acknowledgements.
I am delighted to announce that later this semester, I will be joining the faculty of the Purdue University Libraries as an associate professor. At Purdue, I will focus on emerging technologies in education with a special focus on the first year experience. If you don’t already know about the great work happening in educational technologies at Purdue, I’d recommend checking out their ITaP studio where they’ve developed learning tools that include: an app that integrates with Facebook to increase student engagement, a learning analytics platform, and a badging system. I’ll also continue my current line of research and look forward to the expansive new lines of inquiry I’ll pursue in collaboration with my new Purdue colleagues.
Image credit: martinliao http://www.flickr.com/photos/martinliao/7209746342/