Research shows academic alerts impact grades, withdrawals

A long-standing academic alert system at Oklahoma State University helps improve students’ grades and lessens withdrawal rates, according to a study published today in the journal Studies in Higher Education.

The study, conducted in the 2021–22 and 2022–23 academic years, compared results for freshman- and sophomore-level courses that were identical except for the use of the alerts—including controlling for a swath of demographic variables, from the percentage of rural students to the percentage involved in Greek life, and other demographic factors in either group. Researchers found that students in the group that received the alerts were 30 percent less likely to withdraw from the course and 4 percent more likely to receive a grade higher than a C than those in the control group.

Controlling for demographic factors meant that “we can compare apples to apples instead of apples to oranges,” said lead author Kwideok Han, a data scientist at OSU. “The finding is very clear.”

The alerts at OSU fall into six categories: informing students that they have never attended class, have stopped attending class, have missed a high number of classes, have turned in low-quality work, have failed to turn in an assignment and finally that they can no longer pass the course. The notices are emailed by instructors to the student in question and their adviser, including information about why the alert was submitted and comments from the instructor. They also advise students to speak with their instructor or adviser to try to resolve the issue.

“Our results underscore that being enrolled in a course where instructors use academic alerts benefits students by proactively enhancing their academic outcomes,” the report’s authors conclude. “This provides valuable insights for administrators seeking to implement evidence-based strategies to improve student support services and foster a positive learning environment.”

Academic alert systems—a term that can refer to a range of messages notifying students of their academic progress—are incredibly common at U.S. universities, although many are more targeted toward at-risk students than OSU’s. A 2012 survey, conducted by the John N. Gardner Institute for Excellence in Undergraduate Education, found that 93 percent of four-year institutions utilize what it referred to as “early alert systems,” or EAS.

There is sparse research into the efficacy of these systems, according to the OSU study, and much of it is anecdotal. Research into nudges—tools similar to academic alerts that are defined as low-cost, low-touch interventions that aim to influence someone to take a specific action—has also been somewhat inconclusive. Recent research from Georgia State University showed that nudges had a significant impact on students’ likelihood to complete pressing, high-importance tasks, like paying their tuition balance, but past research also indicated that they had little impact on grades.

In 2014, Erik Moody, a professor of psychology at Marist College, studied an early alert system at his university; faculty were able to opt in to using the system, which then used an algorithm to ping students who were academically at risk. Moody said that, though the alerts did prove effective, it was hard to get buy-in from faculty at the small liberal arts institution, who are typically working with class sizes small enough that they’re able to speak with students about their progress one on one.

He said the method could be especially effective in large first-year lecture courses at bigger institutions, where professors may not be able to give every freshman the guidance they may need.

“First-semester freshmen don’t know if what they’re doing is sufficient, often. Someone may be cruising along, thinking they’re doing great, and that alert comes and it’s the first thing that comes along and makes them think they might need to be doing something different,” Moody said.

Han said her team’s research is evidence that OSU’s notifications do make a difference, but she noted that there are many aspects of academic alerts worth investigating further. For one, she said, the OSU researchers hope to look into whether the timing of the alerts has an impact on student outcomes; other research, including Moody’s at Marist, has indicated that earlier alerts are likely to make a bigger difference by giving struggling students more time to reverse course.

Other factors could also play a part in how effective academic alerts are—which campus resources the message encourages students to use, for instance, or the language the message employs. For example, Moody’s team found when developing their alerts that students are less receptive to harsh, alarmist messaging.

“The concern was you could cause them to withdraw when, in fact, they could still succeed in the course,” he said. “If you were to use a more subtle and softer tone, the students responded to that more positively.”