During the 2018-19 academic year, Georgia State University took its engagement strategy to the next level. In doing so, they proved that SMS text messaging and Behavioral Intelligence are effective tools for driving student retention and persistence.
Finding the solution would require the university to identify and adopt an approach that would help their students take advantage of campus resources and make required tasks feel less complicated. So, GSU’s Dr. Timothy Renick and Lindsey Fifield teamed up with Mainstay (formerly AdmitHub) and independent researchers Lindsay Page, Hunter Gehlbach, and Jeonghyun Lee to do just that.
Keeping students on their post-enrollment path
After a student is enrolled at a college or university, making sense of the ongoing administrative requirements and forms they need to complete can feel daunting. A single misstep, like missing a deadline or failing to submit their completed FAFSA, can mean losing their spot in a class or missing out on access to critical financial aid.
These outcomes can ultimately put a student’s ability to graduate at risk. What’s more, many students are unaware of campus resources that are intended to help streamline these processes and make their lives easier.
“Pouncing” on another student engagement opportunity
As one of the nation’s most innovative institutions when it comes to student engagement, Georgia State University has continued to refine its strategy over the past several years. Through a series of experiments, the university has identified proven ways to meet their key objectives and drive positive student outcomes.
By the 2018-19 academic year, the GSU team had already increased enrollment by 3.9% and achieved a 21.4% reduction in summer melt. Their significant success scaling admissions and enrollment support to incoming first-year students sparked a hypothesis that they could achieve similar success across their enrolled student population, too.
Pounce, GSU’s behaviorally intelligent chatbot from Mainstay (formerly Admithub), was critical to the university’s admissions and enrollment initiative. So, when the team set out to help enrolled students navigate required (and sometimes complex) administrative processes, they naturally decided to enlist Pounce again. This time, they would extend the bot’s reach and test new strategies to make sure students understood the steps they needed to take and utilized the many campus resources available to them.
A brief history of GSU’s randomized controlled trials (RCT) with Mainstay
In 2017, GSU worked with Lindsay Page of Brown University, Hunter Gehlbach of Johns Hopkins University, and Jeonghyun Lee of Georgia Institute of Technology, as well as Mainstay to conduct a randomized controlled trial (RCT) to evaluate the efficacy of their engagement strategy that focused on opening their lines of communications with committed incoming students.
By implementing IQ — one of the critical components of Behavioral Intelligence — the team had scaled their ability to instantly answer students’ questions around the clock through Pounce. They added the components of EQ and CQ by sending timely reminder “nudges,” delivering relevant information about enrollment tasks, and listening to students’ concerns by collecting key survey data.
Understanding that RCTs are the gold standard for evaluating program effectiveness, the team decided to build on their original research. So, in 2018, GSU again partnered with Mainstay and the research team of Page, Gehlbach, and Lee and embarked on another RCT to measure how a chatbot can impact enrolled student persistence and their ability to thrive on campus.
GSU’s VP of Enrollment Dr. Timothy Renick and Lindsey Fifield, Project Director, oversaw this second RCT throughout the 2018-2019 academic year. The experiment involved roughly half of the GSU undergraduate student body. 6,828 students were targeted with customized, text-based outreach from Pounce. Another similarly sized group of students, who were selected for the control study, did not receive access to Pounce.
The students who received engagement campaigns were supported via text messaging across three important contextual dimensions of college life:
Academics.
Encouraging students to participate in activities that contribute to their academic success, including supplemental instruction and advisor meetings.
Social and careers.
Building awareness of community events and cultivating a sense of belonging through non-classroom activities, such as career services events and a Thanksgiving feast.
Administrative processes.
Raising student awareness about administrative processes, including resolving holds due to an outstanding balance, FAFSA filing, and registration for subsequent semesters.
Messaging activity
Throughout the RCT, Pounce sent nearly a quarter-million messages to targeted students and responded to 5,000 incoming questions. Because Mainstay’s Student Engagement Platform continuously learns from interactions and considers contextual relevance in its conversations, the GSU team was able to deliver even more meaningful messages and support to students as the program went on. As a result, out of the thousands of messages the bot received, only 42 required the attention of GSU staff members. The rest were either handled automatically by Pounce or by the dedicated communications team — freeing up significant time for staff to focus on high-value activities.
Over the course of the year-long RCT period, the average student interacted with Pounce on three separate instances. However, the most engaged students sent nearly 100 messages to Pounce. This underscores recent student engagement trends that point to technology as a judgement-free way for students to ask questions and receive critical information.
Meaningful interactions delivered via SMS text messaging can help students feel more connected to their college community. This is particularly true for traditionally underrepresented students (those from low-income backgrounds, first-generation college-goers, and students of color) who report less of a sense of belonging than continuing-generation students or white students at four-year institutions.
Results: Behavioral Intelligence drives critical action and boosts retention
Behavioral Intelligence makes it possible for institutions to actively listen to students at scale, lead with empathy, and communicate with warmth and inclusivity. GSU’s RCT proves that behaviorally intelligent chatbots effectively deliver the personalized, accessible information students need, exactly when they need it, on the channels they prefer to use.
By embracing an engagement strategy powered by Behavioral Intelligence, GSU combined intelligence, empathy, and contextual relevance to achieve these initial results in their engagement experiment:
“Using conversational AI chatbots to assist students in meeting key deadlines and completing critical tasks helped Georgia State boost rates of FAFSA filing and registration for the subsequent fall semester by 3 percentage points [for a nearly 6% overall increase]. Scaled to the entire Georgia State undergraduate student body, an impact of this magnitude translates to more than 1,300 students taking the next step in their college journey who otherwise might have stopped out or been stuck in place.”
– Higher Ed Dive
Now, for the best news: Among students receiving personalized nudges from Pounce, GSU recorded a 50% decrease in withdrawal rates due to outstanding balances, as well as a 3% increase in re-registrations for the fall semester.
The RCT’s researchers noted that “studies focused on text-based nudging in educational contexts have reached a level of maturity where we can begin to hypothesize about which characteristics of nudges are likely effective in shaping.”
They specifically called out the importance of CQ — a hallmark of Behavioral Intelligence — in their guidance for other institutions looking to test similar approaches: Text-based outreach is most effective when messages are highly relevant, timely, and pertaining to a specific action the student needs to take immediately, such as managing an outstanding tuition bill balance. The researchers also noted that a personalized and targeted approach, such as the one taken by GSU, will be key to improved student engagement outcomes.
At Mainstay, we’re inspired by the GSU team’s continued student engagement successes — from curbing summer melt to enhancing the overall student experience in support of retention and persistence. By incorporating personalized conversations into a multi-faceted system of support across academic, social, and administrative dimensions, they’re setting up students for long-term success — throughout the academic journey and beyond.
To learn more, download the full RCT report, “Conditions under which college students can be responsive to nudging,” today.
