Since 2016, Georgia State University (GSU) has consistently evolved their use of conversational AI in the strategy and operations of Student Success Coaching. To benefit other institutions, GSU committed to rigorous research to prove Pounce not only reduces melt, increases enrollment, boosts retention, and reduces DFW rates, but also helps to close equity and achievement gaps.
Part 1: Tackling Summer Melt
In 2015, Scott Burke, AVP for Undergraduate Admissions at GSU, was struggling to solve Summer Melt: one-in-five incoming GSU freshmen was not making it to their first day of class. By all measures, these students were college-ready and indistinguishable from those who did enroll except that various barriers got in their way of enrolling.
Dr. Timothy Renick, former EVP of Enrollment and now Executive Director of GSU’s National Institute for Student Success, posed the question that would change everything: “Are we the problem?”
Renick wondered if the very systems meant to help students matriculate might actually hinder them with a series of complicated steps. Indeed, each student’s journey was so complex that it required a level of personalized guidance that GSU could not provide. Renick believed this might be why so many first-generation and historically resilient students struggled to enroll.
The GSU team was also familiar with recent research by Lindsay Page and Ben Castleman that text message nudges had the potential to impact Summer Melt. However, Burke and Renick estimated that they would need up to a dozen full time staff members to manage the volume of students and text messages they expected from GSU’s incoming students.
That’s when Burke met Drew Magliozzi and Kirk Daulerio of Mainstay (known as AdmitHub at the time) which promised an AI enhanced solution to help solve their Summer Melt issue. Moreover Magliozzi and Daulerio had also partnered with Dr. Lindsay Page and Dr. Hunter Gehlbach to rigorously measure and publish the impact this new technology might have on Summer Melt.
And in April of 2016, GSU launched “Pounce,” an AI-powered assistant named for GSU’s mascot, to deliver personalized guidance to admitted students to help them successfully enroll in college.
Mapping the Journey
Thanks to Renick’s imperative to understand all the barriers keeping students from successfully enrolling, Burke and the GSU team had a detailed list of fourteen key issues — from FAFSA verification to attending orientation to submitting immunization forms. In deep partnership with Mainstay, GSU designed a proactive communication plan around each of 14 student barriers along with a substantial knowledge base of questions and answers.
By aligning messaging to these moments along the student journey, the team hoped to turn barriers into milestones. Conversations were designed to move students to the next step, and in the critical moments when a human needed to get involved, Pounce escalated the issue to a human admissions counselor.
Tackling Summer Melt
Under the supervision of outside researchers — Dr. Page and Dr. Gehlbach — GSU launched Pounce in April of 2016, as a randomized controlled trial (RCT). Known as the gold-standard of research, RCTs involve a rigorous design where students are randomly assigned to a “treatment” or “control” group where all variables are equal except for the intervention being measured: in this case “Pounce” an AI-powered enrollment coach.
Fewer than 10% of the treatment group opted-out of Pounce’s messages, with more than 60% exchanging an average of 60 messages. Less than 2% of the 50,000 messages Pounce received from students required escalation to a human, giving advisors time to focus on the students in-need of critical support.
One such student was Austin Birchil, who exchanged more than 200 messages with Pounce when his financial aid award was substantially lower than expected. Austin was a first-gen college student whose father had recently passed away after a long bout with ALS. Georgia State had been Austin’s dream school for years, and at 1am with that dream seemingly slipping away, Austin turned to Pounce with a series of questions that were quite literally keeping him up.
Thanks to Pounce, Austin’s case was flagged to the financial aid office, which figured out that two digits of his social security number had been flipped. With the issue resolved, Austin continued his journey to Georgia State with Pounce celebrating every milestone along the way. Afterward, Austin was even selected to join a student delegation to meet with Bill Gates, whose interest in GSU’s innovative approach to student success, led him to publish this blog post, which included details about Pounce.
In addition to Austin, the research study published by Dr. Page and Dr. Gehlbach demonstrated a 21% reduction in Summer Melt and a 3.3 percentage point increase in enrollment. Given the scale at GSU, that amounts to nearly 300 students just like Austin who the data suggested would not have enrolled without Pounce’s support.
Beyond the headline impact on Summer Melt, the data also showed that Pounce had moved the needle on every single barrier on the journey to college. Most of all, 94% of all students recommended offering Pounce to the next incoming class!
GSU’s AI Strategy to Reduce Summer Melt
- Define the Problem
‘Summer Melt’ is a critical issue affecting student enrollment. - Map the Student Journey
Carefully outline each step required of a student, from admission to enrollment. - Identify Friction Points
Pinpoint specific barriers where students lose momentum and staff struggle to intervene, such as completing the FAFSA, attending orientation, submitting paperwork, meeting an advisor, selecting courses, and more - Develop a Proactive Communication Plan
Coordinate proactive, personalized conversations designed to turn barriers into milestones, especially for first-gen students without the know-how and support system to navigate college enrollment on their own. - Human + Artificial Intelligence
Ensure AI is trained to coach students to success in the same way humans are taught, and when important issues arise, ensure there is an escalation path to get a person involved. - Measure and Learn
Conduct rigorous research with third-party evaluators to not only assess the overall impact of the intervention but also the intermediate data around each known friction point.
- Define the Problem
