AI Is a Flashlight, Not a Crystal Ball

Jason Fife

May 30, 2025

Not long ago, a school district made headlines for sending students home with a summer reading list—filled with books that didn’t exist.

Parents searched for titles.
Librarians came up empty.

Eventually, it became clear: the list had likely been generated with help from AI. The titles sounded real—convincing, even. But they weren’t. The machine hadn’t lied. It had guessed.

And that’s the trap.

AI is brilliant at scale, speed, and pattern recognition.
But judgment? That’s still our forte.

Personalization, Scale, and One Small but Telling Detail

At Mainstay, like many of our partners, we think constantly about how to connect with our audience in ways that feel personal—even when we’re communicating at scale.

Not long ago, I was preparing outreach to thousands of colleges and universities. One small—but very visible—detail stood out: how to refer to each school in a way that felt familiar. “Elon” instead of “Elon University.” “UCLA” instead of “University of California, Los Angeles.”

Not because that alone makes a message feel 1:1—but because getting it wrong is the kind of thing that breaks the illusion immediately.

So I looked for a better way.

I didn’t want to hand-code a giant lookup table. I didn’t need another vendor list. And I wasn’t about to spend hours writing regex. Instead, I made a custom GPT—trained on a Wikipedia page of commonly used school names. In a few minutes, I had a bot that could help me quickly match well-known institutions to the names people actually use in the real world.

It only covered about 10% of the list. But that 10% gave me something powerful: a base layer of crowd-sourced judgment. These weren’t guesses—they were conventions, already vetted by usage.

From there, I asked ChatGPT to take a shot at generating names for the rest.
And what came back was… imperfect:

“Elon University” became “EU.”
“Michigan State” became “Michigan.”
🥴

Plenty of perfectly good schools got collapsed into cold, two-letter acronyms that looked familiar—but made it obvious I didn’t actually know who I was talking to.

Technically fast. Strategically wrong.
Not because AI failed—because no one told it what mattered yet.

AI Is Great at Scale. We’re Still Better at Judgment.

That’s where the real lesson comes in. I wasn’t expecting perfection—I was using AI to get closer, faster, so I could focus my attention where it mattered.

I started layering in structure:

  • Drop terms like “Community College,” “College,” and “University” when they appear at the end of the name and the institution remains uniquely identifiable without them

  • Block acronyms that are fewer than three characters—they’re too short to reliably convey identity

  • Flag schools with in-state rivals for disambiguation—because Michigan State ≠ Michigan

Each small judgment I applied made the output better. But more importantly, it made the remaining edge cases easier to see.

AI doesn’t finish the job.
It just shows us where to step next.


That’s exactly what it’s for.

AI doesn't finish the job. It shows us where to step next.

That’s How Our Partners Use Mainstay, Too

This isn’t just a personal productivity hack. It reflects the exact way our partners use Mainstay’s AI to drive incredible outcomes.

Georgia State University didn’t become a national model for success by launching a bot. They chipped away at friction—one barrier, one behavior, one opportunity at a time.

Their focus was on the small moments that quietly derail a student’s progress:

  • completing the FAFSA on time,
  • clearing verification,
  • showing up for advising,
  • registering for classes,
  • attending career and internship fairs (to name a few).

Left unaddressed, any one of these can become the reason a student stops out. But with the right support, each becomes a moment of momentum.

Mainstay helped Georgia State turn those barriers into action—starting with what could be automated.

  • Reminders.
  • Answers to routine questions.
  • Gentle nudges.
  • All the things that don’t require human judgment.

And when something did require more, the system made that clear—pointing the way so people could step in with purpose.

That same model powers the work we do with partners across the country.
We use AI to detect patterns in behavior and communication, act immediately on what can be resolved without delay, and flag the situations that need real-time support. It lets your team focus where their energy moves the needle most.

Because that’s what AI is for: not to replace humans, but to give them more room to do what only they can.

Where AI Shines: Midstream Framing and Momentum

There’s another place AI shines—not at the start or finish, but midstream, when you’re already in motion and need a second brain.

When I was working through the Michigan vs. Michigan State issue, I described the dilemma to ChatGPT. I didn’t need an answer. I needed a lens.

What it gave me was:
“Schools with in-state rivals need disambiguation.”

Not a fact—just a frame. And it made the next step obvious.

That’s how we think about student support, too. AI doesn’t just give answers—it reveals patterns, prompts reflection, and helps educators learn about their students at scale. It’s not “done for you”—it’s “focused for you.”

We use AI that way every day. Not just for nudges and reminders, but to help students explore their interests, reflect on how they’re feeling, and start setting their own goals. It’s like putting a coach on their phone—no judgment, just momentum.

And the best part? Every one of those conversations generates insight.

So the AI doesn’t just help students move forward—it helps you understand who needs what, and when.
It shines a light where your team can make the biggest difference.
That’s not just scale—it’s precision.

Use AI Where It’s Useful

We don’t need AI to be perfect.
We need it to be useful.

Useful means surfacing patterns, speeding up repetition, and making invisible friction visible. But it also means knowing where human insight still matters most.

Let AI handle the scale. Let people make the calls.
That’s not a limitation—it’s the opportunity.

It’s how we help our partners do more of what only they can do:
connect, support, and remove barriers—one insight at a time.

Want to light the path forward for you and your team?

Share on social

Facebook
Twitter
LinkedIn

Subscribe for free

Sign up to receive Mainstay's latest insights on AI and Student Success — right in your inbox.

Book a call with us:

Get a guided tour of Mainstay’s Engagement Platform

Mainstay’s Engagement Platform uses Behavioral Intelligence to help businesses build meaningful relationships, optimize insights, and drive action.

Sign up for your personalized demo to find out how Mainstay makes it easy to spark conversations with your employees and guide them toward positive results.

Channels

Instant support — anytime, anywhere

Mainstay’s behaviorally intelligent chatbots make it easy to engage with employees on the channels they prefer to use:

SMS text messaging
Live chat
Web chat
Social media

Let's talk

Get a guided tour of Mainstay’s Engagement Platform

Mainstay’s Engagement Platform uses Behavioral Intelligence to help schools and businesses build meaningful relationships, optimize insights, and drive action.

Sign up for your personalized demo to find out how Mainstay makes it easy to spark conversations with your students or employees and guide them toward positive results.

Channels

Instant support — anytime, anywhere

Mainstay’s behaviorally intelligent chatbots make it easy to engage with students and employees on the channels they prefer to use:

SMS text messaging
Live chat
Web chat
Social media

Let's talk