Personalizing Activation Emails with AI: Workshop Insights

A practical exploration of how AI can transform user reactivation through infinite personalization

Introduction

User activation emails are one of the trickiest challenges in product management. They need to resonate with diverse users at different stages, with different goals, and different levels of engagement. Traditional approaches force us to create one-size-fits-all messages that often feel generic and expire quickly as products evolve.

But what if every user could receive an activation email tailored specifically to their profile, goals, and behavior? This is exactly what we explored in our recent AI feature workshop, where we built a personalized activation email system for a language learning platform.

Why AI in Products is Different from ChatGPT

Before diving into the experiment, it's important to understand how integrating AI into products differs from casual ChatGPT usage.

When you use ChatGPT, you're solving one specific problem with your unique input. But when you integrate AI into products, you're solving the same problem at scale for infinite users with different inputs.

Think of it this way:

ChatGPT UI: You see your message input, but behind the scenes are invisible system instructions providing guardrails and context
AI in products: Your system prompt (instructions) must work consistently across thousands of different user inputs

This means your instructions need to be concrete enough to handle many test cases while maintaining quality and consistency.

The Power of Infinite Personalization

One of AI's most compelling capabilities is infinite personalization—the ability to customize experiences for every individual user, not just the majority.

Traditional product development forces prioritization: "If we implement this feature, how many users will it serve?" This means edge cases and niche needs often go unaddressed.

AI flips this equation. You can now serve the 1%, the 5%, the unique individual—without fragmenting your product or roadmap.

The Experiment: Duolingo-Style Activation Emails

For our workshop, we imagined we were building an activation system for Duolingo users who hadn't returned to the platform in a while.

The Setup

We created a system prompt with clear instructions:

Task: Generate personalized activation emails that are engaging, friendly, and tailored to user profiles
User data inputs: Name, age, learning goals, experience level, and last active timestamp
Output requirements: Subject line, personalized greeting, compelling message, relevant features, call to action

The User Profiles

We tested with 25 diverse user profiles, including:

Ahmed (35-44, wants to improve English for work promotions, intermediate, last active yesterday)
Emma (early 20s, learning German to get a job, beginner)
Dr. Rachel Moore (professional, exploring Japanese culture through language)

Testing Multiple Models

We ran the experiment across 6 different AI models (from Anthropic, OpenAI, Perplexity) to compare outputs—300 total generated emails. Importantly, we hid which model generated which output during evaluation to avoid bias.

What We Discovered

1. Tone Adaptation by Demographics

The AI naturally adjusted its tone based on user age and context. Younger users received more casual, energetic messages with emojis, while professional users like "Dr. Moore" received more formal communications—some models even addressed her as "Dr. Moore" rather than by first name.

2. Context-Aware Messaging

For Ahmed, who was active yesterday, some models correctly said "You were active yesterday, let's pick it up today!" Others mistakenly said "We noticed you haven't been practicing since November"—revealing that LLMs struggle with dates and timestamps without explicit handling.

3. The Hallucination Problem

One email mentioned "our new technical German module"—a feature that doesn't exist. This is critical: AI will confidently make things up if you don't provide specific constraints.

For well-known products like Facebook, models may have embedded knowledge. But for your product, only you know what features exist and how they work. Domain expertise is essential.

4. Formatting Inconsistencies

Some outputs included emojis, others didn't. Some had more structured formatting. Some said "warm regards," others "to your success." These variations show where you need clearer instructions if consistency matters.

The Role of Domain Expertise

This experiment revealed two critical moments where domain expertise makes all the difference:

1. Writing the Instructions

Someone with deep product knowledge can include context like:

"Focus on the 5-minute daily commitment"
"Highlight the streak feature"
"Mention specific course types we offer"

Without this expertise, instructions stay generic and results feel less authentic.

2. Evaluating the Results

AI always returns something—but is it the best thing? Only domain experts can judge:

Does this tone match our brand?
Would this message actually motivate our users?
Are we overpromising on features?

Interestingly, evaluating quality is much easier when you see multiple options side by side rather than judging a single output in isolation.

Key Takeaways for Product Managers

1. AI Enables Previously Impossible Features

Personalizing emails based on free-text user goals (not just predefined options) would be nearly impossible with traditional template systems. AI makes this trivial.

2. Instructions Are Your Product

The system prompt is your product specification. Invest time in making it clear, specific, and comprehensive. Include guardrails for edge cases.

3. Test Across Multiple Models

Don't assume the "best" model is best for your specific task. Frontier models excel at general reasoning but sometimes perform worse at specific tasks than smaller, specialized models.

4. Iterate Based on Real Outputs

Start with your best instructions, generate outputs across diverse test cases, identify patterns in what works and what doesn't, then refine your instructions.

5. Remove Your Bias

Hide model names during evaluation. You likely have preconceptions about which providers are "better" that can cloud your judgment of actual results.

Beyond Chat Interfaces

One final point: AI in products isn't always about chat interfaces. The activation email system we built has no chat component—it's an intelligent backend feature that personalizes content.

Other non-chat AI applications include:

Content moderation for user-generated content
Personalized product descriptions (showing what matters to each shopper)
Smart onboarding flows that adapt to user responses
Automated classification and routing systems

What's Next?

The possibilities for AI-powered personalization extend far beyond activation emails:

Onboarding flows that adapt based on user expertise
Product recommendations explaining why they're relevant to you specifically
Support responses tailored to user history and sentiment
Content that adjusts complexity based on reader background

The key is identifying where personalization creates genuine value—and where your domain expertise can guide AI to deliver results that traditional systems never could.

Want to try this yourself? The full experiment setup will be shared as a sample so you can run your own activation email experiments and see how different instructions and models perform for your use case.