A/B Testing
Optimize your campaigns by testing different versions of your emails.
A/B Testing
A/B testing allows you to compare different versions of your emails to identify what works best.
Info A/B testing is included in the Pro plan.
What is A/B Testing?
A/B testing (or split testing) consists of:
- Creating two versions (A and B) of an email
- Sending each version to a portion of your contacts
- Measuring which version performs better
- Using these learnings for your future campaigns
What you can test
Email subjects
The most impactful element on open rate.
Test examples:
| Version A | Version B |
|---|---|
| Quick question | , quick question |
| Improve your sales | 3 techniques to double your sales |
| Invitation | You're invited |
Email content
Impact on click and response rate.
Test examples:
- Length (short vs detailed)
- Tone (formal vs conversational)
- Structure (paragraphs vs lists)
- CTA (call-to-action)
Sender
The sender name influences opens.
| Version A | Version B |
|---|---|
| John Smith | John from FluenzR |
| FluenzR Marketing | john@fluenzr.co |
Create an A/B test
1. Enable A/B test
In the workflow editor:
- Click on an Email node
- Enable the A/B Test option
- A version B is automatically created
2. Configure versions
Version A:
- Fill in subject A
- Write content A
Version B:
- Click on the "Version B" tab
- Fill in subject B
- Write content B
3. Define distribution
| Parameter | Description | Recommendation |
|---|---|---|
| Distribution | % of contacts for each version | 50/50 |
| Success criterion | Metric to optimize | Open rate |
4. Option: Automatic winner sending
Enable this option to:
- Test on a sample (e.g., 20% of contacts)
- Wait for results (e.g., 24h)
- Automatically send the winning version to the remaining 80%
Comparison metrics
Open rate
Relevant for testing subjects.
| Version | Sent | Opened | Rate |
|---|---|---|---|
| A | 500 | 125 | 25% |
| B | 500 | 175 | 35% |
→ Version B wins (+40% opens)
Click rate
Relevant for testing content and CTAs.
| Version | Opened | Clicks | Rate |
|---|---|---|---|
| A | 125 | 15 | 12% |
| B | 175 | 14 | 8% |
→ Version A wins for click rate
Response rate
The ultimate metric for prospecting.
| Version | Sent | Replies | Rate |
|---|---|---|---|
| A | 500 | 8 | 1.6% |
| B | 500 | 15 | 3% |
→ Version B wins for replies
Analyze results
Statistical significance
A result is reliable if:
- The sample is sufficient (>100 per version)
- The difference is significant (>10% gap)
- The test lasted long enough (>48h)
Warning With small samples, results may be due to chance. Wait for enough data before concluding.
Interpretation
| Gap | Interpretation |
|---|---|
| <5% | No significant difference |
| 5-15% | Trend to confirm |
| >15% | Significant difference |
Best practices
Test only one element
To know what makes the difference, change only one thing:
| Bad | Good |
|---|---|
| Different subject + Different content | Only the subject |
| Everything is different | One variable at a time |
Sufficient sample
| List size | Recommended test sample |
|---|---|
| <500 | Test on 100% (50/50) |
| 500-2000 | Test on 50% |
| >2000 | Test on 20-30% |
Test duration
Wait long enough for reliable results:
| Email type | Minimum duration |
|---|---|
| Prospecting | 48-72h |
| Newsletter | 24-48h |
| Urgent | 12-24h |
Document your tests
Keep a journal of your A/B tests:
- Starting hypothesis
- What was tested
- Quantified results
- Actionable conclusion
Test ideas
Subjects to test
| Element | Example A | Example B |
|---|---|---|
| Personalization | "Question" | ", question" |
| Question vs Statement | "How to increase your sales?" | "Increase your sales by 30%" |
| Length | "Available this week?" | "Are you available for a 15-min call this week?" |
| Emoji | "Quick question" | "Quick question 👋" |
| Urgency | "An idea for you" | "This week only" |
Content to test
| Element | Version A | Version B |
|---|---|---|
| Length | 3 paragraphs | 1 paragraph |
| CTA | "Reply to this email" | "Click here to book a slot" |
| Social proof | Without testimonial | With customer testimonial |
| Format | Full text | Bullet list |
FAQ
Can I do an A/B/C test?
No, FluenzR only supports 2-version tests. To test 3 options, run successive tests.
Does the test work on follow-ups?
Yes, each email in the sequence can have its own A/B test.
How are contacts distributed?
Randomly. Each contact is assigned to a version at send time.
Can I modify a test in progress?
No, modifying a test in progress would skew the results. Create a new campaign if needed.