Understanding A/B Test Results
📊 Where to View Your Test Results
- Go to Upsell Engine → A/B Testing
- Select the test you want to analyze.
- You’ll see a results summary for each variant, including metrics like:
- Views
- Conversions
- Revenue
- Click-through rate
- Traffic split
- Duration and surface
Each test is displayed with real-time performance data so you can see which offer performs better.
🧩 Understanding the Layout
Your result card shows two test cases —
- Control (A): the original offer
- Variant (B): the modified version you tested
For each version, you’ll see:
Metric | Description |
|---|---|
Views | Number of visitors who saw this version of the offer |
Conversions | How many visitors completed a purchase after seeing it |
Revenue | Total amount of sales generated by that version |
Click-Through Rate | Percentage of customers who clicked on the offer |
Traffic % | Proportion of visitors assigned to this version (default 50%) |
🧮 Example: Reading Your Test Results
In this example:
- Test Case 1 (4% off) — Winner
- Test Case 2 (Control) — Lower performance
Metric | Control (4% off) | Variant (4% off) |
|---|---|---|
Views | 1 | 1 |
Conversions | 0 | 1 |
Revenue | $0.00 | $28.80 |
Click-through Rate | 0 | 1 |
Traffic | 50% | 50% |
Result | ❌ Control | ✅ Winner |
📈 Projected Gain: +$28.80
🕒 Duration: 11/12/2025–11/18/2025
💰 Surface: Order Status Page
This means the Variant generated more conversions and revenue, so it’s marked as the Winner.
🧠 Interpreting “Significant” Tests
You may see status tags like:
- Active → Test is currently running.
- Significant → The difference between A and B is statistically valid.
- Winner Identified → One version has outperformed the other by a measurable margin.
Account Editor uses these signals to help you know when a test has collected enough data to draw conclusions confidently.
🏆 Applying the Winning Variant
When a test reaches significance, you’ll see an Apply button beside the winning version.
- Click Apply to implement the winner.
- A confirmation window will appear showing which version will go live and which will be drafted.
Example:
Apply Winner Screen
Change | Action |
|---|---|
Test Case 2 | Will be moved to Draft |
Test Case 1 | Will be Applied and become the active offer |
You’ll also see a Performance Impact Summary:
- Expected conversion rate improvement
- Projected gain amount
- Confirmation that the test has reached statistical significance
Click Done to finalize. The winning version becomes your new live offer automatically.
💡 When to End a Test
You should end or apply results when:
- The test has run for at least 2 weeks (or reached statistical significance).
- One version shows a clear performance improvement.
- Traffic and conversions have stabilized.
Avoid ending too early — short-term spikes can be misleading.
📋 Post-Test Checklist
✅ Confirm that your test ran long enough to gather meaningful data.
✅ Review all metrics (not just revenue).
✅ Apply the winner if results are consistent.
✅ Archive or duplicate successful tests for future experiments.
🔁 What Happens After You Apply the Winner
- The winning version becomes active and visible to all shoppers.
- The control version moves to draft mode.
- The system automatically updates performance tracking under Analytics → Surface Performance and Top Performing Offers.
This way, your ongoing analytics reflect only the current best-performing version.
💬 Pro Tip
Run one A/B test per surface at a time.
Testing multiple offers on the same surface simultaneously can dilute accuracy and make interpretation harder.
🔎 Next Article → Upsell Analytics Overview
In the next article, we’ll explore the Analytics Dashboard, where you can:
- Monitor revenue lift from upsells and A/B tests
- View customer and surface performance
- Analyze detailed conversion funnels and offer-level data
Updated on: 27/11/2025
Thank you!