Articles on: Upsell

A/B Testing FAQs & Troubleshooting

❓ General FAQs

Note: Checkout page upsells are only available for Shopify Plus merchants. Shopify restricts checkout extensions to Plus stores only.

1. Why can’t I start a test?

You need at least two active offers on the same surface (Control + Variant A). If you only have one, create a duplicate offer first.

2. Can I run more than one test at once?

Yes, but only one per surface (e.g., one on Checkout, another on Thank You). Running multiple on the same surface will distort traffic splits.

3. How long should I run a test?

At least 14 days or until traffic reaches statistical significance (roughly 500 views per variant for most stores).

4. What happens when the end date passes?

The test stops automatically; data freezes for review. You can still apply the winner afterward.

5. Do discounts or timers reset between variants?

No — each variant follows its own settings. Make sure urgency timers and discount rules match unless those are the variables you’re testing.

Issue	Likely Cause	Solution
Test not starting	Status left Inactive	Toggle to Active after saving.
Traffic not splitting	Only one offer published	Publish both variants on the same surface.
Winner not showing	Too little data	Extend duration or widen conditions to get more traffic.
Metrics look identical	Test period too short	Keep running until you see at least 200–300 sessions per version.
Unexpected winner	Confounding variable	Verify that only one element differs between variants.

Start simple: Test one small element (e.g., button copy) before moving to complex ones.
Document every hypothesis — so you know what change drove improvement.
Use Analytics after each test to verify revenue lift holds in real traffic.
Archive old tests instead of deleting them; they help build your internal knowledge base.

Once you’ve identified a winner, head to Upsell → Analytics to measure how it improved conversion and revenue over time.

Updated on: 27/11/2025

Was this article helpful?

Thank you!