This lets you compare different experiences by showing 50% of users the Control group and 50% the Test group, helping you see which widgets drive better engagement and conversions.
Note: A/B Testing is only for product pages.
💡 Pro Tip: Ensure that any widgets you want to include in your A/B test, whether in the Control or Test group, are enabled in the Shopify Theme Editor.
To create a new A/B test, enter a test name , followed by configuring your Control and Test groups. You can pick between the various widgets and also choose to test without any video widgets.
Note: Control group shows your baseline experience: what shoppers would normally see. The Test group shows a variation, such as a different widget or no video widget at all.
Performance Analytics and Insights
Whatmore lets you metrics side by side helps so you clearly see whether the Test group performs better or worse than the Control group.
Left side – Control Group: This shows the performance of your baseline setup (for example, a Floating Video widget). It’s the reference point used to compare results.
Right side – Test Group: This shows the performance of the variation you’re testing (for example, a Carousel widget).
💡 Pro Tip: Click on View Report to open a detailed breakdown of the metrics explained below, segmented by product. You can also download this data as an Excel file.
Metrics shown for both groups
Total Sales | Total Sessions | Conversion Rate |
The total revenue generated from users who were exposed to that group. | The number of user sessions where shoppers saw that experience. | The percentage of sessions that resulted in a purchase. |
Scroll down to view graphs for each metric, where the Control and Test groups are compared side by side within the same graph.
Floating Videos vs. Carousels: World of Asaya’s A/B Test
World of Asaya compared floating videos (control) with carousels (test) to evaluate engagement and conversion impact.
| Floating Videos | Carousels |
Sessions interacted with videos | 10.7% | 1.5% |
Increase in VCR with each video widget | 59.6% | 195.1% |
Floating videos drove stronger engagement, with 10.7% of sessions interacting with videos, compared to 1.5% for carousels.
On the conversion side, Video Conversion Rate (VCR) increased across both formats when shoppers watched videos. For floating videos, VCR was 59.6% higher when users watched a video, while carousels saw a 195.1% lift in VCR for video watchers.
Takeaway: While floating videos led in engagement, video consumption, across both formats, had a clear positive impact on conversions.