# How to create and run A/B tests to optimize page

## Overview

This guide introduces **Ecombe A/B Testing** and walks you through how to create an A/B test to determine which page version delivers better results. You’ll learn the purpose of A/B testing, how to enable it, and how to interpret test results to optimize your store’s performance effectively.

## Ecombe A/B testing, what is that?

Ecombe A/B testing is a simple but powerful method for comparing two versions of a page. It helps you understand which version performs better in driving user actions such as clicks, add to cart, product views, or conversions. 

In this test, traffic is split between two versions: Version A (the original) and version B (the variation). By measuring user behavior, you can make data-driven decisions to improve your store.

For example, you can test two different “Add to Cart” button colors to see which one receives more clicks. This method reduces guesswork and increases your chances of improving sales and engagement.

## What can Ecombe A/B testing do?

Ecombe A/B Testing allows you to evaluate and compare different page variations to understand which version delivers the best performance. With this feature, you can:

* **Test page performance across different devices** 

  Run separate tests for Desktop, Tablet, and Mobile to to analyze behavior on each device.
* **Measure key performance metrics** 

  Track essential behavioral and conversion metrics, including Add to cart rate; Click rate; Product view rate; Conversion rate.
* **Test by visitor type** 

  Choose which audience segment will participate in the experiment: All visitors, New visitors, Returning visitors. This helps you testing and understanding of the behavior of specific target groups.
* **Control traffic distribution:** 

  Assign a specific percentage of traffic to each variant to test evenly or to prioritize certain versions.
* **Automatically apply the winning variant** 

  When the test concludes, the system can automatically set the best-performing version as the final page design.

## Why A/B testing is important for Shopify merchants

A/B Testing is essential for Shopify merchants who want to optimize store performance without relying on assumptions. Without A/B Testing, your pages are exposed to lost sales, inconsistent user experiences, and guess-based decisions.

With A/B Testing, you gain:

* **Data-driven insights**

  Make confident decisions based on real shopper behavior, not assumptions.
* **Higher conversion rates**

  Identify which layout, product presentation, or call-to-action generates more engagement and sales.
* **Smarter, faster growth**

  Continuously improve your store by testing ideas, discovering what works, and automatically applying winning versions.

A/B Testing empowers Shopify merchants to turn every page into a high-performing, revenue-driving asset.

## Enable Ecombe A/B testing to track page conversion

Ecombe A/B testing uses data from Google Analytics (GA4) to calculate and analyze metrics. You need to enable Ecombe Analytics in order to set up and track page performance. If you have already enabled Ecombe Analytics, you can skip this step. If not, please refer to [this guide](https://help.ecombe.com/editor-and-functions/store-optimization/enable-ecombe-analytics) and continue setting up Analytics.

## How to Set Up an A/B test and View Insights

In this section, you will learn how to create a complete A/B test and view its metrics.

### Set Up a A/B test

To create an A/B test, your page must be published so users can access it and generate measurable data. Make sure the page is published.

#### 1. Create a new test

Go to the page editor where you want to create an A/B test, click **the A/B test Icon** in the Left Sidebar then click **Create a new test**. 

<figure><img src="/files/ygCAPEsxvPH93glA60Ou" alt=""><figcaption></figcaption></figure>

#### 2. Configure the test

Set up the A/B test according to your measurement needs.

<figure><img src="/files/DrRPwd0AXNQyAgZXBaBf" alt=""><figcaption></figcaption></figure>

<table data-header-hidden><thead><tr><th width="52.77978515625"></th><th width="306.0797119140625"></th><th></th></tr></thead><tbody><tr><td>1</td><td><strong>Test title</strong></td><td>Enter a name for the test.</td></tr><tr><td>2</td><td><strong>Test device</strong></td><td>Select the device type you want to test: Desktop, Tablet, or Mobile.</td></tr><tr><td>3</td><td><strong>Measuring type</strong></td><td><p>Choose the action used to determine the performance of versions A and B:</p><ul><li><strong>Add to cart rate:</strong> the percentage of sessions where a product is added to cart.</li><li><strong>Click rate:</strong> the percentage of sessions where visitors click any element on the page.</li><li><strong>Product view rate:</strong> the number of times a specific product is viewed within a selected period.</li><li><strong>Conversion rate:</strong> the percentage of sessions where visitors click a trackable element from the page.</li></ul></td></tr><tr><td>4</td><td><strong>Test visitor type</strong></td><td><p>Select A/B test participants:</p><ul><li><strong>All visitor:</strong> all users</li><li><strong>New visitor:</strong> users visiting your store for the first time </li><li><strong>Returning visitor:</strong> users who have visited your page before.</li></ul></td></tr><tr><td>5</td><td><strong>Set traffic for each variant</strong></td><td>Control how much visitor traffic is sent to each page version during the test.</td></tr><tr><td>6</td><td><strong>Start test immediately</strong></td><td>Start the A/B test immediately after creation, or schedule a start date.</td></tr><tr><td>7</td><td><strong>Auto-end after period</strong></td><td>End the A/B test after the selected duration.</td></tr><tr><td>8</td><td><strong>Auto-apply winner after test ends</strong></td><td>Automatically set the winning version as the main page after the test ends, based on the selected measuring type.</td></tr></tbody></table>

#### 3. Start test

Click **Start test now** if you chose “Start test immediately,” or choose **Create Schedule test** if you selected a scheduled date.

<figure><img src="/files/GvQdVXSPcwyNTqqYku3O" alt=""><figcaption></figcaption></figure>

#### 4. Editing Variant version (B)&#x20;

After creating and starting an A/B test, the system automatically generates two page versions: **Version A (Original)** and **Version B (Variant)**.\
To run a meaningful experiment, you will need to edit **Version B** and adjust its layout or content so that it differs from **Version A**.

1. In the editor, use the toggle at the top to switch between the two versions:&#x20;

* **Version A – Original**&#x20;
* **Version B – Variant**

2. Select **Version B** to begin customizing the variant.

**After making your changes, remember to click&#x20;*****Save*****&#x20;to apply them.**

<figure><img src="/files/cXer91wsytjmpcEYGqB6" alt=""><figcaption></figcaption></figure>

If you need help customizing or updating your theme for this variant, please [contact us](/getting-started/setup-and-quick-start/contact-support.md). Our team is happy to assist with your theme changes or improvements.

### View Insights

After your A/B test starts, you can quickly view A/B test information directly inside the editor.

<figure><img src="/files/rhAQw1tOISIkzqE5jwwp" alt=""><figcaption></figcaption></figure>

To see all detailed information about the A/B test, click “Go to A/B analytics to view full test results” in the editor or go through: **Analytics** > **Manage A/B test** > **Select an A/B test** to view all Insights.

{% stepper %}
{% step %}
**Access Manage A/B test**

Click **Analytics** on the Shopify sidebar > **Manage A/B test**.

<figure><img src="/files/fLbWyHVg88PctbgjWOWR" alt=""><figcaption></figcaption></figure>
{% endstep %}

{% step %}
**Choose the A/B test to view Insights**

Select the A/B test you want to view Insights.

<figure><img src="/files/J0Op3OhvUNvNfb3gSln5" alt=""><figcaption></figcaption></figure>
{% endstep %}
{% endstepper %}

{% hint style="warning" %}
After you run a A/B test, the data will take approximately **24 to 48 hours** for Google to collect.
{% endhint %}

## A/B testing Insights explained

This section helps you understand the data and metrics shown in the A/B test result dashboard. It guides you on how to use the results effectively to decide which Original (A) or Variant (B) is the best.

#### Test Result

<table data-header-hidden><thead><tr><th width="168.78997802734375"></th><th></th></tr></thead><tbody><tr><td><strong>Test status</strong></td><td><p>Displays the status and outcome of the A/B test:</p><ul><li><strong>Scheduled:</strong> The test will begin at the scheduled start time.</li><li><strong>Active:</strong> The test is currently running.</li><li><strong>Complete:</strong> The test has finished.</li></ul></td></tr><tr><td><strong>Test result</strong></td><td>Display results during and after the A/B test.</td></tr></tbody></table>

#### Campaign Information

<table data-header-hidden><thead><tr><th width="169.9822998046875"></th><th></th></tr></thead><tbody><tr><td><strong>Overview</strong></td><td><p></p><ul><li><strong>Goal metric:</strong> The target metric used to determine the better-performing version.</li><li><strong>Device:</strong> The device type used in the test.</li><li><strong>Visitor type:</strong> The visitor segment included in the test.</li></ul></td></tr><tr><td><strong>Runtime</strong></td><td><p></p><ul><li><strong>Start time:</strong> When the test began.</li><li><strong>End time:</strong> When the test ended.</li><li><strong>Time elapsed:</strong> Remaining time until the campaign auto-ends.</li></ul></td></tr><tr><td><strong>Traffic allocation</strong></td><td><p></p><ul><li><strong>Page Traffic Tested:</strong> Total traffic included in the campaign.</li><li><strong>Traffic tested split:</strong> The percentage of traffic allocated to versions A and B.</li></ul></td></tr></tbody></table>

#### Test overview

Provides an overview of the results for both Original version (A) and Variant version (B).

<table data-header-hidden><thead><tr><th width="169.56829833984375"></th><th></th></tr></thead><tbody><tr><td><strong>Win probability</strong></td><td>Win rate of the version</td></tr><tr><td><strong>Add to cart</strong></td><td>Number of add-to-cart actions</td></tr><tr><td><strong>Product views</strong></td><td>Number of product views</td></tr><tr><td><strong>Visitors</strong></td><td>Number of unique visitors</td></tr><tr><td><strong>Sessions</strong></td><td>Number of sessions</td></tr></tbody></table>

In addition to the metrics, Ecombe A/B Test provides charts to help you easily visualize differences between versions A and B.

<figure><img src="/files/lTxeDn7HL12uirnSMsjw" alt=""><figcaption></figcaption></figure>

## Best Practices for A/B testing

#### Step 1: Collect (and analyze) data

The goal of this step is to identify pages with issues such as high bounce rate, drop-off rate, low add to cart, or low conversions. You can use [Ecombe Analytics](/editor-and-functions/store-optimization/analytics.md) to collect data and identify issue.

#### Step 2: Define the direction and goals of the test

For example, increasing website traffic, raising conversion rate, lowering bounce rate, or reducing cart abandonment.

#### Step 3: Form a hypothesis

List ideas and hypotheses for A/B testing. For example: moving a signup box to the top left may increase registrations, changing the button color may increase clicks, etc.

#### Step 4: Determine sample size and test duration

Typically, you can test with at least 500 sessions and 10 target events to make a prediction.

#### Step 5: Create the new version for A/B testing

The new version should change only **one variable** to ensure clear comparison and conclusions about its impact on the goal defined in Step 2.

#### Step 6: Analyze results and conclude

If the new version performs better, apply the change. If not, continue testing further to find the winning version.

## FAQ

**1. What happens when an A/B test ends?**

When you create an A/B testing campaign, the system asks whether you want to “Auto-apply winner after test ends.”

* If you choose **“Yes, set the winner as main page,”** the system automatically applies the winning version as the main page.
* If you choose **“No, keep the original page as main,”** the system allows you to decide manually.

**2. What is the difference between Visitors and Sessions?**

Although both measure traffic, **Sessions** and **Visitors** track completely different things.

* **Visitors:** the number of unique people who visit your page. One person counts as **1 visitor**, even if they visit multiple times.&#x20;

  Example: 1 person visits 3 times → still **1 visitor**.
* **Sessions:** One **visit/session** to your page, starting when the user opens the page and ending when they leave. One visitor can create **multiple sessions**.&#x20;

  Example: 1 person visits in the morning and again in the afternoon → **2 sessions**

{% hint style="info" %}
Have a question? Contact us via **in-app live chat,** we'll reply in few minutes.
{% endhint %}


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://help.ecombe.com/editor-and-functions/store-optimization/ab-testing.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.