How to Measure ChatGPT Discovery for Your Shopify Store

To measure whether customers discover your Shopify store through ChatGPT or another AI assistant, combine two sources: referral data showing recorded visits and a post-purchase survey asking buyers where they first heard about you. Keep the results separate. A customer can remember an AI recommendation even when their eventual visit comes through a search engine or a bookmark.

This guide gives you a short question to use, a reporting template, and a way to decide whether the signal deserves more attention. The numerical example below is illustrative, not an OrderSurvey customer result.

What each source can tell you

Shopify's marketing reports describe recorded referrers and offer attribution models for interpreting customer interactions. A survey adds the buyer's recollection of discovery. Neither gives a complete account of every influence on a purchase.

For example, someone might ask an assistant for a gift idea, remember your brand, then search for it later. Their recorded search visit and their survey answer can both be accurate. Do not overwrite one with the other just to make your reports agree.

Start with one discovery question

Use this wording:

Where did you first hear about our store?

The word first helps distinguish discovery from the reminder that brought someone back to buy. Ask a separate question if your real objective is to understand the final purchase influence.

Start with options that fit your business, such as search, social media, a creator, a friend or family member, a publication, and Other. These are suggested categories, not a universal questionnaire. Keep the wording stable while comparing results across time.

If you do not yet have evidence that AI is a meaningful source, let customers name it in an optional Other follow-up. If mentions recur, you can add AI assistant, such as ChatGPT as an option. Record the change date: explicitly naming a channel can change how people answer, so the periods before and after are not directly equivalent.

See the discovery survey answer-options guide for more ways to organize your channel list.

Use a conditional follow-up for context

When someone selects the AI option, a useful follow-up is:

What were you looking for when you found us?

Keep it optional and allow a short written response. A customer who remembers looking for a compact desk lamp gives you more useful context than a channel label alone. If identifying the assistant matters more, ask which tool they used instead. Avoid turning a short checkout survey into a long interview.

You do not need the customer's entire chat transcript. Ask only for the business context you can use, and avoid inviting them to share personal information from their conversations.

OrderSurvey supports conditional follow-up questions. Configure the follow-up to appear for the relevant answer, then preview both that path and an unrelated answer before enabling it.

Place and verify the survey

For a purchase-discovery question, the Thank you page provides an opportunity to ask immediately after an order. You can also configure the Order status page. Shopify documents how to add app blocks through its checkout and accounts editor.

With OrderSurvey:

1. Create the question, answer choices, and optional follow-up. Choose the intended pages and audience, then enable the survey.
2. Open the setup page in OrderSurvey and follow its link to Shopify's editor.
3. On the Thank you page, add the Post-purchase survey block under OrderSurvey, position it, and save.
4. If you also want the Order status placement, switch to that page, add its block, and save separately.
5. Use a test order that matches the audience and schedule. Confirm the intended question and follow-up appear and that a submitted answer is saved.

Enabling a survey does not add the Shopify blocks automatically. Both the survey settings and page placement need to be configured. Keep your test answers identifiable so they do not enter your customer-discovery analysis.

Our Thank you page setup guide covers the placement in more detail. If you select first-order targeting, also check the documented Order status limitation: OrderSurvey needs the order classification received through the Thank you block. Do not assume a later visit can always supply that information.

Report counts and denominators together

Here is a hypothetical four-week reporting period:

This supports the statement: Six respondents named an AI assistant, representing 5% of completed discovery responses. It does not support the statement that AI caused 5% of total revenue.

If you cannot measure actual displays for the same order cohort, leave that denominator unknown. Do not substitute all orders and label the result a display-to-response rate. Keep each order represented once when combining data from both pages.

Review recorded AI referrals separately for the same dates. Do not add referral-attributed orders and survey mentions together without checking overlap; they can include the same buyers. Avoid assuming that respondents represent all nonrespondents.

Turn the signal into a small experiment

Start by reviewing the follow-up answers. Look for a recurring product question you can answer accurately on your own site, such as fit, dimensions, compatibility, materials, or intended use.

Choose one improvement, record its publication date, and keep the questionnaire stable while watching subsequent cohorts. Examples include clarifying a product's dimensions, answering a common comparison question, or correcting an outdated product page. Track the number of responses as well as the percentage mentioning AI. A percentage can rise simply because fewer people answered.

A before-and-after change is a useful observation, but it is not proof of cause. Promotions, seasonality, product availability, and changes in the mix of customers can move the results too. Avoid large budget shifts based on a handful of mentions.

Start collecting the missing context

OrderSurvey helps you ask buyers about discovery on Shopify's Thank you and Order status pages. It does not promise AI rankings or automatically repair missing referral tracking.

The Free plan includes 100 responses per rolling 30-day period, conditional branching, and targeting rules. Pro is $39 USD per month and adds unlimited responses, CSV export, webhook alerts, and priority support.

Install OrderSurvey, or book personal setup help to choose the question and verify your placements.

Frequently asked questions