How to Change Shopify Survey Questions Without Misreading Results

Before changing a Shopify survey, save its current wording, answer choices, audience and conditions. Record when the change goes live, then report the periods separately. Editing the survey does not make the older answers worthless, but the new results may answer a different question.

This matters when you add ChatGPT as a discovery source, split Instagram into several options, or replace a broad question with a more specific one. A larger share for a newly named option can be useful information without proving that the channel suddenly grew.

The workflow below combines a manual change log with OrderSurvey's saved answers. It does not require a new survey every time you correct a typo, and it does not assume the app stores a complete questionnaire version history.

Decide what the change is meant to improve

Write one sentence about the decision you need to make. For example: “We want to distinguish Instagram creator discovery from Instagram advertising because the team reviews those activities separately.”

Then choose the smallest useful change. If the main question already works, an optional follow-up may provide the detail you need while retaining the main categories. See the discovery answer-options guide for wording choices.

Pew Research Center's question-design guidance explains that wording, available choices and question order can influence responses. Its guidance on tracking change emphasizes keeping wording and context consistent. These are reasons to document a questionnaire change, not reasons to keep an unclear question forever.

For example, “Where did you first hear about us?” and “What brought you back today?” serve different purposes. Combining their answers into one discovery trend would hide that difference.

Keep a short change log before you save

Copy this table into a shared document. The version names are labels for your own records; they are not automatic OrderSurvey version IDs.

Keep a screenshot of the old configuration if it helps a colleague reconstruct it. Save the questionnaire record before making the edit; a later screenshot only shows the current configuration.

If you use Unlimited, keep an untouched Responses CSV export as well. An export preserves answer rows; it is not a replacement for the full option list and targeting record.

Update affected conditions and test both pages

Open the survey in OrderSurvey and make the planned edit. If an answer label changes, review conditions that depend on that answer. A follow-up intended for “Instagram” may need a different condition after you split that option into several choices.

After saving, check the intended path on Thank you and Order status using eligible test orders. Check the newly added answer, an unaffected answer, and a changed selection that should hide or clear a follow-up. Confirm a submitted test answer in Responses, and exclude test orders from your business analysis.

Use the conditional-survey test checklist for a fuller path review. Verify page assignments too: editing a survey does not publish a missing Shopify block.

Record any transition uncertainty. A response timestamp records submission, not necessarily when the buyer first opened the page. If an order was already open while you changed the questionnaire, do not assume its timestamp alone proves which configuration the buyer encountered. Keep uncertain transition responses separate when the distinction matters.

Example: splitting Instagram into three answers

Suppose you keep the question “Where did you first hear about us?” but replace Instagram with Instagram ad, Instagram creator, and Other Instagram content.

The following numbers are invented for illustration, not OrderSurvey customer results. Each period contains one answer to the discovery question per answering order; Other and uncertain answers remain in the denominator.

The after-period roll-up is (15 + 10 + 15) / 120 = 33.3%. The same number of answering orders chose an Instagram-related answer, while the denominator increased.

Two conclusions would go beyond this evidence:

• “Instagram ads went from zero to 15 orders.” The earlier survey did not offer that separate category; its value is unknown, not zero.
• “Instagram performance fell by 6.7 percentage points.” Answer share fell in this example, but the questionnaire, responding buyers and marketing context may all differ. The calculation does not isolate campaign performance.

A useful report says: “After splitting the option, 40 of 120 answering orders selected an Instagram category. The earlier broad option received 40 of 100 answers. We cannot separate the earlier responses into ads, creators and other content.”

Read OrderSurvey's reports with those limits in view

Editing a question preserves saved responses. In the survey overview, answers are grouped by their saved question identity, wording and type. Answers to a removed question remain in the breakdown, and earlier or removed questions are labeled. Where older responses lack details, the report discloses its use of available current information.

There are two limits to keep in mind:

• Option-only edits may share a group. If the question wording and type stay the same, the overview can combine answers collected under different option lists. Historical option lists are not reconstructed. Use your change log and reporting dates to separate them.
• The dashboard is a summary. Its survey distribution uses the first recorded answer in each response over the last 90 days. Open the question breakdown before treating that combined summary as one unchanged question. The survey overview uses up to the 5,000 most recent responses.

For a comparison worksheet, open Responses, choose the survey and an appropriate date range, then select Export CSV. Export a wider period if needed and filter the Date column in your working copy. This export uses UTC submission timestamps and one row per answered question.

Filter to the intended question and keep the locations and audience context explicit. Resolve repeated order IDs before counting answering orders; a three-question response is not three different orders. If two questions have identical wording or a response falls in an uncertain transition period, leave the classification unresolved rather than guessing. The free-text reporting guide explains how to preserve original answers while adding analysis categories.

Start collecting, then upgrade for the reporting workflow

OrderSurvey's Free plan includes 100 responses per rolling 30 days and conditional follow-ups. Unlimited is $24 USD/month and adds unlimited responses, CSV export, advanced analytics and webhook alerts. Unlimited does not add a complete questionnaire version history; the change log in this guide remains a manual record.

Install OrderSurvey from the Shopify App Store to create a survey on your Thank you and Order status pages. If you want help choosing a question, checking conditions or verifying a saved response, book a personal setup walkthrough.

Frequently asked questions