Survey design checklist

A survey is useful only when its responses can change a named decision. Define that decision before writing questions, then collect the least amount of information needed to act.

Before writing

• Write one sentence: "The responses will help us decide..."
• Name the audience and how it differs from the population you want to understand.
• Decide how each answer could change a PRD, issue, plan, or recommendation.
• Remove questions whose answers are already available in analytics, code, docs, or prior research.

Question checklist

• Ask about one concept at a time. Split questions joined by "and" when either half could have a different answer.
• Use simple, familiar language. Remove jargon, double negatives, unnecessary context, and vague time ranges.
• Avoid leading, loaded, biased, or assumptive wording. Include a legitimate neutral or negative response when one could exist.
• Make response options balanced, mutually exclusive, and collectively useful. Add "Other" only when you can act on unlisted answers.
• Fully label the ends of scales. Prefer a five-point scale for ordinary agreement or confidence questions unless the decision needs different resolution.
• Keep scales vertical and choice lists short enough to scan on mobile. Avoid dense matrix questions when separate items would be clearer.
• Make sensitive or identifying questions optional. Include "Prefer not to answer" when omitting the question is not practical.
• Use open text selectively for evidence or rationale that choices cannot capture.

Flow checklist

• Start with an easy, relevant question; place difficult or sensitive questions later.
• Group related questions and keep the order logical because earlier wording can influence later answers.
• Require only answers without which the response would be unusable.
• Use show_if to ask targeted follow-ups instead of showing every question to everyone.
• For longer studies, use one substantive question per page and enable settings.progress_bar.
• Pilot the survey with people similar to the audience. Ask what they thought each question meant, then revise ambiguity before treating results as evidence.

Analysis checklist

• Report the response count and who was invited.
• Separate answer distributions from your interpretation.
• Preserve disagreement, outliers, and missing answers.
• State sampling limits and avoid claims broader than the respondents support.
• Tie each recommendation to the question or response pattern that supports it.

Methodology sources

This checklist adapts guidance from:

• Pew Research Center: Writing Survey Questions on wording, question order, open and closed formats, and pretesting.
• Purdue OWL: Creating Good Interview and Survey Questions on biased, assumptive, double-barreled, confusing, and unrelated questions.
• University of Iowa: Survey Design Best Practices on plain language, labeled scales, mobile presentation, matrix use, and sensitive questions.

The recommendations above are concise adaptations for Motivation Form, not copied survey instruments. Pretest every questionnaire in its actual product context.

Start from apps/motivation-form/forms/agent-feedback-survey.md, then follow the agent feedback workflow.