THE SHORT ANSWER
A question panel is a defined selection of realistic questions tested repeatedly under documented conditions. It reveals observations for that selection, not a complete share of the AI-answer market. The protocol proposed here helps keep tests comparable and prevents a change from being prematurely attributed to one particular content or technical intervention.
For the broader picture of technical foundations, useful content and visibility measurement, read the pillar guide SEO, GEO and AEO for SaaS: visibility that supports buying decisions.
Derive questions from real tasks
Start with buyer conversations, support cases and concrete research tasks. Distinguish general problem questions, method comparisons and supplier-specific enquiries. A collection consisting only of brand-adjacent wording can make visibility look favourable while saying little about new discovery. Define categories, selection reasons and deliberately excluded tasks. German and English questions should test equivalent tasks without requiring literal translation. Preserve exact wording under stable identifiers. If a question changes later, create a new version. Otherwise a changed response may be misread as a change in the system or the website rather than the effect of a different question. Explore this further in the guide Building AI test cases: turn real failures into an evaluation set.
Define conditions before execution
Record the platform, visible model or mode, language, date and relevant account settings. Decide whether to use fresh sessions and exclude prior conversation context. Keep conditions as consistent as practical while acknowledging factors that cannot be controlled. Location and personalisation may limit comparability. Document what is known and what remains unknown. Do not adjust the protocol halfway through a run merely because some answers look unfavourable. If a technical problem prevents a test, record it separately rather than counting it as a negative brand mention in the same denominator as a successfully generated answer.
Preserve complete answers and sources
Save the full response with question identifier, time and visible source links. A screenshot alone may hide links or collapsed references, so retain suitable text and link details as well. Repeat according to a predeclared plan rather than collecting only favourable answers. State the number of valid runs for each question and platform. Respect the systems’ permitted access methods and usage conditions. The objective is a modest inspectable observation, not uncontrolled bulk querying. Exclude personal or confidential information from prompts; authentic buyer tasks can generally be represented without customer names or private project details.
Use a consistent coding scheme
Define in advance what counts as a brand mention, source link, correct service attribution and factual error. These are different events: a name without a link is not a citation, and a citation can accompany an incorrect claim. Have a second person inspect difficult cases using the same criteria. Preserve disagreements and improve the scheme for a later run where needed. Do not present an opaque composite score as an objective visibility measure. When summarising results, show the underlying categories and denominators. This makes it possible to distinguish more links from more appropriate explanations or merely different patterns of brand naming.
Example: comparable rather than perfect repetition
A hypothetical consultancy builds a panel of specific SaaS marketing questions. It tests German and English tasks separately in defined modes and saves each valid run. A later repetition returns different sources on one platform while the substantive explanation remains similar. The team records that variation rather than displaying only the better run. It notes a provider update as contextual information. The panel now supports an inspectable discussion about particular questions. It does not prove complete visibility for all users or a causal effect of the latest copy change. Those boundaries belong beside the findings so they remain visible when shared.
Evolve the panel through versions
Retain a stable core for comparisons and label new tasks as additions. A changed question mix can shift an aggregate rate without changing answers to identical questions. Compare the same identifiers as well as totals. Keep panel observations separate from website visits and CRM outcomes. Google explains that its AI search responses and links may vary and that serving is not guaranteed. This supports cautious interpretation without making the proposed panel an officially standardised measurement procedure. Its usefulness comes from recorded conditions, repeatable coding and an honest account of what was actually observed within the defined sample. Explore this further in the guide Evaluating GEO changes through cautious experiments.
INTERACTIVE / FROM READING TO ACTION
Three checkpoints for your implementation
Choose a checkpoint and track your reading progress. Checks are neither saved nor sent and do not score your business.
CHECKPOINT 01
Preserve complete answers and sources
Save the full response with question identifier, time and visible source links.
Explore in the article ↗CHECKPOINT 02
Use a consistent coding scheme
Define in advance what counts as a brand mention, source link, correct service attribution and factual error.
Explore in the article ↗CHECKPOINT 03
Example: comparable rather than perfect repetition
A hypothetical consultancy builds a panel of specific SaaS marketing questions.
Explore in the article ↗Frequently asked questions
Does the panel measure total AI market share?
No. It describes selected questions under recorded conditions.
Why repeat runs?
Responses and sources can vary; repetitions reveal that variation.
Are mentions and citations identical?
No. Code names, links and factual accuracy as separate categories.
How can I put “Derive questions from real tasks” into practice?
Start with buyer conversations, support cases and concrete research tasks. Distinguish general problem questions, method comparisons and supplier-specific enquiries. A collection consisting only of brand-adjacent wording can make visibility look favourable while saying little about new discovery.
What matters when working on “Define conditions before execution”?
Record the platform, visible model or mode, language, date and relevant account settings. Decide whether to use fresh sessions and exclude prior conversation context. Keep conditions as consistent as practical while acknowledging factors that cannot be controlled.
