SpyderBot Documentation

Understanding SpyderBot measurements

SpyderBot measures what it observes in a defined set of questions, AI systems, search opportunities or connected traffic sources. A metric describes that scope. It is not a measurement of every AI user, every answer or the internal state of an AI model.

Every numeric result should be read with its unit, eligible population, time window, evidence support and method version. The same word can describe different scopes: an assistant mention rate, exposure across search opportunities and an external traffic estimate are not interchangeable.

How balanced aggregation works

A headline rate is not normally the total positive answer count divided by all collected answers. That pooled calculation would give more influence to prompts or systems that happened to receive more repetitions.

Under the standard balanced profile, calculation proceeds through the declared measurement structure:

  1. For one exact prompt variant, channel, system and surface, calculate the outcome rate across accepted observable repetitions.
  2. Combine declared surfaces within the same actual system with equal surface weight.
  3. Combine independent actual systems within a channel with equal system weight.
  4. Combine channels using the named profile. One channel receives all weight in a single-channel result; the explicitly balanced assistant-and-search profile assigns half to each channel.
  5. Combine approved wording variants within their Core Prompt with equal variant weight.
  6. Combine eligible Core Prompts with equal strategic weight.

The scope and profile are fixed before observing outcomes. Prompt priority, traffic estimates, proxy labels, agent count, retry volume and extra repetitions do not create extra strategic votes. Two labels for the same actual model configuration do not automatically count as two independent systems.

For example, in a complete two-system, one-prompt scope, 8 mentions in 10 accepted answers from system A and 2 in 2 from system B produce system rates of 80% and 100%. Equal-system visibility is 90%. The raw count remains 10 mentions in 12 accepted answers. SpyderBot must not invent a raw numerator by multiplying 90% by 12.

These elementary aggregation rules define the meaning of the reported number. The internal methods used to recognise entities, interpret recommendations and validate classifications are separate.

Conditional metrics

Some metrics apply only when a known condition exists. Sentiment applies when the entity is represented and can be classified; Average Rank applies when a valid target rank exists; search-answer presence applies when a usable AI answer is generated.

Calculate within the known applicable observations, then combine them through the same declared hierarchy. At each level, known inapplicable groups do not contribute to that conditional average; the original weights are rebalanced over applicable groups. Missing or unresolved groups cannot be treated as inapplicable. Show applicability and coverage alongside the conditional metric.

Two brands with different applicable answer sets do not automatically support a fair head-to-head conditional comparison. Use common applicable scope for a comparison or clearly present the results separately with limited comparability.

What enters the denominator

Question eligibility is established before the answer is observed. Organic discovery, aided evaluation, named comparisons, competitor-led alternatives, ranking, commerce and risk questions remain distinct. An entity supplied in the effective question context can make the task aided even if it is absent from the visible short question. Metadata used only to analyse an answer does not, by itself, seed that answer.

For an eligible unconditional metric, a complete observable answer without the target outcome contributes a negative outcome. This can include an answer that declines to recommend anything. A failed collection, incomplete answer or unresolved extraction is missing evidence. Do not remove unfavourable answers simply because they did not recommend a brand.

Each logical observation contributes at most once to a metric. Retries and reused artifacts do not multiply evidence. A substantive mention is assessed in context, not by raw keyword frequency. A quoted claim, a rejection, an admissible option, active advice and a primary preference are different outcomes.

Zero, unavailable and not applicable

DisplayMeaningExample
0%The applicable outcome was observed and did not occur, with enough support to report it.Complete eligible answers contain no substantive target mentions.
Not applicableA required condition is known not to exist.No generated search AI answers for an answer-conditional metric.
Unavailable / MissingRequired evidence is absent or unresolved.A search collection failed before the answer state could be verified.
Insufficient evidenceObservations exist but do not support the stated result.Only one repeated answer for a pair-agreement diagnostic.
SuppressedThe value is withheld because a specific publication requirement is unmet.Claim accuracy lacks adequate verification support.
Not included / Access restrictedThe feature or evidence is outside the current entitlement or viewer permission.A viewer cannot open underlying answer evidence.

Do not use “0”, a neutral midpoint or a generic dash for all of these states. Access restrictions do not imply the metric is zero or that no evidence exists.

A fixed-panel headline cannot silently drop a missing prompt, system or channel and keep the original label. An explicitly reduced-scope diagnostic is a separate result with its own scope, evidence level and limitations. Known no-answer search outcomes are not missing collection.

Counts, rates, shares and positions

Counts describe the evidence collected. Rates describe outcomes within an eligible population and can be balanced rather than pooled. Shares compare components against a named total. Positions are ordinal values within an eligible ranking task. A source-provided external category rank is different from an AI answer rank.

One answer can mention several entities, recommend several candidates or cite several domains. Presence rates across entities or domains may sum above 100%. A declared allocation share can sum to 100%. Rounded displayed values can create small total differences.

Do not rebuild a headline from rounded table rows when the underlying calculation uses greater precision. Retain raw counts as raw counts and preserve the declared metric denominator.

Comparing results over time

Compare the same entity set, eligible questions, language, location, systems, surfaces, channels, sampling design, date policy and compatible method versions. A result from a changed scope may be useful, but it answers a changed question.

For percentage rates, use percentage-point changes. From 20% to 30% is +10 percentage points; it is +50% relative growth. A missing baseline is not a 0% baseline. An AI model or search surface can change independently of your website.

What the methodology explains

SpyderBot explains what each metric measures, which observations qualify, the calculation and aggregation basis, data limitations and how to interpret the result. Available report evidence helps you inspect the observations behind your own result, subject to access permissions.

Internal extraction prompts, classifier implementation, calibration procedures, infrastructure and detection safeguards are not part of the public methodology. Keeping these implementation details private does not change the published definition or allow missing evidence to be presented as measured performance.

Core Prompts and wording variants

A Core Prompt is one strategic question in your confirmed measurement set. Approved wording variants express that question in different ways. Variants and repeated answers help observe variation; they do not turn one strategic question into several equally weighted Core Prompts. Changing the underlying question changes the measurement scope.

Measurement cells and observations

A measurement cell is one declared combination of question wording, channel, actual AI system, surface and other fixed conditions such as language and location. A repeated observation is another accepted collection under that configuration. A retry is a collection attempt; it is not automatically another accepted observation.

Question groups and eligibility

A question group, also called a cohort, contains questions with the same relevant measurement purpose. Open discovery, named-brand evaluation, named comparison and risk-seeking questions create different opportunities. Eligibility says whether a question is appropriate for a particular metric before any answer is observed.

Observation opportunities

An opportunity is the unit at which an outcome could occur. For assistant response metrics it is an eligible observable answer. For search exposure it is an eligible verified search opportunity, including a verified result that produces no AI answer. For a conditional metric, an additional known condition determines which observations apply.

Observed, estimated and diagnostic results

Observed results describe collected answers, search outcomes or connected activity. Estimated results come from a stated external estimate or modelled planning source. Diagnostic results assess tested conditions, such as URL delivery or content accessibility. These labels describe different evidence bases and are not substitutes for one another.