SpyderBot Documentation
Recommendations and ranking
Mentions, candidate inclusion, active recommendations and primary recommendations describe different outcomes. Their conditions are assessed in context, not from list formatting or isolated keywords. A task’s eligibility is established before its answer is observed.
The rates below use the balanced aggregation method ↗. Different cards can have different eligible prompt populations; their percentages should not be compared as if they all used the same denominator.
On this page
Organic Recommendation Rate · Primary Recommendation Rate · Candidate Inclusion Rate · Branded Endorsement Rate · Comparative Preference Rate · Alternative Recommendation Rate · Ordered Top-1 Rate · Ordered Top-3 Rate · Average Rank When Ranked · Ranked-Target Coverage · Ranking Instruction Compliance · Observed Leader
Organic Recommendation Rate
Unit: Percent
How often AI actively advises considering, choosing or using the target in eligible open recommendation tasks.
How it is calculated
Within every eligible measurement group, count active recommendations among all complete, usable answers to recommendation-eligible questions. Include relevant explicit conditional advice. A complete neutral list, refusal or answer without a recommendation remains an observed opportunity with no positive recommendation. Combine group rates using balanced aggregation.
How to interpret it
The denominator is not restricted to answers that already recommend something. A mention, quoted praise or negative recommendation is not automatically an active recommendation.
What to do next
Review the needs and conditions for which the target is or is not recommended.
Illustrative example
“Options include A and B” can be candidate inclusion without active advice. “Consider A if you need offline access” can be an active recommendation when that condition fits the request.
Related concepts: Candidate Inclusion Rate ↗ · Primary Recommendation Rate ↗
Measurement scope and aggregation ↗ · Evidence and availability ↗
Primary Recommendation Rate
Unit: Percent
How often the target is explicitly preferred or placed in an explicitly preferred top tier.
How it is calculated
Use the same eligible open recommendation panel as Organic Recommendation Rate, but count only an explicit primary choice or preferred top tier. Apply balanced aggregation.
How to interpret it
An unordered shortlist does not establish a primary choice. Several entities may share an explicitly preferred top tier.
What to do next
Inspect whether the target is considered but rarely preferred.
Measurement scope and aggregation ↗ · Evidence and availability ↗
Candidate Inclusion Rate
Unit: Percent
How often the target is presented as an admissible option in an eligible open recommendation task.
How it is calculated
Count inclusion as an admissible candidate among all usable answers in the eligible recommendation panel, then aggregate with the same balanced method. Neutral option lists can qualify; a mere reference or explicit rejection does not automatically qualify.
How to interpret it
Inclusion is broader than active recommendation, but narrower than any substantive mention on the same answer pool. Headlines on different prompt populations need not follow this numerical ordering.
What to do next
Compare inclusion and active recommendation on the same scope to find consideration without endorsement.
Measurement scope and aggregation ↗ · Evidence and availability ↗
Branded Endorsement Rate
Unit: Percent
How often AI affirmatively endorses a choice or claim that explicitly introduces the target.
How it is calculated
Use aided questions that actually ask for endorsement or choice validation. Count an affirmative endorsement of that requested choice in context, then balance within that cohort.
How to interpret it
A general trust or factual question is not automatically a purchase endorsement opportunity. This rate does not enter organic recommendation.
What to do next
Examine qualifications and cautions alongside affirmative endorsements.
Measurement scope and aggregation ↗ · Evidence and availability ↗
Comparative Preference Rate
Unit: Percent
How often the target is explicitly preferred in a named comparison.
How it is calculated
Use the declared named-candidate comparison panel. Count a supported explicit preference for the target, including an explicitly tied preference where applicable, across all usable eligible answers. Apply balanced aggregation.
How to interpret it
The specified candidate set constrains the result. This does not measure open-market discovery.
What to do next
Keep comparison criteria and candidates stable when evaluating changes.
Measurement scope and aggregation ↗ · Evidence and availability ↗
Alternative Recommendation Rate
Unit: Percent
How often AI actively recommends the target as an alternative to the specified competitor.
How it is calculated
Use the competitor-led alternative questions for the stated anchor. Count active target recommendations among all usable eligible answers, then balance within that anchor-specific scope.
How to interpret it
It is narrower than Competitive Capture, which measures substantive presence. Changing the anchor changes the question.
What to do next
Review the use cases where the target is a credible alternative.
Related concepts: Competitive Capture ↗
Measurement scope and aggregation ↗ · Evidence and availability ↗
Ordered Top-1 Rate
Unit: Percent
How often the target receives a validated first rank when the question requests an ordered comparison.
How it is calculated
Use all usable answers to eligible ordered-ranking tasks. Count a validated target rank of 1; target absence, lack of required ordering or no qualifying rank contributes no Top-1 outcome. Apply balanced aggregation.
How to interpret it
Formatting position alone is not a semantic rank. Explicit rank-one ties may allow several entities to qualify; this is not a share that must sum to 100%.
What to do next
Check ordering compliance and the common ranking criteria.
Measurement scope and aggregation ↗ · Evidence and availability ↗
Ordered Top-3 Rate
Unit: Percent
How often the target receives a validated rank of 1, 2 or 3 in eligible ordered-ranking tasks.
How it is calculated
Calculate validated Top-3 outcomes across all usable answers to ranking-eligible questions, including observed noncompliance and target absence as non-positive outcomes. Use balanced aggregation.
How to interpret it
The denominator includes answers where the target has no rank. A naturally numbered answer to a task that did not request ranking is diagnostic evidence, not automatically an eligible ranking observation.
What to do next
Read Top-3 with ranked-target coverage and ordering compliance.
Measurement scope and aggregation ↗ · Evidence and availability ↗
Average Rank When Ranked
Unit: Rank
The target’s average validated rank among applicable answers where it was ranked.
How it is calculated
Average known target ranks within applicable measurement groups, then use conditional balanced aggregation. Groups known to contain no target rank are not applicable; unresolved extraction is missing. No target ranks means no average rank.
How to interpret it
A rank of 1 based on rare appearances can coexist with low coverage. Missing ranks are not replaced by zero or an invented last position. Lower is better only within comparable ranking criteria.
What to do next
Always inspect ranked-target coverage and supporting counts.
Related concepts: Ranked-Target Coverage ↗
Measurement scope and aggregation ↗ · Evidence and availability ↗
Ranked-Target Coverage
Unit: Percent
How often a usable answer to an eligible ranking task assigns the target a validated rank.
How it is calculated
Calculate the balanced share of all usable eligible ranking answers containing a valid target rank.
How to interpret it
This is the companion needed to understand Average Rank. It does not say whether the rank is high.
What to do next
Investigate low coverage before interpreting a good conditional average.
Measurement scope and aggregation ↗ · Evidence and availability ↗
Ranking Instruction Compliance
Unit: Percent
How often the answer provides the requested meaningful ordering on the stated comparison basis.
How it is calculated
Calculate the balanced share of usable eligible ranking answers that comply with the requested semantic ordering.
How to interpret it
A numbered list is not sufficient by itself. Low compliance limits what the ranking outcomes can tell you.
What to do next
Review task clarity and observed answers; keep any redesigned prompt separate in history.
Measurement scope and aggregation ↗ · Evidence and availability ↗
Observed Leader
Unit: Entity and observed lead state
The entity with the highest observed result on the explicitly named metric and shared comparison scope.
How it is calculated
Entities are compared on one task-aligned metric and a common observation pool. The highest observed value determines the leader; exact ties remain ties. The leader state distinguishes no positive outcome, a single candidate, narrow leadership and insufficient evidence. The runner-up margin is the difference on that same metric when a comparable runner-up exists.
How to interpret it
This replaces the old blended Winner Score. An observed leader is not a statistically proven superior brand. Pure factual or trust evaluation has no default cross-entity winner.
What to do next
Check the basis, candidate roster, margin and evidence before acting.
Illustrative example
An enumeration task can have an observed mention leader; it must not be labelled the recommendation winner.
Measurement scope and aggregation ↗ · Evidence and availability ↗