SpyderBot Documentation
AI Page Access
This page answers which pages received observed AI activity, what access outcomes were measured, and which important pages require investigation.
Core metrics
| Metric | Definition |
|---|---|
| Eligible Inventory Pages | Public canonical pages included in the current inventory policy |
| AI-Active Pages | Eligible pages with qualifying AI bot activity in the selected period |
| Observed Page Coverage | Share of eligible inventory pages with AI activity |
| Critical Page Coverage | Share of eligible critical pages with AI activity |
| Known Status Coverage | Share of AI activity with HTTP status evidence |
| Successful Access Coverage | Share of pages with known successful (2xx) AI activity among pages with status evidence |
| Pages with No Observed AI Activity | Eligible inventory pages with no qualifying event in the period |
Every percentage should be read with its numerator, denominator, source, and time window.
Interpretation guardrails
- URL normalization defines canonical page identity; raw URL variants remain available as evidence.
- No observed activity is not equivalent to invisible, blocked, or undiscoverable.
- A successful response does not prove content extraction or understanding.
- HTTP conclusions require adequate status coverage.