SpyderBot Documentation
LLM Tracking Overview
LLM Tracking is SpyderBot's AI Interaction Intelligence platform.
Overview
It helps organizations understand how AI systems discover, access, interpret, and interact with their websites throughout the AI ecosystem.
While Brand Insights explains how AI systems understand your organization, and Prompt Intelligence explains why AI systems behave the way they do, LLM Tracking focuses on an equally important question:
How do AI systems interact with your website?
Understanding these interactions helps organizations improve AI accessibility, monitor AI traffic, optimize website configuration, and identify opportunities to strengthen AI Visibility.
Why LLM Tracking Matters
AI systems interact with websites differently from traditional search engines.
In addition to crawling webpages, modern AI systems may:
- Discover new content.
- Access documentation.
- Retrieve structured information.
- Generate referrals.
- Reference website content.
- Respect access policies.
- Continuously revisit important resources.
Understanding these interactions is becoming an essential part of managing AI Visibility.
LLM Tracking provides visibility into that interaction layer.
Business Decision
LLM Tracking helps answer one strategic decision:
Is our website prepared for effective interaction with modern AI systems?
Organizations increasingly optimize websites not only for human visitors and search engines, but also for AI systems.
LLM Tracking helps evaluate whether websites support that objective.
Business Questions
LLM Tracking helps answer questions such as:
- Which AI systems are interacting with our website?
- How are AI systems discovering our content?
- Which AI systems generate referrals?
- How much AI-related traffic reaches our website?
- Are AI crawlers being allowed or restricted appropriately?
- Which website improvements may strengthen AI accessibility?
The AI Interaction Lifecycle
LLM Tracking follows a continuous interaction lifecycle.
Observe
↓
Understand
↓
Control
↓
Optimize
↓
Monitor
Each stage represents a different aspect of how AI systems interact with your website.
Together they provide a complete picture of AI interaction.
Core Intelligence Areas
LLM Tracking combines several complementary intelligence layers.
AI Crawl Intelligence
Understand how AI systems discover and access your website.
AI Crawl Intelligence helps identify which AI crawlers visit your website, how they access content, and how crawling behavior evolves over time.
AI Crawl Intelligence
AI Referral Intelligence
Understand how AI-generated responses lead users to your website.
Referral Intelligence helps organizations measure AI-originated visits and understand which AI systems contribute to website traffic.
AI Referral Intelligence
AI Traffic Intelligence
Understand the broader patterns of AI-related website traffic.
Traffic Intelligence helps organizations evaluate how AI interactions contribute to overall website engagement and identify emerging trends.
AI Traffic Intelligence
AI Access Control
Understand how website configuration influences AI interaction.
This includes controls such as robots.txt, llms.txt, and other mechanisms that determine how AI systems access website content.
AI Access Control helps organizations verify that these configurations align with their AI Visibility objectives.
AI Access Control
AI Optimization
Transform interaction insights into practical optimization opportunities.
AI Optimization helps organizations identify website improvements that may strengthen AI accessibility, content discovery, and long-term AI Visibility.
AI Optimization
Monitoring
Monitor changes in AI interaction over time.
Monitoring continuously observes AI interaction patterns and helps organizations identify meaningful operational changes that may require attention.
Monitoring
How LLM Tracking Fits Within SpyderBot
LLM Tracking complements the other SpyderBot products by focusing on website interaction rather than AI perception or prompt behavior.
A typical workflow looks like this:
Brand Insights
↓
Understand AI Perception
↓
Prompt Intelligence
↓
Understand AI Behavior
↓
LLM Tracking
↓
Understand AI Interaction
↓
Optimize Website
↓
Measure Again
Together, these products provide a comprehensive understanding of how AI systems perceive organizations, generate responses, and interact with websites.
Typical Workflow
Although every organization has different objectives, a common workflow includes:
1. Observe AI Interaction
Identify how AI systems currently access and interact with your website.
2. Understand Interaction Patterns
Review crawl activity, referrals, traffic, and access configuration.
Determine whether current interaction aligns with business objectives.
3. Improve Website Accessibility
Implement website improvements that strengthen AI accessibility and content discoverability.
4. Continue Monitoring
Monitor interaction continuously to identify meaningful changes and evaluate the long-term impact of optimization efforts.
AI interaction evolves continuously.
Effective AI Visibility therefore requires ongoing observation.
Related Concepts
To better understand LLM Tracking:
Related Products
LLM Tracking works closely with:
- Products → Brand Insights — Understand how AI perceives your organization.
- Products → Prompt Intelligence — Understand why AI behaves the way it does.
Together these products provide visibility across AI perception, AI behavior, and AI interaction.
Next Steps
Continue with:
AI Crawl Intelligence explains how AI systems discover, access, and revisit your website, forming the foundation of AI Interaction Intelligence.