SpyderBot Documentation
Log import for AI Traffic
Use Log import when you need AI crawler evidence from server or CDN access logs — including historical windows before live measurement started.
What Log import may provide
- AI crawler hits and requested URLs
- HTTP status / access outcomes from the log line
- Bot identity signals from known crawler user-agents
- Coverage of days present in the uploaded or Logpush payload
Log import does not measure human AI referrals or chat-referred humans, and does not prove citation, recommendation, or training use.
Setup checklist
- Confirm Log import is available for your account. If it is not, contact support.
- Open AI Traffic → Setup → Log import for the website.
- Upload a supported access log, or point Cloudflare Logpush at
/api/logpushwith headerX-Tracking-Key: <project tracking key>(Logpush is documented available on Free–Enterprise; confirm HTTP destination in your Cloudflare plan if missing). - Leave overlap at historical-only unless you intentionally accept live-day overlap.
- Validate imported bot hits in AI & Bot Activity for a known crawl window.