SpyderBot Documentation

Log import for AI Traffic

Use Log import when you need AI crawler evidence from server or CDN access logs — including historical windows before live measurement started.


What Log import may provide

  • AI crawler hits and requested URLs
  • HTTP status / access outcomes from the log line
  • Bot identity signals from known crawler user-agents
  • Coverage of days present in the uploaded or Logpush payload

Log import does not measure human AI referrals or chat-referred humans, and does not prove citation, recommendation, or training use.


Setup checklist

  1. Confirm Log import is available for your account. If it is not, contact support.
  2. Open AI Traffic → Setup → Log import for the website.
  3. Upload a supported access log, or point Cloudflare Logpush at /api/logpush with header X-Tracking-Key: <project tracking key> (Logpush is documented available on Free–Enterprise; confirm HTTP destination in your Cloudflare plan if missing).
  4. Leave overlap at historical-only unless you intentionally accept live-day overlap.
  5. Validate imported bot hits in AI & Bot Activity for a known crawl window.