SpyderBot Documentation
Evaluate progress after an improvement
Your team updated a page, corrected a claim, or changed an access configuration. Now you need to decide whether to continue, adjust, expand the work, or stop.
Your outcome: an assessment that separates implementation, observed AI or website behavior, and business outcomes, followed by a justified next decision.
Start with the change you actually made
Record the asset, action, completion date, and intended result in your existing task record. Be specific: "Corrected the integration page's support conditions" is assessable; "improved GEO" is not.
Use the original finding and observation reference if you kept them. You do not need to rerun a broad report just to create a retrospective baseline.
| Your situation | What you can assess |
|---|---|
| Relevant before-and-after observations exist | Progress within their comparable scope |
| No suitable earlier observation exists | Implementation and current state; establish a baseline for future checks |
| The panel or Context changed | A supported comparison if available; otherwise separate the periods |
| The change is implemented but later evidence is pending | Completion now, outcome later |
1. Decide what success would look like
Use the result directly connected to the task:
| Task | Completion check | Relevant later observation |
|---|---|---|
| Correct a product fact | Approved information is published and accessible | AI describes the fact accurately in the affected scope |
| Repair an access failure | Relevant technical check passes | A subsequent real request succeeds |
| Improve a landing-page next step | The intended experience works | Measured human goal completion, where configured |
| Improve support for a buying requirement | Accurate evidence is available on the relevant asset | Portrayal or recommendation behavior for that requirement |
Choose the primary observation before inspecting results. Keep additional metrics as context. This reduces the temptation to call the work successful because an unrelated number improved.
2. Confirm implementation before judging effectiveness
Check the published content or implemented configuration. For technical work, use the relevant audit or verification result. Confirm that the change affects the intended URL, locale, and variants.
If implementation is incomplete or the test still fails, return to the owner. There is little value in interpreting AI outcomes as a test of a repair that has not actually been completed.
3. Check whether the observations are comparable
Confirm the question selection, Context, systems, metric, and competitor where relevant. Check observation dates and evidence availability. Use the product's supported comparable periods; do not assume two adjacent calendar dates provide equivalent data.
If you added easier questions or removed a weak-performing system, a better aggregate may reflect the new scope. Where a valid common-scope comparison is available, use it and explain the scope. Otherwise report the later result as a new baseline.
4. Read the outcome and the surrounding evidence
Use the relevant Observatory change intelligence, benchmark observations, report evidence, or AI Traffic verification and outcomes. Follow the change to the affected questions, systems, sources, or pages when it matters to the decision.
Record other relevant events: a product release, competitor change, source update, measurement repair, or scope change. A result observed after your intervention does not establish that your intervention alone caused it.
For business outcomes, use measured human sessions and configured goals. A citation or bot request is not a conversion. Revenue evidence, where available, still requires its own attribution limits.
5. Choose what to do next
| Evidence | Decision | Practical next step |
|---|---|---|
| Change not implemented correctly | Complete or repair | Return the failed acceptance check to the owner. |
| Intended observation improves with adequate support | Continue; consider a bounded expansion | Test the same approach on another relevant asset with its own evidence. |
| Intended observation is unchanged with suitable evidence | Adjust | Revisit the claim, source, user need, or implementation assumption. |
| Outcome evidence is still insufficient | Wait with a review condition | Specify the missing observation and a review date. |
| Evidence points to an adverse effect | Investigate; consider a targeted reversal | Confirm the affected experience and owner before further rollout. |
| The work no longer supports a business objective | Stop or defer | Record why and preserve useful findings. |
If several review checkpoints remain inconclusive, decide whether this measurement still supports an actionable question. Check the execution and scope, then either revise the evidence plan or defer the investment. Do not label limited evidence a failed intervention, but do not keep spending on identical runs without a decision they could change.
Illustrative example. A brand corrects an outdated integration page. Later comparable answers describe the integration accurately, but recommendation behavior remains unchanged. The team records success for the factual portrayal objective. It investigates remaining selection criteria before committing to a broader content rollout. It does not describe the unchanged recommendation result as proof that the factual correction failed.
Write the conclusion in three parts
Implemented: what changed and how it was verified.
Observed: what happened in the relevant scope, including evidence limits.
Decision: continue, adjust, expand, wait, or stop-with an owner and next checkpoint.
You are done when the evidence supports a next decision, including a decision to wait for specific missing data. If you have no baseline, say so and start one; a new run cannot recreate past AI behavior.
Next: use Report progress and recommend the next investment to communicate the assessment to leadership or a client.