Living Online

Agency OS

Crawl audit

Technical SEO Audit

Runs a public-site crawl audit with Browser Run, deterministic technical SEO checks, PageSpeed evidence, and an R2/Asana evidence package.

Status

Hosted audit path

Current hosted implementation state.

Trigger Surface

Asana task

Created or updated in Jessie Intake and assigned to Jessie.

Delivery

HTML artifact

Attached to the source task or linked as a stable artifact.

Approval

Review required

No client-facing or external-system action without human approval.

Consultant Prompt

Hosted audit path

Example Asana request

Run a technical SEO audit for https://www.example.com.au.

Routes when a Jessie-assigned Asana task asks for a technical SEO audit, crawl audit, Screaming Frog-style audit, indexability review, or canonical review.

Default Inference

  • Infers the target public website when there is one confident client/site match.
  • Uses same-host crawling with a default bounded page limit.
  • Uses render-disabled Browser Run crawling by default and targeted rendering only for JavaScript SEO risk checks.
  • Asks for clarification for private, staging, authenticated, or ambiguous targets.

Data Sources

  • Cloudflare Browser Run /crawl output.
  • Direct robots.txt and XML sitemap fetches.
  • Sampled PageSpeed Insights/Core Web Vitals evidence.
  • Targeted rendered validation and screenshots when useful.
  • R2 audit package and Asana attachment fallback.

Output Contract

  • Consultant-facing static HTML technical audit report.
  • Supporting CSV and JSON evidence package.
  • Critical crawl-quality finding when Browser Run returns empty or incomplete evidence.
  • Developer-ready fix briefs for High and Medium findings.

Guardrails

  • Public websites only.
  • No robots.txt bypass, CAPTCHA bypass, login-wall bypass, or authenticated crawling.
  • No website, CMS, tracking, or client-facing changes.
  • Do not use this for monthly SEO performance reporting.

Current System Context

Recent changes
  • This skill is deliberately deeper than the small Browser Run checks used inside monthly forensic reporting.
  • Evidence package shape is defined for technical-seo-audits/{hostname}/{yyyy-mm-dd}-{run-id}/.
  • R2 is preferred for larger packages, with Asana attachment fallback when R2 is unavailable.

Prompt Source

.agents/skills/technical-seo-audit/SKILL.md
---
name: technical-seo-audit
description: Run consultant-facing technical SEO audits from Asana-triggered public website crawl tasks. Use when a task asks for a technical SEO audit, site crawl audit, Screaming Frog-style audit, crawl/indexability/canonical review, or technical SEO analysis of a website.
---

# Technical SEO Audit

Use this skill when the Asana task asks for a public-site technical SEO crawl audit. Do not use it for monthly SEO performance reporting, organic traffic movement, GA4/Search Console analysis, rankings, backlinks, or content strategy; those belong to `seo-reporting` or another skill.

## Operating Rules

- Treat the Asana task as the trigger, review trail, and delivery destination.
- Infer the target website from plain-language task text or trusted client config when there is one confident match.
- Crawl public websites only. Do not handle staging, password-protected, authenticated, private, or account areas in v1.
- Crawl same hostname only, default 500 HTML pages.
- Use Cloudflare Browser Run as the primary crawler with `render: false`.
- Use targeted rendering only for JavaScript SEO risk validation or screenshot evidence.
- Do not bypass robots.txt, bot protection, CAPTCHA, login walls, or access controls.
- Do not create follow-up Asana tasks in v1.
- Do not publish client-facing output or make website/CMS/tracking changes.
- Never expose credentials. Cloudflare/API credentials must be in Cloudflare Secrets, Workers Secrets, or an approved secret manager.

## Workflow

1. Confirm the task routes to `technical-seo-audit`, not `seo-reporting`.
2. Extract or infer the target website. If missing or ambiguous, comment on the Asana task asking for the public website.
3. Run the hosted Agency OS technical SEO audit path. It should:
   - crawl with Cloudflare Browser Run `/crawl`
   - keep render disabled by default
   - analyze raw HTML technical SEO signals
   - fetch robots.txt and XML sitemaps directly for deterministic evidence
   - run sampled PageSpeed Insights checks for Core Web Vitals and Lighthouse evidence
   - run targeted rendered validation only for representative JavaScript SEO risk pages
   - store the audit package in R2 when the binding is available
   - attach `report.html` to the triggering Asana task
   - expose `crawl-data.json`, `page-inventory.csv`, `crawl-debug.json`, V2 supporting exports, and issue CSV evidence through R2 links or Asana fallback attachments
   - post a summary comment back to the same task
4. If Browser Run credentials are not configured, report the missing secret clearly. If R2 is unavailable, the audit should still deliver by attaching the evidence files to Asana. Do not invent an ad hoc local crawler unless explicitly asked.
5. Read [references/rubric.md](references/rubric.md) only when you need the issue categories, severity rules, or v1 scope boundaries.

## Report Contract

The delivered report is consultant-facing, issue-focused, and static HTML. It must include:

- executive summary
- crawl quality and limitations
- top findings
- findings by severity and action status
- robots.txt and XML sitemap sections
- indexability and canonical findings
- metadata findings
- heading/content basics
- internal broken links
- JavaScript SEO risk
- structured data parse/sanity checks
- basic hreflang checks
- platform/CMS detection with confidence/evidence
- lightweight crawl-visible performance hints
- sampled Core Web Vitals and Lighthouse performance evidence
- direct robots.txt and XML sitemap evidence
- targeted rendered validation evidence where useful
- developer-ready fix briefs for High/Medium findings
- appendix links to supporting CSV/JSON evidence

Findings must be deterministic and backed by crawl evidence. AI may summarize and group evidence, but must not invent findings.

An empty Browser Run result is not a successful audit. If polling times out, the crawl remains running, or no usable same-host records are returned, the report must include a Critical/blocking crawl-quality finding with the job id, start URL, poll attempts, last status, API caveat/error, incomplete reason, and recommended next action.

## Output Package

Each audit produces a timestamped package:

```text
technical-seo-audits/{hostname}/{yyyy-mm-dd}-{run-id}/
  report.html
  crawl-data.json
  page-inventory.csv
  crawl-debug.json
  performance-cwv.json
  performance-summary.csv
  robots-sitemaps.json
  sitemap-samples.csv
  rendered-validation.json
  rendered-validation.csv
  issues/*.csv
  evidence/screenshots/*   # only when captured as targeted evidence
```

The Asana task receives the HTML report attachment and a summary comment with the R2 package reference, crawl-data reference, issue export reference, top findings, and crawl quality summary. If R2 is unavailable, attach the crawl data, page inventory, crawl debug, and issue CSV files directly to the triggering Asana task.

Related Docs

1 references
  • Team Agent Work Intake docs/runbooks/team-agent-work-intake.md