The agent

thespider-agent is TheSpider's thin execution agent — a single Rust binary. It holds only a server URL, an auth token, and model_code → command bindings. All brain logic and all state live server-side; what reaches the client is one rendered prompt per claimed slice, by design.

Use the command surface

thespider-agent login          # open <server>/activate, paste enrollment code (stdin), store token
thespider-agent logout         # clear the stored token (keychain + file)
thespider-agent status         # org/projects/entitlements, config sanity, token source
thespider-agent models         # server catalog vs local [[model]] bindings, as a diff
thespider-agent run [flags]    # claim slices, run models locally, upload results
thespider-agent gate --campaign <id>   # poll the PR gate; exit 0 pass / 1 fail / 2 pending
thespider-agent daemon [flags] # persistent runner: poll → managed-clone → claim/run/submit

Enroll with login

login prints (and best-effort opens) <server>/activate, reads a short enrollment code from stdin, and POSTs /v1/enroll. It is identical in SaaS and on-prem — the /activate page is server-side, and enrollment codes are minted in the web UI or via bun run src/mint-enrollment-code.ts.

Run a campaign

thespider-agent run [--campaign <id>] [--once] [--models a,b] [--target-path <dir>] [--redact]

The loop:

1. GET /v1/me → resolve org/project/entitlements.
2. Campaign: --campaign <id>, else create a full campaign from local git detect (branch/commit/remote_url/dirty; the client detects, the server records).
3. Claim a batch (sized to concurrency), respecting the lease/timeout policy. For each lease: write the prompt to a temp file, substitute the model command, spawn sh -c, capture stdout/stderr/exit/duration, heartbeat the lease while the model runs, then POST the raw stdout echoing prompt_hash.
4. Response handling: accepted (200), accepted-as-artifact / superseded (202), prompt_hash mismatch (409), too large (413). Network errors, 429, and 5xx retry with exponential backoff + jitter.
5. --once processes a single claim batch and stops (CI / one-shot). Without it, run loops until no claimable slices remain.

Redact on the client (opt-in)

The server secret scrubber is always on. --redact additionally, on the client, before upload:

1. Masks secret-shaped substrings in raw stdout using the same pattern family — PEM private-key blocks, JWTs, Authorization: Bearer tokens, AWS access-key ids, GitHub/Slack/Google/OpenAI keys, and long high-entropy base64/hex runs — replacing them with «redacted». Ordinary prose, short identifiers, and file paths are preserved.
2. Truncates stdout to at most 1 MiB (stricter than the server's default artifact cap), appending a visible marker.

It never parses or reshapes findings semantics — it operates purely on the raw stdout string.

Run the persistent daemon (PR/CI pattern)

thespider-agent daemon [--interval <secs>] [--workdir <dir>] [--models a,b] [--redact] [--max-iterations <n>]

Run the daemon once on a team box or self-hosted CI runner holding a project-scoped token and local model creds; CI jobs then only poll the gate — no model credentials or model runtime in CI.

Loop:

1. GET /v1/me → resolve the project.
2. Poll GET /v1/projects/{id}/campaigns/claimable — campaigns with pending / expired-lease slices, newest first. These are created by the GitHub webhook (PR opened/synchronized → incremental campaign) or by a developer's run.
3. For each workable campaign: managed clone — fetch and check out the campaign's exact commit into <workdir>/<project>/<remote-hash> (a stable per-repo directory, reused across polls), detached — then claim/run/submit its slices exactly like run, with {target_path} = the managed checkout.
4. Sleep a jittered interval (which backs off while idle) and poll again.

Handle daemon robustness

• Graceful shutdown on SIGINT/SIGTERM — the in-flight campaign finishes; no new slices are claimed.
• Per-campaign error isolation — a bad clone/checkout/run logs and the loop continues.
• A campaign with no remote_url/commit (e.g. a CLI dirty-tree campaign) is skipped, not errored.
• --max-iterations <n> bounds the loop (E2E / one-shot CI).
• The daemon prints a daemon summary: iterations=… campaigns_worked=… leases=… accepted=… reports=… failed=… line.

Authenticate the managed clone

Git auth is git's own — the daemon shells out to git clone/git fetch:

• SSH remotes (git@github.com:owner/repo.git) → a loaded ssh-agent key (or ~/.ssh/config identity) the daemon's user can use non-interactively.
• HTTPS remotes → a token embedded in the remote URL (https://x-access-token:<TOKEN>@github.com/owner/repo.git) or a configured git credential helper.

The repo_remote_url a project stores is what the webhook maps PRs to and what the daemon fetches — set it to a URL the runner can actually clone.

Next steps

• PR gate — wire the webhook and poll from CI.
• Campaigns — full / incremental / retest semantics.