Set up Agent swarm

Agent swarm uses a Meticulous-hosted agent to explore and test your application for a pull request. Meticulous uploads the build from your CI job and opens it in a recorder-instrumented browser, where the agent follows your instructions to exercise happy paths, edge cases, and potential regressions. The Agent swarm page reports each test case's result with step-by-step screenshots, and the discovered flows are saved as sessions that extend your replay-test coverage.

Use this guide to add Agent swarm to your pull requests. It complements ordinary Meticulous replay tests; it does not replace the sessions you record or the normal PR-test workflow.

Agent swarm is currently in beta and is not enabled until we turn it on for your project. Contact us before merging any workflow change and wait for confirmation that your project is opted in. If you add a workflow first, launches will fail until we enable it.

When you reach out, include your Meticulous org/project, which launch you want (see Choose how runs launch), whether you plan to use local mocks or a staging backend, and — if you need staging login — the details in Login setup.

Agent swarm can start a run in one of two ways. Pick one for your pull requests:

• Automatic launch (recommended). Meticulous starts a run from the build your PR test run already uploads, so you add nothing to CI. It works with uploaded static assets, uploaded containers (with network traffic mocked), and preview URLs. We turn it on for your project, for either the first commit of each PR or every PR commit.
• ci agent-test from CI. Your CI job builds the app and launches the run itself. Use it when the automatic launch can't cover your setup: the agent needs a staging backend (--backendUrl), it should test a different build from the one you replay, a run needs its own instructions file (--instructionsFile), or you want to launch runs on demand.

Agent swarm keeps one run per pull request: a newer run cancels any earlier one still in progress. With the automatic launch on and a ci agent-test job on pull_request, whichever launches second cancels the other, so each push builds twice for one result. If you need both, for example to test against staging on demand, trigger the ci agent-test workflow manually (workflow_dispatch only) and leave pull requests to the automatic launch.

With the automatic launch, follow Add agent instructions and Verify the first run, and skip the CI steps.

You need:

• A Meticulous project linked to the repository and opted in to the beta (see above).

For ci agent-test, you also need:

• A project-scoped API token stored in your CI provider as METICULOUS_API_TOKEN.
• A build command that produces a directory of static frontend assets.
• A GitHub Actions workflow (or equivalent CI job) that runs on each push to a pull request, plus a way to re-run manually.

Choose one environment for the agent:

The rest of this guide walks through the first two options.

Create and commit .meticulous/agent-swarm-instructions.md. Describe the app in terms the agent can act on: routes, important controls, test accounts, and the expected flows. Keep secrets out of this file.

Meticulous reads this file from the repository at the commit being tested for every Agent swarm run, whether it was launched from CI with ci agent-test or triggered automatically. This means a pull request can change its own instructions, but uncommitted changes are not visible to Meticulous.

For example:

# Storefront

Start at `/`. Browse the catalogue, add an item to the cart, and complete the checkout form. Use `test@example.com` and the test account details supplied by the environment. Also visit `/orders` and check empty and populated states.

Good instructions name the goal and the UI affordances that lead to it. They should also call out variants worth exercising, such as validation errors, filters, feature flags, or a dark-mode toggle.

If a first-time tour, tip, or consent banner can cover the application, name it and its visible dismiss control in the instructions. Agent swarm automatically closes common consent banners and vendor tours before starting a flow, and it remembers any other disposable overlay it has had to dismiss so later runs close it automatically too. To have an overlay closed from the very first run, add a data-meticulous-overlay-dismiss attribute to its dismiss control, or name the overlay and its dismiss control in the instructions. Do not tell the agent to close a dialog that is part of the behavior you want tested.

For applications with configured staging login, the most deterministic option is to complete the onboarding state as part of that project login setup. Meticulous captures the resulting cookies and local storage and seeds each case browser with them, so the popup never appears or adds a dismissal action to the recorded session. Tell us which tour-completion state or action is required when you send the login setup details.

Pass --instructionsFile <path> to use a different markdown file for one CLI-launched run. The override replaces .meticulous/agent-swarm-instructions.md; the two files are not merged. This is useful for environment- or mode-specific instructions.

If you already use .github/agent-review/instructions.md, move it to .meticulous/agent-swarm-instructions.md so automatically triggered runs can find it. Alternatively, keep passing the old path with --instructionsFile for CLI-launched runs.

Skip this step if you use the automatic launch.

Add a separate pull-request workflow, or a job after your existing build. The example below runs on every push to the PR, so each new commit is tested automatically. If you prefer fewer runs, restrict the trigger with types: [opened, reopened].

The following GitHub Actions job builds a static site in build/ and launches Agent swarm for the PR's head commit:

name: Agent swarm

on:
  pull_request:
  workflow_dispatch:

jobs:
  generate-sessions:
    runs-on: ubuntu-latest
    env:
      METICULOUS_API_TOKEN: ${{ secrets.METICULOUS_API_TOKEN }}
    steps:
      - uses: actions/checkout@v4
      - uses: pnpm/action-setup@v4
      - uses: actions/setup-node@v4
        with:
          node-version: "22"
          cache: pnpm
      - run: pnpm install --frozen-lockfile
      - run: pnpm build
      - name: Launch Agent swarm
        run: |
          npx -y @alwaysmeticulous/cli@latest ci agent-test \
            --assetsDir build \
            --commitSha "${{ github.event.pull_request.head.sha || github.sha }}"

Replace build with your build output directory. The command runs the agent in Meticulous; your CI runner does not need Docker or LLM credentials.

To launch a run manually at any time — for example to re-test a later commit when the trigger is restricted — use Actions → Agent swarm → Run workflow (the workflow_dispatch trigger).

On a pull_request event, github.sha can be GitHub's temporary merge commit. Pass github.event.pull_request.head.sha || github.sha so the generated sessions and their result are associated with the commit that Meticulous tests and displays for the PR.

To prove the command and paths are valid without launching an agent, append --dryRun. You can also run the same command locally after building, with METICULOUS_API_TOKEN exported.

A staging backend needs ci agent-test: the automatic launch runs against mocked network traffic.

If the uploaded frontend needs a live backend, serve a disposable staging environment over public HTTPS and proxy the frontend's relative API paths to it:

      - name: Launch Agent swarm with staging API
        env:
          METICULOUS_STAGING_USERNAME: ${{ secrets.STAGING_AGENT_USERNAME }}
          METICULOUS_STAGING_PASSWORD: ${{ secrets.STAGING_AGENT_PASSWORD }}
        run: |
          npx -y @alwaysmeticulous/cli@latest ci agent-test \
            --assetsDir frontend/dist \
            --backendUrl "https://staging.example.com" \
            --backendProxyPaths /api \
            --commitSha "${{ github.event.pull_request.head.sha || github.sha }}"

Your frontend must make same-origin requests under the configured prefixes (for example, fetch('/api/tasks')). --backendProxyPaths defaults to /api, so you can omit it when that is the only prefix you need. If your pages are rendered by the backend, pass --backendProxyPaths / to proxy every path, so pages and the assets they reference come from the same backend build. Absolute API URLs bypass this proxy; contact Meticulous to configure an egress policy for every required cross-origin host. The backend must be publicly reachable on HTTPS, accept the proxied requests, and use a disposable account because the agent can perform writes.

--backendUrl works with --localImageTag too. The staging origin is injected into the container at runtime, in the environment variable named by --backendUrlEnvVariable (default METICULOUS_BACKEND_URL); your image must use that variable to route its backend traffic, which is the preferred setup and the only one that covers server-side (SSR) fetches. Pass --backendProxyPaths to enable the experimental fronting server: the browser opens http://localhost:8000, backend paths go to staging and all other requests go to your container. Use --appPort to override its port. With --backendProxyPaths, requests reach your container with Host: localhost:8000 (or localhost:<appPort> for your chosen port), so allow localhost in any host allowlist, such as Django ALLOWED_HOSTS, Rails config.hosts, or Vite allowedHosts. Cookies and redirects work natively; frontend JavaScript can read cookies that are not HttpOnly, so use a disposable staging identity. The backend must be publicly reachable on HTTPS and use a disposable account because the agent can perform writes.

If the agent needs to sign in against your staging backend, we configure the login flow for your project — it is not something you set in CI. Send us the following before your first credentialed run (ideally when you request beta access):

You can paste this into Slack or email:

Project: <org/project>
Staging backend URL: https://…
Login type: username/password on app | SSO/IdP | other (describe)
Login URL/path: …
Username field: (default ok / selector: …)
Password field: (default ok / selector: …)
Submit: (default ok / selector or button text: …)
After login: lands on … / session cookie …
Post-login tour/tip: none | completion action/state: …
Test account: (username/password go in CI secrets only)
MFA/captcha on staging for this account: yes/no

Once we have configured login, provide credentials and any other login options only through METICULOUS_STAGING_* environment variables: METICULOUS_STAGING_USERNAME and METICULOUS_STAGING_PASSWORD, plus any flow-specific values we agree on — for example METICULOUS_STAGING_TOTP_SECRET (base32 TOTP seed) for an MFA flow, or METICULOUS_STAGING_SKIP_EMAIL_CLIENT_ID (trusted-automation client id) when the login flow must bypass an email verification challenge. If your backend can mint a sign-in link for the run, pass it as METICULOUS_STAGING_MAGIC_LINK_URL instead of a username and password; it must be on the same origin as --backendUrl. Every environment variable with the METICULOUS_STAGING_ prefix is forwarded to the login flow, so adding a new option never requires a CLI upgrade. Do not put these values in agent instructions or commit them to the repository.

1. Open a pull request with a small, visible change (or manually re-run the workflow).
2. With ci agent-test, confirm the Agent swarm job uploads the build and prints a workflow run identifier. With the automatic launch, the run starts once the PR's Meticulous test run has its build.
3. Open the Meticulous test run for the PR commit, then open Agent swarm to inspect the generated test cases, results, and screenshots.
4. Review the recorded flows and adjust .meticulous/agent-swarm-instructions.md if important routes or states were missed.

If the agent cannot reach an API request, first check that the frontend uses a relative URL under --backendProxyPaths, or ask Meticulous to add an egress policy for its cross-origin host; then check that the staging deployment is healthy and the test credentials can sign in. If a container returns DisallowedHost, "Blocked hosts", or a similar host-rejection error, check the localhost allowlist requirement under With a container build. For command syntax and the container option, see CLI commands.

When a flow can't be verified against your staging backend because of how the test environment is set up (for example missing test data, test-user permissions, a feature flag, sign-in, an unhealthy backend or an external dependency), Agent swarm retries it once with API responses served from recorded sessions. A flow that passes that retry still counts as a pass, but is labelled mocked backend and explained as "This test passed against a mocked backend": its frontend behaviour was verified, but your backend was not tested. The original setup blocker and its suggested fix stay attached to the flow, so you can still fix the environment.

Coding agents can read Agent swarm results through the Meticulous MCP server or CLI, using the same authentication as the other agent tools:

agent-swarm-run and agent-swarm-run-case pick a run the way the other test-run commands do: by swarmRunId, or the latest execution run for a testRunId, commitSha or prNumber (its head commit). The CLI defaults to the current git HEAD; MCP tools need one of them explicitly. With --json, each CLI command prints the same JSON as its MCP tool. While a run is still in progress, agent-swarm-run and agent-swarm-run-case wait up to 10 minutes for it to finish, and their MCP tools return { status: "processing", message } until it has.

The failure checker's review quotes your source code, so it is returned only when the caller may read the project's code. If source-code access is disabled for the project, or the signed-in user can't view its code, the review and fix prompt are withheld and checkWithheld says why. A case's status already reflects the checker's verdict either way.