> Meticulous records real user sessions from a web app and replays them against each commit to catch visual regressions.
>
> Full AI-readable docs index (complete project setup journey + every page as markdown): https://app.meticulous.ai/llms.txt

# Writing a custom check

This guide walks through building a **network request capacity check** from scratch: a custom check that makes sure sessions on the head test run do not issue many more network requests than they did on the base.

It is organised in four sections, mirroring what every custom check does:

1. **Read the snapshot data** captured during replay.
2. **Compute the result** of the check by comparing base and head.
3. **Report the result** back to Meticulous.
4. **Set up CI** so the check runs on every PR.

Everything lives in a single `report.ts` file that we build up as we go.

A complete, runnable version of this check lives in the [`custom-checks-examples`](https://github.com/alwaysmeticulous/custom-checks-examples) repository.

## What we will build

Each recorded session has a baseline on the base test run — the number of meaningful HTTP requests that session normally issues. That baseline is the session's **capacity**. On every PR we compare head against base per session, and if head exceeds capacity by too much the check surfaces a warning or failure before the change merges.

## Prerequisites

Before following this guide, make sure you have:

- A Meticulous project with [CI tests set up](/docs/ci) so every PR produces head and base test runs.
- Custom checks enabled for your project. Contact the Meticulous team to enable the custom checks.
- Your Meticulous API token.

## Pick an example test run to develop against

Before writing any code, choose a **completed test run** to use as a running example while you build the reporter. You will point the script at it repeatedly to check that your filtering, thresholds, and report read the way you expect. Grab its id from the test run's URL in the Meticulous app — `https://app.meticulous.ai/projects/<org>/<project>/test-runs/<testRunId>`.

It helps to pick a test run that actually exhibits the regression you are checking for, so you can confirm the check *fires*, not just that it runs. For a network request capacity check, a simple way to produce one is to open a PR that deliberately increases network traffic — for example, adding a short polling interval that re-fetches an endpoint — and use the test run associated with that PR as your example. Once the check is working, you can drop the PR.

## 1. Read the snapshot data

### Set up the reporter project

A custom check is just a small Node script — the **reporter** — that you run after a Meticulous test run finishes. It talks to Meticulous through the published [`@alwaysmeticulous/custom-checks`](https://www.npmjs.com/package/@alwaysmeticulous/custom-checks) SDK, which handles authentication, resolving test runs, downloading data, and posting results.

Create a dedicated directory (for example `custom-checks/` at the repo root) with its own `package.json` and a single `report.ts` we will build up as we go:

```json
{
  "name": "my-custom-checks",
  "private": true,
  "scripts": {
    "report": "ts-node report.ts"
  },
  "dependencies": {
    "@alwaysmeticulous/custom-checks": "^2.296.0"
  },
  "devDependencies": {
    "ts-node": "^10.8.1",
    "typescript": "^5.9.3"
  }
}
```

Use the latest [`@alwaysmeticulous/custom-checks`](https://www.npmjs.com/package/@alwaysmeticulous/custom-checks) release from [npm](https://www.npmjs.com/package/@alwaysmeticulous/custom-checks) — the version above is current at the time of writing. Install dependencies with your package manager (`npm install`, `pnpm install`, etc.).

The [`custom-checks-examples`](https://github.com/alwaysmeticulous/custom-checks-examples) repo has this project ready to run if you'd rather skip the boilerplate.

### Resolve a test run

Before computing anything, get a minimal `report.ts` running that connects to Meticulous and resolves your example test run. This confirms your token and SDK setup work before you add any check logic.

Two SDK functions get us started:

- `createClient(...)` opens an authenticated connection to Meticulous from your API token. Every other SDK call takes the client it returns.
- `findTestRunForCustomChecks(...)` takes a test run id and returns the resolved run together with its **base** run (waiting for the run to finish if it is still in progress). That head/base pair is what a check compares.

Start with all the imports we will need across the script:

```typescript
import {
  createClient,
  findTestRunForCustomChecks,
  findTestRunByCommitForCustomChecks,
  getSnapshotsFromTestRun,
  reportCustomCheckResults,
  type MeticulousClient,
  type Snapshot,
  type CustomCheckVerdict,
  type ReportedCustomCheckResult,
} from "@alwaysmeticulous/custom-checks";

const testRunId = process.argv[2];

const client = createClient({
  apiToken: process.env.METICULOUS_API_TOKEN,
  appInfo: "my-app/custom-checks",
});

const { testRun } = await findTestRunForCustomChecks({
  client,
  testRunId,
  // We are only exploring here, not reporting — don't register this run as
  // expecting custom checks (explained under "Report the result").
  skipRegisteringExpectedCustomChecks: true,
});

console.log(`Resolved test run ${testRun.id} (${testRun.status}): ${testRun.url}`);
```

Run it against the example test run you picked earlier:

```shell
METICULOUS_API_TOKEN=<token> pnpm run report -- <testRunId>
```

We pass `skipRegisteringExpectedCustomChecks: true` here because we are only exploring and will not report results — the *Report the result* section explains this flag. Everything from here on is added to this same `report.ts` file.

### Understand snapshots

The data a custom check compares comes from **snapshots**. While Meticulous replays each session — on both the head and base deployments — it records structured data points called snapshots and stores them alongside the replay. Each snapshot is tagged with:

- `sessionId` — which session it was captured in, so you can align the same session on base and head. A session is essentially a recorded user flow that Meticulous replays.
- `sessionDescription` — a short, human-readable summary of what the user was doing in that session (for example `Added an item to the cart`), useful for labelling sessions in your report. It is `null` when the session has no description, so treat it as optional.
- `type` — the kind of snapshot (for example `network-requests`).
- `stageDuringSession` — which screenshot in the session timeline the data belongs to.
- `data` — the payload, whose shape depends on the snapshot type.

Some snapshot types are **built in** and need no application code — Meticulous captures them automatically. The built-in types today are `network-requests` (every `fetch` / XHR a session made), `js-bundle-sizes` (the JavaScript bundles it loaded), and `react-component-renders` (a per-component breakdown of which components re-rendered). You can also record your own from browser code; see [Built-in snapshot types](/docs/custom-checks/built-in-snapshot-types) and [recording custom snapshots](/docs/custom-checks/recording-custom-data).

This check uses `network-requests`. Each such snapshot's `data` describes a single request:

```typescript
interface NetworkRequestSnapshotData {
  url: string;
  method: string;
  requestBody?: string;
  status: number | null;
  responseBody?: string;
  // ...plus request/response headers and other metadata
}
```

So one session that made 12 requests produces 12 `network-requests` snapshots, all sharing that session's `sessionId`.

### Fetch the snapshots

The SDK function for reading snapshot data is `getSnapshotsFromTestRun(...)`. You pass it the client, a test run id, and the `snapshotTypes` you care about; it downloads every matching snapshot for both the head run and its base and returns them as two arrays, `baseSnapshots` and `headSnapshots`. This is the entry point for *any* check, whatever snapshot type it reads.

Add the snapshot type constant and a `computeNetworkRequestsCheck` function near the top of `report.ts`, above the resolving code you added in *Resolve a test run*. For now it just fetches and logs how much data we got:

```typescript
const NETWORK_REQUESTS_SNAPSHOT_TYPE = "network-requests";

const computeNetworkRequestsCheck = async (
  client: MeticulousClient,
  testRunId: string,
): Promise<void> => {
  const { baseSnapshots, headSnapshots } = await getSnapshotsFromTestRun({
    client,
    testRunId,
    snapshotTypes: [NETWORK_REQUESTS_SNAPSHOT_TYPE],
  });

  console.log(
    `Fetched ${baseSnapshots.length} base and ${headSnapshots.length} head snapshots.`,
  );
};
```

Then call it after resolving the run:

```typescript
await computeNetworkRequestsCheck(client, testRun.id);
```

Re-run the script — you should see non-zero counts. For large test runs this download can take some time, since one snapshot is written per captured request across every session on both base and head. We will change the return type to a check result once we have something to report.

## 2. Compute the result of the check

### Define the capacity model

We now know the raw shape of the data. Next, capture the rules of the check as types and constants so the comparison logic stays readable. Add these near the top of `report.ts`:

```typescript
/** Stable id shown in the Meticulous UI. */
const CHECK_ID = "network-requests";

/** Surface a non-blocking warning once a session exceeds base capacity by this much. */
const WARN_PERCENT_INCREASE_THRESHOLD = 10;
/** Require reviewer acknowledgement once a session exceeds base capacity by this much. */
const REQUIRE_ACK_PERCENT_INCREASE_THRESHOLD = 20;

/** Ignore low-traffic sessions where +1 request is noise. */
const MIN_REQUESTS_FOR_ALARM = 3;

interface EndpointComparison {
  label: string;
  baseCount: number;
  headCount: number;
  delta: number;
}

interface SessionComparison {
  sessionId: string;
  // Short description of what the user did in the session (e.g. "Added an item
  // to the cart"), used to label the session in the report. `null` when the
  // session has no description.
  sessionDescription: string | null;
  baseCount: number;
  headCount: number;
  delta: number;
  percentIncrease: number;
  endpoints: EndpointComparison[];
}
```

A custom check reports one of three verdicts, typed by the SDK as `CustomCheckVerdict`:

- `pass` — no regression; the check is green and no report is surfaced.
- `warn-without-requiring-user-ack` — surfaces a report in the Meticulous UI as a signal for reviewers to glance at, but does **not** block the pull request or require anyone to act on it.
- `warn-and-require-user-ack` — surfaces a report that a reviewer must explicitly acknowledge (review) in the Meticulous UI before the run is considered actioned, the same way an unreviewed visual diff blocks until someone accepts or ignores it.

Note there is no `fail` verdict: instead of failing outright, a check escalates by *requiring acknowledgement*. (A check that errors while *running* is a separate, run-level concern, not a verdict.)

That distinction drives the two thresholds here. We surface a non-blocking warning once a session exceeds its base capacity by 10%, and require acknowledgement at 20% — reserving the acknowledgement-required verdict for clear, actionable regressions so a single retried API call does not gate a PR. `SessionComparison` and `EndpointComparison` are the per-session and per-endpoint breakdown we will produce next.

### Filter out noise

Not every HTTP request reflects your application's behaviour. Analytics beacons, error trackers, font CDNs, and the Meticulous recorder itself all issue requests during replay. Counting them would let a session "regress" on telemetry noise rather than on product traffic.

Each downloaded snapshot is a `Snapshot` — the SDK type for the data points from *Understand snapshots*, carrying `sessionId`, `sessionDescription`, `type`, `stageDuringSession`, and a `data` field typed as `unknown` (the SDK does not know the shape of every snapshot type). So declare the minimal shape you need and decide which requests are meaningful:

```typescript
interface NetworkRequestData {
  url?: string;
  method?: string;
  requestBody?: string;
}

const networkRequestData = (snapshot: Snapshot): NetworkRequestData =>
  (snapshot.data ?? {}) as NetworkRequestData;

/** Hostname substrings to exclude from the comparison. */
const IGNORED_REQUEST_HOST_SUBSTRINGS = [
  "sentry.io",
  "segment.io",
  "google-analytics.com",
  "googletagmanager.com",
  // Add the third-party hosts your app loads during replay.
];

const isMeaningfulRequest = (snapshot: Snapshot): boolean => {
  const { url } = networkRequestData(snapshot);
  if (!url) {
    return true;
  }
  try {
    const hostname = new URL(url).hostname.toLowerCase();
    return !IGNORED_REQUEST_HOST_SUBSTRINGS.some((needle) =>
      hostname.includes(needle),
    );
  } catch {
    // Relative URLs (e.g. "/api/graphql") are same-origin app traffic.
    return true;
  }
};
```

Tailor `IGNORED_REQUEST_HOST_SUBSTRINGS` to your stack. Same-origin and relative URLs should always count as meaningful.

### Count requests per session

To explain *which* endpoints regressed, group meaningful requests by a human-readable label, then bucket them into per-session, per-endpoint counts:

```typescript
/** GraphQL calls by operation name; everything else as `METHOD /path`. */
const describeRequest = (snapshot: Snapshot): string => {
  const { url, method } = networkRequestData(snapshot);
  if (!url) {
    return "(unknown request)";
  }
  const verb = (method ?? "GET").toUpperCase();
  try {
    const { pathname } = new URL(url);
    return `${verb} ${pathname}`;
  } catch {
    return `${verb} ${url.split("?")[0]}`;
  }
};

type RequestCountsByEndpoint = Map<string, number>;

const countRequestsBySession = (
  snapshots: Snapshot[],
): Map<string, RequestCountsByEndpoint> => {
  const countsBySession = new Map<string, RequestCountsByEndpoint>();
  for (const snapshot of snapshots) {
    if (snapshot.type !== NETWORK_REQUESTS_SNAPSHOT_TYPE) continue;
    if (!isMeaningfulRequest(snapshot)) continue;

    let byEndpoint = countsBySession.get(snapshot.sessionId);
    if (!byEndpoint) {
      byEndpoint = new Map();
      countsBySession.set(snapshot.sessionId, byEndpoint);
    }
    const label = describeRequest(snapshot);
    byEndpoint.set(label, (byEndpoint.get(label) ?? 0) + 1);
  }
  return countsBySession;
};
```

### Compare head against base capacity per session

Align sessions by `sessionId` and compare each session's head request count against its base capacity. Skip sessions that only ran on head — they have no baseline capacity to compare against:

```typescript
const sumCounts = (byEndpoint: RequestCountsByEndpoint): number =>
  [...byEndpoint.values()].reduce((total, count) => total + count, 0);

const compareEndpoints = (
  base: RequestCountsByEndpoint,
  head: RequestCountsByEndpoint,
): EndpointComparison[] => {
  const labels = new Set([...base.keys(), ...head.keys()]);
  return [...labels].map((label) => {
    const baseCount = base.get(label) ?? 0;
    const headCount = head.get(label) ?? 0;
    return { label, baseCount, headCount, delta: headCount - baseCount };
  });
};

// First non-null `sessionDescription` seen per `sessionId`, so each session can
// be labelled by what the user did rather than by its opaque id.
const collectSessionDescriptions = (
  ...snapshotLists: Snapshot[][]
): Map<string, string | null> => {
  const descriptions = new Map<string, string | null>();
  for (const snapshots of snapshotLists) {
    for (const snapshot of snapshots) {
      const existing = descriptions.get(snapshot.sessionId);
      if (existing == null) {
        descriptions.set(snapshot.sessionId, snapshot.sessionDescription ?? null);
      }
    }
  }
  return descriptions;
};

const compareSessions = (
  baseSnapshots: Snapshot[],
  headSnapshots: Snapshot[],
): SessionComparison[] => {
  const baseCounts = countRequestsBySession(baseSnapshots);
  const headCounts = countRequestsBySession(headSnapshots);
  const descriptions = collectSessionDescriptions(baseSnapshots, headSnapshots);
  const comparisons: SessionComparison[] = [];

  for (const sessionId of headCounts.keys()) {
    // Only compare sessions that also ran on base.
    if (!baseCounts.has(sessionId)) continue;

    const baseByEndpoint = baseCounts.get(sessionId) ?? new Map();
    const headByEndpoint = headCounts.get(sessionId) ?? new Map();
    const baseCount = sumCounts(baseByEndpoint);
    const headCount = sumCounts(headByEndpoint);

    comparisons.push({
      sessionId,
      sessionDescription: descriptions.get(sessionId) ?? null,
      baseCount,
      headCount,
      delta: headCount - baseCount,
      percentIncrease:
        baseCount === 0
          ? headCount > 0
            ? Infinity
            : 0
          : ((headCount - baseCount) / baseCount) * 100,
      endpoints: compareEndpoints(baseByEndpoint, headByEndpoint),
    });
  }

  return comparisons.sort((a, b) => b.delta - a.delta);
};
```

### Decide whether capacity was exceeded

Every check must end on a single verdict, typed by the SDK as `CustomCheckVerdict` — the union `"pass" | "warn-without-requiring-user-ack" | "warn-and-require-user-ack"`. Turn the per-session comparisons into one of those values: did any session exceed its base capacity beyond the warn or acknowledgement thresholds? Use integer arithmetic on counts so boundary conditions (exactly +10% or +20%) are stable:

```typescript
const hasEnoughTraffic = (c: SessionComparison) =>
  Math.max(c.baseCount, c.headCount) >= MIN_REQUESTS_FOR_ALARM;

const requiresAck = (c: SessionComparison) =>
  hasEnoughTraffic(c) &&
  c.baseCount > 0 &&
  c.headCount * 100 >=
    c.baseCount * (100 + REQUIRE_ACK_PERCENT_INCREASE_THRESHOLD);

const isWarning = (c: SessionComparison) => {
  if (!hasEnoughTraffic(c) || c.delta <= 0 || requiresAck(c)) return false;
  if (c.baseCount === 0) return true; // new meaningful traffic on head
  return (
    c.headCount * 100 >=
    c.baseCount * (100 + WARN_PERCENT_INCREASE_THRESHOLD)
  );
};

const computeVerdict = (
  comparisons: SessionComparison[],
): CustomCheckVerdict => {
  if (comparisons.some(requiresAck)) return "warn-and-require-user-ack";
  if (comparisons.some(isWarning)) return "warn-without-requiring-user-ack";
  return "pass";
};
```

### Build a report and return the result

A verdict on its own does not tell a reviewer *why* the check failed. Meticulous renders a markdown report in the Checks tab, so build one that lists the sessions that exceeded capacity, with a per-endpoint table showing where the extra requests came from.

Because this check compares **per session**, link each session in the report to its view in the Meticulous app. That page shows the base vs head comparison for the session — screenshots, timeline, and simulation details — so reviewers can see what changed without hunting for the session id. Label the link with the session's `sessionDescription` (e.g. "Added an item to the cart") so reviewers recognise the flow at a glance, falling back to `session1`, `session2`, … by position when a session has no description. The URL pattern is:

```
https://app.meticulous.ai/projects/<organization>/<project>/test-runs/<testRunId>/sessions/<sessionId>
```

`<organization>` and `<project>` are the URL slugs from your project's path, `<testRunId>` is the head test run you are reporting against, and `<sessionId>` is the session id from the snapshot data.

```typescript
const PROJECT_PATH =
  "https://app.meticulous.ai/projects/<organization>/<project>";

const sessionUrl = (testRunId: string, sessionId: string): string =>
  `${PROJECT_PATH}/test-runs/${testRunId}/sessions/${sessionId}`;

const buildReport = (
  verdict: CustomCheckVerdict,
  comparisons: SessionComparison[],
  testRunId: string,
): string => {
  const alarming = comparisons.filter((c) => requiresAck(c) || isWarning(c));
  const lines = [
    "# Network request capacity",
    "",
    `**Verdict:** ${verdict}`,
    "",
  ];

  alarming.forEach((comparison, index) => {
    // Prefer the session's recorded description; fall back to its position when
    // the session has none.
    const label = comparison.sessionDescription ?? `session${index + 1}`;
    lines.push(
      `## [${label}](${sessionUrl(testRunId, comparison.sessionId)}) — ${comparison.baseCount} → ${comparison.headCount} requests`,
      "",
      "| Endpoint | Base | Head | Δ |",
      "| --- | ---: | ---: | ---: |",
    );
    for (const endpoint of comparison.endpoints.filter((e) => e.delta > 0)) {
      lines.push(
        `| ${endpoint.label} | ${endpoint.baseCount} | ${endpoint.headCount} | +${endpoint.delta} |`,
      );
    }
    lines.push("");
  });

  return lines.join("\n");
};
```

A check hands its outcome back as a `ReportedCustomCheckResult` — the SDK shape that pairs your `checkId` with the `verdict`, a one-line `summary` shown inline in the UI, and a `report` (here `{ type: "markdown", markdown }`). Every check, whatever it measures, returns this same shape.

Now finish the `computeNetworkRequestsCheck` function we started in *Fetch the snapshots*. Change its return type to `Promise<ReportedCustomCheckResult>` and, after fetching the snapshots, compute the comparisons, verdict, and report:

```typescript
const computeNetworkRequestsCheck = async (
  client: MeticulousClient,
  testRunId: string,
): Promise<ReportedCustomCheckResult> => {
  const { baseSnapshots, headSnapshots } = await getSnapshotsFromTestRun({
    client,
    testRunId,
    snapshotTypes: [NETWORK_REQUESTS_SNAPSHOT_TYPE],
  });

  const comparisons = compareSessions(baseSnapshots, headSnapshots);
  const verdict = computeVerdict(comparisons);

  return {
    checkId: CHECK_ID,
    verdict,
    summary:
      verdict === "pass"
        ? "No sessions exceeded their network request capacity"
        : `${comparisons.filter(requiresAck).length || comparisons.filter(isWarning).length} session(s) exceeded network request capacity`,
    report: {
      type: "markdown",
      markdown: buildReport(verdict, comparisons, testRunId),
    },
  };
};
```

## 3. Report the result

### Run the reporter locally

Two more SDK functions complete the round trip:

- `findTestRunByCommitForCustomChecks(...)` resolves a run from a **commit SHA** (waiting for it to finish). It is the CI-friendly counterpart to `findTestRunForCustomChecks`, which takes a run id — CI usually knows the commit, not the run id.
- `reportCustomCheckResults(...)` posts your computed results back to Meticulous in a single call (more on the "single call" rule below).

Wire the bottom of `report.ts` to resolve the test run, compute the check, and either print the result (while iterating) or post it back to Meticulous. Accept **either** a positional test run id (handy while developing against your example run) **or** `--commitSha` (what CI uses). Replace the resolving block from *Resolve a test run* with:

```typescript
const dryRun = process.argv.includes("--dryRun");

const resolveTestRun = async () => {
  const commitShaFlagIndex = process.argv.indexOf("--commitSha");
  const commitSha =
    commitShaFlagIndex !== -1
      ? process.argv[commitShaFlagIndex + 1]
      : undefined;

  if (commitSha) {
    return findTestRunByCommitForCustomChecks({
      client,
      commitSha,
      // On a dry run we are only experimenting, so don't tell the backend that
      // results are coming for this run (see below).
      skipRegisteringExpectedCustomChecks: dryRun,
    });
  }

  // First positional argument after `pnpm run report --`.
  const testRunId = process.argv
    .slice(2)
    .find((arg) => arg !== "--" && !arg.startsWith("-"));
  if (testRunId) {
    return findTestRunForCustomChecks({
      client,
      testRunId,
      skipRegisteringExpectedCustomChecks: dryRun,
    });
  }

  throw new Error(
    "Pass a testRunId argument or --commitSha to identify the test run.",
  );
};

const { testRun } = await resolveTestRun();

const checks = [await computeNetworkRequestsCheck(client, testRun.id)];

if (dryRun) {
  for (const check of checks) {
    console.log(`${check.checkId}: ${check.verdict}\n${check.report.markdown}`);
  }
} else {
  await reportCustomCheckResults({
    client,
    testRunId: testRun.id,
    results: { status: "complete", checks },
  });
}
```

Both `findTestRunForCustomChecks` and `findTestRunByCommitForCustomChecks` **register** the run as expecting custom check results by default — that is what makes the **Checks** tab appear in the Meticulous UI and show a "waiting for checks" state until you report. On a real run that is exactly what you want. But while iterating locally with `--dryRun` you are not going to report anything, so registering would leave that run stuck showing a "waiting for checks" tab that never resolves.

Pass `skipRegisteringExpectedCustomChecks: true` to suppress that signal during dry runs. Tying it to the `dryRun` flag (as above) means real runs still register and report normally, while local experiments stay invisible. Note that calling `reportCustomCheckResults` always marks the run regardless, so this only matters on the dry-run path that never reports.

While building the check, keep using your example test run id:

```shell
METICULOUS_API_TOKEN=<token> pnpm run report -- <testRunId> --dryRun
```

In CI you pass the commit SHA instead (see *Set up CI* below). Read the printed verdict and markdown carefully before reporting for real — confirm that the sessions linked, thresholds, and endpoint breakdowns all make sense for the change under review. When you are satisfied, drop `--dryRun` to report results to Meticulous.

If anything misbehaves, compare against the runnable version in the [`custom-checks-examples`](https://github.com/alwaysmeticulous/custom-checks-examples) repo.

### Report all checks in a single result

A test run accepts custom check results **exactly once**. Every check you run must be computed and submitted together in one `reportCustomCheckResults` call, with all of them in the `checks: [...]` array.

You **cannot** report checks separately — for example, posting the network requests result from one CI job and the bundle-size result from another. A test run only accepts one set of results, so a second report is rejected.

So when you add more checks later, compute them all in `report.ts` and send the complete set in a single API call — one script (or one CI step) that computes every check, then includes every result in the single `checks` array passed to `reportCustomCheckResults`.

### Viewing results

Open the test run in the Meticulous app and select the **Checks** tab. Each reported check shows its verdict, one-line summary, and rendered markdown report. For checks that require acknowledgement, reviewers can accept or ignore them from the UI, similar to reviewing visual diffs.

## 4. Set up CI

Add the reporter as a CI step **after** the step that kicks off your Meticulous test run. When resolved by commit SHA, the reporter waits for the test run to complete before computing and reporting results, so it can run right alongside the rest of your pipeline. A typical GitHub Actions step:

```yaml
- name: Report custom checks
  if: always()
  working-directory: custom-checks
  env:
    METICULOUS_API_TOKEN: ${{ secrets.METICULOUS_API_TOKEN }}
  run: pnpm run report -- --commitSha ${{ github.sha }}
```

> **warning: Which commit SHA to pass on GitHub**
> `${{ github.sha }}` is usually **not** the tip of the PR branch. For `pull_request` workflows it is the temporary **merge commit** GitHub creates between your branch and the base branch — and that is the commit GitHub Actions checks out by default. Meticulous associates the test run with that SHA, so pass `${{ github.sha }}` here rather than the PR head commit shown in the GitHub UI. On other CI providers, pass the commit SHA your Meticulous upload step used.

Place this in the same workflow that triggers Meticulous tests (see [GitHub Actions setup](/docs/github-actions-v2)). The job needs the same API token you use for CI uploads.

## Choosing a comparison granularity

This check compares **per session**: it aligns each session's snapshots on base and head by `sessionId` and judges every session independently. That suits network request capacity, where each user flow has its own expected amount of traffic.

Other checks may want a different granularity:

- **Across the whole test run.** Aggregate the data over every replay without distinguishing sessions — for example, the total JavaScript bundle size shipped, or the set of unique endpoints called anywhere in the run — and compare the base aggregate against the head aggregate.
- **Per phase.** Use each snapshot's `stageDuringSession` to compare only the events that fired before a particular screenshot — for example, the requests made during page load versus after a specific interaction. This catches regressions localised to one part of a flow that a whole-session total would average out.

Pick whichever granularity makes the regression you care about easiest to detect and explain.