Setting up Meticulous tests to run in your CI provider

Visit https://github.com/apps/alwaysmeticulous/installations/new to install our GitHub App.

Select the project below that contains the sessions you wish to simulate, copy and paste the API token, and add it to your GitHub repo as a secret named METICULOUS_API_TOKENCopy to clipboard:

<API_TOKEN>

Be very careful with this API token, since it allows the holder access to your recorded sessions.

To run Meticulous on CI add a new .github/workflows/meticulous.yamlCopy to clipboard file, or, if you already use GitHub Actions, you can add it as a job to a an existing workflow. The workflow needs to run on both pushes to your main branch and on pull requests.

This workflow file should use the alwaysmeticulous/report-diffs-action/cloud-computeCopy to clipboard action to run your tests. This can be either against a local build of your app (which should be served using a non-development server) or against the latest preview URL for the commit (if you use a service such as Vercel or Netlify that generates preview URLs).

Note: If you are running the action against a local build of your app, you only need to build and serve frontend assets. Because Meticulous mocks out network requests by default, you don't need to run your backend server. You can read more about how Meticulous handles network requests here.

If you hit any issues then book a call with us and we'll help you get set up.

See below for an example workflow file, which you can add to your repo as '.github/workflows/meticulous.yaml'. Note that you'll need to update it to build and serve your frontend app.

File name: .github/workflows/meticulous.yamlCopy to clipboard.

File contents:

# Workflow for serving app locally & running Meticulous tests against it

name: Meticulous

# Important: The workflow needs to run both on pushes to your main branch and on
# pull requests. It needs to run on your main branch because it'll use the results
# from the base commit of the PR on the main branch to compare against.
on:
  push:
    branches:
      - main
  pull_request: {}
  # Important: We need the workflow to be triggered on workflow_dispatch events,
  # so that Meticulous can run the workflow on the base commit to compare
  # against if an existing workflow hasn't run
  workflow_dispatch: {}

# Important: The workflow needs all the permissions below.
# These permissions are mainly need to post and update the status check and
# feedback comment on your PR. Meticulous won’t work without them.
permissions:
  actions: write
  contents: read
  issues: write
  pull-requests: write
  statuses: read

jobs:
  test:
    name: Report diffs
    runs-on: ubuntu-latest

    steps:
      - name: Checkout repository
        uses: actions/checkout@v4

      - name: Use Node.js LTS
        uses: actions/setup-node@v4
        with:
          node-version: "20"
          cache: yarn

      - name: Cache node_modules
        uses: actions/cache@v4
        with:
          path: node_modules
          key: node-modules-${{ runner.os }}-${{ hashFiles('**/yarn.lock') }}
          restore-keys: |
            node-modules-${{ runner.os }}

      - name: Install dependencies
        run: |
          yarn install --frozen-lockfile --non-interactive

      - name: Build project
        run: |
          yarn build

      - name: Serve project
        # TODO: Update these commands to serve your app's frontend locally, and
        # then update the app-url to match.
        # We strongly recommend you serve the app with a non-development server.
        # For instance, if you are using Vite, use 'vite preview' instead of 'vite'
        # or 'vite serve' to spin up the app.
        # The sleep is often required to ensure your app is readily being served
        # by the time the Meticulous tests start
        run: |
          yarn serve &
          sleep 5

      - name: Run Meticulous tests
        uses: alwaysmeticulous/report-diffs-action/cloud-compute@v1
        with:
          api-token: ${{ secrets.METICULOUS_API_TOKEN }}
          # TODO: Update the port and protocol below to match your app's frontend
          app-url: "http://localhost:3000/"

Make sure that the 'Run Meticulous tests in the cloud' toggle is enabled in the 'CI & Execution' section of Meticulous settings for your project.

Create a new pull request to add the above workflow. Then validate that Meticulous is able to access your application correctly and is successfully simulating sessions by checking the test run linked to from a comment on the PR.

Sometimes running your application in CI might require setting up additional environment variables or configuration. Not having these set up correctly can cause Meticulous to fail to simulate sessions. This usually manifests as your application erroring, blank screens, or similar issues when you view the test run.

You can create your pull request with [meticulous debug]Copy to clipboard in the PR title to keep the secure tunnel open for debugging. Meticulous will post a comment on your PR on how to access the secure tunnel to debug your application setup.

Merge the PR to add the above workflow. You won't see any results on the PR that adds the workflow because you need to wait for the workflow to run on your main branch for it to detect any diffs.

Once the PR has merged and Meticulous has run on your base branch you can open a new PR to test Meticulous. It should post a comment if your new PR changed any of the screens or logic for the workflows you've recorded sessions for:

You can click the link in the comment to review the diffs before merging your PR.

If you've installed the Meticulous GitHub App Meticulous will add a check on your PR that is red if there are diffs that haven't been approved yet and becomes green once you click the green 'Approve all Visual Differences' button. This button can be found by clicking on the link in the Meticulous comment.

If you wish, you can make this check blocking by following the instructions here. Doing so will prevent developers from merging a PR which has visual differences until they have clicked the button to acknowledge the differences.