Setting up GitHub actions

This doc explains how to setup reference simulations which get tested with GitHub actions to detect major regressions.

1. Setup a reference simulation

Simulate a reference session testing a critical flow of your app, e.g. signing in, and save a reference simulation:

Simulate session

Assuming the dev app is served at http://localhost:3000/ and that the session id is 2022-02-28T13:47:34.163Z_ADvadSSEdvca

npm

npx meticulous simulate --apiToken="<API_TOKEN>" \
  --sessionId="2022-02-28T13:47:34.163Z_ADvadSSEdvca" \
  --appUrl="http://localhost:3000/" \
  --screenshot \
  --save

yarn

yarn meticulous simulate --apiToken="<API_TOKEN>" \
  --sessionId="2022-02-28T13:47:34.163Z_ADvadSSEdvca" \
  --appUrl="http://localhost:3000/" \
  --screenshot \
  --save

This should create a meticulous.json configuration file with the following content:

{
  "testCases": [
    {
      "sessionId": "2022-02-28T13:47:34.163Z_ADvadSSEdvca",
      "baseReplayId": "BmQYthZKWheJxWQEYdT8H"
    }
  ]
}

You can rename the test case for better readability:

{
  "testCases": [
    {
      "title": "Sign in flow",
      "sessionId": "2022-02-28T13:47:34.163Z_ADvadSSEdvca",
      "baseReplayId": "BmQYthZKWheJxWQEYdT8H"
    }
  ]
}

You should save and commit the meticulous.json file to enable testing and diffing.

The meticulous.json file should be located at the root of the repository, alongside package.json.

2. Run tests for new pull requests with GitHub actions

With at least one reference simulation saved in meticulous.json, we can use them to test their end state for visual regressions:

Run Meticulous tests

Assuming the dev app is served at http://localhost:3000/

npm

npx meticulous run-all-tests --apiToken="<API_TOKEN>" \
  --appUrl=http://localhost:3000/ \
  --headless

yarn

yarn meticulous run-all-tests --apiToken="<API_TOKEN>" \
  --appUrl=http://localhost:3000/ \
  --headless

Now we can run this command inside a GitHub workflow.

Here is a template you can add to your repository, at .github/workflows/meticulous-simulate.yaml.

# Workflow to simulate reference user sessions with Meticulous

name: Meticulous Simulations

on:
  push:
    branches:
      - main
  pull_request:
    branches:
      - main

jobs:
  simulate:
    runs-on: ubuntu-latest

    steps:
      - name: Checkout repository
        uses: actions/checkout@v2

      - name: Use Node.js LTS
        uses: actions/setup-node@v2
        with:
          node-version: "16"

      - name: Install dependencies
        run: npm ci

      - name: Simulate
        run: |
          npm run start &
          pid=$!
          sleep 5
          npx meticulous run-all-tests --appUrl="http://localhost:3000" --headless --githubSummary
          kill "${pid}"
        env:
          METICULOUS_API_TOKEN: ${{ secrets.METICULOUS_API_TOKEN }}

Adapt this template to your project, e.g. using yarn instead of npm or change the script used to start the local frontend server.