Ensuring Base Test Runs are Available

When running Meticulous tests in CI, Meticulous compares the test run on your current commit (the "head" commit) against a test run on the base commit (typically the commit on your main branch that your PR branches from). If no base test run exists yet, Meticulous cannot compute visual differences.

The ci prepare command helps ensure that a base test run is available before running your tests. If no base test run exists, this command will automatically trigger one using a script you provide.

You should use ci prepare if you have CI workflows where base test runs might not be automatically created. If you're using GitHub Actions with the standard Meticulous setup, base test runs are created automatically, and you do not need this command.

The ci prepare command:

1. Checks if a base test run exists for the base commit of your PR
2. If no base test run exists, it executes your provided trigger script to create one

You then run ci run-with-tunnel with the --hadPreparedForTests flag, which signals that the command should wait for the base test run to be available before proceeding.

The command is meant to be used in your CI workflow before running tests:

# First, prepare and ensure base test run exists
npx @alwaysmeticulous/cli ci prepare \
  --headCommit <commit-sha> \
  --triggerScript <path-to-trigger-script>

# Build your application
# (your build commands here)

# Run Meticulous tests with the --hadPreparedForTests flag
npx @alwaysmeticulous/cli ci run-with-tunnel \
  --commitSha <commit-sha> \
  --appUrl <your-app-url> \
  --hadPreparedForTests

Your trigger script should accept a commit SHA as its first argument and should:

1. Check out the commit
2. Build the application for that commit
3. Run Meticulous tests for that commit

Here's an example trigger script:

#!/bin/bash

# The commit SHA is passed as the first argument
BASE_COMMIT=$1

# Clone or navigate to your repository
cd /tmp
git clone <your-repo-url>
cd <your-repo-name>

# Check out the base commit
git checkout "$BASE_COMMIT"

# Install dependencies, build, and start the application in background
npm install
npm run build
npm run preview &

# Run Meticulous tests for this commit
npx @alwaysmeticulous/cli ci run-with-tunnel \
  --commitSha "$BASE_COMMIT" \
  --appUrl "http://localhost:8080/"

Make sure your trigger script is executable:

chmod +x trigger-base-test.sh

• --headCommit: The commit SHA of the head commit (the commit you're testing). Auto-detected from git if not provided.
• --triggerScript (required): Path to the script that should be executed to trigger a base test run

• --hadPreparedForTests: Signals that ci prepare was run, and the command should wait for the base test run to be available before comparing results. Note: there is a timeout for this wait, so if the base test run takes too long to complete, the command will not be stuck forever.

• --triggerScript: Alternative approach that combines preparation and execution. Instead of using ci prepare separately followed by --hadPreparedForTests, you can pass --triggerScript directly to ci run-with-tunnel. This is more concise but may take longer to complete since it will trigger the base test run and wait for it to finish before proceeding with the current test run. Parallelizing the generation of the base test run and building the application is usually more efficient.

• Verify your trigger script is executable (chmod +x)
• Check that the script path is correct relative to your CI working directory
• Ensure the script has access to necessary environment variables (like METICULOUS_API_TOKEN)

• Check that your base test run is actually completing successfully
• Verify the base commit SHA is correct

• Check the logs from your trigger script
• Verify the base commit can be built successfully
• Ensure all dependencies are available in the environment where the trigger script runs

For more information on setting up Meticulous in CI, see Setting up Meticulous tests to run in your CI provider.