Detecting Diffs Locally

With the standard Meticulous integration in your CI pipeline, Meticulous will automatically run on every commit to every PR and will comment on your PRs with a summary and link to the diffs.

However, if you want to quickly try it out before integrating with CI, you can detect diffs locally using the Meticulous CLI. This guide will walk you through the necessary steps to do so and help clarify how Meticulous works along the way.

• Have a Meticulous account. If you don't have one yet, you can sign up for free.
• Install the Meticulous CLI. The Meticulous CLI is available via NPM.
• Have a local version of your application running. This version of your app needs to be locally accessible via a URL (e.g. http://localhost:3000).

At a high level, Meticulous works by recording user sessions and then simulating the sessions on different versions of your app to detect any changes. In order for Meticulous to detect a diff on a given screen, Meticulous needs to have recorded a session which rendered that screen.

If you have not yet recorded any sessions, you can use the Meticulous CLI to do so:

npx @alwaysmeticulous/cli record session --apiToken="<API_TOKEN>"

• Note: the above command includes your API token. Make sure to keep this secret - it provides access to all your recorded user sessions.
• The command will open up a web browser with a blank page. You can now navigate to the URL for the local version of your app.
• Your interactions with the site will be recorded. Go through the flow that you wish to record, like signing up.
• Once you are finished you can close the browser. Meticulous will print out links for the sessions recorded.
• If you navigate across multiple pages then Meticulous may record one separate session per page. This allows Meticulous to simulate sessions for multiple pages in parallel.

Meticulous does not take screenshots of your app when recording sessions. Instead, Meticulous records user actions and takes screenshots when simulating those actions against a version of your application. Comparing screenshots between simulations reduces flakes and keeps tests up-to-date as your application evolves.

To manually generate the base screenshots from your sessions, you can trigger a test run without providing a base test run to compare against. This can be done using the Meticulous CLI:

npx @alwaysmeticulous/cli ci run-local --apiToken="<API_TOKEN>" --appUrl="<LOCAL_APP_URL>"

Once the test run completes, the CLI will output a link to the Meticulous web app where you can view the test run results. Because you did not provide a base test run to compare against, the test run page will show a message indicating that there are no diffs. If you want to see what screenshots were taken, you can click on the "View Visual Snapshots Tested" button.

Please note down the test run ID for use in a later step. This can be found in the URL of the test run page after /test-runs/.

Now that you have generated base screenshots, you can modify your application to introduce a diff. Please make a change to a screen that was rendered in one of the recorded sessions and then recompile your app.

Now that you have modified your app, you can run the tests again to see if Meticulous detects any diffs. This time, you will need to provide the test run ID from step 2 as the base test run to compare against:

npx @alwaysmeticulous/cli ci run-local --apiToken="<API_TOKEN>" --appUrl="<LOCAL_APP_URL>" --baseTestRunId="<TEST_RUN_ID>"  --parallelize

Just like in step 2, the test run in step 4 will output a link to the Meticulous web app where you can view the test run results. If Meticulous detects any diffs, you'll see both the base and the new screenshots to help you quickly identify where the diffs occurred.

In this demo, you recorded one or two sessions which will only cover a very small portion of your application. Once you set up the Meticulous recorder, Meticulous will auto-curate a test suite from dozens of new sessions a day and will aim to cover every corner of your application. Additionally, when you integrate Meticulous with your CI, Meticulous will automatically run on every commit to every PR and will comment on your PRs with a summary and link to the diffs. These tests are run in Meticulous's simulation cluster and normally take less than 2 minutes to run.

We're always happy to help you with any issues you encounter while setting up or with anything else you might be unsure about.

Get in touch by emailing support@meticulous.ai.