Nightwatch Integration

Introduction

This guide requires an existing Nightwatch project.
You can alternatively take a look to our example repository.

Sauce Visual provides an integration with Nightwatch through a service that you can add to any existing Nightwatch project.

Sauce Visual adds new commands to the Nightwatch's browser object:

browser.sauceVisualCheck(): Takes a screenshot and send it to Sauce Visual for comparison.
browser.sauceVisualResults(): Waits for diff calculations to complete and returns a summary of results.

Quickstart

Step 1: Add Sauce Visual dependency

Install the Sauce Visual service in your current project.

npm install --save-dev @saucelabs/nightwatch-sauce-visual-service

Step 2: Add SauceVisualService to your Nightwatch configuration

Add the SauceVisualService to your existing configuration (E.g. nightwatch.conf.(js|ts)):

// ...
module.exports = {
  // ...
  // Add the service
  plugins: [
    // ... other plugins
    '@saucelabs/nightwatch-sauce-visual-service',
  ],
  // ...
  test_settings: {
    // ...
    default: {
      // ...
      sauceVisualService: {
        buildName: 'Nightwatch Visual Demo Test',
        project: 'Nightwatch Project',
        branch: 'main',
      },
      // ...
    },
    // ...
  },
  // ...
};

Step 3: Add visual tests in your project

Add a check to one of your tests:

Default
Mocha
CucumberJS

import { NightwatchTests } from 'nightwatch';

const home: NightwatchTests = {
  'Sauce Demo test': () => {
    browser.url('https://saucedemo.com').sauceVisualCheck('Home Page').end();
  },
};

export default home;

describe('Saucedemo example', function () {
  it('Check Home Page - 1', (browser) => {
    browser
      .url('https://saucedemo.com')
      // NOTE: adding the `this` context is required for the service
      // to properly pass the suite and test name to Sauce Labs Visual
      .sauceVisualCheck('Home Page - 1', this);
  });
});

const { Given, When } = require('@cucumber/cucumber');

Given(/^I am on the login page$/, function () {
  return browser.navigateTo('https://www.saucedemo.com');
});

When(/^I check the page before login$/, function () {
  return browser.sauceVisualCheck('Before Login');
});
// ...

Step 4: Configure your Sauce Labs credentials

Sauce Visual relies on environment variables for authentications.
Both SAUCE_USERNAME and SAUCE_ACCESS_KEY need to be set prior starting your Nightwatch job.

Username and Access Key can be retrieved from https://app.saucelabs.com/user-settings.

export SAUCE_USERNAME=__YOUR_SAUCE_USER_NAME__
export SAUCE_ACCESS_KEY=__YOUR_SAUCE_ACCESS_KEY__

Step 5: Run the test

Upon executing your tests for the first time under this step, a visual baseline is automatically created in our system. This baseline serves as the standard for all subsequent Nightwatch tests. As new tests are run, they are compared to this original baseline, with any deviations highlighted to signal visual changes. These comparisons are integral for detecting any unintended visual modifications early in your development cycle. All test builds, including the initial baseline and subsequent runs, can be monitored and managed through the Sauce Labs platform at Sauce Visual Builds.

Remember, the baseline is established during the initial run, and any subsequent visual differences detected will be marked for review.

At the end of the build, a summary will be displayed in the console:

┌───────┬────────────┐
│ Count │ Status     │
├───────┼────────────┤
│ 0     │ Approved   │
├───────┼────────────┤
│ 1     │ Equal      │
├───────┼────────────┤
│ 0     │ Queued     │
├───────┼────────────┤
│ 5     │ Unapproved │
├───────┼────────────┤
│ 0     │ Rejected   │
└───────┴────────────┘

note

More information about the status can be found here

Advanced usage

Customizing Your Builds (Environment Variables)

Below are the environment variables available in the Sauce Visual Nightwatch plugin. Keep in mind that the variables defined in Nightwatch configuration have precedence over these.

Variable Name		Description
`SAUCE_USERNAME`	required	Your Sauce Labs username. You can get this from the header of app.saucelabs.com
`SAUCE_ACCESS_KEY`	required	Your Sauce Labs access key. You can get this from the header of app.saucelabs.com
`SAUCE_REGION`		The region you'd like to run your Visual tests in. Defaults to `us-west-1` if not supplied. Can be one of the following: `'eu-central-1'`, `'us-west-1'` or `'us-east-4'`
`SAUCE_VISUAL_BUILD_NAME`		The name you would like to appear in the Sauce Visual dashboard.
`SAUCE_VISUAL_BRANCH`		The branch name you would like to associate this build with. We recommend using your current VCS branch in CI.
`SAUCE_VISUAL_DEFAULT_BRANCH`		The main branch name you would like to associate this build with. Usually `main` or `master` or alternatively the branch name your current branch was derived from. Follow me to learn more
`SAUCE_VISUAL_PROJECT`		The label / project you would like to associate this build with.
`SAUCE_VISUAL_BUILD_ID`		For advanced users, a user-supplied SauceLabs Visual build ID. Can be used to create builds in advance using the GraphQL API. This can be used to parallelize tests with multiple browsers, shard, or more. By default, this is not set and we create / finish a build during setup / teardown.
`SAUCE_VISUAL_CUSTOM_ID`		For advanced users, a user-supplied custom ID to identify this build. Can be used in CI to identify / check / re-check the status of a single build. Usage suggestions: CI pipeline ID.

Sauce Visual Assertion

note

This feature is only available when using the default and CucumberJS TestRunners. It is not available when using the mocha TestRunner.

At the end of each Test you can assert the status of all the visual comparisons that have been made during the test. Sauce Visual will return a summary of the test results in this format:

{
    QUEUED: number;     // Diffs that are pending for processing. Should be 0 in case the test is completed without any timeouts
    EQUAL: number;      // Diffs that have no changes detected
    UNAPPROVED: number; // Diffs that have detected changes and waiting for action
    APPROVED: number;   // Diffs that have detected changes and have been approved
    REJECTED: number;   // Diffs that have detected changes and have been rejected
}

By using the following assertion you can make sure that all the visual comparisons have been completed and approved:

import { NightwatchTests } from 'nightwatch';
import { DiffStatus } from '@saucelabs/nightwatch-sauce-visual-service';

const home: NightwatchTests = {
  'Check Inventory Page': () => {
    browser
      .url('https://saucedemo.com')
      .setValue('input[data-test="username"]', 'standard_user')
      .setValue('input[data-test="password"]', 'secret_sauce')
      .click('input[data-test="login-button"]')
      .waitForElementVisible('.inventory_list')
      .sauceVisualCheck('Inventory Page')
      .click('[data-test="add-to-cart-sauce-labs-backpack"]')
      .sauceVisualCheck('Added backpack to cart')
      // Here we validate that we don't have any unapproved visual changes
      // See status codes above
      .assert.sauceVisualResults(DiffStatus.Unapproved, 0);
  },
};

export default home;

Build attributes

When creating the service in Nightwatch's configuration, extra fields can be set to define the context, thus acting on which baselines new snapshots will be compared to. (More info on baseline matching).

Options:

buildName: Name of the build, not used for matching, only for display purposes
project: Name of the project, used for matching
branch: Name of the branch, used for matching
defaultBranch: Name of the default branch, used for matching
captureDom: Enable DOM capture globally, can also be enabled on a per-screenshot basis

// ...
module.exports = {
  // ...
  // ...
  test_settings: {
    // ...
    default: {
      // ...
      sauceVisualService: {
        buildName: 'Nightwatch Visual Demo Test',
        project: 'Nightwatch Project',
        branch: 'main',
        // captureDom: true,
      },
      // ...
    },
    // ...
  },
  // ...
};

Ignored regions

In the case you need to ignore some region when running your tests, Sauce Visual provides a way to ignore user-specified areas.

Those ignored regions are specified when requesting a new snapshot.

Component-based ignored region

You can ignore elements from the page based on providing a component by its selector or by a page object reference (if you are using page objects).

Example:

const page = browser.page.homePage();
await browser.sauceVisualCheck('Home Page', {
  ignore: [
    // If you are using page objects
    page.elements.username.selector,
    //
    // NOTE: You can't use the normal page object syntax, like '@username' here. Please use the above.
    // If you do use the normal syntax, then the service will filter out the property, not use it and log a warning like
    //  Ignoring page object reference: '@username'. Please use the 'pageObjectName.elements.username.selector' annotation.
    //
    // If you prefer selectors
    'input[data-test="password"]',
  ],
});

User-specified ignored region

Alternatively, an ignored region can be a specific region that is defined by four elements.

x, y: The location of the top-left corner of the ignored region
width: The width of the region to ignore
height: The height of the region to ignore

Note: all values are pixels

Example:

browser
  .url('https://saucedemo.com')
  .sauceVisualCheck('Home Page', {
    ignore: [
      {
        x: 100,
        y: 100,
        width: 200,
        height: 200,
      },
    ],
  })
  .end();

Selective Diffing

Sauce Visual allows selective diffing that permits to ignore changes from a certain kind (more information here).

warning

Selective diffing is only available with Balanced diffing method AND with DOM capture enabled.

Area-specific configuration

Sauce Visual Binding allows to configure which kinds of changes should be effective specific regions of the snapshot.

Example:

  const login = browser.page.login();
  login
    .navigate()
    .waitForElementVisible('@username')
    .sauceVisualCheck('Home Page', {
      diffingMethod: DiffingMethod.Balanced,
      captureDom: true,
      regions: [
        {
          element: login.elements.username.selector,
          enableOnly: [],
        },
        {
          element: login.elements.password.selector,
          enableOnly: ['style],
        },
      ],
    });

Capturing the DOM snapshot

Sauce Visual does not capture dom snapshot by default. It can be changed in sauceVisualCheck options.

Example:

browser
  .url('https://saucedemo.com')
  .sauceVisualCheck('Home Page', {
    captureDom: true
  })
  .end();

Clip to an Element

You can clip to a specific element on the page by using the clipElement option when calling Sauce Visual.

Notes:

Clipping is done by taking a screenshot of the page then clipping it to the location of the requested element.
We will attempt to scroll the element into view before taking the snapshot.
We can only take a screenshot of what is visible in the current viewport, however, this can be combined with full page option to enable clipping large vertical elements.

Example:

await browser.sauceVisualCheck('Visible Sale Banner', {
  // An element that we should crop the screenshot to
  clipElement: browser.element('.your-css-selector')
})

Fail on failures

By default, Sauce Visual will not fail the test if there are any failures during the comparison process. A failure will be logged in the Sauce Labs Visual dashboard, but the test will continue to run.

If you want to fail the test when there are failures, you can set the failOnFailures option to true:

// ...
module.exports = {
  // ...
  // ...
  test_settings: {
    // ...
    default: {
      // ...
      sauceVisualService: {
        failOnFailures: true, // Default is false
      },
      // ...
    },
    // ...
  },
  // ...
};

Skip Execution of Visual Tests

In the event you're running / debugging tests locally and would like to disable the Sauce Visual integration temporarily, you can use the SAUCE_VISUAL_SKIP environment variable with any truthy value. When provided, we'll skip build creation, taking snapshots, and all snapshot based assertions will be marked as successful.

Available in @saucelabs/nightwatch-sauce-visual-service@0.6.0 and later.

Example

An example project is available here.

Introduction​

Quickstart​

Step 1: Add Sauce Visual dependency​

Step 2: Add SauceVisualService to your Nightwatch configuration​

Step 3: Add visual tests in your project​

Step 4: Configure your Sauce Labs credentials​

Step 5: Run the test​

Advanced usage​

Customizing Your Builds (Environment Variables)​

Sauce Visual Assertion​

Build attributes​

Ignored regions​

Component-based ignored region​

User-specified ignored region​

Selective Diffing​

Area-specific configuration​

Capturing the DOM snapshot​

Clip to an Element​

Fail on failures​

Skip Execution of Visual Tests​

Example​

Introduction

Quickstart

Step 1: Add Sauce Visual dependency

Step 2: Add SauceVisualService to your Nightwatch configuration

Step 3: Add visual tests in your project

Step 4: Configure your Sauce Labs credentials

Step 5: Run the test

Advanced usage

Customizing Your Builds (Environment Variables)

Sauce Visual Assertion

Build attributes

Ignored regions

Component-based ignored region

User-specified ignored region

Selective Diffing

Area-specific configuration

Capturing the DOM snapshot

Clip to an Element

Fail on failures

Skip Execution of Visual Tests

Example