Running Tests

This page details the different options for using the saucectl run command to execute your tests in the JavaScript framework of your choice:

  • Run tests locally with Docker and send results to Sauce Labs
  • Run tests remotely on Sauce Labs virtual machines
  • Run tests against a local app server / app running on localhost

What You'll Need#

default test location

Unless you specify a test directory, saucectl executes tests based on the test directory generated for the framework you selected during installation. For example, with Cypress, saucectl will attempt to locate cypress.json, as well as the default cypress directory.

Test on Docker (Local Testing)#

The following steps outline how to run your tests on your local machine with the containerized solution. This is the preferred option if you wish to run tests on your local machine, or if you wish to accelerate test execution in CI.

docker Syntax Page

See the docker syntax reference page for further details.

mode Description

See the mode syntax reference page for further details.

For example, if you want to run all the suites in docker, set the default mode in config.yml to docker, as in the following example:

defaults:
mode: docker

Specify a Docker Image#

saucectl can provide a default image to run your tests. You can also specify a Docker image in the config.yml, as shown in the following example:

docker:
image: saucelabs/stt-cypress-mocha-node:v5.6.0

Refer to the framework version support matrix for a list of framework-specific images.

Transfer Test Files to the Container#

The configuration field: fileTransfer, instructs saucectl how to copy the test files into the docker container. There are currently two choices mount or copy.

fileTransfer Syntax

Please refer to the fileTransfer syntax page for further detals.

docker:
fileTransfer: mount # Defaults to `mount`. Choose between mount|copy.
image: saucelabs/stt-cypress-mocha-node:v5.6.0

Test on Sauce Labs#

supported frameworks: Cypress

If you wish to run your tests on Sauce Labs VMs, set the default mode in config.yml to sauce, as shown in the following example:

defaults:
mode: sauce

or

suites:
- name: saucy test
mode: sauce

If you wish to increase your VM concurrency you can also use the flag --ccy <vm number>.

saucectl run --ccy 2

You can also designate concurrency in the config.yml like so:

sauce:
concurrency: 3
region: us-west-1

Please note that VM concurrency depends on the suite number rather than the number of .spec.js |.test.js files. Plese visit the CLI Reference for more information regarding command parameters:

Your concurrency and VM entitlements also depend on your Sauce Labs subscription tier. For more information please visit the pricing guide

Cross-Browser Tests#

supported frameworks: Cypress

When you run tests on Sauce Labs VMs you have access to a wide range of OS + browser combinations. In order to test against multiple different browsers, you can indicate the desired combinations in the suites > browser field:

suites:
# Chrome
- name: "Swag Labs Login Chrome"
browser: "chrome"
platformName: "Windows 10"
screenResolution: "1400x1050"
config:
testFiles: [ "**/login.*" ]
# MicrosoftEdge
- name: "Swag Labs Login MicrosoftEdge"
browser: "microsoftedge"
platformName: "Windows 10"
screenResolution: "1400x1050"
config:
testFiles: [ "**/login.*" ]
# Firefox
- name: "Swag Labs Login Firefox"
browser: "firefox"
platformName: "Windows 10"
screenResolution: "1400x1050"
config:
testFiles: [ "**/login.*" ]

For full examples, please visit this repository

Using Sauce Connect#

If you're running tests on Sauce Labs VMs, but the site under test is protected behind strict network security/policies, you can utilize Sauce Connect Proxy to circumvent the problem.

You can use the --tunnel-id flag with saucectl in order to use an existing Sauce Connect tunnel with your test session:

saucectl run --tunnel-id <tunnel-id>

For more information on how to use the --tunnel-id flag, please visit the CLI Reference.

To enable Sauce Connect Proxy in the config.yml, use the tunnel field:

sauce:
concurrency: 3
tunnel:
id: sauce-ci-tunnel
region: us-west-1

For more information regarding how to install, setup, and configure Sauce Connect Proxy, please visit the Sauce Connect Proxy documentation

Analyze Test Results in Sauce Labs#

After tests complete, Testrunner Toolkit uploads the test assets (logs, test results, and test videos) to your Sauce Labs account and displays a job link like so:

https://app.saucelabs.com/tests/<job-number>

From this job link you can review, share, and analyze the test results just as you would with any other test framework executed on Sauce Labs.

Quick demo#

saucectl Demo

Run Tests Against a Local App#

If you plan to run tests against a local app server / app running on localhost (either on your host machine or in a CI pipeline) there are specific workflows you must follow.

Need to Access Custom Node Modules?

If you have third party, or custom modules that are required test dependencies, you can utilize the npm field in your config.yml in order to include those packages during test execution. Refer to the syntax reference for further details.

Run Tests in Docker Mode and Send Results to Sauce Labs#

Set the default or suite-specific mode in config.yml to docker:

defaults:
mode: docker

or

suites:
- name: saucy test
mode: docker

ensure the docker container can access the local app server (e.g. localhost:<port>/) from your host machine. After the tests complete the results upload to the Sauce Labs results dashboard.

Run Tests on Sauce Labs with Sauce Connect#

If you wish to test the app running on a local app server with Sauce Labs VMs:

  • Download and launch Sauce Connect
  • Specify a tunnel-id (either in the config or using the --tunnel-id CLI flag)
Working Example

Here is a working example of this use case using Sauce Connect and GitHub Actions.

Further Details

Please see Using Sauce Connect for further details.

Automation Framework Examples#

The examples here show how Pipeline testing can be used. Try them and find your own use cases.

Every Testrunner image comes with a preconfigured setup that allows you to focus on writing tests instead of tweaking with the configurations. Our initial testrunner flavors come either with Cypress, Playwright, or TestCafe as an automation framework.

Below are example snippets in the following frameworks: Cypress, Playwright, and TestCafe.

context('Actions', () => {
beforeEach(() => {
cy.visit('https://example.cypress.io/commands/actions')
})
it('.type() - type into a DOM element', () => {
// https://on.cypress.io/type
cy.get('.action-email')
.type('fake@email.com').should('have.value', 'fake@email.com')
})
})
Last updated on by Nancy Sweeney