Skip to main content

saucectl run

Execute framework-agnostic tests using the saucectl test orchestrator.

Usage

$ saucectl run [OPTIONS]

Extended Description

Execute tests according to the environment, framework, and test suite specifications defined in your configuration file or via command line options described in this document.

Options Summary

KeyShorthandDescription
--artifacts.cleanupClear the artifacts directory before downloading new test data.
--artifacts.download.directoryA local directory to which test assets are downloaded.
--artifacts.download.matchWhich asset files to download.
--artifacts.download.whenCriteria for downloading assets.
--asyncLaunch tests without waiting for preceding test results.
--buildAdd a build reference to your tests.
--ccyMaximum tests to run concurrently.
--config-cAlternate location of the configuration YAML file.
--dry-runSimulate a test run without running tests.
--env-eDefine environment variables.
--fail-fastStop suite execution after first failure.
--help-hUsage information for the run command.
--launch-orderModify the execution order of suites.
--live-logsTail the live log output from a running Sauce Orchestrate container.
--no-auto-taggingDisable the automatic tagging of jobs with metadata, such as CI or Git information.
--no-colorDisable colored console output.
--region-rSauce Labs target data center.
--reporters.json.enabledToggles saucectl's JSON test result reporting on/off.
--reporters.json.filenameSpecifies the report filename. (default "saucectl-report.json").
--reporters.json.webhookURLSpecifies the webhook URL. When saucectl test is finished, it'll send a HTTP POST payload to the configured webhook URL.
--reporters.junit.enabledToggles saucectl's own junit reporting on/off.
--reporters.junit.filenameSpecifies the report filename. (default "saucectl-report.xml").
--retriesNumber of times to rerun a failed test suite.
--root-dirSpecifies the project directory. Not applicable to mobile frameworks. (default ".")
--sauceignoreSpecifies the path to the .sauceignore file. (default ".sauceignore")
--select-suiteExecute a particular test suite.
--show-console-logInclude the console.log contents in the output for all tests.
--tagsAdd reference tags to your tests.
--timeout-tGlobal timeout that limits how long saucectl can run in total. Supports duration values like '10s', '30m' etc. (default: no timeout).
--tunnel-nameUse a running Sauce Connect tunnel to test.
--tunnel-ownerThe tunnel owner, if it is not the testing account.
--upload-timeoutSet a max upload duration.
--verboseEnable detailed output.

Options Details

--artifacts.cleanup

| OPTIONAL | BOOLEAN |

When set to true, all contents of the specified download directory are cleared before downloading any new assets from the current test.

saucectl run --artifacts.cleanup true

--artifacts.download.directory

| OPTIONAL | PATH |

Specifies the path to the folder location in which to download artifacts. A separate subdirectory is generated in this location for each suite for which artifacts are downloaded. Must be set in conjunction with --artifacts.download.match and --artifacts.download.when.

saucectl run --artifacts.download.directory ./artifacts

--artifacts.download.match

| OPTIONAL | FILE/LIST |

Specifies which artifacts to download based on whether they match the name or file type pattern provided. Supports the wildcard character *. Must be set in conjunction with --artifacts.download.directory and --artifacts.download.when.

saucectl run --artifacts.download.match console.log,another.log

--artifacts.download.when

| OPTIONAL | ENUM |

Specifies when and under what circumstances to download artifacts. Valid values are:

  • always: Always download artifacts.
  • never: Never download artifacts.
  • pass: Download artifacts for passing suites only.
  • fail: Download artifacts for failed suites only.

Must be set in conjunction with --artifacts.download.directory and --artifacts.download.match.

saucectl run --artifacts.download.when always

--async

| OPTIONAL | BOOLEAN |

Allows you to launch tests without waiting for results of the preceding tests. This option can be helpful when relying on a CI tool to automatically launch tests. This option does not require a value; including it inline sets it to true.

--build

| OPTIONAL | STRING |

Associates the tests with a build to support easy filtering of related test results in the Sauce Labs UI.

saucectl run --build myBuildID

--ccy

| OPTIONAL | INTEGER |

Sets the maximum number of suites to execute at the same time. If the config defines more suites than the max, excess suites are queued and run in order as each suite completes.

caution

For tests running on Sauce, set this value to equal or less than your Sauce concurrency allowance, as setting a higher value may result in jobs dropped by the server.

saucectl run --ccy 2

--config

| OPTIONAL | FILEPATH |

Specify an alternative configuration file to the default .sauce/config.yml for this execution.

Shorthand: -c <path>

saucectl run -c ./path/to/{config-file}.yml
YAML Required

While you can use multiple files of different names or locations to specify your configurations, each file must be a *.yml and follow the saucectl syntax. If you are less comfortable with YAML, any of a wide variety of free online YAML/JSON validator tools may be helpful.

--dry-run

| OPTIONAL | BOOLEAN |

Simulate a test run without actually running any tests. This option does not require a value; including it inline sets it to true.

saucectl run --dry-run

--env

| OPTIONAL | KEY=VALUE |

An environment variable key value pair that may be referenced in the tests executed by this command. Expanded environment variables are supported.
saucectl run --env <key1>=<value1> --env <key2>=<value2> ...

Supported Frameworks

Frameworks and ProductsSupported?Note
CypressVirtual Devices Only
PlaywrightVirtual Devices Only
TestCafeVirtual Devices Only
Puppeteer Replay
Espresso
XCUITestVirtual Devices Only
API Testing
Sauce Orchestrate

--fail-fast

| OPTIONAL | BOOLEAN |

Stops suites after the first failure. This will not interrupt suites that have been started already. This option does not require a value; including it inline sets it to true.

saucectl run --fail-fast

--help

Usage information for the run command.

Shorthand: -h

--launch-order

| OPTIONAL | STRING |

Modify the execution order of suites.

Supported value fail rate: Jobs with the highest failure rate are prioritized for launch.

saucectl run --launch-order="fail rate"

--live-logs

| OPTIONAL | BOOLEAN | Sauce Orchestrate Only |

Tail the live log output from a running Sauce Orchestrate container.

saucectl run --live-logs

--no-auto-tagging

| OPTIONAL | BOOLEAN |

Disables the automatic tagging of jobs with metadata, such as CI or Git information.

saucectl run --no-auto-tagging

--no-color

| OPTIONAL | BOOLEAN |

Disables colorized console output for saucectl. This is particularly useful for CI environments that cannot display ANSI color codes, which may render the log output illegible. This option does not require a value; including it inline sets it to true.

saucectl run --no-color

--region

| REQUIRED | STRING |

Specifies the Sauce Labs data center through which tests will run. Valid values are: us-west-1, us-east-4 or eu-central-1.

Shorthand: -r

saucectl run --region us-west-1

--reporters.json.enabled

| OPTIONAL | BOOLEAN |

Toggles saucectl's JSON test result reporting on/off. This only affects the reports that saucectl itself generates as a summary of your tests.

saucectl run --reporters.json.enabled=true

--reporters.json.filename

| OPTIONAL | FILEPATH |

Sets the report filename, with the default being saucectl-report.json.

To use this option, make sure to activate the JSON reporter by including --reporters.json.enabled=true.

saucectl run --reporters.json.enabled=true --reporters.json.filename="path/to/my_report.json"

--reporters.json.webhookURL

| OPTIONAL | URL |

Specifies the webhook URL. When saucectl test is finished, it'll send an HTTP POST payload to the configured webhook URL.

To use this option, make sure to activate the JSON reporter by including --reporters.json.enabled=true.

saucectl run --reporters.json.enabled=true --reporters.json.webhookURL="my_webhook_url"

--reporters.junit.enabled

| OPTIONAL | BOOLEAN |

Toggles saucectl's own junit reporting on/off. This only affects the reports that saucectl itself generates as a summary of your tests. Each Job in Sauce Labs has an independent report regardless.

saucectl run --reporters.junit.enabled=true

--reporters.junit.filename

| OPTIONAL | FILEPATH |

Sets the report filename, with the default being saucectl-report.xml.

To use this option, make sure to activate the JUnit reporter by including --reporters.junit.enabled=true.

saucectl run --reporters.junit.enabled=true --reporters.junit.filename="path/to/my_report.xml"

--retries

| OPTIONAL | INTEGER |

Instructs saucectl to rerun failed tests this many times.

saucectl run --retries 2

--root-dir

| OPTIONAL | FILEPATH |

Specifies the project directory. Not applicable to mobile frameworks. Default: ..

saucectl run --root-dir="path/to/my/root_dir"

--sauceignore

| OPTIONAL | FILEPATH |

Specifies the path to the .sauceignore file. Default: .sauceignore.

saucectl run --sauceignore="path/to/my/.sauceignore"

--select-suite

| OPTIONAL | STRING |

Specifies a test suite to execute by name rather than all suites defined in the config file.

saucectl run --select-suite <suite_name>
Formerly --suite

Versions of saucectl before v0.52.4 use the option --suite instead.

--show-console-log

| OPTIONAL | BOOLEAN |

Includes the contents of the suite's console.log in the output of the command regardless of the test results. By default, the console log contents are shown for failed test suites only. This option does not require a value; including it inline sets it to true.

saucectl run --show-console-log

--tags

| OPTIONAL | LIST |

Keywords that may help you distinguish the test in Sauce Labs, and also help you apply filters to easily isolate tests based on metrics that are meaningful to you.

saucectl run --tags e2e,team2

--timeout

| OPTIONAL | DURATION |

Sets a limit (in seconds or minutes) for how long saucectl can run this test (no limit by default).

Real Device Max Duration

When setting the timeout values for your suites, consider that native framework tests on real devices enforce a maximum test duration limit of 60 minutes.

saucectl run --timeout 10s
saucectl run --timeout 30m

--upload-timeout

| OPTIONAL | DURATION |

Uploads timeout that limits how long saucectl will wait for an upload to finish. Supports duration values like '10s' '30m' etc. (default: 5m)

saucectl run --upload-timeout 10s
saucectl run --upload-timeout 30m

--tunnel-name

| OPTIONAL | STRING |

Specifies an active Sauce Connect tunnel to establish a secure connection to run this test on Sauce Labs.

note

Replaces the former --tunnel_id option, which is deprecated.

saucectl run --tunnel-name my-tunnel

--tunnel-owner

| OPTIONAL | STRING |

Identifies the Sauce Labs user who created the specified tunnel, which is required if the user running the tests did not create the tunnel.

note

Replaces the former --tunnel-parent option, which is deprecated.

saucectl run --tunnel-name not-my-tunnel --tunnel-owner another.sauce.username

--verbose

| OPTIONAL | BOOLEAN |

Enables detailed output during the test run in order to facilitate troubleshooting potential authentication, connection, and/or container issues. This option does not require a value; including it inline sets it to true.

saucectl run --verbose