Here's a Java code sample setting a job's name to "My test":
Appium JS-Executor methods for Real Device Testing in Sauce Labs are limited and are indicated with the following badge: RDC ✓
|Sets the pass/fail status of the job. Options are |
|Sets the job name|
|Sets the job tags in a comma-separated list.|
|Sets the job’s build name.|
|Stops and restart the VM’s network connection (Mac OSX only).|
|Turns off logging for certain commands within the test in order to omit sensitive data from the |
NOTE: This method does not omit the commands from other possible records.
|Sets a Sauce breakpoint in the test. Test execution will pause at this point, waiting for manual control by clicking in the test’s live video.|
|Logs the given line in the job’s Selenium commands list.|
|Sets one or more job information fields to the values sent in the JSON-formatted dictionary.|
|Points to file for testing image injection (e.g. barcode scanning).|
|Allows performance metrics to be collected.|
|Pauses performance metrics collection.|
Spacing in the methods is sensitive, i.e., some methods require a space following sauce: (
job-info), while other methods do not.
Setting the pass/fail status of your tests is important for getting the most out of your insights, as Selenium has only three built-in states: In Progress, Error, and Complete.
You should update your tests to record pass/fail status with our REST API on completion, using a test framework, or the
This code is from a sample Java test script using TestNG. You can find the full version in our Test Frameworks repository.
One of the most difficult aspects of troubleshooting Selenium tests can be matching commands to browser actions. The
sauce:context method provides you with a way to inject text into the command log to associate with a specific command, essentially adding a comment.
For example, in the command log on the left, it's hard to immediately see which command is responsible for following a link to the page, which one submitted a comment, and which one asserted that the comment was valid. In the screenshot on the right, each set of commands has been provided with a context.
TestBase.java script, assign a context with each step of the test is set using the
The following code examples are from the Java-TestNG-Selenium framework in our GitHub repository.
In your test script, specify the text you want associated with each step of the test:
You can manage your tests more effectively from your Dashboard and Archives with annotations. The Sauce Labs REST API includes an update_job method that you can use to set a name, tags, pass/fail status, and custom data for your test after it runs. To automate test annotation with this method, you'll want to create a simple set of functions to perform the put request for you. We've developed a Java library to do just that, with examples for Python and Ruby on GitHub.
Adding Pass/Fail Status and Build Numbers to Test Results with Frameworks
In addition to using the REST API to set these annotations once your test completes, you can use one of the Sauce Labs test framework examples to set these and other annotations for you automatically as part of the test execution.
You can set the following test configuration options to keep track of your jobs:
|Test Names||Used to record test names for jobs and make it easier to find individual tests||string|
|Build Numbers||Used to associate jobs with a build number or app version, displayed on the Dashboard and Archives views||string|
|Tagging||User-defined tags for grouping and filtering jobs in the Dashboard and Archives||list|
|Pass/Fail Status||Selenium and Appium handle sending commands to control a browser or app, but don't report to the server whether a test passed or failed. To record pass/fail status on Sauce, set the passed flag on the job.|
Since you can't know in advance whether a test passed or failed, this flag can't be set in the initial configuration.
|Custom Data||User-defined custom data that will accept any valid JSON object, limited to 64KB in size.||object|
By assigning unique attributes (such as test name, tags, and build ID) in your test capabilities, you can then apply these annotations to filter results on your Sauce Labs Test Results and Archive pages. Although not required, following this best practice can make it easier to monitor tests and builds in your CI pipeline.
You can set these capabilities to be any combination of letters and numbers. To differentiate between builds, it's also a good practice to add a timestamp or CI job/build number at the end of your build tag.
The build name and tags capabilities are not supported in automated real device testing at this time, please check back for future updates with regards to this functionality.
While it's technically possible to use the same build name for multiple test runs, this will cause all of your test results to appear incorrectly as part of a single run. This, in turn, will cause your test results for those builds to be inaccurate.