Skip to main content

Appium Testing with Real Devices

Sauce Labs provides thousands of real mobile devices for nearly every phone and tablet model and applicable OS version. You can run your Appium tests on these devices through the Sauce Labs Real Device Cloud (RDC) to ensure your app behaves accurately and consistently across different devices in the real world. Sauce Labs offers a massive pool of public devices available for all customers, as well as a private option in which customers can create a selection of devices for use by only their organization.

Appium automated real device testing supports tests designed to run against a web app in a mobile browser or a native app on a mobile device.

See When to Test on Real Devices for deails about real device testing use cases, benefits, and system requirements.

note

Sauce Labs now supports ADB commands for Appium. To use ADB and mobile:shell commands, please sign up for our BETA through this form and indicate the desired ADB commands you would like to run. We will be supporting a limited list of ADB commands through mobile:shell.


ADB can already be used during live testing.

What You'll Need​

Installing Your Mobile App on Real Devices​

If your Appium tests are intended to test a native mobile app on real devices, the app file must be available to Sauce Labs so it can be installed on the devices selected for testing. Sauce Labs provides a variety of methods for doing this, including:

  • Upload your app to Sauce App Storage using the Sauce Labs UI or REST API
  • Install your app to a real device from a remote location How?

The following app file types are supported for real device tests:

See Appium Versions for information about Appium versions supported for real device testing.

Using the W3C WebDriver Specification​

As the W3C WebDriver Protocol is supported in Appium v1.6.5 and higher, and required for Appium v2.0, we recommend and support using it exclusively in your test scripts instead of the JSON Wire Protocol (JWP). See Migrating Appium Real Device Tests to W3C for more information.

The W3C WebDriver Protocol test capability syntax differs from that of JWP, so it's important to make sure you configure your tests accurately so your intended protocol is followed and your settings are applied correctly.

How Sauce Labs Determines Your Protocol​

When Sauce Labs executes your test configuration, it looks for the presence of certain indicators in the session creation request to determine whether it should apply the JWP or W3C protocol. The following table outlines how Sauce Labs evaluates your creation request.

IndicatorDetermination
sauce:options node is present within the capabilities node.W3C
desiredCapabilities node is absent.W3C
desiredCapabilities node is present AND sauce:options node is absent.JWP

Examples of JWP and W3C Configurations​

The main difference between JWP and W3C is the format required to specify your test capabilities. In JWP, all capabilities are defined within the desiredCapabilities node, while in W3C, all capabilities are defined under the capabilities node within the firstMatch property. Each capability uses a namespace to indicate one of the following three categorizations:

  • WebDriver standard capabilities have no namespace
  • Appium specific capabilities require the prefix appium:
  • Sauce Labs custom capabilities are set within the sauce:options node

The following examples illustrate this difference in the respective specifications.

"desiredCapabilities": {
    "platformName" : "android",
    "app","storage:filename=mapp.pk",
    "deviceName" : "Samsung.*Galaxy.*",
    "orientation" : "PORTRAIT",
    "platformVersion" : "8.1",
    "appiumVersion" : "1.21.0",
    "sessionCreationRetry" : "2",
    "sessionCreationTimeout" : "300000",
    "name" : "MobileWebsiteTest (jwp)"
 }
note

You can avoid having to add the appium: prefix to Appium specific capabilities by upgrading your Appium client library to a version that automatically applies the prefix.

Configuring Appium Tests for Real Devices​

Our Test Configuration Options reference documentation provides a complete index of required and optional parameters for Appium. Be aware that not all of the Appium capabilities list are supported for both virtual and real device tests and that some capabilities have driver-specific options for Android and iOS client libraries.

The following sections provide context and instructions for test configurations that are essential when using Appium to run automated tests on Sauce Labs real devices.

Specifying the platformName​

You can use real devices to test both native apps and web apps in a mobile browser. The platformName capability is the only test configuration that is mandatory regardless of which type of mobile test you are writing, as it specifies whether the test is for iOS or Android.

Specifying Your app​

For native app tests, the app capability is the only other required configuration. If it is omitted, Sauce Labs infers the test is written for a mobile browser and automatically sets a default browserName based on the specified platformName.

For native app tests on real devices, you must provide a location from which your mobile app can be accessed in the app capability so your app can be installed on the test devices. You can specify a Sauce Labs App Storage ID or filename, or a remote location to which Sauce Labs has access. See Application Storage for details.

'appium:app', 'storage:filename=mapp.ipa'
'appium:app', 'https://github.com/test-apps/ios-app.ipa'

You can also install a dependent app or an app upgrade during a test by using the driver.installApp('path-to-app') command.

driver.installApp(
'https://github.com/saucelabs/my-demo-app-rn/releases/download/v1.3.0/Android-MyDemoAppRN.apk'
)
Limitations
  • The provided app path needs to be publicly available as this method does not have access to your local path/storage.
  • This method does not have access to apps in Sauce Storage. Only apps that are publicly available can be installed with this command. Therefore, we also can't re-sign and instrument the app. The Instrumentation will not work for apps installed using the driver.installApp('path-to-app') command (see App Settings to learn more).
  • This method will not work for iOS due to signing. Each iOS app needs to be resigned so it is allowed to be installed on our devices. To make this work you must use a private device and add the UDID of the private device to the provisioning profile for iOS (see our resigning process to learn more).

For more information about this command, see the Appium documentation.

Excluding the browserName​

When testing a native mobile app, no browser is accessed, so if you are re-using the capabilities from your mobile or desktop browser tests, omit the browserName capability. This is an important exclusion because if values are set for both app and browserName, Sauce Labs defaults to the browserName. Similarly, if neither capability is specified, Sauce Labs automatically populates the browserName value that matches the platformName (Safari for iOS and Chrome for Android).

Selecting Your Test Device​

Testing on real devices requires you to specify a device on which you plan to test your app. You can do this by specifying either:

  • A specific device from a Sauce Labs device pool (public or private) by its ID (static allocation)
  • A set of device attributes to use as criteria for selecting any available matching device (dynamic allocation)

Static Device Allocation​

Static Allocation allows you to specify a known device by its unique ID. This can be beneficial if, for example, you are testing features only available on a very specific device setup. However, what you gain in precision may be offset by the time it takes for a specific device to become available, especially if your tests do not require that level of precision. If you do require a specific device, you should always configure the device's availability before launching your tests.

"appium:deviceName" : "HTC_One_M8_real";

You can obtain a list of available devices, including their IDs, using the Get Devices API request.

Alternatively, you can find a device's ID in the Sauce Labs app:

  1. Log into Sauce Labs.
  2. Navigate to Live >> Mobile App.
  3. Choose your mobile app (or an applicable demo app) from the list and click Choose Device to bring up the pool of devices for your organization.
  4. Hover over the device you want a click Details to bring up the identification information for that device.

Dynamic Device Allocation​

Dynamic Allocation allows you to specify the device attributes that are important to you and then run your test against the first available device from the pool that matches your specifications, giving you greater flexibility and, likely, a faster test execution time, particularly if you are running tests in parallel.

note

If you have both private AND public devices, dynamic device allocation will search for available matching private devices first, and if not found it will then search for available matching public devices.

Dynamic allocation is advised, in particular, for all automated mobile app testing in CI environments.

To enable dynamic device allocation, you will have three options:

  • Only provide platformName to find the first available Android or iOS device. This could be a phone or a tablet.
  • Provide platformName AND deviceName to narrow the search to a specific device based on the provided value.
  • Provide platformName AND platformVersion to narrow the search to a specific platform version based on the provided value.

The following table provides information about accepted values.

note

The following sample values are presented using case for readability, but capabilities values are not case-sensitive, so there is no distinction between iPhone and iphone, for example.

CapabilityRequiredDescription and Sample Values
deviceNameNo

Provide a device display name, or use regular expressions to provide a partial name, thus increasing the potential pool of matches. Some examples include:

Any iPhone: "appium:deviceName", "iPhone.", "iPhone ."

Any device with the word "nexus" in its display name: "appium:deviceName", ".nexus."

Either iPhone 7 or iPhone 6: "appium:deviceName", "iPhone [67]" or "iPhone [6-7]"

Either iPhone 7S or iPhone 6S: "appium:deviceName", "iPhone [67]S" or "iPhone [6-7]S"

platformNameYesSpecify the mobile operating system to use in your tests (i.e., android or ios.
platformVersionNoSpecify the OS version to use in your tests (i.e., 4 or 4.1.

This property uses a substring match, so you can specify major and/or incremental versions. For example, if you set only a major version 4, any devices running incremental versions (e.g., 4.1, 4.2, 4.2.1, 4.4.4) will also match. This behavior extends to minor and point versions as well, so 11.4 matches 11.4.0 and 11.4.1.

In addition to the required capabilities for device matching, you can also specify any of the following optional Sauce custom capabilities to ensure your tests run on a device that matches your ideal environment. These capabilities need to be put in the "sauce:options": {}.

Using cacheId and noReset​

By default, every time you complete a test session, the real device cloud uninstalls your app, performs device cleaning, and de-allocates the device. If you're running multiple tests on the same device, this is inconvenient and inefficient:

  • You must to wait for the cleaning process to complete between every test.
  • You lose time in your testing while your app gets reinstalled to the same device each time.
  • There is a small chance that the device could get allocated to another tester before your next test picks it up.

To optimize device availability, consistency, and efficiency for multiple tests, assign a cacheId to your tests, which keeps the device allocated to you for 10 seconds after each test completes and skips the allocation and device cleaning process if you immediately start another test. The app and its data will still be uninstalled and reinstalled for the next test, however.

"sauce:options" : {
  "cacheId" : "jnc0x1256",
}

To skip the uninstallation and reinstallation of your app from the device, you can set noReset to true in conjunction with using a cacheId. This setting adds efficiency, but may not be suitable for test setups that require the app's state to be reset between tests.

"appium:noReset" : "true",
"sauce:options" : {
  "cacheId" : "jnc0x1256",
}

When using cacheId the value must match for all tests slated to run on the cached device. In addition, the app must be the same for all tests, as must the values for the following capabilities:

  • deviceName
  • platformName
  • platformVersion
  • tabletOnly
  • phoneOnly
  • privateDevicesOnly
  • publicDevicesOnly
  • automationName
  • autoGrantPermissions
  • appiumVersion

Example Configuration Code Snippets​

iOS and Android Project Configuration​

Appium capabilities for an iPhone project using iOS version 15:

MutableCapabilities caps = new MutableCapabilities();
caps.setCapability("platformName", "iOS");
caps.setCapability("appium:platformVersion", "15.0");
caps.setCapability("appium:deviceName", "iPhone .*");
caps.setCapability("appium:orientation", "portrait");
caps.setCapability("appium:app", "storage:filename=<file-name>");
MutableCapabilities sauceOptions = new MutableCapabilities();
sauceOptions.setCapability("username", "SAUCE_USERNAME");
sauceOptions.setCapability("accessKey", "SAUCE_ACCESS_KEY");
caps.setCapability("sauce:options", sauceOptions);

Appium capabilities for Samsung Galaxy device using Android version 11:

MutableCapabilities caps = new MutableCapabilities();
caps.setCapability("platformName", "Android");
caps.setCapability("appium:platformVersion", "11");
caps.setCapability("appium:deviceName", "Samsung.*Galaxy.*");
caps.setCapability("appium:orientation", "portrait");
caps.setCapability("appium:app", "storage:filename=<file-name>");
MutableCapabilities sauceOptions = new MutableCapabilities();
sauceOptions.setCapability("username", "SAUCE_USERNAME");
sauceOptions.setCapability("accessKey", "SAUCE_ACCESS_KEY");
caps.setCapability("sauce:options", sauceOptions);

Additional Test Configuration Options​