Skip to main content

Appium Testing with Real Devices

Sauce Labs provides thousands of real mobile devices for nearly every phone and tablet model and applicable OS version. You can run your Appium tests on these devices through the Sauce Labs Real Device Cloud (RDC) to ensure your app behaves accurately and consistently across different devices in the real world. Sauce Labs offers a massive pool of public devices available for all customers, as well as a private option in which customers can create a selection of devices for use by only their organization.

Appium automated real device testing supports tests designed to run against a web app in a mobile browser or a native app on a mobile device.

See When to Test on Real Devices for deails about real device testing use cases, benefits, and system requirements.

note

Sauce Labs does not support automatic testing of ADB commands for Appium. To use ADB and shell commands, the usage of vUSB with private devices is necessary as Sauce Labs does not support ADB without the use of vUSB.


ADB can also be used during live testing.

What You'll Need

Installing Your Mobile App on Real Devices

If your Appium tests are intended to test a native mobile app on real devices, the app file must be available to Sauce Labs so it can be installed on the devices selected for testing. Sauce Labs provides a variety of methods for doing this, including:

  • Upload your app to Sauce App Storage using the Sauce Labs UI or REST API
  • Install your app to a real device from a remote location How?

The following app file types are supported for real device tests:

The following Appium versions are supported on our Real Device Cloud:

  • Appium 2.0 Beta
  • 1.22.2
  • 1.22.0
  • 1.21.0
  • 1.20.1
  • 1.19.0
  • 1.18.1
  • 1.17.1
  • 1.17.0
  • 1.15.1
  • 1.15.0
  • 1.14.0

Using the W3C WebDriver Specification

As the W3C WebDriver Protocol is supported in Appium v1.6.5 and higher, and required for Appium v2.0 (currently in beta), we recommend and support using it exclusively in your test scripts instead of the JSON Wire Protocol (JWP). See Migrating Appium Real Device Tests to W3C for more information.

The W3C WebDriver Protocol test capability syntax differs from that of JWP, so it's important to make sure you configure your tests accurately so your intended protocol is followed and your settings are applied correctly.

How Sauce Labs Determines Your Protocol

When Sauce Labs executes your test configuration, it looks for the presence of certain indicators in the session creation request to determine whether it should apply the JWP or W3C protocol. The following table outlines how Sauce Labs evaluates your creation request.

IndicatorDetermination
sauce:options node is present within the capabilities node.W3C
desiredCapabilities node is absent.W3C
desiredCapabilities node is present AND sauce:options node is absent.JWP

Examples of JWP and W3C Configurations

The main difference between JWP and W3C is the format required to specify your test capabilities. In JWP, all capabilities are defined within the desiredCapabilities node, while in W3C, all capabilities are defined under the capabilities node within the firstMatch property. Each capability uses a namespace to indicate one of the following three categorizations:

  • WebDriver standard capabilities have no namespace
  • Appium specific capabilities require the prefix appium:
  • Sauce Labs custom capabilities are set within the sauce:options node

The following examples illustrate this difference in the respective specifications.

"desiredCapabilities": {
"platformName" : "android",
"app","storage:filename=mapp.pk",
"deviceName" : "Samsung.*Galaxy.*",
"orientation" : "PORTRAIT",
"platformVersion" : "8.1",
"appiumVersion" : "1.21.0",
"sessionCreationRetry" : "2",
"sessionCreationTimeout" : "300000",
"name" : "MobileWebsiteTest (jwp)"
}
note

You can avoid having to add the appium: prefix to Appium specific capabilities by upgrading your Appium client library to a version that automatically applies the prefix or installing the relaxed-caps plugin, but Sauce Labs advises including the prefix to avoid any confusion in the future.

Configuring Appium Tests for Real Devices

Our Test Configuration Options reference documentation provides a complete index of required and optional parameters for Appium. Be aware that not all of the Appium capabilities list are supported for both virtual and real device tests and that some capabilities have driver-specific options for Android and iOS client libraries.

The following sections provide context and instructions for test configurations that are essential when using Appium to run automated tests on Sauce Labs real devices.

Specifying the platformName

You can use real devices to test both native apps and web apps in a mobile browser. The platformName capability is the only test configuration that is mandatory regardless of which type of mobile test you are writing, as it specifies whether the test is for iOS or Android.

Specifying Your app

For native app tests, the app capability is the only other required configuration. If it is omitted, Sauce Labs infers the test is written for a mobile browser and automatically sets a default browserName based on the specified platformName.

For native app tests on real devices, you must provide a location from which your mobile app can be accessed in the app capability so your app can be installed on the test devices. You can specify a Sauce Labs App Storage ID or filename, or a remote location to which Sauce Labs has access. See Application Storage for details.

App
"appium:app","storage:filename=mapp.ipa"
Remote
"appium:app","https://github.com/test-apps/ios-app.ipa"

Excluding the browserName

When testing a native mobile app, no browser is accessed, so if you are re-using the capabilities from your mobile or desktop browser tests, omit the browserName capability. This is an important exclusion because if values are set for both app and browserName, Sauce Labs defaults to the browserName. Similarly, if neither capability is specified, Sauce Labs automatically populates the browserName value that matches the platformName (Safari for iOS and Chrome for Android).

Selecting Your Test Device

Testing on real devices requires you to specify a device on which you plan to test your app. You can do this by specifying either:

  • A specific device from a Sauce Labs device pool (public or private) by its ID (static allocation)
  • A set of device attributes to use as criteria for selecting any available matching device (dynamic allocation)

Static Device Allocation

Static Allocation allows you to specify a known device by its unique ID. This can be beneficial if, for example, you are testing features only available on a very specific device setup. However, what you gain in precision may be offset by the time it takes for a specific device to become available, especially if your tests do not require that level of precision. If you do require a specific device, you should always configure the device's availability before launching your tests.

"appium:deviceName" : "HTC_One_M8_real";

You can obtain a list of available devices, including their IDs, using the Get Devices API request.

Alternatively, you can find a device's ID in the Sauce Labs app:

  1. Log into Sauce Labs.
  2. Navigate to Live >> Mobile App.
  3. Choose your mobile app (or an applicable demo app) from the list and click Choose Device to bring up the pool of devices for your organization.
  4. Hover over the device you want a click Details to bring up the identification information for that device.

Dynamic Device Allocation

Dynamic Allocation allows you to specify the device attributes that are important to you and then run your test against the first available device from the pool that matches your specifications, giving you greater flexibility and, likely, a faster test execution time, particularly if you are running tests in parallel.

Dynamic allocation is advised, in particular, for all automated mobile app testing in CI environments.

To enable dynamic device allocation, you should specify the deviceName and platformName, and platformVersion capabilities at a minimum. The following table provides information about accepted values.

note

The following sample values are presented using case for readability, but capabilities values are not case-sensitive, so there is no distinction between iPhone and iphone, for example.

CapabilityDescription and Sample Values
deviceName

Provide a device display name, or use regular expressions to provide a partial name, thus increasing the potential pool of matches. Some examples include:

Any iPhone: "appium:deviceName", "iPhone.*", "iPhone .*"

Any device with the word "nexus" in its display name: "appium:deviceName", ".*nexus.*"

Either iPhone 7 or iPhone 6: "appium:deviceName", "iPhone [67]" or "iPhone [6-7]"

Either iPhone 7S or iPhone 6S: "appium:deviceName", "iPhone [67]S" or "iPhone [6-7]S"

platformNameSpecify the mobile operating system to use in your tests (i.e., android or ios.
platformVersionSpecify the OS version to use in your tests (i.e., 4 or 4.1.

This property uses a substring match, so you can specify major and/or incremental versions. For example, if you set only a major version 4, any devices running incremental versions (e.g., 4.1, 4.2, 4.2.1, 4.4.4) will also match. This behavior extends to minor and point versions as well, so 11.4 matches 11.4.0 and 11.4.1.

In addition to the required capabilities for device matching, you can also specify any of the following optional Sauce custom capabilities to ensure your tests run on a device that matches your ideal environment. These capabilities need to be put in the "sauce:options": {}.

Using cacheId and noReset

By default, every time you complete a test session, the real device cloud uninstalls your app, performs device cleaning, and de-allocates the device. If you're running multiple tests on the same device, this is inconvenient and inefficient:

  • You must to wait for the cleaning process to complete between every test.
  • You lose time in your testing while your app gets reinstalled to the same device each time.
  • There is a small chance that the device could get allocated to another tester before your next test picks it up.

To optimize device availability, consistency, and efficiency for multiple tests, assign a cacheId to your tests, which keeps the device allocated to you for 10 seconds after each test completes and skips the allocation and device cleaning process if you immediately start another test. The app and its data will still be uninstalled and reinstalled for the next test, however.

"sauce:options" : {
"cacheId" : "jnc0x1256",
}

To skip the uninstallation and reinstallation of your app from the device, you can set noReset to true in conjunction with using a cacheId. This setting adds efficiency, but may not be suitable for test setups that require the app's state to be reset between tests.

"appium: noReset" : "true",
"sauce:options" : {
"cacheId" : "jnc0x1256",
}

When using cacheId the value must match for all tests slated to run on the cached device. In addition, the app and project ID must be the same for all tests, as must the values for the following capabilities:

  • deviceName
  • platformName
  • platformVersion
  • tabletOnly
  • phoneOnly
  • privateDevicesOnly
  • publicDevicesOnly
  • automationName
  • autoGrantPermissions
  • appiumVersion

Example Configuration Code Snippets

iOS and Android Project Configuration

Appium capabilities for an iPhone project using iOS version 15:

MutableCapabilities caps = new MutableCapabilities();
caps.setCapability("platformName", "iOS");
caps.setCapability("appium:platformVersion", "15.0");
caps.setCapability("appium:deviceName", "iPhone .*");
caps.setCapability("appium:orientation", "portrait");
caps.setCapability("appium:app", "storage:filename=<file-name>");
MutableCapabilities sauceOptions = new MutableCapabilities();
sauceOptions.setCapability("username", "SAUCE_USERNAME");
sauceOptions.setCapability("accessKey", "SAUCE_ACCESS_KEY");
caps.setCapability("sauce:options", sauceOptions);

Appium capabilities for Samsung Galaxy device using Android version 11:

MutableCapabilities caps = new MutableCapabilities();
caps.setCapability("platformName", "Android");
caps.setCapability("appium:platformVersion", "11");
caps.setCapability("appium:deviceName", "Samsung.*Galaxy.*");
caps.setCapability("appium:orientation", "portrait");
caps.setCapability("appium:app", "storage:filename=<file-name>");
MutableCapabilities sauceOptions = new MutableCapabilities();
sauceOptions.setCapability("username", "SAUCE_USERNAME");
sauceOptions.setCapability("accessKey", "SAUCE_ACCESS_KEY");
caps.setCapability("sauce:options", sauceOptions);

Additional Test Configuration Options

Once you're up and running with your real device tests, check out our Best Practices for making the most of your testing. Some possible configurations include: