Skip to main content

Appium Testing with Real Devices

This topic describes automated Appium testing on the Sauce Labs Real Device Cloud (RDC). You can accelerate your test execution by running parallel automated tests across thousands of mobile device/OS combinations, in our public real device cloud and/or a private device pool of your own.

What You'll Need#

Uploading Mobile Apps from a Remote Location#

There may be situations where you want to install an app from a downloadable remote location (e.g., AWS S3 bucket, a GitHub repository). The app is completely removed from the real device after the test completes, providing an added layer of security for your app.

Please review the following guidelines below before uploading your app:

  1. Make sure your app meets the requirements for Android and iOS Mobile App Testing.

  2. Upload your app to the hosting location.

  3. Ensure Sauce Labs has READ access to the app URL.

  4. In your Appium test script, enter the URL for the app as the "app" desired capability. Example Java Snippet:

    caps.setCapability("app", "https://github.com/saucelabs/sample-app-mobile/releases/download/2.3.0/Android.SauceLabs.Mobile.Sample.app.2.3.0.apk?raw=true");

Installing your App on Private Devices#

In some cases, you may need to upload/install your app to a private device and also prevent the device from broad internet access while under test. The steps to achieve this are:

  • Upload your app to an internal git repository, or private hosting solution with the necessary permissions (e.g. Amazon S3 with a strict bucket policy).
  • Ensure the hosted app URL is available to the machine running the automated test.
  • Ensure that you've enabled the Require Sauce Connect/VPN setting in your organization's security settings.
note

Each session is a "fresh" installation of your app, meaning, you will not be able to access information about previous versions of your app.

Uploading Mobile Apps with the Sauce Labs REST API#

Use the App Storage REST API to upload your app for Appium automated mobile testing. We currently support *.apk or *.aab Android app files and *.ipa or *.zip iOS app files (*.zip files are parsed to determine whether a valid *.app bundle exists).

Limited Support for *.aab Files

At this time, *.aab files are only supported for Android real device testing.

Application Storage and Data Center Endpoints#

When uploading your app for testing with real devices, you must identify the Data Center where the devices you are testing are located by specifying the applicable API URL for that data center.

Examples using the EU and US endpoints:

US Data Center (macOS/Linux)

curl -u "$SAUCE_USERNAME:$SAUCE_ACCESS_KEY" --location \--request POST 'https://api.us-west-1.saucelabs.com/v1/storage/upload' \--form 'payload=@"g16K4P8IX/iOS.RealDevice.SauceLabs.Mobile.Sample.app.2.7.1.ipa"' \--form 'name="iOS.RealDevice.SauceLabs.Mobile.Sample.app.2.7.1.ipa"'

US Data Center (Windows)

> curl -u "%SAUCE_USERNAME%:%SAUCE_ACCESS_KEY% -X POST -w "%{http_code}\n" \-H "Content-Type: application/octet-stream" \"https://saucelabs.com/rest/v1/storage/%SAUCE_USERNAME%/Android.SauceLabs.Mobile.Sample.app.x.x.x.apk?overwrite=true" \--data-binary @\path\to\Android.SauceLabs.Mobile.Sample.app.x.x.x.apk

Configuring Appium Tests for Real Devices#

This section describes parameters that we recommend using to configure your Appium tests on Sauce Labs real devices. See Test Configuration Options for the full list of required and optional parameters.

Certain Appium capabilities behave differently when running Appium tests on our Real Device Cloud versus a local Appium server. On our Real Device Cloud:

  • Some Appium capabilities are not supported.
  • Emulator-only capabilities will not work.
  • The app capability will be always be overwritten; it will point to the app file you uploaded to our system.
  • The noReset capability will only work if device caching is enabled.
  • Different setups may have different ways of handling capabilities and/or different requirements. Check to make sure you're providing all of the required capabilities.

Supported Use Cases for Sauce Labs Real Device Testing#

  • Execute Appium tests against a private real device hosted in the U.S., using your Sauce Labs username and access key.
  • Use our application storage for Appium testing as you usually do for emulators and simulators tests.
  • Analyze Appium test executions, on Sauce Labs similar to the way you do it for desktop, emulators and simulators.
  • Consume Real Device Cloud (RDC) API similar to the way you do for emulators and simulators (with applicable RDC settings).

Example Test Script

private URL createUrl() throws MalformedURLException {    return new URL("https://$SAUCE_USERNAME:$SAUCE_ACCESS_KEY@ondemand.us-west-1.saucelabs.com/wd/hub");}
@BeforeEachvoid setUp() throws MalformedURLException {    DesiredCapabilities desiredCapabilities = new DesiredCapabilities();    desiredCapabilities.setCapability("platformName", "iOS");    IOSDriver driver = new IOSDriver(createUrl(), desiredCapabilities);}

Setting Your appiumVersion#

If you omit the appiumVersion in your test configuration, your test will be running with our default Appium version. We recommend that you specify one of the newer Appium versions that provides a more extended API and fixes to known bugs.

Checking the Appium Version for Your Test#

  1. Log in to Sauce Labs.
  2. Go to Test Details.
  3. Find and select the test that you ran using Appium.
  4. Click the Metadata tab.
  5. Look for the Logs row and select Appium Log. The first line should indicate the Appium version. For example, 2019-05-05T17:45:07.541Z - info: Welcome to Appium v1.10.1.

Setting the browserName#

When testing a native mobile app, the value for browserName is an empty string, as in caps.setCapability("browserName", "");.

Setting the Location of Your Mobile App#

If the app you want to test has been uploaded to a location other than our App Storage, you need to specify this location for app, and make sure that this location is accessible to Sauce Labs browsers. For example, caps.setCapability("app","storage:filename=mapp.zip");.

Setting the automationName for Android Apps#

If you're testing a native mobile app against Android versions 4.0-4.1, or a hybrid mobile against Android versions 4.0 - 4.2, you need to set the capability "automationName","selendroid".

These Android versions are only supported via Appium’s bundled version of Selendroid, which utilizes Instrumentation. Later versions of Android are supported via Appium’s own UiAutomator library.

Enabling Location Services for iOS Devices#

If you want to enable location services on an iOS simulator - to test GPS-dependent apps, for example - set these capabilities in your Appium script:

locationServicesEnabled=truelocationServicesAuthorized=true

Example Configuration Code Snippets#

iOS and Android Project Configuration#

Appium capabilities for an iPhone project using iOS version 12.2:

DesiredCapabilities caps = DesiredCapabilities();    caps.setCapability("username", "SAUCE_USERNAME");    caps.setCapability("accessKey", "SAUCE_ACCESS_KEY");    caps.setCapability("deviceName","iPhone .*");    caps.setCapability("orientation", "portrait");    caps.setCapability("platformVersion","12.2");    caps.setCapability("platformName", "iOS");    caps.setCapability("browserName", "");    caps.setCapability("app", "storage:filename=<file-name>");

Appium capabilities for Samsung Galaxy device using Android version 8.1:

DesiredCapabilities caps = DesiredCapabilities();    caps.setCapability("username", "SAUCE_USERNAME");    caps.setCapability("accessKey", "SAUCE_ACCESS_KEY");    caps.setCapability("deviceName","Samsung.*Galaxy.*");    caps.setCapability("orientation", "portrait");    caps.setCapability("browserName", "");    caps.setCapability("platformVersion","8.1");    caps.setCapability("platformName","Android");    caps.setCapability("app", "storage:filename=<file-name>");

Native vs. Hybrid Apps#

iPhone 6 Real Device

{deviceName:'iPhone 6 Device',platformName:'iOS',platformVersion:'8.4',app:'storage:filename=SampleIOSApp.ipa',"appium-version":"1.4.16"}

Samsung Galaxy S5 Real Device

{deviceName:'Samsung Galaxy S5 Device',platformVersion:'4.4',platformName:'Android',"appActivity": ".ContactManager","appPackage": "com.example.android.contactmanager",app:"http://saucelabs.com/example_files/ContactManager.apk","appium-version":"1.4.16"}

Samsung Galaxy S4

{deviceName:'Samsung Galaxy S4 Device',platformVersion:'4.4',platformName:'Android',"appActivity": ".ContactManager","appPackage": "com.example.android.contactmanager",app:"http://saucelabs.com/example_files/ContactManager.apk","appium-version":"1.4.16"}

Dynamic Device Allocation#

Dynamic Allocation involves providing basic parameters for the platform and operating system, or the type of device you want to use in your tests, and a device with those specifications is selected from the device pool.

While static allocation allows you more fine-grained control over the device used in your tests, it can also cause delays in your test execution if that device isn't available when you run your tests. If you only need to test on a particular platform and OS version, such as an Android 4.1, or on a particular type of device, you should use dynamic allocation, and we recommend that you use dynamic allocation for all automated mobile application testing in CI/CD environments.

Required Capabilities#

Below are capabilities required for dynamic allocation of iOS and/or Android real devices for your tests.

CapabilityCapability Explanation
platformName

Defines the type of mobile platform to use in your tests (i.e., Android or iOS). The values for capabilities are not case-sensitive, so android is the same as Android, and ios is the same as iOS.

platformVersion

The platform version to use in your tests, for example "4" or "4.1". This is a substring match. You can specify both major versions and incremental versions of an operating system.

For example, if you set only a major version 4, you also have access to all devices running incremental versions (e.g., "4.1"," 4.2", "4.2.1", "4.4.4").

This also extends to minor and point versions. For example, if you specify "11.4", it will match "11.4.0", "11.4.1".

deviceName

The display name of the device to use, such as "Samsung S7". You can also use regular expressions for setting the deviceName. Some examples:

To allocate any iPhone:

"iPhone.*", "iPhone .*"

To allocate any device with the word "nexus" in its display name.

".*nexus.*"

To allocate either "iPhone 7" or "iPhone 6" device.

"iPhone [67]" or "iPhone [6-7]"

To allocate either "iPhone 7S" or "iPhone 6S" device.

"iPhone [67]S" or "iPhone [6-7]S"

To allocate "iPhone 7" or "iPhone 7S", or any device that starts with the display name "iPhone 7".

"iPhone 7.*"

NOTE: Regular expressions are not case sensitive.

Optional Capabilities#

Below are optional capabilities for dynamic allocation of iOS and/or Android real devices for your tests.

Static Device Allocation#

With Static Allocation, you can specify the device to use in your tests, but if that device is not available when you run your tests, it will delay test execution. For this reason, you should always make sure that the device you want to use is available before launching your tests. The platformName and platformVersion capabilities will be set by default, and if these are included in your test script, they will be ignored.

CapabilitySetting
deviceNameThe ID of the device you want to use in your test (e.g., LG_Nexus_5X_real). To find a device's ID number, go to its listing in the device selection menu, then click Details .

Device Caching#

cacheId Capability#

By default, every time you complete a test session, the real device cloud uninstalls your app, performs device cleaning, and de-allocates the device. This means that if you're running multiple tests on the same device, you would need to wait for this cleaning process to complete between every test.

To get around this, you can use the capability cacheId, which keeps the device allocated to you for 10 seconds after each test completes. If you immediately start another test on the device, you won't need to wait for the allocation and device cleaning process to be repeated. In this case, no device cleaning will take place in between sessions, with the only exception being the application under test and the data it owns.

CapabilitySetting
cacheId

A random string. This value for cacheId must be the same for all test methods that you want to run on the cached device. In addition, the application and project ID used for the tests must remain the same, along with the values for these capabilities:

  • deviceName
  • platformName
  • platformVersion
  • tabletOnly
  • phoneOnly
  • privateDevicesOnly
  • publicDevicesOnly
  • automationName
  • autoGrantPermissions
  • appiumVersion

Using Device Caching with noReset#

You can also use the cacheId capability in conjunction with the standard noReset Appium capability. In the default case, where noReset is set to false, your application will be uninstalled and reinstalled after every test. If noReset is set to true, the application you are testing won't be reinstalled after every test run. This might save you further time, but it won't be suitable for test setups that require the application's state to be reset between tests. Note that then cacheId is set, no device cleaning will take place in between sessions, regardless of noReset value.

Monitoring Real Device Performance for Appium Tests#

By including the desired capability recordDeviceVitals in your Appium test script, you can collect performance statistics for the real devices used in your tests, including CPU, Network, Memory, and Thread performance. This topic describes how to set up the desired capability in your tests, and collect the statistics when the test completes.

Setting recordDeviceVitals

Include this setting as part of your Appium test script capabilities:

capabilities.setCapability("recordDeviceVitals", true);

Collecting Performance Metrics

  1. When your test completes, log into the Real Device Testing web interface, and select the app that you used in the test from your dashboard.
  2. Click Automated Testing > Appium.
  3. Select the Test Results for your test.
  4. Under Mobile Vitals, click the link to download the CSV file containing your test performance metrics.

Performance Metrics for Android Devices

The CSV file will contain these performance metrics for Android devices.

MetricDescription
unix_epoch_millisecondsUnix epoch timestamp in milliseconds, which can be matched to events within your automated tests
device_local_time Device’s local date and time
cpu_totalSystem-wide CPU usage in percentage across all CPU cores. 4 cores at max use would be shown as a value of 400%.
cpu_userCPU usage for user processes in percentage across all CPU cores. 4 cores at max use would be shown as a value of 400%.
cpu_kernelAndroid OS CPU usage in percentage across all CPU cores. 4 cores at max use would be shown as a value of 400%.
n_processorsAmount of available CPU cores. Use this to divide the cpu values into per core
n_threadsTotal threads in use by the app
memory_size_kb Total memory currently used by device in kilobytes
memory_resident_kbMemory currently in use by application in kilobytes
memory_shared_kbAnonymous shared memory currently in use by system shared between application(s) and system
network_wifi_receive_bData in bytes received over wifi connection
network_wifi_sent_bData in bytes sent over wifi connection
network_mobile_receive_bData in bytes received from the mobile carrier network
network_mobile_sent_b Data in bytes sent over mobile carrier network

Performance Metrics for iOS Devices

The CSV file will contain these performance metrics for iOS devices.

MetricDescription
Unix_epoch_millisecondsUnix epoch timestamp in milliseconds, which can be matched to events within your automated tests
device_local_timeDevice’s local date and time
cpu_totalSystem-wide CPU usage in percentage. Values between 0 and 100%.
cpu_userUser processes CPU usage in percentage. Values between 0 and 100%.
cpu_processApp under test CPU usage in percentage. Values between 0 and 100%.
n_threadsTotal threads in use by the process
process_memory_used_bMemory used in bytes by the app that is under test
net_bytes_inTotal data in bytes received
net_bytes_outTotal data in bytes sent
net_packets_inTotal packets received
net_packets_outTotal packets sent
disk_write_opsDisk write operations during time period
disk_bytes_writtenBytes written to disk during time period

Additional Test Configuration Options#

Once you're up and running with your real device tests, check out our Best Practices for making the most of your testing. Here are some examples:

Full Example Scripts#

These Appium script examples can help streamline your real device testing process. They use the pytest test framework. Feel free to clone these scripts directly from GitHub, and follow the instructions in the README file.

Visit our sample test frameworks GitHub repository for more detailed language-specific examples.

For Example Purposes Only

The code in these scripts is provided on an "AS-IS” basis without warranty of any kind, either express or implied, including without limitation any implied warranties of condition, uninterrupted use, merchantability, fitness for a particular purpose, or non-infringement. Your tests and testing environments may require you to modify these scripts. Issues regarding these scripts should be submitted through Sauce Labs GitHub. These scripts are not maintained by Sauce Labs Support.