Using Real and Virtual Mobile Devices for Testing

Get the most out of your live and automated testing by including a healthy mix of emulators, simulators, and real devices as your mobile testing platforms. Why? Because there are aspects of the mobile experience that you can't test on emulators or simulators, such as location-based apps that use manufacturer-specific device sensors, memory consumption, and CPU usage.

There are a variety of use cases to consider when you're deciding on the mix of emulators, simulators, and real devices to use in your testing.

Full list of Sauce Labs supported devices and browsers.

When to Test on Emulators and Simulators#

If you need...

  • Massive concurrency
  • To reduce build times
  • To save costs
  • Immediate availability
  • Extremely low error rates for your test environment

System Requirements#

iOS Mobile AppsAndroid Mobile Apps
Apple logoAndroid logo
Requirements

Your iOS app must be:

Compiled for the simulator/device version of your choice

Compressed into a .zip package/archive file (must include app directory)

Uploaded and hosted in a place that Sauce Labs can access (for example: AWS, GitHub, or Sauce Storage)

Your Android app must be:

Compiled for the simulator/device version of your choice

Configured to have internet permissions

Built into an .apk package/archive file

Uploaded and hosted in a place that Sauce Labs can access (for example: AWS, GitHub, or Sauce Storage)

Versions supportediOS versions 10.3 and higherAndroid versions 5.0 and higher
Tips

If you're using Sauce Storage, get the returned location, which will look something like sauce-storage:myApplication.zip.

In your test capabilities, specify the location of the .zip file, or the sauce-storage:myApplication.zip URL as described in App Storage.

This StackOverflow article contains instructions on how to build an .apk file in Eclipse.

In your test capabilities, specify the location of the .apk file, or the sauce-storage:app.apk URL as described in Sauce Storage.

For the full list of supported real devices, see Supported Browsers and Devices.

When to Test on Real Devices#

If you need...

  • A breadth of device types for panel/compatibility testing
  • To replicate an issue to match exact model as reported
  • Pixel-perfect display testing
  • To test hardware dependencies like CPU, memory, display, GPS
  • To test native ARM Libraries
  • To test on a custom OS (e.g., Samsung TouchWiz, OnePlus OxygenOS)
  • To test on a native framework like Espresso and Robotium
  • To test scenarios that require network connectivity (e.g., phone calls, send SMS messages)

Supported Features#

The Sauce Labs Real Device Cloud (RDC) provides you with the ability to run live and automated tests across Android and iOS devices. Some of the features that are specific to our RDC include:

  • Manual testing on real devices.
  • Choice of public real devices or private real devices unique to your organization.
  • Support for Appium, Robotium, Espresso, and XCUITest frameworks.
  • Support on hundreds of device/OS combinations.
  • IPSec VPN secure connections to private device clouds.
  • Devices cleaning process between sessions to ensure maximum privacy.
  • Carrier Network Connectivity (devices with SIM cards).

System Requirements#

iOS Mobile AppsAndroid Mobile Apps
Apple logoAndroid logo
Requirements

Your iOS app must be:

Formatted as an .ipa file. Refer to the documentation on how to create an .ipa file

Uploaded and hosted in Sauce Labs storage or installed from a remote location.

Your Android app must be:

Built into an .apk package/archive file.

Configured to have internet permissions

Uploaded and hosted in Sauce Labs storage or installed from a remote location.

Versions supportediOS versions 9.3.6 and higherAndroid versions 5.0 and higher

For the full list of supported real devices, see Supported Browsers and Devices.

Public vs. Private Real Devices#

Sauce Labs offers access to both public and private device clouds for your real device testing. Review the use cases below to see which option will suit your testing needs. Learn more here.

Use Cases: Public Devices#

Our public cloud contains a wide selection of thoroughly cleaned devices.

  • The devices available on the public cloud are sufficient for your testing coverage.
  • You need to reproduce bugs on a selection of hundreds of iOS and Android devices.
  • You need to upload and spot check apps on devices you don’t have access to.
  • You need to share manual test sessions and devices across teams worldwide.
  • You are looking for a low-cost real device testing option.

Use Cases: Private Device#

Enterprise Plans Only

Dedicated pool of devices just for your organization.

  • You need a very specific set of devices that aren't supported on the public cloud.
  • Your security team insists on dedicated devices.
  • You require a secure tunnel between your tests and our cloud to test.
  • You want to run automated/parallel tests across multiple devices.
  • You need specific settings which are set on the devices all the time.

Security#

Real Device Cleaning#

We use a proprietary process that wipes every real device clean at the end of the testing session:

  • User accounts and data are cleared from the device.
  • History and user data is cleared from the browser.
  • Network settings are reset.
  • Device settings are reset.
  • Your app is uninstalled.
  • Cached data is deleted.
  • Device is de-allocated from you.
caution

While we take these actions to clean public real devices after each test session, we do not perform factory resets nor do we have anti-virus software installed on them. It is possible that other users of the public RDC may engage in malicious, careless or unsecure activity, and that sophisticated, persistent malware could therefore be present on any device in the public RDC.

For more information on Sauce Labs security settings, see Security Settings for Organizations.

Data Center Security#

Real Device Cloud Data Center security is described in Data Center Endpoints.

Static and Dynamic Device Allocation#

Regardless of the test frameworks you're using (Appium, Espresso, XCUITest), you can configure your real device tests using static and dynamic device allocation. While the syntax may be different (i.e., --device, deviceName), the functionality is the same across all frameworks.

Static Device Allocation#

Specifying the exact device to use in your tests by providing the Device ID, which you can find under Live > Mobile-App > Choose device > Find Your Device > Details.

Sauce Labs Device ID example

Static allocation example — exact device names are provided.

--devices iPhone_11_13_5_real_us,iPhone_5
--device Samsung_Galaxy_S20_real

Dynamic Device Allocation#

Specifying basic parameters for the platform, operating system, and/or type of device you want to use in your tests using regular expressions (regex). A device(s) with your specifications will be selected from the real device pool.

Regex InputDynamic Allocation Action
"iPhone.*" , "iPhone .*"Allocates any iPhone.
".*nexus.*"Allocates any device with the word "nexus" in its display name.
"iPhone [67]" or "iPhone [6-7]"Both will allocate either an iPhone 7 or iPhone 6 device.
"iPhone [67]S" or "iPhone [6-7]S"Both will allocate either an iPhone 7S or iPhone 6S device.
"iPhone 7.*"Allocates any device that starts with the display name "iPhone 7" (e.g., iPhone 7, iPhone 7S).

Dynamic allocation example - finds any device that starts with the display name "Google".

capabilities.setCapability("deviceName", "Google.*");

<<<<<<< HEAD Excludes a specific device.

capabilities.setCapability("deviceName", "^((?!.Google_Pixel_XL_real_us.).)*$");

NOTE: A matching device must be present in your account in order for the test to run. Regex values are not case-sensitive (i.e., "iphone .*S" and "IPHONe .*s" are the same).

Additional Resources#

Last updated on by Kim