Skip to main content

Using Real and Virtual Mobile Devices for Testing

Getting the most out of your live and automated testing means including a healthy mix of emulators, simulators, and real devices as your mobile testing platforms. Why? Most aspects of the mobile experience you can test on emulators or simulators, while some scenarios require testing on physical real devices (e.g., memory consumption, CPU usage, location-based apps that use manufacturer-specific device sensors).

There are a variety of use cases to consider when you're deciding on the mix of emulators, simulators, and real devices to use in your testing.

We support thousands of device/OS combinations and test automation frameworks such as Appium, Espresso, XCUITest, and Robotium. For a full list, click here.

When to Test on Emulators and Simulators#

Use Cases#

If you need...

  • Massive concurrency.
  • To reduce build times.
  • To save costs.
  • Immediate availability.
  • Extremely low error rates for your test environment.

System Requirements#

iOS Mobile AppsAndroid Mobile Apps
Apple logoAndroid logo
Requirements

Your iOS app must be:

Compiled for the simulator/device version of your choice

Compressed into a .zip package/archive file (must include app directory)

Uploaded and hosted in a place that Sauce Labs can access (for example: AWS, GitHub, or Sauce Labs App Storage)

Your Android app must be:

Compiled for the simulator/device version of your choice

Configured to haveĀ internet permissions

Built into an .apk package/archive file

Uploaded and hosted in a place that Sauce Labs can access (for example: AWS, GitHub, or Sauce Labs App Storage).

Appium only. For Espresso, saucectl uploads the referenced app for you.

Versions supportediOS versions 10.3 and higherAndroid versions 5.0 and higher
Tips

If you're using App Storage, get the returned location, which will look something likeĀ storage:filename=myApplication.zip.

In your test capabilities, specify the location of the .zip file, or the storage:filename=myApplication.zip URL as described inĀ App Storage.

This StackOverflow articleĀ contains instructions on how to build anĀ .apkĀ file in Eclipse.

In your test capabilities, specify the location of theĀ .apkĀ file, or theĀ storage:filename=app.apkĀ URL as described in App Storage.

When to Test on Real Devices#

The Sauce Labs Real Device Cloud (RDC) provides you with the ability to run live and automated tests across Android and iOS devices. Learn more here.

Use Cases#

If you need...

  • A breadth of device types for panel/compatibility testing.
  • Manual, interactive testing on actual physical devices.
  • To replicate an issue to match exact model as reported.
  • Pixel-perfect display testing.
  • To test hardware dependencies like CPU, memory, display, GPS.
  • To test native ARM Libraries.
  • To test on a custom OS (e.g., Samsung TouchWiz, OnePlus OxygenOS)
  • To test on a native framework like Espresso and Robotium.
  • To test scenarios that require carrier network connectivity (e.g., making phone calls and sending SMS messages to devices with SIM cards).

Public Device Cloud#

Our public cloud, available to all users regardless of pricing plan, contains a wide selection of thoroughly cleaned devices. They are subject to availability. On the mobile device selection screen, if a device is in use, it'll be marked with a Busy flag. Here are some use cases:

  • The devices available on the public cloud are sufficient for your testing coverage.
  • You need to reproduce bugs on a selection of hundreds of iOS and Android devices.
  • You need to upload and spot check apps on devices you donā€™t have access to.
  • You need to share manual test sessions and devices across teams worldwide.
  • You are looking for a low-cost real device testing option.

Private Device Cloud#

Enterprise Plans Only

This is dedicated pool of devices just for your organization. On the mobile device selection screen, your private devices are marked with a person silhouette icon. Here are some use cases:

  • You need to use a very specific set of devices that aren't supported on the public cloud.
  • Your security team insists on dedicated devices.
  • You want to run automated parallel tests across multiple devices simultaneously.
  • You need specific settings which are set on the devices all the time.
  • You need to establish a secure IPSec VPN connection between your network and the Sauce Labs cloud.

System Requirements#

iOS Mobile AppsAndroid Mobile Apps
Apple logoAndroid logo
Requirements

Your iOS app must be:

Formatted as a .app or .ipa file. Refer to the documentation on how to create an .ipa file

Uploaded and hosted in Sauce Labs storage or installed from a remote location.

Appium only. For XCUITest, saucectl uploads the referenced app for you.

Your Android app must be:

Built into an .apk package/archive file.

Configured to have internet permissions

Uploaded and hosted in Sauce Labs storage or installed from a remote location.

Appium only. For Expresso, saucectl uploads the referenced app for you.

Versions supportediOS versions 9.3.6 and higherAndroid versions 5.0 and higher

For the full list of supported real devices, see Supported Browsers and Devices.

Security#

Real Device Cleaning#

We use a proprietary process that wipes every real device clean at the end of the testing session:

  • User accounts and data are cleared from the device.
  • History and user data is removed from the default system browser. Non-default browsers are uninstalled.
  • Network settings are reset.
  • Device settings are reset.
  • Your app is uninstalled.
  • Cached data is deleted.
  • Device is de-allocated from you.
caution

While we take these actions to clean public real devices after each test session, we do not perform factory resets nor do we have anti-virus software installed on them. It is possible that other users of the public RDC may engage in malicious, careless or unsecure activity, and that sophisticated, persistent malware could therefore be present on any device in the public RDC.

Passcode Usage on Public Devices#

Passcodes cannot be set on public devices. If your app requires authentication, you can:

For more information on Sauce Labs security settings, see Security Settings for Organizations.

Data Center Security#

Real Device Cloud Data Center security is described in Data Center Endpoints.

Static and Dynamic Device Allocation#

Regardless of the test frameworks you're using (Appium, Espresso, XCUITest), you can configure your real device tests using static and dynamic device allocation. While the syntax may be different (i.e., --device, deviceName), the functionality is the same across all frameworks.

Static Device Allocation#

This is specifying an exact device for your test by setting deviceName to the Device ID, which you can find under Live > Mobile-App > Choose device > Find Your Device > Details.

Sauce Labs Device ID example

When using this, there's no need to specify the platformName and platformVersion because they'll be set by default (i.e., if you include these separately included in your test script, they will be ignored).

Static allocation example ā€” exact device names are provided.

--devices iPhone_11_13_5_real_us,iPhone_5--device Samsung_Galaxy_S20_real

Dynamic Device Allocation#

This is specifying basic parameters for the platform, operating system, and/or type of device you want to use in your tests using regular expressions (regex) to dynamically allocate a device. A device(s) with your specifications will be selected from the real device pool.

Regex InputDynamic Allocation Action
"iPhone.*" , "iPhone .*"Allocates any iPhone.
".*nexus.*"Allocates any device with the word "nexus" in its display name.
"iPhone [67]" or "iPhone [6-7]"Both will allocate either an iPhone 7 or iPhone 6 device.
"iPhone [67]S" or "iPhone [6-7]S"Both will allocate either an iPhone 7S or iPhone 6S device.
"iPhone 7.*"Allocates any device that starts with the display name "iPhone 7" (e.g., iPhone 7, iPhone 7S).

Dynamic allocation example - finds any device that starts with the display name "Google".

capabilities.setCapability("deviceName", "Google.*");

Excludes a specific device.

capabilities.setCapability("deviceName", "^((?!.Google_Pixel_XL_real_us.).)*$");
note

A matching device must be present in your account in order for the test to run. Regex values are not case-sensitive (i.e., "iphone .*S" and "IPHONe .*s" are the same).

Additional Resources#