Skip to main content

Live Mobile Application Testing

With Sauce Labs, you can test your mobile applications on a variety of Android and iOS devices. If you do not have an app, consider using the Sauce Labs Swag Labs sample app for validating your account functionality as well as your tests.

What You'll Need#

Uploading an App#

You can upload your app via the Sauce Labs UI or via the REST API. For information about uploading via the API, see Upload Files with the REST API.

To upload an app via the Sauce Labs UI:

  1. On Sauce Labs, in the left panel, click LIVE and then click Mobile App.

  2. Click App Upload. You can either drag and drop an application, or browse for and select the file. We currently support *.apk Android app files, *.aab Android App Bundle files and *.ipa or *.zip iOS app files (*.zip files are parsed to determine whether a valid *.app bundle exists). Non-app file uploads are not supported in the UI at this time, but can be uploaded through the API.

    If you don't have an app to test, you can use the Sauce Labs sample mobile app.

Upload an application

Deleting an App#

Deleting an app in Sauce Labs will delete the whole application (i.e., the group of builds belonging to the same app package). Files associated with app identifiers (i.e., belonging to the same platform and accessible to the same team) are indicated by the + symbol next to version number. Also, the version number shown is the most recently updated file, not necessarily the latest version of the application.

To delete an application, on the Mobile App test page, hover over the test and then click Delete.

Delete an application

App Settings#

To view or change the application settings, on the Mobile App test page, hover over the test and then click Settings.

Application settings

To easily copy a test's file name or ID, hover over the test and then click the clipboard icon.

Copy a file name or ID
note

The application settings screen is only available for real device testing.

Default App Settings

SettingDescription
Device LanguageUse the dropdown to select the device language.
Device OrientationUse the dropdown to set the device orientation (Landscape or Portrait).
ProxyEnable/disable the use of a proxy. Enter the Hostname and Port and then click Update.
InstrumentationEnable/disable device instrumentation. Enabling allows you to use advanced features when testing your app in the real device cloud, like image injection and taking screenshots of secure views.
Image InjectionEnable/disable image injection. Image injection allows you to mimic camera behavior when testing applications by letting you upload an image and present it to the application as if it were read by the device camera.
Bypass Screenshot Restriction

ANDROID ONLY

Enable/disable Bypass Screenshot Restriction (not supported on applications uploaded to the legacy sauce storage). Enabling Bypass Screenshot Restriction allows you to take screenshots of your app during tests, even if your app does not allow screenshots for security reasons.
System Alerts Display

iOS Only

Enable/disable a system alerts delay. Enabling delays alerts, such as asking for permission to access the camera, to prevent app crashes at startup.
Biometrics InterceptionEnable/disable biometrics. Enabling allows you to choose authentication options if your mobile app requires a biometric authentication, such as fingerprint or face recognition on Android, and Face ID or Touch ID on iOS.
This setting is disabled by default for iOS apps.
Group Folder Redirect

iOS Only

Enable/disable a group directory redirect. Enabling allows you to use your app's private app container directory instead of the shared app group container directory. When your app gets resigned, the shared directory is not accessible.
note

Any changes you make to the application settings will affect all uploaded versions of the application.

Example Settings - iOS

Application settings - iOS

Example Settings - Android

Application settings - Android

Most settings update automatically, however, when you make changes to the proxy setting, click Update to finish.

Selecting a Device#

You must select a device prior to launching a session.

On the App Selection page, hover over the app you want to test and then click Choose Device.

Choose a device

The device selection page will open, with the option to test on a real device or a virtual device.

note

If you are testing an iOS app, the Virtual Devices tab will only appear if the app is configured for simulators.

Real Devices#

On the device selection page, click the Mobile Real tab. Use the search box and filters to find the device you want to test on, or select the device in the grid.

Mobile Real tab

Virtual Devices#

On the device selection page, click the Mobile Virtual tab. Use the dropdowns to select the details for the virtual device you want to test on, and then click Start Session.

Mobile Virtual tab

Public vs. Private Devices#

There is a distinction between Public Devices and Private Devices.

  • Public devices are accessible by anyone with a Sauce Labs account and are subject to availability. If a device is in use, you will see a yellow Busy flag across the thumbnail.
Busy public device
  • Private devices are associated with your account and are an enterprise only feature. Private devices are indicated by a person icon.

Launching a Test#

You can launch a test from the following screens:

  • Hover over the device in the grid and then click Launch.

  • Hover over the device in the grid and then click Details. On the Details screen, click Launch.

    Launch a test from the Details screen

    You'll see a loading screen, and then the app will launch in a live test window using the device you selected.

    Mobile real device test interface

Live Test Interface#

IconNameDescription
Unpin/Pin Toolbar iconUnpin/Pin ToolbarUnpins or pins the live testing toolbar.
Session Info iconSession InfoOpens the Current Session window, which includes app and device details.
Take Screenshot iconTake ScreenshotTakes a screenshot of the current device screen. The image downloads automatically as a .png file.
Share Session iconShare SessionOpens the Share Device window. For a sharable link to the device, click Get Link.
Users must be logged in to be able to view the test.
Rotate Device iconRotate DeviceRotates the device between portrait and landscape.
Home iconHomeOpens the device home screen.
More Device Options iconMore Device OptionsSet Location - Set the GPS location using coordinates or by dropping a pin on the map.
Camera Injection - Opens the Camera Injection window. See Camera Image Injection for more information.
Biometric Injection - Opens the Biometric Authentication window.
Restart App iconRestart AppRestarts the app.
Switch App Version iconSwitch App VersionOpens the Switch App Version window. To change the version of the app you are testing, hover over the version and then click Choose version.
Clipboard iconClipboardOpens the Paste Content Into Device window.
Install Dependency iconInstall DependencyOpens the Install Dependent App window.
Developer Options iconDeveloper OptionsOpens the Developer Options panel, which includes the Device Log, Device Vitals, and Dev Tools tabs.

Device Log#

Device Log
IconNameDescription
Log LevelLog Level
  • VERBOSE
  • DEBUG
  • INFO
  • WARN
  • ERROR
  • ASSERT
Search Log iconSearch LogOpens the Search log interface. Enter a term or terms in the search box and select or deselect the following checkboxes as necessary:
  • REGEX
  • IGNORE CASE
  • INVERT
Pause Log iconPause LogPauses the log feed.
Wrap Text iconWrap TextWraps text in the log for easier reading.
Clear Log iconClear LogClears the log feed.
Download Log iconDownload LogDownloads the log as a .txt file.

Device Vitals#

Device Vitals is a feature that collects useful performance data in real time from a device during a live session. Data such as network, CPU, and memory usage helps users understand the general performance of a device and the application under test. Users can view a graph of this performance data in real time as the app is processing.

Device Vitals
IconNameDescription
Graph HeightGraph HeightChange the size of the graphs (S, M, L).
Grid Layout iconGrid LayoutDisplays graphs side by side.
Row Layout iconRow LayoutDisplays graphs as as a list.
Download Vitals iconDownload VitalsDownloads the device vitals as a .txt file.

Performance Metrics for Android/iOS Devices

The graph and csv file will contain these performance metrics for devices.

MetricDescription
cpu_totalSystem-wide CPU usage in percentage across all CPU cores. 4 cores at max use would be shown as a value of 400%
cpu_userCPU usage for user processes in percentage across all CPU cores. 4 cores at max use would be shown as a value of 400%
cpu_kernalAndroid OS CPU usage in percentage across all CPU cores. 4 cores at max use would be shown as a value of 400%
n_threadsTotal threads in use by the app
memory_size_kbTotal threads in use by the app
memory_resident_kbMemory currently in use by application in kilobytes
memory_shared_kbAnonymous shared memory currently in use by system shared between application(s) and system
network_wifi_receive_bData in bytes received over wifi connection
network_wifi_sent_bData in bytes sent over wifi connection
network_mobile_receive_bData in bytes received from the mobile carrier network
network_mobile_sent_bData in bytes sent over mobile carrier network

Changing an App Version#

Sometimes you need to conduct A/B testing, or document and validate feature parity between different versions of the same application. You can change the app version, as well as the real device, and launch a new test session.

  1. On the App Upload page, click the +n in the Version column.
App with multiple versions
  1. On the Settings page, in the versions list, hover over the version you want to launch.
  2. Click Choose Device.
Change the version of an app

Testing Apple Pay in Mobile Apps#

There are three ways to test Apple Pay with Sauce Labs:

  • Using simulators
  • Using real private devices with an Apple Pay Sandbox Testing account
  • Using real private devices with a real production account and real credit cards

Apple Certificates#

Apple certificates are used to ensure security in their systems, and they are much more strict about them than Android. This level of security makes certificates a very complex part of making Apple Pay work with devices in a cloud.

To give you an example, Android apps can be installed without any specific signing on whatever real device you want. With Apple you have two options, or you need to add a remote device to your developer certificate and the provisioning profile, so you are allowed to install the app on that specific device. Or you need to use an enterprise certificate where the Apple device that has that certificate installed allows you to install the app. Similarly, when you install an iOS app on a device, we re-sign the app with a Sauce Labs enterprise certificate so you can install your app on all Sauce Labs public/private devices.

note

Apple Pay has a limitation that it cannot work with an enterprise certificate. You need to use the developer certificate where the device has been added to the provisioning profile in order to make this work. This can only be done for Sauce Labs private devices on which you have disabled the resigning.

Apple Pay on Real Private Devices#

To make Apple Pay work on Sauce Labs real private devices:

  1. Follow Apple’s steps to enable Apple Pay (see Setting Up Apple Pay Requirements). Apple is strict about certificates, so they require you to follow very specific steps:

  2. Set up Apple Pay integration in your app.

  3. Register the Merchant ID in your Apple developer account.

  4. Set up an Apple sandbox tester account (see Create a sandbox tester account for more information).

  5. Build your app. Apple Pay doesn’t work with enterprise certificates, so it will not work with Sauce Labs out of the box. The first step is to add the Sauce Labs real private devices to your Apple developer certificate before building the app. You can do that in one of the following ways:

  6. Manually adding the device and its UDID to the device list for your developer certificate.

    note

    Your device list can be found on Apple’s Certificates, Identifiers & Profiles page for your developer account, and you can get the UDID of your private device by contacting your Sauce Labs CSM.

  7. Using the Sauce Labs Virtual USB solution:

    1. Start a session with Virtual USB (see Testing with Virtual USB on Real Devices for more information).

    2. When the connection is established, open XCODE.

    3. Select the device from the device list.

      Apple Pay setup - device list

      On the Signing & Capabilities tab, you will see that the device has not yet been added.

      Apple Pay setup - device not added
    4. Click Register Device to add the device to your developer certificate.

      Apple Pay setup - add device to certificate
    5. Once the UDID of the device is added to the developer certificate, you can build the application (manually or automatically):

      1. Select your build scheme and then select Generic iOS Device.
      2. To build the application, click Product and then click Archive.
      3. Click Distribute App.
      4. Distribute the application with Ad Hoc and Automatically manage signing.
      5. Store the application on your local machine.

    If the application has been built, you should not yet upload it to Sauce Labs. The device to be tested needs to be prepared. If you have already prepared the device, then you can skip to step 4.

  8. Prepare the device. Set up the first Sauce Labs private device to use Apple Pay with the Apple sandbox account that was created in step 1.

Disable the Passcode#

Apple Pay requires that you have set a passcode on your phone, and you can’t add cards to your wallet without it. But setting a passcode on a device can break Appium automation because Appium can’t automate the passcode screen. To prevent the testing device from displaying the passcode screen:

  1. On the device, go to Settings > Display & Brightness and disable Auto-Lock.
  2. Ask your CSM to disable rebooting the device by providing them with the unique device name, found in the device details.
    note

    There is no guarantee that the device won’t reboot or show the passcode screen. The test run on the device might be less reliable if the passcode screen appears during the automated session.

Add the Testing Account#

  1. On the device, go to Settings and then click Sign in to your iPhone. Sign in using your Apple sandbox tester account.
Apple Pay setup - sign in to account
  1. If prompted, enter the device’s passcode.
Apple Pay setup - passcode

If you weren’t prompted for a passcode, set it by going to Settings > Face ID & Passcode and tapping Turn Passcode On.

Add Apple Sandbox Test Cards#

Apple test cards can be found on Apple’s Sandbox Testing page.

  1. On your device, go to Wallet. If you didn’t set a passcode, Apple will show a notification.
Apple Pay setup - passcode notification
  1. In Wallet, tap the plus sign to add a new card. Use the card information on Apple’s Sandbox Testing page.
Apple Pay setup - Add new card
  1. Prepare Sauce Labs. As mentioned before, Sauce Labs uses an enterprise certificate to install an app on public and private devices. But Apple Pay can’t work with the enterprise certificate, so the app needs to be signed with the developer certificate. You need to instruct Sauce Labs to not re-sign the app when it is installed.

Disable Re-Signing#

  1. On Sauce Labs, in the left navigation, click Live and then click Mobile-App.
Apple Pay setup - Sauce login

You will see an overview of the already uploaded apps. If no app has been uploaded, then upload the app. Once uploaded, open the app settings by hovering over the row until you see this:

Apple Pay setup - Settings
  1. Click Settings.
Apple Pay setup - Settings
  1. Under Default settings, toggle Instrumentation to Disabled.
Apple Pay setup - Disable instrumentation

Disabling this allows the app to use Apple Pay and the developer certificate and provisioning profile that you used when you built the app.

note

Disabling re-signing will break the installation of the app on public devices. The app will only be allowed to be installed on private devices that have been added to the developer certificate and provisioning profile.

Once the app has been uploaded and re-signing has been disabled, you can start the device and let Sauce Labs install the application on the device.

  1. Test the app. View the Sauce Labs Demo Payments app:
Apple Pay setup - Demo appApple Pay setup - Demo appApple Pay setup - Demo appApple Pay setup - Demo app