Camera Image Injection

Real Devices Only

Do you have a mobile app with the ability to take images on the device camera, then process or store them within the app (e.g., scanning/depositing a check in a banking app)?

Camera Image Injection, also known as camera mocking, is a Sauce Labs Real Device Cloud (RDC) core feature that simulates taking a picture through your mobile app. It then allows you to test your app's camera-based functionality and deliver the best possible user experience.

You employ the built-in device camera in your live and automated testing and perform test cases that require taking images with any of the device cameras. To mimic the system camera behavior during a test, you'll provide the app with an image of your choosing. Your app will access the camera and instead of getting back the picture of the device camera, it'll retrieve the uploaded image.

caution

Make sure you have a debuggable AND not obfuscated version of your application uploaded to Mobile App Storage.

What You'll Need

• A Sauce Labs account (Log in or sign up for a free trial license).
• Upload your app to Sauce Labs prior to testing. The Camera Image Injection functionality points to Sauce Labs storage to get your app information.

Key Specs

Camera Image Injection is available for testing on all Sauce Labs Android and iOS real devices, public and private real device clouds, and front-facing and rear-facing system device cameras. Image uploads must be JPG, JPEG, or PNG format, up to 5MB.

	Supported	Not Supported
**Device Type **
Android real devices	✓
iOS real devices	✓
Android Emulators		✕
iOS Simulators		✕
App Type
Flutter(iOS/Android)		✕
React Native(iOS/Android)		✕
Cordova (iOS/Android)		✕
Framework Type
Appium	✓
Espresso (Android)		✕
XCUITest (iOS)		✕

Not Supported

• Mobile browsers and pre-installed system apps.
• Cross-platform development frameworks like Flutter, React Native, and Cordova (libraries and frameworks are not supported).
• Ephemeral apps (apps with temporary, disappearing messages and photos).

Android System Camera

For Android devices, there are multiple ways to capture an image, as described in the Android Camera API developer documentation. We support the following:

• ACTION_IMAGE_CAPTURE Intent: opens the system camera and notifies the calling app gets when the image is taken.
• camera2 API: everything is configured and handled from within the app.
• cameraX: leverages the capabilities of camera2, but uses a simpler, use case-based approach that is lifecycle-aware.
• Camera API (deprecated): partially supported. As with camera2, everything is handled in the app itself. QR Code readers often use Camera#setPreviewCallback. We pass the injected image to this method, but the rest of this deprecated API is not supported. UI Elements will not likely display the injected image.

iOS System Camera

For iOS devices, the camera can be configured with different outputs. We support the following:

• AVCapturePhotoOutput: for capturing still images. The results are received via the AVCapturePhotoCaptureDelegate and the method captureOutput:didFinishProcessingPhoto:error: The other methods in this delegate are either deprecated or handle live photos, which we don't support.
• AVCaptureVideoDataOutput: for capturing video frames and processing them. The frames are received via AVCaptureVideoDataOutputSampleBufferDelegate and the method captureOutput:didOutputSampleBuffer:fromConnection:.
• AVCaptureMetadataOutput: for reading QR-Codes. The QR Codes are passed to the app via captureOutput:didOutputMetadataObjects:fromConnection:. We are detecting the AVMetadataMachineReadableCodeObject and QR Codes are part of that.

Example Use Cases

Below are common use cases ideal for implementing Camera Image Injection in your tests.

• Scanning a check for a mobile banking app deposit: Many mobile banking apps allow customers to deposit checks using their smartphone. The customer takes and uploads an image of their physical paper check, and the image is then submitted to the bank for processing.
• Using a QR code to link to an embedded URL: QR codes are often used as a way to bridge print media to digital. Users take a photo with a QR code reader app, the app scans the code and directs them to an embedded URL. For use cases that involve scanning barcodes or QR codes, your own application in testing must do the actual image processing. Camera Image Injection passes your uploaded image directly to your app as if it came from the device camera; it does not do any processing.
• Taking a selfie for a user profile photo: This could be taking a selfie or uploading a picture for apps that require a user profile photo. You can use Camera Image Injection to test image formats and sizes.
• Taking an image to store or send via mobile app: Whether it’s a social media app or photo sharing, this use case can encompass many different scenarios. In its simplest form, it could be taking pictures from the front or back camera to send and/or archive within the app.

Live Testing

During a live test, you'll be prompted in Sauce Labs to upload a photo that will be fed to your app, rather than using your device camera to take the photo.

1. In Sauce Labs, click App Management, hover over the test and then click Settings.

1. On the Settings page, ensure that Instrumentation and Image Injection is enabled and then return to the App Management page.

4. On the App Management test page, hover over the test and then click Start Test.

5. On the device selection page, hover over a device and then click Start Test.

6. When you want to capture an image of the test, in the right toolbar, click Tools and then click Camera Upload.

7. Click Choose Image and navigate to the image you want to use.

If the image upload is successful, you will see a thumbnail of the image and a successful upload message.

8. Activate the camera inside of your app. The device will show your uploaded image in the app as if the image was taken by the device camera. The image will continue to be available, should you go back to the camera during your test session, or you can upload another image and capture it with the camera.

Testing with QR Codes

You can use the Camera Image Injection feature to upload QR codes for testing.

1. In the live test window, in the right toolbar, click Tools and then click Camera Upload.

2. Click Choose Image and navigate to the QR code file.

If the image upload is successful, you will see a thumbnail of the image and a successful upload message.

Padding

Sometimes when you upload a QR code image your app will not recognize it because it doesn't fit in the target area defined by your application. This is because image injection strictly overrides what is seen by the camera and your image will be scaled linearly to fit the camera image size as best it can. But this might not always be the same as what your application is expecting.

If your application only registers a code being read when it is within this target area, you will need to take this behavior into account when testing your application. Adding extra space (or "padding") around your image before uploading it for image injection will ensure that the QR code is within the target area that your application defines.

Automated Testing

During an automated test, you'll pass an image to the image injection endpoint. Image injection intercepts the camera input and replaces the camera output with the image file via the camera APIs. When the app requests an image from the device's camera, we inject your uploaded image into the response (the app doesn't know the difference).

In your test script, you'll need to input the desired capabilities specific to Camera Image Injection (see below code snippets). The code will execute your image upload and opening of the device camera.

1. First, add the camera instrumentation desired capability command, imageInjection, to your test script. This capability enables image injection functionality.

Webdriver.io example

exports.config = {
//...
capabilities: [
{
deviceName: 'Samsung Galaxy S10',
platformName: 'Android',
platformVersion: '10',
automationName: 'UiAutomator2',
// Enable image-injection on RDC
imageInjection: true
}
]
//...
}

Java example

var desiredCapabilities = new DesiredCapabilities();
desiredCapabilities.setCapability("deviceName", "Samsung Galaxy S10");
desiredCapabilities.setCapability("platformVersionName", "10");
...
// Enable image-injection on RDC
desiredCapabilities.setCapability("imageInjection", true);

2. In this step, you're adding the image injection method to your test script by providing a file path to your image. You can put the below code snippet into your test script in one or more places. Then, when you execute your test, the code snippets will call the endpoint and pass the image to the app for further processing or for other use.

To change the image, you'll need to send a custom command with a different image. Note that your image file path must be converted to base64 encoding.

Webdriver.io example

const { readFileSync } = require('fs')
const { join } = require('path')

// Read the file from your project and transform it to a base64 string
const qrCodeImage = readFileSync(
join(process.cwd(), 'assets/qr-code.png'),
'base64'
)

// Provide the transformed image to the device
driver.execute(`sauce:inject-image=${qrCodeImage}`)

Java example

import java.util.Base64;
import static org.apache.commons.io.IOUtils.toByteArray;

// Read the file from the classpath and transform it to a base64 string
FileInputStream in = new FileInputStream("/Users/enriquegonzalez/Desktop/Gorilla.png");
qrCodeImage = Base64.getEncoder().encodeToString(
        toByteArray(in)
);
// Provide the transformed image to the device
((JavascriptExecutor)driver).executeScript("sauce:inject-image=" + qrCodeImage);

Common Errors

Here are some common errors you may see in the course of testing with camera image injection and how to resolve them.

Error: Image injection failed

This error is displayed when you attempt to inject your image before the app fully loads during your initial test session startup. You must wait until your app has fully loaded prior to injecting your image.

Error: Image injection is not enabled for the application

This error is displayed due to one or more of these reasons:

• Enable Image Injection checkbox is not checked; this needs to be checked.
• For Android tests, the debuggable flag (android:debuggable="true") is missing from your app's manifest file.

Additional Resources

• Sauce Labs Image Injection code examples (GitHub).
• Android Camera API | Google Developer Documentation.