Headless Browser Testing 101

Headless browsers are gaining in popularity as a viable option for testing web apps. However, when determining your test strategy, it’s important to understand how headless browsers can help improve your efforts, and where they might hinder them.

This article explains what headless browser testing is and which use cases it support. It also identifies the limitations of headless browser testing, and discusses situations in which you may not want to use it.

What Is a Headless Browser?

“Headless browser” is a funny-sounding term that refers to something that software test engineers use all the time: a web browser that is not configured with a Graphical User Interface.

A headless browser is a type of software that can access webpages but does not show them to the user and can pipe the content of the webpages to another program. Unlike a normal browser, nothing will appear on the screen when you start up a headless browser, since the programs run at the backend.

Like a normal browser, a headless browser can parse and interpret webpages (although behaviors can differ between different browser implementations), so it can provide a real browser context without any of the memory and speed costs of running a full-fledged one with a GUI. For example, a headless browser should be capable of parsing JavaScript, clicking on links, coping with any downloads, etc. Executing a headless browser typically means doing so via a command line interface or using network communication.

Google Chrome and Firefox both have versions of their web browser with a headless option. Other browser options that offer some form of a headless implementation include: HtmlUnit, TrifleJS, Splash, and SimpleBrowser.

Headless browsers may not be very useful for surfing the Web, but they are a great tool for testing. Here are some things you might want to know about headless browser testing and the advantages and disadvantages of leveraging it for your testing needs.

Headless Browser Testing: How Can It Be Useful?

Automation

Since headless browsers can provide automated control of webpages, they can be used for automating tasks, scripts, and User Interface tests against a browser without the need for starting up the User Interface of the browser. Webpage interactions can be automated such as form submissions, keyboard inputs, mouse clicks, etc. You can also run automated tests for JavaScript libraries. Headless browser testing can enable up-to-date automated tests in a browser environment.

Layout Testing

Headless browser testing can facilitate a lot of layout checks since headless implementations can render and interpret HTML and CSS like a real browser. This means that it is possible to test style elements such as page layouts (e.g. determining the default width of the page, the coordinates of where an element is), color selection for any elements, font used, etc. It can also test for JavaScript and AJAX execution. You can even automate screen captures for layout checks.

Data Extraction

A full instance of a browser would be overkill if you just want to perform website scraping and data extraction. You may not necessarily want to start up a website, go to it, and scrape it. It is much easier and quicker to navigate to a website headlessly, scrape for data, and use the results to test multiple webpages as part of a quality control effort.

Performance

Headless browsers are your friend if performance is critical. They are faster than real browsers since they do not come with all the overhead of starting up a browser GUI, and this equates to quicker results for your tests. For your developers, this means a lot of time saved when they just want to perform quick and small smoke tests where the UI is irrelevant (i.e., “Can I log into the application?” or “Is SSL enabled?”). Being able to perform non-exhaustive tasks via the command line without the need to manually refresh a page or start a browser can save a lot of effort. Keep in mind that the overall performance still depends on the actual tests and the system being tested.

Other Uses

Here are some other use cases that can be leveraged with headless browser testing:

• Running tests on machines without a Graphical User Interface
• Generating screenshots and PDFs of webpages
• Monitoring network application performance
• Capturing a timeline trace of a website for performance diagnosis
• Simulating multiple browsers on a single machine without the resource hog

Limitations of Headless Browser Testing

As illustrated above, many things that can be tested manually in a real browser can be tested using a headless implementation, but you cannot actually use it for all your browser testing needs, and you should perform testing on real browsers most of the time.

At the end of the day, your end users are not using a headless browser, and so you do not want to focus on bugs that only happen in headless mode. In order to test for user experience and do a functional test, you need to mimic a real user and get precise feedback on what they will actually experience on the website. You want to observe the actual behavior and interaction between an end user and the browser.

Sometimes it is necessary to see your test run on a real browser. This can often be an easy and good debugging technique. If you need to do a lot of browser debugging, this can be challenging in headless mode. A headless browser is a different browser, and tests can catch bugs and throw errors that won’t occur on a real browser.

Conclusion

Automated headless browser testing is great for when you need fast feedback on components of you application. They are quick to set up, and can ensure that snippets of code are working with the end-to-end experience. Headless testing early on in the pipeline, for example, on every pull request, can help deliver instant feedback to developers before committing code to the master branch. However, headless browsers can be limiting when performing full UI tests, as it won’t interact with the browser the way your users would. In addition, headless browsers can be limiting if you want to test on the full variety of browser and OS combination to ensure coverage. Finally, debugging tools such as video validation and live (manual) testing cannot be accessed with headless. Therefore, a sound testing strategy should include a mix of both types of browsers, as they offer both QA and development the infrastructure they need to test quickly and reliably.

UPDATE: Sauce Labs has released an industry first, cloud-based headless testing platform - Sauce Headless. Learn more about the product, and how to get started, on the Sauce Headless page.

Daisy Tsang is a Software Developer currently based in Berlin. She has worked at companies in Canada and Germany in the past few years, loves learning about new technologies, and is particularly interested in the open-source space. Her non-technical interests include photography, knitting, baking, and learning new natural languages.