Data Privacy when using SaaS services for browser and mobile testing

SaaS solutions are popular options to enable cross browser and mobile testing at scale for software development and testing teams. They come with a rich feature set and a large variety of browser / operating system combinations, which is essential in today’s shift left driven organizations.

There is however one crucial and critical detail that is often overlooked by organizations that use these type of services— DATA PRIVACY.

When using SaaS solutions for browser and mobile testing, the typical setup is as follows:

While the customer’s build and test infrastructure is behind the corporate firewall, the browsers and mobile devices of the SaaS provider are outside of the corporate network, sitting in a public space.

In an automated or manual test, a request is sent to the SaaS provider, where a browser is started and controlled. In addition to the commands (e.g. open URL, click on button), the test data (e.g. name, birthday, financial information) is also sent to the browser (and therefore to the SaaS provider).

Your data is visible to the SaaS provider

The transfer of the commands and data from the customer to the SaaS provider typically happens through encrypted tunnels which is considered secure. However, when the data arrives at the SaaS provider and is entered into the browser (e.g. via Selenium), all the information is visible in clear text to anyone who has access to the system where the browser or the mobile device is running.

This is not an issue when purely synthetic test data is used for the tests. But many organizations use “production like” or “production” data for test purposes.

When you send data outside of your network you may breach data privacy regulations

When testing with production data, the moment that data leaves an organization’s network, data privacy regulations are potentially severely breached. The potential consequences of allowing private data to be accessed by an external organization are numerous: legal action taken against the company; loss of client trust and business; negative effects on the reputation of the company; identity theft, fraud, etc.

“Terms of Service” of SaaS providers

You grant rights in your data to the SaaS provider

In order to better understand the issue with data privacy, let’s look at what SaaS providers stipulate in their Terms of Service with regards to customer data and data privacy. In most SaaS providers’ Terms of Service you will find something like this:

• You are not supposed to use any personally identifiable information (e.g. production or Pseudonymized data) for your tests.
• For any data you send (production but also synthetic, anonymized etc.), you grant the SaaS provider the right to use, copy, store, modify and display the data. This effectively means that anyone at the SaaS provider can look at and use your data.
• You as an organization need to make sure that you have the right to make customer data available / export to a 3rd party and also that you are allowed to assign rights in that data to a 3rd party.

What are the alternatives?

One of the main reasons organizations are using SaaS providers is the simplicity and ease of running cross browser and mobile tests, without the struggle of building and maintaining their own Selenium Grid.

Element34’s SBOX — Enterprise Selenium Grid enables organizations to run their entire testing pipeline behind their firewall while at the same time enjoying all the rich feature sets and comforts of SaaS solutions.

Bring your browser and mobile testing infrastructure securely behind your firewall while enjoying the same rich features that SaaS solutions offer

SBOX is an enterprise level Selenium Grid that runs securely inside an organization’s network. No data leaves the corporate network and no external access is required.

‍