Frequently Asked Questions

No, SBOX is a solution running inside your corporate network on your own infrastructure making it extremely secure and scalable. No external access to the application under test is required.

Building and maintaining your own Selenium Grid is complex and a high level of resources. The open-source Selenium Grid provides basic functionality for running cross-browser tests. However, it is missing the scalability and enterprise-level functionality that many organizations require today. With every new browser, driver, and Selenium release, there is significant effort involved to ensure that the new browser versions are running properly in the Selenium Grid. SBOX is a maintenance-free solution, as the browser, driver, and Selenium updates are managed by us. As a client, you can fully focus on building tests rather than test infrastructure.  SBOX also provides enterprise-level features like live view, video recording, access control via LDAP, and more.

No. SBOX automatically downloads and installs new browsers, drivers and Selenium versions after they are tested and released by Element34. This makes the SBOX completely maintenance free.

No. SBOX implements the standard Selenium/web driver protocol. You can switch to another solution at any time.

No, we do this for you. When we release new images to the SBOX, our customers can be sure that the versions are compatible with each other.

No. Each test is run in an isolated environment and does not share any resources with other tests. A new browser is started for each test. After the test finishes, the browser is closed and the environment is destroyed. Each test is completely isolated and there is no chance of side effects between browsers/environments.

Yes. SBOX has an advanced logging and monitoring mechanism, which can easily be customized and integrated into the customer environment.

After we conduct our tests, new images are usually released to the SBOX within 24-48 hours of the initial release.

Most of our customers have a central SBOX instance running where the various teams connect. SBOX is designed to handle a high load from various sources (i.e. CI system, developers).

Yes. Access control and restriction are a built-in feature of SBOX. If you are using an identity provider i.e. LDAP you can easily set up access groups and restrict the visibility of tests to a certain group of people.

No, there is no limit on the number of tests that you can run. Based on your license tier, there may be a limit on the concurrency (the number of tests you can run at the same time).

While this is entirely up to you (and may depend on your policies), you can run tests with any kind of data on SBOX. The test and test data never leave your network with SBOX. SBOX runs completely inside your network and with that, it is safe to run tests with production data.

Typically we see an increase of factor 3-5 in execution times when the same test is run on SBOX vs. a SaaS solution. This is because SBOX runs inside your network and is close to the rest of the CI/test infrastructure. Latencies are minimized hence the significant increase in speed.

Background: Selenium / WebDriver is an HTTP-based protocol that requires a roundtrip from the client -> server -> browser -> application under test and back for each command. When running on SaaS solutions the browser is running outside of the customer network with potentially large latencies in between.

No. You can run as many tests in parallel as your hardware resources are capable of. If you want to run more tests, you simply need to add more CPU / RAM to the SBOX system.

SBOX runs on any Linux system. Most of our clients run RHEL or CentOS. SBOX typically runs in one or more virtual machines but can also run on bare metal. We can provide detailed technical requirements upon request.

After new browsers are released by the browser vendors we thoroughly test that the browsers are compatible with the driver and Selenium versions. Once all the tests pass on our end we release the bundles to the SBOX installations. SBOX immediately picks up the newly released versions and makes them available for use.

Yes. SBOX integrates seamlessly with your corporate LDAP where you can define usage and access rights for your organization.

Yes. SBOX supports self-signed SSL certificates.

Yes. This is our recommended approach. Larger companies often have specific requirements in terms of infrastructure, network, etc.

We will send you a detailed list of requirements for the proof of concept. On a high level, you need 1-2 Linux machines (ideally VMs) with 8 logical CPU cores each; 16 GB RAM per VM and 150 GB of disk space. We can send you scripts and documentation on how to check that you fulfill the requirements.

Yes. SBOX provides a live view feature which allows you to watch tests in real-time while it is running.

Yes. You can manually interact with the browser where the test is running. This also allows you to open a debug console and debug in real-time.

SBOX provides a sophisticated report to analyze your tests while running or after completion of a run. At the same time, SBOX seamlessly integrates into your client system in case you already have reports set up and want to plug in the SBOX tests.

The fundamental difference between a cloud service and SBOX is that SBOX runs completely in-house or in your private cloud. This results in superior performance over cloud solutions and allows for much tighter integration into your surrounding systems such as CI, identity management and reporting. In addition you have better debugging, monitoring and reporting capabilities as well as greater stability and support for custom requirements i.e. self-signed SSL certificates, unlimited scaling option.

We support Chrome, Firefox, Internet Explorer, Edge, MacOS Safari, WebKit and OpenFin in the most recent and all older versions.

Yes. After a new browser or Selenium image is released to the SBOX, older browser versions are still available. Tests can be executed on the latest as well as older browser versions.

No. SBOX uses the standard Selenium / WebDriver protocol without any modifications.

Yes. SBOX was designed to operate independently from outside access. Banks, insurance and security sensitive organizations are our main client base.

SBOX is based on an annual or multi-year subscription.

Yes. SBOX has built in End to End HTTPS support.

Yes, SBOX fully supports the Appium protocol. You can run your tests on Android and iOS native apps, mobile web and hybrid apps.

No. You can run as many tests in parallel as your hardware resources are capable of. If you want to run more tests, you simply need to add more CPU / RAM to the SBOX system.

No,  all browsers (except OSX Safari) run inside the Linux host. You don’t need to provision and manage any Windows machines for running Internet Explorer and Microsoft Edge browsers.  This is one of our key technical advantages that we are able to run Windows browsers natively in a Windows OS inside Linux.

No. SBOX is a software solution that runs on your corporate servers or in your private cloud.

SBOX is a browser and mobile execution environment. The decision if a test is passed or failed is not taken by SBOX but rather by the client side (e.g. CI system). Therefore, it makes most sense to use a client side reporting tool (e.g. Allure, Extent) to visualize the test results. SBOX also provides an option to pushing the test results from the client to the SBOX. We however discourage this option in favour of client side reporting.

Security (because everything is running inside your network. No data leaves your network, so there are no implications regarding data privacy etc.)Performance (tests are executed extremely fast. There are no latencies, because SBOX is running inside your network and close to the rest of your infrastructure)Integration (SSO, proxies, APIs, authentication)Scalability (you can run as many tests in parallel as you want)

SBOX supports Safari in two ways. You can either use Apple hardware as an executor in which case you can run native Safari browsers and iOS simulators. Alternatively, you can also run a Safari/WebKit browser on Linux which for functional testing purposes provides a safe target to test Safari. Using this approach means you can use your existing SBOX executors and don’t need to provision Apple hardware as executors.

WebKit is the open-source web platform implementation used by Apple Safari and Epiphany. Running a WebKit browser on Linux differs from Apple Safari in the following ways:

- it uses a non-macOS network stack
- it uses non-core Animation to composite scenes and produces image raster. This means that screenshots on Linux will not perfectly match screenshots from macOS.

In terms of running functional tests via Selenium or Playwright the same WebKit code will layout the page and run JavaScript—therefore it will match how WebKit works in Safari.

To summarize, running Safari/WebKit on Linux will use a different compositing pipeline than macOS, but it can be safely used for e2e testing of web applications in WebKit-powered browsers.

SBOX is a fully self-contained system. There is no effort required to manage the SBOX when running in auto-update mode.

Yes. We fully support custom proxies based on your network architecture and requirements.

Yes, SBOX is fully cloud compatible and can be run on physical servers, VMWare clusters, AWS, Google Cloud, Azure etc.

A proof of concept will give the certainty that SBOX satisfies your needs BEFORE purchasing. You will have a fully functional system available for evaluation. During the evaluation period you can run your own tests on SBOX to ensure that you are seeing the results that you are aiming for.

Yes. A video can be taken on a per-test basis. The videos can be easily integrated into the reporting system or CI system.

Yes. You can turn on manual interaction for your tests. This allows you to control the browser and debug your tests in more depth.

Yes. You can see all the Selenium commands executed in the live view window.

Yes. Because SBOX is hosted in-house, it is no problem to also run tests with any supported browser against a build on your local machine.

Yes, you can also run Chrome in headless mode, if you wish.