10 cases when it’s worth looking for workarounds
In companies that work on Agile and DevOps, it’s difficult to achieve fast and high-quality releases without testing automation. But the course of one hundred percent automation with the same share can become a failure, because there are limits to everything. Sometimes autotests require even more money and time than manual ones. And it happens that tests cannot be automated at all through a browser using Selenium. Let’s look at situations where automation using this tool doesn’t make sense.
What is Selenium good at?
Selenium is an open source tool that is used to automate testing of web applications. Since its creation in 2004 by Jason Huggins, it has become the most popular instrument. Its wide popularity was promoted by open source code, a large number of integrated frameworks and libraries, as well as free use. In addition, test scripts can be written in seven popular programming languages, and tests can be carried out on Windows, Mac and Linux and in five widely used browsers (Mozilla Firefox, Internet Explorer, Google Chrome, Safari or Opera).
Selenium has become so popular in quality assurance services that it’s already identified with automated testing. According to the Test Automation Landscape report for 2020, 54% of specialists used Selenium as a testing automation tool. However, it’s not universal, and there are a number of test scenarios when it’s impractical to use it.
CAPTCHA
CAPTCHA is a fully automated public Turing test that allows you to distinguish computers from people. CAPTCHA is used by any site that seeks to limit the visit of bots. For example, to prevent spam in registration systems, to impede the creation of a fake account, to hamper false entries to free events, and so on.
Therefore, CAPTCHA cannot be verified by automated tests. If this suddenly happens, then it means that it’s poorly implemented on the site.
But there are a couple of loopholes that allow you to bypass CAPTCHA checks and crank automated tests. For example, disable it in your test environment by asking the development team to make a similar function in the software under test. Or use a crawling service that has an API to return the CAPTCHA text. The average response arrives within 9 seconds, and the result is about 90%.
Visual testing
During visual automatic testing, the correct display of the page for the user is checked. Whether the elements are positioned correctly, whether the color of the “Buy” button is set correctly, whether the right font is used in the header, whether the page is scaled correctly on devices of different sizes. Selenium is not suitable for this, it’s better to use a special tool for visual testing.
Two-factor authentication
With automated Selenium tests, it’s not possible to verify two-factor authentication (2FA). After all, a one-time password generated by mobile authenticators comes to the user via SMS or email.
But there are nuances here too. You can, for example, disable 2FA for certain users in a test environment and use all credentials for automation. Or disable 2FA for certain IP addresses.
Uploading files
Automation scripts for uploading files via the UI is an extremely difficult mission, because the API doesn’t display the download process. In addition, this is not such an important nuance of simulating user interaction with the application.
Therefore, it’s worth thinking carefully about how to complete the work-around. By the way, you can use Selenium to find a link along with all the necessary cookies and transfer it to the HTTP request library (REST Assured (Java), fetch (JavaScript) or libcurl (cross-platform)).
HTTP response codes
HTTP status codes are standard response codes (for example, 404 or 500) provided by the server. With their help, you can find out the reason why the web page doesn’t load or the server doesn’t find it. In automated functional testing, checking the HTTP response code is not the most important detail of a test failure. The preceding steps are more valuable.
There is no direct method for checking the response status in Selenium WebDriver, so you should use another library for this: REST Assured (Java), fetch (JavaScript) or RestSharp (.NET).
Log in to email and social networks
It’s not recommended to use WebDriver to log in to sites of this type. This violates the terms of use, works slowly and chaotically. After all, longer tests are less reliable, which significantly increases the probability of failure.
To eliminate the risks, software testing companies such as Andersen should pay attention to the APIs offered by email providers. Or, if we talk about the Facebook network, use the developer tools service, which provides an API for creating test accounts.
Using the API may seem like a difficult task, but in return you will get speed, reliability and stability. The probability that the API will change is small. But it’s worth bearing in mind that web pages and HTML locators often change, so you need to update the test environment.
Performance testing
It is usually not recommended to conduct performance testing using Selenium and WebDriver. This tool is simply not optimized for such a task and the tests have some external and internal vulnerabilities: for example, the speed of browser startup, the speed of HTTP servers, the response of third-party servers hosting JavaScript or CSS, and so on. These variations can affect the result and speed of the performance test in different ways.
To improve the performance of a web application, it’s worth being able to evaluate overall productivity regardless of differences in the environment, detect poor-quality code methods, and break down the efficiency of individual resources. There are performance testing tools that are capable of implementing such tasks, providing reports and analytical data, and can even offer options for improvement.
Pause on links
Using WebDriver to transfer links is also not a good idea, because the tool needs time to run. The process can take from a few seconds to one minute to go to the page and go through the document object model. The speed depends on how well the test is written. To describe the logic of navigating through pages and capturing links, you can use simpler methods (libraries or special services).
Video streaming
The issue of testing the quality of streaming is so complicated that even at one of the last Google testing conferences (GTAC), the company couldn’t solve it. Today, the best way to test a video is to ask people to watch the video and rate its quality on a scale from 1 to 10. But such a method is expensive, slow and non-scalable.
Automated video quality testing is a code that “scans” each frame of a video in different network conditions, on different devices, and examines pixel patterns of an image for failures. But this method is imperfect: video quality can deteriorate for many reasons, and its full impact on the viewer is difficult to assess.
Therefore, streaming video testing is not easy to automate through a user interface using Selenium. This tool is not capable of recognizing video controls. Instead, you should use the capabilities of StreamTest and other programs that can measure the quality of the end user’s work.
Crash recovery
Recovery testing is one of the software testing services that checks whether the program is able to resume work after failures. It’s advisable not to automate these checks, but to carry out them manually. To test reliability, data is collected at various stages of development. Tests are limited due to cost and execution time.
These are the ten main cases when tests shouldn’t be automated using the user interface. These nuances don’t overshadow the usefulness of Selenium for software testing companies. We just wanted to warn about situations when it’s worth connecting narrower services to help, looking for loopholes. Even if there are workarounds, it’s always wise to consider the reasons why specific elements shouldn’t be automated in the first place.
