Authoring and Execution of tests is an important aspect of test automation. Tests should be simple to write, to understand, and to execute across projects. The chosen platform should give the flexibility to both record and playback tests and write custom code to extend the functionalities of the automation framework. I recently came across Angie’s article on 10 features every codeless test automation tool should offer. She does a great job of discussing different aspects of test automation that needs to be an integral part of any automation platform.
Angie’s breakdown appeals to the heart and soul of what we set out to do when we built Testim. Starting from her explanation of why record and playback tools fail (we discuss some of these issues in this post as well) to the different challenges mentioned in her article.
We are really proud of what we do at Testim and wanted to address how we handle some of the important aspects of test automation pointed out in her article. We also highlight how we use AI to solve the “maintenance” issue which is arguably the hardest challenge of test automation.
Testim’s founder (Oren Rubin) coined the term “Smart element locators” in 2015, when he gave the first demo of Testim. He showed us how AI can be used to improve locators. Using hundreds of locators instead of a single static one, Testim can learn from each run and improve over time.
With static locators (e.g. CSS-Selector/XPath), we use only one attribute of an element to uniquely identify it on a page and if this changes, the test breaks and as testers, we end up spending a considerable amount of time troubleshooting the problem and fixing it. Based on research, about 30% of testers’ time is consumed in just maintaining tests. Can you imagine the opportunity cost associated with this effort?
A Testers’ time is valuable and better spent on actually exploring the application and providing information to help stakeholders make informed decisions about the product. With AI based testing we can overcome this problem by using dynamic locators. Dynamic Locators is a concept where we use multiple attributes of an element to locate it on on the page instead of a single attribute. This way, even if one attribute changes, the element can still be successfully located with the help of other attributes that have already been extracted from the DOM by the AI. Testim is based on Dynamic Location Strategy making your tests more resilient to change.
- Conditional waiting
- Control structures
Testim supports “if” statements and “loops”. Looping can be applied on the test level by parameterizing your tests with different datasets (aka Data Driven) or on a specific subset of actions (something super hard, that only Testim supports). These conditions (when to stop the loops) can either be simple, predefined (such as element or text being visible) or can be more complex with custom code. This has been an integral part of Testim since the first version.
- Easy assertions
Assertions are one of the most widely performed actions with test automation. You want to validate an element based on different conditions. For example – If a particular element needs to appear on a particular page of your website, we need to add an assertion to validate the presence of element. With Testim, we made it easy for users to add assertions with a single mouse click and built all of them within the tool itself.
Users have various validation options that include:
- Validate element visible
- Validate element not visible
- Validate element text
- Validate via API call
- Validate file download
- Validation via custom JS code running in the browser (great for custom UI)
- Validation via custom JS code running in node.js (great for pdf and DB validations)
- Visual validation – via Applitools integration*.
Testim integrates seamlessly with Applitools, a Visual Assertion platform, which allows you to validate not only the text, but also its appearance like font and color.
- Modification without redo
Testim not only supports easy modification of steps, the platform also supports full version control, including creating branches and auto-sync with github.
In Testim, you can add a step at any point in your test.
You can also easily delete or modify any step in your test
- Reusable steps
Testim supports reusability by grouping several steps together and the ability to reuse the same group in another test (and passing different parameters e.g. a login),
For Example – The simple steps to log into an application, is one of the most commonly repeated steps in test automation. In Testim, you can create a reusable “Login” step by selecting the steps we want to group together and click on “Add new Group” as shown below.
Not only does Testim support the creation of reusable components as explained above, the platform also supports maximizing the use of reusable component through a feature called Group Context. Imagine you have one or more components (E.g. gallery of images) within a page or across several pages, and you need to easily state on which component to perform the group of actions. Although this is relatively doable in coding (via the Page Object design pattern), this was extremely hard to implement in low code tools until now with the release of Group Context. Testim is the only codeless platform that can currently support this action.
- Cross-browser support
Testim uses Selenium underneath the hood, and supports test execution on ALL browsers, even mobile web, and mobile native (iOS/Android) which is currently in private beta. Signup for a free trial to execute tests on different browser combinations that include Chrome, Safari, Edge, Firefox and IE11.
It is vital to get quick feedback on your test runs, especially root cause analysis. The reports generated should be easy to read and needs to have relevant information on the state of the test. In Testim, there are different levels of reporting to help users know what exactly happened within each test run. Some of the features worth mentioning here include.
While each test is recorded, the platform takes screenshots of all the Passed and Failed results for each step. As a result, users find it easier to troubleshoot problems and understand what happens underneath the hood.
The user gets feedback on all the Passed or Failed steps in a test by showing a “Green” or “Red icon” on the top left portion of the step as shown below.
- Entire DOM captured on failure
On failure, the user also has the option of interacting with the real HTML DOM and see what objects were extracted during the run.
Logs are a rich source of information on what happened underneath the AUT. Testim provides test logs when the user runs the tests on the grids. The option can be found in the in top section of editor.
We have suite and test runs views that enables the user to get granular details on each test run in terms of when the test ran, what was the result, the duration of the run, the level of concurrency, what browser the test ran on and much more. We also have filters to drill down based on different options.
We make it easy to get a high level health check of your tests using the Reports feature. There is no other testing platform that currently exists, that can give this level of details about the tests and this is done all in one single dashboard. We give details that include what percentage of test runs passed, number of active tests, average duration of tests, how many new tests were written, how many tests were updated, how many steps have changed, what are the most flaky tests in your test suite and all these details can be filtered based on the current day, 7 day or a 30 day period.
- Ability to Insert Code
In addition, Testim has test-hooks to run before or after each test/suite.
For example, if you want to validate a particular price on a web page, you can grab the price, convert the string to a number and do the necessary validation. In the below example we are validating that, the price is not over $1000.
- CI/CD Integration
Testim easily integrates with ALL CI servers (e.g. Jenkins, Circleci, VSTS) by literally just copying the automatically generated CLI command and pasting it in your build servers. We integrate with ALL 3rd party grids hosting, that supports Selenium and Appium (Amazon/SauceLabs/BrowserStack…). We also support locally hosted grids and also provide our own Testim grids.
Apart from all the above mentioned features to help build stable automated tests, we also have a notable feature to make tester’s life a lot easier by saving time to report bugs
One of the most time consuming aspects of testing is bug reporting, where in, after finding a bug, we need to report it to the developer with relevant information, to speed up the troubleshooting and fixing of issues.
With Testim you can do this with a single click, with the help of our chrome extension. All the details related to the bug are automatically generated for you in matter of seconds.
In summary, we wanted to build a tool that could help in the authoring, maintenance and collaboration, which we consider the 3 pillars of test automation. Hopefully this post helps to highlight this aspect.
We also love to hear your feedback about our tool, so feel free to reach out to us by not only trying out Testim for FREE but also getting a free consultation on Test Design and Test Automation on your existing testing frameworks and practices. Remember as Steve Jobs said “Simplicity is the ultimate sophistication” and this is the basis on which Testim was created.