Author

Raj Subramanian

Browsing

As you may already know, Testim provides the flexibility to extend the functionalities of our platform by giving teams the ability to add their own JavaScript code. That being said, there maybe a situation where you need to use inbuilt javascript methods within the Testim JavaScript editor.  For Example – Say you want to find out of if an element is enabled on the page, you could do this

if(!element.disabled) {

 return true;

}

Now how do you know that, you could use the inbuilt method disabled() here?  The answer is simple.  Just follow the below steps-

  1. Open any web page on your Chrome browser
  2. Right click on an element and select “Inspect”
  3. Navigate to Console tab
  4. Start typing “$0.”
  5. Then observe all the inbuilt methods which are supported by Chrome

The same methods are supported by Testim as well.  Additionally, we support commonly used javascript methods such as reload(), split(), trim() and much more.  For more javascript examples check out our help document here.

Authoring and Execution of tests is an important aspect of test automation. Tests should be simple to write, to understand, and to execute across projects. The chosen platform should give the flexibility to both record and playback tests and write custom code to extend the functionalities of the automation framework. I recently came across Angie’s article on 10 features every codeless test automation tool should offer. She does a great job of discussing different aspects of test automation that needs to be an integral part of any automation platform.

Angie’s breakdown appeals to the heart and soul of what we set out to do when we built Testim. Starting from her explanation of why record and playback tools fail (we discuss some of these issues in this post as well) to the different challenges mentioned in her article.

We are really proud of what we do at Testim and wanted to address how we handle some of the important aspects of test automation pointed out in her article. We also highlight how we use AI to solve the “maintenance” issue which is arguably the hardest challenge of test automation.

 

  • Smart element locators

Testim’s founder (Oren Rubin) coined the term “Smart element locators” in 2015, when he gave the first demo of Testim. He showed us how AI can be used to improve locators. Using hundreds of locators instead of a single static one, Testim can learn from each run and improve over time.

With static locators (e.g. CSS-Selector/XPath), we use only one attribute of an element to uniquely identify it on a page and if this changes, the test breaks and as testers, we end up spending a considerable amount of time troubleshooting the problem and fixing it. Based on research, about 30% of testers’ time is consumed in just maintaining tests. Can you imagine the opportunity cost associated with this effort?

A Testers’ time is valuable and better spent on actually exploring the application and providing information to help stakeholders make informed decisions about the product. With AI based testing we can overcome this problem by using dynamic locators. Dynamic Locators is a concept where we use multiple attributes of an element to locate it on on the page instead of a single attribute. This way, even if one attribute changes, the element can still be successfully located with the help of other attributes that have already been extracted from the DOM by the AI. Testim is based on Dynamic Location Strategy making your tests more resilient to change.

  1. Conditional waiting

Testim supports conditional waits and can be added in a click of a button. We provide built in wait functionalities based on element visibility, text (or regex), code based (custom) waits using JavaScript, waits based on downloaded file and of course the hardcoded “sleep” (which is not generally advisable to use in tests unless you have no other option).

  1. Control structures

Testim supports “if” statements and “loops”. Looping can be applied on the test level by parameterizing your tests with different datasets (aka Data Driven) or on a specific subset of actions (something super hard, that only Testim supports). These conditions (when to stop the loops) can either be simple, predefined (such as element or text being visible) or can be more complex with custom code. This has been an integral part of Testim since the first version.

  1.  Easy assertions

Assertions are one of the most widely performed actions with test automation. You want to validate an element based on different conditions. For example – If a particular element needs to appear on a particular page of your website, we need to add an assertion to validate the presence of element. With Testim, we made it easy for users to add assertions with a single mouse click and built all of them within the tool itself.

Users have various validation options that include:

  • Validate element visible
  • Validate element not visible
  • Validate element text
  • Validate via API call
  • Validate file download
  • Validation via custom JS code running in the browser (great for custom UI)
  • Validation via custom JS code running in node.js (great for pdf and DB validations)
  • Visual validation – via Applitools integration*.

Testim integrates seamlessly with Applitools, a Visual Assertion platform, which allows you to validate not only the text, but also its appearance like font and color.

  1. Modification without redo

Testim not only supports easy modification of steps, the platform also supports full version control, including creating branches and auto-sync with github.

In Testim, you can add a step at any point in your test.

You can also easily delete or modify any step in your test

  1. Reusable steps

Testim supports reusability by grouping several steps together and the ability to reuse the same group in another test (and passing different parameters e.g. a login),

For Example – The simple steps to log into an application, is one of the most commonly repeated steps in test automation. In Testim, you can create a reusable “Login” step by selecting the steps we want to group together and click on “Add new Group” as shown below.

 

 

Not only does Testim support the creation of reusable components as explained above, the platform also supports maximizing the use of reusable component through a feature called Group Context. Imagine you have one or more components (E.g. gallery of images) within a page or across several pages, and you need to easily state on which component to perform the group of actions. Although this is relatively doable in coding (via the Page Object design pattern), this was extremely hard to implement in low code tools until now with the release of Group Context. Testim is the only codeless platform that can currently support this action.

  1. Cross-browser support

Testim uses Selenium underneath the hood, and supports test execution on ALL browsers, even mobile web, and mobile native (iOS/Android) which is currently in private beta. Signup for a free trial to execute tests on different browser combinations that include Chrome, Safari, Edge, Firefox and IE11.

  1. Reporting

It is vital to get quick feedback on your test runs, especially root cause analysis. The reports generated should be easy to read and needs to have relevant information on the state of the test. In Testim, there are different levels of reporting to help users know what exactly happened within each test run. Some of the features worth mentioning here include.

  • Screenshots

While each test is recorded, the platform takes screenshots of all the Passed and Failed results for each step. As a result, users find it easier to troubleshoot problems and understand what happens underneath the hood.

  • Feedback on each step

The user gets feedback on all the Passed or Failed steps in a test by showing a “Green” or “Red icon” on the top left portion of the step as shown below.

  • Entire DOM captured on failure

On failure, the user also has the option of interacting with the real HTML DOM and see what objects were extracted during the run.

  • Test Logs

Logs are a rich source of information on what happened underneath the AUT. Testim provides test logs when the user runs the tests on the grids. The option can be found in the in top section of editor.

  • Suite and Test Runs

We have suite and test runs views that enables the user to get granular details on each test run in terms of when the test ran, what was the result, the duration of the run, the level of concurrency, what browser the test ran on and much more. We also have filters to drill down based on different options.

 

  • Reports

We make it easy to get a high level health check of your tests using the Reports feature. There is no other testing platform that currently exists, that can give this level of details about the tests and this is done all in one single dashboard. We give details that include what percentage of test runs passed, number of active tests, average duration of tests, how many new tests were written, how many tests were updated, how many steps have changed, what are the most flaky tests in your test suite and all these details can be filtered based on the current day, 7 day or a 30 day period.

  1. Ability to Insert Code

Testim gives the flexibility for organizations to extend the functionalities of our platform using JavaScript by either running it in the browser; where Testim can help by finding elements for you in the DOM (aka dependency injection), or by running the code on node.js, which allows loading many common libraries (that includes accessing databases or inspect downloaded PDFs).

In addition, Testim has test-hooks to run before or after each test/suite.

For example, if you want to validate a particular price on a web page, you can grab the price, convert the string to a number and do the necessary validation. In the below example we are validating that, the price is not over $1000.

  1. CI/CD Integration

Testim easily integrates with ALL CI servers (e.g. Jenkins, Circleci, VSTS) by literally just copying the automatically generated CLI command and pasting it in your build servers. We integrate with ALL 3rd party grids hosting, that supports Selenium and Appium (Amazon/SauceLabs/BrowserStack…). We also support locally hosted grids and also provide our own Testim grids.

Apart from all the above mentioned features to help build stable automated tests, we also have   a notable feature to make tester’s life a lot easier by saving time to report bugs

 

Capture Feature

One of the most time consuming aspects of testing is bug reporting, where in, after finding a bug, we need to report it to the developer with relevant information, to speed up the troubleshooting and fixing of issues.

With Testim you can do this with a single click, with the help of our chrome extension. All the details related to the bug are automatically generated for you in matter of seconds.

 

In summary, we wanted to build a tool that could help in the authoring, maintenance and collaboration, which we consider the 3 pillars of test automation. Hopefully this post helps to highlight this aspect.

We also love to hear your feedback about our tool, so feel free to reach out to us by not only  trying out Testim for FREE but also getting a free consultation on Test Design and Test Automation on your existing testing frameworks and practices. Remember as Steve Jobs said “Simplicity is the ultimate sophistication” and this is the basis on which Testim was created.

Testim is #1 in innovation among all other competitors for several years now. This can be seen from the new features we have been releasing constantly to make automation faster, more stable and much more collaborative than ever. In continuation with this, we are excited to bring you our next big feature which we are calling Group Context. Imagine you have one or more components (E.g. gallery of images) within a page or across several pages, and you need to easily state on which component to perform the group of actions. Although this is relatively doable in coding (via the Page Object design pattern), this was extremely hard to implement in codeless tools until now with the release of Group Context. Testim is the only codeless platform that can currently support this action

For those of you not familiar with Testim, let’s start by defining “Context”?

“Context” is key in real life and in coding. You provide the necessary information in order to perform an action. For Example – Say you make a reservation at a restaurant for a party of four people, you would be required to provide a name for the reservation. Here the name is “contextual information” and you making a reservation is the “context”. Similarly, in test automation, it is important to know the context of elements from the DOM level in order to perform an action.

A Context in Test Automation

Context is all the more important when you have reusable components in your automation suite. For Example – let’s take the below web page.

 

It is a simple web page, containing a gallery of items. In this case, it has names of places with description and an option to book them as part of a reservation. In each item in the gallery, you have similar elements such as an image, text, and button, this is because the code for generating those instances is the same. So, on a high level, all these gallery items are exactly the same except for the different data showing up for each element in the gallery item.

Say we create a reusable group to:

  1. Validate whether an image and text is present in the gallery item 1 which is “Madan”
  2. Validate whether there is a “Book” button in the gallery item
  3. Click on the “Book” button
  4. Do some validations in the “Checkout” screen

It would look something like this:

 

Now, what if I want to use group on Item 2 of the gallery which is “Shenji” instead of “Madan” (which was the 1st item) ?

 

Typically, we would have to create another reusable group to make it work for gallery item 2, which is time consuming and does not make sense when the whole gallery shares the same DOM structure.

When using code (e.g. Selenium), you can use the Page Object design pattern, and just pass the context in the constructor, either as a locator or a WebElement (see slides 40 and 41 here).

In codeless/low-code platforms, this wasn’t possible until now with the release of our new feature “Group Context”. Now you can maximize reuse by assigning entire groups to different elements of the page and across pages within a single click. Taking the above example, you would not have to reassign all the elements/properties of steps within the group when you want to perform same set of actions, on another item in the gallery, with the exact same DOM structure. This means, we can use the same group on gallery item 2 which is “Shenji” without having to do any rework by just choosing the Context-> Custom option from the properties panel as shown below

 

Use Cases for Group Context

Group Context can be used in many other scenarios, such as:

  • Repeating elements: when you have similar elements repeating in the page and want to execute the same steps on all of them.
  • Table rows: when you want to execute the same steps on different rows in a table.
  • Tabs or frames: when you want to reuse a group of steps recorded on one tab on other tabs in the same or different page

 

Summary

We constantly keep working hard to improve the functionality and usability of our autonomous testing platform to support your software quality initiatives. We strongly believe this feature will benefit teams and help in making automation much smarter. Group Context is currently in beta and we would love your feedback.

Please use our chat support to give us feedback or e-mail us at product@testim.io. Happy Testing!!!

We recently hosted a webinar on AI and its influence on test automation with an awesome panel consisting of Jason Arbon, Oren Rubin, Dionny Santiago and me being the moderator. There were lot of great discussions on this topic and we wanted to share it with the community as well.

Below you will find the video recording of the webinar, the list of questions and answers that we couldn’t get to during the webinar and different resources to learn about AI and testing. Please feel free to reach out to me in case of any questions at raj@testim.io or any of the panel members.

Video Recording

 

Q&A

I am a performance engineer and am working on AI for quality gates in load testing results…that needs to be a high priority for the “future” which is “now”.How do you think bots can be used in this area?
@Jason: UI-Bots can help generate user-like load directly via the application.  Though, for most load testing problems, would recommend something like charles proxy, or internal ways to spin up load, and only use the ‘expensive’ UI-based bots to see how the app works E2E for the user under load.

With rapid changes in agile requirements, how do we make the machines learn or adapt to the changes every time?
@Jason: The ai bots most folks are working on these days (vendors) will auto discover new features in the app and exercise them.  At test.ai we have a set of 5k+ tests written for common flows in apps, so if you add a new feature to your app that looks like something similar on another app, the bots will auto apply that test case to the new build.

@Raj: The more tests you run, the smarter the AI becomes in detecting changes in the application. It will start detecting changes in application’s UI, element attributes and start adapting the tests automatically to these changes due to its self-healing mechanism. It can identify flaky tests, optimize waits in between steps and also proactively fix issues for us before they occur.

With BDD model and shift left and demand for testing at unit and service/api layer where does E2E testing stand?
@Jason: Dionny’s work can help generate valid permutations of existing API test case parameters/flows.  Also, clustering methods can help identify misbehaving servers via logs of activity during api testing or production.

Where can we find Dionny’s paper on AI testing, you were talking about?
Dionny’s paper

Lot of automation test scripts fail due to test data issues, can we use AI to tackle those kind of issues?
@Jason: Thats a broad category of failure types, but yes, ‘AI’ can be taught to auto associate correct data with the right application states.  Google also shared some ‘test selection’ findings using ML to help decide what to do with all those failing tests:  https://testing.googleblog.com/2018/09/efficacy-presubmit.html

I wanted to understand, what does really mean by AI in testing?if it mean by machine will perform testing? if machine will test then if they will be already defined with scenarios which needs to be tested? is it same as automation testing as there also we don’t need manual intervention?
@Jason: Generally, AI in testing, means applying machine learning / AI techniques to test applications.  There is also ‘Testing AI’ which refers to approaches to test AI/ML-based products and features. There are a variety of ways to apply AI to testing, some leverage pre-written test cases and the AI is used to automatically execute the tests, create variations, or analyze the results.  Some AI based systems are trained to mimic general human behavior and can execute basic ‘flow testing’ for many apps, without pre-written test scenarios.  The bots we build at test.ai can read BDD/Scenarios and execute them against a set of applications.  As for need for human intervention, like automation, there is still the need for plenty of human intervention in AI-based testing approaches these days 🙂 Humans gather oracle/training data for the AI. Humans measure the correctness of the ‘AI’, and humans evaluate the significance of the AI-based test results as they relate to business/shipping decision.

@Raj: In addition to what @Jason was saying,  I wanted to mention that, AI can have a positive impact on several facets of software testing especially test automation. There have been so many different tools and frameworks that have come up trying to solve different kinds of problems related to test automation but one problem that has been a constant challenge till date, is the aspect of “maintenance”. One of the main reasons for this is the use of static locators. With static locators, we use only one attribute of an element to uniquely identify it on a page and if this changes, the test breaks and as testers we end up spending a considerable amount of time troubleshooting the problem and fixing it. Based on research, about 30% of testers’ time is consumed in just maintaining tests. Can you imagine the opportunity cost associated with this effort? It is mind blowing. Testers’ time is valuable and better spent on actually exploring the application and providing information to help stakeholders make informed decisions about the product. With AI based testing we can overcome this problem by using dynamic locators. Dynamic Locators is a concept where we use multiple attributes of an element to locate it on on the page instead of a single attribute. This way, even if one attribute changes, the element can still be successfully located with the help of other attributes that have already been extracted from the DOM by the AI.

Can you guys elaborate on how do AI-based tests learn acceptance criteria that normally has to be defined by humans?
@Jason: Depends on the AI system being used.  The bots at test.ai execute human-written test cases.  Acceptance tests are written at a very high level of abstraction, and the bots do all the test execution.  Reporting is as normal for test automation.  In summary, just tell the bots what your acceptance criteria are.

What an automation tester needs to learn to align with the future of ai in testing ?
@Jason: A good set of links to learn are here:  https://www.linkedin.com/pulse/links-ai-curious-jason-arbon/ . You can also start to leverage/experiment with “AI” via the current testing vendors. If you are already familiar with selenium/appium like testing, there is a new open source API that uses AI for element selection that you can use today:  https://medium.com/testdotai/adding-ai-to-appium-f8db38ea4fac?sk

Is AI platform dependent. like desktop application or web/mobile?
@Jason: 
Depends on the AI approach/solution.  Many are platform dependent.  The bots we build at test.ai though are not platform dependent–a key feature.  The bots are platform-independent as the machines are trained to recognize UI elements much like humans do, and humans are platform dependent ;).

Is there an Open Source project that allows to apply AI to locate the elements?
@Jason: Yes, for appium today and likely Selenium soon:  https://medium.com/testdotai/adding-ai-to-appium-f8db38ea4fac?sk

How can AI be used for improving test coverage ?
@Jason: AI can help generate many more validate test scenarios than a human could create. AI also enabled re-use of test artifacts so a test written for one app, can also execute on a similar app with no human intervention.

@Raj:  Now with AI, you can also connect your production apps to the testing cycle. This means that we can create tests based on actual flows done by the user in production. Also, the AI can observe and find repeated steps and cluster them to make reusable components in your tests. For Example – Login, Logout scenarios. So now we have scenarios that are actually created based on real production data instead of us assuming what the user will do in production. In this way, we also get good test coverage based on real data.

Will AI testing replace selenium appium and all tools and technologies?
@Jason: Asymptotically.

Is AI really better?
@Dionny: Traditional testing teams focus on either a single app, or a small set of apps; whereas, AI can learn from millions of different examples and apps. The more data we show the AI, the better it gets. Also, the AI never gets tired!

What are the immediate benefits of using AI?
@Raj: Apart from the benefits already mentioned in the answers above, AI can also help in increasing team collaboration. The field of test automation has historically been a technical tester focused community. This stigma can also change with AI. What this means is, non-technical resources no longer need to fear code and technology, rather AI will help to bridge the gap between the technical know-how and authoring and execution of tests making life easier for teams.

Will our jobs be replaced?
@Raj: Over the past decade technologies have evolved drastically, there have been so many changes happening in the technology space but one thing constant is human testers’ interaction with them and how we use them for our needs. The same holds true for AI as well. Secondly, to train the AI, we need good data combinations (which we call a training dataset). So to work with modern software we need to choose this training dataset carefully as the AI starts learning from this and starts creating relationships based on what we give to it. Also, it is important to monitor how the AI is learning as we give different training datasets. This is going to be vital to how the software is going to be tested as well. We would still need human involvement in training the AI. Finally, it is important to ensure while working with AI the security, privacy and ethical aspects of the software are not compromised. All these factors contribute to better testability of the software. We need humans for this too.

In summary, we will continue to do exploratory testing manually but will use AI to automate processes while we do this exploration. It is just like automation tools which do not replace manual testing but complement it. So, contrary to popular belief, the outlook is not all ‘doom-and-gloom;’ being a real, live human does have its advantages. For instance, human testers can improvise and test without written specifications, differentiate clarity from confusion, and sense when the ‘look and feel’ of an on-screen component is ‘off’ or wrong. Complete replacement of manual testers will only happen when AI exceeds those unique qualities of human intellect. There are a myriad of areas that will require in-depth testing to ensure safety, security, and accuracy of all the data-driven technology and apps being created on a daily basis. In this regard, utilizing AI for software testing is still in its infancy with the potential for monumental impact.

Will intelligence machines take over the world?
@Raj: Hollywood movies do have an influence on our lives don’t they 🙂 ? At most of the conferences I speak at, there is this weird notion that, in 3 years, AI powered robots are going to take over the world and we will become slaves to them. Which sounds interesting on paper but in reality I don’t think that is going to be the case.

Currently there are are some section of the people who believe fully developed AI that can react and think like humans , will be developed by 2055 and there are are other sections of people who think it will take several hundred years for that to happen. No one knows the exact answer yet. That being said, there are several organizations trying to ensure the AI currently being developed is safe for the human society. For example – The future of life institute was formed for the exact same purpose and has the brightest minds in the AI field working in that group on AI safety research. We also have groups like the World Economic forum keeping a close eye on the impact of AI on society.

So, I do not think machines will take over the world,  just yet!!! 🙂

AI Resources

Courses

And there are more courses available online. Just google search for “Deep Learning courses”, “Machine Learning courses” as keywords.

 

Free Resources/Courses

 

Books

Introduction

We work hard to improve the functionality and usability of our autonomous testing platform to support your software quality initiatives. This month we’re thrilled to release a few of your most requested features; Shared Group Indicator, Numbered Test Steps, New Base URL Parameter. Check them out and let us know what you think.

Shared Group Indicator

What is it?

When trying to change a Shared step the users will now get a notification that they are editing a shared step. Further clicking on “See affected tests” takes the user to the list of tests that are using the shared step.

 

 

Why should I care?

You no longer have to worry about someone changing a shared step unknowingly, as you now see the shared group indicator letting you know the effects of a change before it is done. This is useful when teams are collaborating to build test suites and when multiple people are working on the same set of tests. Now individuals have more visibility to how their changes might impact overall testing.

 

Base URL as a Parameter

What is it?

Users now have the ability to access the base url through a variable within your custom actions. The new variable that automatically stores the url value is named BASE_URLLearn More

Why should I care?

You no longer have to add extra code to get the url value of the web page used in the test. Instead, you just use the BASE_URL parameter and perform any actions necessary inside our custom actions. For example – If we want to print out the url of the web page to ensure the same page is still displayed after certain number of validations, you could just say

console.log(“The current base url is” + BASE_URL)

 

Numbered Test Steps

What is it?

Step numbers help to uniquely identify each step in a test. You now have the step number displayed next to the name of each and every step that is added to your test.

Why should I care?

Having numbered steps help to easily refer to a particular step in a test. This is helpful in cases where you want

  • To edit a particular step
  • To collaboratively work on a particular step of a test with other team members
  • To talk to our support team to debug a particular step in a test

Introduction

We work hard to improve the functionality and usability of our autonomous testing platform to support your software quality initiatives. This month we’re thrilled to release a few of your most requested features; Result Labels, Test Run Navigation Icon, Grid Management. Check them out and let us know what you think.

Result Labels

What is it?

The “Result Labels” allows you to name each remote run. On the “Suite Runs” and “Test runs” pages, you can easily filter your runs by choosing a result label.

Testim Result Labels

Why should I care?  

You now have the ability to label your runs. This is especially useful when you need to drill down into specific runs based on environment, application version, sprint numbers etc. For example you can label your runs as “nightly-scheduler”, “v1.42.34”, “Jenkins”, “Troubleshooting”, “Staging”.

Result labels can be added to the CLI using the parameter –result-label “<user defined name of the run>”. Learn more

Test Run Navigation Icon

What is it?

The new navigation icon opens the results of a test in a new tab.

Testim test run navigation

Why should I care?

You now have  the ability to switch back and forth between test and the test runs via the tabs.

Grid Management

What is it?

To run your tests remotely, you need to integrate either with Testim grid, your own local grid or other 3rd party grids like Sauce Labs and Browserstack. Learn more

testim grid management

Why should I care?  

Grid management now offers the ability to easily manage multiple grids providing an abstraction layer for your devops. The grid information is automatically added to the CLI based on the already configured grids and will appear in this format –grid “<grid name>”.

Customers have access to these new features now. Check it out and let us know what you think. If you’re not a customer, sign up for a free trial to experience autonomous testing. We’d love to hear what you think of the new features. Please share your thoughts on Twitter, LinkedIn or Facebook.

Introduction

We work hard to improve the functionality and usability of our autonomous testing platform to support your software quality initiatives. This month we’re thrilled to release a few of your most requested features; Hidden Parameters, Data Driven testing  via config files and Element Text condition. Check them out and let us know what you think.

Hidden Parameters

What is it?

When you use parameters in your tests, the values that are passed in during run time are saved and shown in the UI. Sometimes this information is sensitive and you may want the value to be hidden. This is now possible using the hidden parameters option available in the project settings page of the Testim editor. Learn more

Why should I care?  

You no longer have to worry about revealing sensitive information in your tests. This is especially true if your application is related to banking, security, insurance or any other domain that handles a large amount of sensitive data.

Data Driven testing now supports CSV, database and other external sources

What is it?

Now users have the ability to pass data sets at run time via config files. The newly added “overrideTestData” parameter in the beforeSuite hook will allow users to pass in multiple parameters to multiple tests at the same time. The same parameter can also be used to extract data from external sources such as CSV, Databases etc.

Why should I care?  

Data Driven testing is no longer just restricted to passing a json file within the tests. Now, you have the flexibility to pass this data at run time through a single config file. Also, you can extract data from external sources and use it within your tests. Everything happens automatically for you during run time. This makes test data setup much more extensible and reusable. Learn more

Talking about working with excel;  we already have detailed documentation of an alternate way to import excel data into Testim. You can learn more about here.

Element Text condition

What is it?

Testim provides several predefined conditions (“if statements”) to be used with steps.  For example, whether an element is visible or not. We just introduced a new condition which checks whether an element has a specific text. Just pass in a string, regex, or a js statement (you can use variables too!).

Why should I care?
Now you have the flexibility to add conditions based on element text instead of just checking for element being visible on the screen. Learn more

 

Customers have access to these new features now. Check it out and let us know what you think. If you’re not a customer, sign up for a free trial to experience autonomous testing. We’d love to hear what you think of the new features. Please share your thoughts on Twitter, LinkedIn or Facebook.

 

One of the most important factors related to automated tests is Maintenance. A lot of effort is spent on maintaining the tests than writing actual tests.  A recent study suggested about 30% of testers time is spent on maintenance.This leads to wastage of valuable time and effort by the resources, which they could have rather spent on testing the actual application.

Imagine a world where the software can maintain tests without human interaction? This world has become a reality with Testim.io. We use Artificial Intelligence (AI) underneath the hood, which provides self-healing maintenance i.e problems are detected by the AI and automatically fixed without human intervention.

Testim.io also help to speed up the maintenance of tests by providing the follow features within our platform-

  1. Version Control

At any given time, it is important to have logs of what changes were made to a particular test. This way we can always revert back to an older version of test as and when required. Our platform provides this functionality by showing all the version history by going to the Properties panel of the setup step and clicking on “See old revisions”

  1. Branching

At Testim.io, we firmly believe in the “Shift Left Paradigm” where Development and Testing must start in parallel as early as possible in the software development lifecycle. Keeping this in mind, we provide the functionality to teams to create separate branches for each team member and work on the same projects and tests. This way, no one can overwrite the changes of the other team members and teams can work on the same code base at any instant of time

In our platform, we just need to select “Fork” to create a new branch and we can also switch between existing branches


        3.  Scheduler

Users have the option of scheduling their tests. This helps to run the tests automatically at a certain day and time without any manual intervention. We can also get notified via email in case of any errors

 

Troubleshooting

As testers, we spend considerable amount of time troubleshooting issues. To help in troubleshooting, our platform offers different options to the user to narrow down the scope of the problem. These options  are as follows-

  1. Screenshots

The screenshot feature explained in the “Authoring and Execution” section helps users to know what was the baseline image and what was the actual image found.

  1.   Properties Panel

The properties panel helps to capture the error messages and display it to the user. The user also has the option of interacting with DOM and see what objects were extracted during the run

  1. Test Logs

Logs are a rich source of information on what happened underneath the UI. We provide test logs when the user runs the tests on our grid or a 3rd party grid. The option can be found in the in top section of editor

  1. Bug Reporting

One of the most time consuming aspects of testing is after finding a bug, we need to report it to the developer with relevant information, to speed up the troubleshooting and fixing of issues.

With Testim.io you can do this with a single click with the help of our chrome extension. All the details related to the bug are automatically generated for you.

  1. Documentation

We put in a lot of effort to document most of the features of the tool in our User Documentation found under the “Educate” tab.

We also have detailed videos on how to troubleshoot your tests quickly

Troubleshooting Part 1- Element is not visible

Troubleshooting Part 2 – Element not found

Troubleshooting Part 3 – Timing issues

Troubleshooting Part 4 – Issues related to mouse hover

With the above features, Testim.io helps to create stable tests that are highly maintainable.

The below posts gives more in depth analysis of Testim in terms of different features that make authoring and execution of tests really simple and how to create reusable components that improves the extensibility of the tool

https://blog.testim.io/bringing-simplicity-to-authoring-and-execution-of-automated-tests/

https://blog.testim.io/how-to-make-reusable-and-extensible-code-using-testim-io/

 

Artificial Intelligence (AI) and machine learning (ML) are advancing at a rapid pace. Companies like Apple, Tesla, Google, Amazon, Facebook and others have started investing more into AI to solve different technological problems in the areas of healthcare, autonomous cars, search engines, predictive modeling and much more. Applying AI is real. It’s coming fast. It’s going to affect every business, no matter how big or small. This being the case how are we as Testers going to adapt to this change and embrace AI? Here is the summary of different things you need to know about using AI in software testing.

Let’s summarize how the testing practice has evolved over the last 4 decades

  • In the 1980’s majority of software development was waterfall and testing was manual
  • In the 1990’s, we had bulky automation tools which were super expensive, unstable and had really primitive functionality. During the same time, there were different development approaches being experimented like Scrum, XP, RAD (Rapid Application Development)
  • From 2000, the era of open source frameworks began
    • People wanted to share their knowledge with the community
    • Started encouraging innovation and asking community of like minded people to help out in improving testing
    • Agile became a big thing, XP, Scrum, Kanban became a standard process in the SDLC
    • There were need for faster release cycles as people wanted more software features delivered faster
  • In the 2010’s, it was all about scale, how to write tests fast and find bugs faster
    • Crowdtesting started
      • Encouraging other people to give feedback on the application. Free and Paid services
    • Cloud testing started
      • People started realizing they need more
        • Server space
        • Faster processing
      • Started to realize the problem of maintenance. How expensive it is to buy hardware and software for maintaining your tests
      • Then we have
        • DevOps
        • Continuous Testing
        • CI/CD integration
  • I believe the Future will be about Autonomous Testing using Machine Learning and AI

 

Basics of AI

Let’s start by de-mystifying some of the terminologies related to AI

  • Artificial Intelligence (AI) is an area of computer science that emphasizes the creation of intelligent machines that work and react like humans
  • Machine Learning (ML) evolved from the study of pattern recognition and computational learning theory (studying design and analysis of ML algorithms) in AI. It is a field of study that gives computers ability to learn without being explicitly programmed
  • Deep Learning(DL) is one of the many approaches to ML. Other approaches include decision tree learning, inductive logic programming, clustering and Bayesian networks. It is based on neural networks in the human body. Each neuron keeps learning and interconnects with other neurons to perform different actions based on different responses

 

There are 3 types of widely used ML algorithms

  • Supervised Learning – We are giving the right training data (input/output combinations) for the algorithm to learn
    • Examples
      • Give bunch of e-mails and identify spam e-mails
      • Extracting text from audio
      • Fill out a loan application and find the probability of the user repaying the loan
      • How to make user click on ads by learning their behavior
      • Recommendation engines on Amazon, Netflix where customer is recommended products and movies
      • Amazon uses AI for logistics
      • Car Optimization
      • Autonomous cars
  • Un-supervised learning – We give a bunch of data and see what we can find
    • Examples
      • Taking a single image and creating a 3D model
      • Market Segmentation
  • Reinforced learning – Based on concept of reward function. Rewarding Good/Bad behavior and making the algorithm learn from it. E.g. Training a Dog

 

Real life AI applications to visually see how it works

  • Quick Draw from Google
  • Weka is an open source project where they are using ML algorithms for data mining

 

What challenges can AI solve?

Let’s discuss the challenges the industry faced while transitioning to agile and what’s still remains a challenge:

How can we use AI to solve testing problems?

There are many companies taking multiple approaches to solve different problems related to software testing and automation. Testim.io is one such company

Testim.io uses Dynamic Locators, The Artificial Intelligence (AI) underneath the platform in real time, analyzes all the DOM objects of a page and extracts the objects and its properties. Finally, the AI decides the best location strategy to locate a particular element based on this analysis. Due to this, even if a developer changes the attribute of an element, the test still continues to run and this leads to more stable tests. As a result, the authoring and execution of automated tests are much faster and more stable.

 

Here is the detailed insight of how our AI works – https://www.softwaretestinghelp.com/testim-io-tool-tutorial/

One of the good practices of writing automated tests is creating reusable components that can be used in different parts of our test suite.

Why is this important?

Creating reusable components is important because it

  • Helps to increase the readability of the automated tests
  • Saves effort by not repeating the same set of steps in different parts of the tests
  • Any changes to the reusable step needs to be done only in one place and it is reflected throughout the tests, across different projects
  • Makes the automated tests more extensible

 

Testim.io helps to ensure Reusability by “Grouping” and “Parameterization”.

  • Grouping

Any number of related steps can be grouped into one reusable component.

For Example – The “Login” scenario is one of the most commonly used steps in any application. The way we can create a reusable “Login” step would be to select the steps we want to group together and clicking on “Add new Group” as shown below

  1. Parameterization

Our platform gives the option of testing application through various input combinations via Parameterization.

This can be achieved in various ways. One way to do this is to give all the input parameters we would need to test the application in the form of a JSON file in the Setup step (The first step of our tests) as shown below

Then add the variable names used in the json file in the appropriate fields of the step as show below

 

Another important aspect of automation is building your tests such that it is extensible.

Why is this important?

As the product and teams grow, there will be need to test more complex functionalities which would require building upon already existing tests. This being the case, the automation suites need to be simple, understandable and should be easy to add more tests to already existing test suites with low coupling and high cohesion.

Testim.io gives the flexibility for organizations to extend the functionalities of our platform using JavaScript and HTML. This way, any functionality our platform does not handle; the user can write their own code to build a robust automation framework

For Example – Say we want to validate the “Select Destination” button from our previous examples. The way to do this would be.

  • Click on “Add custom action”
  • Give a name to the New Step and click on “Confirm”
  • Click on “PARAMS” and Select “HTML” for this example
  • Add Custom Code

The new step with Custom Code gets added to the list of already existing steps

The above features help to make the automation suite more reusable and extensible.

The below posts gives more in depth analysis of Testim in terms of different features that make authoring and execution of tests really simple and easy to maintain

https://blog.testim.io/bringing-simplicity-to-authoring-and-execution-of-automated-tests/

https://blog.testim.io/maintenance-of-tests-made-easy-with-testim-io/

Be smart & save time...
Simply automate.

Get Started Free