Tag

artificial intelligence

Browsing

One of the most important factors related to automated tests is Maintenance. A lot of effort is spent on maintaining the tests than writing actual tests.  A recent study suggested about 30% of testers time is spent on maintenance.This leads to wastage of valuable time and effort by the resources, which they could have rather spent on testing the actual application.

Imagine a world where the software can maintain tests without human interaction? This world has become a reality with Testim.io. We use Artificial Intelligence (AI) underneath the hood, which provides self-healing maintenance i.e problems are detected by the AI and automatically fixed without human intervention.

Testim.io also help to speed up the maintenance of tests by providing the follow features within our platform-

  1. Version Control

At any given time, it is important to have logs of what changes were made to a particular test. This way we can always revert back to an older version of test as and when required. Our platform provides this functionality by showing all the version history by going to the Properties panel of the setup step and clicking on “See old revisions”

  1. Branching

At Testim.io, we firmly believe in the “Shift Left Paradigm” where Development and Testing must start in parallel as early as possible in the software development lifecycle. Keeping this in mind, we provide the functionality to teams to create separate branches for each team member and work on the same projects and tests. This way, no one can overwrite the changes of the other team members and teams can work on the same code base at any instant of time

In our platform, we just need to select “Fork” to create a new branch and we can also switch between existing branches


        3.  Scheduler

Users have the option of scheduling their tests. This helps to run the tests automatically at a certain day and time without any manual intervention. We can also get notified via email in case of any errors

 

Troubleshooting

As testers, we spend considerable amount of time troubleshooting issues. To help in troubleshooting, our platform offers different options to the user to narrow down the scope of the problem. These options  are as follows-

  1. Screenshots

The screenshot feature explained in the “Authoring and Execution” section helps users to know what was the baseline image and what was the actual image found.

  1.   Properties Panel

The properties panel helps to capture the error messages and display it to the user. The user also has the option of interacting with DOM and see what objects were extracted during the run

  1. Test Logs

Logs are a rich source of information on what happened underneath the UI. We provide test logs when the user runs the tests on our grid or a 3rd party grid. The option can be found in the in top section of editor

  1. Bug Reporting

One of the most time consuming aspects of testing is after finding a bug, we need to report it to the developer with relevant information, to speed up the troubleshooting and fixing of issues.

With Testim.io you can do this with a single click with the help of our chrome extension. All the details related to the bug are automatically generated for you.

  1. Documentation

We put in a lot of effort to document most of the features of the tool in our User Documentation found under the “Educate” tab.

We also have detailed videos on how to troubleshoot your tests quickly

Troubleshooting Part 1- Element is not visible

Troubleshooting Part 2 – Element not found

Troubleshooting Part 3 – Timing issues

Troubleshooting Part 4 – Issues related to mouse hover

With the above features, Testim.io helps to create stable tests that are highly maintainable.

Artificial Intelligence (AI) and machine learning (ML) are advancing at a rapid pace. Companies like Apple, Tesla, Google, Amazon, Facebook and others have started investing more into AI to solve different technological problems in the areas of healthcare, autonomous cars, search engines, predictive modeling and much more. Applying AI is real. It’s coming fast. It’s going to affect every business, no matter how big or small. This being the case how are we as Testers going to adapt to this change and embrace AI? Here is the summary of different things you need to know about using AI in software testing.

Let’s summarize how the testing practice has evolved over the last 4 decades

  • In the 1980’s majority of software development was waterfall and testing was manual
  • In the 1990’s, we had bulky automation tools which were super expensive, unstable and had really primitive functionality. During the same time, there were different development approaches being experimented like Scrum, XP, RAD (Rapid Application Development)
  • From 2000, the era of open source frameworks began
    • People wanted to share their knowledge with the community
    • Started encouraging innovation and asking community of like minded people to help out in improving testing
    • Agile became a big thing, XP, Scrum, Kanban became a standard process in the SDLC
    • There were need for faster release cycles as people wanted more software features delivered faster
  • In the 2010’s, it was all about scale, how to write tests fast and find bugs faster
    • Crowdtesting started
      • Encouraging other people to give feedback on the application. Free and Paid services
    • Cloud testing started
      • People started realizing they need more
        • Server space
        • Faster processing
      • Started to realize the problem of maintenance. How expensive it is to buy hardware and software for maintaining your tests
      • Then we have
        • DevOps
        • Continuous Testing
        • CI/CD integration
  • I believe the Future will be about Autonomous Testing using Machine Learning and AI

 

Basics of AI

Let’s start by de-mystifying some of the terminologies related to AI

  • Artificial Intelligence (AI) is an area of computer science that emphasizes the creation of intelligent machines that work and react like humans
  • Machine Learning (ML) evolved from the study of pattern recognition and computational learning theory (studying design and analysis of ML algorithms) in AI. It is a field of study that gives computers ability to learn without being explicitly programmed
  • Deep Learning(DL) is one of the many approaches to ML. Other approaches include decision tree learning, inductive logic programming, clustering and Bayesian networks. It is based on neural networks in the human body. Each neuron keeps learning and interconnects with other neurons to perform different actions based on different responses

 

There are 3 types of widely used ML algorithms

  • Supervised Learning – We are giving the right training data (input/output combinations) for the algorithm to learn
    • Examples
      • Give bunch of e-mails and identify spam e-mails
      • Extracting text from audio
      • Fill out a loan application and find the probability of the user repaying the loan
      • How to make user click on ads by learning their behavior
      • Recommendation engines on Amazon, Netflix where customer is recommended products and movies
      • Amazon uses AI for logistics
      • Car Optimization
      • Autonomous cars
  • Un-supervised learning – We give a bunch of data and see what we can find
    • Examples
      • Taking a single image and creating a 3D model
      • Market Segmentation
  • Reinforced learning – Based on concept of reward function. Rewarding Good/Bad behavior and making the algorithm learn from it. E.g. Training a Dog

 

Real life AI applications to visually see how it works

  • Quick Draw from Google
  • Weka is an open source project where they are using ML algorithms for data mining

 

What challenges can AI solve?

Let’s discuss the challenges the industry faced while transitioning to agile and what’s still remains a challenge:

How can we use AI to solve testing problems?

There are many companies taking multiple approaches to solve different problems related to software testing and automation. Testim.io is one such company

Testim.io uses Dynamic Locators, The Artificial Intelligence (AI) underneath the platform in real time, analyzes all the DOM objects of a page and extracts the objects and its properties. Finally, the AI decides the best location strategy to locate a particular element based on this analysis. Due to this, even if a developer changes the attribute of an element, the test still continues to run and this leads to more stable tests. As a result, the authoring and execution of automated tests are much faster and more stable.

 

Here is the detailed insight of how our AI works – https://www.softwaretestinghelp.com/testim-io-tool-tutorial/

One of the good practices of writing automated tests is creating reusable components that can be used in different parts of our test suite.

Why is this important?

Creating reusable components is important because it

  • Helps to increase the readability of the automated tests
  • Saves effort by not repeating the same set of steps in different parts of the tests
  • Any changes to the reusable step needs to be done only in one place and it is reflected throughout the tests, across different projects
  • Makes the automated tests more extensible

 

Testim.io helps to ensure Reusability by “Grouping” and “Parameterization”.

  • Grouping

Any number of related steps can be grouped into one reusable component.

For Example – The “Login” scenario is one of the most commonly used steps in any application. The way we can create a reusable “Login” step would be to select the steps we want to group together and clicking on “Add new Group” as shown below

  1. Parameterization

Our platform gives the option of testing application through various input combinations via Parameterization.

This can be achieved in various ways. One way to do this is to give all the input parameters we would need to test the application in the form of a JSON file in the Setup step (The first step of our tests) as shown below

Then add the variable names used in the json file in the appropriate fields of the step as show below

 

Another important aspect of automation is building your tests such that it is extensible.

Why is this important?

As the product and teams grow, there will be need to test more complex functionalities which would require building upon already existing tests. This being the case, the automation suites need to be simple, understandable and should be easy to add more tests to already existing test suites with low coupling and high cohesion.

Testim.io gives the flexibility for organizations to extend the functionalities of our platform using JavaScript and HTML. This way, any functionality our platform does not handle; the user can write their own code to build a robust automation framework

For Example – Say we want to validate the “Select Destination” button from our previous examples. The way to do this would be.

  • Click on “Add custom action”
  • Give a name to the New Step and click on “Confirm”
  • Click on “PARAMS” and Select “HTML” for this example
  • Add Custom Code

The new step with Custom Code gets added to the list of already existing steps

The above features help to make the automation suite more reusable and extensible.

Authoring and Execution of tests is an important aspect of test automation. Tests should be simple to write, understand and execute across projects. The automation framework or tool chosen should give the flexibility to record and playback tests as well as, write custom code to extend the functionalities of the automation framework.

This is where Testim.io can help you out. We follow a  Hybrid Approach and make authoring and execution of tests really simple in such a way that both technical and non-technical members can collaborate and start writing automated tests quickly. This is achieved with the use of “Dynamic Locators”.

What are Dynamic Locators?

The Artificial Intelligence (AI) underneath the platform in real time, analyzes all the DOM objects of a page and extracts the objects and its properties. Finally, the AI decides the best location strategy to locate a particular element based on this analysis. Due to this, even if a developer changes the attribute of an element, the test still continues to run and this leads to more stable tests. As a result, the authoring and execution of automated tests are much faster and more stable.

As we can notice from the above image, the AI parses through all the DOM objects, lists them in the Properties Panel along with the rankings of each and every location strategy for that particular element. In this way, even if the attribute of an element changes, the AI can use a different location strategy from the already parsed list of DOM objects.

Thus, the user does not have to worry about flaky tests.

Some of the basic authoring and execution features Testim.io provides to its customers, are explained below.

  1. How to create a test

We create a new Test by clicking on “Create New” or “New Test”

 

  1. Recording and Saving a test

Once we click the “Record” button, we can record different user actions in our application. After recording different actions, click on “Stop Recording” button to finish recording our tests. Use the “Save” button to save the tests.

 

  1. Validations and Assertions

Our platform helps to make validation of different attributes of an element and API’s really simple. We provide various options for users such as

  • Adding Custom Validations using JavaScript and HTML
  • Validate element visibility
  • Validate element text
  • Pixel level validation
  • API level validation

 

  1. Screenshots

While each test is recorded, the platform takes screenshots of all the Pass and Failed results of each and every step. As a result, users find it easier to troubleshoot problems and understand what happens underneath the hood.

 

  1. Feedback on each step

The user also gets feedback on each step in terms of whether the tests Passed or Failed by showing a “Green” or “Red icon” on the top left portion of each step as shown below

 

  1. Labeling tests

Testim.io provides the feature to label each and every test a user creates. There are 2 reasons why we may want to label a test

  • Helps to identify the reason the test was created in the first place
  • Helps to run tests with the same label all at once through our CLI feature

The way we create labels is by clicking on the “Label” button and either select an existing a label or create a new one.

 

  1. User Documentation

At Testim.io, we took the effort to provide users with all the documentation they will need to use different features of our platform. Most of the answers about using our platform can be found by clicking on the “Educate” tab and Visiting our documentation site as shown below

With the above features, Testim.io helps to make the authoring and execution of tests really fast and simple for our users. Within a matter of seconds a user can record, replay and save the tests. This is surprisingly one of the most overlooked aspects of test automation and our platform takes care of it for our users.

 

Be smart & save time...
Simply automate.

Get Started Free