Tag

agile testing

Browsing

One of the most important factors related to automated tests is Maintenance. A lot of effort is spent on maintaining the tests than writing actual tests.  A recent study suggested about 30% of testers time is spent on maintenance.This leads to wastage of valuable time and effort by the resources, which they could have rather spent on testing the actual application.

Imagine a world where the software can maintain tests without human interaction? This world has become a reality with Testim.io. We use Artificial Intelligence (AI) underneath the hood, which provides self-healing maintenance i.e problems are detected by the AI and automatically fixed without human intervention.

Testim.io also help to speed up the maintenance of tests by providing the follow features within our platform-

  1. Version Control

At any given time, it is important to have logs of what changes were made to a particular test. This way we can always revert back to an older version of test as and when required. Our platform provides this functionality by showing all the version history by going to the Properties panel of the setup step and clicking on “See old revisions”

  1. Branching

At Testim.io, we firmly believe in the “Shift Left Paradigm” where Development and Testing must start in parallel as early as possible in the software development lifecycle. Keeping this in mind, we provide the functionality to teams to create separate branches for each team member and work on the same projects and tests. This way, no one can overwrite the changes of the other team members and teams can work on the same code base at any instant of time

In our platform, we just need to select “Fork” to create a new branch and we can also switch between existing branches


        3.  Scheduler

Users have the option of scheduling their tests. This helps to run the tests automatically at a certain day and time without any manual intervention. We can also get notified via email in case of any errors

 

Troubleshooting

As testers, we spend considerable amount of time troubleshooting issues. To help in troubleshooting, our platform offers different options to the user to narrow down the scope of the problem. These options  are as follows-

  1. Screenshots

The screenshot feature explained in the “Authoring and Execution” section helps users to know what was the baseline image and what was the actual image found.

  1.   Properties Panel

The properties panel helps to capture the error messages and display it to the user. The user also has the option of interacting with DOM and see what objects were extracted during the run

  1. Test Logs

Logs are a rich source of information on what happened underneath the UI. We provide test logs when the user runs the tests on our grid or a 3rd party grid. The option can be found in the in top section of editor

  1. Bug Reporting

One of the most time consuming aspects of testing is after finding a bug, we need to report it to the developer with relevant information, to speed up the troubleshooting and fixing of issues.

With Testim.io you can do this with a single click with the help of our chrome extension. All the details related to the bug are automatically generated for you.

  1. Documentation

We put in a lot of effort to document most of the features of the tool in our User Documentation found under the “Educate” tab.

We also have detailed videos on how to troubleshoot your tests quickly

Troubleshooting Part 1- Element is not visible

Troubleshooting Part 2 – Element not found

Troubleshooting Part 3 – Timing issues

Troubleshooting Part 4 – Issues related to mouse hover

With the above features, Testim.io helps to create stable tests that are highly maintainable.

Artificial Intelligence (AI) and machine learning (ML) are advancing at a rapid pace. Companies like Apple, Tesla, Google, Amazon, Facebook and others have started investing more into AI to solve different technological problems in the areas of healthcare, autonomous cars, search engines, predictive modeling and much more. Applying AI is real. It’s coming fast. It’s going to affect every business, no matter how big or small. This being the case how are we as Testers going to adapt to this change and embrace AI? Here is the summary of different things you need to know about using AI in software testing.

Let’s summarize how the testing practice has evolved over the last 4 decades

  • In the 1980’s majority of software development was waterfall and testing was manual
  • In the 1990’s, we had bulky automation tools which were super expensive, unstable and had really primitive functionality. During the same time, there were different development approaches being experimented like Scrum, XP, RAD (Rapid Application Development)
  • From 2000, the era of open source frameworks began
    • People wanted to share their knowledge with the community
    • Started encouraging innovation and asking community of like minded people to help out in improving testing
    • Agile became a big thing, XP, Scrum, Kanban became a standard process in the SDLC
    • There were need for faster release cycles as people wanted more software features delivered faster
  • In the 2010’s, it was all about scale, how to write tests fast and find bugs faster
    • Crowdtesting started
      • Encouraging other people to give feedback on the application. Free and Paid services
    • Cloud testing started
      • People started realizing they need more
        • Server space
        • Faster processing
      • Started to realize the problem of maintenance. How expensive it is to buy hardware and software for maintaining your tests
      • Then we have
        • DevOps
        • Continuous Testing
        • CI/CD integration
  • I believe the Future will be about Autonomous Testing using Machine Learning and AI

 

Basics of AI

Let’s start by de-mystifying some of the terminologies related to AI

  • Artificial Intelligence (AI) is an area of computer science that emphasizes the creation of intelligent machines that work and react like humans
  • Machine Learning (ML) evolved from the study of pattern recognition and computational learning theory (studying design and analysis of ML algorithms) in AI. It is a field of study that gives computers ability to learn without being explicitly programmed
  • Deep Learning(DL) is one of the many approaches to ML. Other approaches include decision tree learning, inductive logic programming, clustering and Bayesian networks. It is based on neural networks in the human body. Each neuron keeps learning and interconnects with other neurons to perform different actions based on different responses

 

There are 3 types of widely used ML algorithms

  • Supervised Learning – We are giving the right training data (input/output combinations) for the algorithm to learn
    • Examples
      • Give bunch of e-mails and identify spam e-mails
      • Extracting text from audio
      • Fill out a loan application and find the probability of the user repaying the loan
      • How to make user click on ads by learning their behavior
      • Recommendation engines on Amazon, Netflix where customer is recommended products and movies
      • Amazon uses AI for logistics
      • Car Optimization
      • Autonomous cars
  • Un-supervised learning – We give a bunch of data and see what we can find
    • Examples
      • Taking a single image and creating a 3D model
      • Market Segmentation
  • Reinforced learning – Based on concept of reward function. Rewarding Good/Bad behavior and making the algorithm learn from it. E.g. Training a Dog

 

Real life AI applications to visually see how it works

  • Quick Draw from Google
  • Weka is an open source project where they are using ML algorithms for data mining

 

What challenges can AI solve?

Let’s discuss the challenges the industry faced while transitioning to agile and what’s still remains a challenge:

How can we use AI to solve testing problems?

There are many companies taking multiple approaches to solve different problems related to software testing and automation. Testim.io is one such company

Testim.io uses Dynamic Locators, The Artificial Intelligence (AI) underneath the platform in real time, analyzes all the DOM objects of a page and extracts the objects and its properties. Finally, the AI decides the best location strategy to locate a particular element based on this analysis. Due to this, even if a developer changes the attribute of an element, the test still continues to run and this leads to more stable tests. As a result, the authoring and execution of automated tests are much faster and more stable.

 

Here is the detailed insight of how our AI works – https://www.softwaretestinghelp.com/testim-io-tool-tutorial/

One of the good practices of writing automated tests is creating reusable components that can be used in different parts of our test suite.

Why is this important?

Creating reusable components is important because it

  • Helps to increase the readability of the automated tests
  • Saves effort by not repeating the same set of steps in different parts of the tests
  • Any changes to the reusable step needs to be done only in one place and it is reflected throughout the tests, across different projects
  • Makes the automated tests more extensible

 

Testim.io helps to ensure Reusability by “Grouping” and “Parameterization”.

  • Grouping

Any number of related steps can be grouped into one reusable component.

For Example – The “Login” scenario is one of the most commonly used steps in any application. The way we can create a reusable “Login” step would be to select the steps we want to group together and clicking on “Add new Group” as shown below

  1. Parameterization

Our platform gives the option of testing application through various input combinations via Parameterization.

This can be achieved in various ways. One way to do this is to give all the input parameters we would need to test the application in the form of a JSON file in the Setup step (The first step of our tests) as shown below

Then add the variable names used in the json file in the appropriate fields of the step as show below

 

Another important aspect of automation is building your tests such that it is extensible.

Why is this important?

As the product and teams grow, there will be need to test more complex functionalities which would require building upon already existing tests. This being the case, the automation suites need to be simple, understandable and should be easy to add more tests to already existing test suites with low coupling and high cohesion.

Testim.io gives the flexibility for organizations to extend the functionalities of our platform using JavaScript and HTML. This way, any functionality our platform does not handle; the user can write their own code to build a robust automation framework

For Example – Say we want to validate the “Select Destination” button from our previous examples. The way to do this would be.

  • Click on “Add custom action”
  • Give a name to the New Step and click on “Confirm”
  • Click on “PARAMS” and Select “HTML” for this example
  • Add Custom Code

The new step with Custom Code gets added to the list of already existing steps

The above features help to make the automation suite more reusable and extensible.

Authoring and Execution of tests is an important aspect of test automation. Tests should be simple to write, understand and execute across projects. The automation framework or tool chosen should give the flexibility to record and playback tests as well as, write custom code to extend the functionalities of the automation framework.

This is where Testim.io can help you out. We follow a  Hybrid Approach and make authoring and execution of tests really simple in such a way that both technical and non-technical members can collaborate and start writing automated tests quickly. This is achieved with the use of “Dynamic Locators”.

What are Dynamic Locators?

The Artificial Intelligence (AI) underneath the platform in real time, analyzes all the DOM objects of a page and extracts the objects and its properties. Finally, the AI decides the best location strategy to locate a particular element based on this analysis. Due to this, even if a developer changes the attribute of an element, the test still continues to run and this leads to more stable tests. As a result, the authoring and execution of automated tests are much faster and more stable.

As we can notice from the above image, the AI parses through all the DOM objects, lists them in the Properties Panel along with the rankings of each and every location strategy for that particular element. In this way, even if the attribute of an element changes, the AI can use a different location strategy from the already parsed list of DOM objects.

Thus, the user does not have to worry about flaky tests.

Some of the basic authoring and execution features Testim.io provides to its customers, are explained below.

  1. How to create a test

We create a new Test by clicking on “Create New” or “New Test”

 

  1. Recording and Saving a test

Once we click the “Record” button, we can record different user actions in our application. After recording different actions, click on “Stop Recording” button to finish recording our tests. Use the “Save” button to save the tests.

 

  1. Validations and Assertions

Our platform helps to make validation of different attributes of an element and API’s really simple. We provide various options for users such as

  • Adding Custom Validations using JavaScript and HTML
  • Validate element visibility
  • Validate element text
  • Pixel level validation
  • API level validation

 

  1. Screenshots

While each test is recorded, the platform takes screenshots of all the Pass and Failed results of each and every step. As a result, users find it easier to troubleshoot problems and understand what happens underneath the hood.

 

  1. Feedback on each step

The user also gets feedback on each step in terms of whether the tests Passed or Failed by showing a “Green” or “Red icon” on the top left portion of each step as shown below

 

  1. Labeling tests

Testim.io provides the feature to label each and every test a user creates. There are 2 reasons why we may want to label a test

  • Helps to identify the reason the test was created in the first place
  • Helps to run tests with the same label all at once through our CLI feature

The way we create labels is by clicking on the “Label” button and either select an existing a label or create a new one.

 

  1. User Documentation

At Testim.io, we took the effort to provide users with all the documentation they will need to use different features of our platform. Most of the answers about using our platform can be found by clicking on the “Educate” tab and Visiting our documentation site as shown below

With the above features, Testim.io helps to make the authoring and execution of tests really fast and simple for our users. Within a matter of seconds a user can record, replay and save the tests. This is surprisingly one of the most overlooked aspects of test automation and our platform takes care of it for our users.

 

Growing up as Testers in the software testing industry, we often go through a lot of thoughts and have several questions in mind such as:

  • How do I learn about software testing?
  • I like my job but how do I get to the next level?
  • Am I good at my job?
  • There are so many testing jargons people are using and I do not understand any of them
  • You are trying to communicate effectively but people still do not understand you

Based on testing software for a over a decade now, reading articles and blogs, interacting with practitioners from all over the world and analyzing my success and failures as a tester in the past several years; I discovered everything comes down to 3 Key Factors that paves the path to becoming a strong tester. These factors form the Strong Tester Model Stability as shown below.

Strong TestersFactor 1Motivation

  • “Run Your Own Race” – As Testers, we constantly keep comparing ourselves with other people, try to do more without paying attention to what our goals are and finally end up getting stressed as a result of being overworked and concentrating on lesser priority things. In life and in testing, we need to remember that, the ONLY person we are competing with; is OURSELVES. We need to identify our strengths and answer to our conscience, rather than comparing ourselves with others who totally have different set of goals.
  • Embrace Your Talents – Recognize your strengths and weaknesses. Embrace them and work to your skill sets with well defined goals and deadlines. Hold yourself accountable.
  • Go Explore – We will find our true passion only when we explore different things and take chances. Everyone starts from somewhere and no one is an overnight success. So start your journey and exploration. Try anything at least once with full dedication and determination. Remember “If you take a chance in life sometimes good things happen and sometimes bad things happen. But if you don’t take a chance, nothing happens.”
  • Tips and tricks for sustained and continuous motivation:
    • Have inspirational quotes to get you going when you are down or feel lost. Everyone has a trigger point, what is yours?
    • Have a Testing Notebook to note down all things testing when you read, explore and talk to people. Looking at your notes will spark different ideas for new techniques, strategies, articles, talks and so on. The opportunities are endless.
    • Use Mind Maps to visualize your thoughts and goals. This gives you something concrete to think about and helps in prioritizing each one of them.
    • Listen to inspiring podcasts and read motivational books.
    • Do deep work.
    • Have trusted Mentors to help you out in your journey. They help to challenge ideas, brainstorm solutions and guide you. Meet with them regularly via Skype or on a 1:1 basis.

Factor 2Communication

  • Intercultural communication – In a corporate world, we work with people from different cultures and regions. That being said, be cognizant of the cultural differences, usage of idioms and phrases and help them adapt to these differences. The learning goes both ways in terms of people from different cultures having an open mind and learning from the local people and vice versa.
  • Body Language – About 55% of our communication is through body language. Thus, having effective body language is important when working with other people. Be a good listener, have proper eye contact and pay attention.
  • Tone of Voice – Raising our voice in meetings to express our opinions or concerns does not work. When we raise our voice, the point does not get communicated across to the other person. The only thing noticeable during these times is the fact that, the other person is shouting and it automatically makes people react in a less amicable way.
  • Mental Models – People create their own mental models about objects, systems, ideas, description and so on. It is important to notice this and be open to hearing other people’s ideas.
  • Know your audience – Different people need different kinds of information. We need to modify our testing story based on our audience.
  • Safety Language – Prevent digging a hole for yourself by using safety language. Use word fillers like “could be”, “may have”, “would it be” etc. For Example – You can say “This defect could be reproduced under these conditions” instead of saying “This defect will be reproduced only in this way.”

Factor 3Education

  • Validated Learning – The“Lean startup” principles of Build->Measure->Learn holds good in testing as well. Always build a Minimal Viable Product or Proof of Concept, then solicit feedback. Based on the feedback keep making the product better. Follow an iterative approach.
  • Pairing
    • Developer – Tester pairing
      • Pairing while writing unit tests helps to identify gaps in development testing and these gaps can be addressed by Testers during their testing process.
      • Pairing in code reviews helps to find vulnerabilities in code.
      • Pairing also helps in learning the programming language.
    • Pair Testing with Testers/Developers
      • Paired Testing helps in continuous exchange of ideas/observations and helps to better explore the system
  • We gain experience only by committing mistakes. Remember “Things are never as bad as they feel or good as they sound.”
  • Conferences
    • The track sessions help to learn about different topics relevant to the industry. If you do not like one session feel free to go to another one. A lot of money is invested by you and your company, so take advantage of it.
    • Do your research – Before going to a conference, identify people you want to network with, then meet up with them during the conference to learn and exchange ideas.
    • Hallway conversations and networking – A lot of learning takes place outside the conferences rooms in the hallways and networking events during the conference. Ensure you exchange business cards; in the back of the cards note down hints about the person and follow up with him/her after the conference.
    • Share your ideas, thoughts and problems with the community. Use blogs, LinkedIn and Twitter to help other people like you.

If you want different results, you need to be willing to do things differently and different things” (from 12 Week Year)

In case of any questions, please feel free to contact me at raj@testim.io or visit my website at www.rajsubra.com

To get inspired, visit this page which is my source of inspiration – http://www.rajsubra.com/inspirational-books-articles/

Finally, my youtube videos can be found here – http://www.rajsubra.com/my-youtube-channel/

 

Be smart & save time...
Simply automate.

Get Started Free