Category

QA

Category

By: Sofía Palamarchuk for Abstracta

If you work in the software industry, you’ve most likely heard about the popular term, “shift-left testing.” With Agile practices like TDD, BDD, CI and DevOps becoming mainstream, “shift-left” is the answer to how testing fits in, and must be done in order for them to become a reality. Instead of taking a backseat during the development process, testing is planned in advance and begins earlier in the SDLC (therefore “shifts left”). It could even start before a single line of code is written! Making this shift changes the view of testing instead of traditional QA, it transforms into QE: Quality Engineering.

What Does Shift-Left Testing Look Like?

Thanks to the rise of automation, and the aid of tools that use AI and machine learning, testers have more time to dedicate to being more strategic about their work, instead of having their hands tied, running tests manually every day.

For testers to be successful today, they have to not only be great at testing, but also be engineers of the Agile testing process by collaborating with development and operations while analyzing quality during every stage of development:

Shift Left Testing

Shift-left testing activities include:

  • Testers helping developers implement unit testing
  • Planning, creating, and automating integration test cases
  • Planning, creating, and employing virtualized services at every stage and component level
  • Gathering, prioritizing, and processing feedback

Several process changes occur when teams shift left. Instead of a developer waiting weeks to add his or her code to the rest of the team’s code, it can be done every day, or even several times a day. Instead of manually performing all the tests, most are automated and run every day, or even several times a day. And, instead of detecting problems at the end, the team as a whole analyzes quality as the development progresses.

Not sure if it’s the right move for your organization? Here are some of the pros and cons of shift left testing.

PROS

LOWER THE COST OF TESTING & DEVELOPMENT

It’s well known that the sooner a bug is found, the cheaper it is to fix. One of the aims of Agile testing is detecting errors as soon as possible. With shift-left testing it’s possible to detect in real time, the exact moment in which an error was inserted into the system in order to resolve it in a timely manner. When testing is done with each build (especially during unit testing), the errors that are found are smaller, easier to detect and locate, and subsequently, less costly to fix.

INCREASE EFFICIENCY & QUALITY

With the increased levels of automation when shifting left, teams can benefit from:

  • Increased test coverage since more tests can be run in the same amount of time
  • More time for testers to focus on more challenging and rewarding tasks
  • Reduced human error rate
  • Monitoring performance over time
  • Code quality checks
  • Built-in security checks
  • Reducing issues in production that users may encounter

Beyond these benefits, being able to start testing sooner invariably results in a higher quality product, as testers are less rushed to find all the errors at the end, when there’s little time left to fix them.

COMPETE MORE FIERCELY

In today’s competitive technological landscape, the barriers to compete are minimal, so the best way to survive is to be able to move fast and defend one’s stature by innovating in iterations, which is possible thanks to Agile. As everyone can agree that it’s important to deliver software more quickly, it also mustn’t be rushed out the door (causing a possible backfire). Shift-left testing answers the problem of accelerating development without sacrificing quality.

Anotheryet less obviousbenefit of shifting left is that it can help businesses position themselves as an attractive employer to top talent. Because it is becoming more mainstream, with about two thirds of IT workers reportedly using Agile or leaning towards agile (according to a recent study by HP), it’s what the most forward thinking software professionals expect from their teams. Therefore, if you want to be an attractive employer or at least on-par with the rest, it is important to adopt the modern practices that both testers and developers want to master in order to stay relevant in today’s labor market.

Cons

EASIER SAID THAN DONE

For shift-left testing to be a success, an often drastic change in culture must occur first, which requires a team effort. Teams are usually set in their traditional ways of working, and when they consider shifting, they must consider how the methods, processes, skills, tooling, etc. will need to change. Even more important, what will need to happen to get all the roles within the organization to align properly?

RISK OF BOTTLENECKS

Yes, agile and shift-left aim to eliminate testing as a bottleneck, but it is true that agile teams can find themselves stuck waiting in a queue once all of the pieces come together in the performance and user acceptance testing phases, due to the complexity of environments and composite applications. One way to overcome this is to to utilize service virtualization. Service virtualization emulates the behavior of essential components that will be present in production, enabling integration tests to take place much earlier in development. This is how you can eliminate that key bottleneck, while also benefiting from eliminating errors earlier on. Along with service virtualization, there are several tools to setup automated systems and CI such as Jenkins.

A Worthwhile Undertaking

In the end, shift left testing is certain to have pros that outweigh its cons. Testers will find themselves delegating some of their work to developers and assigning them more testing activities. In mature teams, the testers become “coaches,” training developers on how to write better code, avoid bugs, and own unit testing. The advantage of this is that the tester who used to be busy, bogged down by writing test cases, now has time to delve deeper into the product, working on business cases, penetration testing, performance testing, implementing smarter testing solutions that use artificial intelligence like Testim.io, and so on. This sharing of the responsibility over testing leads to a higher level of achieved quality, as more of the bases get covered, quicker!

What do you think? Still not on-board to shift left testing? Or have you managed to do so already?

About the Author: Sofía Palamarchuk

Sofia Palamarchuk

Sofía is the Chief Executive Officer and Chief Product Officer of software testing services company, Abstracta. With a B.S. in Computer Engineering, Sofía started working in application performance optimization, system monitoring and load testing for the corporate sector for many years. With a solid background in performance tuning and automation, Sofía has become a business development leader and is responsible for managing all aspects of Abstracta’s US operations as well as its mobile testing tool, Monkop.

One of the most important factors related to automated tests is Maintenance. A lot of effort is spent on maintaining the tests than writing actual tests.  A recent study suggested about 30% of testers time is spent on maintenance.This leads to wastage of valuable time and effort by the resources, which they could have rather spent on testing the actual application.

Imagine a world where the software can maintain tests without human interaction? This world has become a reality with Testim.io. We use Artificial Intelligence (AI) underneath the hood, which provides self-healing maintenance i.e problems are detected by the AI and automatically fixed without human intervention.

Testim.io also help to speed up the maintenance of tests by providing the follow features within our platform-

  1. Version Control

At any given time, it is important to have logs of what changes were made to a particular test. This way we can always revert back to an older version of test as and when required. Our platform provides this functionality by showing all the version history by going to the Properties panel of the setup step and clicking on “See old revisions”

  1. Branching

At Testim.io, we firmly believe in the “Shift Left Paradigm” where Development and Testing must start in parallel as early as possible in the software development lifecycle. Keeping this in mind, we provide the functionality to teams to create separate branches for each team member and work on the same projects and tests. This way, no one can overwrite the changes of the other team members and teams can work on the same code base at any instant of time

In our platform, we just need to select “Fork” to create a new branch and we can also switch between existing branches


        3.  Scheduler

Users have the option of scheduling their tests. This helps to run the tests automatically at a certain day and time without any manual intervention. We can also get notified via email in case of any errors

 

Troubleshooting

As testers, we spend considerable amount of time troubleshooting issues. To help in troubleshooting, our platform offers different options to the user to narrow down the scope of the problem. These options  are as follows-

  1. Screenshots

The screenshot feature explained in the “Authoring and Execution” section helps users to know what was the baseline image and what was the actual image found.

  1.   Properties Panel

The properties panel helps to capture the error messages and display it to the user. The user also has the option of interacting with DOM and see what objects were extracted during the run

  1. Test Logs

Logs are a rich source of information on what happened underneath the UI. We provide test logs when the user runs the tests on our grid or a 3rd party grid. The option can be found in the in top section of editor

  1. Bug Reporting

One of the most time consuming aspects of testing is after finding a bug, we need to report it to the developer with relevant information, to speed up the troubleshooting and fixing of issues.

With Testim.io you can do this with a single click with the help of our chrome extension. All the details related to the bug are automatically generated for you.

  1. Documentation

We put in a lot of effort to document most of the features of the tool in our User Documentation found under the “Educate” tab.

We also have detailed videos on how to troubleshoot your tests quickly

Troubleshooting Part 1- Element is not visible

Troubleshooting Part 2 – Element not found

Troubleshooting Part 3 – Timing issues

Troubleshooting Part 4 – Issues related to mouse hover

With the above features, Testim.io helps to create stable tests that are highly maintainable.

The below posts gives more in depth analysis of Testim in terms of different features that make authoring and execution of tests really simple and how to create reusable components that improves the extensibility of the tool

https://blog.testim.io/bringing-simplicity-to-authoring-and-execution-of-automated-tests/

https://blog.testim.io/how-to-make-reusable-and-extensible-code-using-testim-io/

 

Artificial Intelligence (AI) and machine learning (ML) are advancing at a rapid pace. Companies like Apple, Tesla, Google, Amazon, Facebook and others have started investing more into AI to solve different technological problems in the areas of healthcare, autonomous cars, search engines, predictive modeling and much more. Applying AI is real. It’s coming fast. It’s going to affect every business, no matter how big or small. This being the case how are we as Testers going to adapt to this change and embrace AI? Here is the summary of different things you need to know about using AI in software testing.

Let’s summarize how the testing practice has evolved over the last 4 decades

  • In the 1980’s majority of software development was waterfall and testing was manual
  • In the 1990’s, we had bulky automation tools which were super expensive, unstable and had really primitive functionality. During the same time, there were different development approaches being experimented like Scrum, XP, RAD (Rapid Application Development)
  • From 2000, the era of open source frameworks began
    • People wanted to share their knowledge with the community
    • Started encouraging innovation and asking community of like minded people to help out in improving testing
    • Agile became a big thing, XP, Scrum, Kanban became a standard process in the SDLC
    • There were need for faster release cycles as people wanted more software features delivered faster
  • In the 2010’s, it was all about scale, how to write tests fast and find bugs faster
    • Crowdtesting started
      • Encouraging other people to give feedback on the application. Free and Paid services
    • Cloud testing started
      • People started realizing they need more
        • Server space
        • Faster processing
      • Started to realize the problem of maintenance. How expensive it is to buy hardware and software for maintaining your tests
      • Then we have
        • DevOps
        • Continuous Testing
        • CI/CD integration
  • I believe the Future will be about Autonomous Testing using Machine Learning and AI

 

Basics of AI

Let’s start by de-mystifying some of the terminologies related to AI

  • Artificial Intelligence (AI) is an area of computer science that emphasizes the creation of intelligent machines that work and react like humans
  • Machine Learning (ML) evolved from the study of pattern recognition and computational learning theory (studying design and analysis of ML algorithms) in AI. It is a field of study that gives computers ability to learn without being explicitly programmed
  • Deep Learning(DL) is one of the many approaches to ML. Other approaches include decision tree learning, inductive logic programming, clustering and Bayesian networks. It is based on neural networks in the human body. Each neuron keeps learning and interconnects with other neurons to perform different actions based on different responses

 

There are 3 types of widely used ML algorithms

  • Supervised Learning – We are giving the right training data (input/output combinations) for the algorithm to learn
    • Examples
      • Give bunch of e-mails and identify spam e-mails
      • Extracting text from audio
      • Fill out a loan application and find the probability of the user repaying the loan
      • How to make user click on ads by learning their behavior
      • Recommendation engines on Amazon, Netflix where customer is recommended products and movies
      • Amazon uses AI for logistics
      • Car Optimization
      • Autonomous cars
  • Un-supervised learning – We give a bunch of data and see what we can find
    • Examples
      • Taking a single image and creating a 3D model
      • Market Segmentation
  • Reinforced learning – Based on concept of reward function. Rewarding Good/Bad behavior and making the algorithm learn from it. E.g. Training a Dog

 

Real life AI applications to visually see how it works

  • Quick Draw from Google
  • Weka is an open source project where they are using ML algorithms for data mining

 

What challenges can AI solve?

Let’s discuss the challenges the industry faced while transitioning to agile and what’s still remains a challenge:

How can we use AI to solve testing problems?

There are many companies taking multiple approaches to solve different problems related to software testing and automation. Testim.io is one such company

Testim.io uses Dynamic Locators, The Artificial Intelligence (AI) underneath the platform in real time, analyzes all the DOM objects of a page and extracts the objects and its properties. Finally, the AI decides the best location strategy to locate a particular element based on this analysis. Due to this, even if a developer changes the attribute of an element, the test still continues to run and this leads to more stable tests. As a result, the authoring and execution of automated tests are much faster and more stable.

 

Here is the detailed insight of how our AI works – https://www.softwaretestinghelp.com/testim-io-tool-tutorial/

One of the good practices of writing automated tests is creating reusable components that can be used in different parts of our test suite.

Why is this important?

Creating reusable components is important because it

  • Helps to increase the readability of the automated tests
  • Saves effort by not repeating the same set of steps in different parts of the tests
  • Any changes to the reusable step needs to be done only in one place and it is reflected throughout the tests, across different projects
  • Makes the automated tests more extensible

 

Testim.io helps to ensure Reusability by “Grouping” and “Parameterization”.

  • Grouping

Any number of related steps can be grouped into one reusable component.

For Example – The “Login” scenario is one of the most commonly used steps in any application. The way we can create a reusable “Login” step would be to select the steps we want to group together and clicking on “Add new Group” as shown below

  1. Parameterization

Our platform gives the option of testing application through various input combinations via Parameterization.

This can be achieved in various ways. One way to do this is to give all the input parameters we would need to test the application in the form of a JSON file in the Setup step (The first step of our tests) as shown below

Then add the variable names used in the json file in the appropriate fields of the step as show below

 

Another important aspect of automation is building your tests such that it is extensible.

Why is this important?

As the product and teams grow, there will be need to test more complex functionalities which would require building upon already existing tests. This being the case, the automation suites need to be simple, understandable and should be easy to add more tests to already existing test suites with low coupling and high cohesion.

Testim.io gives the flexibility for organizations to extend the functionalities of our platform using JavaScript and HTML. This way, any functionality our platform does not handle; the user can write their own code to build a robust automation framework

For Example – Say we want to validate the “Select Destination” button from our previous examples. The way to do this would be.

  • Click on “Add custom action”
  • Give a name to the New Step and click on “Confirm”
  • Click on “PARAMS” and Select “HTML” for this example
  • Add Custom Code

The new step with Custom Code gets added to the list of already existing steps

The above features help to make the automation suite more reusable and extensible.

The below posts gives more in depth analysis of Testim in terms of different features that make authoring and execution of tests really simple and easy to maintain

https://blog.testim.io/bringing-simplicity-to-authoring-and-execution-of-automated-tests/

https://blog.testim.io/maintenance-of-tests-made-easy-with-testim-io/

Authoring and Execution of tests is an important aspect of test automation. Tests should be simple to write, understand and execute across projects. The automation framework or tool chosen should give the flexibility to record and playback tests as well as, write custom code to extend the functionalities of the automation framework.

This is where Testim.io can help you out. We follow a  Hybrid Approach and make authoring and execution of tests really simple in such a way that both technical and non-technical members can collaborate and start writing automated tests quickly. This is achieved with the use of “Dynamic Locators”.

What are Dynamic Locators?

The Artificial Intelligence (AI) underneath the platform in real time, analyzes all the DOM objects of a page and extracts the objects and its properties. Finally, the AI decides the best location strategy to locate a particular element based on this analysis. Due to this, even if a developer changes the attribute of an element, the test still continues to run and this leads to more stable tests. As a result, the authoring and execution of automated tests are much faster and more stable.

As we can notice from the above image, the AI parses through all the DOM objects, lists them in the Properties Panel along with the rankings of each and every location strategy for that particular element. In this way, even if the attribute of an element changes, the AI can use a different location strategy from the already parsed list of DOM objects.

Thus, the user does not have to worry about flaky tests.

Some of the basic authoring and execution features Testim.io provides to its customers, are explained below.

  1. How to create a test

We create a new Test by clicking on “Create New” or “New Test”

 

  1. Recording and Saving a test

Once we click the “Record” button, we can record different user actions in our application. After recording different actions, click on “Stop Recording” button to finish recording our tests. Use the “Save” button to save the tests.

 

  1. Validations and Assertions

Our platform helps to make validation of different attributes of an element and API’s really simple. We provide various options for users such as

  • Adding Custom Validations using JavaScript and HTML
  • Validate element visibility
  • Validate element text
  • Pixel level validation
  • API level validation

 

  1. Screenshots

While each test is recorded, the platform takes screenshots of all the Pass and Failed results of each and every step. As a result, users find it easier to troubleshoot problems and understand what happens underneath the hood.

 

  1. Feedback on each step

The user also gets feedback on each step in terms of whether the tests Passed or Failed by showing a “Green” or “Red icon” on the top left portion of each step as shown below

 

  1. Labeling tests

Testim.io provides the feature to label each and every test a user creates. There are 2 reasons why we may want to label a test

  • Helps to identify the reason the test was created in the first place
  • Helps to run tests with the same label all at once through our CLI feature

The way we create labels is by clicking on the “Label” button and either select an existing a label or create a new one.

 

  1. User Documentation

At Testim.io, we took the effort to provide users with all the documentation they will need to use different features of our platform. Most of the answers about using our platform can be found by clicking on the “Educate” tab and Visiting our documentation site as shown below

With the above features, Testim.io helps to make the authoring and execution of tests really fast and simple for our users. Within a matter of seconds a user can record, replay and save the tests. This is surprisingly one of the most overlooked aspects of test automation and our platform takes care of it for our users.

The below posts gives more in depth analysis of Testim in terms of creating reusable components, extensibility and maintenance

https://blog.testim.io/how-to-make-reusable-and-extensible-code-using-testim-io/

https://blog.testim.io/maintenance-of-tests-made-easy-with-testim-io/

 

When we hear the phrase “Record and Playback”, a majority of the people cringe with fear and skepticism as they relate it to primitive, unstable and flaky tests. Organizations have viewed it as a sign of vulnerability in automation and have continued to discourage teams from doing record and playback tests for many years now. The main reason for this is, it leads to-

  • Higher maintenance of tests
  • Lesser stability of tests as it breaks if any element changes
  • Unclear test coverage
  • Tests are highly coupled

This is not a new phenomenon, this has been the case for the past 20 years within which the state of automation has evolved by leaps and bounds.

I am not going to refute the above points as it is true in some cases but people fail to realize there is a time and place for everything which includes “record and playback tests”. These type of tests are valuable to-

  • Do fuzz testing (a.k.a monkey testing), which involves recording large amounts of random data through vast number of valid and invalid actions/assertions and observe the application under test. This helps to uncover issues like memory leaks, unexpected crashes and helps to evaluate the system under extreme conditions that otherwise may be hard to do with normal structured automated tests.

Monkey Testing

  • Perform automated exploratory testing, where the user tries out multiple scenarios and records multiple actions while simultaneously learning about the application and the tool used for automated testing.

a b testing

  • Help in load testing, by quickly recording a bunch of tests and simulate thousands of users concurrently performing the same set of recorded actions on the application.

load testing

  • It helps to get the whole team involved in test automation irrespective of their skillsets.

Now, you may think, “Why am I highlighting the advantages and disadvantages of Record and Playback tests”? The answer is, we at Testim.io, recognized these factors and came up with a hybrid approach to solve the problems with record and playback, by building a platform based on Artificial Intelligence (AI).

Testim.io follows a hybrid approach where we give organizations and users the ability to record and playback tests; while at the same time giving the users flexibility to programmatically manipulate these recorded tests. These tasks can be performed easily using inbuilt functionalities of the platform. It also gives teams the freedom to add their own wrappers around the platform (if needed) by using Javascript and HTML.

hybrid testing approach

To increase stability of tests irrespective of the way the tests are written, Testim.io uses Dynamic Locators. The AI underneath the platform in real time, analyzes all the DOM objects of a page and extracts the object trees and its properties. Finally, it decides on the best location strategy to locate a particular element based on this analysis. The more tests you run, the smarter the AI becomes in increasing the stability of the automated tests. So, even if your strategy for automation is only record and playback, re-running the recorded tests multiple times helps to make those tests stable even if the attribute of an element changes in the future. As a result, the authoring and execution of tests are much faster.

In summary, there are various approaches to test automation. Each approach has its own merits/demerits. Understanding and using the approach that makes more sense in the context of the project is crucial to help in better testing using automation tools, platforms and frameworks. As these options continue to mature, it will become all the more important to follow the hybrid approach to cater to different type of skill sets, needs and expectations of teams and organizations. Hybrid approach to testing is the new era of test automation.

Curious to see how we implement the hybrid approach? Sign up for our free trial.

Do you ever ask yourself these questions?

  • How do I know if my quality is improving?
  • What did the team do this week and what was the impact?
  • Do we have enough test coverage in the sprint?

With Testim’s new Managerial Reports, you never have to worry about getting answers to these questions again. Get deep insight into project status and quality metrics which provide granular details into execution cycles, active runs, run durations and success rates – all available online or sent weekly to your inbox.

Testim Test Automation Managerial Reports

These reports, dashboards and KPIs quickly summarize the team’s effort invested over the course of the week, identify tests that require attention as well as if additional effort is required to improve your quality score. Easily track trends week over week and see how your quality coverage is improving.

Testim Test Automation Managerial Reports

For the next two months Testim is offering these reports free of charge. Moving forward, additional licensing will be required to take advantage of these new insights. Sign into Testim to see the new reports now or contact your account manager with any questions.

Guide: Evaluating UI and Mobile Automated Testing Solutions

Growing up as Testers in the software testing industry, we often go through a lot of thoughts and have several questions in mind such as:

  • How do I learn about software testing?
  • I like my job but how do I get to the next level?
  • Am I good at my job?
  • There are so many testing jargons people are using and I do not understand any of them
  • You are trying to communicate effectively but people still do not understand you

Based on testing software for a over a decade now, reading articles and blogs, interacting with practitioners from all over the world and analyzing my success and failures as a tester in the past several years; I discovered everything comes down to 3 Key Factors that paves the path to becoming a strong tester. These factors form the Strong Tester Model Stability as shown below.

Strong TestersFactor 1Motivation

  • “Run Your Own Race” – As Testers, we constantly keep comparing ourselves with other people, try to do more without paying attention to what our goals are and finally end up getting stressed as a result of being overworked and concentrating on lesser priority things. In life and in testing, we need to remember that, the ONLY person we are competing with; is OURSELVES. We need to identify our strengths and answer to our conscience, rather than comparing ourselves with others who totally have different set of goals.
  • Embrace Your Talents – Recognize your strengths and weaknesses. Embrace them and work to your skill sets with well defined goals and deadlines. Hold yourself accountable.
  • Go Explore – We will find our true passion only when we explore different things and take chances. Everyone starts from somewhere and no one is an overnight success. So start your journey and exploration. Try anything at least once with full dedication and determination. Remember “If you take a chance in life sometimes good things happen and sometimes bad things happen. But if you don’t take a chance, nothing happens.”
  • Tips and tricks for sustained and continuous motivation:
    • Have inspirational quotes to get you going when you are down or feel lost. Everyone has a trigger point, what is yours?
    • Have a Testing Notebook to note down all things testing when you read, explore and talk to people. Looking at your notes will spark different ideas for new techniques, strategies, articles, talks and so on. The opportunities are endless.
    • Use Mind Maps to visualize your thoughts and goals. This gives you something concrete to think about and helps in prioritizing each one of them.
    • Listen to inspiring podcasts and read motivational books.
    • Do deep work.
    • Have trusted Mentors to help you out in your journey. They help to challenge ideas, brainstorm solutions and guide you. Meet with them regularly via Skype or on a 1:1 basis.

Factor 2Communication

  • Intercultural communication – In a corporate world, we work with people from different cultures and regions. That being said, be cognizant of the cultural differences, usage of idioms and phrases and help them adapt to these differences. The learning goes both ways in terms of people from different cultures having an open mind and learning from the local people and vice versa.
  • Body Language – About 55% of our communication is through body language. Thus, having effective body language is important when working with other people. Be a good listener, have proper eye contact and pay attention.
  • Tone of Voice – Raising our voice in meetings to express our opinions or concerns does not work. When we raise our voice, the point does not get communicated across to the other person. The only thing noticeable during these times is the fact that, the other person is shouting and it automatically makes people react in a less amicable way.
  • Mental Models – People create their own mental models about objects, systems, ideas, description and so on. It is important to notice this and be open to hearing other people’s ideas.
  • Know your audience – Different people need different kinds of information. We need to modify our testing story based on our audience.
  • Safety Language – Prevent digging a hole for yourself by using safety language. Use word fillers like “could be”, “may have”, “would it be” etc. For Example – You can say “This defect could be reproduced under these conditions” instead of saying “This defect will be reproduced only in this way.”

Factor 3Education

  • Validated Learning – The“Lean startup” principles of Build->Measure->Learn holds good in testing as well. Always build a Minimal Viable Product or Proof of Concept, then solicit feedback. Based on the feedback keep making the product better. Follow an iterative approach.
  • Pairing
    • Developer – Tester pairing
      • Pairing while writing unit tests helps to identify gaps in development testing and these gaps can be addressed by Testers during their testing process.
      • Pairing in code reviews helps to find vulnerabilities in code.
      • Pairing also helps in learning the programming language.
    • Pair Testing with Testers/Developers
      • Paired Testing helps in continuous exchange of ideas/observations and helps to better explore the system
  • We gain experience only by committing mistakes. Remember “Things are never as bad as they feel or good as they sound.”
  • Conferences
    • The track sessions help to learn about different topics relevant to the industry. If you do not like one session feel free to go to another one. A lot of money is invested by you and your company, so take advantage of it.
    • Do your research – Before going to a conference, identify people you want to network with, then meet up with them during the conference to learn and exchange ideas.
    • Hallway conversations and networking – A lot of learning takes place outside the conferences rooms in the hallways and networking events during the conference. Ensure you exchange business cards; in the back of the cards note down hints about the person and follow up with him/her after the conference.
    • Share your ideas, thoughts and problems with the community. Use blogs, LinkedIn and Twitter to help other people like you.

If you want different results, you need to be willing to do things differently and different things” (from 12 Week Year)

In case of any questions, please feel free to contact me at raj@testim.io or visit my website at www.rajsubra.com

To get inspired, visit this page which is my source of inspiration – http://www.rajsubra.com/inspirational-books-articles/

Finally, my youtube videos can be found here – http://www.rajsubra.com/my-youtube-channel/

 

Guide: The Key to Implementing CI/CD

Time is money. How many times have you heard that?

We are truly a “Startup nation” constantly racing against the clock to deliver multiple features and execute sprint content, to meet our customer’s demands. This intense pace is an everyday reality. As a QA Engineer that has worked with and in different organizations, I have experienced it up-close and personal.

On one side there are owners and investors – they want to see growth. On the other side, there are customers – they want features and capabilities that work. And then there is us, Testers – we want to deliver quality.

Where does software quality lie in the release process?

Let’s start by defining software quality and how would you measure it?  In the context of software engineering,  software quality refers to how well it meets functional and non-functional requirements.  In terms of how well you can measure it; there are various metrics associated with it and is complex. This is like asking, “how would you define a high-quality watch, car, or a clothing item?”

Could it be that from using the product you can feel that its creator/maker used good materials (even if it means that the price would be higher)? If you use it for a long time, would it still be preserved from standard wear and tear? Is it designed to be comfortable? Fun to use? Does it break or overheat when you accelerate to a high speeds or drive long distances?

With that said, a Mercedes costs 10 times more than a Honda. Does that mean that a Honda is not a good quality car?

All of these examples teach us that quality is based on a perceived notion of price to value.

  • Does the product serve its purpose in a good way?
  • Can it stand up to our “use demands” in a reliable, long-lasting way?

Price can be interpreted in different ways as well – for example – implementation and maintenance time.  Don’t be fooled,  breaking this perception is a lot easier than building it in the eyes of our users. I can think of more than one car brand that has managed to break our perception of it in the last decade or so.

The farther you run, the more expensive it is to go back.

What I’m about to write will not be a shock to anyone, and still, you will be surprised to hear the number of organizations I see that just don’t seem to assimilate that idea. The earlier you will incorporate quality practices into the product’s life cycle – the less money you will spend on fixing it in the long runI have seen how a wide aspect feature is being invented, designed and developed, only to understand that it is different from the poet’s intention, or does not serve its purpose the best way.

What can possibly go wrong? Nothing much, just:

  • Features developed contrary to the customer’s tastes/needs
  • Modules that do not carry out the action for which they were designed
  • Time-consuming debates on whether it was a bug or maybe the characterization was not unequivocal and clear enough
  • Going back and forth by working on/fixing the same module for several times
    • As a result: lack of time to perform all of the evaluated “sprint content” = less features, less progress
  • Features that aren’t clear enough for the user and result in complaints or support issues
  • Unsatisfied customers
  • Bad user experience
  • Bugs in production
  • Wasting time = money

Make a decision, start the change!

Start by understanding the importance of having a wide range testing infrastructure in your organization. Sure, it will not happen overnight. Yet still, it is an investment that produces huge value.

Suit up, sneakers on, and start the process:

  • Start designing a work procedure that includes clear written requirements. People never completely understand one another. If there is no written documentation, it can become challenging to examine what to develop and what to test. I am realistic, there is no place or time for detailed requirements in a startup. However, we can design a format which is short enough to be written in our race and clear enough for the developer to understand what to perform.  
  • Implement the use of a knowledge base and management tools that will suit your needs. Lets define knowledge base.  As far as I’m concerned, it can be a Google-Drive but you still need one place that I can go to if I need to know how a feature works. Now, about management tools, lets just say that it is a topic of a whole another article and still what you need to understand is that there are free requirement management, bug management and process driven tools that could help you organize your everyday tasks.
  • Start building a work process which will suit your company’s needs and corporate culture. The same as there are rules to running a marathon, there has to be rules to running a startup. When do you start the run? When can the tester join? What is the criteria that indicates that the lap is over? What can and can’t be done in the middle of the sprint? Every game needs its set of rules.
  • Implement productive communication between product and QA. How? Well for one thing here is personal example. Start by implementing a productive and positive discussion. Every individual requires an approach so they will not “lock up” to a defensive position and observe your words. Take some time to learn your environment and how to approach each member.
  • Assign a quality lead that will provide you with the added value of a productive process.
  • Incorporate quality practices as early on as possible in the concept, design, and development. You will be surprised how effective it is to review requirements (even before one line of code has been written) and how much time it can save you in the long run.  
  • Make an organizational level decision to make quality an high priority issue, and let QA do their job with the appropriate amount of authority.

Two points I would like to add to all that has been written:

  1. It is a process and conception that takes time, but it will return its investment.
  2. It’s not magic, there will always be bugs! The question is in what amount and what severity?

Remember the question we asked at the beginning of the article? (How do we fit in this never-ending race?)

I would like to wrap up this article with a quote by the famous Tester Michael Bolton:

“I wish more testers understood that testing is not about “building confidence in the product”. That’s not our job at all. If you want confidence, go ask a developer or a marketer. It’s our job to find out what the product is and what it does, warts and all. To investigate. It’s our job to find out where confidence isn’t warranted where there are problems in the product that threaten its value.”

As many yoga instructors do, they encouraged students to find balance. It’s an effective bit of advice in that it puts the onus on the practitioner. It’s also a popular concept for software testing, as industry experts often recommend that software teams find the balance between automation and manual testing practices. Another similarity is the trajectory of their adoption rates. One does not dive into yoga, but slowly adopts it over time until it becomes a part of their daily routine. The same goes for test automation: you can’t expect to start automating everything from scratch. Good test automation is the culmination of work over time.

Why not all manual?

Manual testing is fine, and was once the status quo, but with high adoption of Agile and continuous delivery methodologies, it just doesn’t scale. For software development, every enhancement or new feature that’s rolled out for an app must have a corresponding set of tests to ensure the new functionality works and does not break the code from the previous version. In this regard, to check all of the file directories, database, workflows, integrations, rules and logic manually would be extremely time consuming.

For companies looking to improve time to market and increase test coverage, automation provides significant advantages over manual testing. A typical environment can host thousands of test cases of varying complexity to be executed effortlessly and nonstop. As a result, automation dwindles the time required to perform mundane, repetitive test cases from days to hours and even minutes if you run in parallel. That inherent velocity increase is what attracts teams to automated testing.

Why not all automation?

The benefits of automation are obvious. Automation provides the ability to quickly react to ever-changing business requirements and generate new tests continuously. Is it reasonable then to assume that the more you automate the more benefits you reap? Then why not just automate everything?

The short answer, its very difficult. According to the World Quality Report 2017-18 Ninth Edition, these are the top challenges enterprise are facing in regards  to test automation.

Traditionally you would automate only once your platform is stable and the user flow is pretty consistent. This was primarily due to the amount of time it took to write a test, sometimes hours and even days. Plus the amount of time it took to update and maintain tests as code changed. The ROI was not justified up to the point your user flow is stable.

Feedback from our customers such it has changed. We recently heard a number of our clients who automate everything and even as early as wireframes are ready. We reduced authoring time by about 70% of the time and maintenance time by about 90%. Creating a user flow now takes a few minutes with Testim and so is updating the user flow. Reducing the time per test to a few minutes instead of hours and days makes it much more affordable to automate, providing immediate ROI from feedback loop benefits. This is made possible with the help of our AI (Artificial Intelligence) underneath the hood making the tests much more faster and stable.

So should you strive to 100% automation? Well no but you can get to 80%-90% which was unheard of until recently. There are still scenarios that only humans can do such as Face ID. There are also other aspects of your Software Development Lifecycle (SDLC) that automation is dependent on.

Summary

There is an ongoing struggle to keep up with the rate dictated customer pressure and competition to produce on a continuous basis. And not just produce for production’s sake, but a quality product that’s been thoroughly tested. That’s why we created Testim…

To help software teams:

  • Increase coverage by 30% in 3 months
  • Reduce number of missed bugs by 37%   
  • Increase team productivity by 62%
  • Reduce regression testing costs by 80%  
  • Speedup testing schedules by 3 months

Guide: The Key to Implementing CI/CD

Sources

  1. https://techbeacon.com/world-quality-report-2017-18-state-qa-testing
  2. https://dzone.com/articles/automated-vs-manual-testing-how-to-find-the-right
  3. https://blog.qasource.com/the-right-balance-of-automated-vs.-manual-testing-service
  4. http://sdtimes.com/software-testing-is-all-about-automation/#sthash.5Idkps2O.dpuf
  5. http://sdtimes.com/report-test-automation-increasing/#sthash.HlHQmD2l.dpuf
  6. https://appdevelopermagazine.com/4935/2017/2/8/test-automation-usage-on-the-rise/
  7. https://techbeacon.com/devops-automation-best-practices-how-much-too-much

Be smart & save time...
Simply automate.

Get Started Free