Category

Agile

Category

Maybe you’ve heard what test automation is but you’re unsure about how to fit it all together in practice. If that’s you, then you feel just like I did when I was first getting my head around how test automation works. So many conversations about testing quickly become abstract and sometimes we simply want to be practical—just show me the code!

If you feel the same way, then you’re in luck! Today we’re going to cover how automated tests work in detail. To do this, we’ll cover the test pyramid and how it relates to testing, which will set a foundation so we can go through and break down each category of testing—from unit tests to integration tests and finally to journey tests. And we did promise to keep things practical, so we’ll talk through some real-world code examples too.

By the end of this post, you should know what the test pyramid is and how it relates to automation testing. You will also learn in detail what unit, integration, and journey tests are and how they’re written.

What Are Automated Tests?

Before we begin, let’s start with a quick definition:

Automated tests perform machine-based assertions on an application to determine the state of the current application against the assertions.

In simple terms, we ask questions of our systems through testing, and the current state of the system (how it’s coded) informs our test results.

Don’t worry if it still feels abstract, as we’ll be breaking it down.

What Is the Test Pyramid?

Before we dive into our examples on each of our testing steps, let’s define them. A lot of the nuance of how automated tests should be written hinges on the theory around test pyramids, so it’s important to have a strong grasp before continuing.

At its heart, the testing pyramid is about feedback loops, and it forms a recommendation on how to structure your tests to achieve the fastest possible feedback loop.

But, what do I mean by feedback loop? I mean the time it takes from modifying some code until you know that it’s wrong (and, importantly, where it’s wrong). The notion of the pyramid is important as it represents how we should heavily load the bottom layer while keeping the top layer thin in order to retain our pyramid shape (more on balancing testing layers soon!).

Test pyramids can come in slightly different formats. For today, let’s use a simplified version, which breaks down into unit, integration, and journey tests.

Unit Tests

Unit tests are the base of our pyramid—they run fast and validate individual functions work as we expect. However, since we’re only testing at the function level, these tests don’t tell us whether our application works as a whole. But importantly, unit tests run blazingly fast. And when they fail, they tell us exactly what went wrong to the precision of a single function. So while unit tests don’t tell us if the application is broken as a user would deem it, they give us the fastest feedback possible.

Integration Tests

Integration tests form the middle of the testing pyramid. If unit tests together sum up to a single software component, then our integration tests ensure that each software component can work with another. Usually, integration tests run over a network boundary of some form. Integration tests essentially validate whether all the aforementioned functions are put together correctly and that other software components can successfully communicate with the component in question. Integration tests are less about what the functions do (that’s our unit tests), but more that our functions are put together correctly.

Journey Tests

These tests form the last layer and the top of our pyramid. Journey tests typically execute a journey as our user would see it. We don’t tend to cover edge cases in journey tests as they should be covered at base levels of the pyramid (either unit or integration).

And journey tests are merely a few, short tests that cover a lot of code. Because we’re writing so few of them and they cover so much code, they give us a lot of bang-for-your-buck for establishing if our application is working as expected since they are covering core journeys. But—frustratingly—they won’t really tell you what is broken (at least, not to the same degree that a unit test would). Additionally, journey tests can be very slow to run and brittle to maintain (but more on why this is later).

Putting the Pyramid Into Practice

Hopefully, by now you have a decent understanding of the test pyramid and how the layers fit together. Don’t worry if you don’t completely understand at this point. We’re going to break down each layer and show you how it works in practice.

Unit Tests: An Example

Unit tests run against individual pieces of code. So let’s take an example.

The following test is written using the Jest unit test library. Jest gives us the “test” and the “expect” function. With these tools, we can then describe the scenarios we want to run for our tests and also our expected output. Tests can have any number of assertions, but typically you’ll have only one (or a small amount). Keeping assertions small means we can fit them to our descriptions easier.

function isOdd(a) {

    if (typeof a != “number”) {

        throw new Error(“Must pass a number”);

    }

    return a % 2 != 0

}

test(“Passing an odd number returns true”, () => {

    expect(isOdd(1)).toBe(true);

});

test(“Passing an even number returns false”, () => {

    expect(isOdd(1)).toBe(true);

});

test(“Passing a string throws error”, () => {

    expect(() => isOdd(“Hello”)).toThrow(“Must pass a number”);

});

Here we can see that we’ve written:

  • A happy path test
  • An unhappy path test
  • A dynamic type checking and error handling test

The test coverage is therefore 100 percent. However, we may want to write additional cases in the future for edge cases if we believe they add more value. For instance: what should the function do if we pass superfluous arguments? And what about edge cases? Negative numbers? Passing a zero value? Or passing a very large value? All of these are valid tests, but adding more tests has diminishing value over time. If we’re incredibly thorough with our testing at all levels, we likely won’t ship any code. It’s a balance. Choosing whether to add these additional cases will depend on factors such as: how much business impact would there be if the code were faulty? Are there other tests you could write that deliver more value?

Integration Tests: An Example

Building on top of our unit tests we have our integration tests. Below is an example of a test written with SuperTest. SuperTest is a small helper library around Jest (which we used for the unit test example). SuperTest gives us the request functionality (and its chained assertion methods).

The following code takes an endpoint, in our case “/user” makes a request to it, asserting the response to be a 200 success. SuperTest also allows us to simply test HTTP endpoints. Remember when we said integration tests often happen over a network boundary? This is precisely what we meant. We tell our test what our setup criteria are, and we can then assert that we received the correct response, error code, headers, etc.

describe(‘GET /user’, function() {

  it(‘responds with json’, function(done) {

    request(app)

      .get(‘/user’)

      .set(‘Accept’, ‘application/json’)

      .expect(‘Content-Type’, /json/)

      .expect(200, done);

  });

});

But you might be thinking: in the above example are we simply testing that the endpoint responds? Not that it has expected values?

At first, this may seem like a low-value test. However, since we have already covered a lot of the functional testing in our unit tests at the integration test level, we are really only testing that our application links up all of those functions and responds without errors. Of course, we can write more detailed scenarios, but we want to be careful that we’re not replicating unit test code in integration tests. Unit tests run faster, so we should ideally write our tests there when possible.

Journey Tests: An Example

Last up is the journey test. The following is an example of a journey test using the Cypress testing framework. Cypress gives us a test runner and an assertion library. The “describe”, “it”, and “cy” commands all come from the Cypress library.

describe(“When at checkout”, () => {

    it(“Can buy”, () => {

        cy

            .get(“.buy”)

            .click();

        cy

            .get(“.summary”)

            .should(“have.text”, “Thanks for purchasing!”);

    });

});

In our journey test, we are testing the physical interface of the application, which should not only include what the user sees, but also behave as a user would when conducting a task (we call this a journey). In our example, we are navigating to our route, finding a button by its class name, clicking it, and then asserting that we can see our summary.

Can you spot why we said these tests could be brittle? For instance, if a developer decides to change an element class name (which in theory might not break functionality!), this change may result in a test failure. Oh no! We can use clever ways of structuring our code to reduce the fragility of our tests, but we cannot remove the brittleness completely—it’s inherent in the nature of the journey test.

The End of Our Tour of Automated Testing

And that concludes our foray into automated testing today. I hope this post gave you a clearer picture of how automated testing works in practice. We’ve seen how we need to have a balanced pyramid and also some examples of how our pyramid could start to be implemented.

In the real world, automated testing requires constant collaboration communication and continual improvement of our codebase. As we find bugs in our code, we work to ensure our coverage is updated. But in updating our codebase we need to ensure we balance our pyramid to keep our feedback fast.

It’s not easy. But, do it right and automated testing yields incredible results that are far superior to their manual testing counterparts. Learn how to identify the best automation platform for your needs with our simple, five step process. 

And remember, the best way to learn is to get your hands dirty and have a go! So take the examples and attempt to implement your own small application with each layer of the test pyramid!

Author bio: This post was written by Lou Bichard. Lou is a JavaScript full stack engineer with a passion for culture, approach, and delivery. He believes the best products emerge from high performing teams and practices. Lou is a fan and advocate of old-school lean and systems thinking, XP, continuous delivery, and DevOps.

Over the past few years, our team has been working hard to improve the functionality and usability of our autonomous testing platform. This mindset is what made us successful in building a product that people love and trust. In 2019, we released many new features that help testers author and execute tests more efficiently.

In this webinar, we discussed how the AI works underneath the hood, what are some of the features that help in faster authoring, execution and maintenance of automated tests and finally, discussed the top features within Testim that were released this year.

Here is the recording of the webinar

Below are some useful links related to features discussed in this webinar

Some useful blog posts to quickly get to know Testim

Regression testing is a specific form of testing that verifies whether a given piece of software suffered regressions after undergoing changes. “Regression” here means “going back to a previous undesired state.” So automated regression testing is nothing more than the process of automatically verifying that the application has not regressed to a previous undesired state.

Ideally, regression testing—whether automated or not—should be performed every time a software application is changed in some way, whether by receiving a new feature, an improvement, or a bug fix.

In today’s post, we’ll define and explain what automated regression testing means. We’ll start by explaining what a regression is, how costly they are for software teams, and why you should employ regression testing to avoid them. Since manual regression testing would be both time-consuming and error-prone, we’ll then proceed to cover the need for automation, offering advice and tips on how to actually implement the technique for your teams.

Let’s get started!

Automated Regression Testing: Let’s Come Up With a Definition

Before we dive into the “how” of automated regression testing, we need to clarify its “what” and “why.” What is it, after all? What does automated regression testing mean, and why should we use it?

This section will answer the questions above and more.

What Does “Regression” Mean in “Automated Regression Testing”?

The first step in order to understand “automated regression testing” is to get rid of the “automated” part, at least for now. First, we need to understand what “X” means, then “Automated X” will be self-evident. Or, at the very worst, easier to grok.

If we want to understand what “regression testing” means, the first step is to define “regression.” In this context, is “regression” good or bad? Should we welcome it or fear it? The answer to the questions above is straightforward. Here, regression means essentially the same thing it means in everyday conversation—to go back to a previous state. So, in the context of software development, we say we’ve got a regression when our application unintentionally reverted to a previous state.

Why Do We Need Regression Testing?

As we’ve just defined in the previous section, “regression” means an unintentional reversion to a previous state. However, something really important was left implicit in the definition: software regression, more often than not, means the application went back to a bad previous state.

Have you ever had the experience of making some existing feature stop working after implementing a new one? What about that nasty bug returning months after you thought you had completely eliminated it? These are examples of regression. The application has regressed to a previous inferior state.

Regression Testing Is Vital in Software

The field of software development is disproportionately prone to regression problems. Each addition or change made by any developer has the potential to cause unexpected problems in areas (supposedly) unrelated to the spot where it was performed. Any non-trivial software project, maintained by a team with more than, say, five people, has an incredibly high number of potential regressions during each release.

Developers should always keep in mind that all the changes they make, no matter how small, simple, or insignificant they seem, have the potential to cause surprising side effects. They can break functionalities that don’t have anything to do with the changes being made. By performing regression tests, the developer checks that not only does their change behave as it’s supposed to, but also that it plays well with all of the code that was written up until that point.

That’s where regression testing comes in handy. Regression testing is nothing more than the execution (partial or total) of a test suite in order to verify that a given application hasn’t returned to a previous undesired state. If manually done, though, regression testing can be extremely time-consuming and error-prone, which leads us to our next point.

Regression Testing: Why Automate It?

I often say that everything that’s automatable should be automated. Meaning, if you can automate a process, then you probably should do so. If you can automate a process but you’re still doing it manually, I’m afraid you’re leaving money on the table. Chances are the manual process is slow, time-consuming, tedious, and error-prone. That implies you’re losing money in at least three different ways.

For starters, you waste money by having well-paid professional performing tasks that could be automated. Then, there’s always the opportunity cost. The people who are performing the tests could be doing more valuable tasks. Such tasks could have the potential to generate way more value. Finally, since the manual process is error-prone, then people are bound to make mistakes, which will result in losses.

Automated Regression Testing: How to Put It Into Practice

Now that we’re done defining concepts, it’s time for some practical tips on how to actually implement regression testing. There’s good news. If you already write some kind of automated tests for your application—e.g. unit tests—then you’re already performing regression testing without even knowing about it.

Regression testing is not a “new” category of automated tests. It’s not an alternative to unit tests or integration tests. On the contrary, your automated tests —unit tests, integration tests, and similar—written from day one in the project can and should act as regression tests. After each new change to the codebase,  just re-run all of the relevant tests on the suite to ensure they’re not failing.

Automated Regression Testing Best Practices

Additionally, there are some best practices you should implement when adopting regression testing, regardless of the specifics of your implementation:

  • Adopt a Test Management Software: This is vital for all projects that are larger than a one-developer toy project. You’ll have, realistically, way more tests cases than you’d be able to track and manage individually. Luckily for you, there are lots of test management tools at your disposal.
  • Keep a Testing Schedule: You should maintain a strict testing schedule throughout the entire project. This will ensure the final project is thoroughly tested. Additionally, the schedule will encourage the team to adapt to a frequent testing regimen.
  • Write a New Failing Test for Every New Bug Found: Imagine your code has an unequivocal, reproducible bug. But all of your tests are passing. That means that either the current tests are wrong, or your test suite is lacking tests. If you find yourself in this scenario, write a new failing test to document the bug.
  • Categorize All of Your Tests:  You should split your test suite into smaller categories. Your test management tool will most likely provide you with the ability to categorize your tests. That way, your team members can easily identify each kind of test.

Automated Regression Testing: Put It to Work for Your Team ASAP

Software development is a creative endeavor like no other, but the path to reaping its rewards is full of risks. One of the dangers when writing code is to break existing functionality while adding new ones. Even when fixing bugs, we can make older ones—that we thought were dead—come back to life. Each small and seemingly insignificant change poses the risk of a dreaded regression.

If you want to get rid of regressions—and you should want to, believe me—then a suite of regression tests is the solution you seek.

Author bio: This post was written by Carlos Schults.Carlos is a .NET software developer with experience in both desktop and web development, and he’s now trying his hand at mobile. He has a passion for writing clean and concise code, and he’s interested in practices that help you improve app health, such as code review, automated testing, and continuous build.

In a world where technology moves at breakneck speed, most of us know the importance of keeping pace. That’s why traditional software development models are rapidly giving way to methodologies that are more dynamic.

DevOps is a partnership that emerged from this climate. The DevOps approach emphasizes communication, collaboration, and integration. And it does that by bringing your developers and your operations folks together.

But the term DevOps includes only development and operations in its name, and the reality is that we rely heavily on a third entity as well. To deliver software that’s up to today’s standards, you need to consider your testers an equal part of that equation. And so that’s what we’ll talk about today—why the term DevTestOps is a more fair and accurate representation of how companies deliver great software.

A Brief History

With traditional development models, software releases consisted of a series of handoffs. Specifically, developers coded to technical specifications and then passed their code to QA to test. Then, there was a final handoff to operations, who pushed the code into the production environment.

Now, you might notice that QA—the integral step between development and operations—isn’t reflected in the DevOps title. This is a problem. One of the primary components of DevOps is continuous delivery. And for continuous delivery to be successful, you need a substantial amount of testing.

Now, don’t get me wrong. When the tech industry shifted toward DevOps, it was a great thing. In fact, key findings from the 2016 State of DevOps report show the measurable benefits reported by organizations that have successfully incorporated DevOps:

  • Project teams deploy 200 times more frequently.
  • Teams spend 22% less time on unplanned work and fixes.
  • They also spend 29% more time on new features or code.
  • And finally, teams spend 50% less time on security issues.

The report says it best: “By better integrating information security objectives into daily work, teams achieve higher levels of IT performance and build more secure systems.”

Since many companies are eager to have results like these, we’re seeing an increase in DevOps adoption. After all, who doesn’t want the benefits of an accelerated development cycle? But for your software to be resilient, testers need to ensure that quality is built into the final product.

As you can see, DevOps, as it stands, is missing a key component.

Testers Pull Their Weight

If you’re not convinced that testers have earned their place among development and operations, consider this.

More data than ever is traversing the networks. According to DOMO, there’s an estimated 2.5 quintillion bytes of data created daily. And the world is releasing products at a pace that renders manual testing ineffective.

With all that’s going on, automation is essential. Testing such large amounts of data at fast-paced intervals necessitates it.

If you do smoke/regression testing, performance testing, and static tests, automation does the bulk of the work. That means more time for manual testing when there are subjective factors at play. We all know that sometimes you need human judgment.

However, this may prove to be a tall order. Over 70% of smaller companies have very little automation.

That means that the folks responsible for testing must deal with the challenge of test orchestration and automation on their own—either that or they must try to bear the entire burden with manual testing. Not only must QA teams ensure that code changes are fully functional and integrate well but they also must see that frequent releases don’t render the product unstable.

Yes, your developers and your operations folks are critical. But testers bear equal responsibility for the quality of the software you put out. With all these challenges, it would be hard to argue they don’t pull their weight just as much as the “dev” and “ops” in DevOps do.

Testers Help Break Down Silos, Too

Beyond logging defects, testers in environments work cross-functionally. Sound familiar? It’s what DevOps is all about.

Testers share the spirit of collaboration that defines the DevOps movement. They work to ensure that the entire system, not just an individual feature, is operating at an optimal level. From verifying app performance to validating output results from APIs, testing in continuous development environments covers a broad range of tasks.

And with the added responsibility comes greater visibility. Software testing is no longer limited to that of a supporting role. These folks are partners. QA and testing teams have come to the forefront as key players, in line with development and engineering. They’re breaking down silos as much as anyone else in a DevOps organization.

Based on this type of environment, you can see why the term DevTestOps is more fitting. The trifecta of development, testing, and operations are already working together as equals. In fact, just as the word DevTestOps blends three words into one, this approach blurs the lines between roles and responsibilities. And to a large extent, it also eliminates the need for those previously discussed handoffs—for developers to hand off to QA and then for QA to hand off to operations—in order to promote code into production. Instead, each discipline opens its communication channels and collaborates as one team towards a common goal.

One Final Reason to Put the Test in DevTestOps: Money

We’ve seen plenty of good reasons to call things like they are and adopt the term DevTestOps. The combination of development, testing, and operations brings about many benefits. But let’s get to the heart of why companies would want to make the shift in thinking.

The “test” in DevTestOps adds business value across the board, from rapid delivery to improved security. And herein lies an important point: revenue and profits rely on speedy development. So it’s imperative that you head off problems before they become major events. App crashes must be mitigated. Bugs need to be caught early. And we want the end-to-end user experience to be error-free.

QA is an essential part of making sure this is the case. You have to test in order to save yourself from costly problems like bugs. And the earlier in the development cycle you catch a bug, the cheaper it is to fix it.

So let’s give “test” the weight it deserves. Let’s move to the next step in DevOps evolution: DevTestOps. If we do that, it means we have our priorities in the right place, and we’re going to have more profitable businesses.

If we’re talking convincing reasons, this should be the most convincing one of all.

Moving Into the Future

Testers and QA teams champion their cause, influencing processes and decisions project-wide. A business’s concerns aren’t just matters for development and operations. As key players in continuous delivery, QA organizations have seized the opportunity to evolve, taking testing from error detection to error prevention. Now, it’s our turn to evolve from DevOps to DevTestOps organizations, acknowledging their equal importance in creating great software.

“Automated testing” used to be a phrase that caused developers to scratch their heads. Now, it seems to me that most developers are at least familiar with the notion of automated testing. Fortunately, many devs go beyond familiarity. They turn theoretic notion into practical action by actually employing automated testing on a day-to-day basis. And nowadays there’s immense variety in the types of automated tests that can help you achieve higher software quality. In today’s post, I want to focus on a very specific type of automated testing—UI testing.

We’ll start by briefly defining what UI testing is, and then explain how it can benefit you. Then we’ll proceed to show you, step-by-step, how to get started with UI testing using Testim.

Let’s get started.

UI Testing: What It Is and Why Do You Need It

As you surely know, UI stands for “user interface.” So, defining UI testing doesn’t sound that hard, at first sight—it’s just exercising the application through its user interface. But, is that all there is to it?

Actually, it’s not. First of all, some people seem to think that UI testing isn’t necessary if you already have other types of testing in place, such as unit tests. That couldn’t be farther from the truth. UI testing is not a replacement for other types of testing—instead, it’s a compliment. Unit testing exercises the application’s units in isolation. It’s an important verification to have, but it’s not enough. It’s also important to verify how those units fit together. You have to check how the whole system behaves, from the UI, passing through each layer and then back. So, UI testing can be thought of as more of an “end-to-end” type of testing.

Other people might think that, while UI testing has some importance, you can have one or more developers just click around and see if the application behaves correctly. But that isn’t true. Manual UI testing might work for very small and simple applications. But when it comes to applications with complex business rules and rich user interfaces, manually testing quickly becomes insufficient. It is a tedious, error-prone, expensive, and hard-to-scale process.

At this point, automated UI testing is the best choice.

UI Testing Guide: How To Get Started

It’s time to get your hands dirty. We’re going to offer you a brief guide on how to get started with UI testing. It’ll by no means be an exhaustive tutorial, but it should teach you to create a basic approach upon which you can build and evolve into a more sophisticated approach later.

For this guide, we’re going to use Testim to test a very simple web app. Let’s get started.

An Example of Manual UI Testing

Before we start creating our tests, we need something to test, right? So, let’s create a toy application. It’s going to be a super simple web app, with just two controls—an input box and a button. It will allow the user to input numbers, separated by a comma. After clicking on the button, the application will add all the numbers and display the results. If the user types just one number, the result should be that number. If the input box is left empty, the result should be zero. Simple enough, right?

Creating a Toy Application

Ok, let’s get to it. This application will consist of a single HTML page with a little bit of Javascript. Nothing too fancy. Open up your favorite text editor, create a new file, and add the following content to it:

<!DOCTYPE html>

<html>

<head>

<title>String Calculator</title>

<style type=”text/css”>

label { display:block; margin: 10px;}

</style>

</head>

<body>

<h1>String Calculator</h1>

<form id=”form1″>

<label for=”numbers”>Numbers: <input type=”text” id=”numbers” name=”numbers”/></label>

<label for=”addButton”><input type=”submit” value=”Add” id=”addButton”/>

</form>

<script type=”text/javascript”>

document.getElementById(“addButton”).addEventListener(‘click’, function(){

    var text = document.getElementById(“numbers”).value.trim();

    if (!text) {

        text = ‘0’;

    }    

    var integers = text

        .split(“,”)

        .map(x => parseInt(x));

    var result = integers.reduce(((a, b) => a + b), 0);

    alert(result);

});

</script>

</body>

</html>

After that, save the file with the “.html” extension. Then, double-click the newly saved file. You should see your default browser showing you something like this:

Performing Manual UI Testing

Now let’s do some (manual) UI testing. Click on the input box, type 0 and click on the Add button. You should see this:

Now, redo the test, but leave the input box empty. The result should also be zero:

Repeat the test a few more times, always using a single number. The result should always be the number itself. Then, perform some tests with more than one number, separated by a comma. The result should be the sum of the numbers, as the following example shows:

After this brief session of manual UI testing, I’d be convinced that our little toy app works as intended. But, is that testing enough? For this example application, I’d say yeah, probably. But keep in mind that this silly string calculator is just a placeholder for a much more complex application, closer to the ones you’re likely working on in real life.

Getting Started With Testim

Now it’s time to show you how to get started with Testim so you can see what automated UI testing looks like.

First Steps: Creating an Account and Installing the Testim Browser Extension

First, go to https://testim.io and  click on “Start Trial.” You’ll be asked to sign up for an account. You can do that using your Google or GitHub account. If neither option is OK for you, sign up with an e-mail address and create a password.

After successfully logging in, you’ll see a screen like the following image:

If you click on “Create test” button, you’ll see a message asking you to install the Testim Chrome extension:

After downloading and installing the extension, you’ll be ready to start creating automated tests. This time, though, instead of testing our silly little string calculator, we’ll test a real one—the Google calculator.

Creating Your First Test With Testim

On Chrome, go to the address bar, type “calculator” and hit enter. You should see this:

Just a good old calculator. Now, we’re going to create a test for the “+” button.  While still on the same tab, click on the Testim extension’s icon, as shown in the following image:

After clicking on the extension icon, you should see the following menu:

Click on “CREATE AUTOMATED TEST.” That will open Testim and attach the current tab to it. Then, you’ll return to the calculator tab and see the following message:

What you have to do now is very simple—just use the application and your actions will be recorded. Let’s create a test for the “2 + 2” sum. Perform the sum as you would normally do it. After clicking on the “equals to” button, go back to the tab where Testim is open. You should now see this:

Click on the “SAVE” button. You’ll then see a window asking you for a name and a description for your test:

For the name, enter “Add test”. On “Description”, type “Verify the Add operation” Then click on “OK.” Congratulations, you’ve just created your first test with Testim!

Running Your Test

You’ve just created your first automated UI test using Testim. That’s nice, but how do you actually run the test?

On the left panel, click on the “Automate” button, like in the following image:

After doing so, you’ll see your test list which contains only the test you’ve just created:

To run the test, just mark its checkbox and click on the “play” button:

After clicking on the “Run Selected Tests” button, you’ll see a message warning you to avoid touching the mouse and the keyboard during the test execution:

After clicking on “OK”,  Testim will open up a new Chrome tab and run the test. Testim will execute the exact same steps you’ve recorded. After Testim finishes running the test, it’ll close the window.  Go back to the Testim tab and see the test result:

You’ve successfully run your first test using Testim. Congratulations! But…isn’t there something missing?

Adding a Verification

You’ve probably noticed that the test you’ve just created doesn’t really test anything. Yeah, the test passes but that doesn’t really mean much. In terms of unit testing, it’d be the same as having a test method with no assertions.

Our test lacks something, and that something is a verification. We need our test to check that the result of “2 + 2” is indeed four. How do we do that?

On the left panel, click on the “Automate” button. You’ll go back to the test listing. By clicking on the test name, you’ll reopen the test editor so you can change it:

You can see that the test has six steps, and the last step is clicking on the “=” sign. What we have to do now is add an extra step, whose goal is to perform the validation. To add a new step, click on the “+” button after the sixth step. You’ll see the following options:

Click on the third option and you’ll see the menu change to this:

Now, click on “Validate element text.” Testim will take you back to the calculator tab. There, click on the result field:

After doing that, go back to Testim. Now, you should see that your test has a new step:

Hover over the newly added step and click on the engine icon to show the properties panel:

Now it’s just a matter of editing the “Expected value” field, setting it to ‘4’. After doing that, click on the “SAVE” button again. You’ll see a box prompting you for a message, explaining your change. Type “Add validation step” and click on OK:

If you run the test again after saving it, it will continue to pass. However, Testim is now comparing the expected result to the actual result, just like a unit test assertion does.

UI Testing Goes Way Beyond “Clicking Around”

There are still some misconceptions about UI testing out there. One of them is that UI testing isn’t needed if you already employ different categories of testing. Another myth is the belief that informal manual UI testing—a.k.a. just clicking around—is enough.

Neither of those beliefs is true.

We hope this post has shown you not only that UI testing is indeed necessary, but also that it isn’t hard getting started with it, especially if you employ a tool like Testim.

Thanks for reading, and see you next time!

Author bio: This post was written by Carlos Schults. Carlos is a .NET software developer with experience in both desktop and web development, and he’s now trying his hand at mobile. He has a passion for writing clean and concise code, and he’s interested in practices that help you improve app health, such as code review, automated testing, and continuous build.

Quite often there is a need to validate the content of a downloaded document which could be a PDF, Word Document, CSV file or databases such as MySQL, MongoDB. All these are possible within Testim using the CLI action step. This feature helps to create and execute custom Node.js scripts at runtime; to perform different validations. Below are some tips to help with CLI action steps.

Tip 1: How to find out what node.js packages you can use?

The different node.js packages that can be used within Testim are the ones found in https://www.npmjs.com/. This is the official website that has references to all the open source node.js packages. Say for example, you are searching for a package related to pdf documents; just search for “pdf” and you will get a list of all the node.js packages available for pdf.

Tip 2:  What are the available CLI action steps within Testim?

Below are the available CLI action steps within Testim-

  • Add CLI action
  • Add CLI validation
  • Add CLI wait for
  • Validate download
  • Wait for download

Tip 3:  How to add node.js packages to Testim?

To add node.js packages to Testim, follow the below steps

  • Add one of the above mentioned CLI action steps
  • In the Properties Panel, click on PARAMS-> PACKAGE

Tip 4: How to execute CLI action steps locally?

To execute node.js scripts via CLI action steps locally, you need to first set up the runtime environment using the below command. Ensure this command is executed from the command line before running a test with a CLI action step.

npm i -g @testim/testim-cli && testim –agent

Putting everything together

Lets see how all the tips work together to execute node.js scripts. Say you want to validate the content of a downloaded document such as a PDF file. The steps to do this would be-

  • Ensure CLI Action feature is enabled (Contact our CSM or customer support for this)
  • Add the “Validate Download” action
  • Add the necessary node.js package for the pdf content validation. In this case, we are adding the latest version of the package pdf-parse
  • Then, add the necessary code to do the content validation

Here is a sample test to show how to do PDF content validation – PDF Validation

For more CLI action examples, check out the last section of this doc – CLI Action

Today’s post is going to be all about automated functional testing. What’s this category of tests really about? Why do you need it? How do you go about implementing it? What are the challenges you might face when doing so? With this post, we’re going to answer these questions—and more.

We’ll start out by defining functional testing. Then, we’ll go on to explain the benefits of functional testing so you understand why you need it, even if you already employ other types of tests. After that, we’ll reach the “automated” part of “automated functional testing.” We’ll define automation, explaining how it’s imperative to apply it to testing. Then, we’ll be ready to cover automated functional testing.

With the definitions out of the way, it’s time to get to more specific advice. You’ll see how to get started with automated functional testing, the possible challenges on your journey, and how you can beat them.

Functional Testing: What It Is and Why You Need It

Functional testing, as its name implies, concerns itself with how a given application functions. Does the application meet its functional requirements? Does it solve the problems of its users? These are the kind of questions that should be answered by functional test cases.

Functional testing is, then, a type of black-box testing. In other words, functional testing couldn’t care less whether you’ve used an array or a list to model a given concept. The only thing that matters is that the application’s external behavior meets the requirements.

Unit Tests Are Not Enough

In some software development cycles, there are people who seem to think of unit tests as the ultimate software panacea, the mythical silver bullet that will solve all software crises for good.

There’s no such thing, of course.

Make no mistake: unit tests are awesome. They can either lead to a cleaner, low-coupled design or—in the worst scenario—make you acutely aware of poorly designed, high-coupled code. They often cause developers to learn about good development practices, such as immutability, purity of functions, and separation between logic and presentation. More importantly than all of that, a comprehensive suite of unit tests empower developers to fearlessly refactor, knowing they have a safety net if they break something.

Did you notice who is missing from the previous paragraph? The user.

Let’s Think of the Final Users

The previous section covered some of the benefits of unit testing. You’ll notice, though, that those benefits seem to benefit mainly the developers themselves. Sure, cleaner code and a more maintainable application will indirectly benefit users, no doubt about that. But that’s not enough.

There’s a school of thought that argues that “unit testing” is an unfortunate name for an otherwise great tool, and I agree with that. I’ve seen people use “executable specifications” as an alternative name. The point is this: unit tests aren’t really tests. They’re verifications that developers use in order to check that the code they wrote does what they think it does.

But who’s checking whether the code they wrote does what the user expects it to do? That’s the job of functional testing.

It is possible to write a correct, elegant and efficient solution…to the wrong problem. While unit tests will make you confident you’re solving the problem correctly, functional testing is what ensures that you solve the right problem, for the right users.

Automated Functional Testing: Understanding Its Benefits

Up until now, you’ve seen what functional testing is and why you need it, even if you already use other, also very useful, types of tests. Now it’s finally time to add “automation” into the mix. We’re going to define automation and show why its use is imperative in testing. We’ll then zoom in and talk specifically about automating functional tests.

Automation Is Great. Are We Using It?

As developers—and software development professionals in general—we can consider that automation is the whole raison d’être of our jobs.  Why shouldn’t we use it to improve the efficiency of the software development process itself?

Sure, programmers have been automating aspects of their job since the dawn of software engineering. But in recent years, the industry’s use of automation increased to unprecedented scales. We can see this reflected in the growing use of automated testing and, more generally, in the DevOps movement.

Automation in Testing Is a Must

If there’s one area in the whole software development process that really benefits from automation, that area is testing. That’s due to one of the most unique characteristics of software—even the smallest change in one area of an application can cause unforeseen consequences in a totally different area. Yep, that’s a serious problem. In an ideal application, with sound architecture, something like that should never happen. We don’t live in the ideal world, unfortunately, and things like that do happen frequently.

The implication of this unique software characteristic is crucial. After making some change on a codebase, the only way to be sure the whole application still works properly is to retest the entire app. Just testing your own change isn’t enough. You could’ve broken something and you’d never know.

That means that, in modern software development, applying automation to testing is imperative.

Automated Functional Testing: You Can Have Your Cake and Eat It Too

Automated functional testing is, in a way, the best of both worlds—it brings to the table the benefits that automation provides to all kinds of tests, along with the functional testing ethos of putting the user’s needs first. It takes automated testing to its full potential, making it not only a tool for developer confidence (which is already awesome in itself), but also a tool for customer satisfaction. It bridges the gap between QA professionals and developers, creating a shared space where they can collaborate.

Automated functional testing sounds awesome—and it is—but how do you actually get started with it? I’m glad you asked.

The first step is to have specs. You can’t have tests that verify whether the application conforms to the specifications if there aren’t any. For this first moment, it shouldn’t matter that much whether the specifications are embedded into the code or recorded in an external document. The important thing is that you have them.

Then, it’s time to start thinking about the possible test cases for your application. Start your app, go to its main screen, and look at it critically. What are the tasks the user is supposed to perform here? How could they go wrong? It’s even better if you don’t do this on your own. Instead, bring in some coworkers to help. Do a brainstorm session on the current screen. Write down all of the ideas that you can come up with.

Then do the same with other areas of the application.

After all of this, do you have a reasonably sized list of possible use cases? Great, now it’s time to go shopping for tools. This post is meant to be platform agnostic, so we won’t actually recommend particular testing frameworks. But it doesn’t matter your language or tech stack of choice, rest assured, there are automated functional testing tools out there for you.

Automated Functional Testing Challenges: What They Are and How To Overcome Them

As promised, now it’s finally time to cover some of the common challenges when implementing automated functional testing. What are those? How do you beat them? Keep reading to learn more.

QA Skepticism

One of the most common challenges when implementing all kinds of automated testing is skepticism. It’s 2019 and there are still many professionals that are unfamiliar with automated testing, believe it or not. There are also others who know about automated testing but don’t quite buy into the usefulness. How do you deal with the skeptics? Education.

You must use every kind of material at your disposal—books, online courses, on-site training, and what have you. After some training, make your QA people get their hands dirty. Have them start creating automated functional tests, first on a very small scale. Then, start to gradually increase said scale. In true Agile fashion, use small steps, reviewing your results in short iterations. Rinse and repeat. That’s pretty much it.

Developers Skepticism

When it comes to developers, it’s possible that some of them have jumped on the “unit testing = software panacea” bandwagon. If that’s the case, make them get off the bandwagon. The tool for doing so? You guessed it, education. Hands-on education.

Have your developers collaborate with QA people. Create small, cross-functional teams on which QA and developers can work side by side. As automation experts, developers might be of incredible help, mentoring QA people on the nuances of automated testing frameworks and tools.

This approach will create trust and empathy between developers and QA professionals, bridging an otherwise insurmountable gap, and destroying every trace of skepticism toward automated functional testing on both sides.

Maintenance Burden

Another challenge of automated functional testing—and other types of automated testing—is the maintenance burden. You see, test code is code, after all. It can contain bugs. It might be unreadable. And the list goes on.

How do you overcome this challenge? First of all, education again. Educate yourself and your coworkers on the best practices for writing tests, be them functional or otherwise. Read about ways to keep your test code clean and readable. Learn about the common mistakes people make when writing tests and what you should do to avoid them.

Finally, there are tools that can ease your burden by providing you with test management capabilities that you would otherwise not have, such as comparing versions of the same test, scheduling tests executions, and automating bug reporting.

Automated Functional Testing Helps You Solve the Right Problems at the Right Speed

Software development is a field that evolves at breakneck speed. In order to keep up with such rapid change, companies rely more and more on automation, giving birth to approaches such as DevOps and continuous delivery.

Automated testing plays a crucial role in this scenario. It’s unfeasible for teams to completely retest their application by hand each time someone has made a change. Automated testing is way past the point of being something that’s “nice to have”—software development in the 21st century requires it.

Automated functional testing, in particular, brings the benefits of automation to a type of test that puts the user front and center. It bridges the gap between QA and developers, ensures the application meets the user’s needs, and takes automated testing to its full potential.

Author bio: This post was written by Carlos Schults.Carlos is a .NET software developer with experience in both desktop and web development, and he’s now trying his hand at mobile. He has a passion for writing clean and concise code, and he’s interested in practices that help you improve app health, such as code review, automated testing, and continuous build.

Software Test Automation has been there in one form or another for decades. Organizations had mixed successes with Test Automation even though few have been immensely successful in automating tests as part of their application development process. The benefits to these organizations have been huge in terms of costs, time to market and quality. 

Why is it that many teams are still continuing with their manual tests or struggling as they embark on Test Automation? Can every manual test be automated? How to get started with a Test Automation? What are some of the strategies and best practices? These were some of the questions that were answered in this webinar. We also covered how AI/ML has been helping in implementing autonomous testing.

Here is the recording of the webinar-

Below you will find answers to the questions asked during the webinar-

We always find it difficult to find the right test data and many times we had to cancel the testing. this is because of many factors such as, no access to prod data, not able to mock the data, not able to scrub..etc. How do you handle such situations or can automation play any role in such cases?

Test Data is key in evaluating systems and ensuring they are working as expected. So setting up and maintaining test data is critical for testing. There are many ways this can be done-

  • Have separate QA servers and ensure all the test data needed for testing is set up here. We need to ensure the test data resembles data in production as much as possible, to mimic true behavior while testing. This QA server should not be touched by anyone except for testers. Procedures need to be put in place such that only the admin can change the test data to ensure effective testing
  • Have separate config files or databases that houses all the data. This makes editing and maintaining test data really easy compared to having it stored in different locations of the system
  • Use automation to test different modules with different data sets. You can have one test that runs through different data sets. This will be otherwise hard to do manually
  • If possible mock API requests for getting faster feedback instead of waiting for API’s to be implemented
  • If there are situations where there are no test data available, take the help of developers to come up with a plan to have mock test data or test these modules via developer testing from the code

I want to know more about handling changing requirements in an agile world?

Scope creep in requirements is one of the biggest problems in software development. There are various ways to prevent scope creep and ensure features are developed and tested on time. Here is an article which describes how do this – https://www.infoq.com/articles/quick-guide-atdd/

What is price of Testim and does the tool have manual or tutorials?

Our pricing is based on the number of test runs. It varies based on the number of browsers, grid set up needed, whether you are choosing the cloud, tunneling or On prem solution and a number of other factors. So, it is best to contact us here -> https://www.testim.io/contact-us or get in touch with our 24/7 customer support for more accurate pricing information.

We created a robust help documentation repository that has answers to the majority of questions people may have, when they initially start using Testim. Check out the help docs here – https://docs.testim.io/.

Also, the below blog posts describes how we differentiate ourselves from other tools-

Are there any open source tools for AI/ML automation that is well known?

Currently there is none. But, I won’t be surprised if something comes up in the upcoming months/years.

I am new to automation and still learning can you recommend where to find more information about Design Patterns and Principles Please?

When it comes to automation design principles, we need to ensure the tests are-

  • Reusable
  • Have single responsibility
  • Maintainable
  • Extensible

The different strategies that go along with this are Data Driven Testing, Parameterization and following Page object pattern to structure your automation code.

If you google for any one of these keywords you will find more details about them. There are numerous resources on the web.

Assuming JIRA is used, how do you link test cases to requirements? Assuming Stories are used and not typical requirements?

There are numerous ways JIRA can be used along with stories. It depends on the context. So, it is hard to give one single answer to this question. Based on my experience, here are some possible options-

  • Have the requirements in Wiki Pages. Then add the link to the requirement in the story to show the connection between the two
  • Have the requirements in JIRA. Then have an Epic story that relates to this requirement. Then, have smaller stories linked to the epic story focusing on different parts of the requirement
  • You could use different tools like Test Rail, Zephyr etc to help in the process

Do we assure to automate 100% of the functionality of some app? I always knew any app can’t reach 100% of automation coverage, however, the CEO of a company is very confident that automation would solve all problems, finding bugs and going forward with all functionality?

Saying you can automate 100% of the functionality is like saying you know everything about computers. This is impossible. How can we know every user action a user would perform on our application and automate those scenarios before hand? There are always going to certain functionalities or user flows we did not think about and probably has not been automated.

The above being said, it is important to understand the scenarios automated testing could be a good option. Here are some use cases for it-

  • To automate mundane/repeatable tasks that are really time consuming to do manually
  • To operate in a DevOps environment where we need automation right from the start of the SDLC; till the release and production monitoring phase
  • To get quick feedback about the system when new code is checked in (new feature is implemented). We will have automated tests that get automatically triggered for every code check in
  • To run several tests on a daily basis to ensure the older functionalities of the system are still working as expected
  • To create Test Data that will be used for doing manual exploratory testing which is otherwise time consuming to do manually
  • To test different fields with hundreds of data sets using data driven testing
  • To do load/stress testing where we simulate thousands of users using the application simultaneously which would otherwise be hard to do manually

Automation is not a one size fits all solution. It is not a silver bullet that can solve all the testing problems. It is just an aid in the overall testing process. This being said, it is important to remember that, not everything can be automated and in trying to do so, we will move the focus away from actual testing and finding bugs.

There are many scenarios that cannot be automated or if automated may be unstable and provide less value. Some of them are-

  • Using automation for catching rendering issues in the application (look and feel) is a bad idea. There are few tools out there which does visual validation but it is really difficult to replace humans in this aspect. For example – I had a scenario where my mobile web page looked white on one Mobile phone, whereas it looked dark grey in another mobile phone. Yes, we can try to automate this but I think humans would be better at finding these subtle differences in look and feel of the application
  • Using automation to figure out an element location on a page is a bad idea. The automated tests will become unstable if we start writing tests based on x,y coordinates of elements, as the web page could be viewed in different browsers, devices and OS’s and the coordinates are going to change based on the screen size.This means the automated tests are going to be really flaky.
  • Using automation to test integrated systems which involve software, hardware, web services, API’s and cloud services all communicating in real time with each other would be a bad idea. For example – How would we write an automated test which tests all the end to end scenarios of fitness trackers like Fitbit? We can try as hard as possible to simulate real human movements and mock services, but it is going to be a really difficult task to automate the entire process of a fitness tracker. We could rather have real humans do exploratory testing in parallel to some automated tests

Can you suggest me any best tool can be used for Multimedia device (multimedia device in Car) testing?

It is hard to answer this question without knowing the context of what you are trying to test in the multimedia device. Are you testing the hardware or software portion of it? Is it possible to use simulators do that or everything has to be done on a physical device? 

Can we use data driven testing using testim for local run?

You can set up your test data with different data sets and can ensure the test is able to interact with these data sets by running locally. But, when run locally, the test will go through only the first data set in the list. If you want to run through all the data sets, then you would run it on the grids using our CLI. The reason we designed it this way was, people were trying to run hundreds of datasets locally and it was consuming time and space of the local machine. So, to prevent this from happening, we encourage users to run the tests with datasets on the cloud as it is much faster.

Can testim locate canvas object?

Currently we do not support canvas testing but it is something you can expect Testim to support in the future.

Start your Free Trial today.

When we talk about automation applied to the testing field, there are two expressions you’ll often hear thrown around: “automated testing” and “test automation.” But wait a minute. Aren’t automated testing and test automation the same thing? As it turns out, no. They are related concepts, but each one has a very specific meaning and purpose. And make no mistake: in order for your software organization to thrive, you’ll need both.

If up until now you thought that these two concepts are the same, don’t feel bad. For starters, you’re definitely not alone. People often mistake the two terms for one another. And secondly, your confusion ends today. That’s what this post is all about, after all.

We’ll begin by defining the two terms. Then we’ll dive a little deeper into each one of them, explaining their importance in an organization, also showing how they differently affect each role inside the team. Finally, we part ways by giving some additional tips. Let’s get started!

Defining “Automated Testing”

Right at the start of this post, we claim that “automated testing” and “test automation” aren’t the same thing, even if their names seem to imply otherwise. We’ve also told you that both are critical for your overall testing strategy. Finally, after a brief detour on the importance of testing in general, it’s time to define both approaches, clarifying how they are different—and similar—to one another.

Let’s start out by defining automated testing. And to be fair, that couldn’t be easier. “Automated testing” is just the process of automatically running a specific set of tests and verifying their results, instead of doing it manually. When you run your suite of unit tests on your development machine, you’re performing automated testing. When the CI server runs all of your unit and integration tests when someone pushes new commits to it, it’s also performing automated testing.

That’s pretty much all there is to it. With that out of the way, let’s continue to “test automation”, where things will probably be a little bit trickier.

Defining “Test Automation”

Now it’s time to define “test automation.” What is it and how it differs from the definition you’ve just read in the previous section? While automated testing is basically the running of automated tests, “test automation” is a way broader concept. It refers to completely automating the whole process of managing the different testing needs inside an organization.

Out of context, the line above most likely doesn’t explain much. Things will get a lot easier in the following sections, in which we’ll contextualize the two approaches, explaining how they fit into the overall quality strategy of a software organization and who each one benefits the most.

Automated Testing: Who Does It Help?

To really draw the distinction between automated testing and test automation, it’s important to show how they affect the different roles inside a software organization. Let’s start with automated testing. Which role in an organization benefits the most from this approach? The way I see it, the answer is clear: developers. And why is that?

A Unit Testing Example

For the sake of the argument, let’s focus on unit testing for a while. “Unit tests” is not a really great name if you think about it. No one seems to agree on what “unit” really means, for starters. But the main problem is that they aren’t really “tests.” The best way to think about unit tests is to consider them executable specifications. The term “programmer tests” has also been proposed a few times, and it makes sense. Unit tests are written by programmers, for programmers, in order to “prove”, at least to some degree of confidence, that a given piece of code really does what it’s supposed to do.

Automated Testing Is a Developer Confidence Booster

It isn’t that much of a stretch to expand our definition of “executable specifications” to also include integration tests. A common definition of integration tests would be “unit tests, kind of, but they use the real database/filesystem/etc instead of mocking/stubbing it.” Let’s do just that, then: let’s stretch our definition of “executable specifications” in order to also include integration tests. And let’s go further and pull all of that under the “automated testing” umbrella. What do we get from all of that defining and stretching?

The answer seems clear to me: the main beneficiary of “automated testing” in a software organization is the developer. Or, in other words: automated testing is a tool to achieve developer sanity. It allows developers to fearlessly refactor, knowing that they have a reliable safety net in the form of a comprehensive test suite that will warn them if they break something.

Test Automation: Who Does It Help?

We’ll now turn our focus to test automation and repeat the same analysis we just did with automated testing. Who benefits the most from test automation? The answer, again, is clear: the organization as a whole, with a special focus on the DevOps professionals. To understand why, we have to go back in time, to the (not so) good old days of the waterfall methodology.

Once Upon A Time, During The Dark Ages

In organizations that employ this methodology (and similar ones), testing is nothing but another phase in the whole cycle. And the testing phase tends to happen at the end of the cycle. More often than not, this is too little, too late. The feedback cycles slow down, reducing the usefulness and efficiency of the whole testing strategy. Nowadays, companies are moving toward a world of continuous development, deployment. They can’t afford to wait forever for the feedback from testing. The testing itself has to be continuous so that quality can be ensured at all phases of development. Automation can come in handy in this situation.

Here Comes The Enlightenment (i.e. Continuous Testing)

Automation is the missing link. It can provide the speed and consistency that you need in order to have continuous testing up-and-running. Manually managing the testing needs in such an environment would not only be incredibly hard and error-prone but also time-consuming, which in fact defeats the whole point of doing it in the first place.

By automating the managing of said testing needs, test automation helps organizations keep quality at the highest standards during the whole development cycle. Furthermore, it frees QA professionals from managing the minutia of testing needs, which means they can focus on creating more efficient case tests to ensure the application’s quality.

Automated Testing And Test Automation. You Need Both. ASAP

In today’s post, we’ve clarified the confusion between “Automated Testing” and “Test Automation”. Yep, the naming could use a little less ambiguity. But the concepts themselves are really different. However, they’re equally important if you want your organization to continue delivering software to the highest standards of quality. Automated testing is important, especially for giving your developers the confidence to fearlessly refactor their code,  which contributes to higher quality code, without a doubt.

But test automation becomes critical when your company starts moving towards a continuous development/continuous deployment scenario. Do you want your company to continue thriving in this brave new DevOps world? Put test automation to work for you and your organization.

This post was written by Carlos Schults.Carlos is a .NET software developer with experience in both desktop and web development, and he’s now trying his hand at mobile. He has a passion for writing clean and concise code, and he’s interested in practices that help you improve app health, such as code review, automated testing, and continuous build.

So your company is going Agile. If you’re a tester or QA person, that can bring about frightening thoughts about what your world will look like. For example, will you now be expected to test a codebase that changes daily, with no breaks or developer deadlines? Alternatively, you’ve heard that some teams do away with QA and testers completely once they go agile. How will you survive in this scenario?

Fear not, for there’s still a need for QA in our new agile world. However, instead of the old paradigm, where your goal involves finding defects, your new goals include preventing defects and helping ship products faster. And with this guide, you’ll be on your way to creating a robust and healthy QA process.

In this post, we’re going to take a look at what going agile means from a QA perspective and how you can change to thrive in this environment.

What Does Going Agile Mean for QA?

When reviewing the values and principles of agile, we don’t see testing addressed as clearly as we’d like. In fact, we’re not even sure if traditional QA has a place. But we need to read between the lines and picture what this means for us. Here are the four values that came from the Agile Manifesto:

  • Working software over comprehensive documentation
  • Customer collaboration over contract negotiation
  • Responding to change over following a plan
  • Individuals and interactions over processes and tools

Now let’s look at how these values fit together with QA.

Working Software Over Comprehensive Documentation

For the first value, we learn that working software comes before comprehensive documentation. Now, this doesn’t mean you can’t have any documentation, no matter what your development team tells you.

It means you should create documentation only if it provides value. In the testing world, we should spend less time writing out manual test plan documentation and more time automating the tests.

And we do have some things that will always need documentation. For example, putting together documentation for the customer can be a big part of QA’s job, as they’re most familiar with how to use the system.

As another example, you’ll need to document how to recreate a defect within the application. But don’t spend too much time filling out forms, tickets, or any other unnecessary work. Look for streamlined ways you can clearly communicate the defect to the development team without much overhead.

Whatever you do, make sure it drives the result of having working software.

Customer Collaboration Over Contract Negotiation

In the previous section, we mentioned putting together documentation for customers. That should be one of the last steps. Before the documentation, we should be working closely with our customers to understand how they use the system and why they might run into defects or bugs. Learn what their pain points are so you can share what you’ve learned with the development team.

Overall, your relationship with the customer should look like a partnership, not a criminal case.

Responding to Change Over Following a Plan

One big concern testers have when it comes to delivering software frequently involves keeping up with the testing. Everything changes in agile, and fairly quickly. Stop trying to build a plan for a future that might not come true. Instead, work to put in automated tests and guardrails so that you’re able to react quickly to changing priorities and functionality.

Sprinkle in as much automation as you can so that when change happens, you’re ready.

Individuals and Interactions Over Process and Tools

In our old waterfall world, the QA team would get a few weeks prior to release to learn and test all the new functionality. And, as waterfall usually went, those few weeks would shrink from four to three to two weeks max as software development deadlines slipped. We still needed to ship our product, and the QA team would feel much of this pain.

In this new agile paradigm, QA should be involved throughout the whole process. In fact, testers should interact daily with the team. Whether it’s pairing on a task or discussing how a story should be tested, they’re integral in discussions around testing and quality.

Another important note: agile teams have frequent ad hoc design discussions, so testers should be co-located and available to jump in on those discussions whenever they happen. So don’t create more paperwork and process to share ideas—instead, involve yourself in the whole software delivery process.

What Should QA Do Now?

Now that we’ve covered the agile values, we still might have questions as to how we’re supposed to work when agile. In this section, let’s cover some of the ways that our QA skills can help the team deliver quality software in a continuous way.

From a day-to-day standpoint, involve yourself in stand-ups, retros, and demos. Stay on top of the work the team is doing so you’re aware of dependencies between stories and what additional testing will be necessary.

What else should you do?

Join the Team

The biggest change involves not having a separate QA team. Instead, QA should be part of the development team. Since we’re part of the team, we’re able to assist in continuous testing, instead of doing it all at the end of the sprint.

Also, when we’re on the team from start to finish, we have the chance to affect the delivery of the product much more. Why is that? In an agile team, roles aren’t as clearly defined. You may be able to write your own story by stepping in as a testing coach or automation SME.

When the team focuses on delivering working software faster to the customer, you’ll have the opportunity to provide flexible support to your teammates.

Understand the Big Picture

While the developers dive into specific details of the task they’re working on, you have the responsibility of understanding the big picture. You’ll need deep knowledge of the working system to define tests and scenarios that might not be apparent from the outside. And with that whole-system view, you’ll be able to write the documentation that’s necessary for your customers.

Additionally, keep in mind application design. Understanding your product’s design can help identify interesting edge cases that should be considered.

Understanding the big picture can also help with a blind spot the engineers might miss. The big picture will help you uncover assumptions the team has made about the system. And those assumptions could leave a gaping quality hole in your end product. Therefore, you’ll always want to have a keen understanding of the system as a whole.

Automate Your Test Suite

Since agile testing is done in parallel to development, automation becomes critical. Without it, you’ll find yourself retesting the same functionality three or more times in a sprint or iteration.

Your developers will write unit and integration tests. However, they write those tests to a spec and typically can’t cover the use case from end to end. This is where you come in as a QA professional.

Now, you may think that you’ll need to learn to code to automate all these tests. Learning to code—or at least learning some light scripting—definitely has its benefits. But also take advantage of tools like Testim to easily put together automated functional tests for your product.

These automated tests can be based on the acceptance criteria in developer stories. Create tests that ensure that your product works correctly. And look for where there may be gaps in understanding, and therefore a need for additional tests.

Test Manually for the Right Reasons

Exploratory testing can identify gaps in automated tests. This shouldn’t involve going through the same manual processes over and over.

Instead, this time is for asking those “what if?” questions.

What if the order gets canceled after the shipping paperwork gets printed? What if I lose my internet connection while sending the payment?

Put time into exploratory testing to give your team more confidence that they didn’t miss a critical bug or loss of functionality.

Enable Faster Delivery

As mentioned before, because it’s so critical, we need to enable faster delivery of software. As the QA person on the team, this can mean a lot of different things. For example, help your team define what a completed story looks like. Assist with story creation. Help the team identify what’s missing or out of scope for a particular story. Guide them to ensure that you deliver your product with quality.

But most importantly, don’t be a bottleneck or a block. Enable the team to go faster.

Share Testing Practices

As QA, you have an understanding of good testing practices. Help the team by sharing your experience. Consider how a feature would be easier to test. Perhaps look for ways to make the design more testable.

Continuously Improve

Though quality is the whole team’s job, you should be driving continuous improvement of testing practices. Work to become an expert on agile testing methodologies and strategies. Help the devs create integration tests that aren’t flaky but ensure that the system is working. Help build maintainable and healthy test suites.

Summary

In an agile team, everyone is responsible for quality. And this can be your time to shine by sharing your expertise with the team. Work with your team to build quality into the development process. As I mentioned earlier, the goal no longer involves finding bugs and defects, but preventing them.

Find ways to use agile practices and values to make that happen in collaboration with the team and the customers. You’ll be able to find comfort in your new agile workplace without fear or apprehension.

Author bio: This post was written by Sylvia Fronczak. Sylvia is a software developer that has worked in various industries with various software methodologies. She’s currently focused on design practices that the whole team can own, understand, and evolve over time.

Be smart & save time...
Simply automate.

Get Started Free