Category

Product

Category

Introduction

We work hard to improve the functionality and usability of our autonomous testing platform to support your software quality initiatives. This month we’re thrilled to release a few of your most requested features; Extract Value Step enhancements, Validate Checkbox/Radio button, Failed Test Retries option in Scheduler. Check them out and let us know what you think. 

Extract Value Step Enhancements

What is it?

As part of the extract value step, you now have the ability to extract specific values from an element on the page using the new extract mode field. This allows to extract the entire string, just the numbers, date or use regular expressions.

NOTE:  You also have the flexibility to extract the first, last or all the numbers from the element on the page when using “Number” option.

Why should I care? 

There is no longer a need to add custom code to get the specific value from the element on the page.  For example – You can now use the “Number” option and extract just the price (numeric value) from the text “$1980.45”.

Validate Checkbox/Radio Button

What is it?

You now have a built in step to validate checkbox and radio buttons on the page.

Why should I care? 

You no longer need to add custom code to check whether a checkbox or radio button is checked.

Failed Test Retries Option (Scheduler)

What is it?

This is a follow up to the Failed Test Retry flag which was released earlier to be used as part of the CLI. When a value is set in this field (1-20), a failed test will be executed repeatedly until either the test passes or the max number of retries has been reached (in which case the test will finish execution with a failed status). 


Why should I care?

You now have the ability to use this option via scheduler as well; apart from just the CLI.

Over the past few years, our team has been working hard to improve the functionality and usability of our autonomous testing platform. This mindset is what made us successful in building a product that people love and trust. In 2019, we released many new features that help testers author and execute tests more efficiently.

In this webinar, we discussed how the AI works underneath the hood, what are some of the features that help in faster authoring, execution and maintenance of automated tests and finally, discussed the top features within Testim that were released this year.

Here is the recording of the webinar

Below are some useful links related to features discussed in this webinar

Some useful blog posts to quickly get to know Testim

“Automated testing” used to be a phrase that caused developers to scratch their heads. Now, it seems to me that most developers are at least familiar with the notion of automated testing. Fortunately, many devs go beyond familiarity. They turn theoretic notion into practical action by actually employing automated testing on a day-to-day basis. And nowadays there’s immense variety in the types of automated tests that can help you achieve higher software quality. In today’s post, I want to focus on a very specific type of automated testing—UI testing.

We’ll start by briefly defining what UI testing is, and then explain how it can benefit you. Then we’ll proceed to show you, step-by-step, how to get started with UI testing using Testim.

Let’s get started.

UI Testing: What It Is and Why Do You Need It

As you surely know, UI stands for “user interface.” So, defining UI testing doesn’t sound that hard, at first sight—it’s just exercising the application through its user interface. But, is that all there is to it?

Actually, it’s not. First of all, some people seem to think that UI testing isn’t necessary if you already have other types of testing in place, such as unit tests. That couldn’t be farther from the truth. UI testing is not a replacement for other types of testing—instead, it’s a compliment. Unit testing exercises the application’s units in isolation. It’s an important verification to have, but it’s not enough. It’s also important to verify how those units fit together. You have to check how the whole system behaves, from the UI, passing through each layer and then back. So, UI testing can be thought of as more of an “end-to-end” type of testing.

Other people might think that, while UI testing has some importance, you can have one or more developers just click around and see if the application behaves correctly. But that isn’t true. Manual UI testing might work for very small and simple applications. But when it comes to applications with complex business rules and rich user interfaces, manually testing quickly becomes insufficient. It is a tedious, error-prone, expensive, and hard-to-scale process.

At this point, automated UI testing is the best choice.

UI Testing Guide: How To Get Started

It’s time to get your hands dirty. We’re going to offer you a brief guide on how to get started with UI testing. It’ll by no means be an exhaustive tutorial, but it should teach you to create a basic approach upon which you can build and evolve into a more sophisticated approach later.

For this guide, we’re going to use Testim to test a very simple web app. Let’s get started.

An Example of Manual UI Testing

Before we start creating our tests, we need something to test, right? So, let’s create a toy application. It’s going to be a super simple web app, with just two controls—an input box and a button. It will allow the user to input numbers, separated by a comma. After clicking on the button, the application will add all the numbers and display the results. If the user types just one number, the result should be that number. If the input box is left empty, the result should be zero. Simple enough, right?

Creating a Toy Application

Ok, let’s get to it. This application will consist of a single HTML page with a little bit of Javascript. Nothing too fancy. Open up your favorite text editor, create a new file, and add the following content to it:

<!DOCTYPE html>

<html>

<head>

<title>String Calculator</title>

<style type=”text/css”>

label { display:block; margin: 10px;}

</style>

</head>

<body>

<h1>String Calculator</h1>

<form id=”form1″>

<label for=”numbers”>Numbers: <input type=”text” id=”numbers” name=”numbers”/></label>

<label for=”addButton”><input type=”submit” value=”Add” id=”addButton”/>

</form>

<script type=”text/javascript”>

document.getElementById(“addButton”).addEventListener(‘click’, function(){

    var text = document.getElementById(“numbers”).value.trim();

    if (!text) {

        text = ‘0’;

    }    

    var integers = text

        .split(“,”)

        .map(x => parseInt(x));

    var result = integers.reduce(((a, b) => a + b), 0);

    alert(result);

});

</script>

</body>

</html>

After that, save the file with the “.html” extension. Then, double-click the newly saved file. You should see your default browser showing you something like this:

Performing Manual UI Testing

Now let’s do some (manual) UI testing. Click on the input box, type 0 and click on the Add button. You should see this:

Now, redo the test, but leave the input box empty. The result should also be zero:

Repeat the test a few more times, always using a single number. The result should always be the number itself. Then, perform some tests with more than one number, separated by a comma. The result should be the sum of the numbers, as the following example shows:

After this brief session of manual UI testing, I’d be convinced that our little toy app works as intended. But, is that testing enough? For this example application, I’d say yeah, probably. But keep in mind that this silly string calculator is just a placeholder for a much more complex application, closer to the ones you’re likely working on in real life.

Getting Started With Testim

Now it’s time to show you how to get started with Testim so you can see what automated UI testing looks like.

First Steps: Creating an Account and Installing the Testim Browser Extension

First, go to https://testim.io and  click on “Start Trial.” You’ll be asked to sign up for an account. You can do that using your Google or GitHub account. If neither option is OK for you, sign up with an e-mail address and create a password.

After successfully logging in, you’ll see a screen like the following image:

If you click on “Create test” button, you’ll see a message asking you to install the Testim Chrome extension:

After downloading and installing the extension, you’ll be ready to start creating automated tests. This time, though, instead of testing our silly little string calculator, we’ll test a real one—the Google calculator.

Creating Your First Test With Testim

On Chrome, go to the address bar, type “calculator” and hit enter. You should see this:

Just a good old calculator. Now, we’re going to create a test for the “+” button.  While still on the same tab, click on the Testim extension’s icon, as shown in the following image:

After clicking on the extension icon, you should see the following menu:

Click on “CREATE AUTOMATED TEST.” That will open Testim and attach the current tab to it. Then, you’ll return to the calculator tab and see the following message:

What you have to do now is very simple—just use the application and your actions will be recorded. Let’s create a test for the “2 + 2” sum. Perform the sum as you would normally do it. After clicking on the “equals to” button, go back to the tab where Testim is open. You should now see this:

Click on the “SAVE” button. You’ll then see a window asking you for a name and a description for your test:

For the name, enter “Add test”. On “Description”, type “Verify the Add operation” Then click on “OK.” Congratulations, you’ve just created your first test with Testim!

Running Your Test

You’ve just created your first automated UI test using Testim. That’s nice, but how do you actually run the test?

On the left panel, click on the “Automate” button, like in the following image:

After doing so, you’ll see your test list which contains only the test you’ve just created:

To run the test, just mark its checkbox and click on the “play” button:

After clicking on the “Run Selected Tests” button, you’ll see a message warning you to avoid touching the mouse and the keyboard during the test execution:

After clicking on “OK”,  Testim will open up a new Chrome tab and run the test. Testim will execute the exact same steps you’ve recorded. After Testim finishes running the test, it’ll close the window.  Go back to the Testim tab and see the test result:

You’ve successfully run your first test using Testim. Congratulations! But…isn’t there something missing?

Adding a Verification

You’ve probably noticed that the test you’ve just created doesn’t really test anything. Yeah, the test passes but that doesn’t really mean much. In terms of unit testing, it’d be the same as having a test method with no assertions.

Our test lacks something, and that something is a verification. We need our test to check that the result of “2 + 2” is indeed four. How do we do that?

On the left panel, click on the “Automate” button. You’ll go back to the test listing. By clicking on the test name, you’ll reopen the test editor so you can change it:

You can see that the test has six steps, and the last step is clicking on the “=” sign. What we have to do now is add an extra step, whose goal is to perform the validation. To add a new step, click on the “+” button after the sixth step. You’ll see the following options:

Click on the third option and you’ll see the menu change to this:

Now, click on “Validate element text.” Testim will take you back to the calculator tab. There, click on the result field:

After doing that, go back to Testim. Now, you should see that your test has a new step:

Hover over the newly added step and click on the engine icon to show the properties panel:

Now it’s just a matter of editing the “Expected value” field, setting it to ‘4’. After doing that, click on the “SAVE” button again. You’ll see a box prompting you for a message, explaining your change. Type “Add validation step” and click on OK:

If you run the test again after saving it, it will continue to pass. However, Testim is now comparing the expected result to the actual result, just like a unit test assertion does.

UI Testing Goes Way Beyond “Clicking Around”

There are still some misconceptions about UI testing out there. One of them is that UI testing isn’t needed if you already employ different categories of testing. Another myth is the belief that informal manual UI testing—a.k.a. just clicking around—is enough.

Neither of those beliefs is true.

We hope this post has shown you not only that UI testing is indeed necessary, but also that it isn’t hard getting started with it, especially if you employ a tool like Testim.

Thanks for reading, and see you next time!

Author bio: This post was written by Carlos Schults. Carlos is a .NET software developer with experience in both desktop and web development, and he’s now trying his hand at mobile. He has a passion for writing clean and concise code, and he’s interested in practices that help you improve app health, such as code review, automated testing, and continuous build.

Quite often there is a need to validate the content of a downloaded document which could be a PDF, Word Document, CSV file or databases such as MySQL, MongoDB. All these are possible within Testim using the CLI action step. This feature helps to create and execute custom Node.js scripts at runtime; to perform different validations. Below are some tips to help with CLI action steps.

Tip 1: How to find out what node.js packages you can use?

The different node.js packages that can be used within Testim are the ones found in https://www.npmjs.com/. This is the official website that has references to all the open source node.js packages. Say for example, you are searching for a package related to pdf documents; just search for “pdf” and you will get a list of all the node.js packages available for pdf.

Tip 2:  What are the available CLI action steps within Testim?

Below are the available CLI action steps within Testim-

  • Add CLI action
  • Add CLI validation
  • Add CLI wait for
  • Validate download
  • Wait for download

Tip 3:  How to add node.js packages to Testim?

To add node.js packages to Testim, follow the below steps

  • Add one of the above mentioned CLI action steps
  • In the Properties Panel, click on PARAMS-> PACKAGE

Tip 4: How to execute CLI action steps locally?

To execute node.js scripts via CLI action steps locally, you need to first set up the runtime environment using the below command. Ensure this command is executed from the command line before running a test with a CLI action step.

npm i -g @testim/testim-cli && testim –agent

Putting everything together

Lets see how all the tips work together to execute node.js scripts. Say you want to validate the content of a downloaded document such as a PDF file. The steps to do this would be-

  • Ensure CLI Action feature is enabled (Contact our CSM or customer support for this)
  • Add the “Validate Download” action
  • Add the necessary node.js package for the pdf content validation. In this case, we are adding the latest version of the package pdf-parse
  • Then, add the necessary code to do the content validation

Here is a sample test to show how to do PDF content validation – PDF Validation

For more CLI action examples, check out the last section of this doc – CLI Action

Artificial Intelligence (AI) has taken over the world in the past few years. Not a day goes by without hearing something new that has come up in the field of finance, healthcare, ecommerce, computer vision and much more. Now we have AI in the field of software testing.

In this webinar we discussed how Testim has implemented AI to speed up authoring, execution and maintenance of automated tests and has significantly reduced the cost of maintenance. We covered-

  • How the AI works underneath the hood?
  • How we reduce the time spent on authoring, execution and maintenance of tests?
  • How we integrate Testim in a CI/CD pipeline?
  • What kind of reporting we get when using Testim?

Here is the recording of the webinar

Introduction
We work hard to improve the functionality and usability of our autonomous testing platform to support your software quality initiatives. This month we’re thrilled to release a few of your most requested features; Multi Tab Screenshots, Generate Random Value Step, Suite View Free Text Search, Test List Export Feature. Check them out and let us know what you think.

Multi Tab Screenshots

What is it?

As a follow up to the multi tab indicator feature released last month, we now provide additional visibility when working with multi tab steps. You will now see test fail screenshots from all test tabs.

NOTE: The bottom carousel displays all tabs. There is indication of which tab the step executed on (the “Run Tab” label) as well as an indication of which tab is currently presented in the main viewport.

Why should I care? 

There is no longer a need to open a step and look at the screenshots or other related information to know what tab the step ran on. With the tab number indication and the screenshots, the user has more visibility into test runs that involve multiple tabs/windows. Learn More

Generate Random Value Step

What is it?

You now have a step to generate random values containing text, numbers and certain prefixes during run time. The value gets stored in a variable name which can be used in other steps as well. By default the variable name is randomValue.

In the interest of making this step more robust, we’ve added a few simple (but useful) features:

● Ability to generate letters only, numbers only, or letters and numbers (mixed) strings.

● Ability to control the length of the output string

● Ability to add a prefix to the generated value (can be a string, variable, or JS expression)

Why should I care? 

Until now, you had to add custom code to generate random strings. With this new step everything is handled by Testim within one click and the need for code is eliminated.

Suite View Free Text Search

What is it?

You now have the ability to do free text search in the Suite Runs tab to search for a particular suite.  As you type in the characters, a list of suggestions (based on the existing test suites) are automatically pre-populated for you to click on.

Why should I care? 

There is no longer the need to scroll through all the suites and then click on the required one. Instead, you just search for the suite and it automatically shows up.

Test List Export Feature

What is it?

You will now see an export button in the test list page. Clicking this button will download the test list as a CSV file, which can then be imported into excel or Google sheets.

NOTE: We have the same feature in the results page as well. Learn More

Why should I care? 

We now have the ability to easily share the tests across teams with people who use and do not use Testim. This also gives the flexibility to feed the generated CSV file to any external tool or framework for more customized reporting. The possibilities are endless.


Quite often there are some elements and attributes that change dynamically on a page. For example – Say you have a currency conversion application. Today, 1 US Dollar would be equal to 0.89 Euros. The same dollar could be 0.90 Euros the very next day; as they change dynamically on a daily basis. In Testim, you have the following ways to handle dynamically changing elements-

Tip 1: Validating Dynamically Changing Elements

Whenever you want to validate dynamically changing elements, Testim has export parameters to handle this situation. 

For example – Say you want to validate the price of your flight trip. The price changes dynamically on a daily basis based on multiple factors. In this case, you can use export parameters; where in, you can store the element in an export variable and use the variable in different steps as parameters and also use it to do different validations in a page.

Tip 2: Generating random values during run time

Quite often there is a need to generate random values containing text, numbers and certain prefixes during run time. This is possible using the generate random value step in Testim. The value gets stored in a variable name which can be used in other steps as well. By default the variable name is randomValue.

Introduction

We work hard to improve the functionality and usability of our autonomous testing platform to support your software quality initiatives. This month we’re thrilled to release a few of your most requested features; Multi Tab Indicator, Advanced Merge, Failed Test Retry Flag and Refresh Option. Check them out and let us know what you think.

Multi Tab Indicator

What is it?

You now have a numeric value that displays in the top right section of each step that executes in multiple tabs.

Why should I care?

There is no longer a need to open a step and look at the screenshots or other related information to know what tab the step ran on. With the tab number indication, the user has more visibility into test runs that involve multiple tabs/windows. Learn More

Advanced Merge

What is it?

When merging changes into a branch, a modal window will now pop up showing different changes that are being merged into the branch. The changes are categorized into Test, Shared Steps and Suites.

At the top level, users will be able to see what changed in each category – how many new items were created, updated, or deleted. Expanding each item, will display more details about individual changes.

Why should I care?

You now have better visibility and confidence before merging branches. All the details that are getting into the branch are clearly detailed in the modal window.

Failed Test Retry Flag

What is it?

When this flag is set, a failed test will be executed repeatedly until either the test passes or the max number of retries has been reached (in which case the test will finish execution with a failed status). This flag is passed in via the CLI by using the below syntax

––retries <max_num_of_retries>

Why should I care?

You now have the ability to re-run failed tests automatically, with the use of this flag. So even if a test fails once or twice because of some unexpected issues, it will automatically run again so that the test passes the next time.  

NOTE: When a test passes after one or more retries, it will be indicated in the UI as shown below

Refresh Option

What is it?

This new option will completely reload the page before proceeding to the next step.

Why should I care?

You no longer have to add custom action to reload the page. Now, there is a one click option to do it.

Testim gives you the flexibility to add regular expressions (RegEx) to help in easier string searching and validations.  It is extremely helpful in extracting required information from a web page or when there is a need to validate strings that has a portion of it changing dynamically.

For Example – Say you want to validate the label  “Price” in the below page.

The price value is going to change dynamically based on the itinerary booked; every time you run the test. So, if you want to ensure the label “Price” is displayed correctly in the page no matter what the price value maybe, RegEx can be of great help here.

You could add /^Price/ in the Expected Value Field in the properties panel of the Test Step; within Testim.  What this does is, it validates whether the text starts with the word “Price”, allowing the rest of the text to be dynamic and still pass the validation.

Commonly used RegEx and their syntax are as follows-

More references on how to use RegEx can be found below-

https://docs.testim.io/advanced-steps/advanced-text-validations

https://www.w3schools.com/jsref/jsref_obj_regexp.asp

https://regexone.com/



Testim gives you the ability to playback tests in incognito mode. The reason you may want to use incognito mode is, to get the true behavior of the application without any cached data. This is similar to running tests on the grid, where in each test runs on a new browser instance without any cached data (same as running in incognito mode). The different tips to help you playback tests in incognito mode are as follows-

NOTE: Ensure you allow Testim to run in incognito, before playing the tests.

Tip 1: Running a single test in incognito

If you want to play a test you just created in incognito, follow the below steps

  • Click on the drop down arrow next to the play button
  • Click on “Run in Incognito mode”

Tip 2: Running multiple tests in Incognito

Multiple tests can be run in incognito mode by using the CLI. Each time a test is run on the grid a fresh browser instance opens up without any cached data. Follow the below steps to run multiple tests in incognito all at once-

NOTE: Tests cannot be run in incognito mode from the Test List view. The CLI needs to be used to run multiple tests in incognito.

Be smart & save time...
Simply automate.

Get Started Free