Category

QA

Category

What are Loops?

Loops are one of the most powerful concepts in programming and sometimes could be a little hard to understand. At Testim, we made it easier for users to use Loops in their tests; by building it within the framework itself. This means, the user does not need to write code to repeat a group of steps a certain number of times.

Why do we need Loops?

Loops are useful when we need to repeat an action several times in our test.

For example –  Say we want to check whether the “Post” button on Twitter works consistently. We could have a scenario where we want to click the button 50 times to ensure it works consistently without any problems. In this scenario, are we going to record 50 steps to click the button 50 times or just record the step once and repeat it 50 times?

This is where Loops can be our best friend. It helps in repeating an action several times without needing to repeat the steps again and again. As a result, we save a lot of time in test authoring and execution.

How to use Loops?

Loops could still be a little hard concept to grasp. So here is a quick tip on how to easily use loops within Testim.

Let’ say we have the below code

for(i = 1; i < 4; i++) {

//Steps to repeat 3 times

}

What we are doing here is –

  • We are initializing a variable “i” to be equal to 1 before the loop starts. This is our Iterator.
  • We are specifying a condition to repeat the steps a certain number of times by giving “i<4”. In this case we want to repeat a set of actions 3 times.
  • Each time we exit a loop we need to increment our Iterator. In this case, we are doing “i++” which will increment the variable i from 1 to 2, 2 to 3 and 3 to 4. Then, eventually we exit the loop when i = 4, as we have a condition which checks to see if i<4 (Remember 4 is NOT LESS than 4 but EQUAL)

The same logic applies to Loops in Testim as well. Where we have 3 steps –

Step 1 – Initialize the Iterator

Step 2 – Specify a condition to repeat the steps a number of times

Step 3 – Increment the Iterator

This is what we do in this Demo Example, where we-

 

STEP 1Initialize the Iterator

 

STEP 2Specify a condition to repeat the steps a number of times

  • Group the steps to be repeated in one single reusable component
  • Go to the Properties Panel and add Custom code
  • Give condition to repeat the group a certain number of times

 

STEP 3Increment the Iterator

  • Add a custom action as the last step inside the newly created group
  • Increment the Iterator

 

The same steps applies to any set of actions you may want to repeat using our Loops functionality. You can also use do…while loops which follows similar concepts.

Also remember, Testim supports loops based on element conditions apart from just custom code.

We recently hosted a webinar on Real Use Cases for using AI in Enterprise Testing with an awesome panel consisting of Angie Jones and Shawn Knight and me being the moderator. There were lot of great discussions on this topic and we wanted to share it with the community as well.

Below you will find the video recording, answers to questions that we couldn’t get to during the webinar (will be updated as and when we get more answers) and some useful resources mentioned in the webinar as well. Please feel free to share these resources with other testers in the community via email, twitter, linkedin and other social channels. Also, reach out to me in case of any questions at raj@testim.io or any of the panel members.

Video Recording

 

Q&A

Any pointers to the existing AI tools for testing?

@Raj: I am assuming this question is about things to know with existing AI tools in the market. If this is the case, then first and foremost, we need to figure out what problems we are trying to solve with an AI based tool that cannot already been done with other possible solutions. If you are looking for decreasing time spent on maintenance, getting non-technical folks involved in automation and making your authoring and execution of UI tests much faster, then AI tools could be a good solution. I may be biased in this but it is definitely worth checking out Testim and Applitools if any of the points I mentioned is your area of interest/pain point as well.

As discussed in the webinar, currently there are a lot of vendors (including us) who use all these AI buzzwords. This may result in you getting confused or overwhelmed in choosing a great solution for your problems. My recommendation is-

  • Identify the problem you are trying to solve
  • Pick different AI tools, frameworks to help solve that problem
  • Select the one that meets the needs of your project
  • Then, proceed with that tool

 

As a tester working with Automation, what should I do to not lose my job?

@Raj: First of all, I believe manual testing can never be replaced. We still need human mind to think out of the box and explore the application to find different vulnerabilities in our product. AI will be used complementary to manual testing.

Secondly, we need humans to train these AI bots to try to simulate human behavior and thinking. AI is still in its initial stages and is going to take another 10 -15 years to completely mature.

In Summary, I think this is the same conversations we had 10 years ago when there were automated tools coming into the market. Then, we concluded that automated tools helps to complement manual testing BUT NOT replace it. I think it is the same analogy here where AI is going to complement manual testing BUT NOT replace it.

As long as people are open to constantly learning and acquiring different skillsets, automation is only going to be make our lives easier while we can pivot and focus on other aspects that cannot be accomplished with automation. This mainly involves things related to creativity, critical thinking, emotion, communication and other things that are hard to automate. The same thing holds through for Artificial Intelligence. While we use AI to automate some of the processes to save us time, we can use this saved time in focusing on acquiring other skills and stay abreast with the latest in technology.

So the question here is, not more about automation/AI replacing humans but about how do we stay creative and relevant in today’s society? That is done by constant learning, development and training.

 

Do we have some open source tool on the market for AI testing?

@Raj: Not really, we do have a small library which was added to the Appium project to give a glimpse of how AI can be using in testing — https://medium.com/testdotai/adding-ai-to-appium-f8db38ea4fac?sk. This is just a small sample of the overall capabilities.

 

What should be possible in testing with ai, in 3 years time? And how do you think testing has changed (or not)?

@Raj: We live in this golden age of testing, Where there are so many new tools, frameworks and libraries that are available to us, to help make testing more effective, easier and collaborative. We are already seeing the effects of AI based testing tools in our daily projects with introduction of new concepts in the areas of location strategies of elements, visual validation, app crawling and much more.

In 3 years, I can see the following possibilities in testing-

  • Autonomous Testing

I think autonomous testing will be more matured and a lot of tools will include AI in their toolset. This means we can create tests based on actual flows done by the user in production. Also, the AI can observe and find repeated steps and cluster them to make reusable components in your tests. For Example – Login, Logout scenarios. So now we have scenarios that are actually created based on real production data instead of us assuming what the user will do in production. In this way, we also get good test coverage based on real data. Testim already does this and we are trying to make it better.

  • UI Based TDD

We have heard of ATDD, BDD, TDD and also my favorite SDD (StackOverflow Driven Development) 🙂 . In 3 years, we will have UITDD. What this means is, when the developers get mockups to develop a new feature; the AI potentially could scan through the images in the mockups and start creating tests, while the developer is building the feature in parallel. Eventually, by the time the developer has finished implementing the feature, the AI would have already written tests for it based on these mockups using the power of image recognition. We just need to run the tests against the new feature and see whether it passes/fails.

  • AI for Mocking Responses

Currently we mock server requests/responses for testing functionalities that depend on other functionalities which haven’t been implemented yet or for making our tests faster by decreasing response time of API requests. Potentially, AI can be used to save commonly used API requests/responses and prevent unnecessary communication to servers when the same test is repeated again and again. As a result, your UI tests will be much faster as the response time has been drastically improved with the help of AI managing the interaction between the application and the servers.

 

Will our jobs be replaced?

@Raj: Over the past decade, technologies have evolved drastically. There have been so many changes happening in the technology space but one thing constant is human testers’ interaction with them and how we use them for our needs. The same holds true for AI as well. Secondly, to train the AI, we need good data combinations (which we call a training dataset). So to work with modern software we need to choose this training dataset carefully as the AI starts learning from this and starts creating relationships based on what we give to it. Also, it is important to monitor how the AI is learning as we give different training datasets. This is going to be vital to how the software is going to be tested as well. We would still need human involvement in training the AI.

Finally, it is important to ensure while working with AI the security, privacy and ethical aspects of the software are not compromised. All these factors contribute to better testability of the software. We need humans for this too.

In summary, we will continue to do exploratory testing manually but will use AI to automate processes while we do this exploration. It is just like automation tools which do not replace manual testing but complement it. So, contrary to popular belief, the outlook is not all ‘doom-and-gloom;’ being a real, live human does have its advantages. For instance, human testers can improvise and test without written specifications, differentiate clarity from confusion, and sense when the ‘look and feel’ of an on-screen component is ‘off’ or wrong. Complete replacement of manual testers will only happen when AI exceeds those unique qualities of human intellect. There are a myriad of areas that will require in-depth testing to ensure safety, security, and accuracy of all the data-driven technology and apps being created on a daily basis. In this regard, utilizing AI for software testing is still in its infancy with the potential for monumental impact.

 

Did you have to maintain several base images for each device size and type?

Has anyone implemented MBT to automate regression when code is checked in?

 

Resources relevant to discussions in the webinar

“When I talk to my colleagues and tell them that we have a team of developers that are performing manual testing guided by a test-plan nobody believes me… ”

Let me just start by saying that I’m very proud of our team and how far we have come.

Since my arrival, we have tripled in size and have significantly improved our quality standards. When I Joined Testim, we were a small R&D team based in Israeli. We had no QA department or defined testing process and had to implement it all from scratch. There are many challenges in trying to create a process where testing is integrated into our CI/CD workflows. For one, there is awareness. The process of assimilating the Idea that we have to test our produced features and acknowledging there are consequences if we don’t. We have a very talented and speed oriented Dev team.  But, it comes to a point when you have to stop and take a step back to really plan for the future. As we strive for continuous improvement, we have to periodically evaluate our process from end to end, making sometimes small adjustments which can improve the flow holistically. We had to assimilate the idea that it is better to “waste” a week of testing than 2 weeks of refactoring. Or worse – to lose our customers trust in our product.

I’m very happy to say that one of the things I love most about our team – is that we do not have inner battles. (At least not anymore). We truly understand that we are all working for the same cause and moving as a team to get the job done.

Some history…

I started at Testim as a one-man show. I arrived with a  background in quality assurance and solid automation experience working for larger companies. The startup environment was new to me.  I took the first month to learn the culture, build rapport with the people, understand the customers and their objectives, and observe the various internal and external process end to end.  From these learnings, I could conclude which things needed to be changed, what was good and should be preserved, and which tools or methodologies could suit our needs the best way.

For example, the product-dev-testing workflow needed some tuning. With that said, what suits an enterprise would be terrible overhead for a fast-paced startup. Typically, a startup requires a lot of risk analysis, fine-tuning and wise tool selection. We needed flexible tools that were easy to use and configure as our processes would quickly evolve over time. We did not have time for lengthy implementations which involved consultants or systems integrators, we needed to deliver value instantly without any setbacks. 

I see a lot of guys being recruited to a small startup coming from a big enterprise, and without fully evaluating their surroundings, start to incorporate heavy test management, reporting, and bug tracking tools just because “This is the way it should be done.” Frequently they will encounter resistance from the team because they simply do not see the need, or it creates an unnecessary burden on the routine without really delivering added value to the current work process. The thing is that there is no “single right way” to do things, and each team needs to collaborate to create an efficient workflow that works for them.

The 5 steps we took to building our quality driven development process

  1. Effective Product-Dev-Testing flows – We needed some more concrete requirements, tech designs, with lean and effective documentation. We also needed to establish what was the entrance and exit point to all of the “stops” in our cycle (And how the ping-pong would be addressed during the cycle). That took some work but I think that we are in a very good place now and we continue to learn and iterate.
  2. Incorporating quality initiatives into our roadmap – An important thing to realize – plans tend to change in a fast-paced startup. With that said, if we don’t bring up things for the agenda – they will not go into our mindset (and eventually into our roadmap). I mapped the things I needed to implement in regards to quality and automation, ensuring it was a part of the plan and scheduled work procedure. One more thing I needed to change was proper time estimations for feature releases. While we do need to move fast and give reasonable time estimations, we started incorporating testing and automation coverage time into our time assessments for each feature. That way, the time we needed to release a well tested and “covered” product is being properly evaluated.
  3. Tool choice – We decided on tools, frameworks and document formats that suited our needs the best way and worked on perfecting the flow with every sprint. The key guideline was to make the tool work without making us work for the tool.
  4. Testing and automation coverage – We worked on reaching a point where our own automation gives us a very solid picture of our product and risk-analysis to cover what needed to be performed manually. In addition to that, we made it part of the process for each and every one of us to write automated coverage for the features we deliver. The standardized approach and guidelines are maintained with some insights from me and our Chief Architect. We also implemented an effective CI/CD process and pipeline to assure our fast pace and quality standards could go hand-in-hand.
  5. Ownership & teamwork – It is true that in a startup there is a lot more versatility in everybody’s roles.  With that said, over time we learned our strong points and took responsibility for things where we can contribute to making our work more productive. I took the Quality Assurance and Test-Automation part because it was my background. There was a time when testing was my sole responsibility. Time went by, we grew to understand that a bottleneck is not a good place to be stuck in (If you know what I mean). That lead us to adopt a whole new approach. We always worked as a team, and as such, everybody feels responsible to make their task come to an end with maximum quality and fast delivery time. A part of that is to take responsibility and test your own work. When I tell my colleagues that we have a team of developers writing test automation and performing manual testing according to a test plan – nobody believes me. It’s actually very simple math. We had a lot of work to do, and a limited staff. The alternative is to deliver features without testing or to delay releases, and that was not an option. Test your work and be responsible for it – it’s as simple as that. Ones you push that deploy button – you have to be confident that you did what you could to assure it’s quality. Psychologically speaking, when you know that you don’t have a “Gatekeeper” checking after you, you would do a more thorough job. With that said – at times you do need an additional set of eyes or it’s better to let someone else take a look after you have boiled your own code for a long time – and that’s where I come in – or someone else from the team who is available. The popular conception is that only testers can test a product. But, what is the difference between a tester and a developer? The main difference is that a tester decided to learn how to test and a developer decided to learn how to develop. Is it possible for someone to learn how to do both? Traditionally, organizations preferred that it was separate functions. However, today’s Agile and DevOps practices try to reduce the handoffs between various stages of the process to create a continuous workflow. Once you teach a developer how to test, and give them the correct mindset for testing – they can tackle any feature they develop. Since they know the code and its weak points, they can do an amazing job of testing it., Everybody who “Touches” the product and its feature at any stage of the development lifecycle has to do their best to deliver quality work. The bottom line is as I mentioned before, there is not a “one size fits all” model – it takes time and continuous improvement to adopt a process that works best for your team.

I think the main key here is making quality an organizational initiative. Whether you are in product, engineering, sales or marketing, focusing on the customers experience should be the responsibility of everyone.  We try to leverage as much data as possible to provide cross-functional team transparency and not let emotion and anecdotal assessments get in the way of our decisions. We all share the same agenda and my mission is to influence and foster quality awareness and maintain a productive workflow which combines a detailed code review, effective CI/CD, unit testing and some TDD elements – our winning recipe for success.

 

It’s no surprise that mobile is big and only getting bigger. There are currently 8.7 billion mobile connections in the world, more than a billion active devices than there are people. Ensuring seamless user experiences for mobile users requires extensive software testing. However, testing mobile is much easier said than done – until now!

We are thrilled to unveil Testim Mobile.  SIGN UP  to our Mobile Beta program to experience super fast automated test creation and execution with minimal maintenance.

For those who are familiar with Testim, now you get all of the same AI capabilities you are accustomed to in the web version, now available in mobile. With this release, we have become the only fully integrated AI test automation platform that supports Web, Mobile Web and Native applications.

For those who are not familiar with Testim, we help engineering teams of all sizes across the globe improve test coverage, speed up release cycles and reduce risk in their delivery process. Companies like Wix, Globality, Autodesk, Swisscom and LogMeIn use Testim to integrate testing into their evolving Agile software development processes.

Playback Tests on Real Devices or Emulators

To get the true behavior of the application, it is recommended to playback tests on real devices, and not settle for an emulator. The main challenge with real device coverage is the plethora of available devices and configurations, which can include screen resolutions, performance levels, different firmwares from manufacturers, different operating systems and much more.

With Testim, you can directly playback tests on the physical device seamlessly; apart from just using the emulator.

Record actions in real time

When the user performs gestures such as tap, text inputs, scroll etc on the device/emulator, they are recorded in real time in the Testim editor. This allows you to see exactly which actions are being captured.

Get screenshots for each and every step

As we start recording different flows and creating multiple tests, it becomes critical to visually see the UI elements of the app when recording and playing the tests. Testim provides screenshots for each and every step that is recorded. Once the same test is played back, you can see the baseline and the expected image side by side. This way you know exactly what is happening underneath the hood.

Assertions

One of the most important capability of any automation framework is its ability to do assertions, this holds true for mobile as well. With Testim, you can do various validations to ensure the UI elements on the page are displayed as expected.

Feedback on each step

The user also gets feedback on each step in terms of whether the tests Passed or Failed by showing a “Green” or “Red icon” on the top left portion of each step as shown below

Ease of Use

The downfall of most of the automation frameworks is the complexity that goes into setting up the physical device with the framework, recording/playing back tests, maintaining and organizing the tests into meaningful components. With Testim, everything is handled within the the Testim editor just like how it is for users who are accustomed to doing Web and Mobile Web automation using Testim. There is no context switching between different windows, tabs, tools or frameworks. Just pair the device and start recording. It is that simple. We take care of the rest. All the features you have been using such as branching, version control, test suite/plans, test runs works the same in Testim Mobile.

In addition, you can maintain your Web, Mobile Web and Native automation projects all in one place and easily switch between projects and manage your tests.

Robust CLI actions

Users have the ability to create custom Node.js scripts that can be executed in the CLI (Command Line Interface). This will give users the flexibility to not only perform UI level validations on the mobile application but also the opportunity to do more advanced actions such as database validations.

Inserting Custom Code

Custom javascript code can be added to perform addition validations (if needed). We can perform manipulation/validation of parameters.

Control structures built within Testim

Save time by using the while loop to repeat steps. This eliminates the need to duplicate steps or write code to perform certain actions on your mobile application repeatedly. This feature helps to build simple and robust tests.

Multi Application Recording

With Testim Mobile you have the the flexibility to record actions from multiple applications within the same test. This means, you can easily switch between applications during recording and perform end to end testing, without any additional configuration.

 

Upcoming features in future releases

We will be releasing more features for Testim Mobile in the upcoming months. Here is a quick sneak peek into some of them-

Support for advanced gestures

Users will be able to record/playback more gestures on their applications that includes double tap, nested scroll, multi-touch, long press menus and much more.

Pixel level validation

Testim provides a step by step baseline and expected images out of the box, as a basic means of debugging your test runs. In addition, you will also be able to do visual validation. This feature is helpful when you need to ensure the UI of the mobile application is exactly what is supposed to be, across different devices.

Support for multiple IME’s

With multiple IME’s (Input Method Editor) available in the Android ecosystem, it makes sense to give users the flexibility to record/playback tests with the desired IME. We will soon be able to use alternative input methods such as on-screen keyboards, speech input etc, on our devices and record/playback tests with Testim Mobile.

 

Get started with Mobile Native App Automation

Don’t take our word for it, test drive the Mobile Beta and find out first hand for yourself to see how AI driven mobile automation can be super fast, stable, and simple.

It is also worth mentioning that, users who sign up for the program get a personalized on-boarding process from our team, which also includes 24/7 customer support. The best part of all this is, everything is FREE. We are more interested in your feedback and would like to showcase our AI powered mobile automation framework.

So, what are you waiting for? Enroll to our beta program and take your first step towards easy and seamless mobile native app test automation using AI.

As you may already know, Testim provides the flexibility to extend the functionalities of our platform by giving teams the ability to add their own JavaScript code. That being said, there maybe a situation where you need to use inbuilt javascript methods within the Testim JavaScript editor.  For Example – Say you want to find out of if an element is enabled on the page, you could do this

if(!element.disabled) {

 return true;

}

Now how do you know that, you could use the inbuilt method disabled() here?  The answer is simple.  Just follow the below steps-

  1. Open any web page on your Chrome browser
  2. Right click on an element and select “Inspect”
  3. Navigate to Console tab
  4. Start typing “$0.”
  5. Then observe all the inbuilt methods which are supported by Chrome

The same methods are supported by Testim as well.  Additionally, we support commonly used javascript methods such as reload(), split(), trim() and much more.  For more javascript examples check out our help document here.

Authoring and Execution of tests is an important aspect of test automation. Tests should be simple to write, to understand, and to execute across projects. The chosen platform should give the flexibility to both record and playback tests and write custom code to extend the functionalities of the automation framework. I recently came across Angie’s article on 10 features every codeless test automation tool should offer. She does a great job of discussing different aspects of test automation that needs to be an integral part of any automation platform.

Angie’s breakdown appeals to the heart and soul of what we set out to do when we built Testim. Starting from her explanation of why record and playback tools fail (we discuss some of these issues in this post as well) to the different challenges mentioned in her article.

We are really proud of what we do at Testim and wanted to address how we handle some of the important aspects of test automation pointed out in her article. We also highlight how we use AI to solve the “maintenance” issue which is arguably the hardest challenge of test automation.

 

  • Smart element locators

Testim’s founder (Oren Rubin) coined the term “Smart element locators” in 2015, when he gave the first demo of Testim. He showed us how AI can be used to improve locators. Using hundreds of locators instead of a single static one, Testim can learn from each run and improve over time.

With static locators (e.g. CSS-Selector/XPath), we use only one attribute of an element to uniquely identify it on a page and if this changes, the test breaks and as testers, we end up spending a considerable amount of time troubleshooting the problem and fixing it. Based on research, about 30% of testers’ time is consumed in just maintaining tests. Can you imagine the opportunity cost associated with this effort?

A Testers’ time is valuable and better spent on actually exploring the application and providing information to help stakeholders make informed decisions about the product. With AI based testing we can overcome this problem by using dynamic locators. Dynamic Locators is a concept where we use multiple attributes of an element to locate it on on the page instead of a single attribute. This way, even if one attribute changes, the element can still be successfully located with the help of other attributes that have already been extracted from the DOM by the AI. Testim is based on Dynamic Location Strategy making your tests more resilient to change.

  1. Conditional waiting

Testim supports conditional waits and can be added in a click of a button. We provide built in wait functionalities based on element visibility, text (or regex), code based (custom) waits using JavaScript, waits based on downloaded file and of course the hardcoded “sleep” (which is not generally advisable to use in tests unless you have no other option).

  1. Control structures

Testim supports “if” statements and “loops”. Looping can be applied on the test level by parameterizing your tests with different datasets (aka Data Driven) or on a specific subset of actions (something super hard, that only Testim supports). These conditions (when to stop the loops) can either be simple, predefined (such as element or text being visible) or can be more complex with custom code. This has been an integral part of Testim since the first version.

  1.  Easy assertions

Assertions are one of the most widely performed actions with test automation. You want to validate an element based on different conditions. For example – If a particular element needs to appear on a particular page of your website, we need to add an assertion to validate the presence of element. With Testim, we made it easy for users to add assertions with a single mouse click and built all of them within the tool itself.

Users have various validation options that include:

  • Validate element visible
  • Validate element not visible
  • Validate element text
  • Validate via API call
  • Validate file download
  • Validation via custom JS code running in the browser (great for custom UI)
  • Validation via custom JS code running in node.js (great for pdf and DB validations)
  • Visual validation – via Applitools integration*.

Testim integrates seamlessly with Applitools, a Visual Assertion platform, which allows you to validate not only the text, but also its appearance like font and color.

  1. Modification without redo

Testim not only supports easy modification of steps, the platform also supports full version control, including creating branches and auto-sync with github.

In Testim, you can add a step at any point in your test.

You can also easily delete or modify any step in your test

  1. Reusable steps

Testim supports reusability by grouping several steps together and the ability to reuse the same group in another test (and passing different parameters e.g. a login),

For Example – The simple steps to log into an application, is one of the most commonly repeated steps in test automation. In Testim, you can create a reusable “Login” step by selecting the steps we want to group together and click on “Add new Group” as shown below.

 

 

Not only does Testim support the creation of reusable components as explained above, the platform also supports maximizing the use of reusable component through a feature called Group Context. Imagine you have one or more components (E.g. gallery of images) within a page or across several pages, and you need to easily state on which component to perform the group of actions. Although this is relatively doable in coding (via the Page Object design pattern), this was extremely hard to implement in low code tools until now with the release of Group Context. Testim is the only codeless platform that can currently support this action.

  1. Cross-browser support

Testim uses Selenium underneath the hood, and supports test execution on ALL browsers, even mobile web, and mobile native (iOS/Android) which is currently in private beta. Signup for a free trial to execute tests on different browser combinations that include Chrome, Safari, Edge, Firefox and IE11.

  1. Reporting

It is vital to get quick feedback on your test runs, especially root cause analysis. The reports generated should be easy to read and needs to have relevant information on the state of the test. In Testim, there are different levels of reporting to help users know what exactly happened within each test run. Some of the features worth mentioning here include.

  • Screenshots

While each test is recorded, the platform takes screenshots of all the Passed and Failed results for each step. As a result, users find it easier to troubleshoot problems and understand what happens underneath the hood.

  • Feedback on each step

The user gets feedback on all the Passed or Failed steps in a test by showing a “Green” or “Red icon” on the top left portion of the step as shown below.

  • Entire DOM captured on failure

On failure, the user also has the option of interacting with the real HTML DOM and see what objects were extracted during the run.

  • Test Logs

Logs are a rich source of information on what happened underneath the AUT. Testim provides test logs when the user runs the tests on the grids. The option can be found in the in top section of editor.

  • Suite and Test Runs

We have suite and test runs views that enables the user to get granular details on each test run in terms of when the test ran, what was the result, the duration of the run, the level of concurrency, what browser the test ran on and much more. We also have filters to drill down based on different options.

 

  • Reports

We make it easy to get a high level health check of your tests using the Reports feature. There is no other testing platform that currently exists, that can give this level of details about the tests and this is done all in one single dashboard. We give details that include what percentage of test runs passed, number of active tests, average duration of tests, how many new tests were written, how many tests were updated, how many steps have changed, what are the most flaky tests in your test suite and all these details can be filtered based on the current day, 7 day or a 30 day period.

  1. Ability to Insert Code

Testim gives the flexibility for organizations to extend the functionalities of our platform using JavaScript by either running it in the browser; where Testim can help by finding elements for you in the DOM (aka dependency injection), or by running the code on node.js, which allows loading many common libraries (that includes accessing databases or inspect downloaded PDFs).

In addition, Testim has test-hooks to run before or after each test/suite.

For example, if you want to validate a particular price on a web page, you can grab the price, convert the string to a number and do the necessary validation. In the below example we are validating that, the price is not over $1000.

  1. CI/CD Integration

Testim easily integrates with ALL CI servers (e.g. Jenkins, Circleci, VSTS) by literally just copying the automatically generated CLI command and pasting it in your build servers. We integrate with ALL 3rd party grids hosting, that supports Selenium and Appium (Amazon/SauceLabs/BrowserStack…). We also support locally hosted grids and also provide our own Testim grids.

Apart from all the above mentioned features to help build stable automated tests, we also have   a notable feature to make tester’s life a lot easier by saving time to report bugs

 

Capture Feature

One of the most time consuming aspects of testing is bug reporting, where in, after finding a bug, we need to report it to the developer with relevant information, to speed up the troubleshooting and fixing of issues.

With Testim you can do this with a single click, with the help of our chrome extension. All the details related to the bug are automatically generated for you in matter of seconds.

 

In summary, we wanted to build a tool that could help in the authoring, maintenance and collaboration, which we consider the 3 pillars of test automation. Hopefully this post helps to highlight this aspect.

We also love to hear your feedback about our tool, so feel free to reach out to us by not only  trying out Testim for FREE but also getting a free consultation on Test Design and Test Automation on your existing testing frameworks and practices. Remember as Steve Jobs said “Simplicity is the ultimate sophistication” and this is the basis on which Testim was created.

Software Development ranks as the number one job in 2018, spanning industries such as healthcare, finance and manufacturing sectors. This being the case, it becomes really important to write good quality code when developing programs and apps for these industries. Even the slightest coding mistake can lead to big problems in the future. Many coding professionals use Log Analyzer – Real-time Log Analysis by Papertrail to monitor and fix coding errors in their programs.

Below you will find 7 Critical Tips to generate high-quality code.

Image via Pexels

  1. Developing Test Frameworks is Essential

With all of the different tools available to programmers, choosing the right ones can be a bit confusing. Utilizing the power of testing frameworks is a great way to see how well your code will perform. Most of the programs that offer these frameworks are automated and very easy to use.

If you are trying to perform a unit test, then using some of the commonly used unit test frameworks is a great idea. For programs in need of GUI testing, working with tools like Testim is beneficial. Regardless of what your testing needs are, there is a tool/framework out there for you.

  1. Have a Coding Standard in Place is Crucial

Most newcomers to the world of programming fail to realize just how easy it is to write bad and unorganized code. In order to write good code, you will need to put some time into developing standards. Following standards that require you to do things like having name conventions and using good formatting is a must.

By following these standards, you can rest assured that your code can be easily read by you and anyone else that comes across it. There are a number of widely accepted coding standards like the W3C standards that can take all of the guesswork out of this process.

  1. Code Documenters Can Be Helpful

If you are working on larger coding projects that span numerous namespaces and classes, working with software that instantly generates API documentation is a wise choice. With this software, you can help your team identify key pieces of the code without investing a lot of time in research. This type of software is also helpful for developers who are working on a number of projects simultaneously. For years, programmers have used JavaDoc  as their code documenter of choice.

Image via Pexels

  1. You Need to Make Sure Your Comments Are Useful

One of the most important pieces of a sound coding structure is good comments. If you take a break from a thousand line script you are developing, the notes you leave will help you make sense of it all upon your return. These comments can also be helpful to the people who will have to maintain your code once you are finished. Vague lines should have meaningful single-line comments attached to them.

Functions and methods should have meaningful names and parameter descriptions. You should also focus on keeping each of the comments you write up to date. Failing to change these comments when major alterations are made to the existing code is a recipe for confusion and disaster.

  1. Automated Build Tools can be a Life Saver

Using tools like Maven or Ant can assist you in getting your source code developed, compressed and deployed in a relatively short amount of time. By building your program or application with a single command, you can avoid errors and omissions, as well as performing repetitive tasks. 

  1. Get Familiar With Code Refactoring

Failing to refactor your code on a daily basis may lead to big problems in the future. Most programmers believe that refactoring is the key element to keeping their code healthy. When taking on this process, you need to make sure that everything from your program’s architecture to methods and functions are properly refactored.

Functions that have more than 25 lines of code, will generally have far too much logic contained in it. This means you should refactor it and split it into smaller functions or methods. While learning how to properly refactor your code will take time, it is well worth the effort and time you invest.

  1. Try to Avoid Using Global Code

Most new programmers fail to realize what a mess global variables and loops can create. Using global code will usually lead to a number of clashes and error developing. Ideally, you should avoid having blocks that are defined globally. Making sure your methods are written inside of class definitions can be very beneficial. You should also focus on putting class and function definitions within the right namespaces as well.

By following the advice in this article, you can avoid major headaches when developing new code and ensure good quality code.

Testim is #1 in innovation among all other competitors for several years now. This can be seen from the new features we have been releasing constantly to make automation faster, more stable and much more collaborative than ever. In continuation with this, we are excited to bring you our next big feature which we are calling Group Context. Imagine you have one or more components (E.g. gallery of images) within a page or across several pages, and you need to easily state on which component to perform the group of actions. Although this is relatively doable in coding (via the Page Object design pattern), this was extremely hard to implement in codeless tools until now with the release of Group Context. Testim is the only codeless platform that can currently support this action

For those of you not familiar with Testim, let’s start by defining “Context”?

“Context” is key in real life and in coding. You provide the necessary information in order to perform an action. For Example – Say you make a reservation at a restaurant for a party of four people, you would be required to provide a name for the reservation. Here the name is “contextual information” and you making a reservation is the “context”. Similarly, in test automation, it is important to know the context of elements from the DOM level in order to perform an action.

A Context in Test Automation

Context is all the more important when you have reusable components in your automation suite. For Example – let’s take the below web page.

 

It is a simple web page, containing a gallery of items. In this case, it has names of places with description and an option to book them as part of a reservation. In each item in the gallery, you have similar elements such as an image, text, and button, this is because the code for generating those instances is the same. So, on a high level, all these gallery items are exactly the same except for the different data showing up for each element in the gallery item.

Say we create a reusable group to:

  1. Validate whether an image and text is present in the gallery item 1 which is “Madan”
  2. Validate whether there is a “Book” button in the gallery item
  3. Click on the “Book” button
  4. Do some validations in the “Checkout” screen

It would look something like this:

 

Now, what if I want to use group on Item 2 of the gallery which is “Shenji” instead of “Madan” (which was the 1st item) ?

 

Typically, we would have to create another reusable group to make it work for gallery item 2, which is time consuming and does not make sense when the whole gallery shares the same DOM structure.

When using code (e.g. Selenium), you can use the Page Object design pattern, and just pass the context in the constructor, either as a locator or a WebElement (see slides 40 and 41 here).

In codeless/low-code platforms, this wasn’t possible until now with the release of our new feature “Group Context”. Now you can maximize reuse by assigning entire groups to different elements of the page and across pages within a single click. Taking the above example, you would not have to reassign all the elements/properties of steps within the group when you want to perform same set of actions, on another item in the gallery, with the exact same DOM structure. This means, we can use the same group on gallery item 2 which is “Shenji” without having to do any rework by just choosing the Context-> Custom option from the properties panel as shown below

 

Use Cases for Group Context

Group Context can be used in many other scenarios, such as:

  • Repeating elements: when you have similar elements repeating in the page and want to execute the same steps on all of them.
  • Table rows: when you want to execute the same steps on different rows in a table.
  • Tabs or frames: when you want to reuse a group of steps recorded on one tab on other tabs in the same or different page

 

Summary

We constantly keep working hard to improve the functionality and usability of our autonomous testing platform to support your software quality initiatives. We strongly believe this feature will benefit teams and help in making automation much smarter. Group Context is currently in beta and we would love your feedback.

Please use our chat support to give us feedback or e-mail us at product@testim.io. Happy Testing!!!

Selenium has been a popular automation testing framework for the past several decades. But, as applications have become more complex in the past several years, especially with the use of popular JavaScript frameworks such as Angular.js, Vue.js, React.js and Ember.js for building web applications; Selenium has found it hard to adapt to these technologies.

For example– If you are a currently using Selenium, have you ever experienced any of the below problems-

  • Spending majority of your valuable testing time fixing flaky tests?
  • Unable to make automation progress due to the lack of skilled programmers to write automated tests?
  • Not finding enough support in the open source community when new libraries and updates break existing tests and you have no idea what to do?
  • Need visual validation when a step fails, to visually understand the exact reason for the failure?
  • Insufficient logging information when your tests fail?
  • Finding it hard to seamlessly integrate your automated tests within your CI/CD pipeline?

If you answered YES, to any of the above questions, then you are not alone!!! According to new Gartner research, “Selenium is the de facto standard for front-end test automation of modern web technologies due to the flexible and powerful browser automation capabilities of WebDriver… It’s a sophisticated tool that isn’t easy to learn. It requires your team to have programming expertise and familiarity with object-oriented languages.” So, Selenium is good if you have the necessary programming expertise in your team, if not it becomes an obstacle.

There is no need to fear as there are good alternatives to Selenium. Below are the top 3 widely used alternatives currently available to give you and your team the flexibility to build robust automation test suites/frameworks.

NOTE:  Selenium is an open source community. People volunteer to make the framework better so the limitations of Selenium is not their fault as it is the responsibility of the user using the framework to extend its functionalities and make it better for his/her own automation needs.

  1. Cucumber

This is an open source testing tool that focuses on BDD (Behavior Driven Development). It emphasizes writing requirements in plain english in the form of Given, When and Then statements. This is commonly referred to as “Gherkin” syntax. You then convert these GWT statements into code using Java, JavaScript, Ruby or Kotlin.  This helps to enforce collaboration and bring more clarity to requirements.

On the flip side, it still needs someone with programming background to write the binding code to convert these GWT statements into usable actions. Also, the GWT format leads to a maintenance nightmare especially when many people collaborate and start making changes to different steps. Finally, it does not have any visual validation and has insufficient logging features making it hard to troubleshoot errors.

  1. SikuliX

This is an open source GUI based automation tool. It uses image recognition powered by OpenCV to automate anything with a UI. This includes desktop and web applications. It supports multiple programming languages including Python, Ruby and JavaScript.

Although this tool has robust UI validation functionalities, it lacks a lot of necessary features that are needed for stable automation such as locating elements based on attributes, creating reusable components and modularizing your tests for easier maintenance.

  1. Testim

Testim uses artificial intelligence (AI) for the authoring and execution of automated tests. The focus is on functional testing, end-to-end testing and UI testing. The Dynamic Location strategy used within the tool sets it year’s apart from its competitors. The Artificial Intelligence (AI) underneath the platform in real time, analyzes all the DOM objects of a page and extracts the objects and its properties. Finally, the AI decides the best location strategy to locate a particular element based on this analysis. Due to this, even if a developer changes the attribute of an element, the test still continues to run and this leads to more stable tests. This is the major advantage of using Testim compared to other frameworks like Selenium which uses static locators. Also, the “self-healing” mechanism of the AI helps to proactively detect problems in your test and fixes it automatically for you. As a result, the authoring and execution of automated tests are much faster and more stable.

Testim is not a completely code-less tool; you can use JavaScript and HTML to write complex programming logic (if needed) for your applications. This helps to make the automation suite more extensible. It provides easy integration with your CI/CD pipeline and most importantly helps to involve the whole team in automation including technical and non-technical people.

In summary, Selenium has really good capabilities but knowing there are other alternatives out there helps to give your more options to make your automation effort more easier and stable.

 

In the current era, where applications are becoming more complex due to the varied functionalities to support growing customer demands, it becomes all the more important to have a sophisticated IDE to write tests and automate these types of applications.

Selenium IDE was once popular with their Firefox and Chrome extensions but as applications became more complex in the past several years, there have been continual problems while using the Selenium IDE, such as-

  • Inability to handle dynamically changing elements on the web page
  • No inbuilt functionalities to automate actions that involve multiple tabs, hovers, scroll and other complex user actions
  • The need to give explicit and implicit waits manually to make tests more stable
  • Flakiness with the inbuilt options of the IDE such as Record/Playback where it seizes to work in some situations for no reason

Knowing these limitations; it is good to be open to other alternatives. Below are the top 3 alternatives that are currently popular among teams and organizations.

NOTE:  Selenium is an open source community. People volunteer to make the framework better so the limitations of Selenium is not their fault as it is the responsibility of the user using the framework to extend its functionalities and make it better for his/her own automation needs.

  1. Unified Functional Testing (UFT)

UFT (originally QTP – Quick Test Professional) was part of Mercury Interactive and was quite popular in the testing community. It was acquired by Hewlett Packard (HP) in 2006, then was renamed to UFT before it got sold to MicroFocus in 2016.

UFT is an automated function testing tool used for functional and regression testing. Using UFT, you can automate different user actions on multiple desktop and mobile browsers with different users and data sets, on various Windows OS versions.The down side of UFT is that, the IDE is built mainly for Windows OS. It has limited support for Mac and no support for Linux OS. Also, the execution of tests is really slow, especially when there is parallelism. So, there seems to be a huge gap between the amount of features offered and the price you pay for the tool given its limitations.

  1.  Protractor

This framework was born as a result of the inabilities of Selenium to handle javascript frameworks such as Angular.js. It is an end to end testing framework to specifically test Angular and Angular JS applications.  It has built in functions to identify different types of elements in these applications. One of the coolest feature of the framework is its support for parallel testing, where you can run several browser instances simultaneously.

Although protractor has good features to do web automation it does have its limitations. It only supports JavaScript and automating Angular.js applications. It does not support mobile apps. Also, the biggest complaint from users is, there is not enough developers in the open source community working to fix the current issues with the framework. As a result. it is really time consuming for teams and organizations to wait on fixes which becomes a big obstacle in automation progress and affects release cycles.

  1. Testim

Testim uses artificial intelligence (AI) for the authoring and execution of automated tests. The focus is on functional testing, end-to-end testing and UI testing.The Dynamic Location strategy  used within the tool sets it year’s apart from its competitors. The Artificial Intelligence (AI) underneath the platform in real time, analyzes all the DOM objects of a page and extracts the objects and its properties. Finally, the AI decides the best location strategy to locate a particular element based on this analysis. Due to this, even if a developer changes the attribute of an element, the test still continues to run and this leads to more stable tests. This is the major advantage of using Testim compared to other frameworks like Selenium which uses static locators. Also, the “self-healing” mechanism of the AI helps to proactively detect problems in your test and fixes it automatically for you. As a result, the authoring and execution of automated tests are much faster and more stable.

Testim is not a completely code-less tool; you can use JavaScript and HTML to write complex programming logic (if needed) for your applications. This helps to make the automation suite more extensible. It provides easy integration with your CI/CD pipeline and most importantly helps to involve the whole team in automation including technical and non-technical people. It supports multiple browsers – Firefox, Chrome, Edge, IE and Safari, multiple OS versions in Windows, Mac and Linux and also supports parallel testing on both web and mobile applications.

In summary, Selenium IDE does have great functionalities but unless like minded developers in the open source community come together and fix the existing issues with the IDE, users will have to look for better alternatives that are highlighted in this post.

Be smart & save time...
Simply automate.

Get Started Free