Software Development ranks as the number one job in 2018, spanning industries such as healthcare, finance and manufacturing sectors. This being the case, it becomes really important to write good quality code when developing programs and apps for these industries. Even the slightest coding mistake can lead to big problems in the future.
Below you will find 7 Critical Tips to generate high-quality code.
With all of the different tools available to programmers, choosing the right ones can be a bit confusing. Utilizing the power of testing frameworks is a great way to see how well your code will perform. Most of the programs that offer these frameworks are automated and very easy to use.
If you are trying to perform a unit test, then using some of the commonly used unit test frameworks is a great idea. For programs in need of GUI testing, working with tools like Testimis beneficial. Regardless of what your testing needs are, there is a tool/framework out there for you.
Have a Coding Standard in Place is Crucial
Most newcomers to the world of programming fail to realize just how easy it is to write bad and unorganized code. In order to write good code, you will need to put some time into developing standards. Following standards that require you to do things like having name conventions and using good formatting is a must.
By following these standards, you can rest assured that your code can be easily read by you and anyone else that comes across it. There are a number of widely accepted coding standards like the W3C standards that can take all of the guesswork out of this process.
Code Documenters Can Be Helpful
If you are working on larger coding projects that span numerous namespaces and classes, working with software that instantly generates API documentation is a wise choice. With this software, you can help your team identify key pieces of the code without investing a lot of time in research. This type of software is also helpful for developers who are working on a number of projects simultaneously. For years, programmers have used JavaDoc as their code documenter of choice.
One of the most important pieces of a sound coding structure is good comments. If you take a break from a thousand line script you are developing, the notes you leave will help you make sense of it all upon your return. These comments can also be helpful to the people who will have to maintain your code once you are finished. Vague lines should have meaningful single-line comments attached to them.
Functions and methods should have meaningful names and parameter descriptions. You should also focus on keeping each of the comments you write up to date. Failing to change these comments when major alterations are made to the existing code is a recipe for confusion and disaster.
Automated Build Tools can be a Life Saver
Using tools like Maven orAnt can assist you in getting your source code developed, compressed and deployed in a relatively short amount of time. By building your program or application with a single command, you can avoid errors and omissions, as well as performing repetitive tasks.
Get Familiar With Code Refactoring
Failing to refactor your code on a daily basis may lead to big problems in the future. Most programmers believe that refactoring is the key element to keeping their code healthy. When taking on this process, you need to make sure that everything from your program’s architecture to methods and functions are properly refactored.
Functions that have more than 25 lines of code, will generally have far too much logic contained in it. This means you should refactor it and split it into smaller functions or methods. While learning how to properly refactor your code will take time, it is well worth the effort and time you invest.
Try to Avoid Using Global Code
Most new programmers fail to realize what a mess global variables and loops can create. Using global code will usually lead to a number of clashes and error developing. Ideally, you should avoid having blocks that are defined globally. Making sure your methods are written inside of class definitions can be very beneficial. You should also focus on putting class and function definitions within the right namespaces as well.
By following the advice in this article, you can avoid major headaches when developing new code and ensure good quality code.
“Context” is key in real life and in programming. You provide the necessary information in order to perform an action. For Example – Say you make a reservation at a restaurant for a party of four people, you would be required to provide a name for the reservation. Here the name is “contextual information” and you making a reservation is the “context”. Similarly, in test automation, it is important to know the context of elements from the DOM level in order to perform an action.
A Context in Test Automation
Context is all the more important when you have reusable components in your automation suite. For Example – let’s take the below web page.
It is a simple web page, containing a gallery of items. In this case, it has names of places with description and an option to book them as part of a reservation.
When inspecting the DOM, the gallery items looks like this
As you can see we have a gallery of items on a web page.
Also, if you notice each item in the gallery has similar elements such as an image, text, and button, this is because the code for generating those instances is the same. So, on a high level, all these gallery items are exactly the same except for the different data showing up for each element in the gallery item.
Validate whether an image and text is present in the gallery item 1 which is “Madan”
Validate whether there is a “Book” button in the gallery item
Click on the “Book” button
Do some validations in the “Checkout” screen
It would look something like this:
Now, what if I want to use Clicking Gallery Item on Item 2 of the gallery which is “Shenji” instead of “Madan” (which was the 1st item) ?
Typically, we would have to create another reusable group to make it work for gallery item 2, which is time consuming and does not make sense when the whole gallery shares the same DOM structure.
When using code (e.g. Selenium), you can use the Page Object design pattern, and just pass the context in the constructor, either as a locator or a WebElement (see slides 40 and 41 here). In codeless/low-code platforms, this wasn’t possible until now with our new feature called “Group Context”. Now you can maximize reuse by assigning entire groups to different elements of the page and across pages within a single click. Taking the above example, you would not have to reassign all the elements/properties of Clicking Gallery Item when you want to perform same set of actions, on another item in the gallery, with the exact same DOM structure. This means, we can use Clicking Gallery Item on gallery item 2 which is “Shenji” without having to do any rework.
Use Cases for Group Context
Group Context can be used in many other scenarios, such as:
Repeating elements: when you have similar elements repeating in the page and want to execute the same steps on all of them.
Table rows: when you want to execute the same steps on different rows in a table.
Tabs or frames: when you want to use a group of steps recorded on one tab, in another tab.
We constantly keep working hard to improve the functionality and usability of our autonomous testing platform to support your software quality initiatives. We strongly believe this feature will benefit teams and help in making automation much smarter. Group Context is currently in beta and we would love your feedback.
Please use our chat support to give us feedback or e-mail us at <email@example.com>. Happy Testing!!!
For example– If you are a currently using Selenium, have you ever experienced any of the below problems-
Spending majority of your valuable testing time fixing flaky tests?
Unable to make automation progress due to the lack of skilled programmers to write automated tests?
Not finding enough support in the open source community when new libraries and updates break existing tests and you have no idea what to do?
Need visual validation when a step fails, to visually understand the exact reason for the failure?
Insufficient logging information when your tests fail?
Finding it hard to seamlessly integrate your automated tests within your CI/CD pipeline?
If you answered YES, to any of the above questions, then you are not alone!!! According to new Gartner research, “Selenium is the de facto standard for front-end test automation of modern web technologies due to the flexible and powerful browser automation capabilities of WebDriver… It’s a sophisticated tool that isn’t easy to learn. It requires your team to have programming expertise and familiarity with object-oriented languages.” So, Selenium is good if you have the necessary programming expertise in your team, if not it becomes an obstacle.
There is no need to fear as there are good alternatives to Selenium. Below are the top 3 widely used alternatives currently available to give you and your team the flexibility to build robust automation test suites/frameworks.
NOTE:Selenium is an open source community. People volunteer to make the framework better so the limitations of Selenium is not their fault as it is the responsibility of the user using the framework to extend its functionalities and make it better for his/her own automation needs.
On the flip side, it still needs someone with programming background to write the binding code to convert these GWT statements into usable actions. Also, the GWT format leads to a maintenance nightmare especially when many people collaborate and start making changes to different steps. Finally, it does not have any visual validation and has insufficient logging features making it hard to troubleshoot errors.
Although this tool has robust UI validation functionalities, it lacks a lot of necessary features that are needed for stable automation such as locating elements based on attributes, creating reusable components and modularizing your tests for easier maintenance.
Testim uses artificial intelligence (AI) for the authoring and execution of automated tests. The focus is on functional testing, end-to-end testing and UI testing. The Dynamic Location strategy used within the tool sets it year’s apart from its competitors. The Artificial Intelligence (AI) underneath the platform in real time, analyzes all the DOM objects of a page and extracts the objects and its properties. Finally, the AI decides the best location strategy to locate a particular element based on this analysis. Due to this, even if a developer changes the attribute of an element, the test still continues to run and this leads to more stable tests. This is the major advantage of using Testim compared to other frameworks like Selenium which uses static locators. Also, the “self-healing” mechanism of the AI helps to proactively detect problems in your test and fixes it automatically for you. As a result, the authoring and execution of automated tests are much faster and more stable.
In summary, Selenium has really good capabilities but knowing there are other alternatives out there helps to give your more options to make your automation effort more easier and stable.
In the current era, where applications are becoming more complex due to the varied functionalities to support growing customer demands, it becomes all the more important to have a sophisticated IDE to write tests and automate these types of applications.
Selenium IDE was once popular with their Firefox and Chrome extensions but as applications became more complex in the past several years, there have been continual problems while using the Selenium IDE, such as-
Inability to handle dynamically changing elements on the web page
No inbuilt functionalities to automate actions that involve multiple tabs, hovers, scroll and other complex user actions
The need to give explicit and implicit waits manually to make tests more stable
Flakiness with the inbuilt options of the IDE such as Record/Playback where it seizes to work in some situations for no reason
Knowing these limitations; it is good to be open to other alternatives. Below are the top 3 alternatives that are currently popular among teams and organizations.
NOTE:Selenium is an open source community. People volunteer to make the framework better so the limitations of Selenium is not their fault as it is the responsibility of the user using the framework to extend its functionalities and make it better for his/her own automation needs.
UFT (originally QTP – Quick Test Professional) was part of Mercury Interactive and was quite popular in the testing community. It was acquired by Hewlett Packard (HP) in 2006, then was renamed to UFT before it got sold to MicroFocus in 2016.
UFT is an automated function testing tool used for functional and regression testing. Using UFT, you can automate different user actions on multiple desktop and mobile browsers with different users and data sets, on various Windows OS versions.The down side of UFT is that, the IDE is built mainly for Windows OS. It has limited support for Mac and no support for Linux OS. Also, the execution of tests is really slow, especially when there is parallelism. So, there seems to be a huge gap between the amount of features offered and the price you pay for the tool given its limitations.
Testim uses artificial intelligence (AI) for the authoring and execution of automated tests. The focus is on functional testing, end-to-end testing and UI testing.The Dynamic Location strategy used within the tool sets it year’s apart from its competitors. The Artificial Intelligence (AI) underneath the platform in real time, analyzes all the DOM objects of a page and extracts the objects and its properties. Finally, the AI decides the best location strategy to locate a particular element based on this analysis. Due to this, even if a developer changes the attribute of an element, the test still continues to run and this leads to more stable tests. This is the major advantage of using Testim compared to other frameworks like Selenium which uses static locators. Also, the “self-healing” mechanism of the AI helps to proactively detect problems in your test and fixes it automatically for you. As a result, the authoring and execution of automated tests are much faster and more stable.
In summary, Selenium IDE does have great functionalities but unless like minded developers in the open source community come together and fix the existing issues with the IDE, users will have to look for better alternatives that are highlighted in this post.
We recently hosted a webinar on AI and its influence on test automation with an awesome panel consisting of Jason Arbon, Oren Rubin, Dionny Santiago and me being the moderator. There were lot of great discussions on this topic and we wanted to share it with the community as well.
Below you will find the video recording of the webinar, the list of questions and answers that we couldn’t get to during the webinar and different resources to learn about AI and testing. Please feel free to reach out to me in case of any questions at firstname.lastname@example.org or any of the panel members.
I am a performance engineer and am working on AI for quality gates in load testing results…that needs to be a high priority for the “future” which is “now”.How do you think bots can be used in this area? @Jason: UI-Bots can help generate user-like load directly via the application. Though, for most load testing problems, would recommend something like charles proxy, or internal ways to spin up load, and only use the ‘expensive’ UI-based bots to see how the app works E2E for the user under load.
With rapid changes in agile requirements, how do we make the machines learn or adapt to the changes every time? @Jason: The ai bots most folks are working on these days (vendors) will auto discover new features in the app and exercise them. At test.ai we have a set of 5k+ tests written for common flows in apps, so if you add a new feature to your app that looks like something similar on another app, the bots will auto apply that test case to the new build.
@Raj: The more tests you run, the smarter the AI becomes in detecting changes in the application. It will start detecting changes in application’s UI, element attributes and start adapting the tests automatically to these changes due to its self-healing mechanism. It can identify flaky tests, optimize waits in between steps and also proactively fix issues for us before they occur.
With BDD model and shift left and demand for testing at unit and service/api layer where does E2E testing stand? @Jason: Dionny’s work can help generate valid permutations of existing API test case parameters/flows. Also, clustering methods can help identify misbehaving servers via logs of activity during api testing or production.
Where can we find Dionny’s paper on AI testing, you were talking about? Dionny’s paper
Lot of automation test scripts fail due to test data issues, can we use AI to tackle those kind of issues? @Jason: Thats a broad category of failure types, but yes, ‘AI’ can be taught to auto associate correct data with the right application states. Google also shared some ‘test selection’ findings using ML to help decide what to do with all those failing tests: https://testing.googleblog.com/2018/09/efficacy-presubmit.html
I wanted to understand, what does really mean by AI in testing?if it mean by machine will perform testing? if machine will test then if they will be already defined with scenarios which needs to be tested? is it same as automation testing as there also we don’t need manual intervention? @Jason: Generally, AI in testing, means applying machine learning / AI techniques to test applications. There is also ‘Testing AI’ which refers to approaches to test AI/ML-based products and features. There are a variety of ways to apply AI to testing, some leverage pre-written test cases and the AI is used to automatically execute the tests, create variations, or analyze the results. Some AI based systems are trained to mimic general human behavior and can execute basic ‘flow testing’ for many apps, without pre-written test scenarios. The bots we build at test.ai can read BDD/Scenarios and execute them against a set of applications. As for need for human intervention, like automation, there is still the need for plenty of human intervention in AI-based testing approaches these days 🙂 Humans gather oracle/training data for the AI. Humans measure the correctness of the ‘AI’, and humans evaluate the significance of the AI-based test results as they relate to business/shipping decision.
@Raj: In addition to what @Jason was saying, I wanted to mention that, AI can have a positive impact on several facets of software testing especially test automation. There have been so many different tools and frameworks that have come up trying to solve different kinds of problems related to test automation but one problem that has been a constant challenge till date, is the aspect of “maintenance”. One of the main reasons for this is the use of static locators. With static locators, we use only one attribute of an element to uniquely identify it on a page and if this changes, the test breaks and as testers we end up spending a considerable amount of time troubleshooting the problem and fixing it. Based on research, about 30% of testers’ time is consumed in just maintaining tests. Can you imagine the opportunity cost associated with this effort? It is mind blowing. Testers’ time is valuable and better spent on actually exploring the application and providing information to help stakeholders make informed decisions about the product. With AI based testing we can overcome this problem by using dynamic locators. Dynamic Locators is a concept where we use multiple attributes of an element to locate it on on the page instead of a single attribute. This way, even if one attribute changes, the element can still be successfully located with the help of other attributes that have already been extracted from the DOM by the AI.
Can you guys elaborate on how do AI-based tests learn acceptance criteria that normally has to be defined by humans? @Jason: Depends on the AI system being used. The bots at test.ai execute human-written test cases. Acceptance tests are written at a very high level of abstraction, and the bots do all the test execution. Reporting is as normal for test automation. In summary, just tell the bots what your acceptance criteria are.
Is AI platform dependent. like desktop application or web/mobile? @Jason: Depends on the AI approach/solution. Many are platform dependent. The bots we build at test.ai though are not platform dependent–a key feature. The bots are platform-independent as the machines are trained to recognize UI elements much like humans do, and humans are platform dependent ;).
How can AI be used for improving test coverage ? @Jason: AI can help generate many more validate test scenarios than a human could create. AI also enabled re-use of test artifacts so a test written for one app, can also execute on a similar app with no human intervention.
@Raj: Now with AI, you can also connect your production apps to the testing cycle. This means that we can create tests based on actual flows done by the user in production. Also, the AI can observe and find repeated steps and cluster them to make reusable components in your tests. For Example – Login, Logout scenarios. So now we have scenarios that are actually created based on real production data instead of us assuming what the user will do in production. In this way, we also get good test coverage based on real data.
Will AI testing replace selenium appium and all tools and technologies? @Jason: Asymptotically.
Is AI really better? @Dionny: Traditional testing teams focus on either a single app, or a small set of apps; whereas, AI can learn from millions of different examples and apps. The more data we show the AI, the better it gets. Also, the AI never gets tired!
What are the immediate benefits of using AI? @Raj: Apart from the benefits already mentioned in the answers above, AI can also help in increasing team collaboration. The field of test automation has historically been a technical tester focused community. This stigma can also change with AI. What this means is, non-technical resources no longer need to fear code and technology, rather AI will help to bridge the gap between the technical know-how and authoring and execution of tests making life easier for teams.
Will our jobs be replaced? @Raj: Over the past decade technologies have evolved drastically, there have been so many changes happening in the technology space but one thing constant is human testers’ interaction with them and how we use them for our needs. The same holds true for AI as well. Secondly, to train the AI, we need good data combinations (which we call a training dataset). So to work with modern software we need to choose this training dataset carefully as the AI starts learning from this and starts creating relationships based on what we give to it. Also, it is important to monitor how the AI is learning as we give different training datasets. This is going to be vital to how the software is going to be tested as well. We would still need human involvement in training the AI. Finally, it is important to ensure while working with AI the security, privacy and ethical aspects of the software are not compromised. All these factors contribute to better testability of the software. We need humans for this too.
In summary, we will continue to do exploratory testing manually but will use AI to automate processes while we do this exploration. It is just like automation tools which do not replace manual testing but complement it. So, contrary to popular belief, the outlook is not all ‘doom-and-gloom;’ being a real, live human does have its advantages. For instance, human testers can improvise and test without written specifications, differentiate clarity from confusion, and sense when the ‘look and feel’ of an on-screen component is ‘off’ or wrong. Complete replacement of manual testers will only happen when AI exceeds those unique qualities of human intellect. There are a myriad of areas that will require in-depth testing to ensure safety, security, and accuracy of all the data-driven technology and apps being created on a daily basis. In this regard, utilizing AI for software testing is still in its infancy with the potential for monumental impact.
Will intelligence machines take over the world? @Raj: Hollywood movies do have an influence on our lives don’t they 🙂 ? At most of the conferences I speak at, there is this weird notion that, in 3 years, AI powered robots are going to take over the world and we will become slaves to them. Which sounds interesting on paper but in reality I don’t think that is going to be the case.
Currently there are are some section of the people who believe fully developed AI that can react and think like humans , will be developed by 2055 and there are are other sections of people who think it will take several hundred years for that to happen. No one knows the exact answer yet. That being said, there are several organizations trying to ensure the AI currently being developed is safe for the human society. For example – The future of life institute was formed for the exact same purpose and has the brightest minds in the AI field working in that group on AI safety research. We also have groups like the World Economic forum keeping a close eye on the impact of AI on society.
So, I do not think machines will take over the world, just yet!!! 🙂
If you work in the software industry, you’ve most likely heard about the popular term, “shift-left testing.” With Agile practices like TDD, BDD, CI and DevOps becoming mainstream, “shift-left” is the answer to how testing fits in, and must be done in order for them to become a reality. Instead of taking a backseat during the development process, testing is planned in advance and begins earlier in the SDLC (therefore “shifts left”). It could even start before a single line of code is written! Making this shift changes the view of testing— instead of traditional QA, it transforms into QE: Quality Engineering.
What Does Shift-Left Testing Look Like?
Thanks to the rise of automation, and the aid of tools that use AI and machine learning, testers have more time to dedicate to being more strategic about their work, instead of having their hands tied, running tests manually every day.
For testers to be successful today, they have to not only be great at testing, but also be engineers of the Agile testing process by collaborating with development and operations while analyzing quality during every stage of development:
Shift-left testing activities include:
Testers helping developers implement unit testing
Planning, creating, and automating integration test cases
Planning, creating, and employing virtualized services at every stage and component level
Gathering, prioritizing, and processing feedback
Several process changes occur when teams shift left. Instead of a developer waiting weeks to add his or her code to the rest of the team’s code, it can be done every day, or even several times a day. Instead of manually performing all the tests, most are automated and run every day, or even several times a day. And, instead of detecting problems at the end, the team as a whole analyzes quality as the development progresses.
Not sure if it’s the right move for your organization? Here are some of the pros and cons of shift left testing.
LOWER THE COST OF TESTING & DEVELOPMENT
It’s well known that the sooner a bug is found, the cheaper it is to fix. One of the aims of Agile testing is detecting errors as soon as possible. With shift-left testing it’s possible to detect in real time, the exact moment in which an error was inserted into the system in order to resolve it in a timely manner. When testing is done with each build (especially during unit testing), the errors that are found are smaller, easier to detect and locate, and subsequently, less costly to fix.
INCREASE EFFICIENCY & QUALITY
With the increased levels of automation when shifting left, teams can benefit from:
Increased test coverage since more tests can be run in the same amount of time
More time for testers to focus on more challenging and rewarding tasks
Reduced human error rate
Monitoring performance over time
Code quality checks
Built-in security checks
Reducing issues in production that users may encounter
Beyond these benefits, being able to start testing sooner invariably results in a higher quality product, as testers are less rushed to find all the errors at the end, when there’s little time left to fix them.
COMPETE MORE FIERCELY
In today’s competitive technological landscape, the barriers to compete are minimal, so the best way to survive is to be able to move fast and defend one’s stature by innovating in iterations, which is possible thanks to Agile. As everyone can agree that it’s important to deliver software more quickly, it also mustn’t be rushed out the door (causing a possible backfire). Shift-left testing answers the problem of accelerating development without sacrificing quality.
Another—yet less obvious—benefit of shifting left is that it can help businesses position themselves as an attractive employer to top talent. Because it is becoming more mainstream, with about two thirds of IT workers reportedly using Agile or leaning towards agile (according to a recent study by HP), it’s what the most forward thinking software professionals expect from their teams. Therefore, if you want to be an attractive employer or at least on-par with the rest, it is important to adopt the modern practices that both testers and developers want to master in order to stay relevant in today’s labor market.
EASIER SAID THAN DONE
For shift-left testing to be a success, an often drastic change in culture must occur first, which requires a team effort. Teams are usually set in their traditional ways of working, and when they consider shifting, they must consider how the methods, processes, skills, tooling, etc. will need to change. Even more important, what will need to happen to get all the roles within the organization to align properly?
RISK OF BOTTLENECKS
Yes, agile and shift-left aim to eliminate testing as a bottleneck, but it is true that agile teams can find themselves stuck waiting in a queue once all of the pieces come together in the performance and user acceptance testing phases, due to the complexity of environments and composite applications. One way to overcome this is to to utilize service virtualization. Service virtualization emulates the behavior of essential components that will be present in production, enabling integration tests to take place much earlier in development. This is how you can eliminate that key bottleneck, while also benefiting from eliminating errors earlier on. Along with service virtualization, there are several tools to setup automated systems and CI such as Jenkins.
A Worthwhile Undertaking
In the end, shift left testing is certain to have pros that outweigh its cons. Testers will find themselves delegating some of their work to developers and assigning them more testing activities. In mature teams, the testers become “coaches,” training developers on how to write better code, avoid bugs, and own unit testing. The advantage of this is that the tester who used to be busy, bogged down by writing test cases, now has time to delve deeper into the product, working on business cases, penetration testing, performance testing, implementing smarter testing solutions that use artificial intelligence like Testim.io, and so on. This sharing of the responsibility over testing leads to a higher level of achieved quality, as more of the bases get covered, quicker!
What do you think? Still not on-board to shift left testing? Or have you managed to do so already?
About the Author: Sofía Palamarchuk
Sofía is the Chief Executive Officer and Chief Product Officer of software testing services company, Abstracta. With a B.S. in Computer Engineering, Sofía started working in application performance optimization, system monitoring and load testing for the corporate sector for many years. With a solid background in performance tuning and automation, Sofía has become a business development leader and is responsible for managing all aspects of Abstracta’s US operations as well as its mobile testing tool, Monkop.
One of the most important factors related to automated tests is “Maintenance”. A lot of effort is spent on maintaining the tests than writing actual tests. A recent study suggested about 30% of testers time is spent on maintenance.This leads to wastage of valuable time and effort by the resources, which they could have rather spent on testing the actual application.
Imagine a world where the software can maintain tests without human interaction? This world has become a reality with Testim.io. We use Artificial Intelligence (AI) underneath the hood, which provides self-healing maintenance i.e problems are detected by the AI and automatically fixed without human intervention.
Testim.io also help to speed up the maintenance of tests by providing the follow features within our platform-
At any given time, it is important to have logs of what changes were made to a particular test. This way we can always revert back to an older version of test as and when required. Our platform provides this functionality by showing all the version history by going to the Properties panel of the setup step and clicking on “See old revisions”
At Testim.io, we firmly believe in the “Shift Left Paradigm” where Development and Testing must start in parallel as early as possible in the software development lifecycle. Keeping this in mind, we provide the functionality to teams to create separate branches for each team member and work on the same projects and tests. This way, no one can overwrite the changes of the other team members and teams can work on the same code base at any instant of time
In our platform, we just need to select “Fork” to create a new branch and we can also switch between existing branches
Users have the option of scheduling their tests. This helps to run the tests automatically at a certain day and time without any manual intervention. We can also get notified via email in case of any errors
As testers, we spend considerable amount of time troubleshooting issues. To help in troubleshooting, our platform offers different options to the user to narrow down the scope of the problem. These options are as follows-
The screenshot feature explained in the “Authoring and Execution” section helps users to know what was the baseline image and what was the actual image found.
The properties panel helps to capture the error messages and display it to the user. The user also has the option of interacting with DOM and see what objects were extracted during the run
Logs are a rich source of information on what happened underneath the UI. We provide test logs when the user runs the tests on our grid or a 3rd party grid. The option can be found in the in top section of editor
One of the most time consuming aspects of testing is after finding a bug, we need to report it to the developer with relevant information, to speed up the troubleshooting and fixing of issues.
With Testim.io you can do this with a single click with the help of our chrome extension. All the details related to the bug are automatically generated for you.
We put in a lot of effort to document most of the features of the tool in our User Documentation found under the “Educate” tab.
We also have detailed videos on how to troubleshoot your tests quickly
With the above features, Testim.io helps to create stable tests that are highly maintainable.
The below posts gives more in depth analysis of Testim in terms of different features that make authoring and execution of tests really simple and how to create reusable components that improves the extensibility of the tool
Artificial Intelligence (AI) and machine learning (ML) are advancing at a rapid pace. Companies like Apple, Tesla, Google, Amazon, Facebook and others have started investing more into AI to solve different technological problems in the areas of healthcare, autonomous cars, search engines, predictive modeling and much more. Applying AI is real. It’s coming fast. It’s going to affect every business, no matter how big or small. This being the case how are we as Testers going to adapt to this change and embrace AI? Here is the summary of different things you need to know about using AI in software testing.
Let’s summarize how the testing practice has evolved over the last 4 decades
In the 1980’s majority of software development was waterfall and testing was manual
In the 1990’s, we had bulky automation tools which were super expensive, unstable and had really primitive functionality. During the same time, there were different development approaches being experimented like Scrum, XP, RAD (Rapid Application Development)
From 2000, the era of open source frameworks began
People wanted to share their knowledge with the community
Started encouraging innovation and asking community of like minded people to help out in improving testing
Agile became a big thing, XP, Scrum, Kanban became a standard process in the SDLC
There were need for faster release cycles as people wanted more software features delivered faster
In the 2010’s, it was all about scale, how to write tests fast and find bugs faster
Encouraging other people to give feedback on the application. Free and Paid services
Cloud testing started
People started realizing they need more
Started to realize the problem of maintenance. How expensive it is to buy hardware and software for maintaining your tests
Then we have
I believe the Future will be about Autonomous Testing using Machine Learning and AI
Basics of AI
Let’s start by de-mystifying some of the terminologies related to AI
Artificial Intelligence (AI) is an area of computer science that emphasizes the creation of intelligent machines that work and react like humans
Machine Learning (ML) evolved from the study of pattern recognition and computational learning theory (studying design and analysis of ML algorithms) in AI. It is a field of study that gives computers ability to learn without being explicitly programmed
Deep Learning(DL) is one of the many approaches to ML. Other approaches include decision tree learning, inductive logic programming, clustering and Bayesian networks. It is based on neural networks in the human body. Each neuron keeps learning and interconnects with other neurons to perform different actions based on different responses
There are 3 types of widely used ML algorithms
Supervised Learning – We are giving the right training data (input/output combinations) for the algorithm to learn
Give bunch of e-mails and identify spam e-mails
Extracting text from audio
Fill out a loan application and find the probability of the user repaying the loan
How to make user click on ads by learning their behavior
Recommendation engines on Amazon, Netflix where customer is recommended products and movies
Amazon uses AI for logistics
Un-supervised learning – We give a bunch of data and see what we can find
Taking a single image and creating a 3D model
Reinforced learning – Based on concept of reward function. Rewarding Good/Bad behavior and making the algorithm learn from it. E.g. Training a Dog
Real life AI applications to visually see how it works
Weka is an open source project where they are using ML algorithms for data mining
What challenges can AI solve?
Let’s discuss the challenges the industry faced while transitioning to agile and what’s still remains a challenge:
How can we use AI to solve testing problems?
There are many companies taking multiple approaches to solve different problems related to software testing and automation. Testim.io is one such company
Testim.io uses Dynamic Locators, The Artificial Intelligence (AI) underneath the platform in real time, analyzes all the DOM objects of a page and extracts the objects and its properties. Finally, the AI decides the best location strategy to locate a particular element based on this analysis. Due to this, even if a developer changes the attribute of an element, the test still continues to run and this leads to more stable tests. As a result, the authoring and execution of automated tests are much faster and more stable.
One of the good practices of writing automated tests is creating reusable components that can be used in different parts of our test suite.
Why is this important?
Creating reusable components is important because it
Helps to increase the readability of the automated tests
Saves effort by not repeating the same set of steps in different parts of the tests
Any changes to the reusable step needs to be done only in one place and it is reflected throughout the tests, across different projects
Makes the automated tests more extensible
Testim.io helps to ensure Reusability by “Grouping” and “Parameterization”.
Any number of related steps can be grouped into one reusable component.
For Example – The “Login” scenario is one of the most commonly used steps in any application. The way we can create a reusable “Login” step would be to select the steps we want to group together and clicking on “Add new Group” as shown below
Our platform gives the option of testing application through various input combinations via Parameterization.
This can be achieved in various ways. One way to do this is to give all the input parameters we would need to test the application in the form of a JSON file in the Setup step (The first step of our tests) as shown below
Then add the variable names used in the json file in the appropriate fields of the step as show below
Another important aspect of automation is building your tests such that it is extensible.
Why is this important?
As the product and teams grow, there will be need to test more complex functionalities which would require building upon already existing tests. This being the case, the automation suites need to be simple, understandable and should be easy to add more tests to already existing test suites with low coupling and high cohesion.
For Example – Say we want to validate the “Select Destination” button from our previous examples. The way to do this would be.
Click on “Add custom action”
Give a name to the New Step and click on “Confirm”
Click on “PARAMS” and Select “HTML” for this example
Add Custom Code
The new step with Custom Code gets added to the list of already existing steps
The above features help to make the automation suite more reusable and extensible.
The below posts gives more in depth analysis of Testim in terms of different features that make authoring and execution of tests really simple and easy to maintain
Authoring and Execution of tests is an important aspect of test automation. Tests should be simple to write, understand and execute across projects. The automation framework or tool chosen should give the flexibility to record and playback tests as well as, write custom code to extend the functionalities of the automation framework.
This is where Testim.io can help you out. We follow a Hybrid Approach and make authoring and execution of tests really simple in such a way that both technical and non-technical members can collaborate and start writing automated tests quickly. This is achieved with the use of “Dynamic Locators”.
What are Dynamic Locators?
The Artificial Intelligence (AI) underneath the platform in real time, analyzes all the DOM objects of a page and extracts the objects and its properties. Finally, the AI decides the best location strategy to locate a particular element based on this analysis. Due to this, even if a developer changes the attribute of an element, the test still continues to run and this leads to more stable tests. As a result, the authoring and execution of automated tests are much faster and more stable.
As we can notice from the above image, the AI parses through all the DOM objects, lists them in the Properties Panel along with the rankings of each and every location strategy for that particular element. In this way, even if the attribute of an element changes, the AI can use a different location strategy from the already parsed list of DOM objects.
Thus, the user does not have to worry about flaky tests.
Some of the basic authoring and execution features Testim.io provides to its customers, are explained below.
How to create a test
We create a new Test by clicking on “Create New” or “New Test”
Recording and Saving a test
Once we click the “Record” button, we can record different user actions in our application. After recording different actions, click on “Stop Recording” button to finish recording our tests. Use the “Save” button to save the tests.
Validations and Assertions
Our platform helps to make validation of different attributes of an element and API’s really simple. We provide various options for users such as
Validate element visibility
Validate element text
Pixel level validation
API level validation
While each test is recorded, the platform takes screenshots of all the Pass and Failed results of each and every step. As a result, users find it easier to troubleshoot problems and understand what happens underneath the hood.
Feedback on each step
The user also gets feedback on each step in terms of whether the tests Passed or Failed by showing a “Green” or “Red icon” on the top left portion of each step as shown below
Testim.io provides the feature to label each and every test a user creates. There are 2 reasons why we may want to label a test
Helps to identify the reason the test was created in the first place
Helps to run tests with the same label all at once through our CLI feature
The way we create labels is by clicking on the “Label” button and either select an existing a label or create a new one.
At Testim.io, we took the effort to provide users with all the documentation they will need to use different features of our platform. Most of the answers about using our platform can be found by clicking on the “Educate” tab and Visiting our documentation site as shown below
With the above features, Testim.io helps to make the authoring and execution of tests really fast and simple for our users. Within a matter of seconds a user can record, replay and save the tests. This is surprisingly one of the most overlooked aspects of test automation and our platform takes care of it for our users.
The below posts gives more in depth analysis of Testim in terms of creating reusable components, extensibility and maintenance