Tag

shift left

Browsing

By: Sofía Palamarchuk for Abstracta

If you work in the software industry, you’ve most likely heard about the popular term, “shift-left testing.” With Agile practices like TDD, BDD, CI and DevOps becoming mainstream, “shift-left” is the answer to how testing fits in, and must be done in order for them to become a reality. Instead of taking a backseat during the development process, testing is planned in advance and begins earlier in the SDLC (therefore “shifts left”). It could even start before a single line of code is written! Making this shift changes the view of testing instead of traditional QA, it transforms into QE: Quality Engineering.

What Does Shift-Left Testing Look Like?

Thanks to the rise of automation, and the aid of tools that use AI and machine learning, testers have more time to dedicate to being more strategic about their work, instead of having their hands tied, running tests manually every day.

For testers to be successful today, they have to not only be great at testing, but also be engineers of the Agile testing process by collaborating with development and operations while analyzing quality during every stage of development:

Shift Left Testing

Shift-left testing activities include:

  • Testers helping developers implement unit testing
  • Planning, creating, and automating integration test cases
  • Planning, creating, and employing virtualized services at every stage and component level
  • Gathering, prioritizing, and processing feedback

Several process changes occur when teams shift left. Instead of a developer waiting weeks to add his or her code to the rest of the team’s code, it can be done every day, or even several times a day. Instead of manually performing all the tests, most are automated and run every day, or even several times a day. And, instead of detecting problems at the end, the team as a whole analyzes quality as the development progresses.

Not sure if it’s the right move for your organization? Here are some of the pros and cons of shift left testing.

PROS

LOWER THE COST OF TESTING & DEVELOPMENT

It’s well known that the sooner a bug is found, the cheaper it is to fix. One of the aims of Agile testing is detecting errors as soon as possible. With shift-left testing it’s possible to detect in real time, the exact moment in which an error was inserted into the system in order to resolve it in a timely manner. When testing is done with each build (especially during unit testing), the errors that are found are smaller, easier to detect and locate, and subsequently, less costly to fix.

INCREASE EFFICIENCY & QUALITY

With the increased levels of automation when shifting left, teams can benefit from:

  • Increased test coverage since more tests can be run in the same amount of time
  • More time for testers to focus on more challenging and rewarding tasks
  • Reduced human error rate
  • Monitoring performance over time
  • Code quality checks
  • Built-in security checks
  • Reducing issues in production that users may encounter

Beyond these benefits, being able to start testing sooner invariably results in a higher quality product, as testers are less rushed to find all the errors at the end, when there’s little time left to fix them.

COMPETE MORE FIERCELY

In today’s competitive technological landscape, the barriers to compete are minimal, so the best way to survive is to be able to move fast and defend one’s stature by innovating in iterations, which is possible thanks to Agile. As everyone can agree that it’s important to deliver software more quickly, it also mustn’t be rushed out the door (causing a possible backfire). Shift-left testing answers the problem of accelerating development without sacrificing quality.

Anotheryet less obviousbenefit of shifting left is that it can help businesses position themselves as an attractive employer to top talent. Because it is becoming more mainstream, with about two thirds of IT workers reportedly using Agile or leaning towards agile (according to a recent study by HP), it’s what the most forward thinking software professionals expect from their teams. Therefore, if you want to be an attractive employer or at least on-par with the rest, it is important to adopt the modern practices that both testers and developers want to master in order to stay relevant in today’s labor market.

Cons

EASIER SAID THAN DONE

For shift-left testing to be a success, an often drastic change in culture must occur first, which requires a team effort. Teams are usually set in their traditional ways of working, and when they consider shifting, they must consider how the methods, processes, skills, tooling, etc. will need to change. Even more important, what will need to happen to get all the roles within the organization to align properly?

RISK OF BOTTLENECKS

Yes, agile and shift-left aim to eliminate testing as a bottleneck, but it is true that agile teams can find themselves stuck waiting in a queue once all of the pieces come together in the performance and user acceptance testing phases, due to the complexity of environments and composite applications. One way to overcome this is to to utilize service virtualization. Service virtualization emulates the behavior of essential components that will be present in production, enabling integration tests to take place much earlier in development. This is how you can eliminate that key bottleneck, while also benefiting from eliminating errors earlier on. Along with service virtualization, there are several tools to setup automated systems and CI such as Jenkins.

A Worthwhile Undertaking

In the end, shift left testing is certain to have pros that outweigh its cons. Testers will find themselves delegating some of their work to developers and assigning them more testing activities. In mature teams, the testers become “coaches,” training developers on how to write better code, avoid bugs, and own unit testing. The advantage of this is that the tester who used to be busy, bogged down by writing test cases, now has time to delve deeper into the product, working on business cases, penetration testing, performance testing, implementing smarter testing solutions that use artificial intelligence like Testim.io, and so on. This sharing of the responsibility over testing leads to a higher level of achieved quality, as more of the bases get covered, quicker!

What do you think? Still not on-board to shift left testing? Or have you managed to do so already?

About the Author: Sofía Palamarchuk

Sofia Palamarchuk

Sofía is the Chief Executive Officer and Chief Product Officer of software testing services company, Abstracta. With a B.S. in Computer Engineering, Sofía started working in application performance optimization, system monitoring and load testing for the corporate sector for many years. With a solid background in performance tuning and automation, Sofía has become a business development leader and is responsible for managing all aspects of Abstracta’s US operations as well as its mobile testing tool, Monkop.

One of the most important factors related to automated tests is Maintenance. A lot of effort is spent on maintaining the tests than writing actual tests.  A recent study suggested about 30% of testers time is spent on maintenance.This leads to wastage of valuable time and effort by the resources, which they could have rather spent on testing the actual application.

Imagine a world where the software can maintain tests without human interaction? This world has become a reality with Testim.io. We use Artificial Intelligence (AI) underneath the hood, which provides self-healing maintenance i.e problems are detected by the AI and automatically fixed without human intervention.

Testim.io also help to speed up the maintenance of tests by providing the follow features within our platform-

  1. Version Control

At any given time, it is important to have logs of what changes were made to a particular test. This way we can always revert back to an older version of test as and when required. Our platform provides this functionality by showing all the version history by going to the Properties panel of the setup step and clicking on “See old revisions”

  1. Branching

At Testim.io, we firmly believe in the “Shift Left Paradigm” where Development and Testing must start in parallel as early as possible in the software development lifecycle. Keeping this in mind, we provide the functionality to teams to create separate branches for each team member and work on the same projects and tests. This way, no one can overwrite the changes of the other team members and teams can work on the same code base at any instant of time

In our platform, we just need to select “Fork” to create a new branch and we can also switch between existing branches


        3.  Scheduler

Users have the option of scheduling their tests. This helps to run the tests automatically at a certain day and time without any manual intervention. We can also get notified via email in case of any errors

 

Troubleshooting

As testers, we spend considerable amount of time troubleshooting issues. To help in troubleshooting, our platform offers different options to the user to narrow down the scope of the problem. These options  are as follows-

  1. Screenshots

The screenshot feature explained in the “Authoring and Execution” section helps users to know what was the baseline image and what was the actual image found.

  1.   Properties Panel

The properties panel helps to capture the error messages and display it to the user. The user also has the option of interacting with DOM and see what objects were extracted during the run

  1. Test Logs

Logs are a rich source of information on what happened underneath the UI. We provide test logs when the user runs the tests on our grid or a 3rd party grid. The option can be found in the in top section of editor

  1. Bug Reporting

One of the most time consuming aspects of testing is after finding a bug, we need to report it to the developer with relevant information, to speed up the troubleshooting and fixing of issues.

With Testim.io you can do this with a single click with the help of our chrome extension. All the details related to the bug are automatically generated for you.

  1. Documentation

We put in a lot of effort to document most of the features of the tool in our User Documentation found under the “Educate” tab.

We also have detailed videos on how to troubleshoot your tests quickly

Troubleshooting Part 1- Element is not visible

Troubleshooting Part 2 – Element not found

Troubleshooting Part 3 – Timing issues

Troubleshooting Part 4 – Issues related to mouse hover

With the above features, Testim.io helps to create stable tests that are highly maintainable.

The below posts gives more in depth analysis of Testim in terms of different features that make authoring and execution of tests really simple and how to create reusable components that improves the extensibility of the tool

https://blog.testim.io/bringing-simplicity-to-authoring-and-execution-of-automated-tests/

https://blog.testim.io/how-to-make-reusable-and-extensible-code-using-testim-io/

 

Should the goals and principles of DevOps be the same for every company? Is it possible for every company to reach CI/CD? 

Software delivery teams come in all different shapes and sizes. Each team has its own DNA that has organically evolved through generations of diverse experiences, skill sets, tools, technologies and sets of processes.

So, if every project team looks entirely different than the next, where do you start?

RESERVE YOUR SEAT for  this roundtable discussion where we will cover different types of organizational situations and the common as well as unique challenges each situation may face in their DevOps transition. We will review ways to address specific inefficiencies in each situations development process as well as how to plan accordingly and minimize risk along the way.

Date: Wednesday, November 15
Time: 9:00am PT

We will cover:

  • How to transition to DevOps and align your development, testing and release processes
  • How to plan for your DevOps transition based on your current operational reality
  • A breakdown of common as well as unique DevOps obstacles
  • A method for systematically resolving your development issues to enable test automation

 RESERVE YOUR SEAT and bring your questions for the panelists:

  • Tanya Kravtsov is the Director of QA at Audible. She is building a new QA organization to support innovative web product development at scale. Previously, as head of automation and continuous delivery at ROKITT, senior QA manager at Syncsort, and VP at Morgan Stanley, Tanya focused on quality, automation, and DevOps practices, working with internal and external customers to transform development and testing processes. Tanya is passionate about process automation, continuous integration, and continuous delivery. She is a founder of the DevOpsQA NJ Meetup group and a frequent speaker at STAREAST, QUEST, and other conferences and events. Follow her on Twitter @DevOpsQA.
  • Bob Crews is the President of Checkpoint Technologies. He is a consultant and trainer with almost three decades of I.T. experience including full life-cycle development involving development, requirements management and software testing. He is also the President of the Tampa Bay Quality Assurance Association. He has consulted and trained for over 200 different organizations in areas such as effectively using automated testing solutions, test planning, implementing automated frameworks and developing practices which ensure the maximum return-on-investment with automated solutions.

Practical steps to shift left

Last month we published “Do you know how much your quality costs?” where we discussed the importance of shifting left and shortening the feedback loop as it relates to the cost of quality. According to Bank of America, discovering a bug in development would cost about 5 hours to fix and $390. Discovering a bug in QA cost about x30. 127 hours to fix, due to to the time it takes to reproduce, the back and forth between dev and QA and the need to trace back the code. Fixing a bug in production – Estimated at $72,000. Shifting left allows testing close to development as possible and a fast and continuous feedback loop.

Direct cost to find/fix a defect

Four key ingredients to achieving shift left:

  1. Test automation – Traditional QA processes would result in months of testing before you get feedback. A productive tester can get about 20 tests done in a single day. Assuming your test suite includes dozens of tests there are two ways to get the feedback loop down to minutes: Hire hundreds of testers or automate your tests. Automated tests don’t get tired, take days off, go out for lunch etc.
  2. High concurrency – Shifting left means giving near real time feedback i.e minutes. Running dozens of tests, each taking roughly 30-60 seconds, with the expectations of getting the results back in, say 5 min., can only be done if you are running your tests in parallel. The higher your level of parallelism the shorter the feedback loop is. Running your tests in parallel required a grid, a cloud based infrastructure that includes virtual environments allowing you to spin in and out different browsers.
  3. Build-test-deploy automation – Shifting left means you are optimizing the handover across the different stages of the development process. The development cycle includes dozens of handovers and human intervention can always delay that process hence you want to automate the handover using Continuous Integration (CI) servers. CI servers monitor events through your development cycle, such as a new build, and can be programed to trigger a set of actions once a certain event occurred, such as triggering your automated tests.
  4. Suites – It is best practice to optimize your quality assurance processes by segmenting your tests into suites and running different suites at different stages of your development cycle. For example you full regression might include thousands of tests. Running all those tests every time a develop push code would be time and resource consuming. If 2-3 key scenarios failed, there is no point in waiting for the entire suite to be over. You might want to create a small subset of tests, composed of your high priority flows, and run that “sanity” suite often, while running the more extensive suites once if sanity suite ran successfully.

 

Guide: Evaluating UI and Mobile Automated Testing Solutions

Do you how much quality costs? Have you ever calculated it? If you do, you’ll learn that you spend thousands of dollars and hours of productivity on every bug. From the cost of labor, to the cost of deployment, to the cost of finding the bugs in production, no bug fix is free.

Many companies and academics, from IBM to Microsoft to large banks have performed studies to quantify the cost of finding bugs. It’s been well known for many years that the costs go up the closer to production you go, but the actual numbers are a bit staggering.Their findings show the dramatic financial difference between discovering a bug in the wild, and finding it during development.

  • Fixing a bug in production – Very expensive. These take time to fix, and while repairs take place, your business experiences pain. Your teams are working around the clock to fix something that also requires your developer to rework something they wrote months ago and forgot about.
  • Fixing a bug in QA – Somewhat expensive. The QA person has to report the bug. A senior manager has to determine who to be assigned to this. The developer has to step out of his daily tasks, sometimes impacting the current sprint, attend to the bug, reproduce it, and return to edit some code from the last sprint, which the developer has likely already begun to forget about.
  • Fixing a bug in development – Least expensive. takes a few minutes for the developer to fix. If you have an immediate feedback loop, your developer will get feedback minutes after pushing code. If a test fails, it is still within the current sprint and the developer should recognize the code and its implications as everything is still fresh in their mind. The bug didn’t escalate and the likelihood of on time release is much higher.

Shift left is the term coined for a software development cycle that includes a constant and immediate feedback loop. You may have heard stories about how Facebook has put a billion dollars worth of engineering into creating this feedback loop. The company even wrote a PHP-to-C compiler called HipHop to speed up the loading of its test environments.

If Facebook is willing to spend a billion dollars to get that feedback loop, you can spend a few weeks ensuring your own loop is as fast and reliable as possible. This means fast builds, fast test batteries, and short amounts of time between developer compiling, and developer testing.

This requires certain infrastructure that we’ll get to in a future post but organizations that have deployed it, have seen decrease in cost of quality and increase in on-time delivery. You may even have to disassemble your build pipeline to replace a few slower moving items.

Your goal, here, should be one minute or less between build and test. This may seem impossible, or it may already be the way things work for your team. This can require a lot of work, or a little depending on the existing systems. No matter the environment, there is one fundamental truth for all software development: the faster the developer can see their results in a running binary, the faster they can fix problems and the lower the cost of overall quality.

Want to know how to shift left? Stay tuned.

Sources:

Be smart & save time...
Simply automate.

Get Started Free