Author

francis@testim.io

Browsing

Introduction

We work hard to improve the functionality and usability of our autonomous testing platform, constantly adding new features. Our sprints are weekly with minor updates being released sometimes every day, so a lot is added over the course of a month. We share updates by email and social media but wanted to provide a monthly recap of the month’s latests enhancements, summarizing the big and small things we delivered to improve your success with the Testim.

API Testing

What is it?

Do you need to validate data that appears in the app against data from your back-end? Do you need to extract data to use in your tests? Then we have the feature for you.

We provide two types of API steps:

  • API action – Will be used when in need to get data and use it for calculation, or to save it for later use in the test.
  • Validate API – Will be used to validate an element against the API result data.

Why should I care?  
This capability makes it easy to do UI and API simultaneously. No need to toggle between do different systems nor integrate results. Author a UI test and automatically author an API test to ensure the application under test is working correctly. Read more

API Testing

Debug Step Parameters

What is it?

We’ve learned that enriched debugging can be a huge time saver. Debug the console errors and network errors automatically and see the failed steps in the DOM. You can see in each step all the parameters used during the run.

Why should I care?
This feature is helpful when you want to debug your runs and need to figure out which parameters were used in each step. Learn more

Debug step parameters

Company and Project Structure

What is it?

Large enterprises need the ability to manage multiple projects and various groups inside their organization. This feature makes it easy for admins to grant individual users access to specific projects. Earlier this year we’ve added support for a company structure with a company owner managing permissions and owners per project.

Why should I care?
This gives you more flexibility in managing the different groups inside your company and allows control over who has access to which projects. Learn more

Testim Project Management

Customers have access to these features now. Check it out and let us know what you think. If you’re not a customer, sign up for a free trial to experience autonomous testing. We’d love to hear what you think of the new features. Please share your thoughts on Twitter or Facebook.

 

DevOps is a term that refers to the evolving professional movement that supports a collaborative relationship between IT Operations and development, leading to the fast flow of planned work that has high deploy rates, and at the same time increasing the stability, resilience, security, and reliability of the production environment. DevOps came into view in 2009 when a squad of Belgian developers hosted DevOps Days, which supported collaboration between developers and operational staff.  As the cloud is in buzz these days, most developers have readily accepted it that ultimately impacts Ops. DevOps and cloud are closely connected to each other that this post maps out in detail.

DevOps is great as a concept, but it is difficult to execute in reality. As DevOps has grown widespread and differentiated, it has become critical to know how to reach the next level of advancement and truly develops a DevOps driven organization. On this note, have a look at IDC’s recently published DevOps 2.0 MaturityScape.

There is no doubt in saying that DevOps builds a closer working relationship between dev and ops teams, but more significantly it necessitates collaboration with groups spanning QA & testing, security and architecture. To sum up ,  this recommends, DevOps is likely to become everything – which according to my view is right –  DevOps is way better when it integrates cross-functional teams with the aim to deliver high-touch customer experiences quickly.

Implementing and accelerating DevOps are an important topics of discussion; and within this automation, the role of QA & testing, and how to align with security driven DevOps are key considerations. There is a rise in experimentation and utilization of containers and microservices; and more investments in hybrid cloud management, consistent integration/continuous delivery, automated code testing, application performance management have become important to promote the development of a DevOps driven organization. The idea, of course, is to know what investments will support fastening of your organization’s DevOps journey.

As per a recent stats, by the end of 2018, 90% of IT Projects will depend on the principles of experimentation, speed, and quality.

Though operational models and technology investment strategies are still progressing, we begin to observe a widening gap between those businesses in experimentation mode contrary to those that know the benefits and quickly deploying DevOps to speed up a business change.

IDC’s DevOps Conference will make you understand how to accelerate and navigate the DevOps journey and is the right opportunity to know more about this.

How DevOps dictates a new approach to the cloud.

1. Cloud is not always a great solution for systems

DevOps and Cloud offer various potential benefits for companies, but many of them have larger investments in infrastructure, for instance, a mainframe. In the 1990s, the “Death of the Mainframe” has been touted, but the general fact is there is no business case to change many of the systems they host with Cloud-based solutions. The value of these systems can be increased with the help of DevOps techniques and future-proofed to make sure they do not turn into a bottleneck in difficult end-to-end customer journeys.

2. Cloud Native is not many organizations’ first-term goal

Most of our clients are planning to migrate to Cloud, and plan to do so over a two year period, but few have made a move now. Cloud migration is a continuous process, and the decision to do so should depend on business requirements. This involves whether the enterprise shifts entirely to Cloud or selects a hybrid, whichever option they pick, they have to deliver high business value from their current systems up to the point they are replaced or migrated.

3. DevOps tools are interchangeable

DevOps allows businesses to centralize their working practices and culture to gain transformational speed along with the quality of delivery. The tools that support this are interchangeable throughout system types, providing long-lasting benefits no matter how business systems are operated. While simply migrating to Cloud can allow enterprises to scale, it does not increase delivery speed or quality by itself.

4. To maximize the benefits of Cloud in the future

To make the most out of Cloud, businesses need to accept new ways of working. Different services available from main Cloud providers are “DevOps” tools in their right, for instance, AWS CodePipeline. But as mentioned in the previous point “tools support working practices and culture”. With the right working practices and culture in place, you can make the optimum use of tools.

5. Can the Cloud run without DevOps?

As mentioned above, cloud computing offers various benefits. These benefits are particularly useful when strengthened by the DevOps approach, but does the cloud have much to provide if we take out DevOps?

Undoubtedly, “yes”—but not much. You can still use machines in a flexible and scalable manner without DevOps. You can also take advantage of the pay-per-use business model, in which you have the authority to turn off unused resources to stop unnecessary spending.

Though both the cloud and DevOps functions well separately, they are stronger when they are used jointly. You will need to use the cloud model for DevOps to achieve its full potential for flexibility and agility. Similarly, when using the cloud, knowingly or not, you will automate processes and store configuration in version control before you experience that you are really following DevOps practices.

Netflix’s Simian Army is a great example to know about the success that can result from the integration of cloud and DevOps. You might consider this is not right for your company as it follows “on-premises only deployment policy.” Nevertheless, there is a way to get benefited from the cloud approach.

Bottom Line

DevOps and the cloud help businesses to transform user interaction with software delivery strategically. Enterprises no longer struggle adaptation but instead improve adaptability. Consider the cloud as an instrument, then take DevOps as the facilitator. If DevOps is a means, then the driver is the cloud. DevOps and the cloud together foster the development and IT leaps from ‘wait for the right time’ to ‘active frequency’. If you want to learn more about their relation then you can take DevOps training from experts to master its concepts.

Danish Wadhwa


Author bio:

Danish Wadhwa is a strategic thinker and an IT Pro. With more than six years of experience in the  digital marketing industry, he is more than a results-driven individual. He is well-versed in providing high-end technical support, optimizing sales and automating tools to stimulate productivity for businesses.

2017 has been a phenomenal year for Testim, full of exponential growth and product enhancements.

We are proud that so many teams across the globe are utilizing our products to support their software quality initiatives. We want to thank our customers for helping us in making 2017 a huge success! As the leading provider of autonomous testing for Agile teams we would like to recap some of our shared accomplishments.  

100+ Customers Worldwide– Earlier this year, we earned the trust of our 100th customer. Our customers span across a dozen countries, executing more than 1M automated tests per month. We are thrilled to support their CI/CD efforts; reducing risk, faster time to market and releasing higher quality software.

Lightspeed Venture Partners Invests $5.6 Million in Testim– Lightspeed, a Silicon Valley-based early stage venture capital firm focused on accelerating disruptive innovations and trends in the Enterprise and Consumer sectors invested in Testim this past year. This is Testim’s second round of funding in 12 months. The funds will support Testim’s mission to help engineering teams make application testing autonomous and integrative to their agile development cycle.

Testim Delivers Many Highly Requested FeaturesOur customers are the ones that make the Testim community so special and inspire us to innovate. Some of the years most requested customer features include:

We want to express our utmost appreciation and gratitude to our customers, partners and peers in the industry for their continued support. We are thrilled to welcome 2018 and look forward to even more shared success together. Keep your eye out for mobile native, company structure, advanced reporting and much, much more.

Introduction

We work hard to improve the functionality and usability of our autonomous testing platform, constantly adding new features. Our sprints are weekly with minor updates being released sometimes every day, so a lot is added over the course of a month. We share updates by email and social media but wanted to provide a monthly recap of the month’s latests enhancements, summarizing the big and small things we delivered to improve your success with the Testim.

Test Reruns

What is it?

An easy and fast way to rerun a test with the exact parameters used in another run.

Why should I care?

When running large suites, many times you want to rerun a smaller subset of those tests that failed. Since tests fail for a number of reasons, this new capability allows you to test temporary errors or fixes. Reruns let you run a test with the same parameters as a previous test run with a click of a button. This will include all dynamic parameters, from the CLI, Test Data, and parameters exported from tests that ran before this one (via exportsGlobal). Learn more

test suite rerun

To Reuse or not to Reuse

What is is?

Reusing actions is one of the basic principles of programming. Testim always supported reuse, and now we push toward even more developer best practices. For every new group, API call, or custom code step that you create, Testim will prompt you for a (meaningful) name, and whether you would like to make this step shareable or not.

Why should I care?

This feature will save you time since you can author the test once and call it to be used in any suite of tests. This makes it faster to author tests when making changes to a shared group, but do not want it to affect other tests in the suite. You can find more information on Reuse in our docs

test reuse

Mobile Web

What is it?
Testim now supports the authoring and execution of tests for mobile web.

Why should I care?  
The number of mobile devices has suppressed the number of desktop computers. We all consume content through our mobile devices and users expect your application to provide superior experience regardless of the medium. Responsive websites look and behave differently than desktop hence require different tests. Learn more

mobile web

Customers have access to these features now. Check it out and let us know what you think. If you’re not a customer, sign up for a free trial to experience autonomous testing. We’d love to hear what you think of the new features. Please share your thoughts on Twitter or Facebook.

Thank you to everyone who participated in our round table discussion on Choosing the Right DevOps Strategy: How far left should I shift? We had a solid turnout with lots of great questions from the audience. If you missed the live event, don’t worry…

You can watch the recorded session any time:

Tanya Kravtsov, Director of QA at Audible, and Bob Crews, President of Checkpoint Technologies shared their lessons learned from four different types of enterprise DevOps transformations.

They covered:

  • Practical suggestions for assessing your operational reality
  • Major pitfalls in DevOps transformations and how to avoid them
  • How to organize your team to increase productivity
  • Impact on development processes, ownership, tool stack and KPIs

Some of the audience questions they answered:

  • How do you ensure that security requirements are always considered in DevOps?
  • Which KPIs are important to track to ensure successful DevOps transformation?
  • What is an acceptable amount of technical debt? How do we know when it becomes a problem?
  • What are some metrics for measuring risk?

There were several questions that we were not able to address during the live event so I followed up with the panelist afterwards to get their answers.

Q: Why was continuous deployment listed twice on the diagrams?

Francis: Great catch. That was actually a typo, a marketing design bug slipped through to production… The “Continuous Deployment” label shown to the upper left, within the square encompassing the “Build” and “Test” phases in both the upper (Continuous Delivery) and bottom (Continuous Deployment) cycles, should have been labeled “Continuous Integration.”

Q: In reference to CD /CI what are the main benefits of doing smoke testing after production deployment? And, what could be an example of smoke testing?  

Tanya: Smoke test validating the most critical functionality of the product should be executed after every deployment to test the staging and production environments. This will help you uncover any deployment related issues and rollback the release if necessary without impacting the customer.  

Bob: Examples of “smoke testing” would be quick test to check for and validate successful login with valid data, login error message with invalid data, checking for broken links, tests to quickly validate the number and state of objects on a webpage, etc. These are very simple, but important tests. Typically they are run in production as one last check since many, perhaps most, organizations do not have a test environment that is an EXACT duplicate of production.

Q: Are automated tests the entire QA solution across all these organizations? Do the panelists see any value in manual, exploratory QA testers?

Tanya: Automated tests are an important component of the Continuous Delivery solution. It enables exploratory QA testers to focus on what they do best and actually test the new features of the product  by eliminating the need for manually running the same regression tests every time new code is committed.  

Bob: Absolutely. I believe it’s critical to carve out an approach to perform some quick manual, exploratory tests. These can be done as a separate initiative, apart from the validation taking place during these phases.

Q: What approach we should follow to test DevOps solutions?

Tanya: Any scripted DevOps solutions should be treated just like any code,  accompanied by unit and integration tests which run continuously in a test environment before the changes get promoted to the mainline.  In addition,  closely tracking KPIs discussed in the Webinar will alert us to any issues with the current solutions such as build up of technical debt or reduction in quality,  which should be acted upon immediately.   

Bob: I would need a little more information in order to provide a good answer. Knowing the current state of the organization’s testing approach, their objectives, the tools already in place, the skill set of current team members, etc. are just a few questions I would have before being able to provide any insight. If starting from scratch I would say start with becoming entrenched in an Agile approach. If you are already Agile then the next step would be to look at tackling Continuous Integration. Please feel free to connect with me Linkedin and I would be happy to chat more about your question.

About Tanya Kravtsov
Tanya Kravtsov is the Director of QA at Audible. She is building a new QA organization to support innovative web product development at scale. Previously, as head of automation and continuous delivery at ROKITT, senior QA manager at Syncsort, and VP at Morgan Stanley, Tanya focused on quality, automation, and DevOps practices, working with internal and external customers to transform development and testing processes. Tanya is passionate about process automation, continuous integration, and continuous delivery. She is a founder of the DevOpsQA NJ Meetup group and a frequent speaker at STAREAST, QUEST, and other conferences and events. Follow her on Twitter @DevOpsQA.

About Bob Crews
Bob Crews is the President of Checkpoint Technologies. He is a consultant and trainer with almost three decades of I.T. experience including full life-cycle development involving development, requirements management and software testing. He is also the President of the Tampa Bay Quality Assurance Association. He has consulted and trained for over 200 different organizations in areas such as effectively using automated testing solutions, test planning, implementing automated frameworks and developing practices which ensure the maximum return-on-investment with automated solutions.

Should the goals and principles of DevOps be the same for every company? Is it possible for every company to reach CI/CD? 

Software delivery teams come in all different shapes and sizes. Each team has its own DNA that has organically evolved through generations of diverse experiences, skill sets, tools, technologies and sets of processes.

So, if every project team looks entirely different than the next, where do you start?

RESERVE YOUR SEAT for  this roundtable discussion where we will cover different types of organizational situations and the common as well as unique challenges each situation may face in their DevOps transition. We will review ways to address specific inefficiencies in each situations development process as well as how to plan accordingly and minimize risk along the way.

Date: Wednesday, November 15
Time: 9:00am PT

We will cover:

  • How to transition to DevOps and align your development, testing and release processes
  • How to plan for your DevOps transition based on your current operational reality
  • A breakdown of common as well as unique DevOps obstacles
  • A method for systematically resolving your development issues to enable test automation

 RESERVE YOUR SEAT and bring your questions for the panelists:

  • Tanya Kravtsov is the Director of QA at Audible. She is building a new QA organization to support innovative web product development at scale. Previously, as head of automation and continuous delivery at ROKITT, senior QA manager at Syncsort, and VP at Morgan Stanley, Tanya focused on quality, automation, and DevOps practices, working with internal and external customers to transform development and testing processes. Tanya is passionate about process automation, continuous integration, and continuous delivery. She is a founder of the DevOpsQA NJ Meetup group and a frequent speaker at STAREAST, QUEST, and other conferences and events. Follow her on Twitter @DevOpsQA.
  • Bob Crews is the President of Checkpoint Technologies. He is a consultant and trainer with almost three decades of I.T. experience including full life-cycle development involving development, requirements management and software testing. He is also the President of the Tampa Bay Quality Assurance Association. He has consulted and trained for over 200 different organizations in areas such as effectively using automated testing solutions, test planning, implementing automated frameworks and developing practices which ensure the maximum return-on-investment with automated solutions.

Our client is the market leading data authority for the hybrid cloud. They provide a full range of hybrid cloud data services that simplify application management and data across cloud and on-site environment to accelerate digital transformation. Their solution empowers global organizations to unleash the full potential of their data to expand customer engagement, foster greater innovation, and optimize their operations.

The Challenge – Achieve CI/CD

Their application was complex and included many user flows, requiring the creation of thousands of functional tests with the goal of shifting left. In addition, they needed to test as close to development as possible.

Selenium was their first choice and a team of over a dozen testers was assembled to begin the task. After a few months it became apparent that it was going to take more time and man power to achieve their goal of CI/CD. The tests were complex and took three days to author. To make things worse, the tests would often break, leading the team to spend extra time maintaining and fixing the tests. Shifting left would require teaching the developers Selenium as well.

The lack of tools to troubleshoot a failed test (e.g screenshots to compare, details error messages pointing to the right step, test history or the parameters over the flows) led to long time-to-resolution involving a number of team members. A lot of time was wasted, not only trying to figure out why the test failed, but to also explain the discoveries to the developers.

Maintaining tests took a lot of my time. When developers run tests that fail it becomes more of a
distraction
than confidence in bug prevention.Both groups had to stop what they are doing to figure
out if its the functionality or if it’s the test. We found ourselves spending more time trying to stabilize
the tests than actually testing.
” – Company QA Manager

The Solution

Testim’s solution that uses machine learning for the authoring, execution and maintenance of automated test cases was implemented. With Testim they were able to capture the scenarios in minutes, as well as complement that with JavaScript syntaxes. The team was able to spend more time creating new tests, and validating the status of their application.

Today we have hundreds of tests running on every pull, giving the developers feedback to their code
within minutes. The developers themselves easily update test scenarios so QA can focus on increasing
coverage. We also significantly reduced our cost of quality: The rich information we get allows us to
reduce the time to troubleshoot by 80%.

The Results

As a rapidly growing enterprise, they needed a way to optimize processes through automation. This plays a significant role in their move to agile development. Within a couple months of using Testim, The team was able to create hundreds of their UI tests scenarios. Today, they are proud to say that they are authoring tests in under an hour compared to the three days it was originally taking.

Before Testim it would take 3 days to author a single test in Selenium, now even for the less
experiencedtester, writing tests takes under an hour, developers can update tests on the fly and
figure out where the tests are failing without any additional help.

Now, troubleshooting a failed tests takes a fraction of the time. There is an indication on which step failed, including JS syntaxes, screenshots comparing prior runs, and access to the DOM with detailed error messages.

However, the biggest impact was the reduction on maintenance. Tests are stable and trustworthy so when a test fails the user knows it is either due to a bug or the test requires a change in the flow. The team focused most of its time on increasing coverage knowing that the tests adapt to UI changes by the development team.

Today, we are proud to say that they are fully CI/CD, testing on every code push, and running thousands of tests every day.

Introduction

We work hard to improve the functionality and usability of our autonomous testing platform, constantly adding new features. Our sprints are weekly with minor updates being released sometimes every day, so a lot is added over the course of a month. We share updates by email and social media but wanted to provide a monthly recap of the month’s latests enhancements, summarizing the big and small things we delivered to improve your success with the Testim. We’d love to hear your feedback.

Customers have access to these feature now, check it out and let us know what you think. If you’re not a customer, sign up for a free trial to experience autonomous testing.

Group Usage Across Tests

What is it?

Groups allow you to create reuseable flows instead of rewriting them again and again. Group usage allows you to quickly filter the tests that are using a specific group.

Why should I care?  

Whether you make changes to a specific group or have a number of tests that failed and want to trace the commonality among them, group usage allows you to quickly filter the relevant tests as well as their results. This feature has been sought out by a large number of our customers and is already getting love from some our of our users. Read more here

Scheduler –  Email Notifications

What is is?

Easily schedule tests to run on nightly build, or monitor your production app, make sure you’re notified when it fails.

Why should I care?

Plan your schedule and choose your use case:

  • Monitor – Will run the tests on a set interval. Use this option to monitor the health of your application and alert when your service is down.
  • Nightly run – Schedule the tests to run at certain days of the week and time of day. Use this option to automatically trigger test runs such as nightly regression. By the morning you’ll have an email linking to the failed test results.

Read the docs to learn more

Email Scheduler
Email Scheduler

Mobile Web Pilot Program

What is it?

Testim now supports the authoring and execution of tests for mobile web.

Why should I care?  

The number of mobile devices has suppressed the number of desktop computers. We all consume content through our mobile devices and users expect your application to provide superior experience regardless of the medium. Responsive websites look and behave differently than desktop hence require different tests.

We are currently operating a closed beta for a selected number of customers. Reach out to your Testim representative if you are interested in taking part in the program.

We’d love to hear what you think of the new features. Please share your thoughts on Twitter or Facebook.

As many yoga instructors do, they encouraged students to find balance. It’s an effective bit of advice in that it puts the onus on the practitioner. It’s also a popular concept for software testing, as industry experts often recommend that software teams find the balance between automation and manual testing practices. Another similarity is the trajectory of their adoption rates. One does not dive into yoga, but slowly adopts it over time until it becomes a part of their daily routine. The same goes for test automation: you can’t expect to start automating everything from scratch. Good test automation is the culmination of work over time.

Why not all manual?

Manual testing is fine, and was once the status quo, but with high adoption of Agile and continuous delivery methodologies, it just doesn’t scale. For software development, every enhancement or new feature that’s rolled out for an app must have a corresponding set of tests to ensure the new functionality works and does not break the code from the previous version. In this regard, to check all of the file directories, database, workflows, integrations, rules and logic manually would be extremely time consuming.

For companies looking to improve time to market and increase test coverage, automation provides significant advantages over manual testing. A typical environment can host thousands of test cases of varying complexity to be executed effortlessly and nonstop. As a result, automation dwindles the time required to perform mundane, repetitive test cases from days to hours and even minutes if you run in parallel. That inherent velocity increase is what attracts teams to automated testing.

Why not all automation?

The benefits of automation are obvious. Automation provides the ability to quickly react to ever-changing business requirements and generate new tests continuously. Is it reasonable then to assume that the more you automate the more benefits you reap? Then why not just automate everything?

The short answer, its very difficult. According to the World Quality Report 2017-18 Ninth Edition, these are the top challenges enterprise are facing in regards  to test automation.

Traditionally you would automate only once your platform is stable and the user flow is pretty consistent. This was primarily due to the amount of time it took to write a test, sometimes hours and even days. Plus the amount of time it took to update and maintain tests as code changed. The ROI was not justified up to the point your user flow is stable.

Feedback from our customers such it has changed. We recently heard a number of our clients who automate everything and even as early as wireframes are ready. We reduced authoring time by about 70% of the time and maintenance time by about 90%. Creating a user flow now takes a few minutes with Testim and so is updating the user flow. Reducing the time per test to a few minutes instead of hours and days makes it much more affordable to automate, providing immediate ROI from feedback loop benefits.

So should you strive to 100% automation? Well no but you can get to 80%-90% which was unheard of until recently. There are still scenarios that only humans can do such as Face ID. There are also other aspects of your Software Development Lifecycle (SDLC) that automation is dependent on.

Summary

There is an ongoing struggle to keep up with the rate dictated customer pressure and competition to produce on a continuous basis. And not just produce for production’s sake, but a quality product that’s been thoroughly tested. That’s why we created Testim…

To help software teams:

  • Increase coverage by 30% in 3 months
  • Reduce number of missed bugs by 37%   
  • Increase team productivity by 62%
  • Reduce regression testing costs by 80%  
  • Speedup testing schedules by 3 months

Sources

  1. https://techbeacon.com/world-quality-report-2017-18-state-qa-testing
  2. https://dzone.com/articles/automated-vs-manual-testing-how-to-find-the-right
  3. https://blog.qasource.com/the-right-balance-of-automated-vs.-manual-testing-service
  4. http://sdtimes.com/software-testing-is-all-about-automation/#sthash.5Idkps2O.dpuf
  5. http://sdtimes.com/report-test-automation-increasing/#sthash.HlHQmD2l.dpuf
  6. https://appdevelopermagazine.com/4935/2017/2/8/test-automation-usage-on-the-rise/
  7. https://techbeacon.com/devops-automation-best-practices-how-much-too-much

Thank you to everyone who participated in our round table discussion on Cracking Your Test Automation Code: The Path to CI/CD. We had a great turnout with lots of solid questions from the audience. If you missed the live event, don’t worry… Just click here to watch the recorded session. There were several questions that we were not able to answer during the live event so I followed up with the panelist afterwards to get their answers.

Q: How do you implement CI/CD without using any tools?
Bas Dijkstra: Without any tools at all it will not be possible to do CI/CD. Maybe technically there are ways to do it, but it will be massively inefficient. You will need tools that help you do version control, building software and deploying it on a specific environment. If we just look at doing CI/CD without testing  tools, that is possible in theory, but you will need to do the required amount of checking by hand anyway to make sure that the software you delivered has a certain level of quality. Doing that manually will very likely be less efficient and act as a bottleneck in the process.

How can I improve backend testing of a mobile app beyond unit testing?
Oren Rubin: You can expand beyond the unit tests by adding API testing. This is also considered integrations tests, as you testing several units as one big unit. Consider that one big state that you need to initial, and the API calls (either REST, SOAP, etc) are like function calls, some getters, some setters which modify the state.

Q: What are the things we need to cover during API tests?
Bas Dijkstra: In short: connectivity (can I talk to the API at all?), syntax (is the data returned formatted correctly, are headers and status codes OK?), semantics (is the data returned the data I expected?), functionality (does the API call have the required side effects (data stored, processes triggered, etc.), as well as no unintended side effects), performance and security (authorization, authentication).

How should one avoid overlapping in functional and unit test automation?
Oren Rubin: It’s easy to overlap the two, but it’s almost unavoidable as each unit is tested on it’s own. They are all tested via the integration tests, which usually doesn’t care about a specific unit but mostly on part of a user story/flow. Each action (e.g. click) can drive a lot of units and check them and the integrations. Remember that E2E tests come at a high price, so choose them wisely.

Is it good to separate manual test cases from automated test cases?
Bas Dijkstra: Yes. Especially since translating your ‘manual’ test cases (where you’ll likely check a number of things in the process) to automated test cases (which should ideally check one thing only) 1-to-1 will likely lead to inefficient automation. Determining what to automate and what to leave for testers is an art in itself, and simply handing someone a batch of manual test cases to automate isn’t likely to give you good results. 

What is the difference between unit test vs. integration tests in regards to front end testing?
Oren Rubin: There is no difference. Unit tests allow you to test a single unit (e.g. a controller of a component) and mock everything else; use spies, stubs, and mocks. An integration test, which might test the entire UI, where you can check unit loading order works the CSS (which is global by definition, if you exclude future shadow dom), and integrations between the different units. If you don’t mock the servers, then it’s actually End-to-End testing. 

About the panelists 

Bas Dijkstra
Bas is an independent test automation professional who has been in the test automation and service virtualization field for over 10 years now, designing and developing test automation and service virtualization solutions that enhance and improve test teams and test processes. Find out more information about Bas on his LinkedIn profile. For questions and more information you can always send him an email at bas@ontestautomation.com or give me a nudge via @_basdijkstra on Twitter.

Oren Rubin
Oren has over 20 years of experience in the software industry, building mostly test-related products for developers at IBM, Wix, Cadence, Applitools, and Testim.io. In addition to being a busy entrepreneur, Oren is a community activist and and the co-organizer of the Selenium-Israel meetup and the Israeli Google Developer Group meetup. He has taught at Technion University, and mentored at the Google Launchpad Accelerator.

Be smart & save time...
Simply automate.

Get Started Free