Category

Uncategorized

Category

The Software Development Lifecycle (SDLC) consists of various roles and processes that have to seamlessly mesh together, to release high-quality software. This holds true right from the initial planning phase, all the way to production release and monitoring. In terms of roles, we have designers, business analysts (BA), developers, testers, scrum masters, project managers, product owners (PO) and technical architects (TA), who bring a varying level of experience and skill set to the project. They collaborate to discuss and implement different aspects of the software. In terms of processes, based on team size, release schedules, availability of resources and complexity of the software, the amount of processes can vary from no process to strict quality gates, at different phases of the SDLC.

What is the Knowledge Gap?

As teams start to collaborate to work on different features of the software, they often run into the below situations-

  • Each team member has different interpretations of the requirements
  • A majority of the team finds it hard to understand the technical jargons used to describe the working of the system
  • Developers assume the testers have the same level of technical expertise as them, while explaining changes in code during code reviews
  • Developers fail to do development testing and assume testers will test the newly implemented feature completely
  • There is no clear distinction of responsibilities in the team; leading to ambiguity and confusion
  • The PO comes up with a feature to implement and each team member has different interpretations of how the feature should work
  • The BA writes requirements that are hard to understand and implement, due to lack of understanding of the technical aspects of the system
  • The TA explains how a feature should be implemented using technical jargons that are hard to understand by designers, PO’s, BA’s and testers
  • The developer develops the feature without paying attention to the testability of the feature
  • The tester sits in on code reviews and the developers assume he/she has the same level of technical expertise as them when explaining their implementation
  • The tester does not get enough time to complete their testing due to tight release schedules
  • The developer fails to do development testing and makes testers responsible for the quality of the product
  • There is no clear distinction of responsibilities on who would do what task in the SDLC and everyone assumes someone would do the tasks. As a result; majority of them never gets done

And so on…

Now, you may ask? Why do teams get into the above situations more often than expected? The answer is, there is a knowledge gap in teams. This is especially true when teams have a mix of entry-level, mid-level and expert level resources and each one makes different assumptions on the skillset, experience and the domain knowledge every individual brings to the table.

Also, these gaps can stem from a more granular level when teams are using different tools as well. For example – When customers use Testim, we have observed first hand that, different developers/testers think and use our platform differently.

  • Manual testers see Testim as a time saving tool that helps them quickly create and run stable tests, and as something that can help them in reducing the amount of manual effort it takes to test applications
  • Automation Engineers see Testim as an integrated tool that helps them to do coded automated testing with the help of JavaScript and API testing in one single platform instead of using multiple tools for functional and API testing
  • Developers see Testim as a quick feedback tool that helps them to run several UI tests quickly and get fast feedback on the application under test. They also recognize the ability to do more complex tests by interacting with databases and UI, all in one single platform
  • Release Engineers see Testim as a tool that can be easily integrated in their CI/CD pipeline. They also recognize the ability to trigger specific tests on every code check in; to ensure the application is still working as expected
  • Business and other stakeholders view Testim as a collaborative tool that can help them easily get involved in the automation process irrespective of their technical expertise. They also recognize the detailed reports they get from the test runs that eventually helps them to make go/no go decisions

As we can see, within the same team, people have different perceptions of tools that are being used within the project as well. These are good examples of knowledge gaps.

How do we identify knowledge gaps?

Identifying knowledge gaps in the the SDLC is critical not only to ensure the release of high quality software but also to sustain high levels of team morale, productivity, job satisfaction and the feeling of empowerment within teams. The following questions help to identify knowledge gaps-

  • What are the common problems that occur in each phase of the SDLC?
  • What processes are in place during the requirements, design, development, testing, acceptance and release phases of the SDLC?
  • How often does a requirement change due to scope creep?
  • How effective is the communication between different roles in the team?
  • Are the responsibilities of each team member clearly identified?
  • How visible is the status of the project at any instant of time?
  • Do developers/testers have discussions on testability of the product?
  • How often are release cycles pushed to accommodate for more development and testing?
  • Are the teams aware of what kind of customers are going to use the product?
  • Has there been lapses in productivity and team morale?
  • Is the velocity of the team stable? How often does it fluctuate and by how much?

In terms of tools being used:

  • How are teams using different tools within the project? Are they using tools the right way?
  • Are all the resources sufficiently trained to use different tools within the project?
  • Does one sub-group within a team have more problems in using a tool than others?
  • How effective is a particular tool in saving time and effort to perform different tasks?

Answering these questions as a team helps to identify the knowledge gaps and helps in thinking about solutions to these problems.

How do we bridge the knowledge gap?

There are 5 main factors that help to bridge the knowledge gap in teams. They are as follows-

  1. Training

Sufficient training needs to be given to designers, developers, testers, scrum masters, project managers to help them do their job better, in the context of the project and using different tools. Doing this will help designers understand how mockups need to be designed so that developers can effectively implement the feature, testers can attend code reviews without feeling intimidated and use tools more effectively with sufficient technical training, developers will understand why thinking about the testability of the feature being implemented is important and realize tools can help aid their development testing effort, the scrum master can better manage tasks in the project, project managers can ensure they help the team to collaboratively meet release schedules and deadlines and finally, stakeholders can get a high level overview of the project when they learn what reports are generated from different tools and how to comprehend them.

  1. Visibility

If we want the teams to start seamlessly working together like a well oiled machine in an assembly plant; we need to make the results of everyone’s effort visible to the entire team. It is important for us to know how our contributions help in the overall goal of releasing a high quality product within the scheduled date. There are various way to increase visibility in teams such as,

  • Checklists – where there is a list of items to be done in each phase of the SDLC, that helps everyone to be aware of the expectations from each one of them. This is especially helpful when the team consists of members of varying skill sets and experience. If the items in the list are marked as DONE, then there is no ambiguity in terms of what task has been completed
  • Visual Dashboards – Another solution is having visual dashboards giving a high-level overview of the project health and status. This not only helps stakeholders but also individual contributing team members. These can be created on whiteboards, easel boards or software that is accessible to everyone. Everyday, during stand up meetings, the teams should make it a point to address the dashboard and ensure everyone is aware of the high-level project status. For example – In Testim, we provide a dashboard showing what percentage of test runs passed, number of active tests, average duration of tests, how many new tests were written, how many tests were updated, how many steps have changed, what are the most flaky tests in your test suite and all these details can be filtered based on the current day, 7 day or a 30 day period.
  1. Clear definition of responsibilities

There needs to be clearly defined responsibilities in teams. Each one needs to know why he/she is in the team and what task they need to accomplish on a daily, weekly, monthly and a quarterly basis. Goals, objectives and expectations from each team member need to be clearly discussed with their respective peers/managers. This prevents a majority of the confusion that may occur in terms of task completion.

  1. Empowering the team

In this day and age, where individuals are more technical and skilled, what they are lacking in is – getting some level of empowerment and autonomy. Contrary, to popular beliefs that there needs to be one leader for the whole project; the leadership responsibility should be divided within each role in the team. There needs to be one point of contact each from the design, development and testing teams. Each point of contact who also in most cases help to lead their respective sub-teams, meet up with other leads and ensure all the sub-teams within the project are on the same page and working towards the same goals and objectives. This way, the whole team is empowered.

  1. Experimentation

Once the gaps are identified, the whole team needs to sit together (usually in retrospective meetings after sprints) to discuss different solutions to problems. Based on this, the team needs to experiment with different solutions and see what works well/doesn’t. This constant experimentation and feedback loop helps to make the team more creative and empowers them to come up with solutions that work for them.

In summary,

the “knowledge gap” has been one of the major obstacles for teams to reach their fullest potential. Identifying and reducing them, will help to increase efficiency and as a result lead to faster release cycles, with higher quality. Also, Testim can be used as one of the aids in this entire process. Sign up now, to know how it helps to increase collaboration and bridge the gap.

 

As we work with our clients, some themes are recurring. The most common theme is that development organizations are short staffed–they cannot hire enough developers. At the same time the business pressure to ship product features does not let up and pressure is high. So what does the head of engineering do? None of the options are good:

  1. Delay the release – Not acceptable to the business.
  2. Reduce the feature set and only deliver a subset of what the customers expect – Risk or failing to meet customer commitments and losing their business.
  3. Prioritize new feature development over quality and hope for the best.

Unfortunately #3 is what many R&D teams are forced into. Many development executives prioritize the expansion of the development team, and not the QA team. For many growing development teams, adding QA engineers is a luxury. And using sophisticated frameworks like Selenium require skilled engineers.

Yet not doing UI testing invites disaster. The cost of fixing bugs increases over time and so capturing bugs later in the release cycle becomes expensive to fix. Relying on manual UI testing, while better than nothing, provides poor coverage and scale. Many people rationalize their decision not to test the UI, hoping that developers test their own code and whatever their code affects, and maybe we can live without automated UI testing. This often works, until a bug creeps in that stops all forward movement. Suddenly you’re on a fire drill taking valuable resources from multiple departments in the organization. Afterall, developers only test a specific set of the app, on a specific environment.

Does Selenium fix this problem? Not really. The results are OK, but it requires dedicated, skilled engineers to set it up and, more importantly, to maintain. Unmaintained tests lose their value the minute the application changes. And the application is always changing.

The challenge our customers face is how to increase coverage and maintain acceptable quality levels, update their tests to fit code changes and keep up with fast release cycles. If only there was a way to automate the maintenance of tests… possibly leveraging machine learning to ensure the tests keep up with the pace of software delivery…and that the team can grow in this area without adding people.

Unit Tests are in the house!
I’m a big advocate for TDD (Test Driven Development). Research shows TDD has a high correlation to quality code, as it prevents bugs from forming in the first place. Also, the cost of authoring tests and fixing bugs is lower

This is true for the back-end. As for the front-end, although MVC and MVVM based frameworks are doing a great job providing easier tools writing unit testing, we still see low adoption. Some claim it stems from frontend comprising mostly glue-code, and some say it’s harder to write test DOM and event (asynchronous) code.

The need for frontend automation rises
Nowadays, a huge part of any project resides in the frontend. Not only did past challenges remain, the plot thickens as we have more environments (e.g. mobile and tablet), more Operating Systems (Window & Mac are both popular), and more browsers to support (IE11, Chrome, Safari, Firefox, Opera, Edge, not to mention old versions of IE.. may it rest in peace already).

Percentage of core logic – backend vs. frontend

Since more R&D teams shifted to agile, QA has become the bottleneck. A big part of being agile is the frequent releases. When a six month project comprises a three week testing cycle it’s one thing, but when a team wants to deploy weekly or even daily without bugs, QA is challenged with shrinking that to a cycle that takes 1-2 hours. The feedback to the developer should be instantaneous, as research show that longer the feedback cycle is, the longer it takes to fix the problem.

Why testing frontend is difficult
With the move to frontend, high maintenance overhead resurfaced. The maintenance for frontend test automation was as high as 50% twenty years ago. It has only improved marginally in recent years and is still as high as 30%.

To mimic a user interaction, you must first locate the element you wish to act upon. There are two major approaches to locate an element:

  • Visual based – look at the pixels
    Frameworks such as Sikuli use that technique, but it’s super fragile, as render per OS/browser/display-adapter will generate different pixels. Some display adapters are non-deterministic in nature. To have stable tests you must be a computer vision guru.
  • Object based – look at the object hierarchy (DOM).
    Frameworks such as Selenium spread like fire. As opposed to twenty years ago, better software design pattern emerged (such as the Page Object) which helped separate the tests’ business logic from the implementation, and urged reuse.

Nonetheless, both approaches result in flaky and unstable tests. They require high maintenance. or every UI (code) change, a test change is required as well (as both rely on the UI). It’s easier for those practicing the shift left paradigm, where quality is inherent in the development cycle, as finding the cause is easier, but they suffer from the same amount of failures. For those with a QA team, it’s still catastrophic. As almost every morning starts with understanding why tests fail and whether it’s a bug or merely a code change.

Crowdsourcing

Crowdsourcing

For End-to-End testing, some companies turn to crowdsourcing to alleviate the pain. This works great for regression tests which require no prior knowledge of the app. The required ramp up is just to write down the instructions in plain English. The big advantage is the low cost of people willing to click all day long. The problem arises when one tries to implement the shift left paradigm. This requires testing not only in the main branch, but also in all dozens of feature branches. Not only does this makes it not cost effective, but developers are custom to get feedback in seconds or minutes, not hours.
Image a developer promoting a hotfix to be released to fix a critical bug yet having to wait for someone in the other side of the world to wake up and test the new version.
Communication is also challenging at the developer and tester have no experience working together.
As for exploratory testing, crowdsourcing still great. It allows one to test different devices and different networks from all around the world. The rose of the garden is the seductive business model, paying only for proven bugs, comprising a thorn where every little bug is reported and the devs find themselves trying to prove that some of them are not real bugs. Until today, crowdsourcing does not provide a sustainable solution when shifting to continuous delivery.

Guide: The Key to Implementing CI/CD

Be smart & save time...
Simply automate.

Get Started Free