“It took a few hours to incorporate the change but took a few days to test, upsetting the customer. Why? Well, we found out that our QA team was doing too much regression. Wish we could be smarter” – Engineering Manager of a mid-sized IT company.

Have you ever felt this way? Have you wished if you do less regression to release faster?

In the current world of rapid development, software is constantly updated with new features, incremental additions and bug fixes. While features (new & incremental) are the focus for revenue generation and market expansion, bug fixes are necessary to ensure that customers stay.

While on the path of progression towards revenue enhancement, the challenge is “Did I break any existing features that are working well”? That may necessitate a regression test.

Note that as the product grows, so does regression, increasing cost and slowing down releases.

Regress means ‘go backwards’, and in this context, it means ‘checkout prior quality risks to ensure that they still are under control’. This implies that the product is retested from both functionalities and attributes aspects to ensure that functionalities and attributes like performance, security etc. are not compromised.

So, how can one regress smartly?

* Figure out how much not to regress by doing a smarter impact analysis using a scientific approach to understand fault propagation due to change.
* Figure out how much not to regress by analysing defect yields over time to understand what parts of the system have been hardened
* Well, automation is an obvious choice, ensure that the scenarios are “fit enough for automation” so that you don’t end spend much effort maintaining the scripts to be in sync with every change.

Change as we all know is very imminent, and does cause a domino effect. The smartness lies in, validating only those that have the potential for domino effect thereby doing less and exploiting automation to do faster.

Here is the link to TWO aids that can enable your QA team to regress smartly. Oh, ask your QA team to read this article before they use the tool.

The context

In the current world of rapid development, a software is constantly updated with new features, incremental additions and bug fixes. While features (new & incremental) are the focus for revenue generation and market expansion, bug fixes are necessary to ensure that customers stay.

While on the path of progression towards revenue enhancement, the challenge that is “Did I break any existing features that are working well”? That may necessitate a regression test.

Note that as the product grows, so does regression, increasing cost and slowing down releases.

Regression

Regress means ‘go backwards’, in this context this means ‘checkout prior quality risks to ensure that they are still under control’. The product is retested from both functionalities and attributes aspects to ensure that functionalities and attributes like performance, security etc. are not compromised.

But, how do we tackle this?
Given the necessity of ensuring that the functionality and attributes are not compromised, we have to retest the functional/non-functional aspects constantly resulting in repetitive testing.

To do this well, we typically adopt:
1. Massive regression test automation to re-test thoroughly.
2. Deep product knowledge to assess the potential impact of changes and do focused regression.

So, what is the challenge?

1. Well, automation is great, but it requires continual investment to build and maintain.
2. In-depth product knowledge is also limited to a few people, and there are always in high demand!

Hmmm, how can we do better?
Instead of focusing only on how to do more and faster, could we do it less in a smarter way? Let us ask some questions to figure this out:

1. Are you doing too much regression?

Could we do a smarter impact analysis? Could there be a logical approach to analysing change impacts without only relying on deep product knowledge? Yes, one of HyBIST’s technique “Fault propagation analysis” could be useful here. The technique, in a nutshell, states “Given that an entity has been modified and is linked to other entities, what types of defects can indeed propagate and affect the linked entities?”

2. Is your defect yield from regression good enough?

Software with time hardens, i.e. becomes fit. This implies that the same test cases executed yield less defects later, i.e. test case yield drops. So the lingering question is “should be executing these at all?”. Just like living beings who develop resistance to certain diseases over time, the software also can be thought to become ‘resistant to test cases’ with time. In HyBIST, we call this ‘Test case immunity’ and use this to logically ascertain test cases that may be dropped and therefore do less.

3. Are your test scenarios fit enough for automation?

If the software is volatile, automation is more volatile! Changes to software necessitate automation to be in sync. So to ensure rapid modification, frameworks are used. That is great, but do you know that structure, i.e. architecture of test cases also matter? It is not just about frameworks and great code. But it’s about how well the test cases are organised. In HyBIST this is done by using a technique “Levelisation analysis” that ascertains if the test cases are organised into well-formed levels enabling rapid automation with rapid modifiability.

In closing : SMART REGRESSION

In summary, the three questions were all about “How can we do less to do more”? Do less regression. Do less automation maintenance. And therefore perform smart regression to progress further.

Smart regression complements the act of doing faster via automation by enabling one to do lesser.

SaveSave

Posts tagged: smart regression

Is regression hindering your progression?