Despite good testing, defects escape to the customer. Collecting and analysing the customer reported defects is key to improving test effectiveness. So, how can HyBIST help in this?
All we get from the customer is/are issue(s). How do we we proceed?
- Identify the defect with affected EUT. A good understanding of the defect will help us to identify WHAT Flow/Requirement/Feature/Structural-component has this defect (WHERE).
- Now classify this defect of the identified EUT into the appropriate HyBIST level. i.e. what type of defect is this? (WHEN).
- Now classify if this defect is a basic conformance or a robustness issue i.e. Is this defect a basic behaviour issue when used correctly or is this an issue when system has been abused? This helps us to classify the type of test case that may be required to catch this, +ve test case or -ve test case. (WHAT).
Now we have a clear insight of (1) WHERE, (2)WHEN and (3)WHAT – Where is the issue, when could have it been caught and what type of test case could have caught it.
Now all that remains to analysed is why did this slip?
A defect can escape because of:
(a) Test case(s) were not executed maybe due to paucity of time or bad judgement
(b) Test case(s) are not available, indicating insufficiency
4. As a final step identify if there exists test case(s) to uncover this bug. If NO, then this is an indication of insufficiency, so add new test cases. If YES analyse this was not executed and take appropriate corrective action.