The Fishnet
How do we know that we have sufficient yet optimal test cases ?
The inspiration for test adequacy in HyBIST is the fish net. How do we know that the net can indeed catch the fishes that we are after? Well a broad net to cover a large area, deep enough to reach to bottom with the holes small-enough to catch the smallest fish should be sufficient.
In HyBIST test breadth is a measure of the various types of tests to conduct, while depth is about the levels of quality and porosity is a measure of as what the ‘granularity of test data’ is. Test data at lower levels are basic input values while at higher levels test data is coarser i.e. input is a ‘aggregate of basic inputs’ .
Baby Growth
When asked about the quality of software, most often we show defect charts, coverage metrics to give a image of quality. What if we visualised it far better? The picture below shows a brilliant visualization of the growth of baby in the mother’s womb. It is just more than metrics like weight and volume. It shows the actual growth in terms of mass, shape and the organs. And this is the inspiration behind “Cleanliness index” the measure of quality in HyBIST and the visualization of the same.
Fractional Distillation
Remember fractional distillation? A technique to separate mixtures that has components of different boiling points.
Sherlock Holmes
People view the act of uncovering defects in software as being dependent on the experience of the person. So is it only an art? What if we viewed the problem of uncovering defects as solving a mystery? And who is better than Sherlock Holmes to solve mysteries? He solved these not via sheer experience, but via the power of deductive reasoning. By hypothesizing possibilities and then proving each one.
And he is the inspiration behind HyBIST for ‘how to hypothesize potential defect types’ in software – what aspects to look for from what views. And then treat the act of testing as an intelligent method of proving the presence of defects rather than rely on pure experience only.
Fishing Analogy
Testing software is like fishing.
Cast a net to catch defects instead of fishes.
So if you have to increase yield the fishes caught, what would you do ? Work harder or work intelligently? The latter I am sure would be the answer you would give. So what does this mean?
The picture dissects how we could ‘fish intelligently’. Set a clear goal as ‘what fishes to catch’, identify the areas where these fishes may be present and then depending on the ‘size’ of the fish, use an appropriate net to catch these, and then figure out how to catch (use a boat, or sit by the side of the lake) and then finally examine the catch(or the empty net) to refine the fishing.
Properties of Matter
Given any material, how do I judge purity?
Let us take an example. Given a bottle of water, how can I assess if the water is pure?
Hmmm… It should be colourless, should not smell, should be pH neutral etc. What I am doing is to identify the properties that pure water should satisfy and then check if these are indeed met. And we know that matter has two types of properties – physical and chemical.
Now think in the context of software. To assess ‘purity’ or ‘cleanliness’ of a software, can we identify ‘properties’ that it should satisfy and then assess these? Would this not be a good way to ascertain expectations clearly and apply a scientific approach to evaluating correctness?
Understanding expectations of a system in HyBIST has been inspired by “Properties of matter” where expectations of end users of a system are converted to “Cleanliness criteria(CC)”.
Then the act of assessment of these criteria can be done in a logical manner by identifying potential defect types (PDT) that may impede the CC and then performing specific types of tests for detecting these PDTs.
