e

Practice (not process) makes it perfect, RAPIDLY.

Organisations believe in processes as a means to get things done well. However in recent times, the emphasis has shifted to the individual. It is about ‘how to enable individuals’ to deliver their full potential. How can we equip individuals to solve problems smarter and rapidly?

Process is the set of activities we do normally agreed upon by stakeholders in an organisation. Of late, the shift has been to make processes light so that they are nimble and responsive. Process is great in enabling overall discipline but the “individual habits” of good practice are vital to being effective and rapid.

Practice on the other hand is how we do these activities, largely driven by individual brilliance. It is about how we breakdown problems, ideate solutions and rapidly implement them. It is about techniques, principles, guidelines(heuristics) that one uses to solve problems.

Process is like a route map whilst Practice is the driver skills to get to the destination quickly and safely.

Equipping an individual with good method(s) that when practiced continually enables one to deliver great work, the outcome of ‘practice’. Mere processes don’t cut it, however when good individual practices become common place, they get converted to “Best/Good practice” becoming part of the process.

In the context of testing, I have been focused on developing a scientific method/methodology (Hypothesis Based Testing – HBT ) to enable a software engineer to deliver clean code. Recently I have been focused on crafting a style of testing (Immersive Session Testing- IST) that is rapid yet scientific (based on HBT) that equips an individual with good practices spanning the logical left brain thinking with creative right brain ideation to unleash his/her potential.

f

The Power of Geometry

Geometry is a branch of mathematics concerned with questions on shape, size, relative position of figures, and properties of space. The shape or form is key to structural strength and architectural aesthetics in static structures. In dynamics, the shape plays a critical role to higher power transfer/output with lower energy expenditure.

Running form

Let me tell you a story on running to illustrate this. When you are into long distance running (Marathons), the ‘running form’ matters a lot. What is ‘running form’? It is the erect lean-forward stance of the runner with clean forward/backward hand movement. How does this help? Well the erect and lean-forward stance exploits gravity to propel you forward while the rhymthic hand movement enables the abdominal core to power you, reducing the effort on legs. How did it help me? My running performance improved significantly once I got into my running form.

So what does it take to get into the ‘form’? Understanding the science of form, techniques to get into it, and then enormous practice to make this a habit. The result – faster and longer runs, lesser injury and quicker recovery. A significant improvement just by getting into ‘running form’.

Frame geometry

I am on a story telling spree, let me tell you another one.

As an avid endurance cyclist I was keen to improve my average speed on long distance rides. Over the years I have worked on my stamina, food, strength, mindfulness to increase the distances I could do. Now I was keen to improve my average speed. It was but natural to shift my bike from my relaxed hybrid to a race geometry bike, the one with drop bar handles and sleek frame. Having never ridden a road bike, my first ride proved to be marvellous, a 4-5kmph improvement in my average speed. Wow! Nothing has changed in me, the difference was in the frame geometry of the bike. And that is when I realised the power of geometry once again, in cycling. The frame geometry shifted my stance on the bike, enabling me to transfer more power to the pedals while expending the same energy, resulting in higher average speed. Well, the first couple of long rides were interesting; the aggressive geometry resulting in back aches due to a taut back! Once the muscles memorised this stance, it became a habit and the rides turned out to be zippier. Once again it was about knowledge of techniques and then practice to make it a habit.

These experiences of being more effective (more power) yet be efficient (less energy expenditure and therefore longer distance) in two sports set me thinking on the power of geometry. As a passionate tester, it was but natural to think on application of this in the context of software testing.

Geometry in testing
Let us move to testing now…
In both the instances the shape(form) made a big difference to the outcomes. The outcome was faster movement and hence the ability to do longer distances. What is geometry in software testing ? The arrangement of the elements that aids in understand the static system under test (SUT) and the arrangement of test cases to evaluate the dynamic behaviour.

What does arrangement of elements of SUT mean? View the SUT as a set of flows consumed by different end users, where each flow seen is a composition of business requirements, each one delivered by one/more features with each feature delivered by a set of structural components. How does this ‘arrangement(shape)’ help? This ‘geometry’ of the SUT enables clean problem decomposition to clearly understand expectations of end users and also ‘how it is bolted together’. This enables intelligent questioning to understand deeply of what we know and what we do not know, the latter capable of becoming defects in the future.

Now let us look at the shape/arrangement of test cases. The arrangement of test cases into groups that target specific types of defects (for a given entity) ordering them in a hierarchical manner over time (levels). Visualise this a multi-layer filter with each layer catching a set of specific type of defects with each layer having variant meshes to catch different types of issues. How could this shape/arrangement of test case help? Well by decomposing the test design activity to making it simpler, sharper and more comprehensive.

The arrangement of SUT enabled questions to pop up to understand and uncover potential bugs, while here the arrangement improves the ‘filtration capability’ enabling the lurking bugs to pop out.

So what does one need to implement this ‘geometry’ in testing? As with running/cycling, we need to understand the ‘science behind’ and then understand the techniques, which by practice will become a habit.

Geometry in nature – Beauty and Power


Nature uses geometry beautifully. Shapes contribute to strength, result in high utility and ultimately satisfy the should via aesthetics/beauty. All this done efficiently! So it is not just about brute force use of intelligence and techniques but about shaping the problem well to understand deeply and evaluate comprehensively.

 

Look around you. See the various shapes in nature and how they provide the strength and beauty. The honeycomb, the dome of cathedral, the roller coaster, the double helix of DNA.

Shape the problem of understanding question and evaluation to deliver higher yield. Question deeply, understand better and evaluate comprehensively.  Exploit ‘The power of geometry in testing’.

——————————
HBT (Hypothesis Based Testing) is inspired by the power of geometry to setup good baseline, do robust test design equipping you with power to test smartly.

If you are keen on knowing more, there is a 2-day HBT masterclass on HBT happening at Bangalore (June 30-July 1, 2016).  Click here for details.

W6-Poster

The Buddha Story: Good understanding

by T Ashok

Good understanding is not about knowing everything in detail, it is about just knowing what is required and learning to discard what is not necessary or deferring to a later stage.

Let me tell you a beautiful Buddha story that illustrates this.
Once upon a time there lived a wise and learned man with mastery over various languages, philosophies and religions. He was extremely proud of his knowledge, but eager to learn more. He came to know that a person named Buddha lived far away and was a very learned man. Keen to further his knowledge, he went to Buddha, introduced himself as a man well versed with various philosophies, expressed his wish to enhance his knowledge and requested to teach him.

Buddha said he would be happy to do and requested him to kindly sit in the corner of the room. A few hours passed and the learned man was getting fidgety while Buddha continued to do his work. As the sun went down, Buddha suggested that they resume the next day. The learned gentleman after a good night’s rest came back promptly. Buddha again requested him sit down in the corner. The morning turned into afternoon and the dusk set in. The man was throughly upset that Buddha was ignoring him. He went up to him and said “Sir, I am very upset that you are ignoring me, I hope you understand that I am also a man of knowledge”.

Buddha calmly said “Come my dear friend, sit down let us have a cup of tea”. He took the tea pot and poured the tea onto the cup while watching the man. The cup filled to the brim and Buddha continued pouring, spilling the tea on the floor. The man watching this shouted “The cup is full, please stop pouring!”.

Buddha fully aware of what he was doing said “My friend, your mind is like this, filled with knowledge, with no space to acquire additional knowledge. Empty it and you will be ready to learn.” And only then did the learned man realise his folly.

As a test practitioner, understanding a complex system is always always interesting. The big challenge that I face is when passionate folks explain the system sparing no detail. That is when I say “STOP, I do not need to know this, will certainly ask when I need it”.Good understanding requires that the tea cup be only partially full.

Defer what is not needed now to later. Don’t attempt to understand everything at one go.

Smart QA is about having great clarity, a result of good understanding. This occurs when you are actively questioning with an agile mind, not passively absorbing and getting weighed down. Lessen your burden and your mind is like a nimble goat climbing the mountain of complexity. At the top, view the system in totality clearly. And the issues pop out.

HBT (Hypothesis Based Testing) is based on this principle of deferment and active questioning to understand complex systems rapidly and test smartly.


About SmartQA The theme of SmartQA is to explore various dimensions of smartness to leapfrog into the new age of software development, to accomplish more with less by exploiting our intellect along with technology.  Towards this, we will strive to showcase interesting thoughts, expert industry views through high quality content as articles, posters, videos, surveys outlined as a SmartQA Digest weekly emailer. SmartBites is soundbites from smart people”. Ideas, thoughts and views to inspire you to think differently.

l

Tell a story, do not just report

We all make reports/presentations for/to senior management. As technical people, we are typically generous with facts. Since a picture is worth a thousand words, we spice it up with graphs. Feeling happy with the amount of details we have packed in, we are raring to make a great presentation to the senior management. Hey, wait a minute…

What are we trying to achieve? To furnish information or provide insight to assist in decision making? To report facts or tell a story? To showcase value or the quantity of work done?

I prefer story telling. Telling an engaging story that is factual, easy to assimilate, providing insights to facilitate decision making and finally showcase the value of my work. Think it is a good idea? But, what does this take?

Certainly it takes more than just dumping a lot of facts and getting into too much detail in the beginning. In fact less is more! And this is what people find difficult. Let me summarise my recent experience with my colleagues who came to me with a dense and detailed presentation that they intended to present to our customer’s senior management.

1. Understand target & their expectations- “Who is the audience for the story?”

Before you jump into making the presentation, understand to whom are you presenting to and what they are looking for. Typically senior folks look for outcomes and value, not just activities and facts. When you report activities done, align these against the intended objectives. And if you have issues that hinder your activities, state clearly what you need from them to resolve, don’t whine! Don’t just report tons of facts, process these into meaningful information. Remember the senior management is going to ask you the question ’so-what?’ when a fact is presented, hence analyse & process these to meaningful information that can be correlated against intended objectives.

The patience levels of senior management are typically low, hence be clear about the objective, crisp in communication and hold their attention by ‘outline first and detail later’. A good story needs a great plot, well formed characters and a cogent buildup to the climax.

2. Organise the content – “Setup the chapters/scenes”

A good story flows smoothly. Start with a clear exposition of the objective of presentation, and then identify key information to be presented. Arrange the facts (content) into key sections, much like the chapters in a story creating a cogent sequence that form the story line. Then connect these information across these slides. It is the weaving of information that strengthens the story. The interrelationship of the activities, intended goals, and the final outcomes is what strengthens the story and makes it interesting.

3. Detail out the information/facts – “Choose the characters wisely’

A great plot with a cogent buildup is not enough. It is characters that breathes life in a story. In our case, the characters relate to the various facts/information. The information to be presented, their degree of detail, how objective they are, the relevancy to the context is what that makes the character contribute to the story line. Keep in mind that we are presenting to senior management who value objective information and have a high degree of impatience to wishy-washy subjective information.

4. Present the information well – “Dress up the character”

If the characters are not presented well, it is not fun reading the story. They have to fit well into the role, present themselves well, play the part for the right time and exit gracefully.

In addition to what information is presented and how it is presented makes the character play its part well. The way the information is presented include the layout, colour schemes, consistency, spelling, grammar, sentence formation, tone of voice, choice of fonts, use of callouts, using pictures in lieu of text. Short phrases, pleasant colour schemes, right choice of tone (active/passive, 1st/3rd person), appropriate fonts are what ‘dresses up the character’ making them very presentable.

5. Edit, re-edit – “Practice, practice…”

Much like how the characters need to practice multiple times to deliver a stellar performance, writing is iterative, requiring multiple edits, each one refining and embellishing. In every edit, refine by removing information that is superfluous. Remember ‘less is more’, by removing potentially noisy information. When you can remove no more, you probably have reached the end of edit cycle.

6. Read aloud – “Watch the characters perform”

Once you think you are done with the presentation, read the text aloud. In the recent case, I read some of the text in the presentation around and watched the author (my colleague) squirm! Well they could spot the issue and know what they had to modify. This works wonderfully to quickly analyse the content and spot issues. Note that you have to read text in the presentation using the the right modulation so that you feel good, or spot problems. This is very much like watching the performance of the characters and rapidly getting to know as to what you want to tweak.

We started with “Understand target & their expectations” focussing on what they want and now we close by playing the role of a customer by reading it aloud. It is indeed amazing as to how many times I have helped refine content by becoming the customer via reading aloud.

When drab reporting gets converted into a story, the author can make the presentation come alive and engage the typically impatient senior management. Leadership is about great storytelling and when you engage in this manner, you lead the conversation. You are in control then and therefore can get your message across powerfully be it to communicate the value of your work or seek assistance to smoothen bumps you have encountered.

As engineers, we are typically left-brained rooted in logical thinking. On the other hand story-telling is a right-brained activity involving creativity and emotion.

So the next time, you make a presentation, ask yourself “Can I tell this as a story?”. At the least, sit in a quiet corner and read aloud and assess if it feels good. I bet you will know what to do. If you find soliloquy challenging, do this with your best buddy.

Cheers.

the

The Promise

Our focus as testers is on finding issues in the product/application. And it is a good goal to pursue. Step back a little and think… Is there a greater goal? If so what is it?

A product is sold on the promises that marketing makes to its customer base. And a custom application is also sold on the promise of the value it delivers to the business. The greater goal is “The Promise”. And these promises are based the expectation of end users.

When we test a system, we are in fact checking the unwritten promise of the development. That of ascertaining if there are defects that may derail quality and break the promise of good working software. But that alone is not enough. As end users we have expectations that have been built on the promises by marketing folks. And these go beyond the mere good working software. These are about ‘how it enhances my experience and could make my life better’. My life at home or office depending on the product!

So we need to elevate our thinking to a higher level. As one who is responsible for “Ensure that we deliver on the promise”. Of ensuring that the expectations are indeed met. That expectations are meaningful and are indeed value adding.

But it is difficult due to the bias. The bias of knowing deeply about the system. When we test a system, we get to know the system in detail and over time “get used to” the kinks. The kinks that may ultimately derail the promise. The kinks that messes up expectations.

The difficulty is in shifting to the view of being an end user and then evaluating the promise. The difficulty in quietly getting acclimatised to challenges in implementation, to becoming numb to the lack of clarity of the pristine promise and and therefore not doing a stringent job as expected. Kinda like the “Stockholm syndrome”. And this is but natural. So what can we do about this?

How can we get this job done? Let us step back a little and see how we get any job done. We can get a job done by (1)Doing it ourselves or (2) Facilitating to get it done. In this case the ‘doing it by ourself’ seems to pose the challenge due to bias. So maybe we should resort to (2). By facilitating to getting this done.

Before I get to the details, let me explain the genesis of this this article. In a recent customer engagement with a startup company, the conceiver of the product was the Founder & CEO of the company. He was keen that the testing team understand his vision of the product that he has been communicating to his prospective customers and investors so that they can deliver his vision. And he spent a good half day explaining his vision to the team before they commenced testing.

After a few weeks of testing, when the test team had a review with him, he was keenly questioning as to how the high level scenarios related to his promise. And I could see that test team was finding this difficult. Difficult to clearly elucidate how they are assessing his promise because some of the key information to evaluate the promise were indeed unclear/missing. And the test team did not now how to fill this. And the team thought that it was solely their responsibility to find this information. And this was related to kinds of information used in a query, their usage patterns, data volume and finally perception of patience by the end user.

And then is when it dawned that being a facilitator would help. Facilitate by enabling the product conceiver (in this case the CEO) to make the promise testable. Promises do commence by being vague initially, and it is the job of the test team to crystallise it. By building a straw-man, and then working with key stakeholders to identify the elements (of the promise) and then making it clear.

It is not the activity of doing it oneself, but by actively facilitating the thought process. And this is very iterative. In this case, this could be identifying elements of the usage, the situation in which this would be used, the deployment over the next 6-12 months and therefore the potential volume. Being a tester who is very aware of the product and its current limitations, it is necessary to understand this bias and therefore let the other stakeholders figure this out. And play the active role of a facilitator. Of dis-engaging and ensuring that biases do no come in the way. Of elevating and become mature. It is about “The Promise”.

So next time, do look for defects but keep the larger goal in vision. And remember that it is not always about “doing”, but also about “facilitating”.

Requirements traceability is “Necessary but not sufficient”

When asked about “how do you know that your test cases are adequate?”, the typical answer is Requirement Traceability Matrix(RTM) has been generated and that each requirement does indeed have test cases.

Is this logic strong enough? Unfortunately NO! Why? Assume that each requirement had just one test case. This implies that we have good RTM i.e. each requirement has been covered. What we do know is that could there additional test cases for some of the requirements? So RTM is a necessary condition but NOT a sufficient condition.

So, what does it take to be sufficient? If we had a clear notion of types of defects that could affect the customer experience and then mapped these to test cases, we have Fault Traceability Matrix (FTM as proposed by HBT). This allows us to be sure that our test cases can indeed detect those defects that will impact customer experience.

Note that in HBT potential defects types are mapped to the Cleanliness Criteria derived earlier. Cleanliness criteria are those that have to be met to ensure that customer has a good experience with the system.

This is covered in

Properties of a good test scenario/case

We all design scenarios/cases to test system. What are some properties that a good set of scenarios/cases should possess? This post outlines the properties that a good test scenario/case should satisfy.

A test scenario and associated test cases prime objective is to uncover potential issues in the entity under test. In the process we would like to ascertain correctness. The properties outline this line of thinking.

  1. A given scenario and associated test cases should be clear on what it is validating i.e. what entity it is testing. This is common parlance is understood as “Requirements traceability”.
  2. It should be clear as what type of defect it has the power to uncover. This in HBT (Hypothesis Based Testing) is called “Fault traceability”. This makes the scenarios/cases purposeful.
  3. That the number of scenarios/cases shall be proven to complete. This is a “controversial” statement. In HBT, this property is called as “Countability” i.e. that the number of scenarios/cases are no more or no less. This can be arrived only if the intended behavior is modeled and then scenarios generated. The number of scenarios can be arrived by combining the conditions logically and the corresponding test cases generated by combining the values of the data satisfied by the conditions
  4. That the test scenarios/cases should be staged into quality levels rather than being one to uncover a variety of defects so that these are small and purposeful.
  5. That in addition to staging by quality levels, it is good to group scenarios/cases by types of tests to enable better clarity and purposefulness.
  6. The number of scenarios/cases as we progress from the lowest quality level to the highest is shaped like a frustum of a pyramid. i.e. number of test scenarios/cases are lowest level of quality is indeed a lot compared to the numbers at higher levels.
  7. That the distribution of positive and negative test scenarios/cases is indeed good enough. Hmmm – how do we know this? Extending point (3), negative scenarios are generated when conditions that are violated are combined while negative test cases are generated when values of data that do not satisfy the conditions are combined. Given this more conditions implies more scenarios (and also negative scenarios) while more data values implies more test cases. As one proceeds from lowest to highest quality level the distribution of -ve:-+ve skews on lower side. i.e. at higher levels, the scenarios/cases are more conformance oriented.
  8. That the scenarios/cases are ranked in terms of priority guided the entities that they test and the types of defects that they defect, to enable intelligent choosing of which to execute when faced with crunch time.
  9. To aid in ensuring scenarios when automated run as unattended and as long as they, it would be useful to design the execution order of scenarios. i.e which scenario to execute in case the current one fails. This can be factored into automated to allow for “long runs” intelligently.

Understanding properties can enable us to assess the efficacy of scenarios/cases and also yield higher efficiencies. In HBT this is captured in the HBT test case architecture whether nine properties allow segregation of scenarios/cases in a beautiful onion peel shell. The form and structure of this architecture enables better clarity and thus improves effectiveness & efficiency.

We would love to have your comments and if you like it, please share it with your friends.

Have a great day!

Landscaping – a technique to aid understanding

Understanding an application is not just about walking through the various features via the user interface (assuming that the application has an UI).

It requires a systematic walk-through of various elements commencing from the customer’s needs/expectations and then into the application’s deployment environment, architecture, features, behaviour and structure.

We need to construct a “good landscape” of the system and establish a clear baseline for effective testing.

In HBT, there is a technique to aid understanding of a system/application. View the presentation below to know more.

This was also published as an article in “Tea-time with Testers June 2011” – an eZine on Software Testing.

Dangerous Decisions – Assumption Traps

As good engineers, we use metrics to make decisions on quality, testing. My view is that measurements have inherent assumptions that we have to be cognizant to, otherwise decisions can be dangerous. For example, we use defect arrival rate to make decisions on quality, with the inherent assumption that test cases are relevant and complete. We use the metric of coverage to make decisions on adequacy of test cases, with the assumption that all system behaviors have indeed been coded.

I feel that measurements like test case immunity, test case growth, and quality growth could be possibly interesting indicators. These ensure that that we stay focused on the goal of effective testing. Find these thoughts outlined in this short presentation.