Smart checklists make you think

The power of checklists in other disciplines
In other mature disciplines like medicine, aviation, construction where the impact of simple defects is enormous, it is smart checklists that have come to the rescue and saved millions of dollars and saved many lives.

The Checklist Manifesto  by Atul Gawande, a surgeon, extols the power of checklist. He states that any problem can be categorised into simple, complex and complicated, and how the smart checklists can solve these.

‘Simple’ problems are those that are individualistic in nature with a set of stuff to be done, while ‘Complicated’ problems implies multiple teams/people coordination/timing issues, and ‘Complex’ problem is where outcomes are different despite same application.

Checklists are not mindless compliance
Smart checklists are not about mindless compliance, not about ticking boxes, it is really about tickling the brain to think better and ensure fault-proofing rapidly. In our industry, checklists have been seen a cheap brainless activity that is about ticking the checkboxes, and therefore presumed to be useless.

Smart checklists make you think
To solve complex problems, “push the power of decision making from a central authority to the periphery, allowing people to make decision and take responsibility”  (Atul Gawande).

The checklist needs to allow for judgement to be used in the tasks rather than enforce compliance, so that actions may be taken responsibly. Hence, a smart checklist is about enabling your thinking process by having:

  • Set of checks to ensure stupid but critical stuff is not overlooked
  • Set of checks to ensure coordination
  • To enable responsible actions to be taken without having to ask for authority.

Good checklists are precise and easy to use in most difficult situations, it does not spell out everything, but provides reminders of critical & important steps that even highly skilled professionals would miss.

Checklists seem to defend everyone, a kind cognitive net designed to catch flaws of memory and attention, a well designed checklist enables one to DO well, SYNC with others well, and take appropriate ACTions as needed. [5]

“The power of checklists is limited, they help experts remember how to manage a complex process or machine, make priorities clearer and prompt people to function well as a team.  By themselves, however, checklists cannot make anyone follow them.” [Boorman, Boeing]

We use checklists too, but are we effective?
Do we use checklists for early tests like DevTest? Yes, we do. But from what I have seen of numerous checklists like code review/UI checklists , it is more often is used like a compliance assessment, of ticking away a long list of items-to-check for. So this turns to be a mindless job and suffers poor implementation, as it is most ill suited for smart validation.

Smart checklist enables developing clean code
So should we really do dev test? Should we not become sensitive and write cleaner code? Well lean thinking (aka Agile) is of producing less bugs in the first place, not about testing more.

A ‘Smart DevChecklist’ enables one  to precisely accomplish this, to become more sensitive and written code that certainly does L1 through L4 issues. Remember, this is the most cost effective method to good quality. Well, a Smart DevChecklist is an efficient complement to dev test enabling an easy and efficient method to producing great code.

If you are keen to try out a SmartDevChecklist to rapidly do DevTest without the pain of dev test, it is available as part of the e-book listed alongside.


LEAN: It is not doing more, it is about doing less

So, what should be really happening in Agile context?
Lean thinking is what inspired the Agile movement. Lean is about not producing waste in the first place. It is doing things ‘clean’ at at the first place,  so that waste is ideally not there. Waste in software context are bugs. And in the early stages there are ‘unit bugs’. Since our focus in Agile is to find these earlier and to ensure that they are never there whenever we modify, we resort to a high degree of automation. Therefore we have a large body of test cases at the lower levels automated to ensure that we can continually execute them. This is great, but should we not focus on adopting a practice that in essence prevents these issues and lessen the need for large number of unit tests to uncover these?

It is not about doing more, it is about doing less
When we find issues in the product/app especially those that can be caught earlier, we focus on more rigorous dev test with extreme focus on automation. Yes, that seems logical. But what a minute, for a developer already busy writing code, is this the right approach? Given that dev test is largely about issues in L1 thru L4, could we not focus on getting this right or statically assess these via smart checklist?

Great quality early stage code is not about doing more testing, it really is doing about doing less test, by enabling sharper focus on ‘what-can-go-wrong’, ‘have-you-considered-this’.

The ebook outlines in detail outlines how to purposefully do DevTest in the “LEANest” by clearly outlining what issues a dev has to go after and outlines SmartDevChecklist to do this the most LEAN way.


Smart regression testing

FOUR key inferences from regression test survey

We did a quick survey on regression testing as a runup to our webinar “Is regression hindering your progression” consisting of FOUR questions listed below.

1.What fraction of total testing effort does regression tests consume ?
2.What fraction of regression test cases are automated?
3.What is your regression test yield?   RegressionTest yield= (#DefectsFound InRegression #RegressionTestCasesExecuted)
4.I think that a smarter assessment of how much to regress could reduce regression test effort by:

We had a total of forty four (44) anonymous responses, the four key inferences are:

  1. Regression is non-trivial, consuming possibly close to half the test effort.
  2. A long way to go for automation, only a quarter of folks seem to have half their test cases automated.
  3. Low yielding human effort is expensive, only fifth of regression test cases find issues, a low defect yield.
  4. Smart regression is business beneficial, at least 25% effort reduction is what practitioners feel.

Here it is, as an interesting infographic. What do you think? How may $$/time will you save with 25% reduction?

RT Infographic

RT Infographic







joesthe athlete

The promising athlete

“What is the objective of load test?” I asked the participants attending my workshop. “It is to find out if the system can handle the load” replied one of them. A circular answer, this does not help!

Another person chimed in “Having multiple users use the system at the same time and check if the system functions properly”.

“Does that mean that load test is only applicable for multi-user systems?” I asked. Silence followed.

I have observed that the clarity of understanding of “what some of the common non-functional tests like load, stress, performance, scalability, volume are” is typically lacking. Possibly, because the tests are inter-related. Possibly, because the participants did not go beyond of these jargons!

Using “Joe the promising athlete” as an example I have found it is very easy to explain the objectives of these tests and then delve into the test design techniques for these.Let me share this with you now.

Joe is a young athlete, his coach seeing in him a future world champion. Every day Joe spends significant time at the gym building muscles/strength. He lifts weights and is comfortable lifting up to 40kg snatch a few times.

The coach observing that Joe is comfortable with 40 kgs hands him 50 kg weights. Joe is successful but finds it progressively difficult. After a few minutes, he pauses to catch his breath. The coach hands him 60 kg!

“Coach” Joe whines. “Go on Son,” says the coach in a baritone voice. Joe succeeds, his body glistening with beads of perspiration.

After stating this short snippet, I asked the participants in the workshop “So why is Joe whining?”

“Well he is tired,” says a participant

“Why is he tired? I ask.

“It is pretty obvious, he is run out of energy”, says another.

I continue with my next question “So why has he run out energy?” knowing fully well that my audience will think that I am pretty dumb.

“Well he has done a lot of work and that is the reason Joe has run out of energy”.

“Great. So Joe has expended energy doing work. When his energy has run out, he is unable to continue further” I summarized.

“Now, can you tell me how this story relates to load/stress test?”

I see the participants eyes brighten and one of them chips in “Oh Yes, Load is about subjecting the system to do “work” using the energy i.e system resources. 60 kg is the maximum Joe could lift as he was stressed out, and therefore stress test is finding out the maximum load that the system can “lift” before the energy i.e. resources give out”.

“Wonderful guys. Let us summarize. A load test is about understanding if the system can do the typical real-life work with the given resources while the Stress test is about understanding the maximum amount of work that can be done by consuming ‘all’ the resources”.

Now the coach seeing Joe is tired gives him an energy drink. After a few minutes, Joe is ready to go and the coach hands him 75 kg. Joe now energized, lifts it with aplomb!

“Now what can we learn from this?” I ask. The participants think for a while and then “Well I think Joe got an additional shot of energy from the drink and he was able to lift more” said the guy in the last row.

“Wonderful correlation. Yes, he could do more work with the additional energy. Now can you connect this to scalability test?”

After a momentary pause, the person on the far left says “Yes, once stressed, more energy allows us to do continue to do more work and hence scalability test is about checking if the system when stressed out can indeed do more work if given more resources.”

“Brilliant. Now would one of you summarize this please” I ask.

The short guy with glasses in the first row says “Resources (i.e. energy) enables us to do the various System Operations (i.e. work). The three tests Load, Stress & Scalability” are about connecting these in different ways. I will illustrate this on the board”.

He walks to the board and writes:







“Great summary. Indeed simple. Now how does this relate to performance test?”

“That is easy. Performance does with time, the time taken to do the work (operation). Performance test is about measuring if the time taken to perform the operations is indeed acceptable (with the given resources)”

“Does this mean that the load and performance test can be done at the same time? The objective however of these two tests are indeed different, is it not?” asks the backbencher.

“Yes, you are right, they may be done together, however, their objectives are different,” I say.

“Before we conclude, one last question – What is volume test?” I ask, walking to the end of the room.

After a few seconds, a quiet but firm voice behind me says “System Operations is really about processing data and volume test is about checking if the system can indeed process large amounts of data. By the way, in some systems, a large volume can choke and stress the system”.

“Guys, guess Joe and his coach helped us understand Load, Stress, Scalability, Performance and Volume test. I hope you understand that any kind of system (single/multi-user) can be subjected to these tests. Real life load is about subjecting the system to various types of operations (concurrently if multi-user) and not just subjecting the system to a blast of a specific operation” concluding the class.












“Yes” chimed in the participants with a big smile.

Thank you, guys. I am done with the class.

This is one of the lessons in the online course “Load testing using JMeter” in CleanSoft Academy Online at Cleansoft Academy Online











If you find the article interesting, please ‘like’, ‘share’ or leave a comment below.


Behave yourself.

One of the common statements in software testing that I come across is “Domain knowledge is critical to testing the functionality”. The belief is that knowledge of the domain acquired from prior experience is very necessary to perform good functional testing. Not that I disagree, because a prior experience is always handy. But the key question is ‘What if I am encountering a functionality for the first time? Will I be effective?’. Wouldn’t it be nice to be able to do this using a set of techniques in a scientific manner?

Let us first examine the basics… The objective of functional testing is to ascertain if the behavior of the entity under test (EUT) is as intended. So what is behavior? When we tell an unruly kid “Behave yourself”, what we mean is ’please comply with the acceptable norms i.e. rules’. Hence behavior of an EUT is about ’obeying some rules’. These ’rules’ are really the specification. So to test a functionality of any EUT, we need to first understand the intended rules and then create scenarios that are really various behaviors by combining the rules in different ways and then check the outcome(s).

So how do we figure out the rules? Hmmm… If I had domain knowledge then I would know how this should work, and therefore the rules. Note that this can be dangerous too as this sets up a bias, and we may assume certain rules that may be incorrect. Anyways, the issue is, how otherwise? To understand the intended behavior, first, describe what the EUT is supposed to do. Describe the behavior as a series of steps. Each step accepts/uses data and processes the data according to some condition(s). Examine each step and extract the condition(s) hidden in the step. Voila… We now have the various conditions aka rules.

This technique of extracting conditions (aka rules) is what I call “Descriptive to Prescriptive”, that is converting the descriptive specification to a prescriptive specification. “Describing” enables us to understand the behavior from first principles while “Prescribing” helps us to validate the behavior. We all know that clear and comprehensive description consists of the 5W+1H – Who, What, When, Where, Why & How. So when you encounter a functionality for the first time, apply 5W1H and describe the behavior as a series of simple steps.

Now examine each step and identify conditions implicitly or explicitly present. Once the conditions are extracted from each step, re-write each step as a series of rules and what we have now is prescriptive behavior. From this, it is easier to create the test scenarios and then identify the various data sets for each scenario and subsequently generate test cases and therefore for the entire EUT.

This behavior-based approach to functional test design is implemented by two techniques ’Box Model’ and ’Behaviour-Stimuli Approach’ in HBT (Hypothesis Based Testing). This lessens the need for domain knowledge as a prerequisite by enabling you to question better to understand behavior, and then extracting conditions to test effectively. Click here to read some articles on this from

So whether you test a technical feature or a business flow try this approach. Be the doctor – “Ask the patient to describe so that can you can prescribe”. Enjoy the happiness that comes out of the cure.


May you contribute to healthy software and a cleaner world.

If you find the article interesting, please ‘like’, ‘share’ or leave a comment below.


The tale of two doctors

“This is a long story of two doctors who by their different approaches to diagnosis helped Joe a professional tester sort out the typical confusions about black, white and unit, system testing. “

{Be warned this story may take about 7-9 minutes to read!}

In a big city lived Joe, a typical urban yuppie. He was always focused on a great tomorrow. He worked very hard, partied furiously and lived a fast life.

Life was a blast until his body decided to act up. On this Sunday morning, he woke up panting, unable to breathe, body drenched in sweat, with a dull pain in his chest. The previous evening was a blast, a celebration party thrown for his best buddy getting engaged. After an evening spent at bowling, they hit the pubs, closing each one, until they could not find one open.

A typical Sunday morning would commence at noon; today, as he was rudely jolted out of his reverie, the bright LED clock showed 7:00. could not move his arms, it seemed to take a tremendous effort to reach out for the bottle of Evian on the table near his bed. He had read about old age diseases getting younger in these modern times, dismissed them brashly, a reflection of his supreme yuppie confidence. For once he faltered, worried seriously if he could become one of the stories. All these years, he had thought of God, as a fashion statement but today he genuinely wished to believe that God exists. For the next few seconds, which seemed like an eternity, his mind rapidly flew back over the past years on the constant abuse he had heaped on his body. For once he prayed dearly that he would do the right things if he was excused this time. The clock glowed 7:02, and he realized it had been the longest two minutes of his life.

At 8:55 a.m he was at the reception of GoodLife hospital for a 9 AM appointment with Dr. Robert Black Sr., a senior and very experienced cardiologist. He was soon ushered in and was face to face with a severe yet friendly gentleman, a few orderly strands of golden hair on his shiny head with a piercing pair of eyes. “Mr. Joe, would you please tell me your problem?” said Dr. Black. Joe described in detail his travails upon his rude awakening. An old school doctor, he believed in detailed physical examination rather than the fancy modern equipment. “Lie down on the bed and relax Mr. Joe”. He took his stethoscope, placed it on his chest and listened carefully. “Breathe in and out deeply now,” said the doctor as he continued to hover his steth on various parts of the chest. His sharp eyes showed no emotion, as he went about his job confidently. The young nurse standing next to the doctor was petite and beautiful. She dispassionately took out the sphygmomanometer, wrapped the elastic band around Joe’s arm, pumped it up and watched the mercury bobbing up and down, while her other hand measured the pulse. After a minute, she looked at Dr. Black and said 140/120 and 93, in a husky voice.

“Mr. Joe, have you been feeling very tired at the end of day lately?”

“Yes,” he said and added, “It is the busy time of the year at work, a string of late nights.”

“What kind of work do you, Mr. Joe?”

“I work in the software industry. We are in midst of building a cool application for mobile phones”

“Oh I see, you are the software guy. My nephew is in the software business too and is always racing against time.”

“I guess you must be tied to the desk most of the time. Do you exercise?”

“Well doctor, the days are long and busy, I catch up on my sleep over the weekend. I try to workout in the gym over the weekends, but it is challenging.”

“I see that you are a smoker, do you drink? And you a vegetarian?”

“Well doctor, I do drink and I am not a vegetarian.”

Dr. Black was one of the most famous cardiologists in the country. He was a master at diagnosis, believed in the scientific and systematic study of symptoms and their connections. He placed his gold-rimmed glasses on the table, rubbed his finger on his chin, leaned back on his cozy leather chair and his piercing eyes looked straight into Joe and said “Mr. Joe, you have an issue with the blood supply to the heart muscles, there seems an advanced arterial block. It is necessary that you undergo angioplasty, a procedure that relieves the constriction as quickly as possible, say within a fortnight.”

Joe sat still, staring at the statement “The most amazing non-stop machine. Take care.” written on a poster containing the picture of a heart, which prominently on the wall behind the doctor.

Dr. Black who was used to these reactions jolted Joe out of the reverie “Mr. Joe, you need to quit smoking and go vegetarian. You are young, the angioplasty procedure has a high success rate and you should be back on to active life quickly”. Dr. Black believed in conveying news straight and expected patients to face reality and act on the problem. “Mr. Joe, you would need to be in the hospital for 3-4 days, do decide on the date quickly. As I mentioned before, it is important that you act on this within a fortnight. It is a painless procedure and should be fairly straightforward in your case. Do you have any questions?”

“No doctor, I have none” replied Joe mechanically, his ability to think, numbed by the turn of events. As he exited the consulting room, the receptionist, a cheerful and bubbly woman in twenties whispered: “Have a good day Mr. Joe” with a beautiful smile, a genuine one. It had the effect, and for a moment he felt cheerful and returned the smile, a little weakly though.

As soon as he was outside the hospital, his hand went mechanically to his shirt pocket containing the cigarette packet. “Smoking kills,” said the packet loudly and he threw the packet on the wayside.

David, his roommate had just woken up as Joe returned to his apartment. “Hi, have you had breakfast? Got some muffins and bagels, want some?” asked David. “No thanks” mumbled Joe.

After a few minutes, David understood the reason for the strange behavior of his best friend. “Come on man, let us get a second opinion right away”. David was one who never lost his cool and his level-headed thinking in tough situations was one that helped his friends many a time.

At 10:50 they were at ValleyTech hospital for a consultation with Dr. James White. He had been referred by David’s boss, who had undergone a heart bypass a few months at ValleyTech.

At 11:05 Joe was called in. “Good morning Joe. Please sit down. I have read your case sheet and have a few questions. Do you have any recent ECGs?

“No doctor,” said Joe.

“I know your company has a yearly health check plan for all, as our hospital administers it. Have you not taken it this year?”

“No doctor, the last few months have been very hectic and I have not had my yearly checkup yet,” said Joe.

Dr. White was a modern doctor who relied on technology in diagnosis and treatment. A young cardiologist, he believed in seeing the ‘internals’ before the scalpel touched the body. He was amazed at the advancements in radiology and made it a point to recommend a few pictures to be taken before he touched a patient.

Unlike Dr. Black who believed in the power of external examination, Dr. White believed in the looking at internals for diagnosis and treatment. Dr. White had immense faith in scans, lab tests and preferred analyzing reports to spending time on and performing examinations on patients.

“Please get the ECG done now. I would like to see the report first. Thank you.”

Joe went to the diagnostic lab in the adjoining building. When his turn came, he went inside, removed his shirt, the technician smeared jelly on his chest and proceeded to stick colorful leads at various points. In a few seconds, the needle was dancing, drawing patterns on the strip of paper. The technician looked at the squiggles on the paper with a bored expression, and after a few minutes decided the machine has had its share of fun and switched it off. He tore the roll of paper, scanned it intently, and then proceeded to fold it and inserted into a cover. Joe was curious to know what the squiggles meant and asked: “Is it normal?”.

“It seems fine except for a small spike here” replied the technician. He had seen hundreds of such squiggles and knew exactly as to what was normal, but he was no doctor to interpret any abnormalities.

Joe stared at the report, the squiggles held a secret that Joe was scared about. “Hey, let’s go meet to the doc now,” said David breaking Joe’s train of negative thoughts.

“Show me the report Joe,” said Dr. White.

The doc held the strip of paper and rapidly scrolled it forward and then backward.

“Were you treated for any heart-related issue when you were young?” asked Dr. White.

“No,” said Joe, scared to ask questions.

“Joe, there is a slight aberration in the ECG, it may be nothing to worry about. To confirm this, I recommend that you get the 64 slice beating heart scan. This the most advanced technology for diagnosis of heart-related ailments that is available in the world and we are the only hospital in this city to have this. This gives a clear picture of the beating heart and enables clear diagnosis. And also take a chest X-ray too. I will be available until 1:00 PM, please get it done right away and then see me”. He wrote down the lab request and handed it over to Joe.

“Hello, I need to get ‘64 slice beating heart scan’ done. How much does it cost? Joe asked the grim looking gentlemen on the cash counter.

He was shocked at the cost of the hi-tech scan, it seemed to have enough digits to max his credit card. Joe looked at David conveying in his look “They are milking us”.

Joe realized that the second opinion was going to be expensive and needed to think about this before he went on a diagnostic spree.

Joe was a professional software tester who diagnosed software for defects. He decided to chat with David over a cup of coffee to decide whether to go ahead with the expensive scan. David went to get the coffee while Joe sat down at the corner table, gazing at the birds fluttering over the little pond outside.

During this mindless gaze, staring at the chirpy birds, it suddenly flashed on him, the parallels between his profession and the doctor. In his job, he used black box techniques that required him to examine the system externally and find defects. He relied on the deep understanding of the intended behavior, observation of behavior (“symptoms”) to design & refine his test cases. He had at times looked at internal information like architecture, technology, code structure to design test cases that were adept at catching issues related to structure.

His colleagues always used terms like “Black box testing” and “White box testing” and associated these to with system and unit levels respectively. Now he realized the general misconception that unit testing was white box testing and system box testing was black box testing. His train of thought was interrupted by hot coffee spilling on his shoulder followed by the shrill sound of glass breaking. “I am really sorry, hope you are ok,” said the elderly woman who had tripped over the protruding leg of the gentleman at the neighboring table spilling a hot cup of espresso that she was carrying. “I am fine, let me help you,” said Joe as he helped pick up the glass shreds.

He realized that certain types of defects were better caught via “internal examination”(white box test techniques) while some are most suited to be caught via “external examination” (black box test techniques). He now understood that both of these test techniques were required at all levels to uncover effectively and efficiently.

Suddenly the diagnosis approach followed by Dr. Robert Black and Dr. James White became clear. As soon as David laid the steaming cup of Cafe Latte on the table, Joe had made the decision. He was not withdrawn or worried. The confidence was back and he would not let the ECG scroll spoil his fun.

If you find the article interesting, please ‘like’, ‘share’ or leave a comment below.


“Roadmap to Quality” – Panel discussion at SofTec 2012 Conference

SofTec 2012, Bangalore, July 14, 2012

The panel discussion on “Roadmap to Quality” was brilliant due to the cross-pollination of interesting ideas from non-software domains. Three of the four panelists were from non-software domains – Mehul@Arvind Retail, Soumen@GM, Raghavendra@Trellborg with lone exception of Murthy from Samsung, with moderation done by Ashok@STAG.

The key take ways from the panel discussion are:

  1. Continuous monitoring helps greatly as this is like a mirror that reflects what you do constantly, this is what Mehul@Arvind highlighted as being important in his domain of apparel/retail business. Ashok connected this to dashboards that are becoming vogue in our workplace, more in the Agile context
  2. Soumen@GM stated the importance of early stage validation like Simulation, Behavior modelling in the Automotive industry, as the cost of fix at the later stage is very expensive. The moderator connected this to “Shift Left”, the new term in our SW industry- how can we move validation to earlier stage(s)?
  3. Raghav@Trellborg a component manufacturer of high technology sealing systems stated need of understand of understanding the final context of usage of the component as being very important to know to ensure high quality. He also stated testing is deeply integrated into the “shop floor” i.e. daily work and the most important aspect of quality is not QA or QC but the underlying the Quality Systems in place. How do Q systems ensure that quality is deeply entrenched into the daily life. The moderator highlighted the fact the in software industry we have implemented systems, but these are still at an organizational level and the need of the hour in SW industry is to institutionalize these at a personal level
  4. Finally Murthy stated level of quality needed is not the same in all domains, in certain domains (like mobile) that have disruptive innovation and short life cycles, “we need just enough quality”. He highlighted the need to understand “technical debt” that we can tolerate as a driver for deciding “how much to test”

You can also read the special news on the panel discussion on Silicon India website.

Relavent topics:
a. Software testing lacking serious effort


Purity. Quality. Cleanliness Criteria.

It is normal practice to ascertain the purity of a material by first identifying the properties – physical and chemical – that it should satisfy. Therefore, “purity” is really a degree of how well the properties have been met and properties are those that can be observed when evaluating the behaviour of the material.

In the software world, the quality is about how well a system/software meets the expectations, similar to the notion of purity. Therefore, software quality should also be about satisfying properties. The act of testing is about clarifying / setting-up the expectations by identifying properties and their intended value and then assessing them.

Read the article – by T Ashok – below that highlights the notion of purity, quality and cleanliness criteria in this context.