About HyBIST
Hypothesis-Based Immersive Session Testing (HyBIST) is an intellectual practice of probing via hypothesis based core method to analyse & design, and do immersively in sessions to test deeply & rapidly.
Validation is done in THREE phases of Reconnaissance, Exploration & Recoup using ‘key concepts’ to design smart probes and ‘key practices’ to probe system expansively and rapidly in short immersive sessions.
THE PROLOGUE
Quality seems to be caught up between do-more/test-continuously on one side and finding issues earlier/assure on the other side, without a clue on how to marry them in simple terms. There seems to be extreme reliance on rote continuous testing via automation rather than smart assurance exploiting human intelligence.
Some of the key challenges are : doing too much testing choking capacity, unsure about coverage i.e. test quality & therefore defect leakage, metrics not insightful and automation a drag unable to deliver its full potential.
The key inspirations to science and engineering of testing were:
- Properties of matter to state purity in terms of criteria & potential issues
- Sherlockian deductive thinking to hypothesis & proof
- Fractional distillation of separating elements of a mixture,
- The concept of ‘flow’ to be mindful, immersed
- The fishnet as a metaphor to understand test effectiveness
"The SHIFT" in HyBIST
Probing is deep questioning to gain clarity and preempt/uncover potential issues. Unlike traditional code validation, it goes beyond mere confirmation, focusing on proactive discovery and prevention of risks. Rapid generation of questions, suggestions, observations, and ideas is integral to illuminating uncertainties and uncovering latent problems.
At its core, probing fosters curiosity to understand well and unravel issues. It moves away from conventional test practices such as being limited by plans, scenarios that focus on “how-to”. It emphasises a flexible and adaptive mindset not limited by templates to a broader & deeper spirit of inquiry.
Process is a set of activities we do, normally agreed upon by stakeholders in an organisation. Process is great in enabling overall discipline but the “individual habits” of good practice are vital to being effective and rapid.
Practice on the other hand is how we do these activities, largely driven by individual brilliance. It is about how we breakdown problems, ideate solutions and rapidly implement them. It is about techniques, principles, guidelines(heuristics) that one uses to solve problems.
The shift has been to make processes light so that they are nimble and responsive. Process is like route map whilst Practice are driver skills to get to the destination quickly and safely. Equipping an individual with good method(s) that when practiced continually enables one to deliver great work, the outcome of ‘practice’. Mere processes don’t cut it; however when good individual practices become common place, they get converted to “Best/Good practice” becoming part of the process.
Testing is more than assessing whether something is correct or incorrect. It is an exploration into the unknown, of discovering interesting stuff that demands an intelligent human. Testing is not always postcode-based; it is about doing earlier to detect, prevent, or to enhance clarity.
Figuring out test approach, understanding behaviour and identifying the conditions, understanding the human psyche, the environment in which they use it, and the constraints, expectations, and potential mistakes that one might make are things that require human intelligence.
If we can do faster using a machine, then do so. If some things can be observed, perturbed, or exercised by using a tool or technology, then it is ‘smart’ to exploit it to do so. But understanding, figuring out questions, connecting various parts to see the big picture and using that to evaluate requires human intelligence.
It is necessary to exploit human intelligence, as testing is not merely a notion of evaluation at the end. It demands a systems-thinking approach to “look at individual trees but also understand the larger context of the forest”. Human intelligence plays a definitive role, and today it is aided by the intelligent tools that are more supplemental than replacement.
Repetition of tests and probing that is hard to be done by a human are those that demand tools/technology i.e., machine intelligence, that we state as automated testing. A harmonious mixture of machine intelligence and human intelligence helps deliver clean code.
Testing is not a mere compliance-driven activity to validate if meets specification/needs, rather it is probing to understand well and see out potential interesting issues.
It is about questioning, embracing uncertainties, and discovering scenarios in not explicitly stated in specifications. It’s a human-powered exploration, not always scripted, which leads to intriguing observations or potential issues. These observations might not always yield a binary outcome but prompt further consideration based on their significance to the project.
Testing delves beyond specifications, akin to evaluating system wellness rather than just compliance. It necessitates human intelligence due to its probing nature, although some observations might be automated for subsequent checking.
Domain experience seems to be play a pivotal role to be able to detect issues. Is that good enough? What if we do not have a very deep domain experience? That is where the science & engineering of testing becomes necessary, the METHOD.
Experience is indeed very useful and necessary but not sufficient.The act of bounding the system user test, the ability to go beyond the stated boundary to seeing clarity, hypothesising probable issues and devising means to preempt/detect them is where the METHOD becomes paramount.
Testing is more often seen as an intense activity of evaluation. Is it just about executing or running? Not really. It requires good preparatory work before the act of evaluation, to understand the big picture, ask interesting questions about who, where, what, why, what-if and so on and in the process find missing information, seek clarifications, discover potential issues, jot down observations, and come up with suggestions too. Not just viewing testing as an act of evaluation, but pulling it forward. This process helps clarify things and preempt issues.
Validation is not a physical act of execution; it is intellectual, an invisible process that goes on in one’s brain to figure out possibilities and probabilities of what may go wrong, what may happen that is not yet spec’d out, and come up with interesting questions to answer to preempt, to assure. Just like for good health, activity or exercise is good for the body, but rest and sleep are equally important.
Testing is not always an intense activity, it is silent thinking, quietly exploring in the mind’s eye too, to visualise correct flows and see issues too, to pre empt.
In today’s world, there is an extreme focus on speed, the need to test quickly, to test frequently so that build/release is not compromised. The focus is on automation to accomplish this. Is this good enough?
It is not about doing more, and more quickly it is about doing smart QA to do less to accomplish more. It is about :
– don’t do at all (prevent) – sensitive to probable & pre empt.
– do early – detect issues early so that they do not fester & become a pain later.
– do optimally – regress smartly, test just enough.
– do less – delete via smart automation.
DIVE into HyBIST
Probing is deep questioning to gain clarity and preempt/uncover potential issues. Unlike traditional code validation, it goes beyond mere confirmation, focusing on proactive discovery and prevention of risks. Rapid generation of questions, suggestions, observations, and ideas is integral to illuminating uncertainties and uncovering latent problems. At its core, probing fosters curiosity to understand well and unravel issues. It moves away from conventional test practices such as being limited by plans, scenarios that focus on “how-to”. It emphasises a flexible and adaptive mindset not limited by templates to a broader & deeper spirit of inquiry.
The HyBIST approach to SmartQA is smart probing i.e., designing smart probes and probing a system smartly.
Smart PROBES are those that help uncover issues. There may be viewed as : (1) Intellectual probes that really are observations, questions, suggestions, criteria, potential issues & conditions, ideas integral to illuminating uncertainties and uncovering latent problems to help in *preempting* issues and (2) Executable probes which really SmartChecks and test cases to *detect* issues.
Smart PROBING relates to the act of probing and is about (1) being immersive i.e mindful, take notes in free form , focussing to understand depth and also meandering to understand the broader picture and (2) doing in short sessions of Reconnaissance, Exploration & Recoup.
The act of probing is performed in three phases – Reconnaissance, Exploration & Recoup, in short focussed sessions.
– Reconnaissance: Get a big picture of system, and create maps to explore.
– Exploration: Dive deep to understand entities and then evaluate them.
– Recoup: Analyse what has been done, learn and course correct.
The TWELVE key concepts of HyBIST is either a technique/principle/guideline. These assist you in logically dissect, analyse and understand information to sharpen clarity, preempt issues, figure out test approach, design & review scenarios/cases, analyse impact to regress smartly and assess practice.
The TWELVE key concepts are :
– 360 POV
– Entity granularity
– Quality level
– Orthogonality principle
– Landscape
– Product map
– Test scope
– Design method
– Perturbation analysis
– Potency & Immunity
– Smart checks
– Defect map
The FIVE practices of HyBIST enable one to immerse deeply, respond rapidly and stay connected to the context.
The FIVE key practices are :
– Note taking
– Mindful sessions
– Language style
– Survey, Swoop & Iterate
– Focus & Meander
HyBIST is useful in preempting issues (pre-code), detecting issues (post code) and in deep & focussed analysis of issues(post release) to figure out clearly actions to enhance test effectiveness & efficiency.
The key use cases are :
1.Validate a USER STORY: Probe requirements, Preempt issues.
2.Validate an ENTITY: Probe feature, requirement, flow to detect issues.
3.Validate SPRINT: Probe entities in a build, detect issues.
4.Validate PRODUCT: Probe whole product, uncover issues.
5.Analyse PRACTICE: Assess the probing done, to improve.
KEY CONCEPTS in HyBIST
See system/entity from 360 view to get a holistic picture to review/test/retest smartly.
This is about seeing system or entity in a comprehensive manner, examining it from multiple perspectives: business objectives, end-user needs, deployment strategies, and internal structural aspects. This holistic approach helps understand the entire system’s functioning.
See the system as composed of different sized entities to characterise system well and test smartly.
This is about seeing a system as a collection of different-sized entities, from small structural components to complete business flows, aiding in understanding system composition for effective testing. In HyBIST the various entities are – structural COMPONENT, technical FEATURE, user REQUIREMENT & business FLOW.
See validation as multi staged filtration.
This is about treating validation as a multi-staged filtration process, akin to fractional distillation, with the aim to separate and uncover different issues optimally at different levels that consists of specialised focussed tests.
See validation as multi staged filtration. Understand Levels, Types & Techniques are orthogonal.
This establishes a relationship between quality levels, test types, and design techniques. By applying specific tests and techniques at corresponding levels, one can uncover various potential issues optimally.
See the BIG picture.
This helps in understanding the overall system context, usage profile, and the static structure of the system, to aid in asking pertinent questions and clarifying test objectives. This focuses on gaining a holistic and detailed overview of the system’s context and structure.
Map what is used by who, where & expected to satisfy what.
The various maps help in understanding who uses what, what may be the expectations, how different system entities interact, and the various environments in which the system operates to help you understand well and test effectively & efficiently.
– Persona map to understand who and what they need & expect
– Scope map to understand attributes expected from various entities
– Interaction map that helps you establish relationships between the various entities so that you can regress smartly and
– Environment map to understand the various environments that you need to validate on.
See scope of validation as a cartesian product.
This guides in formulating a plan or strategy based on the validation scope, seen as a cartesian product what to test and test for what. The ‘what’ here signifies the various entities under test while ‘test for what’ are the various types of tests of interest. Aligning entities, levels, and types establishes the scope of validation, laying the foundation for a strategy that evolves into a plan.
SmartDesign- coming up with different scenarios from different views.
This outlines the approach to designing probes to uncover potential issues in the system under test. This is a three-fold approach: using a spec-driven forward approach, an expectation-based backward approach, and fault-oriented middle approach to come top with probes(scenarios). This diversification in viewpoints aids in crafting smart probes from various perspectives.
Analyse impact of change via level wise criteria.
The concept of analysing the impact of change, known as perturbation analysis uses quality levels and associated criteria to understand potential impacts due to a change. It helps discern potential effects of modifications on entities by analysing impact across levels, similar entities, and different granularities, to gauge the extent of perturbation into the broader system.
Effective design & efficient regression (How-to).
Potency and immunity, as concepts, centre around assessing the effectiveness of test cases. Potency evaluates the ability of test cases to uncover specific defects within entities, aiming for minimal yet effective test coverage. Conversely, immunity recognises instances where the system becomes robust against specific test cases due to their continuous passing, prompting a reconsideration of their necessity.
Assistance to probe well – focus, expand, leverage.
Transitioning to a smart check from a checklist aims to refine the process. Smart checks serve as guides rather than strict instructions, encouraging focus, expanding thinking, and leveraging experience. They suggest areas for consideration without imposing rigid mandates, enhancing the testing process.
Map defects well to assess & act.
The concept of a defect map aids in intelligent analysis and purposeful action. It maps defects based on their source, characteristics, and impact on quality, test effectiveness, and process efficiency, enabling informed decisions and actions.
KEY PRACTICES of HyBIST
Jot down interesting bits of information.
The first key practice involves note-taking, a critical aspect of observing and assimilating information when analysing systems, reading specifications, formulating ideas, or exploring the product under test. It aids in tracking interesting observations, issues, suggestions, and questions. Note-taking is fundamental to smart queuing strategies.
Be fully immersed, connected, in a state of flow.
This practice aims to maintain complete immersion, connection, and a state of flow during short, rapid testing sessions. This emphasises being fully engaged and connected during the testing process.
Communicate well.
This practice focuses on effective communication. It explores the styles and modes of communication—be it textual, pictorial, or otherwise—to facilitate thorough and quick communication during testing activities.
Understand well.
This is about ‘how-to understand complex systems’. It encompasses various approaches: surveying from a top-down perspective, smoothing processes from a bottom-up angle, and iterating through multiple cycles to gain comprehensive understanding.
Focus on intent, but explore.
This practice advocates for a balance between staying focused on the task at hand and exploring beyond the typical routes. It emphasises the necessity to not only remain focused but also explore alternate paths for better contextual understanding.
HyBIST in PRACTICE
This is about probing a requirement at early stage to preempt issues rapidly & effectively.
Here probing is about :
-clear about big picture? : who uses it, what stage of dev it is, is it new/enhanced, where may it be deployed and what are expectations of this
– what existing entities could this affect
– clear about intended criteria & behaviours?
– is enough info for first cut design available i.e. conditions, i/o clear & complete? conflicts, attributes ?
– is it testable?
This is about probing a feature/requirement/flow (new or enhanced) to detect issues rapidly & effectively i.e. test a feature rapidly.
Here probing is about :
-understanding the big picture
-figuring out what all to retest
-figuring out criteria & behaviours
-designing first cut scenarios, develop Smart Checks and
-while evaluating, observe & improvise
This is about probing a given build to detect issues rapidly & effectively. i.e. test entities delivered in a sprint.
Here probing is about :
-connecting to the big picture
-figuring out flows to retest
-scoping out what to test/retest
-figuring out criteria & behaviours
-designing first cut scenarios, develop Smart Checks and
-while evaluating, observe & improvise
This is about probing a product to uncover issues rapidly & effectively. i.e. test a completed software/system rapidly.
Here probing is about :
-understanding the big picture
-setting up the baseline in terns of what features, requirements & flows to validate
-setup scope & formulate test plan
-figuring out criteria & behaviours
-designing first cut scenarios, develop Smart Checks and
-while evaluating, observe & improvise
to test the new & enhanced and retesting the impacted existing.
This is about assessing post release issues to ascertain how well the probing was done, to improve.
Here probing is about :
-analyse issues(bugs) in terms of levels, types, entities
-analyse reasons of miss i.e. missing scenarios or missed (re)testing it
KEY OUTCOMES of HyBIST
The key concepts of HyBIST- 360POV, Landscape & Product Map enables rapid surveying to get a brilliant view, sharpen clarity, and pave the way for superior validation.
Diving deep (Exploration phase) in an immersive manner applying key concepts Design method, Perturbation analysis, Test potency results in smart probes that weaves a brilliant net with superior coverage.
Test potency and immunity are interesting key concepts that provide deep insights into test adequacy/ effectiveness. Quality level based escape analysis enables actionable improvements.
The shift from doing to probing results in questioning, hypothesising, proving focussing on testability, preempting & early detection while immersive sessions facilitate deep connection with context to test well.
Masterclass on HyBIST
The Masterclass on The HyBIST Approach to SmartQA provides a deep foundational knowledge on testing smartly as SIXTY FOUR topics and TEN worked out examples in THIRTEEN lessons.
Click here for details of this masterclass.
HyBIST Tool
doSmartQA is an assistant for smart session-based testing. It helps to interrogate, hypothesise issues, design and evaluate user story or a set of stories in a sprint deeply and rapidly.
Read more...