Exploratory testing is a powerful technique to discover hidden issues in a piece of software. Many clients, however, ask QA analysts to forgo exploratory testing in favor of scripted test cases based on predefined product requirements.
Although scripted test cases are valuable tools for uncovering bugs in a product's code, exploratory tests help the QA team discover hard-to-predict issues within the finished product's features.
Ideally, effective QA should include exploratory testing in its suite of tools alongside other methods such as UAT, A/B Testing, Stress testing, Shift Left Testing, etc.
This article will explain how exploratory testing works, what skills a tester must have, and a strategy for designing thorough test cases.
What is Exploratory Testing?
First defined by software engineer Cem Kaner in 1983, exploratory testing is:
"A style of testing that emphasizes the personal freedom and responsibility of the individual tester to continually optimize the value of her work by treating test-related learning, test design, test execution, and test result interpretation as mutually supportive activities that run in parallel throughout the project."
In short, exploratory testing involves designing and executing tests as one explores a software's features.
Instead of only relying on scripted tests prepared before the testing phase, QA testers use their experience and creativity to generate new tests as they learn how to handle the system.
Because exploratory testing evaluates a product's finished design and features, many consider it a black-box testing technique. Black box testing methods evaluate a product without looking directly at the inner workings of the software itself.
Exploratory testing differs from other QA methods as it does not involve any specific technique or evaluate any particular feature type. Instead, exploratory testing emphasizes the commitment of the tester to thoroughly explore the product and develop test cases to assess it as needed.
The Tester's Mindset
The proper testing mindset ensures the success of exploratory testing. In addition to the necessary QA skills an engineer should have, testers using an experimental approach should possess the qualities below. Each contributes to a mentality that motivates one to find potential issues and improve software quality.
Testers Should Always Remain Curious
QA testers must always maintain a high level of curiosity about how a given software works. A QA Engineer without curiosity will fail to explore the limits and defects in a product and falsely approve of poor quality software.
Tester Should Always Be Open-minded
QA engineers must think outside the box and discover alternative ways a user could interact with the software other than how the developers intended. Often, confused users may use the software wrong and end up in unanticipated—and often buggy—scenarios.
Testers must develop a habit of taking the unofficial path to see how a user could potentially fall into a gray area in the software. Without open-mindedness, a tester may miss vulnerabilities and defects that compromise the software quality.
Tester Should Always Assume There Are Defects
Some people may misunderstand the role of testers and say they are only pessimistic users that criticize developers' work. However, testers' only mission is to find defects and help the developers improve their software.
They attempt to use the software unintendedly and find gaps between its defined requirements and actual behavior. Testers must remain skeptical. Otherwise, they may skip over specific features, assuming they work okay. In QA work, we can never make this assumption.
How to Create an Exploratory Map
For QA analysts just getting started on a new job, who still have little knowledge of the software they are testing, I recommend creating an exploratory map of the entire site.
Exploratory maps help engineers keep a detailed account of features and their respective levels of depth in software.
Creating a map would let us know how deep features go and what test cases we need to evaluate. It's a straightforward technique too!
Below, we'll list the three steps to create an exploratory map.
- Create a new Excel spreadsheet and add columns named "Level1"," Level 2", "Level 3," and so on.
- Start navigating the website and divide the content from the home page into different sections. Add the features from the home page to the "Level 1" column. The content inside those features (e.g., buttons, videos, menus, dropdowns, forms, etc.) goes in the "Level 2" column. The interactive items within "Level 2" elements belong in the "Level 3" column.
- Continue this process, finding features and elements and listing them in their respective layer column until we finish covering all the application levels.
Example:
Consider a navigation bar (level 1) content on a site's home page with level 2 elements such as the Home link, About Link, and Products dropdown menu.
Inside the Products dropdown, we find level 3 link elements such as Development, Consulting, and QA tools.
We can also color the different items as we are testing them to know which ones have been already tested or not.
Section: Homepage
Level 1Level 2Level 3Navigation barLink: HomeLink: DevelopmentLink: AboutLink: ConsultingDropdown: ProductsLink: QA tools
Exploratory maps help us measure the effort needed by QA analysts to test the entire site. Maps can also help with regressions or tests that ensure changes and updates don't impair functionality or performance.
With a map, we can start understanding the following information:
- Where content is located and why
- Which interactive items let users control the application
- The resources needed to test particular sections
Remember that the QA team should also determine testing scopes before beginning test execution. Otherwise, our testing strategy may use too many resources. For example, if we detect 1500 interactive controls or items, we cannot execute 1500 test cases every time we assess the application.
Such a procedure would be too resource-demanding and inefficient. Testing phases should always focus on specific requirements instead of addressing all of them.
Many test cases do not always correlate with higher product quality. If test cases are not going through all the levels of an application found in the exploratory map, then the test would be superficial and only detect low-priority defects.
Conclusion
The exploratory testing approach encourages the QA team to explore creative ways to evaluate and improve a product based on human curiosity and a personal commitment to quality.
QA analysts continuously learn and adapt to the software at hand to find defects in unexpected areas. When used alongside other QA methods, exploratory testing proves a valuable resource to effective QA teams.
