9  Stage 6: Test Coverage

9.1 Orientation

Researchers stop collecting for many reasons. A deadline arrives. Access closes. New searches repeat old results. The corpus seems sufficient. These reasons should not be disguised as one claim that the evidence is complete.

9.2 Learn

Coverage testing compares the developing corpus with the dimensions and source families on the map. Look for empty cells, thin cells, dependent cells, contradictory cells, and cells filled by one institutional perspective. A large number of records can conceal weak coverage when they repeat one source stream.

The language of saturation has several meanings and does not fit every qualitative tradition.¹ Information power offers another way to reason about adequacy: a focused aim, specific material, strong dialogue with the evidence, theoretical support, and an intensive analytic strategy can reduce the amount required.² These ideas concern sampling and analysis. An evidence map adapts them cautiously to documentary collection.

Track information yield. A search cycle has high yield when it adds a new source family, changes a definition, reveals a rival explanation, fills an important gap, or alters the interpretation of existing evidence. Yield declines when new records repeat known information from the same underlying sources.

9.3 Worked Example

A terrorism corpus may contain thousands of event records while remaining thin on coding changes, local reporting gaps, or the institutional reasons that one event entered the database and another did not. Coverage testing shifts attention from event totals to the process that produced them.

A fragment of the coverage matrix marks dimensions against source families.

Dimension (row) / family (column)	Scholarly database	Local reporting	Government records
Event counts	Adequate	Thin	Dependent on database
Coding changes	Thin	Empty	Empty
Local meaning	Empty	Thin	Contradictory

Thousands of records can leave whole rows empty. A dependent cell repeats a source already counted elsewhere.

9.4 Try It

Mark each evidence-map cell as empty, thin, adequate for the present claim, contradictory, or inaccessible. Write one sentence defending every “adequate” judgment. Then rank the unresolved cells by their likely effect on the argument.

9.5 Guided AI Workflow

Provide a redacted summary of the map. Ask the model to identify repeated dependence, absent actor perspectives, unsupported comparisons, and claims resting on one source family. A usable prompt: “From this summary of an evidence map, identify repeated dependence, absent perspectives, and claims resting on a single source family.”

Permitted input: A redacted summary of the map, with no participant identities or protected locations.
Do not provide: The evidence itself, restricted records, or protected collection detail.
Verify: Check each observation against the actual register. A model with web or retrieval access may consult sources you cannot see; treat its inputs as unverified.
Record: Log which observations changed the gap analysis, and the model and date.

The model sees the summary supplied to it, not the evidence, unless the tool has retrieval access. When the map cannot be summarized without exposing protected material, run this stage without the model.

9.6 Integrity Checkpoint

Some gaps should remain. Ethical restrictions, participant safety, legal obligations, and community agreements can make collection improper. Confirm before saving:

Protected gaps marked: gaps that ethics or agreements require are recorded as protected, not as failures.
Overlooked gaps separated: a gap left by oversight is distinguished from one left by principle.
No pressure to fill: no protected gap is filled to improve apparent coverage.

9.7 Save the Artifact

Save gap-analysis-v0.1.xlsx. Required fields are the cell (dimension and source family), status (empty, thin, adequate, dependent, contradictory, or inaccessible), priority, consequence, proposed action, access constraint, and decision date. A later researcher should be able to distinguish an overlooked gap from a known limit.

9.8 Advanced Practice

Contradiction is not a defect to eliminate. It may reveal distinct institutional views, temporal change, strategic deception, or differences in measurement. The map should preserve disagreement until the analysis explains why it exists.

Braun, Virginia, and Victoria Clarke. “One Size Fits All? What Counts as Quality Practice in (Reflexive) Thematic Analysis?” Qualitative Research in Psychology 18, no. 3 (2021): 328–52. https://doi.org/10.1080/14780887.2020.1769238.

Malterud, Kirsti, Volkert Dirk Siersma, and Ann Dorrit Guassora. “Sample Size in Qualitative Interview Studies: Guided by Information Power.” Qualitative Health Research 26, no. 13 (2016): 1753–60. https://doi.org/10.1177/1049732315617444.

Saunders, Benjamin, Julius Sim, Tom Kingstone, Shula Baker, Jackie Waterfield, Bernadette Bartlam, Heather Burroughs, and Clare Jinks. “Saturation in Qualitative Research: Exploring Its Conceptualization and Operationalization.” Quality & Quantity 52 (2018): 1893–1907. https://doi.org/10.1007/s11135-017-0574-8.

---

1. See Benjamin Saunders et al., “Saturation in Qualitative Research: Exploring Its Conceptualization and Operationalization,” Quality & Quantity 52 (2018): 1893–1907, https://doi.org/10.1007/s11135-017-0574-8 and Virginia Braun and Victoria Clarke, “One Size Fits All? What Counts as Quality Practice in (Reflexive) Thematic Analysis?” Qualitative Research in Psychology 18, no. 3 (2021): 328–52, https://doi.org/10.1080/14780887.2020.1769238.

2. See Kirsti Malterud, Volkert Dirk Siersma, and Ann Dorrit Guassora, “Sample Size in Qualitative Interview Studies: Guided by Information Power,” Qualitative Health Research 26, no. 13 (2016): 1753–60, https://doi.org/10.1177/1049732315617444.