2 Origins and Purpose
2.1 The First Version
The original project began nearly a decade ago with a practical frustration. We could see the workflow a qualitative project required, from its conceptual foundations through collection, structure, preservation, analysis, and a reproducible path from evidence to claim. But the available tools could manage only fragments of that chain. Researchers were being told that digital technology had democratized knowledge, and in an important sense it had. Search engines opened routes into literatures that once required proximity to a major library. Smartphones placed a camera, recorder, scanner, map, and notebook in one pocket. Cloud storage made files available across continents. Yet each gain created another decision that methods training rarely addressed.
What should be collected? How should it be named? Which metadata should travel with it? When should a record enter a reference manager, a qualitative analysis system, a spreadsheet, or an archive? What must remain linked to the original? What does a researcher do when the same event appears in five incompatible forms?
Our early drafts tried to answer these questions by following the research process. They moved from literature review to field collection, from unstructured material to a working database, and from analysis to public communication. Some passages remain useful. Others read like records of a vanished software market. Product descriptions aged faster than the methodological problems they were meant to solve.
This failure taught us something. A book organized around tools will inherit their expiration dates. A book organized around research decisions can survive them.
2.2 The New Pressure
Large language models supply some of the processing power the original workflow lacked. They change the scale of the problem, but not its nature. A researcher has always relied on aids to memory and classification. Indexes, catalogues, research assistants, codebooks, and bibliographies all extend individual capacity. The new systems differ in three respects. They respond in natural language. They infer patterns across large bodies of text. They produce answers whose fluency can hide the weakness of their evidentiary basis.
The practical temptation is easy to understand. A researcher facing two hundred articles may ask a model for the dominant themes. A scholar entering an unfamiliar archive may request names, dates, and likely repositories. A doctoral student may ask for missing literature. Each request can produce a useful lead. Each can also collapse the distinction between a plausible sentence and a supported claim.
The distinction matters because qualitative research depends on context that systems often discard. A phrase changes meaning when removed from the institutional conflict that produced it. An interview gains or loses force according to who conducted it, when it was recorded, and what the speaker had reason to conceal. An official document may describe an event precisely while omitting the actors excluded from the bureaucracy that created the record. Metadata is part of the evidence.
This is why provenance sits near the center of the revived project. Provenance records three things: the record itself, the activities that created or transformed it, and the agents responsible for those activities. These are the entities, activities, and agents of the W3C provenance model.1 It lets a reader ask what a record is, what produced or altered it, and who answered for each step. The model was built for data workflows. Social evidence adds questions it does not name, among them intent, silence, and power. The guide borrows the vocabulary as a heuristic, not a foundation. It does not require researchers to adopt a technical ontology. It requires them to preserve the questions that ontology makes explicit.
2.3 The Argument
We argue that disciplined collection rests on three commitments. A bounded question limits what enters the corpus. A map of the evidence that could bear on the question guides the search. A record of the decisions that shaped the corpus preserves the reasoning. Not every tradition accepts the first commitment in the same form. Grounded and interpretive projects often hold the question open longer, and the map and the record then travel with a question still in motion.
The three commitments correct different failures. Without a bounded question, collection expands by association. Without an evidence map, the most visible sources crowd out harder forms of evidence. Without a decision record, later readers cannot distinguish absence in the world from absence in the search.
No software can resolve these failures because none is technical alone. A search engine cannot decide whether a municipal ledger matters more than a minister’s memoir. A language model cannot determine whether repeated agreement indicates corroboration or common dependence on one wire report. A reference manager cannot explain why the research team stopped collecting one source family while continuing another.
The researcher must decide. The system should help preserve the decision.
The vocabulary of this guide carries a commitment. Words such as evidence, coverage, and gap assume that a collection can be more or less complete for a stated purpose, and that a reader can judge the difference. Strong interpretive and constructivist traditions contest that assumption. They treat a field’s categories as part of the phenomenon and resist measuring a corpus against the world. This guide does not claim to serve those traditions on their own terms. It offers a structure for making collection decisions visible, and it expects researchers in every tradition to adapt or reject the parts that do not fit their own.
2.4 A Guide with Two Speeds
The novice path makes tacit practice explicit. It explains why a decision matters, presents a worked example, supplies a structured exercise, and identifies common errors. The advanced sections address cases in which the simple procedure breaks down. Concepts remain contested. Teams classify the same source differently. Repositories impose incompatible metadata. Ethical limits create gaps that cannot be filled. The experienced reader can move quickly through the shared method and linger where these complications begin.
The two paths remain on the same page. We do not divide readers into permanent categories. An experienced ethnographer may be a novice in digital preservation. A doctoral student may possess deep knowledge of a language or archive that the senior investigator lacks. Expertise follows the task.
2.5 What This Draft Does
This project is a full living field guide published in reviewed modules. The current manuscript contains the first two. One moves from a question to an evidence map and collection plan. The other turns the scholarly source family into a literature corpus, synthesis, coverage audit, and stopping rule. Both use the same page pattern, preserve consequential decisions, and assign AI bounded roles.
These modules are the first tested layers of the book, not its final boundary. The next revision makes the guide’s epistemological position explicit, adds a composite oral-history case, and carries the collection plan into a documented research corpus. Later modules will address preparation, analysis, claim testing, writing, responsible sharing, and maintenance.
The manuscript also opens a conversation between the original authors. The information environment changed; the purpose remains recognizable. We wanted to help researchers work with greater discipline and less fear of abundance. We still do.
The formal vocabulary of entities, activities, and agents follows the W3C provenance model. See Timothy Lebo, Satya Sahoo, and Deborah McGuinness, “PROV-O: The PROV Ontology,” W3C Recommendation (World Wide Web Consortium, 2013), https://www.w3.org/TR/prov-o/.↩︎