18 Review Articles and Meta-Analysis in Transition
18.1 Orientation
A review article once appeared to be a retrospective product. Its authors surveyed an established literature, organized its main findings, and published a settled account. That model still has value, but it no longer describes the entire task. Digital discovery produces larger and more varied corpora. Open research practices expose protocols, coding decisions, data, and analysis. New studies can make a synthesis stale soon after publication. AI can accelerate several clerical operations, while also multiplying plausible errors.
These changes are turning the review from a static article into a form of research infrastructure. The article remains an argument, but the argument now rests on inspectable and potentially reusable objects: a protocol, search history, screening record, coded corpus, analysis, sensitivity tests, and update policy. A trustworthy review links these objects without pretending that procedural detail can replace judgment.
This chapter explains the present and plausible future of review articles and meta-analyses in the social sciences. Its central claim is simple. New tools can shorten discovery, screening, extraction, and updating, but they cannot decide what evidence is comparable or what variation means. The durable unit of progress is therefore not an automated summary. It is a versioned chain of decisions that another researcher can inspect, challenge, and extend.
18.2 Learn
18.2.1 Begin with the review family
The term literature review covers several products with different purposes. A narrative or integrative review may clarify a concept, reconstruct an intellectual debate, or develop a theoretical account. A scoping review maps the extent and characteristics of a broad body of work. A systematic review answers a bounded question through explicit, reproducible procedures. A qualitative evidence synthesis interprets meanings or mechanisms across studies. An evidence and gap map describes the distribution of evidence without necessarily estimating a common effect. An umbrella review examines existing reviews. A rapid review makes declared concessions to produce a timely answer. Meta-analysis statistically synthesizes estimates that meet defined comparability conditions.1
This review family is not a hierarchy. A systematic review is not automatically superior to an interpretive review. A meta-analysis is not a badge of rigor that can be added to any corpus. Each form answers a different question and makes different assumptions. The first design decision is therefore functional: what must the review allow its reader to understand, estimate, compare, or decide?
A review of how a concept changed across disciplines may require close comparison of definitions and citation lineages. A review of whether an intervention changes an outcome may support statistical synthesis. A review of how participants experience that intervention may require qualitative synthesis. A review of an emerging field may first need a scoping map because its populations, measures, and outcomes remain unstable. Combining these products can be useful, but their inferential roles must remain distinct.
Method names do not settle the design. Authors sometimes label a broad database search a systematic review even when eligibility decisions and synthesis rules remain unclear. Other reviews apply a narrow checklist to a question that requires historical or conceptual interpretation. The defensible sequence runs from question to evidence type, from evidence type to review form, and from review form to procedure.
18.2.2 Understand what meta-analysis adds
Meta-analysis is not a synonym for systematic review. It is a set of statistical methods for describing and explaining a distribution of comparable effect estimates within a defined corpus.2 The systematic review establishes how studies entered that corpus. The meta-analysis addresses what their estimates imply under declared statistical assumptions.
This distinction matters because a precise pooled estimate can conceal a weak evidence base. Studies may use different constructs under the same label, compare unlike populations, report incompatible outcomes, or estimate effects under different causal conditions. A statistical model can combine the numbers, but it cannot make the underlying questions equivalent.
Before pooling, construct an estimand table. Each row should record the population, treatment or exposure, comparison, outcome, timing, design, effect measure, and adjustment set. Add the unit of assignment, unit of analysis, and level at which uncertainty was estimated. The table makes comparability a visible decision instead of an assumption hidden inside software.
The pooled mean is only one possible target. A review may seek the average association across settings, the distribution of effects, an expected effect in a new setting, or the moderators that account for variation. These are different questions. A model chosen for one should not be interpreted as if it answered the others.
18.2.3 Treat heterogeneity as substantive evidence
Social science effects often vary because institutions, histories, populations, measures, and implementation differ. Heterogeneity is therefore more than noise around a universal effect. It may be the principal finding.
A random-effects model acknowledges variation among the effects represented by the included studies. It does not prove that those studies form one meaningful population. Nor does it license an inference to every setting beyond them. The review must explain which sources of variation were expected before analysis and which appeared only after inspecting the data.
Report the distribution of effects, not only its mean. A confidence interval describes uncertainty around an estimated parameter. A prediction interval addresses a different question: where an effect from a comparable future setting might plausibly fall. When heterogeneity is substantial, the prediction interval may cross thresholds that the pooled mean does not. Its width can be more informative for policy than the significance of the mean.
Moderator analyses can examine theoretically specified variation, but they are vulnerable to low power, multiple testing, and ecological interpretation. A moderator measured at the study level cannot establish the corresponding individual-level mechanism. Separate planned tests from exploratory ones. Record which coding distinctions existed before results were examined.
Qualitative and quantitative synthesis can inform each other here. Close reading may identify institutional differences that define plausible moderator categories. Quantitative patterns may identify cases that deserve renewed interpretation. Mixed evidence becomes valuable when the forms remain legible, not when one is used to decorate the other.
18.2.4 Preserve dependence and multiplicity
Many social science studies report several outcomes, time points, subgroups, models, or treatment contrasts. Several articles may also analyze the same dataset. These estimates share observations and design decisions. Treating them as independent produces excessive precision and gives prolific studies or datasets disproportionate influence.
Every effect table should therefore include a study identifier, sample identifier, dataset identifier, outcome family, time point, and model family. This structure reveals dependent effect estimates before analysis. The reviewer can then select one estimate under a preregistered rule, model the hierarchy, use an appropriate multivariate method, or apply robust variance estimation when its assumptions and sample requirements fit the corpus.3
Dependence is also intellectual. A set of studies may repeat one operationalization, share code, or inherit the same measurement error. Statistical correction for clustered standard errors does not remove that shared conceptual ancestry. The written synthesis should distinguish repeated analysis from independent replication.
Model multiplicity creates another problem. Authors can often report many defensible specifications. Reviews that select one estimate per study after seeing the results may reproduce selective reporting. Collecting all relevant estimates and modeling their dependence can be preferable, but it requires careful coding and a declared strategy. Contemporary social science guidance identifies dependence, systematic heterogeneity, and publication selection as recurring issues that every quantitative synthesis should address.4
18.2.5 Examine selective reporting without a mechanical verdict
The available literature is not a neutral sample of completed research. Statistically striking results may be more likely to be written, submitted, accepted, or emphasized. Outcomes and specifications can also be selected within a published study. Gray literature searches may reduce some forms of selection while introducing new differences in reviewability and metadata.
No single funnel plot, regression, or adjustment method can diagnose and repair every form of publication bias. Small-study patterns can arise from heterogeneity, design quality, or genuine differences in populations. Statistical corrections rely on assumptions that may not fit the selection process. Use several sources of evidence: prospective registrations where available, dissertations and reports, requests for unreported results, outcome comparison within studies, sensitivity analyses, and a substantive account of incentives in the field.
The review should state what would have to be missing to change its conclusion. That question turns a generic limitation into a sensitivity claim. It also prevents the adjusted estimate from appearing as a recovered truth. A corrected result remains conditional on a model of what became observable.
18.2.6 Make the synthesis reproducible
Open practice changes a meta-analysis from a report about an analysis into a rerunnable research object. A strong public package includes the protocol, complete search strategies, screening decisions, extraction definitions, coded data, transformation code, analysis code, output tables, and a record of deviations. When materials cannot be shared, the package should describe the restriction and expose the largest safe portion.
Preregistration does not freeze a review against learning. It distinguishes planned decisions from changes made after contact with the evidence. A deviation log can record what changed, why it changed, when the team decided, and which outputs the change affected. Open materials allow readers to evaluate those decisions and allow later teams to update the review rather than reconstruct it.5
Transparency must extend to the quantitative data structure. A spreadsheet of final effect sizes is insufficient if it omits study relationships, transformation rules, excluded estimates, and coding uncertainty. Preserve raw extracted values alongside derived values. Give every effect a stable identifier that links it to its source location and coding decision.
Meta-research has found persistent gaps in the transparency and reproducibility of published meta-analyses.6 A long methods section does not by itself solve the problem. Reproducibility depends on whether the released objects recreate the stated result and make consequential judgment visible.
18.2.7 Design living reviews as governed projects
A living review incorporates new evidence under a continuing workflow. It is an update mode, not another review question type. The same systematic review can be maintained in living form when new research arrives frequently and could change a consequential conclusion.7
Living status requires more than an editable webpage. The team needs a surveillance strategy, update trigger, review interval, ownership rule, version number, correction process, and retirement condition. Each release should preserve a citable snapshot. Readers must be able to identify which evidence and analysis supported the version they used.
Updates also create statistical problems. Repeatedly recalculating and testing a pooled effect as new studies arrive can change error rates. A living meta-analysis therefore needs a declared updating method, especially when policy decisions depend on crossing a threshold. The correct method depends on the inferential target, expected evidence flow, heterogeneity, and cost of error. No universal update rule replaces these design choices.
Maintenance should be selective. A slowly changing historical literature may not justify continuous surveillance. A fast-moving policy question might. A review can leave living mode when new studies become unlikely, the conclusion stabilizes for its decision purpose, or resources no longer support responsible maintenance. Calling every online review living would conceal these commitments.
18.2.8 Give AI bounded work
AI can assist with query expansion, deduplication, screening prioritization, structured extraction, citation checking, code explanation, and update surveillance. Earlier machine-learning research found some mature applications for study identification, while extraction and judgment-intensive tasks required more development.8 Later reviews documented rapid growth in available systems, especially around search, screening, extraction, and synthesis.9
The number of tools does not establish their reliability for a particular review. Performance depends on the question, source type, language, study design, prevalence of eligible records, and definition of an error. A system that ranks relevant records near the top may save labor without being safe for autonomous exclusion. A fluent extraction may still assign a number to the wrong group, outcome, or time point.
Use AI as a proposal layer. The system can suggest records, fields, links, and checks. A controlled review step decides what enters the evidence base. For screening, validate recall on a representative researcher-coded sample and inspect every false exclusion. For extraction, require an exact source span and abstention option. For risk-of-bias assessment, causal interpretation, and final inclusion, retain human adjudication.
Human validation is not a ceremonial final glance. It is a measured procedure. Record the validation sample, sampling rule, reference coding, error categories, model and prompt version, decision threshold, and response to failure. Revalidate after material changes in the corpus or system.
18.2.9 Build skills, agents, and project context as separate layers
Repeated AI use becomes more reliable when a project separates procedure, execution, and memory. A skill is a reusable research procedure. It specifies when a task should run, what inputs it accepts, which steps it follows, what it must produce, and how its output will be checked. An agent is a bounded executor that receives a goal, approved tools, selected context, stopping conditions, and an output schema. It may combine several skills and adjust its plan as evidence appears. Neither should become the authority that decides what the project means.
This distinction prevents a common design error. A long prompt may contain a useful procedure, but it remains difficult to test, version, and reuse. A general agent may take many actions, but breadth makes its errors harder to locate. Skills should hold stable method. Agents should coordinate bounded work. Human gates should control consequential changes to the corpus, protocol, argument, and public record.
18.2.9.1 Design a minimal research skill stack
A skill should encode knowledge that the project would otherwise have to reconstruct. Its manifest should name its purpose, triggers, required inputs, procedure, output schema, verification rules, failure conditions, permissions, version, and benchmark cases. High-risk operations need narrow instructions and deterministic checks. Interpretive tasks need room for alternatives, but they still require evidence identifiers and an abstention route.
Five skills provide a useful starting stack for review-based social research:
| Research function | Reusable skill | Required artifact | Human gate |
|---|---|---|---|
| Literature discovery | Literature-discovery skill | Candidate register, query history, discovery-route coverage | Approve final inclusion and exclusion |
| Gap identification | Gap-analysis skill | Gap-claim register with supporting and contrary corpus entries | Authorize any claim that a field lacks evidence |
| Methodology | Methodology-audit skill | Question-design alignment memo, threat register, proposed tests | Approve protocol or analysis changes |
| Data collection | Data-collection skill | Candidate records, provenance links, validation report | Promote records into the research corpus |
| Project continuity | Project-context skill | Versioned project context packet and proposed change log | Modify the project charter, concepts, or accepted claims |
The literature-discovery skill should translate concepts into search vocabulary, vary discovery routes, preserve queries, deduplicate candidates, and flag coverage risks. It should not produce a list of “the most important” publications from model recall. Its unit of output is a candidate with a discovery route and verification status.
The gap-analysis skill should distinguish several kinds of gap. A corpus may lack evidence about a population, setting, period, mechanism, measure, comparison, or method. It may contain contradictory findings, weak replication, or a concept that has not been operationalized consistently. These are different research problems. The skill must also distinguish absence in the bounded corpus from absence in the field. A false gap occurs when failed retrieval, unfamiliar vocabulary, access limits, language bias, or an incorrect boundary is presented as a missing body of knowledge.
The methodology-audit skill should compare the research question with the evidence, design, unit of analysis, estimand, inferential claim, and known threats. It can propose alternative designs or sensitivity tests. It should not silently rewrite the approved protocol. Every recommendation needs a location, a reason, the evidence affected, and the decision that remains with the researcher.
The data-collection skill should enforce the transformations defined elsewhere in this book. It can create candidate records, check required fields, preserve source spans, detect likely duplicates, and route uncertain cases. It should write only to a staging area. Promotion into the verified corpus requires source review, rights and ethics checks, and a named human decision.
The project-context skill is different from the other four. Its task is not to perform a substantive review operation. It prepares the smallest sufficient account of the project for another task. It retrieves the current research question, approved scope, working concepts, source hierarchy, ethical limits, corpus and codebook versions, accepted decisions, unresolved disputes, artifact locations, and active release. It also reports conflicts among these objects rather than resolving them through a plausible summary.
Skills need tests before routine use. Discovery can be evaluated through recall, false exclusions, route diversity, and labor saved. Gap analysis needs cases where a missing concept is genuine and cases where it is only hidden by vocabulary or indexing. Methodology audits need known design defects and defensible designs that should not be criticized. Data collection needs stratified extraction and provenance benchmarks. Project-context tests should include stale decisions, conflicting versions, renamed concepts, and inaccessible artifacts.
18.2.9.2 Assign agents work packages, not identities
Language-model agents can interleave planning, retrieval, tool use, and revision.10 Research prototypes have also combined observation, retrieval, reflection, and planning in persistent agent architectures.11 These patterns make multi-step research assistance possible. They do not show that an agent can independently establish relevance, a scholarly gap, methodological validity, or causal interpretation.
A research agent should therefore receive a work order. The order names one goal, the skill versions it may invoke, permitted collections and tools, project context version, expected artifacts, prohibited actions, stopping rule, and escalation conditions. The agent should return evidence-linked proposals and an execution log. It should not inherit an open-ended instruction to “finish the literature review.”
The five skills above can support corresponding agents. A literature scout executes searches and updates the candidate register but cannot reject the final record. A gap analyst compares the verified corpus with the review protocol and produces candidate gap claims with counterevidence. A methodology critic tests alignment and assumptions without editing the protocol. A collection steward prepares and validates candidate data while leaving promotion to a reviewer. A context curator assembles the current project packet and proposes corrections without overwriting canonical project records.
Agent specialization should reduce ambiguity, not imitate an academic department. Adding several agents that use the same model, sources, and instructions does not create independent corroboration. Their errors remain correlated. Agreement among them is not consensus, and disagreement is not automatically informative. Even different model families may display distinct directional tendencies that happen to offset one another on a benchmark; averaging their outputs requires prior calibration and does not turn model votes into independent expert judgments.12 Use multiple agents when tasks can be separated by evidence, method, or validation role. Do not create a panel merely to generate more text.
Apply least privilege. A discovery agent usually needs search and read access, not permission to change the corpus. A methodology critic needs the protocol and selected artifacts, not participant data. A context curator needs approved project records, but it should not publish, delete, or revise them. Rights, privacy, security, and cost boundaries belong in the work order rather than in an informal reminder.
Consequential decisions require explicit human gates. These include approving inclusion and exclusion, declaring a research gap, changing the research question or protocol, promoting a candidate record, accepting a new variable, interpreting a causal relationship, and releasing a public claim. The gate should record who decided, which agent output was reviewed, what evidence supported the decision, and what changed downstream.
18.2.9.3 Preserve broad context through governed retrieval
The broad project context should live outside the conversation. A chat transcript mixes instructions, exploration, corrections, and obsolete claims. Treating all of it as memory makes current and superseded decisions difficult to distinguish. Supplying the entire project archive is not a solution either. Empirical work on long-context models shows that access to a long input does not ensure reliable use of information throughout that input.13
Create a project context packet with three layers. The stable layer contains the research question, scope, central concepts, evidence standards, ethics and rights constraints, and authority for decisions. The current-state layer names the active stage, corpus and codebook versions, accepted claims, unresolved questions, recent decisions, and release status. The task layer contains only the materials, instructions, and stopping rule needed for the present work.
Use progressive disclosure. Every agent receives a short project card and the task layer. It retrieves method, corpus, or source details only when the work requires them. Each retrieved item carries a stable identifier, version, date, and authority status. This design preserves the global argument while reducing irrelevant material and exposure of sensitive records.
The project context packet is a map, not a substitute for evidence. It may say that a definition was adopted in decision D-014, that corpus C-03 supports claim CL-08, and that a contrary memo remains unresolved. The agent must follow those identifiers to the relevant artifact before making a consequential recommendation. A compressed statement without provenance remains a lead.
Context updates should follow the same proposal boundary as data extraction. An agent may identify that the active question conflicts with the latest protocol or that a manuscript cites an obsolete corpus. It writes the proposed correction to a staging area. A researcher decides whether to update the canonical packet, records the reason, and increments its version.
This architecture supports continuity across literature review, gap analysis, methodology, and collection. The literature scout sees the current vocabulary and boundaries. The gap analyst sees the verified coverage audit. The methodology critic sees the approved estimand and threats. The collection steward sees the source frame and codebook. The context curator sees how their artifacts connect, but it does not turn their provisional outputs into settled project knowledge.
Evaluate the system as a chain. Record whether a task used the correct context version, whether the appropriate skill triggered, whether the agent stayed within permissions, whether its evidence links resolved, whether it abstained when required, and whether the human gate caught consequential errors. Productivity matters, but speed without these measures can make an incoherent project move faster.
18.2.10 Separate the present from the forecast
The current review article can already be modular and open. Protocols, machine-readable data, executable analysis, linked corrections, and citable releases are available practices. Semi-automated screening and extraction are emerging practices whose value must be demonstrated in the local corpus. Continuous synthesis across interoperable evidence registries remains a conditional future.
| Horizon | Review form | Required safeguard |
|---|---|---|
| Current practice | An article linked to protocol, corpus, code, and a citable release | Reproducibility, provenance, and disclosed deviations |
| Emerging practice | A living review with AI-assisted surveillance, prioritization, and extraction | Local validation, human adjudication, and version governance |
| Conditional future | A continuously updated evidence service built from interoperable study, claim, and effect objects | Durable standards, equitable access, preservation, and accountable ownership |
The conditional future is technically plausible but institutionally demanding. Journals and repositories would need stable citation rules for changing objects. Research teams would need credit for maintenance, correction, data stewardship, and negative updates. Shared schemas would need to preserve construct differences rather than forcing every study into one vocabulary.
The review article may become an interface over a versioned evidence system. Readers could inspect the current conclusion, trace it to effects or qualitative findings, change defensible assumptions, and compare releases. Such an interface would not eliminate authorship. Its design would make authorship more visible by showing where inclusion, coding, modeling, and interpretation shaped the answer.
This future also carries risks. Well-resourced fields may maintain sophisticated living syntheses while other regions and languages remain poorly indexed. Proprietary systems may make public evidence dependent on private infrastructure. Continuous updating can create an illusion of completeness and exhaust the teams responsible for it. The goal is not permanent motion. It is controlled revision when new evidence matters.
18.3 Worked Example
Consider a review asking whether community-policing programs increase public trust in police. The first search identifies experiments, quasi-experiments, surveys, qualitative interviews, and program evaluations across several countries. A single pooled estimate would answer the wrong question because the studies do not share one treatment, outcome, or causal design.
The team divides the project into linked products. A scoping map describes programs, populations, research designs, trust measures, and geographic coverage. A systematic review evaluates studies with a defensible comparison group. A qualitative synthesis examines how residents describe contact, legitimacy, safety, and unequal treatment. The products share a source register but maintain separate eligibility and synthesis rules.
The quantitative subset still contains hidden multiplicity. Several reports use one national survey. Individual studies report trust immediately after contact and again months later. Some programs combine foot patrols with public meetings, while others change only officer assignment. The estimand table exposes these differences. The effect table links multiple estimates to samples, outcomes, and program families.
The main model estimates an average across a defined family of programs. It also reports a prediction interval and planned comparisons by intervention component and institutional setting. A dependence-aware analysis prevents one large survey and its related articles from dominating precision. The review treats the remaining heterogeneity as a finding: program labels conceal distinct mechanisms and political contexts.
Publication-bias analysis finds that small published studies report more favorable estimates. The team does not declare the pattern corrected after one adjustment. It searches evaluation repositories and dissertations, compares registered outcomes where possible, and reports sensitivity to several selection assumptions. The conclusion becomes conditional: favorable average results do not establish that every program or setting will improve trust.
The public package contains the protocol, search histories, decision log, coded corpus, effect transformations, analysis code, and a redacted qualitative matrix. The team schedules annual surveillance but triggers an earlier update if a large multisite evaluation appears. Release 1.0 remains frozen and citable when release 1.1 adds new evidence.
An AI system expands intervention names and ranks new search results. It proposes extraction fields with source spans. Researchers validate screening recall on a stratified sample and adjudicate all exclusions near the decision boundary. The system never decides whether a program belongs in the same causal family or whether a trust measure is substantively comparable. Those judgments remain part of the review’s argument.
The team implements this arrangement as separate skills and agents. The literature-discovery skill records every query and route. A scout agent runs that skill against approved databases and deposits candidates in a staging register. The gap-analysis skill compares the verified corpus with the protocol. Its agent proposes a geographic gap, but the coverage audit shows that two relevant local-language indexes were never searched. The team records an unresolved coverage limit instead of claiming that no such research exists.
The context curator prepares project packet 1.3 for the annual update. It includes the current question, intervention taxonomy, approved trust measures, corpus release, open disagreement about quasi-experimental designs, and links to every decision. The methodology critic identifies that a proposed new synthesis would mix immediate and long-term outcomes. It recommends a protocol amendment but cannot make it. Researchers review the affected records, approve a revised timing rule, and issue packet 1.4 before the other agents resume work.
18.4 Try It
Start with one review question and write the decision it must support. Name the review form that best fits that purpose. Then state why two neighboring forms would answer different questions.
Create an estimand table for five candidate studies. Record population, exposure or intervention, comparison, outcome, timing, design, and effect measure. Add sample and dataset identifiers. Mark which studies could enter one quantitative synthesis and explain every exclusion from pooling.
For the eligible estimates, draw a dependence map. Connect effects that share participants, datasets, outcomes, authorship pipelines, or measurement instruments. Choose a strategy for each connection and state what assumption the strategy requires.
Draft a result paragraph that reports the mean, its uncertainty, heterogeneity, a prediction interval, and one limitation arising from the corpus. Remove any sentence that treats statistical significance as a complete substantive conclusion.
Finally, write an update policy. Specify surveillance sources, cadence, update triggers, owner, release format, correction route, and retirement condition. Label each planned feature as current, emerging, or conditional future. This prevents an aspiration from being presented as an implemented method.
Design one research skill for the review. Write its trigger, required inputs, procedure, output schema, verification rule, permissions, failure condition, and three benchmark cases. Then write an agent work order that may invoke the skill. Give the agent one goal, a context version, approved tools, a stopping rule, and an escalation route.
Create project context packet 0.1. Limit its stable layer to one page. Add a current-state table and a task layer for one operation. Ask a colleague to locate one accepted decision, one unresolved dispute, and one source artifact using only the packet. Revise any entry that cannot be traced.
18.5 Guided AI Workflow
Begin by issuing a versioned project context packet and work order. Then use AI in four bounded passes. In the design pass, provide the research question and intended decision. Ask the system to compare plausible review forms and list the evidence each would require. In the structure pass, provide verified study metadata and researcher-defined fields. Ask it to flag possible construct mismatch, dependence, and missing values without deciding eligibility. In the audit pass, provide the analysis plan and output. Ask for unsupported inferences, omitted sensitivity tests, and discrepancies between the estimand and conclusion. In the update pass, ask it to rank new candidate records and explain each ranking.
A useful audit prompt is:
Review this estimand table, effect structure, analysis plan, and draft conclusion. Identify claims that exceed the eligible corpus, effect estimates that may be dependent, sources of heterogeneity not represented in the model, and statements that confuse a pooled mean with an expected effect in a new setting. Cite only the supplied record identifiers. Return “not established” when the record is insufficient.
Permitted input: Verified metadata, researcher-authored protocols, rights-cleared text, structured extraction fields, de-identified effect data, code, and public analytical output.
Do not provide: Licensed full text without permission, confidential peer-review material, protected participant data, restricted archives, or identifiable sensitive records.
Verify: Check every proposed inclusion, exclusion, extracted value, source span, transformation, dependence link, citation, and analytical claim. Test screening and extraction performance on a representative human-coded sample.
Record: Save model and version, skill and agent versions, project context packet, work order, prompt, input description, tool actions, output, reviewer decision, error category, protocol and corpus versions, analysis release, and date.
Do not ask a model to select the final review form, infer missing statistics without a declared method, adjudicate risk of bias alone, or choose a publication-bias correction after seeing which result is favorable. Do not accept a causal or policy conclusion that cannot be reconstructed from the reviewed evidence objects.
18.6 Integrity Checkpoint
A comprehensive search can still reproduce unequal visibility. Dominant databases index some journals, languages, regions, and publication forms more completely than others. AI systems inherit those boundaries and may rank familiar terminology above locally specific concepts. Audit discovery by language, geography, discipline, publication form, and institutional access.
Quantitative synthesis can erase difference through a common effect measure. Before converting outcomes, ask whether they represent the same construct and decision. Preserve original scales and definitions alongside transformed values. Report exclusions from pooling without treating unpooled studies as inferior evidence.
Open release also has limits. Study-level data may contain confidential information or enable reidentification. Search histories can expose sensitive research interests or restricted holdings. Release a calibrated public package and document the withheld components, access basis, and responsible contact.
AI assistance creates additional disclosure duties. Name the operations it supported, the system version, validation procedure, and known failures. Do not list a general-purpose model as an author or allow it to conceal who made consequential decisions.
Agent delegation can obscure responsibility. A coordinating agent may summarize another agent’s output and remove its qualifications. Several agents may repeat one mistaken extraction and make it appear independently confirmed. Preserve the complete provenance chain, label shared models and inputs, and route agent outputs through the same evidence checks as direct model output.
Project context can contain unpublished arguments, access credentials, participant information, and restricted source locations. Give each agent only the layer required for its task. A context packet prepared for discovery should not expose material needed only for confidential analysis.
Finally, resist prestige by method. A complex model cannot compensate for a weak corpus. A transparent narrative synthesis can be more informative than an invalid pooled estimate. The review earns trust by matching its claim to its evidence and preserving the path between them.
18.7 Save the Artifact
Save a review design memo that links the question, intended decision, review form, eligible evidence, and synthesis method. Preserve the protocol and every dated amendment.
Maintain a source register, screening log, extraction dictionary, coded corpus, dependence map, and synthesis memo. For meta-analysis, add raw and transformed effect data, executable code, session information, model diagnostics, prediction intervals, and sensitivity results.
Create a release manifest. It should name the corpus version, protocol version, codebook version, analysis commit, public files, restricted files, license, citation, review date, correction route, update trigger, and retirement condition. Give each release a stable identifier and preserve earlier releases.
Maintain a research-automation register. For every skill, save its manifest, instructions, scripts or schemas, benchmark set, evaluation results, permissions, owner, and version. For every agent, save its work-order template, permitted skills and tools, expected output, escalation rule, and tests. Archive each project context packet and its approved change log. Do not store credentials or protected source content in these public or reusable definitions.
The accompanying Skills and Agents Lab provides the first complete implementation of this architecture. Release 0.1.0 contains a literature-discovery skill, a read-only scout agent, a three-layer project-context packet, three synthetic benchmark cases, a candidate-register validator, an explicit failure report, and a versioned download. It is a pilot procedure to inspect and test, not evidence that autonomous discovery is reliable.
The article, dataset, code, and review website should point to the same manifest. This shared reference prevents a current webpage from silently changing the evidence behind an older citation.
18.8 Advanced Practice
Prospective meta-analysis coordinates eligible studies and analyses before their results become known. It can reduce some forms of outcome selection and harmonize measures while preserving study-level independence. It requires governance that protects local study ownership and records any changes made after results arrive.
Individual-participant-data meta-analysis can examine common definitions and participant-level variation that published aggregates cannot support. Its advantages depend on data access, harmonization quality, missingness, and representation of studies that cannot share data. The unavailable studies remain part of the inference problem.
Bayesian meta-analysis can encode uncertainty about heterogeneity, combine prior information with new evidence, and support sequential updating. Priors should be justified, sensitivity-tested, and separated from evidence supplied by the included studies. Bayesian computation does not make incomparable effects comparable.
Multiverse meta-analysis can reveal how defensible choices about inclusion, effect selection, dependence, and modeling affect the conclusion. The set of specifications must follow substantive and methodological reasoning. A large grid of arbitrary models can obscure judgment as easily as one preferred model.
The mature review program connects these methods to a Second Brain without collapsing private work into public evidence. Source notes, decision records, and synthesis memos support learning. Stable public releases support scrutiny and reuse. The boundary between them is deliberate.
An advanced system may use a coordinating agent to divide a review update into discovery, gap analysis, methods audit, and collection tasks. Coordination should occur through artifacts rather than conversational summaries. Each specialist receives the same approved project card, a distinct work order, and only the context needed for its task. The coordinator can detect missing outputs and schema conflicts, but human reviewers still control the gates between candidate, verified, accepted, and published states.
Treat the context curator as a librarian, not an executive. It maintains indexes, reports contradictions, assembles task packets, and proposes version changes. It does not decide that a disputed claim has become accepted or that an old limitation no longer matters. Keeping this boundary protects the project’s intellectual history from being rewritten for the convenience of the current task.
The future review article is therefore neither a traditional essay with added links nor a database without an argument. It is a governed scholarly claim over a versioned evidence system. Its authority comes from the fit among question, corpus, method, and interpretation. Its capacity to change becomes a strength only when readers can see what changed, why it changed, and which conclusion the earlier evidence supported.
Maria J. Grant and Andrew Booth, “A Typology of Reviews: An Analysis of 14 Review Types and Associated Methodologies,” Health Information
& Libraries Journal 26, no. 2 (2009): 91–108, https://doi.org/10.1111/j.1471-1842.2009.00848.x; Julia H. Littell, “Conceptual and Practical Classification of Research Reviews and Other Evidence Synthesis Products,” Campbell Systematic Reviews 14, no. 1 (2018): 1–21, https://doi.org/10.4073/cmdp.2018.1; Hannah Snyder, “Literature Review as a Research Methodology: An Overview and Guidelines,” Journal of Business Research 104 (2019): 333–39, https://doi.org/10.1016/j.jbusres.2019.07.039.↩︎Terri D. Pigott and Joshua R. Polanin, “Methodological Guidance Paper: High-Quality Meta-Analysis in a Systematic Review,” Review of Educational Research 90, no. 1 (2020): 24–46, https://doi.org/10.3102/0034654319877153.↩︎
Larry V. Hedges, Elizabeth Tipton, and Matthew C. Johnson, “Robust Variance Estimation in Meta-Regression with Dependent Effect Size Estimates,” Research Synthesis Methods 1, no. 1 (2010): 39–65, https://doi.org/10.1002/jrsm.5.↩︎
Zuzana Irsova et al., “Meta-Analysis of Social Science Research: A Practitioner’s Guide,” Journal of Economic Surveys 38, no. 5 (2024): 1547–66, https://doi.org/10.1111/joes.12595.↩︎
David Moreau and Beau Gamble, “Conducting a Meta-Analysis in the Age of Open Science: Tools, Tips, and Practical Recommendations,” Psychological Methods 27, no. 3 (2022): 426–32, https://doi.org/10.1037/met0000351.↩︎
Joshua R. Polanin, Emily A. Hennessy, and Sho Tsuji, “Transparency and Reproducibility of Meta-Analyses in Psychology: A Meta-Review,” Perspectives on Psychological Science 15, no. 4 (2020): 1026–41, https://doi.org/10.1177/1745691620906416; Lesley Uttley, Yuliang Weng, and Louise Falzon, “What’s the Meta Now? More Updates on the Problems with Systematic Reviews,” Journal of Clinical Epidemiology 198 (2026): 112393, https://doi.org/10.1016/j.jclinepi.2026.112393.↩︎
Julian H. Elliott et al., “Living Systematic Review: 1. Introduction, the Why, What, When, and How,” Journal of Clinical Epidemiology 91 (2017): 23–30, https://doi.org/10.1016/j.jclinepi.2017.08.010.↩︎
Iain J. Marshall and Byron C. Wallace, “Toward Systematic Review Automation: A Practical Guide to Using Machine Learning Tools in Research Synthesis,” Systematic Reviews 8, no. 1 (2019): 163, https://doi.org/10.1186/s13643-019-1074-9.↩︎
Aymeric Blaizot et al., “Using Artificial Intelligence Methods for Systematic Review in Health Sciences: A Systematic Review,” Research Synthesis Methods 13, no. 3 (2022): 353–62, https://doi.org/10.1002/jrsm.1553; M. Sharmila A. Sousa et al., “The Landscape of Artificial Intelligence Tools and Platforms for Evidence Synthesis: A Scoping Review,” Systematic Reviews 15, no. 1 (2026): 82, https://doi.org/10.1186/s13643-025-02842-y.↩︎
Shunyu Yao et al., “ReAct: Synergizing Reasoning and Acting in Language Models,” in The Eleventh International Conference on Learning Representations, 2023, https://openreview.net/forum?id=WE_vluYUL-X.↩︎
Joon Sung Park et al., “Generative Agents: Interactive Simulacra of Human Behavior,” in Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology, 2023, 1–22, https://doi.org/10.1145/3586183.3606763.↩︎
Nils B. Weidmann, Mats Faulborn, and David García, “Large Language Models Are Democracy Coders with Attitudes,” PS: Political Science & Politics 59, no. 1 (2026): 21–23, https://doi.org/10.1017/S1049096525101248.↩︎
Nelson F. Liu et al., “Lost in the Middle: How Language Models Use Long Contexts,” Transactions of the Association for Computational Linguistics 12 (2024): 157–73, https://doi.org/10.1162/tacl_a_00638.↩︎