How Do You Document How a Research Idea Became a Scientific Claim?

Scientific papers tell a cleaner story than science usually lives. The published version moves from question to hypothesis to method to result. Real investigations loop, stall, contradict themselves, discover better questions, and sometimes arrive somewhere the researcher did not expect.

Scientific communication has to compress. The problem is that compression can remove the history needed to answer a deceptively simple question: How did this claim become claimable?

Research provenance is one way of approaching that problem. Rather than reconstructing the story only at publication, a researcher can preserve consequential parts of the investigation while they are happening.

The paper shows the destination. What happened to the route?

Consider a study of whether a learning environment improves problem-solving performance. The researcher begins with a broad question. A paper suggests that feedback timing may matter. An AI assistant independently proposes feedback timing while generating alternative mechanisms. The researcher formulates an immediate-feedback hypothesis and preregisters it. The main analysis does not support it. An exploratory analysis suggests that delayed feedback may help novice participants. That observation becomes a new hypothesis.

The eventual paper may present the final evidence appropriately, but a large amount of epistemic history can disappear unless it is preserved:

• the original question;
• the source and AI interaction that prompted attention to feedback timing;
• the a priori hypothesis;
• the analysis that contradicted it;
• the exploratory observation involving novices;
• the new hypothesis and its post hoc status;
• the follow-up analysis or later study;
• the final bounded claim and the evidence that actually supports it.

The purpose of a research trail is not to preserve every thought. It is to preserve consequential epistemic transitions: events that change, constrain, test, justify or reinterpret the investigation.

Research provenance asks a deceptively simple question: Where did this come from?

Provenance concerns where an object, output or idea came from and what activities or agents were involved in producing it. In research, that can include people, papers, datasets, software, AI systems, code, analyses and earlier versions of the research object.

W3C PROV provides a general framework based on entities, activities and agents. It is useful precisely because it separates the history of an object from claims about whether the object is scientifically correct.

The most important distinction: origin is not evidence

An idea can come from a researcher, collaborator, reviewer, article, dataset, statistical result, AI model or accidental observation. That tells us something about provenance. It does not tell us whether the idea is scientifically warranted.

This distinction matters for both human and AI contributions. “A human suggested it” does not make a proposition valid. “An AI suggested it” does not make it invalid. Scientific status has to be established independently through evidence and method.

Do not record everything. Record what changed the investigation.

Recording everything would create an unusable surveillance log. A more practical rule is to record events that materially affect the investigation.

An idea can change its scientific job

A note that is continuously overwritten tells you what you believe now but not how you arrived there. For research provenance, the transition itself can matter.

question → tentative explanation → hypothesis → prediction → tested hypothesis → contradicted or refined hypothesis → finding → bounded claim

Not every project uses those exact categories, and not every idea travels through all of them. The important point is to preserve changes in status instead of retroactively labeling the earlier state as though the final interpretation had always existed.

What should happen when the hypothesis appears after the result?

New hypotheses after data collection are not inherently illegitimate. Exploration is one way science discovers unexpected patterns. The methodological problem arises when a hypothesis informed by observed results is later presented as though it had been specified before those results were known.

Kerr described this practice as HARKing – Hypothesizing After the Results are Known. Preregistration addresses part of the problem by preserving a planned state before consequential outcomes or analyses are known. Research on preregistration also shows why specificity matters; see work on preregistration quality and specificity.

A prospective research record can complement preregistration by making later emergence visible. A post hoc hypothesis does not need to disappear. It needs to be identified as post hoc, exploratory or emergent, according to the terminology appropriate to the design.

“AI was used” tells us almost nothing

“AI was used” is usually too coarse to reconstruct what happened. AI can participate in very different activities, with different implications for reproducibility and interpretation.

• generating search terms or adjacent literatures;
• suggesting alternative hypotheses;
• summarizing supplied material;
• writing or debugging code;
• transforming data or prose;
• criticizing an argument;
• proposing an interpretation;
• structuring a manuscript.

A useful record therefore identifies the system or tool, the activity, the relevant input or context, the output that mattered, and what the researcher did with that output. The goal is not to infer hidden AI use from writing style. It is to record consequential intervention when it occurs.

Again, attribution and evidential status should remain separate. If an AI proposes hypothesis X and experiment Y later supports X, the AI interaction is provenance; Y is evidence. If a human proposes Z and Y contradicts Z, human origin does not rescue the proposition.

A citation is not the same thing as support

A bibliography tells us which works appear in a paper, but not necessarily what each source did in the reasoning process. One paper may provide background, another a method, another a measurement instrument, another support for a proposition, and another a direct challenge.

CiTO – the Citation Typing Ontology formalizes many citation intentions. SWAN represents scientific discourse including questions, hypotheses, claims and evidence. Micropublications models claims within argument structures connected to evidence, methods, interpretation and challenge. These frameworks show that a generic “cites” relation often hides scientifically useful distinctions.

For a researcher, the practical rule is simple: do not let “I cited this source” silently become “this source supports my claim.” Those are different statements.

Researchers already have pieces of the solution

Nanopublications are especially useful as prior art for separating bounded assertions from their provenance. Reproducible-computational-research guidance such as Sandve et al. and research-notebook guidance such as Schnell address complementary parts of the record-keeping problem.

No single one of these methods has to be imposed on every project. The lesson is that provenance, timing, epistemic role, version history and evidential relationships are different dimensions and should not be collapsed into one vague notion of “documentation.”

Can you document a research trail without special software?

Yes. A spreadsheet, notebook, version-controlled text file or database can implement the basic method. For example:

The important properties are that consequential entries are identifiable, their origins and relationships are visible, and later changes do not erase earlier states.

A practical research-provenance workflow

1. Capture consequential events. Record questions, assumptions, hypotheses, sources, analyses, findings, interpretations, decisions and claims when they materially affect the investigation.
2. Type them provisionally. Record what an object is now without promoting a tentative idea into a claim.
3. Record provenance. Identify the person, source, dataset, AI system, tool or analysis involved.
4. Link relationships. Prompted by, tests, supports, contradicts, refines, supersedes, derives from and uses are examples.
5. Preserve temporal status. Distinguish planned, preregistered, exploratory, emergent and post hoc states where relevant.
6. Preserve transitions. Add a new state or event rather than rewriting the past.
7. Connect research objects. Link code, data, figures, search outputs, preregistrations or archived versions when they matter.
8. Traverse backward at publication. For a substantive claim, ask whether you can reconstruct the findings, analyses, hypotheses, sources and decisions that made it possible.

A perfect audit trail can still document bad science

Timestamps, append-only logs, version histories and hashes can strengthen confidence that a record existed at a particular time or has not been silently altered. They cannot establish that the study design was appropriate, the analysis was correct, or the conclusion is true.

Record integrity ≠ scientific validity. A tamper-evident record can document poor science perfectly. Provenance improves auditability; it does not replace scientific judgment.

Start a research trail in your own work

You do not need to record every thought. Start with the events that materially change the investigation:

• Record the question. What are you trying to understand, and what prompted the question?
• Separate what you know from what you think. Record existing evidence, assumptions, uncertainties and tentative explanations separately.
• Record where consequential ideas came from. Was the idea prompted by you, a collaborator, a source, data, an analysis, an AI system or something else?
• Name what the idea is now. Is it a question, assumption, conjecture, hypothesis, prediction, interpretation, finding or claim?
• Preserve timing. Was it planned in advance, preregistered, exploratory, or generated after seeing a result?
• Connect evidence to ideas. Record what tests, supports, contradicts, refines or supersedes what.
• Record consequential decisions. When a method, analysis or interpretation changes, preserve the reason rather than only the final choice.
• Keep AI contributions specific. Record the activity the system performed and what happened to its output rather than simply writing “AI was used.”
• Do not overwrite the past. When an idea changes status, preserve the earlier state and record the transition.
• Trace important claims backward. Before publication, ask whether you can identify the findings, analyses, hypotheses, sources and decisions that produced each substantive claim.

A notebook, spreadsheet or version-controlled file can do this. The purpose is not administrative completeness. It is to preserve the parts of the investigation that another person — including your future self — would need in order to understand how the research changed.

We built Research Trail because this becomes difficult surprisingly quickly

Trying to preserve these relationships manually reveals how quickly the record becomes difficult to navigate. Questions become hypotheses; hypotheses are contradicted or refined; sources play different roles; analyses generate findings; AI and other tools participate in specific activities; the final paper compresses all of this into a much smaller narrative.

That led us to Research Trail, an experimental interface for prospectively recording and traversing those relationships. The methodological argument does not depend on the application. The application is one attempt to make the method usable during live research.

Learn how Research Trail works →

The tool article explains the entry types, provenance model, use cases, methodological boundaries and current alpha implementation. From there, you can open the live application and the source project.

Frequently asked questions

What is research provenance?

It is information about where research objects, ideas and outputs came from and what entities, activities or agents participated in producing them.

What is a research audit trail?

It is a reconstructable record of consequential research events and decisions. A useful audit trail can include both provenance and the changing scientific role of questions, hypotheses, analyses, findings and claims.

Does preregistration replace a research trail?

No. Preregistration preserves a planned state. A prospective trail can preserve what happens before and after that state, including justified deviations and exploratory discoveries.

Can timestamps prove a claim is true?

No. They can strengthen record integrity and chronology, not scientific validity.

References and methodological foundations

1. W3C (2013). PROV-Overview. Source
2. Ciccarese et al. The SWAN Scientific Discourse Ontology. Source
3. Clark, Ciccarese & Goble. Micropublications: a semantic model for claims, evidence, arguments and annotations. Source
4. CiTO: Citation Typing Ontology. Source
5. Nanopublication community: model and guidelines. Source
6. Kerr (1998). HARKing: Hypothesizing After the Results are Known. Source
7. Bakker et al. on preregistration quality and specificity. Source
8. Sandve et al. (2013). Ten Simple Rules for Reproducible Computational Research. Source
9. Schnell (2015). Ten Simple Rules for a Computational Biologist’s Laboratory Notebook. Source