Skip to content
zarif/automatesGet Still Building
Research guideUpdated · Zarif

Your Research Agent Needs an Evidence Ledger

Set up a research agent so every claim keeps its source, guesses are labeled as guesses, and the brief ends in a decision someone can check.

A magnifying glass examines source cards linked by cyan threads to a synthesis card, while an unconnected card sits separately.

The riskiest sentence in an AI research report is usually the one that sounds most finished.

Take "The company is ready to buy." It might mean someone at the company said so. It might mean they posted a relevant job. Or it might mean the agent strung together a few weak hints and wrote a confident conclusion.

Those are very different things to base a decision on. And in the finished report, they look the same.

A saved brief is useful. A brief where you can trace each important claim back to where it came from is much easier to check and reuse. This guide shows how to set a research agent up to produce the second kind.

Note:

This is a design guide. The example below is illustrative, not a measured client engagement or a completed market study. It does not report tested runtime, cost, or revenue results.

Give the research a decision

Start with a question whose answer would change what you do.

"Research this market" leaves the agent to pick the scope, decide what good looks like, and guess when to stop. "Find out what's known and unknown about this company's document workflow before a discovery call" gives it a real job.

A made-up example brief:

Decision: Which questions should we ask in the first discovery call?
Scope: One company, using public sources only.
Output: A short account brief with supporting sources and open questions.
Constraint: Do not treat hiring, funding, or a technology mention as buying intent.
Stop: Deliver the evidence available within the agreed research budget.

"We don't know" is an allowed answer. It beats a confident guess sliding into the sales process as if it were fact.

Separate three kinds of statement

Every line in the brief is one of three things: something you saw, something you're guessing from what you saw, or something you still need to ask.

TypeExampleTreatment
Observed factA published job description mentions document extraction.Save the source and the relevant passage.
InferenceThe company may be investing in document infrastructure.Explain the reasoning and uncertainty.
Open questionWho owns this workflow and what is failing today?Carry it into discovery; do not invent the answer.

These examples are made up. They show how to label evidence, not findings about a real company.

This isn't just a sales problem. A purchasing comparison, an investment memo or a product plan can go wrong the same way. A guess gets summarized, then summarized again, and somewhere along the way it turns into a fact.

Save the evidence before writing the brief

The fix is an evidence ledger: a running list of each claim and where it came from. It can start as a small table. You don't need a database.

Claim:
Source URL:
Publisher:
Source date:
Date checked:
Supporting passage:
Fact, inference, or open question:
Limitations:

Save the source while you have it in front of you. A list of links pasted under the finished report is hard to check. The reviewer has to guess which link backs up which sentence.

Go to the original where you can. A company's own docs for what its product does. A financial filing for its reported numbers. The original paper for a research finding. If a vendor says its own product is fast, that's still just the vendor saying so until someone else checks.

Let the agent challenge the attractive conclusion

Before the agent writes anything, have it argue with itself.

Ask it which statements rest on old material, which sources disagree, and which observations have a boring explanation. A job opening might just be replacing someone who left. A tool named in a case study might have been dropped since. A company can talk about AI all day without having budget for what you sell.

The point is a brief that helps in a real conversation. Good discovery questions often come from the gap between two explanations that both sound plausible.

Use document processing where it solves a real input problem

Sometimes the source is a messy PDF. Converting it and pulling out its structure first can help before the agent summarizes it. Datalab's documentation lists conversion and structured extraction among its document-processing features. Datalab documentation.

That can be a useful step. But it doesn't tell you whether a company wants to buy anything, and it doesn't replace checking that the extracted passage really supports the final claim. Keep "read the document" and "decide what it means for the business" as separate jobs.

Deliver a brief someone can act on

A short brief can hold:

  • The decision the research supports.
  • A few facts, each with its source.
  • Guesses, clearly labeled as guesses.
  • Where sources disagree, and what's missing.
  • The next questions to ask.
  • The evidence ledger behind it all.

The recommendation should only go as far as the evidence does. "Ask how this team handles the documents that don't fit the normal process" might be justified when "pitch them a replacement platform" isn't.

Make the handoff useful next week

Saving files in an account folder is one thing. Building memory the agent can reuse about that account is another. A first version can do the first without the second.

Write down when the brief was made, which sources were checked, what changed and what's still open. Next time, ask the agent to recheck anything time-sensitive, instead of just summarizing its own last summary.

To see the same habit applied to model answers, read three prompt contracts you can check with evaluation fixtures. It shows how to reject a convincing answer that doesn't match its evidence.

END OF ARTICLE

Want the next story in your inbox?

For people who build with AI and want judgment instead of a roundup.

One story from the AI world, told properly, and what I make of it.