An AI-generated explanation can read well even when some of its claims have not been checked. For a QA investigation, keep track of which findings are supported by code, specifications, or observed results, and which still need checking.
The key idea
Ask for evidence alongside each finding and keep open questions in the report. Someone else should be able to open the same files or documents and check the explanation.
Specify what the AI should investigate
Name the repository, branch, files, and feature. Check that the code matches the environment you are investigating. Otherwise, even an accurate explanation of that code may not explain what users are seeing.
If you only want an investigation, say so in the request. If a change is needed later, describe what may be edited and how the change will be checked in a separate request.
Separate facts, hypotheses, and open questions
Finding an operation in the code does not tell you whether a particular user can use it. Permissions, the conditions that enable it, and the target environment may still need checking. Leave those items marked as unknown until you have evidence.
| Type | What to retain |
|---|---|
| Fact | Directly checked implementation, specification, or result, with a reference |
| Hypothesis | An explanation inferred from facts and the parts not yet supported |
| Open question | What needs another observation or confirmation from an owner |
A prompt for requesting findings and evidence
Open the referenced files and documents yourself. Check whether each one supports the claim being made. A report can include citations and still draw the wrong conclusion from them.
Investigate this feature using the specified documents and code.
Do not modify files.
Organize the findings as follows:
- Confirmed facts
- Supporting files, functions, and document sections
- Hypotheses and the reasons behind them
- What has not been verified
- What to run or check nextCheck the findings before updating your records
Before updating a list or CSV, separate proposed additions, changes, deletions, and open questions. Compare them with the existing record, apply the updates you have verified, and inspect the diff.
Leave unknown values as unknown. Note what still needs checking and who can help, so the next person has a clear place to start.
QA Notebook / AI in practice
Read nextWhat to put in CLAUDE.md for QA investigationsBack to all articles