QC statistical tables with AI-written code
Chat assistants can make arithmetic errors, and research on the problem reports error rates that rise with the length and complexity of a calculation. Asking one to eyeball a table is a weak check. Asking it to write code you can run is a much stronger one. Make the code the checker.
Facts on this page are checked against the sources linked here. The prompt has not yet been run on a live assistant. How we check.
Evidence note: research documents LLM arithmetic errors, but we found no study that directly measures how much code execution improves statistical accuracy. Treat "code is the checker" as sound practice, not a measured result.
The principle
Never paste a table and ask "are these numbers right?". Ask the AI to write independent code that recomputes each cell from the dataset, then compare. You run the code; you review the differences.
Step 1: Paste this prompt
You are assisting a statistical programmer doing independent QC. I will give you (a) a table shell with its specifications and (b) the variable list of the analysis dataset. I am NOT giving you any real data. Write independent R code (and equivalent SAS if I ask) that: 1. Recomputes every cell in the table from the dataset. 2. Writes a reconciliation file with columns: cell, reported_value, recomputed_value, difference, within_tolerance. 3. States the rounding rule and tolerance you used. 4. Lists every assumption (population filter, denominator, missing data). Do not guess dataset contents. If a variable or derivation is unclear, list it under QUESTIONS instead of assuming. TABLE SPEC: [paste] VARIABLES: [paste names and labels only]
Step 2: Run it yourself on simulated or public data first
Test the QC code on a dataset where you already know the answer, such as the public CDISC pilot ADaM data, before relying on it for anything.
Review checklist
- The code does not reuse the production program's logic (independence)
- Population filters and denominators match the spec
- Rounding and tolerance are stated and sensible
- Every discrepancy is explained before sign-off
- You ran the code; the AI never "confirmed" numbers by reading them