Prepare for statistical reviewer questions with AI
Use AI as a skeptical reviewer before the real one arrives. It finds gaps fast; you decide which ones are real.
Facts on this page are checked against the sources linked here. The prompt has not yet been run on a live assistant. How we check.
Step 1: Generate the questions
Act as a skeptical statistical reviewer at a regulatory agency. Review the analysis plan summary below. Produce the 15 most likely questions, grouped by: estimand, missing data, multiplicity, sensitivity analyses, subgroups, model assumptions, data integrity. Rank within each group by seriousness. For each question, say in one line why a reviewer would ask it. SUMMARY: [paste - public or simulated only]
Step 2: Draft response outlines, not answers
For the 5 most serious questions, draft a response OUTLINE: - the position to take - the evidence or analysis I would need to point to - what I must verify before answering Do not cite any guidance document, paper or regulation unless you can name it exactly. Mark every citation [VERIFY CITATION]. Never invent numbers.
Review checklist
- Each question is something a real reviewer could ask (cut the generic ones)
- Every citation was verified against the original source
- No invented results or numbers in any draft response
- Responses reflect decisions your team actually made
- Gaps that are real become action items, not just talking points
Worked example (public data)
For the public record NCT02906930 (oral semaglutide, three doses vs placebo, primary outcome HbA1c change at week 26), the questions an AI reviewer should raise include: how treatment discontinuation and rescue medication are handled in the estimand, how missing week-26 data is treated, how multiplicity across three doses is controlled, and whether the "ratio to baseline" endpoints were analyzed on a log scale. A good run lists these as questions to answer, not answers.
Biggest risk: AI invents plausible-sounding citations. A 2023 study of ChatGPT-written medical papers found 47% of references were fabricated and only 7% were both real and accurate. That used an older model and newer models may do better, but the habit stands: treat every reference as unverified until you have the source in hand.