How we check our workflows
AI gets statistics wrong in confident-sounding ways. So every workflow here is checked before it's published, and the checks are on the record.
What we do
- Public data only. Examples come from public sources such as ClinicalTrials.gov records, the sponsor's own posted protocol and SAP, and the CDISC pilot data.
- Primary sources. We read the original document, such as the SAS documentation or the trial record's raw data, and link it. A summary of a source is not a source.
- A test on a real assistant. Where we say a prompt was tested, we ran it in a named assistant, kept the output, and scored it against the real document.
- Calculations, not assertions. Where a claim can be computed, such as how SAS and R define a percentile, we computed it.
What the status labels mean
- Tested in Copilot. Facts checked. The prompt was run in Microsoft 365 Copilot and the output was compared with the real source. Facts on the page were checked against linked sources.
- Facts checked. Prompt not yet tested. The claims on the page were checked against sources, but nobody has run the prompt on a live assistant yet. We say so until it has been.
Corrections log
We make mistakes and we publish them.
- Our first checklist listed four attributes of an estimand. ICH E9(R1) defines five, including the treatment condition. We corrected it.
- We said the public trial record doesn't state the analysis methods. It does, in the Results section posted after the trial, and the sponsor's SAP is attached to the record. We rebuilt the worked example around that.
- We said no analysis in the record was designated primary. The record frames the in-trial analysis as the primary evaluation and says the on-treatment analysis was "also" done. We corrected it.
Limits
These are educational workflows, not validated procedures. One run of one model on one trial is an example of the behaviour to look for, not a guarantee. Always verify AI output yourself, follow your organization's AI and data policies, and never paste confidential trial data into any AI tool.
Spot an error? We want to know. Tell us and we'll correct the page and add it to the log.