RCT, quasi-experimental or theory-based? Choosing an evaluation design
There is no gold standard — only the design that best answers your question with the time, budget and data available.
Commissioners often ask us for 'the most rigorous design'. The more useful question is: which design gives a credible answer to the question you actually need answered?
When an experimental design fits
Randomised designs are powerful when the intervention is well defined, can be allocated at random without ethical problems, and the question is about average effect size. They are less suited to complex, adaptive programmes or system-level change.
When quasi-experimental designs fit
Where randomisation is not possible but a credible comparison group exists — or can be constructed from good data — methods such as difference-in-differences or matching can estimate effects with reasonable confidence.
When theory-based designs fit
For programmes working on policy, institutions or behaviour in complex settings, contribution analysis, process tracing and outcome harvesting can establish whether and how a programme contributed to change, even without a counterfactual.
There is no gold standard — only the design that best fits the question.