Wilkinson, J., Roberts, S. A., Showell, M., Brison, D. R., & Vail, A. (2016). No common denominator: a review of outcome measures in IVF RCTs. Human Reproduction. https://doi.org/10.1093/humrep/dew227
Wilkinson J, Roberts SA, Showell M, Brison DR, Vail A. No common denominator: a review of outcome measures in IVF RCTs. Human Reproduction. 2016. doi:10.1093/humrep/dew227
Wilkinson, J., et al. "No common denominator: a review of outcome measures in IVF RCTs." Human Reproduction, 2016.
For EndNote, Zotero or Mendeley:
Open access
No open license is recorded for this paper. Reuse terms are set by the publisher.
IVF trials used 815 different outcome calculations in 2013 and 2014
A review of 142 randomized IVF trials from 2013 and 2014 found that the trials used 815 different combinations of numerator and denominator to report results. Only 22 of the 52 trials reporting live birth based the rate on everyone randomized. The authors counted outcomes in published reports. They ran no statistical tests.
Key Findings
After combining similar items, reviewers found 361 numerators and 87 denominators, making 815 distinct combinations. 203 appeared in more than one study.
Of 52 studies reporting live birth, 22 used the point of randomization as the denominator. 23 did not, and in 6 the denominator could not be discerned.
In 17 studies the live birth denominator included only participants reaching transfer (15 studies) or oocyte retrieval (2 studies). A median 8% of participants were excluded (IQR 4-14%, range 2-38%).
Clinical pregnancy appeared in 95 studies (67%) with 19 different denominators. Live birth appeared in 52 studies (37%) with 14 combinations.
Live birth had no stated definition in 27 studies (51%); 19 (36%) defined it as a live birth event or delivery.
Interpretation
The authors tallied outcomes in published reports and ran no statistical tests. The sample holds English-language trials from two years. The authors call their categories arbitrary, so exact frequencies depend on their scheme. The live birth denominator analysis was post hoc, and a few articles may come from the same trial. The authors argue that analyzing only participants who reached transfer or retrieval breaks the balance randomization creates. Differences between groups may then reflect patient characteristics as well as treatment. The review did not measure how much bias this adds.
RRM Context
Success rates from different studies compare only when the numerator and denominator match. A rate per transfer leaves out every couple whose cycle ended earlier. The authors say a rate per cycle started may matter more to patients weighing IVF. Comparing restorative reproductive medicine with IVF fairly takes one denominator on both sides: every couple who started.
Our editorial summary of this paper, not the article's abstract.
Abstract
Study Question
Which outcome measures are reported in RCTs for IVF?
Summary Answer
Many combinations of numerator and denominator are in use, and are often employed in a manner that compromises the validity of the study.
What Is Known Already
The choice of numerator and denominator governs the meaning, relevance and statistical integrity of a studys results. RCTs only provide reliable evidence when outcomes are assessed in the cohort of randomised participants, rather than in the subgroup of patients who completed treatment.
Study Design, Size, Duration
Review of outcome measures reported in 142 IVF RCTs published in 2013 or 2014.
Participants/Materials, Setting, Methods
Trials were identified by searching the Cochrane Gynaecology and Fertility Specialised Register. English-language publications of RCTs reporting clinical or preclinical outcomes in peer-reviewed journals in the period 1 January 2013 to 31 December 2014 were eligible. Reported numerators and denominators were extracted. Where they were reported, we checked to see if live birth rates were calculated correctly using the entire randomised cohort or a later denominator.
Main Results and the Role of Chance
Over 800 combinations of numerator and denominator were identified (613 in no more than one study). No single outcome measure appeared in the majority of trials. Only 22 (43%) studies reporting live birth presented a calculation including all randomised participants or only excluding protocol violators. A variety of definitions were used for key clinical numerators: for example, a consensus regarding what should constitute an ongoing pregnancy does not appear to exist at present.
Limitations, Reasons for Caution
Several of the included articles may have been secondary publications. Our categorisation scheme was essentially arbitrary, so the frequencies we present should be interpreted with this in mind. The analysis of live birth denominators was post hoc.
Wider Implications of the Findings
There is massive diversity in numerator and denominator selection in IVF trials due to its multistage nature, and this causes methodological frailty in the evidence base. The twin spectres of outcome reporting bias and analysis of non-randomised comparisons do not appear to be widely recognised. Initiatives to standardise outcome reporting, such as requiring all effectiveness studies to report live birth or cumulative live birth, are welcome. However, there is a need to recognise that early outcomes of treatment, such as stimulation response or embryo quality, may be appropriate choices of primary outcome for early phase studies.
