Research Methods · Measurement and Statistics

Development and internal validation of the multivariable CIPHER (Collaborative Integrated Pregnancy High-dependency Estimate of Risk) clinical risk prediction model

Payne BA, Ryan H, Bone J, Magee LA, Aarvold AB, Mark Ansermino J, Bhutta ZA, Bowen M, Guilherme Cecatti J, Chazotte C, Crozier T, de Pont AJM, Demirkiran O, Duan T, Kallen M, Ganzevoort W, Geary M, Goffman D, Hutcheon JA, Joseph KS, Lapinsky SE, Lataifeh I, Li J, Liskonova S, Hamel EM, McAuliffe FM, O'Herlihy C, Mol BWJ, Seaward PGR, Tadros R, Togal T, Qureshi R, Vivian Ukah U, Vasquez D, Wallace E, Yong P, Zhou V, Walley KR, von Dadelszen P, CIPHER Group

Published October 30, 2018 Critical care (London, England)
DOI 10.1186/s13054-018-2215-6 PMID 30373675 PMC PMC6206915

RRM Academy Synopsis

CIPHER ICU score estimates maternal risk of death or severe illness

CIPHER sorted pregnant ICU patients by their risk of death or prolonged organ support. This retrospective cohort study included 769 pregnant or recently delivered women in intensive care units (ICUs) at 13 hospitals in 11 countries. The outcome occurred in 127 out of 769 women. After internal validation, the AUROC was 0.82.

Key Findings

  • Of 769 women in the final cohort, 127 (16.5%) had the main outcome: 59 (7.7%) died, and 68 (8.8%) needed organ support alone.
  • After internal validation, the model's AUROC was 0.82 (95% CI 0.81 to 0.84) and its calibration slope was 0.92 (95% CI 0.91 to 0.92).
  • In the highest-risk group (predicted probability of 50% or more), 39 of 59 women (66.1%) had the outcome; in the lowest group (under 5%), 5 of 236 (2.1%).
  • The AUROC was 0.851 (95% CI 0.812 to 0.894) at sites in low- or middle-income countries and 0.774 (95% CI 0.663 to 0.868) at sites in high-income countries.
  • For maternal death, the APACHE II score gave an AUROC of 0.84 (95% CI 0.72 to 0.96) in 433 women with complete data, and CIPHER gave 0.87 (95% CI 0.86 to 0.88).

Interpretation

The CIPHER score came from a retrospective cohort. The team reviewed hospital charts from 2000 to 2012 for pregnant or recently delivered women who spent more than 24 hours in intensive care. In this cohort, the predictors are statistical associations with the outcome. The authors tested the model on the same women using bootstrapping, an internal validation. They recommend external validation and, where needed, local recalibration. Of the outcomes, 88.6% occurred at sites in low- and middle-income countries, and 57.6% at one site in Pakistan. The authors say that because clinicians were not masked to the measurements, recent surgery's apparent protective role may reflect changed admission and treatment thresholds.

RRM Context

The authors note that about two thirds of maternal ICU admissions have obstetric causes, such as hemorrhage, pre-eclampsia, and sepsis. The cohort starts at ICU admission, after critical illness has developed. Restorative reproductive medicine is cause-based. It evaluates a woman's health early, to find risks before an emergency.

Abstract

Background

Intensive care unit (ICU) outcome prediction models, such as Acute Physiology And Chronic Health Evaluation (APACHE), were designed in general critical care populations and their use in obstetric populations is contentious. The aim of the CIPHER (Collaborative Integrated Pregnancy High-dependency Estimate of Risk) study was to develop and internally validate a multivariable prognostic model calibrated specifically for pregnant or recently delivered women admitted for critical care.

Methods

A retrospective observational cohort was created for this study from 13 tertiary facilities across five high-income and six low- or middle-income countries. Women admitted to an ICU for more than 24 h during pregnancy or less than 6 weeks post-partum from 2000 to 2012 were included in the cohort. A composite primary outcome was defined as maternal death or need for organ support for more than 7 days or acute life-saving intervention. Model development involved selection of candidate predictor variables based on prior evidence of effect, availability across study sites, and use of LASSO (Least Absolute Shrinkage and Selection Operator) model building after multiple imputation using chained equations to address missing data for variable selection. The final model was estimated using multivariable logistic regression. Internal validation was completed using bootstrapping to correct for optimism in model performance measures of discrimination and calibration.

Results

Overall, 127 out of 769 (16.5%) women experienced an adverse outcome. Predictors included in the final CIPHER model were maternal age, surgery in the preceding 24 h, systolic blood pressure, Glasgow Coma Scale score, serum sodium, serum potassium, activated partial thromboplastin time, arterial blood gas (ABG) pH, serum creatinine, and serum bilirubin. After internal validation, the model maintained excellent discrimination (area under the curve of the receiver operating characteristic (AUROC) 0.82, 95% confidence interval (CI) 0.81 to 0.84) and good calibration (slope of 0.92, 95% CI 0.91 to 0.92 and intercept of -0.11, 95% CI -0.13 to -0.08).

Conclusions

The CIPHER model has the potential to be a pragmatic risk prediction tool. CIPHER can identify critically ill pregnant women at highest risk for adverse outcomes, inform counseling of patients about risk, and facilitate bench-marking of outcomes between centers by adjusting for baseline risk.

Topics

By this author

Research Methods › Measurement and Statistics › Instrument Development and Validation
Beth A Payne, Helen Ryan, Jeffrey Bone, Laura A Magee, Alice B Aarvold, J Mark Ansermino, Zulfiqar A Bhutta, Mary Bowen, J Guilherme Cecatti, Cynthia Chazotte, Tim Crozier, Anne-Cornélie J M de Pont, Oktay Demirkiran, Tao Duan, Marlot Kallen, Wessel Ganzevoort, Michael Geary, Dena Goffman, Jennifer A Hutcheon, Stephen E Lapinsky, Isam Lataifeh, Jing Li, Sarka Liskonova, Emily M Hamel, Fionnuala M McAuliffe, Colm O'Herlihy, Ben W J Mol, P Gareth R Seaward, Ramzy Tadros, Turkan Togal, Rahat Qureshi, U Vivian Ukah, Daniela Vasquez, Euan Wallace, Paul Yong, Vivian Zhou, Keith R Walley, Peter von Dadelszen
Elizabeth Payne, B Payne, H Ryan, Jeff Bone, J Bone, L Magee, A Aarvold, Z Bhutta, M Bowen, C Chazotte, Timothy Crozier, T Crozier, A Pont, O Demirkiran, T Duan, M Kallen, W Ganzevoort, Mike Geary, M Geary, D Goffman, Jen Hutcheon, Jenny Hutcheon, J Hutcheon, Steve Lapinsky, S Lapinsky, I Lataifeh, J Li, S Liskonova, E Hamel, F McAuliffe, C O'Herlihy, Benjamin Mol, B Mol, R Tadros, T Togal, R Qureshi, D Vasquez, E Wallace, P Yong, V Zhou, K Walley, Pete Dadelszen, P Dadelszen
PMID 30373675 30373675 DOI 10.1186/s13054-018-2215-6 10.1186/s13054-018-2215-6 Payne et al. 2018, Payne 2018