Predicting the 6-month risk of severe hypoglycemia among adults with diabetes: Development and external validation of a prediction model

https://doi.org/10.1016/j.jdiacomp.2017.04.004Get rights and content

Abstract

Aims

To develop and externally validate a prediction model for the 6-month risk of a severe hypoglycemic event among individuals with pharmacologically treated diabetes.

Methods

The development cohort consisted of 31,674 Kaiser Permanente Colorado members with pharmacologically treated diabetes (2007–2015). The validation cohorts consisted of 38,764 Kaiser Permanente Northwest members and 12,035 HealthPartners members. Variables were chosen that would be available in electronic health records. We developed 16-variable and 6-variable models, using a Cox counting model process that allows for the inclusion of multiple 6-month observation periods per person.

Results

Across the three cohorts, there were 850,992 6-month observation periods, and 10,448 periods with at least one severe hypoglycemic event. The six-variable model contained age, diabetes type, HgbA1c, eGFR, history of a hypoglycemic event in the prior year, and insulin use. Both prediction models performed well, with good calibration and c-statistics of 0.84 and 0.81 for the 16-variable and 6-variable models, respectively. In the external validation cohorts, the c-statistics were 0.80–0.84.

Conclusions

We developed and validated two prediction models for predicting the 6-month risk of hypoglycemia. The 16-variable model had slightly better performance than the 6-variable model, but in some practice settings, use of the simpler model may be preferred.

Introduction

Hypoglycemia is a potentially life-threatening complication of diabetes treatment, particularly in individuals treated with insulin or sulfonylurea drugs that stimulate insulin secretion.1., 2., 3., 4. A clinically useful prediction tool for hypoglycemia would potentially enable providers and healthcare systems to identify patients at high risk and initiate anticipatory interventions to reduce the risk of severe hypoglycemia through goal modification, medication changes, or focused patient education.

A recent review by the VA QUERI program examined predictors of severe hypoglycemia in adults with type 2 diabetes.1 Important risk factors for severe hypoglycemia included intensive glycemic control, history of hypoglycemia, renal insufficiency, history of microvascular complications, longer diabetes duration, lower education level, African American race, and history of dementia. Gender, age, and lower body mass index (BMI) were not consistently associated with risk of hypoglycemia, although higher age and lower BMI were associated with increased risk of hypoglycemia in the two largest individual studies. However, this review did not describe a clinically useful and externally validated prediction rule. Using data from DCCT/EDIC on 1441 individuals with type 1 diabetes, Lagani et al. developed a prediction rule for hypoglycemia.2 They used a separate cohort of 393 individuals with type 2 diabetes as a validation data set. Their model included five variables, several of which are not easily collected from electronic medical records (marital status, strict vs. standard insulin regimen, total insulin daily dose, family history of type 2 diabetes, and past history of severe hypoglycemia).

For individuals using frequent home blood glucose monitoring or continuous glucose monitors, either with or without an insulin pump, computer algorithms have been developed to predict the very short term risk of hypoglycemia (over the next minutes to hours).5., 6. However, models based on clinical risk factors commonly available in electronic health records (EHRs) to predict the longer term risk of severe hypoglycemia (over days to months) are lacking.

The aims of this study were: (1) to develop a multivariable model to predict the 6-month risk of severe hypoglycemia requiring medical intervention among individuals receiving pharmacologic treatment for diabetes, within one integrated health care delivery system (Kaiser Permanente Colorado) using information available in the electronic health record and other available clinical data sources; and (2) to externally validate the prediction model at two other sites (Kaiser Permanente Northwest and HealthPartners).

Section snippets

Study population

This study included three members of the SUPREME-DM (SUrveillance, PREvention, and ManagEment of Diabetes Mellitus) consortium, a group of 11 member organizations of the Health Care Services Research Network (HCSRN).7., 8. The development cohort was based in Kaiser Permanente Colorado (KPCO) which serves the Denver-Boulder metropolitan areas. We used two validation cohorts, based in Kaiser Permanente Northwest (KPNW; serving the Portland, OR and Vancouver, WA metropolitan areas) and

Model development

There were 31,674 individuals in the KPCO cohort (Appendix Fig. 1), and 325,529 observation periods, with a mean of 10.3 observation periods per individual. There were 4366 observation periods with severe hypoglycemia events, and total of 4727 severe hypoglycemia events.

The baseline variables at the time of the first observation period are shown in Table 1. Just over 95% of the cohort had type 2 diabetes. The unadjusted hazard ratios and hazard ratios from the full 16-variable model are shown

Discussion

We developed 16-variable and 6-variable models to predict the 6-month risk of severe hypoglycemia among individuals receiving pharmacologic treatment for diabetes. Both models showed good calibration and discrimination, with c-statistics of 0.84 and 0.81. The 16-variable model performed somewhat better, but at the cost of greater complexity.

The individual predictors showed similar relationships to prior studies.1., 25., 26., 27. Age and hemoglobin A1c both had U-shaped relationships, with the

Conclusions

In conclusion, we developed and validated 16-variable and 6-variable models to predict the 6-month risk of severe hypoglycemia among a population of individuals with pharmacologically treatment diabetes. These models should be evaluated in other patient populations, but appear to be promising tools to help quantify hypoglycemia risk, and indicate the need for preventive measures.

Acknowledgements

Funding: This work was supported by the Agency for Healthcare Research and Quality [grant numbers R01HS022963 and R01HS019859]. E.B.S. is supported by the National Institute for Diabetes and Digestive and Kidney Diseases [grant number 1K23DK099237-01]. P.J.O. receives support from the National Institute for Diabetes and Digestive and Kidney Diseases [grant number P30DK092924]. The funders had no role in the study's design, conduct, and reporting. The content is solely the responsibility of the

References (29)

  • V. Lagani et al.

    Development and validation of risk assessment models for diabetes-related complications based on the DCCT/EDIC data

    J Diabetes Complications

    (2015)
  • B. Cariou et al.

    Frequency and predictors of confirmed hypoglycaemia in type 1 and insulin-treated type 2 diabetes mellitus patients in a real-life setting: Results from the DIALOG study

    Diabetes Metab

    (2015)
  • H.E. Bloomfield et al.

    Predictors and consequences of severe hypoglycemia in adults with diabetes — a systematic review of the evidence

  • R.D. Pathak et al.

    Severe hypoglycemia requiring medical intervention in a large cohort of adults with diabetes receiving care in U.S. integrated health care delivery systems: 2005-2011

    Diabetes Care

    (2016)
  • E.R. Seaquist et al.

    Hypoglycemia and diabetes: A report of a workgroup of the American Diabetes Association and the Endocrine Society

    Diabetes Care

    (2013)
  • E. Daskalaki et al.

    An early warning system for hypoglycemic/hyperglycemic events based on fusion of adaptive prediction models

    J Diabetes Sci Technol

    (2013)
  • B. Sudharsan et al.

    Hypoglycemia prediction using machine learning models for patients with type 2 diabetes

    J Diabetes Sci Technol

    (2015)
  • G.A. Nichols et al.

    Trends in diabetes incidence among 7 million insured adults, 2006-2011: The SUPREME-DM Project

    Am J Epidemiol

    (2015)
  • T.R. Ross et al.

    The HMO research network virtual data warehouse: A public data model to support collaboration

    EGEMS

    (2014)
  • G.A. Nichols et al.

    Construction of a multisite DataLink using electronic health records for the identification, surveillance, prevention, and management of diabetes mellitus: The SUPREME-DM Project

    Prev Chronic Dis

    (2012)
  • A.A. Ginde et al.

    Validation of ICD-9-CM coding algorithm for improved identification of hypoglycemia visits

    BMC Endocr Disord

    (2008)
  • M. Klompas et al.

    Automated detection and classification of type 1 versus type 2 diabetes using electronic health record data

    Diabetes Care

    (2013)
  • A.S. Levey et al.

    A new equation to estimate glomerular filtration rate

    Ann Intern Med

    (2009)
  • D.B. Rubin

    Multiple imputation for nonresponse in surveys

    (1987)
  • Cited by (0)

    The other authors have no conflicts of interest to disclose.

    View full text