Correlation between an Independent Electronic Health Record & External Ranking of Children’s Hospitals ()
1. Introduction
All academic medical centers must have some mechanism to care for pediatric patients and train medical students, residents, and fellows in pediatrics. There are a number of different models by which this is accomplished. Some centers have a pediatric unit located within an adult hospital. Some have what is commonly referred to as a pediatric “hospital within a hospital” in which the pediatric services are sequestered in a part of the hospital. That part of the hospital may even be called a children’s hospital in name but resides under the same provider number and corporate structure as the adult hospital. Other academic adult hospitals partner with an independent, free-standing children’s hospital. Within these various models, some of the pediatric health systems have an independent instance of an electronic health record (EHR). Other children’s health systems share a single instance of an EHR with their associated adult system. For the purposes of this study, we will refer to these as children’s hospitals with an independent or shared EHR.
Our organization consists of an overarching university, our school of medicine, an adult hospital system, and a pediatric and maternal hospital. The adult and the pediatric and maternal hospitals are each a separate corporate entity. Both are wholly owned subsidiaries of our university and are also physically connected. Currently, the adult hospital and the pediatric and maternal hospital are on separate instances of the same brand of EHR. Because there are operational inefficiencies and potential communication lapses related to certain groups of patients and certain groups of caregivers having to move back and forth between the two hospital systems, there is sometimes discussion about moving to a single instance of the EHR. Others argue strongly that there are primary differences in the needs of the EHR for children [1] - [11] and that an independent pediatric EHR combined with local pediatric clinical informatics expertise is essential to the EHR being tailored to and providing high-quality quaternary pediatric care.
There are a number of functions that are either unique to or much more important in pediatric health care than in adult care [1] [2] [3] [4] [5]. These include weight-based medication dosing and need for related safety features such as dose range checking; requirements related to immunization; graphing normal growth and calculating height and weight percentiles; patient identification issues related to prenatal and neonatal identifiers; lack of government-issues identifier numbers, often required by EHRs, during the neonatal period; issues related to family name changes and ambiguous sex; unique normative data needs for laboratory values, vital signs, and other factors changing by pediatric age group; adolescent privacy standards; complex guardianship related issues, and many other issues [1] - [6]. Though there are reports of improvements in quality and safety of care enabled by an EHR platform [1] - [11], the relationship between a pediatric-specific (independent) EHR for children’s hospitals and quality of care compared to that achieved using a shared EHR system has not been studied specifically. We hypothesize that children’s hospitals that have an independent EHR have the ability to specifically focus on pediatric healthcare delivery and may be advantaged in the quest to provide higher quality care. The purpose of this study was to evaluate the association between an independent pediatric-specific EHR and the quality ranking of children’s hospitals by an independent third-party ranking system.
2. Methods
Evaluation of Top 50Ranked Children’s Hospitals
Children’s hospitals that were ranked in the 2019-2020 US News and World Report (USNWR) rankings were considered. As USNWR was used for the evaluation, this limits the evaluation only to hospitals within the USA. Of these, the top ten children’s hospitals [12] are publically published. As a participant in the USNWR ranking process, we utilized data about points awarded to extend the ranking beyond that publically published to the top 50 children’s hospitals. For each of those top 50 children’s hospitals, it was determined whether that children’s hospital had an independent instance of an electronic health record (EHR) or whether it shared a single instance of an EHR with a partner adult hospital. An independent EHR was defined as when the pediatric healthcare system had an independent license for an EHR system and made independent decisions as to the configuration of that EHR. A shared EHR was defined as when a pediatric healthcare system and the associated adult healthcare system shared a single instance of the EHR and decisions as to the configuration of the EHR were made by a single governing process. Confirmation of the EHR status was made through a number of mechanisms: contacting individuals in the organizations in question through networks of our chief quality officer or chief medical informatics officer, checking public institutional websites, or contacting EHR vendors.
The median ranking of children’s hospitals with an independent EHR was compared to the median ranking of children’s hospitals with a shared EHR for statistical difference using a Wilcoxon rank sum test.
Comparison between Top 50Ranked Children’s Hospitals and Top 21Ranked Adult Hospitals
The 2019-2020 US News and World Report (USNWR) rankings of the top twenty adult hospitals in the United States [13] were also evaluated. Because there was a tie for 20th, there were actually 21 hospitals included in the analysis. Children’s hospitals that were partnered with these top 21 adult hospitals were evaluated and categorized as being supported by an independent or by a shared EHR. Partnering was defined as when the children’s hospital was under the same license or was academically affiliated with an adult healthcare system.
For the children’s hospitals matched with a ranked adult hospital, and median rank of the children’s hospital was calculated for those with an independent and with a shared EHR. Statistically significant differences in these values were evaluated with a Wilcoxon rank sum test. The Wilcoxon rank sum test is a nonparametric statistical hypothesis test that compares two groups. The test calculates the difference between sets of pairs and establishes whether they are statistically significantly different. In addition, the relationship between the rank of the adult hospital and the rank of the associated children’s hospital was evaluated using a paired Wilcoxon rank sum test. Finally, the relationship between the ranking of the adult hospital and whether or not the adult hospital had an independent EHR was evaluated using a Wilcoxon rank sum test.
3. Results
Evaluation of Top 50Ranked Children’s Hospitals
In the evaluation of the USNWR top 50 children’s hospitals (Table 1), there were 23 with independent EHRs and 27 with a shared EHRs. The median USNWR ranking for hospitals with an independent and shared EHR was 13 (inter quartile range (IQR) 6.5 - 30.5) and 30 (IQR 23.5 - 39.5), respectively. The difference was statistically significant (p = 0.002). Analysis of the percentage of children’s hospitals with an independent EHR based on USNWR rank grouping is summarized in Table 2. Of children’s hospitals ranked in the USNWR top eleven, 91% had an independent EHR (Table 3). Of those ranked between 12 - 20, 56% had an independent EHR. For those ranked between 21 - 50, 27% had an independent EHR.
Comparison between Top 50Ranked Children’s Hospitals and Top 21Ranked Adult Hospitals
Results of the evaluation of differences in ranking of children’s hospitals for those with ranked paired adult hospitals are summarized in Table 4. The 21 top ranked adult hospitals were associated with 17 children’s hospitals ranked in the top 50. Of those, 7 had an independent EHR and 10 had a shared EHR. The median ranking for those with an independent and shared EHR was 7 (IQR 1.5 - 10) and 28 (IQR 22.5 - 31.5), respectively. The difference was statistically significant (p = 0.002).
![]()
Table 1. Based on evaluation of the 2019-2020 top 50 USNWR ranked Children’s hospitals, differences in USNWR ranking comparing children’s hospitals with an independent EHR as compared to those with a shared EHR.
![]()
Table 2. Based on evaluation of the 2019-2020 top 50 USNWR ranked Children’s hospitals, comparison of percentage with an independent EHR per ranking group.
![]()
Table 3. 2019-2020 top 50 USNWR ranked Children’s hospitals and whether they do or do not have an independent EHR.
![]()
Table 4. Children’s hospitals that were both paired with a 2019-2020 top 21 USNWR ranked Adult Hospital and also appeared in the 2019-2020 top 50 USNWR Ranked Children’s Hospitals.
![]()
Table 5. Comparison of rankings of 2019-2020 top 21 USNWR ranked adult hospitals that were paired with a 2019-2020 top 50 USNWR Ranked Children’s Hospitals for whether there was an independent or shared EHR.
For the 17 children’s hospitals ranked in the top 50 that were associated with one of the top 21 ranked adult hospitals, the relationship between rank of the adult hospital and the rank of the children’s hospital was not statistically significant (p = 0.05). Also, there was not a statistically significant difference in the rank of the adult hospital for when the adult hospital had an independent or a shared EHR with their partner children’s hospital (p = 0.31) (Table 5).
4. Discussion
In this study, statistically significant correlations have been shown between children’s hospitals that have an independent instance of an EHR and higher rankings of those children’s hospitals in USNWR rankings as compared to those children’s hospitals that share an EHR with their partner adult hospital.
In many ways, this is not surprising. In medicine, it has long been argued that children are not “small adults” and have unique medical needs [1] [2]. This concept has been adapted into pediatric medical practice. The challenges and potential risks when children are cared for in institutions primarily designed to and primarily do care for adults are well known. Over the past 15 - 20 years, it has been recognized that many of these same concepts pertaining to the special needs for pediatric care also apply to EHRs [1] [2] [3].
EHRs were initially primarily constructed to meet the needs of adult patients [3]. There are a number of functions that are either unique to or much more important in pediatric health care than in adult care [1] [2] [3]. In pediatrics as opposed to adults, most medication dosing is based on body weight and EHRs need to have a clear and reliable process for which this can occur with associated safety features related to dose range checking, rounding to safe and convenient doses, age-based dosing decision support, and dosing for the school day [1] [4]. There are specific pediatric needs related to immunization management including the ability to record immunization data, linking EHR entries to state required immunization information systems, and immunization decision support [1] [2] [3]. There are specific pediatric requirements related to growth tracking including the ability to graphically display this information and calculate percentiles [1] [2] [3]. There are unique needs in patient identification including those related to prenatal and neonatal identifiers [1] [2] [3]. Newborns often do not have government-issued identifier numbers, often required by EHRs. There are also unique pediatric challenges related to name changes (related to family changes) and ambiguous sex [1] [2] [3]. There are unique pediatric needs related to normative data [1] [2] [3]. For laboratory values, vital signs, imaging appearance, and other factors what is normal at one age may be completely abnormal at other ages. The ability to depict normative data by age is therefore important. Normative data may also not always be related just to age. In some instances, normal values correlate more closely with height or gender [1]. There is also a need to display some such information by height or other parameters. There are also issues related to privacy. This is most prominent in adolescents where the age of consent varies by state and also by the presenting health issue [1]. For example, different adolescent privacy standards apply in some states when dealing with treatment of mental health disorders, pregnancy, or sexually transmitted diseases [1] [5] [6] [7]. There are also privacy issues related to children in foster or custodial care, consent by Proxy, and adoption [1]. There are also potential and common complex guardianship issues outside the traditional 2-parent household [1]. These issues, and many more, are unique or more important in pediatric healthcare and should be optimally managed in EHRs so that healthcare systems can provide reliable and efficient care.
The unique pediatric needs of EHRs have been recognized by multiple organizations including the American Academy of pediatrics [3], Agency for Healthcare Research and Quality (AHRQ) [8], Centers for Medicare & Medicaid Services (CMS) [9], and the Office of the National Coordinator for Health Information Technology (ONC) [9]. The AHRQ, together with CMS, authored an over 60 page report in 2015 that makes recommendations for EHR enhancements for pediatric care. The ONC has created a process for voluntary certification in pediatric health IT [9]. That process has 10 recommended certification criteria:
1) Use biometric-specific norms for growth curves and support growth charts for children;
2) Compute weight-based drug dosage;
3) Ability to document all guardians and caregivers;
4) Segmented access to information;
5) Synchronize immunization histories with registries;
6) Age- and weight-specific single dose range checking;
7) Transferrable access authority;
8) Associate mother’s demographics with newborn;
9) Track incomplete preventative care opportunities;
10) Flag special health care needs.
While many of these functionalities can be built in standardly by EHR vendors, there is growing evidence that EHRs from the same vendor can have widely varying performance depending on the local implementation and configuration [10] [11]. These principles may be especially applicable in tertiary and quaternary pediatric care settings. The dominant inpatient services in our organization are related to congenital heart disease, solid organ transplant, oncology, stem cell transplant, and maternal-fetal care. Our case mix index (measure of patient complexity) is one of the highest of any children’s hospital in the USA. The extreme differences in size and condition of patients, complex medication regimes, and advanced interventions such as extracorporeal membrane oxygenation and ventricular assistance devices make the need for local informatics teams with pediatric expertise and autonomy to configure the EHR for quaternary pediatric needs even more essential.
This study has a number of limitations. First, some might argue that USNWR Children’s Hospital ranking may not be a perfect proxy for quality. We certainly agree that the USNWR process has numerous flaws. The USNWR is currently, however, the only existing way to compare the performance of children’s hospitals throughout the USA. The children’s hospitals that are considered by many to be of the highest quality always manage to be at the top, despite whatever parameter, weighting, or other process changes are made from year to year. USNWR ranks children’s hospitals by collecting data via a survey completed by participating hospitals. Surveys look at quality parameters, clinical volumes, program components, and reputation of program for 10 different pediatric subspecialties. The top 50 programs in each of the 10 subspecialties are ranked. Based on those rankings, a point system is used to award points by the ranking of each of the 10 subspecialties and uses the sum of those points to rank the participating children’s hospitals.
Secondly, this study demonstrates an association between an independent EHR and increased USNWR ranking. This, however, is not proven to be a direct cause-and-effect relationship. There are likely many factors that influence both whether a children’s hospital has an independent EHR and whether they provide high quality care. Such potential intermediate variables could include size of the children’s hospital (larger being more favorable), governance structures, healthcare system resourcing, greater volumes in pediatric subspecialty areas (greater critical mass) which favors organizations in the USNWR ranking, greater diversity of subspecialty services, and other factors. This study only demonstrates the association between an independent EHR and increased USNWR ranking and that the association is strong. Ideally, a multivariate analysis taking all of these factors into consideration would be an ideal study and should be considered in the future. However, we do not have access to that information at this time.
Another limitation is that by using the USNWR ranking of children’s hospitals as the proxy for quality, the study is limited to children’s hospitals in the USA. This data cannot be used to extrapolate to the situation in other countries, although we do not know of reasons that use of an adult-oriented EHR in caring for children would be different in other countries.
5. Conclusion
In conclusion, this study has demonstrated that there is a statistically significant association between children’s hospitals that have an independent EHR, as compared to those that share an EHR with their partner adult hospital, and a higher ranking on the USNWR survey of children’s hospitals as a proxy for quality. The presence of an independent EHR was a much more significant predictor of children’s hospital performance than was the USNWR ranking of their partner adult hospital. Given the multiple unique pediatric needs of the EHR, it is not surprising that those children’s hospitals that have an increased ability to customize their EHR exclusively to the needs of children would have a favorable performance in USNWR rankings.