"Measuring the Impact of Nurse Staffing on Patient Outcomes: The Effect of Data Aggregation and Estimation Methods"
Please note: All research in progress seminars are off-the-record. Any information about methodology and/or results is embargoed until publication.
Research Objectives: A growing body of evidence shows that nurse staffing levels and composition affect patient outcomes. This evidence has come from difference data sources, with different levels of data aggregation, and used different estimation methods. The problem of unobserved heterogeneity (unobserved characteristics that affect outcomes) is large for this area of research and estimates that don’t address this are almost certainly biased. We used a large, longitudinal, patient-level dataset with monthly, unit-level nurse staffing data to examine how different levels of data aggregation and different statistical methods affected the estimates of the effect of nurse staffing on patient outcomes.
Study Design: Monthly staffing for each unit, for each type of nurse (registered nurse, Licensed Practical Nurses, nursing aides, contract nurses), were obtained from VA accounting data. Payroll data provided education levels and how ng each nurse had worked on the unit (unit tenure). Patient characteristics and length of stay (LOS) were obtained from VA hospital discharge records. Log(LOS) was used as the dependent variable as it captures the effect of many nursing-sensitive patient outcomes. The model controlled for patient age, expected LOS, and patient co-morbidities; the variables of interest were nurse staffing, nurse skill-mix, and unit tenure. The models were estimated using both ordinary least squares (OLS) and fixed-effects (FE) regressions; the latter was used to address unobserved heterogeneity. All regressions were patient-level, with different levels of aggregation for the nurse staffing variables (unit-month, unit-year, hospital-month, and hospital-year) and the unit level models were estimated for all units together, and separately for acute care units and intensive care units.
Population Studied: All VA acute medical care units (including ICUs) for 2003-2006. 1,923,048 patients from 427 units across 138 VA Medical Centers.
Principal Findings: The results were quite sensitive to both estimation method and unit of aggregation. The change in the point estimates of the effects of nurse staffing on LOS of switching from monthly to annual staffing data ranged from 14-1177% for the FE models and 13-276% (plus two reversals, -0.20 to 0.27 and -0.09 to 0.40) for the OLS models. These ranges were even larger across all levels of aggregation. For the same level of aggregation, the difference between the OLS and FE estimates ranged from 0-304% and there were two cases of sign reversal (-0.21 to 0.27 and -0.19 to 0.30).
Conclusions: The magnitude and even the direction of the effects of different elements of nurse staffing on patient outcomes are quite sensitive to the level of aggregation and estimation method.
Implications for Policy or Practice: Interpretation of the results of studies of nurse staffing on patient outcomes needs to account for the level of data aggregation and the statistical methods used. Higher levels of aggregation, both across time and across units, probably masks effects. Thus, studies that measure nurse staffing at the unit-level data with shorter time intervals yield more reliable estimates. Studies that fail to account for unobserved heterogeneity are probably biased. But, FE models also have limits, as they only estimate marginal effects and can’t directly compare the effects of high vs. low staffing levels.