The content development of the SAQ was not described in the Davenport article. The Sexton article details the evolution of the (2006) SAQ from a previous ICU Management SAQ—which was, itself, developed from principles of safety in the flight industry and an instrument called the Flight Management Attitudes Questionnaire (FMAQ). The SAQ version described in the 2006 article retained about 25% of the items in the FMAQ. Sexton and colleagues created about 100 new items through discussions with health care providers and subject matter experts. Retained items had some basis in a theoretical model for analyzing risk and safety (A. Vincent, BMJ, 1998), and/or a conceptual model for assessing quality (A. Donabedian, JAMA, 1988); and they could be grouped logically into proposed factors. These factors were tested via confirmatory factor analysis in their 2006 study, and subsequently, the survey was reduced from 40 to 30 items.
In the Davenport study being reported here, all of the VAMCs in the U.S.A. were invited to participate, and 51% agreed. Similarly, all of the academic medical centers in the U.S.A. were invited to participate and 57% agreed. Monitoring of the NSQIP-guided process of extracting morbidity and mortality data from medical records was done at each local hospital site. The inter-rater reliability of the nurse coding was maintained through “regular conference calls, sample case problems, and formal IRR testing of all nurses.”
Five of the six SAQ scales had acceptable to good internal consistency reliability (Cronbach’s alpha) of 0.71 to 0.82. One scale, “Perceptions of management,” was dropped from the analysis due to low reliability (0.54). The Burnout scale (0.82) was retained. Reliability information on the four communication/collaboration items (one per provider group) was not calculated / reported, likely because the 4 items were treated as separate items (one per provider group) and not expected to form a scale. The mean mortality O/E ratio was 1.08 (range 0.51 to 2.12) and the mean morbidity O/E ratio was 1.04 (range 0.47 to 1.59). These results demonstrated good variability for correlation analysis. These ratios were calculated at the hospital site level.
Relation to Other Variables:
Correlations between the SAQ and the Burnout scales were low and non-significant for both mortality and morbidity rates. The only significant correlation found in the analysis was between the item assessing communication and collaboration with attending MDs and morbidity (r=-0.38, p<0.01). The correlation between communication and collaboration with resident MDs and patient morbidity approached statistical significance (r=0.25, p=0.08).
The authors raise important questions about inferring better patient outcomes based on attitudinal surveys such as the SAQ. One challenge in establishing external (consequential) validity of attitudinal surveys such as the SAQ is that patient complications and deaths due to non-patient (i.e., safety) factors are relatively infrequent, making them more difficult to predict.