What to write
Qualitative and quantitative methods used to draw inferences from the data
Methods for understanding variation within the data, including the effects of time as a variable
Explanation
Various types of problems addressed by healthcare improvement efforts may make certain types of solutions more or less effective. Not every problem can be solved with one method—-yet a problem often suggests its own best solution strategy. Similarly, the analytical strategy described in a report should align with the rationale, project aims and data constraints. Many approaches are available to help analyse healthcare improvement, including qualitative approaches (eg, fishbone diagrams in root cause analysis, structured interviews with patients/families, Gemba walks) or quantitative approaches (eg, time series analysis, traditional parametrical and non-parametrical testing between groups, logistic regression). Often the most effective analytical approach occurs when quantitative and qualitative data are used together. Examples of this might include value stream mapping where a process is graphically outlined with quantitative cycle times denoted; or a spaghetti map linking geography to quantitative physical movements; or annotations on a statistical process control (SPC) chart to allow for temporal insights between time series data and changes in system contexts.
In the first example by Brady et al,1 family activated medical emergency teams (MET) are evaluated. The combination of three methods—statistical process control, a Pareto chart and χ2 testing—makes for an effective and efficient analysis. The choice of analytical methods is described clearly and concisely. The reader knows what to expect in the results sections and why these methods were chosen. The selection of control charts gives statistically sound control limits that capture variation over time. The control limits give expected limits for natural variation, whereas statistically based rules make clear any special cause variation. This analytical methodology is strongly suited for both the prospective monitoring of healthcare improvement work as well as the subsequent reporting as a scientific paper. Depending on the type of intervention under scrutiny, complementary types of analyses may be used, including qualitative methods.
The MET analysis also uses a Pareto chart to analyse differences in characteristics between clinician-initiated versus family initiated MET activations. Finally, specific comparisons between subgroups, where time is not an essential variable, are augmented with traditional biostatistical approaches, such as χ2 testing. This example, with its one-paragraph description of analytical methods (control charts, Pareto charts and basic biostatistics) is easily understandable and clearly written so that it is accessible to front-line healthcare professionals who might wish to use similar techniques in their work.
Every analytical method also has constraints, and the reason for choosing each method should be explained by authors. The second example, by Timmerman et al,2 presents a more complex analysis of the data processes involved in a multicentre improvement collaborative. The authors provide a clear rationale for selecting each of their chosen approaches. Principles of healthcare improvement analytics are turned inwards to understand more deeply the strengths and weaknesses of the way in which primary data were obtained, rather than interpretation of the clinical data itself. In this example,2 rational subgrouping of participating sites is undertaken to understand how individual sites contribute to variation in the process and outcome measures of the collaborative. Control charts have inherent constraints, such as the requisite number of baseline data points needed to establish preliminary control limits. Recognising this, Timmerman, et al used linear regression to test for the statistical significance in the slopes of aggregate data, and used run charts for graphical representation of the data to enhance understanding.
Donabedian said, “Measurement in the classical sense—implying precision in quantification—cannot reasonably be expected for such a complex and abstract object as quality.”3 In contrast to the what, when and how much of quantitative, empirical approaches to data, qualitative analytical methods strive to illuminate the how and why of behaviour and decision making—be it of individuals or complex systems. In the third example, by Dainty et al, grounded theory is applied to improvement work wherein the data from structured interviews are used to gain insight into and generate hypotheses about the causative or moderating forces in multicentre quality improvement collaboratives, including how they contribute to actual improvement. Themes were elicited using multiple qualitative methods—including a structured interview process, audiotaping with independent transcription, comparison of analyses by multiple investigators, and recurrence frequencies of constructs.3
In all three example papers, the analytical methods selected are clearly described and appropriately cited, affording readers the ability to understand them in greater detail if desired. In the first two, SPC methods are employed in divergent ways that are instructive regarding the versatility of this analytical method. All three examples provide a level of detail which further supports replication.
Examples
Example 1
We used statistical process control with our primary process measure of family activated METs (Medical Emergency Teams) displayed on a u-chart. We used established rules for differentiating special versus common cause variation for this chart. We next calculated the proportion of family-activated versus clinician-activated METs which was associated with transfer to the ICU within 4 h of activation. We compared these proportions using χ2 tests.1
Example 2
The CDMC (Saskatchewan Chronic Disease Management Collaborative) did not establish a stable baseline upon which to test improvement; therefore, we used line graphs to examine variation occurring at the aggregate level (data for all practices combined) and linear regression analysis to test for statistically significant slope (alpha=0.05). We used small multiples, rational ordering and rational subgrouping to examine differences in the level and rate of improvement between practices.
We examined line graphs for each measure at the practice level using a graphical analysis technique called small multiples. Small multiples repeat the same graphical design structure for each ‘slice’ of the data; in this case, we examined the same measure, plotted on the same scale, for all 33 practices simultaneously in one graphic. The constant design allowed us to focus on patterns in the data, rather than the details of the graphs. Analysis of this chart was subjective; the authors examined it visually and noted, as a group, any qualitative differences and unusual patterns.
To examine these patterns quantitatively, we used a rational subgrouping chart to plot the average month to month improvement for each practice on an Xbar-S chart.2
Example 3
Key informant interviews were conducted with staff from 12 community hospital ICUs that participated in a cluster randomized control trial (RCT) of a QI intervention using a collaborative approach. Data analysis followed the standard procedure for grounded theory. Analyses were conducted using a constant comparative approach. A coding framework was developed by the lead investigator and compared with a secondary analysis by a coinvestigator to ensure logic and breadth. As there was close agreement for the basic themes and coding decisions, all interviews were then coded to determine recurrent themes and the relationships between themes. In addition, ‘deviant’ or ‘negative’ cases (events or themes that ran counter to emerging propositions) were noted. To ensure that the analyses were systematic and valid, several common qualitative techniques were employed including consistent use of the interview guide, audiotaping and independent transcription of the interview data, double coding and analysis of the data and triangulation of investigator memos to track the course of analytic decisions.4
Training
The UK EQUATOR Centre runs training on how to write using reporting guidelines.
Discuss this item
Visit this items’ discussion page to ask questions and give feedback.
