Measuring up the methodological considerations of quantitative research

Liz Halcomb highlights the areas of discussion and debate in quantitative methods of research and introduces four Nurse Researcher articles relating to the subject

Liz Halcomb highlights the areas of discussion and debate in quantitative methods of research and introduces four Nurse Researcher articles relating to the subject

Measuring research
Picture: Nick Lowndes

Unlike qualitative research that relies on participants’ voices, narrative data and people’s experiences, quantitative research uses statistical techniques to undertake empirical investigations of phenomena. Quantitative research can be broadly classified into experimental and non-experimental research.

In experimental research, an intervention (independent variable) is applied to evaluate what impact it has on the phenomena (dependant variable). In a randomised controlled trial, this intervention will be applied to a randomly selected intervention group, while the control group do not receive the intervention. However, in a quasi-randomised trial either the independent variable will not be manipulated or the allocation to the intervention and control group is not randomised.

In contrast, in non-experimental research, there is no intervention and the research may either simply describe the current situation (descriptive research) or attempt to describe relationships among variables (correlational). 

Nurse Researcher has a number of papers exploring methodological issues around quantitative research. Krishnan (2018) and Zauszniewski et al (2018) present issues related to experimental research, while Hutchinson and East (2018) and Younas and Porr (2018) illustrate models to develop data collection instruments for descriptive research. Together these papers highlight the pertinent issues around quantitative methods that researchers need to consider.

Factors affecting independent variables

Many nurse researchers experience the dilemma of needing to provide evidence that a particular intervention makes an effect on the target group. While textbooks provide examples of clear-cut trials, in the real world, such investigation is often far more complex. In theory, manipulating an independent variable through the delivery of an intervention or randomising individuals to two groups is straightforward; however, in practice it is not always possible to do this. Ethical issues, the natural occurrence of events and the practical inability to manipulate the variable can all affect the study design. Additionally, the potential of contamination between participants or the practicalities of delivering the intervention influence randomisation. 

Krishnan (2018) explores the features and utility of the non-equivalent control group post-test-only design. This design has been the subject of limited previous discussion in the literature, but it is useful in situations where randomisation is precluded for ethical and practical reasons. Despite the potential value of this design, Krishnan (2018) urges careful consideration of the risks to internal validity that should be considered when planning a study using this design.

Measuring parameters

In contrast, Zauszniewski et al (2018) consider the broader criteria around which an intervention should be evaluated. The authors argue that despite the critical importance of establishing the effectiveness of the intervention, scrutinising other parameters – including necessity, acceptability, feasibility, fidelity, safety, efficacy and cost – will impact the future implementation and sustainability of the intervention. Additionally, Zauszniewski et al (2018) highlight the importance of assessing feedback from intervention recipients to explore whether the intervention was perceived as appealing, relevant, meaningful and beneficial. Unless these parameters are evaluated positively, intervention uptake and ongoing sustainability are likely to be problematic. These authors go on to provide practical examples of how each parameter might be measured. 

Although experimental research provides important evidence for the effectiveness of interventions, descriptive research provides essential understandings about a particular situation or the relationship between variables. Having a clear understanding of the current situation can inform the development of interventions to address need, highlight the need for policy change, or identify education or health needs. Similarly, capturing a cross-sectional snapshot of a situation can inform longitudinal evaluation, whereby changes over time can be evaluated. Regardless of the purpose of descriptive research, it is essential that measurement tools are sufficiently robust to collect valid and reliable data.

Robust development of survey tools

The other two papers highlighted here focus on the development of measurement tools for descriptive research. While nurse researchers have been using survey methods for many years, the quality of survey tools has been an area that has received limited attention. Despite the importance of ensuring robust development of survey tools to ensure the reliability and validity of the data collected, many reported studies continue to use investigator-developed instruments that are not informed by principles of good survey design or instrument development. Most investigator-developed instruments are reported as being derived from a review of the literature, although the process of moving from literature review to item generation is seldom clear. 

Five-step process

Hutchinson and East (2018) provide an exemplar of undertaking a meta-summary and factor analysis to refine scale items and establish construct definition in a study defining activities of advanced practice nursing. Through the five-step process the authors describe how they defined the construct, identified domain constructs, established the effect size for domains, developed items and explored content validity to refine the items. While this paper describes a lengthy process of synthesising the literature into a tool, it represents a robust means of instrument development.

Younas and Porr (2018), meanwhile, offer an alternate method of robust scale development. Unlike Hutchinson and East (2018) the process described by Younas and Porr (2018) incorporates data from sources beyond the literature such as interviews and focus groups, the Delphi technique or theoretical and conceptual frameworks to generate the items. Younas and Porr (2018) describe scale development as a six-step process, incorporating conceptualisation of the construct, item generation, developing response set, item selection, psychometric testing and finalisation of the tool. The paper presents a discussion of some of the major considerations around establishing the various types of reliability and validity.

These papers highlight that there are many areas for discourse around quantitative methods. As nurse researchers we need to actively engage in this debate to ensure that our approach to quantitative research is rigorous and the outcomes are robust.

Nurse Researcher references

Hutchinson M, East L (2018) Application of meta-summary to derive a measurement instrument from the literature: a method exemplar. doi: 10.7748/nr.2018.e1599

Krishnan P (2018) A review of the non-equivalent control group post-test-only design. Nurse Researcher. doi: 10.7748/nr.2018.e1582

Younas A, Porr C (2018) A step-by-step approach to developing scales for survey research. Nurse Researcher. doi: 10.7748/nr.2018.e1585

Zauszniewski J, Bekhet A, Herbell K (2018) Comprehensive evaluation of interventions: eight vital parameters. Nurse Researcher. doi: 10.7748/nr.2018.e1603

This article is for subscribers only