Anastasia Mallidou describes the specific stages, approaches and processes involved in summarising research findings
Knowledge translation is the means by which evidence-based practice is used in health care. Knowledge synthesis, a foundational element of knowledge translation, is a systematic, transparent, reproducible, efficient and scientific approach to identifying and summarising research findings for generalisable and consistent messages. Increasing numbers of knowledge synthesis methods are being applied to various types of research and, although these methods take similar approaches, they vary in rigour, process and resources. This article maps knowledge synthesis methods, by describing the specific stages, approaches and processes, and describes and compares different types of knowledge synthesis to help inform healthcare practitioners and policy makers about various designs. It also recommends a map of knowledge-synthesis designs for international agreement.
Nursing Management. 21, 5, 30-39. doi: 10.7748/nm.21.5.30.e1242
Correspondence Peer reviewThis article has been subject to double blind peer review
Conflict of interestNone declared
Received: 27 May 2014
Accepted: 02 June 2014
Published: 28 August 2014
Keywords :
KNOWLEDGE SYNTHESIS is the cornerstone of knowledge translation for healthcare professionals and policy makers for the following reasons. First, science is a cumulative process and the development of new knowledge has proliferated. For example, the University of California estimates that around five billion gigabytes of information were created in 2002 which, if printed, would fill 37,000 libraries the size of the Library of Congress (Kee 2014). Second independent studies or expert opinions can be misleading (Antman et al 1992) in terms of validity, reliability and interpretation of results (Ioannidis 2005); few studies are persuasive enough to change practice and/or policy.
Finally, knowledge synthesis is an efficient and scientific method of summarising the evidence from several studies on a specific question, improving understanding of inconsistencies in varied evidence, identifying gaps in research and suggesting future studies, and providing the best available and generalisable evidence for clinical practice and decisions.
Knowledge synthesis has a long history. In 1747, a Scottish naval surgeon, James Lind, was confronted by a plethora of reports about the prevention and treatment of scurvy, so he gathered and refined all available research and conducted the first randomised trial control study and knowledge synthesis (Chalmers et al 2002). More recently, Chalmers et al (2002) described research synthesis as a new discipline that requires its own methodological development, relevance and rigour. Consequently, a growing range of designs and methods for various types of evidence and knowledge synthesis has emerged. Various scholars take similar approaches, but there are noticeable variations in rigour and processes.
Registered nurses must be competent in understanding and appreciating research processes, and in applying research findings, which are essential elements of evidence-based practice. Yet there are extensive references in the literature to barriers to, and facilitators of, knowledge translation actions in routine practice (Kajermo et al 2008, Brown et al 2009). The proliferation of healthcare evidence makes knowledge of, and skills in, knowledge synthesis strategies fundamental requirements for nurses applying research findings to clinical practice.
The purpose of knowledge synthesis is twofold: first for ‘knowledge support’, to summarise the evidence on a specific clinical question or issue; and second for ‘decision support’, which includes engagement with decision makers for developing the research question and synthesis protocol, interpreting and contextualising the knowledge synthesis results, and developing (context relevant) recommendations (Canadian Institutes of Health Research (CIHR) 2014a).
An increasing number of methods of knowledge synthesis have emerged in the literature recently, including, for example, realist synthesis (Pawson et al 2005). However, the variety of methods means there is a lack of clarity about the most appropriate method to use for a specific research question (Kastner et al 2012). Some of these methods are similar, for example meta-synthesis, meta-summary, meta-ethnography, meta-narrative, meta-study, meta-interpretation and formal grounded theory, and have overlapping definitions and methods (Whittemore 2005, Ring et al 2011).
Grant and Booth (2009) described 14 types of reviews, including critical or literature review, mapping review or systematic map, meta-analysis, mixed studies or mixed methods review or overview, qualitative systematic or evidence review or qualitative synthesis, rapid review, scoping review, state-of-the-art review, systematised review and umbrella review.
Meanwhile, the CIHR has described a number of knowledge synthesis methods from primary studies, such as systematic reviews of quantitative evidence, syntheses of qualitative evidence, mixed methods syntheses; and from broad and diverse studies, such as scoping reviews, multiple treatment meta-analysis or network, meta-narrative synthesis (CIHR 2014b).
Some scholars have described the plethora of knowledge synthesis methods found in the literature (Dixon-Woods et al 2005, Barnett-Page and Thomas 2009), others have built a typology (Grant and Booth 2009) to help inform policy makers (Mays et al 2005), and yet others have described the differences between them (Gough et al 2012). This article gives a comprehensive description of knowledge synthesis methods, their relationships and recommendations for appropriate use.
Any scientific investigation is composed of several stages (Chalmers 2003) that are often used as quality criteria for conducting literature reviews (Whittemore 2005). The stages are as follows:
■ A well-defined problem/research question (purpose and objectives) taking the form ‘PICOT’: population, intervention, comparison, outcome, and time (Melnyk and Fineout-Overholt 2011).
■ Explicit identification of review methods (for example, inclusion and exclusion criteria) by investigators with expertise in content and method. It is important to identify and describe the methods of a knowledge synthesis.
■ Clear specification of review process and development of a knowledge synthesis protocol. Writing a knowledge synthesis protocol is the first step toward an explicit and transparent method of knowledge synthesis.
■ Comprehensive and explicit literature search. For example, identify potential studies, apply pre-determined inclusion/exclusion criteria). Literature search strategies are critical for enhancing the rigour of a knowledge synthesis, because biased searches result in potentially inaccurate results (Cooper 1998).
■ Explicit, unbiased and reproducible data extraction for content and quality.
■ Appraisal of the quality of primary studies. This stage is important for including high quality primary studies in the knowledge synthesis.
■ Systematic synthesis and analysis of data, during which variability of findings are addressed and evidence from primary studies identified.
■ Conclusions based on evidence that capture the complexity of the (clinical) problem.
■ Limitations of the knowledge synthesis needs to be acknowledged in the final report.
■ A structured written report is the outcome of any type of knowledge synthesis.
Evidence Sources of evidence include results of well-designed research, patients’ needs and preferences, and practitioners’ experience and expertise, taking into consideration the contextual characteristics (for example culture) within which evidence may be implemented (Pearson 2010).
Knowledge synthesis In Canada knowledge synthesis is defined as ‘the contextualization and integration of research findings of individual research studies within the larger body of knowledge on the topic. A synthesis must be reproducible and transparent in its methods, using quantitative and/or qualitative methods... Knowledge synthesis could take the form of systematic review that follows the methods developed by the Cochrane Collaboration or Joanna Briggs Institute, or result from a consensus conference or expert panel’. (www.cihr-irsc.gc.ca/e/36331.html)
In the Netherlands a similar definition incorporates more sources of evidence/knowledge: ‘(knowledge synthesis is) a strategy for combining information from research with information from policy-makers and practitioners in a systematic and transparent way in order to promote the use of knowledge by disease prevention workers, health care providers and their professional associations, patients and patient groups, managers of health care, disease prevention institutions, health insurers and policy makers’ (Bos and van Kammen 2007).
Knowledge translation This is described as ‘…a dynamic and iterative process that includes synthesis, dissemination, exchange and ethically-sound application of knowledge to improve the health of Canadians, provide more effective health services and products, and strengthen the healthcare system’. (www.cihr-irsc.gc.ca/e/29418.html)
Grant and Booth (2009) described types of reviews based on three approaches to conducting literature reviews:
■ Typology (vocabulary/terminology used in the literature).
■ Time and resources needed to complete a review.
■ Processes that are tangible, that is well-defined, structured, globally accepted stages in conducting a knowledge synthesis.
These approaches are included in the ‘SALSA’ framework: search, appraisal, synthesis, analysis, in which each tangible process is described and types of each procedural step discussed.
Search The criteria for searching the literature focus on four main elements: sources – for example, databases, grey literature; search strategies – for example, computerised or computer-assisted search, ancestry, journal hand-searching, networking, searching research registries; time or date of publication – for example, the past five to ten years, according to conventions of each database; and scope of the literature search or knowledge synthesis – for example, specific and limited range of view, extensive or broad aim.
Based on these criteria, there are at least seven types of literature search that can be used when conducting a knowledge synthesis (Table 1). Basic search Undertaken to identify significant publications without using specified criteria. This is a questionable search that is neither reproducible nor transparent.
Comprehensive search A maximum number of primary sources are used in combination with at least two or three search strategies (Whittemore and Knafl 2005) without constraints in time and scope of the knowledge synthesis.
Complete search A comprehensive literature search with constraints in time and scope of the knowledge synthesis due to limited resources, such as, time or funds. Usually undergraduate and graduate students, in academia, or healthcare professionals, in practice, use a complete literature search.
Exploratory or purposive/selective search A comprehensive literature search using a sample of primary studies according to a specific purpose Justified, explicit and well documented steps and methods used must be recorded.
Extensive search A literature search using a maximum number of primary sources in combination with more than three strategies with or without constraints in time and scope of the knowledge synthesis.
Exhaustive search All known sources and search strategies are used with constraints in time and scope of the knowledge synthesis.
Systematic search An exhaustive literature search without constraints in time and scope of the knowledge synthesis.
Appraisal The main criteria for assessing the quality of the retrieved and relevant literature include: identification of specified criteria; use of an instrument (standardised or non-standardised); and number of appraised publications – all retrieved or sampling. Based on the above criteria there are six types of literature quality assessment (QA) that can be used when conducting a knowledge synthesis (Table 2):
Judgemental QA Evaluation of the retrieved publications based on their contribution to the literature. This assessment is questionable, however, because it can result in biased evaluation due to vague criteria of appraisal.
Critical QA Evaluation of all included relevant publications using researcher experience and judgement.
Critical (sampling) QA Evaluation of a sample of included publications using researcher experience and judgement.
Typical/informal QA Evaluation of all publications based on researcher experience and judgement, but without using an instrument.
Formal QA without constraints Evaluation of the literature based on researcher experience and judgement assisted by a generic instrument.
Formal QA with constraints (time limited) Evaluation of a sample of the literature based on researcher experience and judgement using a generic standardised instrument.
Synthesis At this point clarification of the two concepts of combining data (literature) – summary and synthesis – is required.
Summary A comprehensive, concise and succinct description of the data without new insight from the writer.
Synthesis Combining the data with the writer’s insights and thoughts to try to answer a pre-defined question. Knowledge synthesis includes one or more of the following elements:
■ Narrative format. A narrative synthesis can be a typical, commentary or minimal. Typical narrative synthesis usually is in chronological or conceptual order of the relevant publications. Commentary narrative refers to comments that compliment other types of synthesis such as graphical or tabular. Minimal narrative synthesis refers to the minimum narrative text that complements other forms of syntheses, for example, graphical, tabular.
■ Graphical diagram is a synthesis of findings in a graph form that may supplement a narrative synthesis; it is the main type of synthesis or it may not be used at all.
■ Tabular illustration is a synthesis of findings in the form of table(s) that may supplement a narrative synthesis; it is the main type of synthesis or it may not be used at all.
Types of literature synthesis based on those elements are shown in Table 3, page 34. A sample of literature synthesis types is described below. Typical narrative (For brevity it is identified as Synthesis 1 in Table 3.) This type of synthesis refers to only narrative synthesis without any tabular or graphical supplements.
Typical narrative synthesis with graphical (or tabular) supplement (Synthesis 2 or Synthesis 3) This is a typical narrative synthesis that includes graphical (or tabular) supplement(s).
Typical narrative synthesis with graphical and tabular supplements (Synthesis 4) A typical narrative synthesis that includes both graphical and tabular supplements.
Commentary (or minimal) narrative (Synthesis 5 or 13) A narrative synthesis that is commentary (or minimal) without any supplements.
Commentary (or minimal) narrative with mainly tabular supplement (Synthesis 7 or Synthesis 15) A commentary (or minimal) narrative synthesis that mainly includes tabular (but not graphical) supplement(s).
Analysis The main elements of a literature analysis include one or more of the following: conceptual analysis – thematic, inclusion of conceptual model; numerical analysis – measure of effect, direction of the effect; combination of conceptual and numerical analysis – mapping, association, identifying research gaps; inclusion of sensitivity analysis; and inclusion of recommendations for practice, policy and research. Based on these elements there are numerous types of literature analysis (Table 4) including the following:
Basic analysis of the literature To identify conceptual contribution of publications to the literature – probably in chronological order. Criteria of analysis are unspecified.
Thematic/conceptual analysis With or without numerical analysis, this can include conceptual models.
Explanatory analysis What works for whom in what circumstances, in what respect, and how? Combination of theoretical views, empirical evidence, context, mechanisms and outcomes.
Mapping A map of qualitative and/or quantitative studies is described with or without conceptual analysis.
Numerical analysis To identify direction and measures of effect size; includes sensitivity analyses and recommendations for practice, policy and research.
Mixed analysis Includes association of qualitative and quantitative studies, mapping quantity and quality of the literature, identification of research gaps, sensitivity analyses (or not), and recommendations for practice, policy and research.
Combining the four tangible processes described above has identified various methods of knowledge synthesis, the most popular of which are discussed below. The SALSA framework is applied to each method alongside its weaknesses.
Integrative review or comprehensive or critical overview This is the most commonly used method of literature review that allows the inclusion of several study designs, such as experimental and non-experimental research, theoretical studies and empirical literature, and which can result in a comprehensive review of complex concepts, theories or healthcare problems (Whittemore and Knafl 2005).
Whittemore and Knafl (2005) and Cooper (1998) have described this method of knowledge synthesis and their work is usually used as a template for an integrative literature review.
Search In theory a comprehensive literature search should be completed. However, complete or purposive search strategies are acceptable if the literature search process is clear and well documented, and includes search terms, names of databases used, search strategies, and the inclusion and exclusion criteria used to determine primary studies. In practice, the most significant concepts in the literature are usually identified without keeping records of the search process.
Appraisal Evaluation of the included publications is based on the contribution of the primary studies to the literature. Specified criteria for assessing the quality of each primary study are not used or reported in integrative literature reviews.
Synthesis A typical narrative in chronological or conceptual order of the findings is usual practice, but conclusions can be reported in tables and/or diagrams.
Analysis Data analysis strategies in integrative reviews are not well developed and reviewers mainly identify significant conceptual components or develop new theories.
Weakness This type of knowledge synthesis is open to bias when there are unspecified criteria. Additionally, lack of specific criteria to evaluate primary studies (due to lack of gold standards) and the complexity of an integrative literature review reduce the quality of this type of knowledge synthesis, which remains weak and vulnerable to bias.
Rapid review This describes a rapid synthesis of knowledge about a policy or clinical practice issue and is generally of interest to policy makers who are required to make evidence-based decisions and policies. Watt et al (2008) define a rapid review as ‘any report or systematic review that has taken between one and six months to produce which contains the elements of a comprehensive literature search’. Although it is speedy, flexible and adaptive it follows all stages of a systematic knowledge synthesis.
Search Complete but with time constraints that result in truncated literature search.
Appraisal Time-limited formal quality assessment of the primary studies.
Synthesis Narrative and tabular.
Analysis Maps quantity and quality of the literature and identifies direction of effect – is the change positive or negative; is the effect the same in all publications and so on.
Weakness The short process of rapid reviews introduces a risk of bias, for example, selection, publication, language of publication, and potential limitations that should be reported, and it is challenging to produce a scientifically rigorous and transparent report within a short period of time (Watt et al 2008). However, the opportunity to contextualise the findings (adaptiveness) is one of its strengths (Hailey et al 2000). Some authors although concerned about rapid assessments, acknowledge their usefulness and potential for influencing policy-making processes. However, they also emphasise the need to follow a rapid review with a full systematic review (Hailey et al 2000).
Overall, there is a lack of standardised and explicit method for undertaking rapid reviews (Harker and Kleijnen 2012), however a consistent method could result in loss of adaptiveness (Watt et al 2008) so the appropriate use of rapid reviews is more important than developing a formalised and consistent method of application (Watt et al 2008). Recommendations include increasing the transparency of the method used, consideration of using a rapid review as an interim until a systematic review can be conducted, and ensuring the appropriate use of rapid reviews (Ganann et al 2010, National Collaborating Centre for Methods and Tools 2010).
Scoping reviews These are defined by Mays et al (2005) as studies that aim ‘to map rapidly the key concepts underpinning a research area and the main sources and types of evidence available, and can be undertaken as stand-alone projects in their own right, especially where an area is complex or has not been reviewed comprehensively before’ (Arksey and O’Malley 2005).
However, other scholars (Anderson et al 2008, Davis et al 2009) argue that Mays’s definition lacks clarity and is widely varied in purpose and methodological rigour. Daudt et al (2013) suggest replacing the word ‘rapidly’ with ‘thoroughly and thoughtfully’ and recommend that scoping reviews should ‘aim to map the literature on a particular topic or research area and provide an opportunity to identify key concepts, gaps in the research, and types and sources of evidence to inform practice, policymaking, and research.
Search Comprehensive or complete.
Appraisal Informal quality assessment of the included publications or no assessment at all. Some authors suggest that undertaking a quality assessment using validated instruments is an important component of scoping reviews (Levac et al 2010, Daudt et al 2013). However, that would mean these reviews would not identify key concepts, gaps in research and sources of evidence by mapping the literature, but would instead assess the quality of studies that support those areas.
Synthesis Tabular and narrative commentary.
Analysis Mapping quantity and quality of the literature and identifying research gaps.
Weakness The main weakness of scoping reviews is that they are not the final output of the knowledge synthesis process, but are an intermediate by-product. Their main strength is their ability to extract the essence of a diverse and significant body of evidence that is both developmental and intellectually creative (Davis et al 2009).
Systematic review Whittemore and Knafl (2005) described systematic reviews as ‘the method of choice for evidence-based practice initiatives’, however they have evolved in the last decade. The Cochrane Collaboration defines a systematic review as ‘a review of clearly formulated questions that uses systematic and explicit methods to identify, select, and critically appraise relevant research, and to collect and analyse data from the studies that are included in the review.
Statistical methods (meta-analysis) may or may not be used to analyse and summarise the results of the included studies’ (Higgins and Green 2011). A systematic review can be used for anything that requires a synthesis of the literature, including, for example, evidence-based clinical practice, evidence-based policy or evidence-based legislation.
Search Comprehensive and exhaustive.
Appraisal Formal quality assessment.
Synthesis Narrative with tabular supplement.
Analysis Recommendations for practice, policy and research are provided.
Weakness Systematic reviews answer simple and well-defined research questions and are not used for complex situations. To conduct a systematic review, a team of experts in the method and content/topic, and knowledge users, are required to work collaboratively.
Realist synthesis The logic of enquiry in a realist synthesis is a qualitative systematic review (based on the ‘realist’ approach) that aims to identify and explain social interventions or programmes, and the interaction between context, mechanisms and outcomes for policy makers.
Search Complete/exploratory, purposive sampling and ‘snowball’ sampling – a strategy used in research to find samples via social networks (Pawson et al 2005).
Appraisal Use of judgement to complement formal quality assessment.
Synthesis Narrative to determine what works for whom, how and under what circumstances. Contradictory evidence is used to generate insights for the contextual effects (Pawson et al 2005).
Analysis Explanatory, recommendations (tentative). Different data are extracted by different studies.
Weakness Realist synthesis is not a protocol-driven approach, but includes an explicit and reflexive quality appraisal process. The findings, for example the effects, are not generalisable.
Meta-analysis This is a transparent, objective and replicable statistical method that aims to synthesise evidence of the effects of interventions, compare effect sizes from relevant, independent, primary, quantitative and ‘combinable’ studies, report a precise summary effect, and support evidence-based practice or policy (Borenstein et al 2009, Cooper et al 2009). Meta-analysis usually follows a systematic review.
Search Systematic.
Appraisal Formal quality assessment and sensitivity analyses.
Synthesis Graphical and tabular with narrative commentary.
Analysis Numerical analysis of effect.
Debate Meta-analysis is a tool with multiple applications that can cause confusion and lead to discussions about what is the right way to perform a research synthesis when there is no single right way (Borenstein et al 2009); it depends on the purpose of the synthesis and the data available. However, there is an ongoing debate about inappropriate use of meta-analysis, for example, comparing apples and oranges.
In systematic reviews there is an element of subjectivity in setting rules to search for primary studies and including them in the review; although, the mechanisms are transparent because all decisions are clearly specified. On the other hand, meta-analysis provides a transparent, objective, and replicable framework (Borenstein et al 2009).
Overview of reviews (or umbrella review) These are designed to identify high quality, reliable systematic reviews on a specific health problem or topic, explore consistency of findings across reviews, and to summarise, integrate, synthesise and compile evidence from multiple systematic reviews into one accessible and usable document using systematic and explicit methods similar to systematic reviews (Hartling et al 2012).
Overviews provide a comprehensive synthesis of evidence and focus mainly on different interventions for the same condition, different outcomes for the same intervention in the same condition, or the same intervention for different conditions or populations to guide healthcare providers and administrators in their clinical decision making (Cheung et al 2012a).
At present, there are no methodological standards or guidelines for conducting an overview of reviews. More information on this subject is available in the Cochrane Handbook (Cochrane Collaboration 2011).
Search Systematic as in systematic reviews.
Appraisal Formal quality assessment using instrument for systematic reviews, for example, Preferred Reporting Items for Systematic Reviews (PRISMA) and meta-analyses.
Synthesis The majority of overviews provide a narrative or qualitative synthesis of the results from the included systematic reviews. However, graphical and tabular syntheses might also be used.
Analysis An overview of reviews often reports the results found in each component systematic review, but those results may be re-analysed using meta-analysis. Recommendations for practice, policy and research are also provided.
Weakness At present overviews are not commonly used, or are not feasible, because of the lack of systematic reviews on all topics relevant to health.
Knowledge synthesis activities in health care usually focus on the methodologically rigorous Cochrane reviews, but in this article the author has described current and future methods of synthesis for clinical and non-clinical healthcare professionals who need to be familiar with the most common types.
There is no agreement among scholars worldwide on definitions of and terms used to describe types of knowledge synthesis, which can result in inconsistent outcomes. For example, an integrative literature review that is described as ‘systematic’ might not follow the standards of a systematic review, and therefore result in some bias (Arksey and O’Malley 2005). Further, Kee (2014) warns that the ‘scholars of tomorrow won’t be able to say they’ve done a systematic literature review… The amount of information will accumulate faster than they can read and understand it. They’ll be trying to drink from a fire hose… There is too much information, not enough time’.
Therefore, healthcare professionals need to be able to read, understand and evaluate the literature before using research findings in practice, and this learning should begin at undergraduate stage through conducting integrative literature reviews as projects or assignments and continue in postgraduate studies through conducting scoping or rapid reviews to allow them to hone their knowledge synthesis skills. Finally, those who undertake doctoral studies must be familiar with, or conduct, a systematic review or a realist synthesis on their topic of interest. However, systematic reviews require a team to complete the enormous amount of work involved, and while some scholars argue that doctoral students can do this to learn the various ways of appraising diverse literature, others suggest that because it requires teamwork it is not a suitable research project for a doctoral student.
There is a need for agreement on knowledge synthesis methods, on specific criteria for evaluation, and on how to perform and appraise knowledge synthesis for its effects and influence on clinical practice and policy making processes. Table 5 displays recommendations for various types of knowledge synthesis for international agreement.
28 August 2014 / Vol 21 issue 5
Lessons learned about art-based approaches for disseminating knowledge
Aim To present a case example of using an arts-based...
