Survey of preferred guideline attributes: what helps to make guidelines more useful for emergency health practitioners?
© Aboulsoud et al.; licensee Springer. 2012
Received: 19 June 2012
Accepted: 24 October 2012
Published: 10 November 2012
Enhancing CPG acceptance and implementation can play a major role in the development and establishment of emergency medicine as a specialty in many parts of the world. A Guideline International Network special interest group established to support collaboration to improve uptake of clinical practice guidelines (CPGs) across the emergency care sector conducted an international survey to identify attributes of guideline likely to enhance their use.
A Web-based survey was undertaken to determine how CPGs were accessed, the preferred formats and attributes of guidelines, and familiarity with GRADE. The criteria used to identify preferred attributes of guidelines were adapted from the AGREE II Tool.
Two hundred six responses were received from 31 countries, 74/206 (36%) from the US, 28/206 (16%) from Canada, 17/206 (8%) from Australia and 15/206 (7%) from the UK. The majority of responses were from physicians (176/206, 85%) with 15/206 (7%) of responses from nurses and 9/206 (4%) from pre-hospital emergency services personnel. The preferred format for guidelines was clinical protocols that incorporated recommendations into workflow, and the most preferred attribute of guidelines was the clear identification of key recommendations. The results also identified that within the group that responded to the question related to GRADE, 66% were unfamiliar with this system for summarizing evidence in relationship to recommendations.
The findings provide the basis for further research to explore the most appropriate formats for guidelines or guidelines resources tailored to the needs of the emergency care providers.
In 2007 the Guidelines International Network (G-I-N) established the Emergency Care (EC) Community as a special interest group of G-I-N to support collaboration to improve both the awareness and uptake of clinical practice guidelines across the emergency care sector internationally. One of the first initiatives of this group was to seek to identify the attributes of guidelines and guideline resources to improve the usability and implementation of guideline recommendations.
The 2006 IOM report “Future of Emergency Care Series, Hospital Emergency Care at the Breaking Point” called for the standardization and implementation of guidelines . This report was based on the a comprehensive independent review that focused on all aspects of emergency care provided by US Emergency Medical Services. The findings of the review have had wide applicability to emergency services internationally. The report identified the increasing importance of guidelines and their implementation to improve patient care and health outcomes based on the best available research.
The gap in general awareness and implementation of guidelines has been well documented over the last 2 decades with specific reference to emergency medicine by Schriger et al. in 1993 [2–5]. Enhancing guideline applicability acceptance and implementation can play a major role in the development of emergency medicine as a specialty in many parts of the world. The EC Community sought to conduct a scoping survey as a hypothesis-generating effort to determine aspects of attitudes, preferences and awareness of guidelines across various professional groups of Emergency Medical Service (EMS) providers.
how CPGs are accessed,
which guideline formats and attributes are preferred, and
the degree of respondent familiarity with GRADE (Grading of Recommendations Assessment, Development and Evaluation).
The criteria used to identify preferred attributes of guidelines were adapted from the AGREE II Tool (Appraisal of Guidelines Research and Evaluation Tool) . The working group also considered it important to gauge the knowledge related to the GRADE system. GRADE is being widely adopted to inform the strength of recommendations based on a transparent explicit approach for rating quality of evidence and connecting evidence to recommendations . The International Liaison Committee on Resuscitation (http://www.ilcor.org), the premier organization for developing evidence-based guidance on emergency care, is transitioning to GRADE in 2015.
The survey (Appendix A) was piloted widely through the EC Community membership. Further feedback was also sought from G-I-N board members. Changes made to the survey included revising the Likert rating to remove a neutral response and ensure that all professional groups were considered in the demographic information. The Australasian College for Emergency Medicine framework was used to classify practice settings based on the Emergency Departments’ level of service available .
A snowballing technique was used to distribute the survey; this method relies on referrals from initially sampled respondents to others who are believed to have the same interest . The survey was e-mailed by the working group and the EC Community members worldwide to utilize their extended networks across medical, nursing and pre-hospital professional groups to invite further participation. The survey was limited to English-speaking emergency healthcare providers from primary and tertiary care including urban and rural settings.
Frequency tables were used to determine the results of from the survey questions. A SPSS package was used to analyze the data. The chi-square test was used to analyze several associations: (1) length of practice and use of guidelines in daily practice, (2) practice setting and preferred guideline formats and (3) practice setting and methods of accessing of guidelines.
Aggregation of the practice setting into community level care and tertiary level care that included the major referral and major regional emergency departments was done to increase the sample size. The responses were aggregated to provide more meaningful interpretation (agree and strongly agree were considered as one category, as was disagree and strongly disagree) . Associations between GRADE familiarity and its usefulness were also analyzed.
Two hundred six responses were received from 31 countries with representation from the US, UK, Canada and Australia. The majority of responses were from physicians (176/206, 85%) with 15/206 (7%) of responses from nurses and 9/206 (4%) from pre-hospital emergency services personnel.
Practice experience of responding emergency practitioners ranged from less than 2 years to greater than 20 years. Of the respondents, 144/206 (16%) were from tertiary-level health facilities, 14/206 (16%) from community- or urban-based healthcare facilities settings and 30/206 (15%) from rural or maritime settings.
Summary of survey responses
Length of practice in the emergency care sector
Up to 2 years
From 2 to 5 years
From 5 to 10 years
From 10 to 15 years
From 15 to 20 years
Greater than 20 years
Is the use of clinical practice guidelines a part of your daily practice?
Always use guidelines
Usually use guidelines
Occasionally use guidelines
Guidelines not used or discussed
Preferred formats of guidelines or guideline resources to support uptake of best practice at point of care?
% Agree responses
Clinical protocols that translate recommendations into work flow
Plain language evidence summaries
Clinical algorithms (flow charts) formats
Electronic order sets with incorporating guidelines
Prioritized list of recommendations, e.g., Care Bundles
Full systematic reviews
Education slide sets
Grading the evidence
% Agree responses
Are you familiar with GRADE?
The preferred attributes of guidelines as adapted from the AGREE II Tool
Preferred attributes of guidelines/guideline resources
% Agree responses
1. The overall objective(s) of the guideline is (are) specifically described
2. The health question(s) covered by the guideline is (are) specifically described
3. The population (patients, public, etc.) to whom the guideline is meant to apply is specifically described
4. The guideline development group includes individuals from all the relevant professional groups
5. The views and preferences of the target population (patients, public, etc.) have been sought
6. The target users of the guideline are clearly defined
7. Systematic methods were used to search for evidence
8. The criteria for selecting the evidence are clearly described
9. The strengths and limitations of the body of evidence are clearly described
10. The methods for formulating the recommendations are clearly described
11. The health benefits, side effects and risks have been considered in formulating the recommendations
12. There is an explicit link between the recommendations and the supporting evidence
13. The guideline has been externally reviewed by experts prior to its publication
14. A procedure for updating the guideline is provided
15. The recommendations are specific and unambiguous
16. The different options for management of the condition or health issue are clearly presented
17. Key recommendations are easily identifiable
18. The guideline describes facilitators and barriers to its application
19. The guideline provides advice and/or tools on how the recommendations can be put into practice
20. The potential resource implications of applying the recommendations have been considered
21. The guideline presents (includes) monitoring and/ or auditing criteria
22. The views of the funding body have not influenced the content of the guideline
23. Competing interests of guideline development group members have been recorded and addressed
This study has provided data from the emergency care practice environment related to the behaviors, attitudes, preferences and knowledge about CPGs. Barriers to the uptake of CPGs across these domains were first reported by Cabana in 1999 ; many of those issues are still prevalent today in a context of many more guidelines being published on a daily basis.
The results of this study support previous research that identified that guideline formats and content are important determinants of usability of guidelines in busy clinical environments such as the emergency care sector [11–13]. A survey of multidisciplinary clinicians in Australian public hospitals reported that concise, quick-reference formats were preferred to detailed texts (35% vs. 6%; P < 0.001) [12–14]. The preferred format for guidelines identified through this survey was clinical protocols that incorporated recommendations into workflow. Understanding the practice environment is a critical aspect to consider when seeking to enhance the usability of guidelines.
The most preferred guideline attribute identified through this survey was the clear identification of key recommendations. The preferences support information related to ‘what to do, why and to whom.’ Knowledge of the CPG development process was also listed as important; the systematic approach taken to review the evidence provides a level of confidence and authority in the recommendations made.
Professional societies and peer review journals were the most frequently used sources to access CPGs. G-I-N and the National Guideline Clearinghouse (NGC) are large web-based repositories of guidelines, which were almost never used to source guidelines among the respondents; some of the reasons for this could be a lack of knowledge of these repositories or that health providers have a higher level of assurance of the quality of guidelines that are published in the peer review literature.
There was no evidence of an association between the level of service provided, e.g., tertiary care or community-based care, preferred formats or where the CPGs were accessed. However, there was a positive association between more years of practice within the setting and use of CPGs. The value of guidelines as a means of establishing standards of care and reducing variation in practice may be better appreciated as more experience is gained working within the health systems.
As the guideline development process continues to evolve, there has been an emergence of grading systems such as GRADE, which is gaining some prominence. This study revealed that 66% of the respondents were unfamiliar with the GRADE system for rating the quality of evidence and recommendations. This is supported by Kotzeva et al. [13–15], who reported that clinicians had limited knowledge, experience and understanding of GRADE. This knowledge deficit related to the grading of recommendations is potentially important as a reflection of respondent capacity to recognize and critically evaluate the approach to rating the strength of evidence used by recently developed guidelines.
This study is the first to explore the preferred attributes of guidelines of those emergency care practitioners who work in an environment that is unique in the diversity of presentations often requiring time-critical decisions on management needing access to relevant high-quality information. The findings suggest the need for further research to explore the most appropriate formats for guidelines or guideline resources tailored to both the needs of the specific setting and healthcare providers within that setting. They also support the usefulness of educational efforts to increase awareness and literacy in emergency systems of grading evidence and recommendations such as GRADE.
Although the results of this survey largely reflect the attitudes of health professionals in well-resourced countries, the following concepts, illustrated in Figure 3, have potentially broad applicability across emergency care services:
• Guideline developers need to engage with end-users to ensure that guideline formats and information are relevant for specific settings and tailored to needs of EMS providers
• Publishers of guidelines need to be aware of EMS provider behaviors for accessing guidelines, and
• Promotion of guidelines to EMS providers at point of care is central to their practice to support enduring evidence-based behaviors.
There are a number of limitations of this study. The distribution of respondents was very heavily biased toward physicians practicing in a tertiary care setting. The sample was predominantly from the US, UK, Australia and Canada with minimal representation from developing countries. The process used was nonrandom selection of participants and relied on the subjective judgments of respondents.
G-I-N Emergency Care Survey 2010
We would like to know a little information about you.
Please select your professional status from the following list:
❒ Physician assistant
❒ Nurse assistant
❒ Emergency services personnel (pre-hospital)
If other, please specify:
Please select which timeframe best describes the period you have been working in or have worked in the emergency care sector.
❒ Up to 2 years
❒ From 2 to 5 years
❒ From 5 to 10 years
❒ From 10 to 15 years
❒ From 15 to 20 years
❒ Greater than 20 years
Please state in which country you predominantly practice.
Please select which best describes the practice environment that you predominantly work in:
❒ Major referral emergency department (tertiary care)
❒ Urban district emergency department (community care)
❒ Major regional/rural base emergency department (tertiary care)
❒ Rural emergency service (community care)
❒ Primary care/remote rural emergency service
If other, please describe.
Preferred formats of guidelines
The following questions relate to your preferred formats of guidelines or other documents that incorporate evidence-based recommendations drawn from guidelines that have been developed from the best available evidence/research.
Is the use of clinical practice guidelines a part of your daily practice?
Full systematic review with evidence tables and associated recommendations
Clinical protocols that translate evidence-based recommendations into a desired workflow or process.
Plain language ‘Evidence into Practice’ resources that summarize the evidence and implications for practice.
Clinical algorithms or flow charts that provide a step-by-step decision-support tool
Educational materials to promote the use of evidence-based recommendations in practice
Government-based agencies, e.g., NICS, NHMRC, SIGN, New Zealand Guideline Group, Singapore Ministry of Health, etc.
From professional specialties or specialty organizations, e.g., the American College of Emergency Physicians (ACEP), International Resuscitation Council, British Thoracic Society, etc.
From the Guideline International Network (G-I-N) Library
From the National Guideline Clearing House (NGC)
Use of the Internet search function, e.g., Google, Yahoo, etc.
Peer review journals
Locally developed guidelines
Other (please specify)
Preferred attributes of guidelines/guideline resources
For the following questions, please consider what information within a guideline is important for you at point of care. These questions are adapted from the criteria used by AGREE II (Appraisal of Guidelines Research & Evaluation). AGREE II is an internationally validated tool for the assessment of clinical practice guidelines; http://www.agreetrust.org/.
The overall objective(s) of the guideline is (are) specifically described.
The health question(s) covered by the guideline is (are) specifically described.
The population (patients, public, etc.) to whom the guideline is meant to apply is specifically described.
The views and preferences of the target population (patients, public, etc.) have been sought.
The target users of the guideline are clearly defined.
Systematic methods were used to search for evidence.
The criteria for selecting the evidence are clearly described.
The strengths and limitations of the body of evidence are clearly described.
The methods for formulating the recommendations are clearly described.
The health benefits, side effects and risks have been considered in formulating the recommendations.
There is an explicit link between the recommendations and the supporting evidence.
The guideline has been externally reviewed by experts prior to its publication.
A procedure for updating the guideline is provided.
The recommendations are specific and unambiguous
The different options for management of the condition or health issue are clearly presented.
Key recommendations are easily identifiable.
The guideline describes facilitators and barriers to its application.
The guideline provides advice and/or tools on how the recommendations can be put into practice.
The potential resource implications of applying the recommendations have been considered.
The guideline presents (includes) monitoring and/or auditing criteria.
The views of the funding body have not influenced the content of the guideline.
Competing interests of guideline development group members have been recorded and addressed.
Grading the evidence
Are you familiar with the GRADE system to assess guideline recommendations?
If your answer is yes to the above question, how useful do you find the GRADE assessment
❒ Very useful
❒ Useful sometimes
❒ Not useful at all
This article reports on the work of the EC Community and does not reflect the views of G-I-N. The EC Community supported G-I-N through the provision of secretarial support and a small grant. The Guidelines International Network (G-I-N; http://www.g-i-n.net) is a Scottish Charity recognized under Scottish Charity No. SC034047.
Members of the G-I-N EC Community Working Group:
S. Aboulsoud, M. Afilalo, R. Alzeidan, S. Bennetts, W. Bernardo, M. Castren, R. Crouch, B. Diner, M. Hosken, W. Yau, P. Wyer, E. Lang and S. Huckson.
- Hospital-based emergency care: at the breaking point/Committee on the Future of Emergency Care in the United States Health System, Board on Health Care Services: Institute of Medicine (U.S.). Committee on the Future of Emergency Care in the United States Health System. II.Series. 2007. accessed Nov 2011 http://www.nap.edu/catalog/11621.html
- Schriger DL, Cantrill SV, Greene CS: The origins, benefits, harms, and implications of emergency medicine clinical policies. Ann Emerg Med 1993, 22: 597–602. 10.1016/S0196-0644(05)81952-3PubMedView Article
- Cabana MD, Rand CS, Powe NR, Wu AW, Wilson MH, Abboud PA, Rubin HR: Why don't physicians follow clinical practice guidelines? A framework for improvement. JAMA 1999,282(15):1458–1465. 10.1001/jama.282.15.1458PubMedView Article
- Baiardini I, Braido F, Bonini M, Compalati E, Walter G: Why do doctors and patients not follow guidelines? Canonica Current Opinion Allergy Clin Immunol 2009, 9: 228–233. 10.1097/ACI.0b013e32832b4651View Article
- Hakkennes S, Dodd K: Guideline implementation in allied health professions. A Syst Rev Lit Qual Saf Health Care 2008, 17: 296–300. 10.1136/qshc.2007.023804View Article
- Brouwers M, Kho ME, Browman GP, Burgers JS, Cluzeau F, Feder G, Fervers B, Graham ID, Grimshaw J, Hanna S, Littlejohns P, Makarski J, Zitzelsberger L, for the AGREE Next Steps Consortium. AGREE II: Advancing guideline development, reporting and evaluation in healthcare. Can Med Assoc J 2010. Available online July 5, 2010 10.1503/cmaj.090449
- Guyatt GH, Oxman AD, Vist GE, Kunz R, Falck-Ytter Y, Alonso-Coello P, et al.: GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ 2008,336(7650):924–926. 10.1136/bmj.39489.470347.ADPubMed CentralPubMedView Article
- Australasian College of Emergency Medicine Statement of emergency department role delineation. 2004. http://www.acem.org.au/media/policies_and_guidelines/S12_Role_delineation_formatted_August_2004.pdf
- Johnson TP: Snowball Sampling. Encyclopedia of Biostatistics. 2005.
- Burns KE, Duffett M, Kho ME, Meade MO, Adhikari NK, Sinuff T, Cook DJ, ACCADEMY Group: A guide for the design and conduct of self-administered surveys of clinicians. CMAJ: Can Med Assoc J 2008, 179: 245–252. no. 3 (July 29, 2008) 10.1503/cmaj.080372View Article
- Gagliardi AR, Brouwers MC, Palda VA, Lemieux-Charles L, Grimshaw JM: How can we improve guideline use? A Conceptual framework Iimplementability. Imple Sci 2011, 6: 26. 10.1186/1748-5908-6-26View Article
- Carlsen B, Glenton C, Pope C: Thou shalt versus thou shalt not: a meta-analysis of GPs attitudes to clinical practice guidelines. Br J Gen Pract 2007, 57: 971–978. 10.3399/096016407782604820PubMed CentralPubMedView Article
- Scott IA, Denaro CP, Bennett CJ, Mudge AM: Towards more effective use of decision support in clinical practice: what the guidelines for guidelines don't tell you. Intern Med J 2004,34(8):492–500. 10.1111/j.1445-5994.2004.00604.xPubMedView Article
- Scott IA, Buckmaster ND, Harvey KH: Clinical practice guidelines: perspectives of clinicians in Queensland public hospitals. Intern Med J 2003,33(7):273–279. 10.1046/j.1445-5994.2003.00366.xPubMedView Article
- Kotzeva A, Solà I, Carrasco JM, Díaz Del Campo P, Gracia FJ, Calderón E, De Gaminde I, Estrada MD, Martínez F, Orrego C, Rotaeche R, Salcedo F, Velázquez P, Alonso-Coello P: Perceptions and attitudes of clinicians in Spain toward clinical practice guidelines and grading systems: a protocol for a qualitative study and a national survey. BMC Health Serv Res 2010, 3: 10–328.
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.