by Michael Turlik, DPM1 
The Foot and Ankle Online Journal 2 (9): 5
Clinical practice guidelines are defined and their use is explained. Two published guidelines dealing with heel pain are evaluated for validity using a common readily available validated instrument which can be accessed on the internet.
Key words: Evidence-based medicine, industry sponsored trials.
This is an Open Access article distributed under the terms of the Creative Commons Attribution License. It permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. ©The Foot and Ankle Online Journal (www.faoj.org)
Accepted: August, 2009
Published: September, 2009
ISSN 1941-6806
doi: 10.3827/faoj.2009.0209.0005
Clinical practice guidelines (CPG) are systematically developed documents which are published to provide specific recommendations to standardize the process of diagnosis and treatment of common clinical disorders for clinicians, patients and healthcare administrators. They are derived from the best available evidence and current best practice. Authors of CPG’s gather appraise and combine evidence similar to systematic reviews however, unlike systematic reviews CPG’s provide actionable recommendations at a clinical level. It is hoped that by publishing guidelines that there is a decrease of ineffective care with a corresponding increase of effective care. Outcomes of CPG’s would be: increased consistency, higher quality care and more predictable healthcare processes. CPG’s are developed by various methods, by diverse stakeholders for different purposes. The process of developing a CPG should be transparent and minimize bias. The recommendations for patient care made should be clear and understandable. Each recommendation should reference the source of the information used to generate the recommendation.
The information cited by the guideline developers for each recommendation should be appraised for its quality and graded for its strength. Primary studies used to make the recommendations in the CPGs may be valid however, the strength will be graded lower if they demonstrate a small imprecise effect or have substantial risks and costs. Using the information provided by the authors of the guideline regarding study quality and grade the podiatric physician will be able to have confidence in the recommendations generated. CPGs are not flawless or a substitute for clinical judgment. CPG’s should not be considered as mandatory statements since not all recommendations may be linked to high quality evidence. In addition, not all podiatric physicians may function in the same environment with the same patients used to develop the CPG’s. CPG’s may be valid but not relevant to the specific clinician or patient.
If CPG’s are used by podiatric physicians they should have an understanding of how to critically analyze the guideline for validity, interpret the results and determine its generalizability to their unique situation. Systematic errors in the development of the CPG can distort the recommendations away from the truth.
What can go wrong; incorrect search strategies can result in loss of important papers, inadequate appraisal and synthesis of the papers found can result in incorrect recommendations, wording, format or structure which is confusing can lead to misunderstanding. Finally, it is important to the podiatric physician to determine who paid for the study, if there was any conflict of interest involving the authors and what was done about it. The purpose of this paper is to provide instruction for podiatric physicians in evaluating CPG’s. Two different CPG’s dealing with a common podiatric complaint heel pain [1,2] will be compared and contrasted for validity and relevance.
Evaluating clinical practice guidelines
There are several published instruments in use to evaluate clinical practice guidelines. The Conference on Guideline Standardization [3] (COGS) developed and published an 18 item instrument to evaluate the validity of clinical practice guidelines. The Grading of Recommendations Assessment, Development, and Evaluation working group4 (GRADE) was begun in 2000 to develop a common, sensible and transparent approach to grading the quality of evidence and strength of recommendations used in clinical practice guidelines. The AGREE instrument5 was developed 2001 by an international group of researchers and policymakers. The AGREE instrument will be used in this article to critically analyze the clinical practice guidelines referenced earlier.
The AGREE website5 provides the instrument to evaluate a CPG and a training manual for the instrument. The AGREE instrument consists of 23 separate items grouped into six quality domains (Table 1) which measure both internal and external validity of CPG’s. It is a validated generic instrument which can be used to evaluate new, existing and revised CPG’s. The AGREE instrument evaluates the process not the content of the CPG.
Table 1 AGREE quality domains.
AGREE Evaluation
Scope and purpose
This domain consists of three separate questions (Q) which evaluate the overall aim of the guideline, the specific clinical questions and target patient population. Answers to these questions allow the podiatric physician to determine if the CPG is relevant to his or her practice setting (generalizability).
Q1 The overall objective (s) of the guideline is (are) specifically described.
The American College of Foot and Ankle Surgeons (ACFAS) heel pain guideline [1] does not explicitly state an overall objective. The American Physical Therapy Association (APTA) guideline [2] is one of a series of guidelines produced by APTA. General purposes for the series of guidelines are explicitly stated.
Q2 The clinical question (s) covered by the guideline is (are) specifically described.
The development of foreground questions utilizing the PICO (Patient, Population, Problem, Intervention, Exposure, Comparison and Outcome) technique have been described elsewhere. [6] The ACFAS heel pain guideline1 does not explicitly state clinical question (s) to be covered. The APTA guideline [2] specifically describes two different tasks which it hopes to accomplish.
Q3 The patients to whom the guideline is meant to apply are specifically described.
Neither the ACFAS heel pain guideline [1] nor the APTA guideline [2] explicitly state the patients in whom the guideline is meant to apply.
Stakeholder Involvement
This domain is composed of a series of questions which focus on the extent to which the guideline represents the views of its intended users. The answers to these questions will help the podiatric physician in determining the CPG’s relevance to his or her clinical practice.
Q4 The guideline development group includes individuals from all relevant groups.
The guideline development group should be diverse to include various stakeholders; end users, policy makers and consumers. [7] It is of interest to the podiatric physician if the guideline development group includes a podiatrist.
The ACFAS heel pain guideline [1] was developed by podiatrists with membership in the ACFAS. No other groups were involved. The APTA guideline [2] was developed principally by physical therapists some with advanced degrees as well as, an orthopedic surgeon specializing in foot and ankle care.
Q5 The patient’s views and preferences have been sought.
It is important in evidence-based medicine to include the values and concerns of patients. [8]
There is no evidence that the ACFAS heel pain guideline [1] or the APTA guideline [2] included patient’s views and preferences in developing the CPG.
Q6 The target users of the guideline are clearly defined.
From a podiatric physician’s viewpoint it is important to consider whether the target user of the guideline specifically lists podiatry.
The ACFAS heel pain guideline [1] does not define target users of the guideline. The authors of the APTA guideline [2] define the target users as orthopedic physical therapy clinicians, students, residents, academic instructors, clinical instructors, fellows and interns. Podiatric physicians are not mentioned in the APTA guideline as authors, reviewers or intended recipients.
Q7 The guideline has been piloted among targeted users.
The ACFAS heel pain guideline [1] has not been piloted among targeted users. The APTA guideline [2] authors provide a detailed and comprehensive explanation of the review process. The guideline was reviewed by multiple varied healthcare practitioners for feedback prior to being finalized.
Rigor of development
This domain relates to the process to gather and synthesize the evidence, the methods to formulate the recommendations and to update them. The answers to these questions will help the podiatric physician in determining the internal validity of the CPG.
Q8 Systematic methods were used to search for the evidence.
An earlier publication has covered this topic in some detail. [9] The ACFAS heel pain guideline [1] provided no information regarding the search strategy in the development of the CPG. The authors of the APTA guideline [2] discussed why a systematic search could not be utilized in the development of the CPG.
Q9 The criteria for selecting the evidence are clearly described.
The ACFAS heel pain guideline [1] does not provide any criteria for selecting the evidence used in the development of the CPG. The APTA guideline [2] in the methods section described the criteria which were used to select the evidence used in the CPG.
Stakeholder Involvement
This domain is composed of a series of questions which focus on the extent to which the guideline represents the views of its intended users. The answers to these questions will help the podiatric physician in determining the CPG’s relevance to his or her clinical practice.
Q4 The guideline development group includes individuals from all relevant groups.
The guideline development group should be diverse to include various stakeholders; end users, policy makers and consumers. [7] It is of interest to the podiatric physician if the guideline development group includes a podiatrist.
The ACFAS heel pain guideline [1] was developed by podiatrists with membership in the ACFAS. No other groups were involved. The APTA guideline [2] was developed principally by physical therapists some with advanced degrees as well as, an orthopedic surgeon specializing in foot and ankle care.
Q5 The patient’s views and preferences have been sought.
It is important in evidence-based medicine to include the values and concerns of patients. [8]
There is no evidence that the ACFAS heel pain guideline [1] or the APTA guideline [2] included patient’s views and preferences in developing the CPG.
Q6 The target users of the guideline are clearly defined.
From a podiatric physician’s viewpoint it is important to consider whether the target user of the guideline specifically lists podiatry.
The ACFAS heel pain guideline [1] does not define target users of the guideline.
The authors of the APTA guideline [2] define the target users as orthopedic physical therapy clinicians, students, residents, academic instructors, clinical instructors, fellows and interns. Podiatric physicians are not mentioned in the APTA guideline as authors, reviewers or intended recipients.
Q7 The guideline has been piloted among targeted users.
The ACFAS heel pain guideline [1] has not been piloted among targeted users. The APTA guideline [2] authors provide a detailed and comprehensive explanation of the review process. The guideline was reviewed by multiple varied healthcare practitioners for feedback prior to being finalized.
Rigor of development
This domain relates to the process to gather and synthesize the evidence, the methods to formulate the recommendations and to update them. The answers to these questions will help the podiatric physician in determining the internal validity of the CPG.
Q8 Systematic methods were used to search for the evidence.
An earlier publication has covered this topic in some detail. [9] The ACFAS heel pain guideline [1] provided no information regarding the search strategy in the development of the CPG. The authors of the APTA guideline [2] discussed why a systematic search could not be utilized in the development of the CPG.
Q9 The criteria for selecting the evidence are clearly described.
The ACFAS heel pain guideline [1] does not provide any criteria for selecting the evidence used in the development of the CPG. The APTA guideline [2] in the methods section described the criteria which were used to select the evidence used in the CPG.
Q15 The recommendations are specific and unambiguous.
The ACFAS heel pain guideline [1] does make recommendations but they are not unambiguous and not specific. In contrast the APTA guideline [2] makes specific and unambiguous recommendations.
Q16 The different options for management of the condition are clearly presented.
The ACFAS heel pain guideline [1] and the APTA guideline [2] both describe different options for management in its CPG.
Q17 The key recommendations are easily identifiable.
The ACFAS heel pain guideline [1] and does not make the key recommendations easily identifiable however; the APTA guideline [2] does.
Q18 The guideline is supported with tools for application.
The entire document is usually large and cumbersome and if a more condensed easily accessed version is not produced for clinicians and patients it is unlikely that it will be utilized as effectively. [13]
The ACFAS heel pain guideline [1] does not provide any tools for application. The APTA guideline [2] does provide a single page listing the recommendations with the grade and strength of the evidence at the beginning of the publication. This allows the recommendations to be easily used by interested parties.
Application
The questions in this domain pertain to the likely organizational, behavioral and cost implications of applying the CPG.
Q19The potential organizational barriers in applying the recommendations have been discussed.
Organizational barriers may limit the usefulness and application of the CPG. A recent article provides an in-depth discussion regarding this topic. [14]
Neither the ACFAS heel pain guideline [1] nor the APTA guideline [2] discussed the potential organizational barriers in applying the CPG.
Q20 The possible cost implications of applying the recommendations have been considered.
Given the rapidly changing dynamics the health care policy of the United States it would be short sighted to not consider the cost implications of recommendations if data were available. [15]
Neither the ACFAS heel pain guideline [1] nor the APTA guideline [2] discussed the cost implications of applying the recommendations.
Q21 The guideline presents key review criteria for monitoring and/or audit purposes.
The ACFAS heel pain guideline [1] does not provide any information concerning criteria for monitoring and/or audit purposes. The APTA guideline [2] recommends the use of validated self-reported instruments to monitor response to treatment and gives several examples.
Editorial independence
Q22 The guideline is editorially independent from the funding body.
The ACFAS heel pain guideline [1] does not discuss who has funded the study but explicit in the document is that it is authored by a committee from the ACFAS. It is not clear if the development of the guideline is editorially independent from the ACFAS. The APTA guideline [2] is authored by members of the orthopedic section of the APTA. It is not clear who has funded the guideline and if the authors are editorially independent of the APTA.
Q23 Conflict of interest of guideline members have been recorded.
It has been shown quite clearly that industry sponsored studies are likely to provide pro industry results. [16] It is important for guideline developers to provide information to the users regarding how conflict of interest was dealt with when found. [17] It is thought that the most common source of bias in CPG’s is financial. [18] In a survey of physician authors of CPG’s 87% had some form of interaction with the pharmaceutical industry. [19]
Neither the ACFAS heel pain guideline [1] nor the APTA guideline [2] discussed a conflict of interest process.
Response scale
Each of the 23 items of the AGREE instrument are individually evaluated on a four-point scale. [5] The scale measures the extent to which the item has been fulfilled. The higher the number the greater the AGREE criteria have been met by the authors of the guideline. Comparing the two different guidelines (Table 2) the guideline produced by ACFAS scored lower using the AGREE instrument when compared to the APTA guideline.
Table 2 Results of comparison ACFAS / APTA guidelines using the AGREE instrument.
In a review of CPG’s published by specialty societies the authors found that 88% did not report information regarding the search strategy and 82% did not report recommendations specifically linked to the quality and grade of the evidence used.20 This is consistent with the results of the CPG produced by the ACFAS. Neither guideline scored well in the domains of applicability and editorial independence. This is consistent with other reviews of CPGs21,22 which found that applicability and editorial independence domains were rated lowest using the AGREE instrument.
Conclusion
Older clinical practice guidelines are characterized by narrative reviews and expert opinions without explicit evaluation of the best evidence available.23 Based upon the results of the AGREE instrument the ACFAS clinical practice guideline follows an older expert based format. The APTA clinical practice guideline follows a more contemporary approach to the development of clinical practice guidelines. It is characterized by its adherence to evidence-based principles. APTA guidelines contain clear explicit actionable recommendations which are linked to evidence which has been evaluated for grade and strength. The APTA guideline does not provide comprehensive recommendations for medical treatment and no recommendations for surgical treatment of heel pain thus limiting its relevance to practicing podiatric physicians.
References
1. Thomas J, Christensen J, Kravitz S, Mendicino R, Schuberth J, Vanore J, Scott Weil L, Zlotoff H, Couture S: Clinical practice guideline heel pain panel: The diagnosis and treatment of heel Pain. JFAS 40: 329 – 340, 2001.
2. McPoil T, Martin R, Cornwall M, Wukich D, Irrgang J, Godges J: Heel pain – Plantar fasciitis: Clinical practice guidelines linked to the international classification of function, disability, and health from the orthopaedic section of the American Physical Therapy Association. J Orthop Sports Phys Ther 38: 629 – 648, 2008.
3. COGS http://gem.med.yale.edu/cogs/ Accessed 7/15/2009
4. GRADE http://www.gradeworkinggroup.org/ Accessed 7/15/09.
5. AGREE http://www.agreecollaboration.org/ Accessed 7/15/2009.
6. Turlik M: Introduction to evidence-based medicine. Foot and Ankle Online Journal 2: 2009.
7. Fretheim A, Schünemann H, Oxman A: Improving the use of research evidence in guideline development: Group composition and consultation process. Health Research Policy and Systems 4:15, 2006.
8. Schünemann H, Fretheim A, Oxman A: Improving the use of research evidence in guideline development: Integrating values and consumer involvement. Health research policy and systems 4: 22, 2006.
9. Turlik, M. Evaluation of a review article. Foot and Ankle Online Journal. 2: 2009.
10. Making group decisions and reaching consensus. http://www.nice.org.uk/niceMedia/pdf/GDM_Chapter9.pdfaccessed Accessed 7/20/2009.
11. Guyatt GH, Oxman AD, Vist GE, Kunz R, Falck-Ytter Y, Alonso-Coello P, Schünemann HJ, GRADE Working Group:. GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ 336: 924 – 26, 2008
12. Shekelle PG, Ortiz E, Rhodes S, Morton SC, Eccles MP, Grimshaw JM, Woolf SH: Validity of the agency for healthcare research and quality clinical practice guidelines how quickly do guidelines become outdated? JAMA 286: 1461 – 1467, 2001.
13. Trevena L, Davey H, Barratt A, Butow P, Caldwell P: A systematic review on communicating with patients about evidence. J Evaluation Clinical Practice. 12: 13 – 23, 2006.
14. Shiffman R, Dixon J, Brandt C, Essaihi1 A, Hsiao A, Michel G, O’Connell R: The GuideLine Implementability Appraisal (GLIA): development of an instrument to identify obstacles to guideline implementation. BMC Medical Informatics and Decision Making 5: 23, 2005.
15. Edejer T: Improving the use of research evidence in guideline development: Incorporating considerations of costeffectiveness, affordability and resource implications. Health Research Policy Systems 4: 23, 2006.
16. Turlik M. Special considerations when reviewing industry sponsored studies. Foot and Ankle Online Journal. 2: 2009.
17. Boyd E, Bero L: Improving the use of research evidence in guideline development: Managing conflicts of interests. Health Research Policy and Systems 4: 16, 2006.
18. Allan S. Detsky A: Sources of bias for authors of clinical practice guidelines. CMAJ 175 (9): 1033, 2006.
19. Choudry N, Stelfox H, Detsky A: Relationships Between Authors of Clinical Practice Guidelines and the Pharmaceutical Industry. JAMA: 287:612 – 617, 2002.
20. Grilli R, Magrini N, Penna A, Mura G, Liberati A: Practice guidelines developed by specialty societies: the need for a critical appraisal. Lancet 355: 103 – 106, 2000.
21. Cates J, Young D, Bowerman D, Porter R: An independent AGREE Evaluation of the occupational medicine practice guidelines. The Spine J 6: 72 – 77 , 2006.
22. Hurdowar A, Graham I, Bayley M, Harrison M, Wood-Dauphinee S, Bhogal S: Quality of stroke rehabilitation clinical practice guidelines. J Evaluation Clinical Practice 13: 657 – 664, 2007.
23. Poolman R, Verheyen C, Kerkhoffs G, Bhandari M, Schünemann H: From evidence to action:
Understanding clinical practice guidelines. Acta Orthopaedics 80: 113 – 118, 2009.
Address correspondence to: Michael Turlik, DPM
Email: mat@evidencebasedpodiatricmedicine.com
1 Private practice, Macedonia, Ohio.
© The Foot and Ankle Online Journal, 2009
