Roddy and Ganz Assessment

Order Description

1) Use “Roddy and Ganz Worksheets” to complete an assessment of the research studies: “Evidence-Based Clinical Guidelines: A New System to Better Determine True Strength of Recommendation” by Roddy et al., and “ICU Nurses’ Oral-Care Practices and the Current Best Evidence” by Ganz et al.

2) In a paper analyze and support whether you believe that the changes recommended are statistically valid and sound enough to make practice changes in a hospital.

Demonstrates clearly and fully the ability to complete an assessment of the research studies, analyze, and support whether the changes recommended are statistically valid and sound enough to make practice changes in a hospital (in the student’s own words). Shows ability to think critically about the validity of the changes.

Supports main points with references, examples, and full explanations of how they apply. Thoughtfully analyzes, evaluates, and describes major points of the criteria.

Journal of Evaluation in Clinical Practice,

12

, 3, 347–352

©

2006 Blackwell Publishing Ltd

347

Blackwell Science, LtdOxford, UKJEPJournal of Evaluation in Clinical Practice1356-1294Blackwell Publishing Ltd 200512

3347352

Original Article

Grading strength of  guideline recommendationsE. Roddy
et al.

Correspondence

Edward Roddy
Academic Rheumatology
Clinical Sciences Building
Nottingham City Hospital
Hucknall Road
NG5 1PB
UK
E-mail:
edward.roddy@nottingham.ac.uk

Keywords:

clinical guidelines,
evidence-based medicine, strength of
recommendation

Accepted for publication:

27 April 2005

Evidence-based clinical guidelines: a new system to better determine
true strength of recommendation

Edward Roddy MRCP (Specialist Registrar in Rheumatology),

1

Weiya Zhang PhD (Senior Lecturer in
Musculoskeletal Epidemiology),

1

Michael Doherty MA MD FRCP (Professor of Rheumatology),

1

Nigel K. Arden MD MSc MRCP (Senior Lecturer in Rheumatology),

2

Julie Barlow PhD (Professor of Health
Psychology),

3

Fraser Birrell MA PhD MRCP (Senior Lecturer in Rheumatology),

4

Alison Carr PhD (Special Lecturer
in Musculoskeletal Epidemiology),

1

Kuntal Chakravarty FRCP (Consultant Rheumatologist),

5

John Dickson FRCP MRCGP (Community Specialist in Rheumatology),

6

Elaine Hay MD FRCP (Professor of
Community Rheumatology),

7

Gillian Hosie FRCP (General Practitioner),

8

Michael Hurley PhD (Reader in
Physiotherapy & ARC Research Fellow),

9

Kelsey M. Jordan MRCP (Rheumatology Research Fellow),

2

Christopher McCarthy PhD (Research Physiotherapist),

10

Marion McMurdo MD FRCP (Professor of Ageing and
Health),

11

Simon Mockett MPhil (Senior Lecturer),

12

Sheila O’Reilly MD MRCP (Consultant Rheumatologist),

13

George Peat PhD MCSP (Research Fellow),

7

Adrian Pendleton MD MRCP (Specialist Registrar in Rheumatology),

14

Selwyn Richards MA MSc FRCP (Consultant Rheumatologist)

15

1

Academic Rheumatology, Clinical Sciences Building, Nottingham City Hospital, Hucknall Road, Nottingham, UK

2

MRC Epidemiology Resource Centre, Southampton General Hospital, Tremona Road, Southampton, Hampshire, UK

3

Interdisciplinary Research Centre in Health, School of Health and Social Sciences, Coventry University, Priory St,
Coventry, UK

4

Musculoskeletal Research Group, University of Newcastle upon Tyne, UK

5

Haroldwood Hospital, Gubbins Lane, Romford, Essex, UK

6

Langbaurgh PCT, Langbaurgh House, Bow Street, Guisborough, Cleveland, UK

7

Primary Care Sciences Research Centre, Keele University, Staffordshire, UK

8

Primary Care Rheumatology Society, Northallerton, North Yorkshire, UK

9

King’s College London, Rehabilitation Research Unit, Dulwich Hospital, East Dulwich Grove, London, UK

10

The Centre for Rehabilitation Science, University of Manchester, Oxford Road, Manchester, UK

11

Department of Medicine, University of Dundee, Ninewells Hospital, Dundee, UK

12

Division of Physiotherapy Education, School of Community Health Sciences, University of Nottingham, Nottingham City
Hospital, Hucknall Road, Nottingham, UK

13

Derbyshire Royal Infirmary, London Road, Derby, UK

14

Craigavon Area Hospital, 68 Lurgan Road, Portadown, Co Armagh, UK

15

Poole Hospital, Longfleet Road, Poole, UK

Abstract

Rationale, aims and objectives

Clinical practice guidelines often grade the
‘strength’ of their recommendations according to the robustness of the sup-porting research evidence. The existing methodology does not allow the
strength of recommendation (SOR) to be upgraded for recommendations
for which randomized controlled trials are impractical or unethical. The pur-pose of this study was to develop a new method of determining SOR, incor-porating both research evidence and expert opinion.

Methods

A Delphi
technique was employed to produce 10 recommendations for the role of
exercise therapy in the management of osteoarthritis of the hip or knee. The
SOR for each recommendation was determined by the traditional method,
closely linked to the category of research evidence found on a systematic
literature search, and on a visual analogue scale (VAS). Recommendations
were grouped A-D according to the traditional SOR allocated and the
mean VAS calculated. Difference across the groups was assessed by one-

E. Roddy

et al.

348

©

2006 Blackwell Publishing Ltd,

Journal of Evaluation in Clinical Practice

,

12

, 3, 347–352

way

ANOVA

variance analysis.

Results

Mean VAS scores for the tradi-tional SOR groups A-D and one proposition which was ‘not recommended’
showed significant linearity on one-way

ANOVA

.  However, certain recom-mendations which, for practical reasons, could not assessed in randomized
controlled trials and therefore could not be recommended strongly by the
traditional methodology, were allocated a strong recommendation by VAS.

Conclusions

This new system of grading strength of SOR is less con-strained than the traditional methodology and offers the advantage of
allowing SOR for procedures which cannot be assessed in RCTs for prac-tical or ethical reasons to be upgraded according to expert opinion.

Introduction

Clinical guidelines have been defined as ‘systemati-cally developed statements to assist practitioner
and patient decisions about appropriate health care
for specific clinical conditions’ (Field & Lohr 1990).
Guidelines that employ an evidence-based format
currently grade each recommendation in two ways:
first, by classifying the ‘category of evidence’ and,
second, by giving a ‘strength of recommendation’.
Although several methods of producing such grades
are described, in most of these, including the method
most commonly used by clinical guidelines in rheu-matology (Pendleton

et al

.  2000; Jordan

et al

.  2003;
Dougados

et al

.  2004; Zhang

et al

.  2004; Roddy

et al

.
2005), the latter is strongly dependent on the former
(Shekelle

et al

.  1999) (Table 1). That is, the strength
of recommendation (SOR) primarily reflects the
robustness of the research evidence, with evidence
from randomized controlled trials (RCTs) and
systematic reviews automatically conferring the
strongest recommendation. However, although this
traditional method allows a downgrading of the SOR
for reasons including side effects or inconsistent
studies, it does not allow an upgrading of recommen-dations in situations where RCTs are impractical or
unethical, e.g. total joint replacement, but effective-ness is not in doubt. Furthermore, the practice of
evidence-based medicine requires the integration of
clinical expertise with the best available evidence
from systematic research (Sackett

et al

.  1996). Dur-ing the development of recent recommendations for
the role of exercise in the management of osteo-arthritis (OA) of the hip or knee (Roddy

et al

.  2005),
we found that the SOR allocated by this method was
often discordant with the consensus opinion of the

Table 1 Traditional hierarchy for category of evidence and strength of recommendation (Shekelle

et al

.  1999)
Categories of evidence

1A. meta-analysis of RCT
1B. at least one RCT
2A. at least one CT without randomization
2B. at least one type of quasi-experimental study
3.   descriptive studies (comparative, correlation, case-control)
4.   expert committee reports/opinions and/or clinical opinion of respected authorities

Strength of recommendation

A.   Directly based on category 1 evidence
B.   Directly based on category 2 evidence or extrapolated recommendation from category 1 evidence
C. Directly based on category 3 evidence or extrapolated recommendation from category 1 or 2 evidence
D.   Directly based on category 4 evidence or extrapolated recommendation from category 1, 2 or 3 evidence

RCT, randomized controlled trial; CT, controlled trial.

Grading strength of guideline recommendations

©

2006 Blackwell Publishing Ltd,

Journal of Evaluation in Clinical Practice

,

12

, 3, 347–352

349

panel. Therefore, we developed an alternative
method to better capture the true opinion of the
panel, whilst still requiring them to consider the
research evidence. We examined whether this alter-native approach affected the support afforded to
each recommendation.

Methods

A multi-disciplinary panel employed a Delphi tech-nique to produce 10 recommendations relating to
the role of exercise in the management of OA of
the hip or knee (Roddy

et al

.  2005). Following a
literature search and summary analysis of results,
the evidence for each recommendation was
assessed. The category of evidence and SOR was
assigned for each according to the method previ-ously described (Shekelle

et al

.  1999). In addition,
each participant was asked to indicate how strongly
they rated each recommendation, based not just on
research evidence but also on all aspects relating
to their knowledge and clinical opinion. This was
recorded using a 10-cm visual analogue scale (VAS)
anchored with two descriptors labelled ‘not recom-mended at all’ at the far left (0 cm) and ‘fully rec-ommended’ at the far right (10 cm). The mean VAS
and standard deviation for each recommendation
were calculated. The recommendations were then
grouped according to their original SOR (A-D) and
the mean VAS and 95% confidence interval calcu-lated for each group. A one-way

ANOVA

variance
analysis was performed to assess the difference
between the groups.

Results

The recommendations and the categories of evi-dence, SOR and VAS for each, are shown in Table 2.
Figure 1 shows the mean VAS and 95% confidence
interval for recommendation groups A, C and D in
addition to one recommendation which was contra-dicted by the research evidence and could not there-fore be graded according to the traditional method
(‘not recommended’). No recommendations were
allocated a grade B SOR. The one-way

ANOVA

vari-ance analysis identified a significant difference across
the groups (

P

<

0.001) and significant linearity
(

P

<

0.001).

Discussion

There was similarity between the SOR produced by
this method and the traditional methodology (Shek-elle

et al

.  1999). The mean VAS for each recommen-dation group (A, C, D) increased with the traditional
SOR, and therefore the category of evidence, and the
lowest mean was seen for the recommendation which
could not be recommended by the research evidence
ie was based solely on expert opinion.
This new system has the advantage of allowing the
SOR to be upgraded or downgraded based on  ex-pert opinion relating to global aspects of health
care delivery, such as generalizability, safety, cost-effectiveness and patient preference, and common
sense. It therefore gives an additional dimension and
weighting to guideline recommendations other than
just the support from research evidence alone. In the
traditional system, the term ‘strength of recommen-dation’ is almost a misnomer as it directly relates
to the category of evidence and provides little extra
information beyond that afforded by the ‘category of
evidence’. This is an important limitation of currently
practised evidence-based guideline methodology
that was overlooked in a recent critique of the meth-odology of OA guidelines (Pencharz

et al

.  2002).
During the development of guidelines there are
many situations for which the existing SOR method-ology (Shekelle

et al

.  1999) is not ideal. Interventions
for which placebo-controlled trials are impractical or
unethical (e.g. total joint replacement) cannot score
highly on the existing hierarchy and yet clearly may

Figure 1 Comparison of mean VAS (95% confidence
intervals) and traditional strength of recommendation.
VAS, visual analogue scale; NR, not recommended.
0
1
2
3
4
5
6
7
8
9
ACDNR Strength of recommendation (Traditional method)
Mean VAS (cm)
VAS = visual analogue scale, NR = not recommended

E. Roddy

et al.

350

©

2006 Blackwell Publishing Ltd,

Journal of Evaluation in Clinical Practice

,

12

, 3, 347–352

be very efficacious and warrant strong recommenda-tion for clinical practice. The new method allows
the recommendation for such interventions to be
upgraded beyond that afforded by the category of
research evidence. Furthermore, when recommenda-tions are not easily assessed in the setting of a clinical
trial yet have clear face validity, as with our third and
fourth recommendations (Table 2), the panel may
feel a much stronger recommendation is warranted
than that permitted by the current research-linked
method. For example, the mean VAS for both prop-ositions 4 and 5B (Table 2) was 7.7, yet the SOR
according to the traditional methodology were D and
1B respectively. This reflects that although proposi-tion 4 would be impractical to assess in the setting of
a RCT, it was highly supported by the expert panel

Table 2 Evidence-based recommendations for the role of exercise in the management of osteoarthritis of the hip or
knee: category of evidence, strength of recommendation (Shekelle

et al

.  1999) and visual analogue score (VAS)

Recommendation
Category of
Evidence (1–4)
Strength of
Recommendation (A-D)
Strength of
recommendation
(VAS) – Mean
(SD) cms

1. Both strengthening and aerobic exercise can
reduce pain and improve function and health
status in patients with knee and hip OA.
Knee 1B
Hip 4
A
C (extrapolated from knee OA)
8.9 (1.1)
6.3 (2.1)
2. There are few contra-indications to the
prescription of strengthening or aerobic
exercise to patients with hip or knee OA.
4C  (extrapolated from adverse
event data)
8.0 (1.5)
3. Prescription of both general (aerobic
fitness training) and local (strengthening)
exercises is an essential, core aspect of
management for every patient with hip or
knee OA.
4D 7.1 (2.5)
4. Exercise therapy for OA of the hip or knee
should be individualized and patient-centred
taking into account factors such as age,
co-morbidity and overall mobility.
4D 7.7 (1.9)
5. To be effective, exercise programmes
should include advice
and education to promote a positive lifestyle
change with an increase in physical activity.
4
1B
D
A
6.1 (2.6)
7.7 (1.4)
6. Group exercise and home exercise are equally
effective and patient preference should be
considered.
1A
4
A
D
8.0 (1.5)
7.6 (2.3)
7. Adherence is the principal predictor of
long-term outcome from exercise in patients
with knee or hip OA.
4D 5.1 (2.4)
8. Strategies to improve and maintain
adherence should be adopted, e.g. long-term
monitoring/review and inclusion of
spouse/family in exercise.
1B A 7.6 (1.5)
9. The effectiveness of exercise is independent
of the presence or severity of radiographic
findings.
4 Not recommended 4.5 (2.8)
10. Improvements in muscle strength and
proprioception gained from exercise
programmes may reduce the progression of
knee and hip OA.
4D 4.2 (2.5)

Grading strength of guideline recommendations

©

2006 Blackwell Publishing Ltd,

Journal of Evaluation in Clinical Practice

,

12

, 3, 347–352

351

whereas robust evidence from RCTs exists to sup-port proposition 5B. Finally, the traditional hierarchy
does not accommodate the scenario where research
evidence contradicts a recommendation, as with our
ninth recommendation (Table 2).
Other guideline methodology groups have
attempted to overcome these limitations and reduce
the dependence of the SOR on the category of
research evidence. However, the grading systems,
produced by American College of Cardiology/
American Heart Association (ACC/AHA) Task
Force (ACA/AHA 2004), the US Preventive Services
Task Force (2003), the National Institute for Clinical
Excellence (NICE) (NICE 2004) and the New
Zealand Guidelines Group (New Zealand Guide-lines Group 2004), derive the SOR primarily from
the category of research evidence. The ACC/AHA
guidelines state that any combination of classification
of recommendation and level of evidence is possible
and that a recommendation can be strongly sup-ported even if it is based entirely on expert opinion
and no research studies have ever been conducted on
the recommendation (ACA/AHA 2004). However,
this system does not provide for the incorporation of
factors such as cost-effectiveness and safety, and the
descriptive and quantitative criteria for assigning the
classification and evidence ratings weight research
evidence and clinical expertise equally, which may
not be appropriate for some modalities, e.g. total
joint replacement. The guideline development
methods of NICE state that when the evidence is
very strong, this should translate directly into a rec-ommendation, yet when the literature search finds no
evidence to answer the clinical question, the guide-line development group should consider using con-sensus methods to identify current best practice,
suggesting that consensus methods are only needed
when there is no robust evidence (NICE 2004).
Furthermore, NICE produces guidance on the role of
individual treatments rather than disease-orientated
recommendations on global treatment strategies. The
recently published GRADE collaboration (Atkins

et al

.  2004), although highlighting the difficulties in
producing clinical guidelines and grading strength of
recommendation, has not produced a simple, practi-cal solution. The VAS, on the other hand, has the
advantage of being simple to apply and allows all
facets to be incorporated, e.g. category of research
evidence, safety, cost-effectiveness, generalizability
and expert opinion.
A limitation of the VAS-SOR methodology is that
as the basis for the VAS is not based on explicit
criteria, it cannot be examined and assessed readily
by external groups. However, we recommend that
the VAS method should be used alongside the tra-ditional method of determining the category of
research evidence supporting each recommendation.
Any discrepancy between the category of evidence
and SOR would therefore be highlighted and should
then be justified in the ensuing discussion. A further
limitation is that this method has only been used
in the setting of recommendations for exercise in
osteoarthritis by a single group of experts, so evi-dence of its generalizability to other fields and other
groups is required.
Other possible methods for grading SOR include
the development of an ordinal scale. A numerical
scale, however, is commonly used to assess self-reported pain and disability in clinical trials, and
applying this principle to SOR seemed preferable.
Although the numerical scale scores themselves do
not have intrinsic comparability between different
sets of guidelines, there is at least scope for grading
or even ranking of different recommendations within
each set of guidelines. Other groups that prefer
verbal scales may wish to develop an ordinal scale
with descriptors to help guide practice in a clinical
setting.
Our guideline development group concludes that,
in comparison to existing traditional methodology,
this new system of grading SOR is less constrained
and offers the advantage of allowing the SOR for
procedures which cannot be assessed in RCTs to be
upgraded according to expert opinion consistent with
the principles of evidence-based medicine (Sackett

et al

.  1996). We would encourage other groups that
develop management recommendations or guide-lines to try this approach, so that its clinical applica-bility and usefulness can be determined more widely.

Acknowledgements

We  are grateful for an educational grant from
MOVE (http://www.move.uk.net) and are also
indebted to the Arthritis Research Campaign, UK
for financial support (ICAC grant D0593; WZ Senior

E. Roddy

et al.

352

©

2006 Blackwell Publishing Ltd,

Journal of Evaluation in Clinical Practice

,

12

, 3, 347–352

Lectureship D0565). We would also like to thank Dr
Jinying Lin, a visiting scholar from The People’s
Hospital of Guangxi Province, China, for assistance
with data entry.
John Dickson during the past 5 years has received
support to attend or organize symposia, or has
received a speaker’s honoraria, or a board member’s
honoraria from one or more of the following com-panies – MSD, Wyeth, Pfizer, BI, TSB Chemedica,
GSK. He has received research funding from Q-med.
He has shares in Merck and Pfizer.
Marion McMurdo is a Director of D D Develop-ments Limited, a University of Dundee company
whose mission is to provide exercise opportunities
for older people. Profits go to ageing research.

References

American College of Cardiology/American Heart Associ-ation (2004)

Manual for ACC/AHA Guideline Writing
Committees. Section II: Tools and Methods for Creating
Guidelines. Step Six: Assign Classification of Recommen-dations and Level of Evidence.

Available at:

http://www.acc.org/clinical/manual/manual_Iistep6.htm

(accessed 13 July 2004).
Atkins D., Best D., Briss P.A.

et al.

(2004) Grading quality
of evidence and strength of recommendations.

BMJ

328

,
1490.
Dougados M., Betteridge N., Burmester G.R.

et al.

(2004)
EULAR standardised operating procedures for the
elaboration, evaluation, dissemination, and implementa-tion of recommendations endorsed by the EULAR
standing committees.

Annals of the Rheumatic Disease

63

,  1172–1176.
Field M. & Lohr K. (1990)

Clinical Practice Guidelines:
Directions for a New Program

.  National Academy Press,
Washington DC.
Jordan K.M., Arden N.K., Doherty M.

et al.

(2003)
EULAR recommendations 2003: an evidence based
approach to the management of knee osteoarthritis:
report of a task force of the Standing Committee for
International Clinical Studies Including Therapeutic
Trials (ESCISIT).

Annals of the Rheumatic Disease

62

,
1145–1155.
National Institute for Clinical Excellence (2004) Guideline
development methods. Chapter 11: creating guideline
recommendations. Available at:

http://www.nice.org.uk/pdf/GDM_Chapter11.pdf

(accessed 13 July 2004).
New Zealand Guidelines Group (2004).

Grading Systems
for Guidelines.

Available at: http://www.nzgg.org.nz
(accessed 13 July 2004).
Pencharz J.N., Grigoriadis E., Jansz G.F. & Bombardier C.
(2002) A critical appraisal of clinical practice guidelines
for the treatment of lower-limb osteoarthritis.

Arthritis
Research

4

,  36–44.
Pendleton A., Arden N., Dougados M.

et al.

(2000)
EULAR recommendations for the management of knee
osteoarthritis: report of a task force of the Standing
Committee for International Clinical Studies Including
Therapeutic Trials (ESCISIT).

Annals of the Rheumatic
Disease

59

,  936–944.
Roddy E., Zhang W., Doherty M.

et al.

(2005) Evidence-based recommendations for the role of exercise in the
management of osteoarthritis of the hip or knee – the
MOVE consensus.

Rheumatology

44

,  67–73.
Sackett D.L., Rosenberg W.M., Gray J.A., Haynes R.B. &
Richardson W.S. (1996) Evidence based medicine: what
it is and what it isn’t.

BMJ

312

,  71–72.
Shekelle P.G., Woolf S.H., Eccles M. & Grimshaw J. (1999)
Clinical guidelines: developing guidelines.

BMJ

318

,
593–596.
US Preventive Services Task Force Ratings (2003)

Strength
of Recommendations and Quality of Evidence. Guide to
Clinical Preventive Services

,  3rd edn. Periodic Updates,
2000–2003. Agency for Healthcare Research and Qual-ity, Rockville, MD. Available at:

http://www.ahrq.gov/clinic/3rduspstf/ratings.htm

(accessed 13 July 2004).
Zhang W., Doherty M., Arden N.

et al.

(2004) EULAR
evidence-based recommendations for the management
of hip osteoarthritis – report of a task force of the Stand-ing Committee for International Clinical Studies Includ-ing Therapeutic Trials (ESCISIT).

Annals of the
Rheumatic Disease.

Published Online First: 7 October
2004, 10.1136/ard.2004.028886.

Roddy Analysis Worksheet

Using the worksheet below, complete an assessment of the following article.

Roddy, E., Zhang, W., Doherty, M., Arden, N. K., Barlow, J., Birrell, F., et al. (2006). Evidence-based clinical guidelines: A new system to better determine true strength of recommendation. Journal of Evaluation in Clinical Practice, 12(3), 347-352. Retrieved from http://library.gcu.edu:2048/login?url=http://search.ebscohost.com/login.aspx?direct=true&db=a9h&AN=20870724&site=ehost-live&scope=site

What was the research question?
What were the independent variables?
What was the dependent variable?
What was the sample size and how was it chosen?
What was the experimental design and use of control group?
Were the instruments of measurement shown to be reliable and valid?
What data types were included?
Describe the statistics used, what they were used for, and the results.
What were the researchers’ conclusions?  How did they answer the research question(s)?
How was error controlled?
Did you see any concerns with the research study?  If so, what?

Ganz Analysis Worksheet

Using the worksheet below, complete an assessment of the following article.

Ganz, F., Fink, N., Raanan, O., Asher, M., Bruttin, M., Nun, M., et al. (2009). ICU nurses’ oral-care practices and the current best evidence. Journal of Nursing Scholarship, 41(2), 132-138.  Retrieved from http://library.gcu.edu:2048/login?url=http://proquest.umi.com.library.gcu.edu:2048/pqdweb?did=1780947491&sid=1&Fmt=3&clientId=48377&RQT=309&VName=PQD

What was the research question?
What were the independent variables?
What was the dependent variable?
What was the sample size and how was it chosen?
What was the experimental design and use of control group?
Were the instruments of measurement shown to be reliable and valid?
What data types were included?
Describe the statistics used, what they were used for, and the results.
What were the researchers’ conclusions?  How did they answer the research question(s)?
How was error controlled?
Did you see any concerns with the research study?  If so, what?

term papers to buy
research papers