Johns Hopkins Nursing Qualitative Research Appraisal Tool

The provided document presents a student's appraisal of a qualitative research article. The student utilized the Johns Hopkins Nursing Evidence-Based Practice Appendix E Research Evidence Appraisal Tool to evaluate the chosen article. The appraisal includes an analysis of the study design, methodology, findings, and conclusions, with specific attention to the level and quality of evidence. The student assesses the article's strengths and limitations, and determines its relevance to evidence-based practice in nursing. The assignment adheres to the requirements of the assignment brief, which includes selecting a qualitative research article and providing a written critique with the completed appraisal tool as an appendix. The appraisal covers key aspects such as the research question, methods, sample, data analysis, and the overall quality of the study. The assignment is a comprehensive evaluation of a qualitative research study.

Johns Hopkins Nursing Evidence-Based Practice
Appendix E
Research Evidence Appraisal Tool
Evidence level and quality rating:
Article title: Attitudes: Mediators of the Relation between Health
and Driving in Older Adults
Number: 35 (S1)
Author(s): Holly Tuokko, Paweena Sukhawathanakul, Laura
Walzak, Alexandra Jouk, Anita Myers, Shawn Marshall, Gary
Naglie, Mark Rapoport , Brenda Vrkljan , Michelle Porter,
Malcolm Man-Son-Hing , Barbara Mazer, Nicol Korner-Bitensky,
Isabelle Gélinas, and Michel Bédard on behalf of the Candrive
II Research Team
Publication date: 05 / 15
Journal: Canadian Journal on Aging
Setting: Canadian Driving Research Initiative for Vehicular
Safety in the Elderly (Candrive) II study cohort
Sample (composition and size):
988, 70 years and more,
Canadian Driving Research
Initiative for Vehicular Safety in
the Elderly (Candrive) II study
cohort
Does this evidence address my EBP question?
Yes
No-Do not proceed with appraisal of this evidence
Is this study:
QuaNtitative (collection, analysis, and reporting of numerical data)
Measurable data (how many; how much; or how often) used to formulate facts, uncover
patterns in research, and generalize results from a larger sample population; provides
observed effects of a program, problem, or condition, measured precisely, rather than
through researcher interpretation of data. Common methods are surveys, face-to-face
structured interviews, observations, and reviews of records or documents. Statistical tests
are used in data analysis.
Go to Section I: QuaNtitative
QuaLitative (collection, analysis, and reporting of narrative data)
Rich narrative documents are used for uncovering themes; describes a problem or
condition from the point of view of those experiencing it. Common methods are focus
groups, individual interviews (unstructured or semi structured), and
participation/observations. Sample sizes are small and are determined when data
saturation is achieved. Data saturation is reached when the researcher identifies that no
new themes emerge and redundancy is occurring. Synthesis is used in data analysis.
Often a starting point for studies when little research exists; may use results to design
empirical studies. The researcher describes, analyzes, and interprets reports, descriptions,
and observations from participants.
Go to Section II: QuaLitative
Mixed methods (results reported both numerically and narratively)
1

Johns Hopkins Nursing Evidence-Based Practice
Appendix E
Research Evidence Appraisal Tool
Both quaNtitative and quaLitative methods are used in the study design. Using both
approaches, in combination, provides a better understanding of research problems than
using either approach alone. Sample sizes vary based on methods used. Data collection
involves collecting and analyzing both quaNtitative and quaLitative data in a single study
or series of studies. Interpretation is continual and can influence stages in the research
process.
Go to Section III: Mixed Methods
2

Johns Hopkins Nursing Evidence-Based Practice
Appendix E
Research Evidence Appraisal Tool
Section I: QuaNtitative
Level of Evidence (Study Design)
Is this a report of a single research study?  Yes No
Go to B
1. Was there manipulation of an independent variable?  Yes  No
2. Was there a control group?  Yes  No
3. Were study participants randomly assigned to the
intervention and control groups?  Yes  No
If Yes to questions 1, 2, and 3, this is a randomized controlled trial
(RCT) or experimental study.
LEVEL I
If Yes to questions 1 and 2 and No to question 3 or Yes to question
1 and No to questions 2 and 3, this is quasi-experimental.
(Some degree of investigator control, some manipulation of an
independent variable, lacks random assignment to groups, and may have
a control group).
LEVEL II
If No to questions 1, 2, and 3, this is nonexperimental.
(No manipulation of independent variable; can be descriptive,
comparative, or correlational; often uses secondary data).
LEVEL III
Study Findings That Help Answer the EBP Question: The findings obtained by Tukko et al.,
(2016), indicated that negative attitudes regarding one’s health or driving were the
mediators influencing one’s perceptions regarding one’s health as well as associated driving
behaviors. Interestingly, one’s perception regarding symptoms not diseases, were found to
strongly facilitate avoidance of challenging driving experiences as well as overall changes in
negative self-perceptions regarding driving.
Skip to the Appraisal of QuaNtitative Research Studies section
2
A

Johns Hopkins Nursing Evidence-Based Practice
Appendix E
Research Evidence Appraisal Tool
Section I: QuaNtitative (continued)
Is this a summary of multiple sources of research
evidence?
 Yes
Continue
 No
Use Appendix
F
1. Does it employ a comprehensive search strategy and
rigorous appraisal method?
If this study includes research, nonresearch, and
experiential evidence, it is an integrative review (see
Appendix F).
 Yes
Continue
 No
Use Appendix
F
2. For systematic reviews and systematic reviews with meta-analysis
(see descriptions below):
a. Are all studies included RCTs? LEVEL I
b. Are the studies a combination of RCTs and quasi-
experimental, or quasi-experimental only? LEVEL II
c. Are the studies a combination of RCTs, quasi-
experimental, and nonexperimental, or non-
experimental only?
LEVEL III
A systematic review employs a search strategy and a rigorous appraisal method, but
does not generate an effect size.
A meta-analysis, or systematic review with meta-analysis, combines and analyzes
results from studies to generate a new statistic: the effect size.
Study Findings That Help Answer the EBP Question
Skip to the Appraisal of Systematic Review (With or Without a Meta-Analysis) section
3
B

Johns Hopkins Nursing Evidence-Based Practice
Appendix E
Research Evidence Appraisal Tool
Appraisal of QuaNtitative Research Studies
Does the researcher identify what is known and not
known about the problem and how the study will
address any gaps in knowledge?  Yes  N
o
Was the purpose of the study clearly presented?  Yes  N
o
Was the literature review current (most sources within
the past five years or a seminal study)?  Yes  N
o
Was sample size sufficient based on study design and
rationale?  Yes  N
o
If there is a control group:
 Were the characteristics and/or demographics
similar in both the control and intervention
groups?
 Yes  N
o N/A
 If multiple settings were used, were the settings
similar?  Yes  N
o N/A
 Were all groups equally treated except for the
intervention group(s)?  Yes  N
o N/A
Are data collection methods described clearly?  Yes  N
o
Were the instruments reliable (Cronbach’s [alpha] >
0.70)?  Yes  N
o N/A
Was instrument validity discussed?  Yes  N
o N/A
If surveys or questionnaires were used, was the response
rate > 25%?  Yes  N
o N/A
Were the results presented clearly?  Yes  N
o
If tables were presented, was the narrative consistent
with the table content?  Yes  N
o N/A
Were study limitations identified and addressed?  Yes  N
o
Were conclusions based on results?  Yes  N
o
Complete the Quality Rating for QuaNtitative Studies section
4

Johns Hopkins Nursing Evidence-Based Practice
Appendix E
Research Evidence Appraisal Tool
Appraisal of Systematic Review (With or Without Meta-Analysis)
Were the variables of interest clearly identified?  Yes  No
Was the search comprehensive and reproducible?
 Key search terms stated  Yes  No
 Multiple databases searched and identified  Yes  No
 Inclusion and exclusion criteria stated  Yes  No
Was there a flow diagram that included the number of studies
eliminated at each level of review?  Yes  No
Were details of included studies presented (design, sample,
methods, results, outcomes, strengths, and limitations)?  Yes  No
Were methods for appraising the strength of evidence (level and
quality) described?  Yes  No
Were conclusions based on results?  Yes  No
 Results were interpreted  Yes  No
 Conclusions flowed logically from the interpretation and
systematic review question  Yes  No
Did the systematic review include a section addressing
limitations and how they were addressed?  Yes  No
Complete the Quality Rating for QuaNtitative Studies section (below)
Quality Rating for QuaNtitative Studies
Circle the appropriate quality rating below:
A High quality: Consistent, generalizable results; sufficient sample size for the study design;
adequate control; definitive conclusions; consistent recommendations based on
comprehensive literature review that includes thorough reference to scientific evidence.
B Good quality: Reasonably consistent results; sufficient sample size for the study design;
some control, and fairly definitive conclusions; reasonably consistent recommendations
based on fairly comprehensive literature review that includes some reference to scientific
evidence.
C Low quality or major flaws: Little evidence with inconsistent results; insufficient sample
size for the study design; conclusions cannot be drawn.
5

Johns Hopkins Nursing Evidence-Based Practice
Appendix E
Research Evidence Appraisal Tool
Section II: QuaLitative
Level of Evidence (Study Design)
Is this a report of a single research study?  Yes
this is
Level III
 No
go to II B
Study Findings That Help Answer the EBP Question
Complete the Appraisal of Single QuaLitative Research Study section (below)
Appraisal of a Single QuaLitative Research Study
Was there a clearly identifiable and articulated:
 Purpose? ❑ Yes ❑ No
 Research question? ❑ Yes ❑ No
 Justification for method(s) used? ❑ Yes ❑ No
 Phenomenon that is the focus of the research? ❑ Yes ❑ No
Were study sample participants representative? ❑ Yes ❑ No
Did they have knowledge of or experience with the research area? ❑ Yes ❑ No
Were participant characteristics described? ❑ Yes ❑ No
Was sampling adequate, as evidenced by achieving saturation of data? ❑ Yes ❑ No
Data analysis:
 Was a verification process used in every step by checking and
confirming with participants the trustworthiness of analysis and
interpretation?
❑ Yes ❑ No
 Was there a description of how data were analyzed (i.e.,
method), by computer or manually?
❑ Yes ❑ No
Do findings support the narrative data (quotes)? ❑ Yes ❑ No
Do findings flow from research question to data collected to analysis
undertaken? ❑ Yes ❑ No
Are conclusions clearly explained? ❑ Yes ❑ No
Skip to the Quality Rating for QuaLitative Studies section
1
A

Johns Hopkins Nursing Evidence-Based Practice
Appendix E
Research Evidence Appraisal Tool
For summaries of multiple quaLitative research
studies (meta-synthesis), was a comprehensive
search strategy and rigorous appraisal method
used?
Yes
Level III
No
go to Appendix F
Study Findings That Help Answer the EBP Question
Complete the Appraisal of Meta-Synthesis Studies section (below)
Appraisal of Meta-Synthesis Studies
Were the search strategy and criteria for selecting primary studies clearly
defined? ❑ Yes ❑ No
Were findings appropriate and convincing? ❑ Yes ❑ No
Was a description of methods used to:
 Compare findings from each study? ❑ Yes ❑ No
 Interpret data? ❑ Yes ❑ No
Did synthesis reflect: ❑ Yes ❑ No
 New insights? ❑ Yes ❑ No
 Discovery of essential features of phenomena? ❑ Yes ❑ No
 A fuller understanding of the phenomena? ❑ Yes ❑ No
Was sufficient data presented to support the interpretations? ❑ Yes ❑ No
Complete the Quality Rating for QuaLititative Studies section (below)
2
B

Johns Hopkins Nursing Evidence-Based Practice
Appendix E
Research Evidence Appraisal Tool
Quality Rating for QuaLitative Studies
Circle the appropriate quality rating below:
No commonly agreed-on principles exist for judging the quality of quaLitative studies. It is a
subjective process based on the extent to which study data contributes to synthesis and
how much information is known about the researchers’ efforts to meet the appraisal criteria.
For meta-synthesis, there is preliminary agreement that quality assessments should be
made before synthesis to screen out poor-quality studies1.
A/B High/Good quality is used for single studies and meta-syntheses2.
The report discusses efforts to enhance or evaluate the quality of the data and the overall
inquiry in sufficient detail; and it describes the specific techniques used to enhance the
quality of the inquiry.
Evidence of some or all of the following is found in the report:
 Transparency: Describes how information was documented to justify decisions, how
data were reviewed by others, and how themes and categories were formulated.
 Diligence: Reads and rereads data to check interpretations; seeks opportunity to find
multiple sources to corroborate evidence.
 Verification: The process of checking, confirming, and ensuring methodologic
coherence.
 Self-reflection and self-scrutiny: Being continuously aware of how a researcher’s
experiences, background, or prejudices might shape and bias analysis and
interpretations.
 Participant-driven inquiry: Participants shape the scope and breadth of questions;
analysis and interpretation give voice to those who participated.
 Insightful interpretation: Data and knowledge are linked in meaningful ways to
relevant literature.
C Lower-quality studies contribute little to the overall review of findings and have few, if
any, of the features listed for High/Good quality.
1 https://www.york.ac.uk/crd/SysRev/!SSL!/WebHelp/6_4_ASSESSMENT_OF_QUALITATIVE_RESEARCH.htm
2 Adapted from Polit & Beck (2017).
3

Johns Hopkins Nursing Evidence-Based Practice
Appendix E
Research Evidence Appraisal Tool
Section III: Mixed Methods
Level of Evidence (Study Design)
You will need to appraise both the quaNtitative and quaLitative parts of the study
independently, before appraising the study in its entirety.
1.Evaluate the quaNitative part of the study using Section I. Level Quality
Insert here the level of evidence and overall quality for this part:
2.Evaluate the quaLitative part of the study using Section II. Level Quality
Insert here the level of evidence and overall quality for this part:
3.To determine the level of evidence, circle the appropriate study design:
 Explanatory sequential designs collect quaNtitative data first, followed by the quaLitative
data; and their purpose is to explain quaNtitative results using quaLitative findings. The level
is determined based on the level of the quaNtitative part.
 Exploratory sequential designs collect quaLitative data first, followed by the quaNtitative
data; and their purpose is to explain quaLitative findings using the quaNtitative results. The
level is determined based on the level of the quaLitative part, and it is always Level III.
 Convergent parallel designs collect the quaLitative and quaNtitative data concurrently for
the purpose of providing a more complete understanding of a phenomenon by merging both
datasets. These designs are Level III.
 Multiphasic designs collect quaLitative and quaNtitative data over more than one phase,
with each phase informing the next phase. These designs are Level III.
Study Findings That Help Answer the EBP Question
Complete the Appraisal of Mixed Methods Studies section (below)
4

Johns Hopkins Nursing Evidence-Based Practice
Appendix E
Research Evidence Appraisal Tool
3 National Collaborating Centre for Methods and Tools. (2015). Appraising Qualitative, Quantitative, and Mixed Methods Studies included in Mixed Studies Reviews: The MMAT. Hamilton,
ON: McMaster University. (Updated 20 July, 2015) Retrieved from http://www.nccmt.ca/ resources/search/232
5
Appraisal of Mixed Methods Studies3
Was the mixed-methods research design relevant to address the
quaNtitative and quaLitative research questions (or objectives)? ❑ Yes ❑ No ❑ N/A
Was the research design relevant to address the quaNtitative and
quaLitative aspects of the mixed-methods question (or objective)? ❑ Yes ❑ No ❑ N/A
For convergent parallel designs, was the integration of quaNtitative
and quaLitative data (or results) relevant to address the research
question or objective?
❑ Yes ❑ No ❑ N/A
For convergent parallel designs, were the limitations associated with
the integration (for example, the divergence of quaLitative and
quaNtitative data or results) sufficiently addressed?
❑ Yes ❑ No ❑ N/A
Complete the Quality Rating for Mixed-Method Studies section (below)
Quality Rating for Mixed-Methods Studies
Circle the appropriate quality rating below
A High quality: Contains high-quality quaNtitative and quaLitative study components;
highly relevant study design; relevant integration of data or results; and careful
consideration of the limitations of the chosen approach.
B Good quality: Contains good-quality quaNtitative and quaLitative study components;
relevant study design; moderately relevant integration of data or results; and some
discussion of limitations of integration.
C Low quality or major flaws: Contains low quality quaNtitative and quaLitative study
components; study design not relevant to research questions or objectives; poorly
integrated data or results; and no consideration of limits of integration.