Quantitative Research Article Appraisal: SMS Reminders in Healthcare

This report provides a critical appraisal of a quantitative research article by Moran, O’Loughin, and Kelly (2018), which investigated the effects of SMS reminders on attendance at a community adult mental health service clinic. The study, conducted in Dublin, Ireland, analyzed outpatient data before and after the introduction of SMS reminders. The researchers used a purposive sampling approach, collecting data from over 6,000 appointments across three phases. The appraisal highlights the study's strengths, such as the large sample size and the comparative analysis of attendance rates. However, it also identifies a limitation: the lack of data consistency to determine the impact of SMS reminders. The findings revealed that while SMS reminders did not directly increase attendance, they provided patients with an opportunity to cancel appointments, optimizing service delivery. The report concludes that the study's findings support the use of reminders to improve the efficiency of mental health services by allowing practitioners to capitalize on available appointment slots. The appraisal includes an evaluation of the study's level of evidence, quality rating, and a detailed analysis of its methodology, results, and limitations, based on the Johns Hopkins Nursing Evidence-Based Practice Appendix E Research Evidence Appraisal Tool.

Running head: QUANTITATIVE RESEARCH ARTICLE APPRAISAL 1
Quantitative Research Article Appraisal
Name of Student
Institutional Affiliation

QUANTITATIVE RESEARCH ARTICLE APPRAISAL 2
Quantitative Research Article Appraisal
Evidence from research has been used to ascertain specific concepts in healthcare. Findings from
evidence-based analyses have provided critical insights and decision baseline when creating strategic measures
in healthcare delivery. However, it is important to evaluate the reliability and validity of the findings by
conducting a comprehensive appraisal of the research. This excerpt provides a critical appraisal of research by
Moran, O’Loughin, and Kelly (2018) on the effects of SMS reminders on the clinic attendance at a local adult
mental health facility.
Moran, O’Loughin, and Kelly (2018) collected data from tow healthcare facilities dealing with mental
health services in Dublin, Ireland. The data was obtained from Ballyfermot and Lucan Community Adult
Mental Health Service. The outpatient data were obtained before the introduction of SMS reminders, after the
introduction of the reminders, and after two and a half years later. The approach adopted for participant
sampling was purposive since the researchers focused on appointments to adult mental health services. Data
from obtained from 2170, 2092, and 2474 appointments for the first, second, and third phases respectively.
Moran, O’Loughin, and Kelly (2018) worked with a large data sample, which is associated with a high level of
reliability. The study compared the appointments issued and the number of attendance and non-attendance.
There were no questionnaires used in this study.
Moran, O’Loughin, and Kelly (2018) has a comprehensive ethics consideration framework because they
first sought the authorization to use the mental health service data in these facilities. By focusing on the
attendance rate and not on patients’ private data, the research met the ethical requirements.
Moran, O’Loughin, and Kelly (2018) adopted a quantitative approach to analyze and interpret the
findings. The scholars used a comparative approach to determine whether there was a difference between the
attendance rate before the introduction of the SMS reminders and when this approach was implemented. The
scholars further compared the experimental and control groups to ascertain the role and implication of SMS
reminders. The analysis approach was effective because it included the period before and after the SMS

QUANTITATIVE RESEARCH ARTICLE APPRAISAL 3
reminder was introduced. At the same time, there was an experimental and control group, which enhances the
validity of the findings.
In this study, there is one major limitation. The study by Moran, O’Loughin, and Kelly (2018) lacked
data consistency to determine whether indeed SMS reminders had an impact on attendance. There is no clear
evidence that shows the study used data from the same group of patients across the entire two and a half years
period.
The scholars found out that before the introduction of SMS reminders, there was a 22.2% non-
attendance, which reduced to 13.9% within the six months. At the same time, attendance cancelation through
SMS stood at 73%. There was no difference between the group that received reminders and those that did not in
terms of the rate of non-attendance. In this case, the introduction of the reminders offered the patients an
opportunity to cancel their appointment in time to enable the hospital to capitalize on created chances to
maximize service delivery. It was possible to show the lack of a relationship between SMS reminders and
attendance rates because of the method of data analysis adopted in this study.
From the findings of this study, it is evident that reminders do not encourage attendance but offers the
opportunity for those who are not going to honor appointments to inform the hospital, which allows for more
appointments to be made. In this case, the findings ascertain how reminders could be used to optimize adult
mental health services delivery by allowing practitioners to capitalize on each available appointment
opportunity.

QUANTITATIVE RESEARCH ARTICLE APPRAISAL 4
References
Moran, L., O’Loughin, K., & Kelly, B. D. (2018). The effect of SMS (text message) reminders on attendance at
a community adult mental health service clinic: Do SMS reminders really increase attendance? Ir J Med
Sci., 187(3), 561 – 564. DOI: 10.1007/s11845-017-1710-0.

QUANTITATIVE RESEARCH ARTICLE APPRAISAL 5
Appendix
Evidence level and quality rating:
Article title: The effect of SMS (text message) reminders on
attendance at a community adult mental health service clinic: Do
SMS reminders really increase attendance?
Number:187(3), 561 – 564. DOI:
10.1007/s11845-017-1710-0.
Author(s): Moran, L., O’Loughin, K., & Kelly, B. D. Publication date: 2018
Journal: Ir J Med Sci.,
Setting: Mental Health Services Sample (composition and size):
2170, 2092, and 2474 for the
three phases
Does this evidence address my EBP question?
Yes
No-Do not proceed with appraisal of this evidence
Is this study:
QuaNtitative (collection, analysis, and reporting of numerical data)
Measurable data (how many; how much; or how often) used to formulate facts, uncover patterns in
research, and generalize results from a larger sample population; provides observed effects of a
program, problem, or condition, measured precisely, rather than through researcher interpretation of
data. Common methods are surveys, face-to-face structured interviews, observations, and reviews of
records or documents. Statistical tests are used in data analysis.
Go to Section I: QuaNtitative
QuaLitative (collection, analysis, and reporting of narrative data)
Rich narrative documents are used for uncovering themes; describes a problem or
condition from the point of view of those experiencing it. Common methods are focus
groups, individual interviews (unstructured or semi structured), and
participation/observations. Sample sizes are small and are determined when data
saturation is achieved. Data saturation is reached when the researcher identifies that no
new themes emerge and redundancy is occurring. Synthesis is used in data analysis.
Often a starting point for studies when little research exists; may use results to design
empirical studies. The researcher describes, analyzes, and interprets reports, descriptions,
and observations from participants.
Go to Section II: QuaLitative
Mixed methods (results reported both numerically and narratively)
Both quaNtitative and quaLitative methods are used in the study design. Using both
approaches, in combination, provides a better understanding of research problems than
using either approach alone. Sample sizes vary based on methods used. Data collection
involves collecting and analyzing both quaNtitative and quaLitative data in a single study
f

QUANTITATIVE RESEARCH ARTICLE APPRAISAL 6
or series of studies. Interpretation is continual and can influence stages in the research
process.
Go to Section III: Mixed Methods
Section I: QuaNtitative
Level of Evidence (Study Design)
Is this a report of a single research study?  Yes No
Go to B
1. Was there manipulation of an independent variable?  Yes  No
2. Was there a control group?  Yes  No
3. Were study participants randomly assigned to the
intervention and control groups?  Yes  No
If Yes to questions 1, 2, and 3, this is a randomized controlled trial
(RCT) or experimental study.
LEVEL I
If Yes to questions 1 and 2 and No to question 3 or Yes to question
1 and No to questions 2 and 3, this is quasi-experimental.
(Some degree of investigator control, some manipulation of an
independent variable, lacks random assignment to groups, and may have
a control group).
LEVEL II
If No to questions 1, 2, and 3, this is nonexperimental.
(No manipulation of independent variable; can be descriptive,
comparative, or correlational; often uses secondary data).
LEVEL III
Study Findings That Help Answer the EBP Question:
There was no difference between the group that received reminders and those that did not in terms of
the rate of non-attendance. In this case, the introduction of the reminders offered the patients an
opportunity to cancel their appointment in time to enable the hospital to capitalize on created chances
to maximize service delivery.
A

QUANTITATIVE RESEARCH ARTICLE APPRAISAL 7
Skip to the Appraisal of QuaNtitative Research Studies section
Section I: QuaNtitative (continued)
Is this a summary of multiple sources of research evidence?  Yes
Continue
 No
Use Appendix
F
1. Does it employ a comprehensive search strategy and rigorous
appraisal method?
If this study includes research, nonresearch, and experiential
evidence, it is an integrative review (see Appendix F).
 Yes
Continue
 No
Use Appendix
F
2. For systematic reviews and systematic reviews with meta-analysis
(see descriptions below):
a. Are all studies included RCTs? LEVEL I
b. Are the studies a combination of RCTs and quasi-
experimental, or quasi-experimental only?
LEVEL II
c. Are the studies a combination of RCTs, quasi-
experimental, and nonexperimental, or non-
experimental only?
LEVEL III
A systematic review employs a search strategy and a rigorous appraisal method, but does not generate
an effect size.
A meta-analysis, or systematic review with meta-analysis, combines and analyzes results from studies
to generate a new statistic: the effect size.

QUANTITATIVE RESEARCH ARTICLE APPRAISAL 8
Study Findings That Help Answer the EBP Question
Skip to the Appraisal of Systematic Review (With or Without a Meta-Analysis) section

QUANTITATIVE RESEARCH ARTICLE APPRAISAL

QUANTITATIVE RESEARCH ARTICLE APPRAISAL
Appraisal of QuaNtitative Research Studies
Does the researcher identify what is known and not known about
the problem and how the study will address any gaps in
knowledge?
 Yes  N
o
Was the purpose of the study clearly presented?
 Yes  N
o
Was the literature review current (most sources within the past
five years or a seminal study)?  Yes  N
o
Was sample size sufficient based on study design and rationale?
 Yes  N
o
If there is a control group:
 Were the characteristics and/or demographics
similar in both the control and intervention
groups?
 Yes  N
o N/A
 If multiple settings were used, were the settings
similar?  Yes  N
o N/A
 Were all groups equally treated except for the
intervention group(s)?  Yes  N
o N/A
Are data collection methods described clearly?
 Yes  N
o
Were the instruments reliable (Cronbach’s [alpha] > 0.70)?
 Yes  N
o N/A
Was instrument validity discussed?
 Yes  N
o N/A
If surveys or questionnaires were used, was the response
rate > 25%?  Yes  N
o N/A
Were the results presented clearly?
 Yes  N
o
If tables were presented, was the narrative consistent with the
table content?  Yes  N
o N/A
Were study limitations identified and addressed?
 Yes  N
o
Were conclusions based on results?
 Yes  N
o
Complete the Quality Rating for QuaNtitative Studies section

QUANTITATIVE RESEARCH ARTICLE APPRAISAL
 The Johns Hopkins Hospital/ The Johns Hopkins University
Quality Rating for QuaNtitative Studies
Circle the appropriate quality rating below:
A High quality: Consistent, generalizable results; sufficient sample size for the study design; adequate
control; definitive conclusions; consistent recommendations based on comprehensive literature review
that includes thorough reference to scientific evidence.
B Good quality: Reasonably consistent results; sufficient sample size for the study design; some control,
and fairly definitive conclusions; reasonably consistent recommendations based on fairly comprehensive
literature review that includes some reference to scientific evidence.
C Low quality or major flaws: Little evidence with inconsistent results; insufficient sample size for the
study design; conclusions cannot be drawn.

Running head: QUANTITATIVE RESEARCH ARTICLE APPRAISAL 1