Critical Appraisal of Two Journal Articles on Mental Health and Evidence Informed Decision Making

This assignment critically appraises two journal articles on mental health and evidence informed decision making. The appraisal considers sample size, research design, ethical concerns, data collection methods, and statistical analysis. The study findings, limitations, and implications for nursing practice are also discussed.

Introduction
This assignment will critically appraise two journal article; one authored by Savic et al. (2013)
on “Separation from family and its impact on the mental health of Sudanese refugees in
Australia.” The second article by Liang et al. (2017) titled “assessing the Competence of
Evidence Informed decision-Making amongst health Service Managers.” The Critical appraisal
will majorly deliberate on the quality of this research articles considering the steps undertaken by
the researchers during the entire research process. The areas to be appraised include sample size,
research design, ethical concerns, data collection methods, statistical analysis. At the same time,
the study findings, the study limitation, and the implication the research findings has on the
nursing practice field will be critically appraised.
Tool selection
Critical Appraisals Skills Programme (CASP) tool was the best choice for critical appraisal of
article 1by Savic et al. (2013) on “Separation from family and its impact on the mental health of
Sudanese refugees in Australia” which is qualitative research. CASP is an international group
that promotes evidenced based approach in health and social care issues. Their tools have check-
list that is critical in enabling individuals make sense of research evidence and application in
practice (Critical Appraisal Skills Programme, 2018). According to Critical Appraisal Skills
Programme (2018), critical appraisal skills enhances an individual’s ability to systematically
determine the trustworthiness, validity of the results published article. Therefore, based on this
organizations unrivalled expertise, their appraisal tool is the best to enable me achieve the
assignment objective.
For the quantitative research article appraisal tool was downloaded from Center for Evidence
Based Management website. This organization provides more information and tools on evidence
based practice (Center for Evidence Based Management, 2018).
Critical appraisal of article 1
The aim of this study has been clearly started by Savic et al. (2013). The study aim was to
elucidate on the perceived impacts of separation on the mental health and resettlement of
Sudanese refugees in Australia and how the refugees cope up with the potential stressor (Page,
1

383-384). The authors have highlighted the existing gaps in knowledge by contextualizing the
research problem through past and current literature review, hence, justifying the study.
The authors of this article conducted a literature review by considering different research work
focusing on the Sudanese community profile, well-being of Sudanese refugee, and determinants
of refugee’s mental health. Though, a big proportion of the reviewed literature were valid, a few
of the researched journal articles were out dated. Therefore, the authors of this article ought to
have adhered to the 10 year time limit for researching an article. The background information of
the research problem have been provided by the authors through thorough literature review.
The researchers of this study adopted a qualitative approach methodology where by key
informants who are well conversant with Sudanese refugee’s affairs were interviewed. The
choice of using the qualitative approach was appropriate for addressing the research objectives.
According to Patten and Newhart (2017), in qualitative research the researchers identifies themes
through interviewing the study participants and the results are analyzed and presented in words.
Furthermore, the qualitative approach has the advantage of providing insight on the meaning of
particular event (Green and Thorogood 2018). This study used phenomenological analysis
approach, the authors focused on various key informants experiences with Sudanese refugee
resettlements in Australia as suggested in this article (p. 384). However, it is worth noting that
the researchers didn’t describe their theoretical perspective of this study, therefore, the researcher
didn’t justify the use of the research design in the study.
The authors have clearly started that 20 key informants were purposively sampled due to their
familiarity with issues around Sudanese refugee mental health (p.384). The purposive sampling
method was adequate for this study, this was to ensure that only persons who could provide the
right information were included in the study. However, the use of purposive sampling method
which is a non-probability method and a small sample size can compromise the research rigour
and validity by introducing selection bias and lower statistical power respectively. As a result the
findings of this study can be questioned and limit its findings being generalized to the general
population.
The sample size of the study participants was 20. The smaller sample size in a qualitative
research guarantees the data quality, and the participants selection was informed by their
knowledge about the topic. According to Chow et al. (2017) using a small sample size one has
2

the opportunity of achieving rich and quality data. The same opinion has been propagated by
Mason (2017), he suggested that when conducting nursing or public health research using
qualitative approach a small sample size would be appropriate. To maintain trustworthiness and
rigour of the research results, a process of constant comparison was adopted and 25 interviews
were conducted until a point of saturation was reached and no new themes emerged (p.384). It is
evident that triangulation was used to maintain rigour. This was achieved attaining data through
interviewing different sources of key informants including six were mental health care providers,
five were primary health care providers, four were health service managers, three were social
workers, and two were policy makers, and the study participants were persons over 18 years of
age (p.384). Therefore, the recruitment strategy of the study participants was appropriate to the
study objective.
The data collected in this study was adequately to address the research question. This study has
clearly started that 25 semi-structured interviews were conducted with key informants in a span
of one year, in-depth-interviews was conducted and recoded by the first author. Furthermore, the
authors of this study have highlighted the questions that were used to guide the interview
(p.384). Therefore, the method used to collect data was explicit. According to Torrance (2012),
an aspect of maintained triangulation and reflexivity ensures good quality of qualitative research
article. For example, there was use of audit trail and the incorporation of divergent viewpoints
about the emerging themes leading to enhanced triangulation and reflexivity in the study (P.384).
A collaborative research with different specialties is core to the quality of any research. In fact,
Parahoo et al. (2017), in his work about the evidenced-based practices overwhelmingly
recognizes the benefits of collaboration between researchers when conducting research.
However, the authors of this article failed to justify the study setting for this research, reasons for
choosing this method of data collection was also not substantiated, and failure to clarify if there
was any modification of the chosen method for data collection is another failure. There is luck of
respondent validation in this study which has a negative impact on its quality. Respondent
validation is an important component of qualitative research because it provides rigour and
validity to the research and it can be achieved by giving feedback to the study participants and
accessing their opinions on interpretations (Torrance 2012; Hannes 2011).
3

In this study a Framework approach to thematic analysis was used to analyses the data that was
collected (P, 384). According to Alhojailan (2012), thematic analysis is a comprehensive process
that enable researchers to categorize several cross-references between emerging themes from
research data. Thematic analysis provides flexibility in the research through inductive and
deductive ways (Guest et al. 2011; Vaismoradi et al. 2013). This makes the thematic analysis the
best choice to analyze data when the research objective is to obtain information to determine a
relationship between independent variable and dependent variable in a qualitative research.
Therefore, this method was appropriate for this type of study.
Critical appraisal of article 2
This study strives to assess the level of competency among mid-level health service managers in
making decisions based on evidence informed decision making (EIDM). Therefore, tittle of this
article is concise and it reflects the main theme of this research. According to Jack et al. (2010),
the general tittle of journal article should provide the fast perception into the intent of the
research. The abstract is succinct and describe the study by introducing the purpose of doing this
study, study setting, highlighting the study findings and a brief conclusion. However, the results
reflected in the abstract contains only wordings it would have been good if the results were
descriptive by expressing statistic parameters and the level of significance . At the same time, it
is not necessary to have abbreviations in the abstract this is because the word count in abstract is
limited and it should be succinct. The main purpose of an article’s abstract according to Parahoo
(2017), is to summarize the journal articles contents and express the main points to readers. The
authors of this study have clearly started the research questions this is a strength that is worth
mentioning
Majority (15 out of 25) of the references used in this article are out dated since they were
published more than 10 years from the time of publication of this article. Therefore, this brings
into focus the reliability of the literature used, hence, some of the information contained in the
old literature can be rendered obsolete. According to Hart (2018), the source of information used
in the research process shouldn’t be more than 10 years old from the time the articles
publication.
In quantitative research a characteristic of a study sample being a representative of the group
from which it has been drawn from cannot be compromised. Therefore, the study sample is a
4

typical of the wider target population to whom the research findings might be generalized (Jack
et al. 2010). In order to attain a threshold of a sample being a representative of a group from
which it is drawn the researcher is required to randomly select the samples and have a large
sample size. Samples selected using non-probability sampling methods can produce findings that
are bias and lack validity due to lack of randomization and spread of confounders (Noble and
Smith 2015). In this study it is clear that the samples were selected using non-probability
sampling method at the same time the sample size used was small (p.21). Therefore, the two
factors combined raises the issue of validity, accuracy, and statistical power of this research
findings and the implication it has in the inferential statistics. However, the authors have cited
the two factors as limitation of this study (P.21).
The data about the manager’s competency in applying EIDM was obtained through
360°subjective assessment and a case study based objective assessment (P.17). Furthermore, the
authors have clearly started the process of filling data collection instrument and behavioral items
validation. Therefore, methods used by the researchers in obtaining the data was sufficient for
the study objective. There is a failure by the researchers of this study to clearly describe both
inclusion and exclusion criteria for the study participants this can contribute to selection bias.
The authors of this article chose SPSS software for the data analysis. SPSS being one of the most
common user friendly software employed currently for analysis of different kinds of data with
guaranteed accuracy as compared to manual calculations. The statistical instruments used by the
authors included a 360° subjective assessment and a case study based objective assessment. The
authors have described into details the assessment instruments were developed and how the 12
behavioral study items used to probe the study participants were validated through thorough
literature review and rigorous consultation with and contribution from management experts from
various health services in Victoria and focus group discussions. The authors used Univariate
analyses for both individuals and groups and t-test was the method of choice. This method of
data analysis was adequate to produce the intended the mean results showing statistical
significance difference of behavioral items. One-way analysis of variance is appropriate for
analyzing the statistical mean difference within the group and Two-way analysis of variance is
good to analyses statistical mean difference between two groups. According to Lakens (2013),
one way- ANOVA enables researchers to differentiate variability caused by individual’s
5

differences from variability caused by effect within-subject design and it has high statistical
power compared to between-subject ANOVA. Therefore, the results produced using this data
analysis method was sufficient to provide answers to the research questions in this study and the
results generalized to the entire population. However, the researcher didn’t provide look into the
demographic characteristics of the study participants including age and sex this are potential
confounding factors or sources errors that could put into question the rigour and validity of the
findings of this research. The authors have used simple and easily understandable tables to
present their results.
The authors have written their conclusion based on the study findings and highlighted the impact
it has in improving EIDM practice among the health service managers. However, the authors
have fall short of making recommendations for feature studies based on their research findings
and limitations.
Ethical issues
The University of Adelaide Human Research Ethics Committee provided the ethical clearance
for the research conducted by Savic et al. (2013) on “Separation from family and its impact on
the mental health of Sudanese refugees in Australia. At the same time, all the study participants
were persons above the age of 18 years (p, 384). However, there is failure by the authors to give
information about the study participants consenting to take part in this research by signing
consent form raising fundamental issue about ethical considerations. According to the guidelines
by American Psychological Association (2015), there are certain exceptions in research whereby
it is mandatory to get signed informed consent from the study participants such research include
where the researchers use anonymous questionnaires or simple naturalistic observations and in
the mentioned situations the study respondents cannot be personally recognized or harm caused
to him/her in any way possible. At the same time, archival research that depends on already
published or publicly available data also don’t require direct informed consent. However, all
other forms of conducted research demands that an informed consent be obtained from the
recruited study respondents before the commencement of the study. Therefore, the research
conducted by Savic et al. (2013) is not an exception based on that they out have mentioned hoe
that ethical consideration was achieved.
6

The authors of the second article by Liang et al. (2017) titled “assessing the Competence of
Evidence Informed decision-Making amongst health Service Managers.” Obtained ethical
approval from La Trobe University before the commencement of their research (p.18). However,
it is not clear if a consent was obtained directly from the study participants before they took part
in the study raising ethical concerns. The research conducted by Liang et al. (2017), doesn’t meet
the criteria for exception of obtaining informed consent from the study participants as mentioned
in the case of the first article. This is because the study participants were known to the
researchers, hence, it was necessary for them to sign consent forms.
Summary of the appraised papers
The basis for analysis of an article is to recognize the problem being addressed, the rationale for
the research, proposed hypothesis, and substance of past and current research on the area of
interest leading to sensible and statistically reliable conclusion. The theoretical context and
research questions should be cited clearly and the sample size should large enough and a
representative of the entire population to provide valid results. Furthermore, the studies should
be conducted with ethical considerations. The two studies have full filled some of these
conditions. However, certain condition were not met including participants signing consent
forms, small sample size, and participant’s validation was lucking. For example, in article
published by Liang et al. (2017) titled “assessing the Competence of Evidence Informed
decision-Making amongst health Service Managers.” This has been well discussed by the authors
they have recognized the fact that the sample size of their study was small, and there was luck of
sample randomization during sampling. All the two discussed limitations are potential sources of
bias leading to lack of rigour and validity of the research findings. The key findings of both
research findings and their implication on the study area were well articulated by the authors of
the two articles. Furthermore, they have made certain recommendations for the feature research.
For example, Liang et al. (2017) recommended that a further research should be carried out in a
larger and more diverse population. At the same time, Savic et al. (2013), recommended further
research on impact of separations together with strategies used by the refugees to cope. Such
argument appreciates the fact that rigour and validity of research findings are critical for
generalization to the general population. In conclusion, the two research articles met the basic
thresholds for a good qualitative and quantitative research.
7

References
Alhojailan, M.I., 2012. Thematic analysis: A critical review of its process and evaluation. West
East Journal of Social Sciences, 1(1), pp.39-47.
American Psychological Association, 2015. Guidelines for clinical supervision in health service
psychology. The American Psychologist, 70(1), p.33.
Brown, C. J. (2009). Self-renewal in nursing leadership: The lived experience of caring for self,
Journal of Holistic Nursing, 27(2), 75-84.
CASP - Critical Appraisal Skills Programme. 2018. CASP Checklists - CASP - Critical
Appraisal Skills Programme. [ONLINE] Available at: https://casp-uk.net/casp-tools-checklists/.
Center for Evidence Based Management. 2018. Center for Evidence Based Management.
[ONLINE] Available at: https://www.cebma.org. [Accessed 15 September 2018] [Accessed 13
September 2018].
Chow, S.C., Shao, J., Wang, H. and Lokhnygina, Y., 2017. Sample size calculations in clinical
research. Chapman and Hall/CRC.
Green, J. and Thorogood, N., 2018. Qualitative methods for health research. Sage.
Guest, G., MacQueen, K.M. and Namey, E.E., 2011. Applied thematic analysis. sage.
Hannes, K., 2011. Critical appraisal of qualitative research.
Hart, C., 2018. Doing a Literature Review: Releasing the Research Imagination. Sage.
Jack, L., Hayes, S. C., Scharalda, J. G., Stetson, B., Jones-Jack, N. H., Valliere, M., LeBlanc. C.,
2010. Appraising Quantitative Research in Health Education: Guidelines for Public Health
Educators. Health Promotion Practice, 11(2), 161–165.
8

Lakens, D., 2013. Calculating and reporting effect sizes to facilitate cumulative science: a
practical primer for t-tests and ANOVAs. Frontiers in psychology, 4, p.863.
Mason, J., 2017. Qualitative researching. Sage.
Nemcek, M. A. (2007). Registered Nurses’ Self-Nurturance and Life and Career Satisfaction,
American Association of Occupational Health Nurses Journal, 55(8), 305-310.
Noble, H. and Smith, J., 2015. Issues of validity and reliability in qualitative research. Evidence-
Based Nursing, pp.ebnurs-2015.
Parahoo, K., 2017. Evidence-Based Practice: misconceptions and challenges.
Patten, M.L. and Newhart, M., 2017. Understanding research methods: An overview of the
essentials. Taylor & Francis.
Savic, M., Chur‐Hansen, A., Mahmood, M.A. and Moore, V., 2013. Separation from family and
its impact on the mental health of Sudanese refugees in Australia: a qualitative study. Australian
and New Zealand journal of public health, 37(4), pp.383-388.
Torrance, H., 2012. Triangulation, respondent validation, and democratic participation in mixed
methods research. Journal of mixed methods research, 6(2), pp.111-123.
Vaismoradi, M., Turunen, H. and Bondas, T., 2013. Content analysis and thematic analysis:
Implications for conducting a qualitative descriptive study. Nursing & health sciences, 15(3),
pp.398-405.
9