Critical Appraisal of Research Methodology

This article provides a critical appraisal of research methodology used in two articles in the field of health and human services management. The article highlights the tools selected for the appraisal, ethical issues, and the strengths and weaknesses of the methodologies used.

Running head: CRITICAL APPRAISAL OF RESEARCH METHODOLOGY
1
Critical appraisal of research methodology
Name of Student
Institutional Affiliation
Name of Professor
Date

CRITICAL APPRAISAL OF RESEARCH METHODOLOGY
2
Critical appraisal of research methodology
Introduction
Critical appraisal is a systematic process of assessing the outcomes of a scientific
research study based on evidence to determine the relevance, value, and its trustworthiness in a
particular field of study (Zeng et al., 2015, pp.2-10). Additionally, it seeks to evaluate the way
studies are conducted against the factors like the internal validity and the generalizability of the
results to the entire population. This appraisal aims at critically examining two articles whereby
one is based on the quantitative research while the other is based on the qualitative research
study in the field of health and human services management. In this study, the first section will
highlight the tools selected while the precedent parts will critically review the research methods
used in the research articles’ case studies. The ethical issues and the summary of the strengths
and weaknesses of the implied methodologies will also be outlined.
Selection of the critical appraisal tools
The critical appraisal tools selected for the research articles’ methodology are the
decision analysis and the qualitative research appraisal tools. According to (Greco, Salvatore,
and Ehrgott, 2016), the decision analysis is an objective critical appraisal method that uses vital
models that represent the particular medical scenario and allows the application of the Evidence-
Based Medicine (EBM). In this tool, the factors involved in the selection of a certain strategy
from a given set of decisions are quantitatively evaluated. Moreover, this tool was suitable for
the (Liang and Wollersheim, 2017, pp.16-23) article on assessing the competence of Evidence-
Informed Decision-Making amongst the service managers since it was the best in evaluating the
steps in this study. However, the qualitative critical appraisal tool was selected for evaluating the
methodology used in the article on separation of families of Sudanese refugees that live in
Australia (Savic, Chur-Hansen, and Mahmood, 2018, pp.383-387). This is due to the nature of
the methodology used in collection and analysis of data, which were based on qualitative
methods.

CRITICAL APPRAISAL OF RESEARCH METHODOLOGY
3
Critical appraisal
Research article 1: (Liang and Wollersheim, 2017, pp.16-23)
This is a quantitative research study aimed at determining the competency in the use of
Evidence-Informed Decision-Making by the mid-level managers in health services. The case
study research also aims to demonstrate the behaviour of mid-level health managers that requires
improvement both at the organizational and individual level. The decision analysis questions are
varied against the methods of selection of participants, collection of data, and the data analysis
methods to determine the critical issues, both the strengths and the weaknesses of the study.
Critical appraisal checklist based on decision analysis tool
The tool seeks to examine whether the following was clearly stated, the patient group
over which the decision analysis should be applied? The patients’ health problem, the setting for
the care, and who is providing the care? These questions in the analysis tool all answered in the
abstract part of the research article on page 16 except the patient group is unspecified but rather
the study applies to all groups in the two Victorian hospitals.
The tool further evaluates the results of the study using the following set of questions.
Were all the strategies and outcomes included in the study? Does the model illustrate the
decision tree diagram? Were all the clinical realistic strategies compared? From the research
article, these questions cannot be determined since the decision tree diagram that could guide the
consumers of the study results from determining these factors was not included in the research
(Gokgoz and Ercan, 2015, pp.138-144). However, the tool further questions whether the relevant
outcomes were considered. These are, does the study include outcomes that matter to patients?
Does it address both the quantity and quality of life? From the article, these matters are covered
using the assessment of the managers on a Likert scale found on page 18 of the article. Such
aspects include the competence to guide the medical stuff, proficient, and superior expertise for
coaching and directing the staff in the effort of caring for the patients.
Another critic from appraisal tool is whether the sensible process was explicitly used to
identify, combine, and select the evidence into probabilities. Did the researcher give enough
information about how they searched for the literature to identify the studies they included? Do

CRITICAL APPRAISAL OF RESEARCH METHODOLOGY
4
they give enough information on how the probability values were assigned? From the research
report, the author has shown enough information used to obtain the quantitative data used. The
research used a 3600 subjective assessment method and an effective case study that is based on
the objective assessment. The analysis methods such as the use of SPSS software used for the
raw data downloaded has been identified in the study as explained on page 18 (Green and
Samuel, 2016). However, the research did not explain how the probability figures displayed from
the statistical competency calculation for the EIDM were arrived at on page 19 of the article.
The next section for critical appraisal was to determine whether the report has shown the
credible sources for the utilities. Are the source of utility rating indicated? Are the individuals
making the ratings a representative of the group the decision analysis is aimed at? The article has
cleared covered the sections whereby the mean score rating of different behavioural activities of
the mid-level managers at the Victorian hospitals is presented in a table format on page 19.
These ratings are for the target managers at the hospitals who are the core determinant of the
quality of services offered to the patients. Additionally, the researcher has outlined all the
sources applied in the article, which are summarized in the reference section on page 23. The
source is also well referenced according to the required standards.
Another important critique to be examined are the results. This intent to investigate the
baseline analysis of the research concerning the research questions. In this section, does the
strategy result in a clinical importance gain for the patients? In addition, if not, is the result a
toss-up? From the article, the results indicate that most managers in the Victorian hospitals are
competent in ensuring that the patients have access to a quality medical care at the hospitals.
(Meys et al., 2016, pp.17-29) deduces that by application of the 3600 subjective assessment
method, the results obtained are shown to be viable and implies the method can be applied to
another particular hospital. Moreover, the results from the article are not a toss-up since they
state the implications to the organisations while addressing the key barriers to the Evidence-
Informed Decision-Making factors that should be addressed to all medical facilities. This is
shown on page 21 through page 22 in the article. Therefore, the results can be projected to other
medical facilities to improve the medical care to the respective patients.

CRITICAL APPRAISAL OF RESEARCH METHODOLOGY
5
Research article 2: (Savic, Chur-Hansen, and Mahmood, 2013, pp.383-387)
This research explored the impacts of separation from the members of the family of the
Sudanese refugees on their wellbeing, their mental health, and the coping strategies in Australia.
The study was conducted using the qualitative research methods where the in-depth interviews
were used as the major source to collect the data. Thereafter, the data analysis method used was
the thematic analysis method. The research aimed at answering the implications undergone by
the Sudanese immigrants in the refugee camps in Australia. Concisely, the critical appraisal tool
used to criticize this research study was the qualitative critical appraisal technique as analyzed
below.
Qualitative critical appraisal technique
The qualitative critical appraisal tool seeks to identify the following from a research
study. Did the article describe the clinical problem that was addressed via the clearly formulated
questions? The second item, was the qualitative research appropriate? From the article, it can be
deduced that the study highlighted the problems faced by the participants in the study on page
383 to 384 that the study aimed at answering them. However, the author did not summarize nor
shortlist the research question to be addressed in the study in a single sentence. The use of the
qualitative approach is conclusively appropriate in the sense that it provided a platform to collect
the first-hand raw data from the victims (Teherani et al., pp.669-670).
The third item examined by the technique is the sampling strategy and justification of this
strategy. Under this, has the method of data sampling adequately described? Have the
researchers studied the most productive range of individuals to be interviewed relevant to the
question? Have the characteristics of the participants been defined? Was it clear why some
participants did not participate? From the article, on page 383 under the subtopic of key
informants, the method of sampling participants has been well described whereby the research
used the most productive participants for the study. These participants included the psychiatrists,
mental health social workers among many others who worked in the refugees’ camp. According
to the research question, this class of people could give reliable information concerning the
refugees, which is very productive for the study. However, the mentally ill victims were not
picked as part of the participants.

CRITICAL APPRAISAL OF RESEARCH METHODOLOGY
6
The fourth issue to be criticized in the study was the method of data collection. The
qualitative critical appraisal technique seeks to examine whether the following questions were
answered. Were the appropriate sources of data studied? Has the method been described in
detail? Were the methods reliable? Were the observations taken in a range of circumstances?
From the article on page 384 under the in-depth interviews subtopic, the research explains the
method of data collection in detail. Through the in-depth interviews, the research could be able
to collect all the relevant information from the participants who interact with the victims on daily
basis (Fusch and Patricia, 2015, pp.1408-1416). The use of audiotapes was used thus making the
method reliable and independently verifiable evidenced on page 384. The researcher used a
range of one year to collect the data thus making it possible to collect the data under the different
range of circumstances.
Was the relationship between the research and the participants explicit? From the article,
the research explains the time taken to interview a participant or an informant to be between 45
minutes to 3 hours on page 384 under in-depth interviews. Deducing from this, it is positive to
claim that the researcher had created a good relationship with the subjects to the research
(Rosenthal, 2016, pp.509-516). Moreover, the researcher observed the ethics, implications, and
the consequences of the findings since only key informants who could reflect the issues were
chosen for participation as explained on page 385 in the first paragraph.
In addition, another critical issue in the technique is the credibility of the results. Do they
address the research question? What conclusions were drawn and were they justified? To what
extent are the findings projected to other clinical settings? From the article under the results
subheading from page 384 to 386, the researcher has clearly stated the themes that emerged from
the study that answers the research question in details. Furthermore, the results can be said to be
credible since the research has gone a step further to include the sequences of several direct
quotations during data collection in the report seen on pages 384 and 385 respectively. The
researcher has also used presented the conclusion and discussion in a simple and coherent
manner that could be understood by the consumers of the findings and results on page 386 and
387.
Ethical issues in the research studies

CRITICAL APPRAISAL OF RESEARCH METHODOLOGY
7
Ethics in research is a requirement that is aimed at protecting the dignity of the
participants and the publication of the information obtained from the research. The ethical issues
from the summary of National Statement Content abstracted from
https://www.nhmrc.gov.au/health-ethicsc website observed by the researchers in the above
article are hereby discussed. From the two articles, the researchers observed an informed
consent code of ethics whereby the participants were persons knowingly intelligent and
volunteers to the study. An informed consent implies that the participant’s autonomy is protected
against any sort of discrimination, assaults on integrity and protection of personal liberty (Tam et
al., 2015, pp.186-198).
Another ethical practice observed is the beneficence (Bromley et al., 2015, pp.900-908).
Moreover, (Madary and Metzinger, 2016, p.3) implies that the hypothesis of the research
whether beneficial or turns out to be non-beneficial should at no cost be harmful to the
participants whether emotionally, physically, or professionally. For instance, in the first article,
the research is obligated not to expose the mid-level manager who might be underperforming.
Concisely, from the second article, if the researcher tries to learn the intimate details of a
participant they should ensure that the potential harm is minimized as much as possible in favour
of the participant.
Additionally, the researchers have impressed the respect for anonymity and
confidentiality code of ethics (Petrova, Elmira and Jan, 2016, pp.442-454). This is directly linked
to the beneficence code of ethics where it obligates the research against disclosing personal
responses by specifying the individuals. (Bertram et al., 2016, pp.3-7) in his findings suggests
that the privacy code of ethics is breached when private information such as attitudes, opinions,
beliefs, and records are shared without the consent of the participants or subjects. In addition, the
code of ethics also serves to protect the vulnerable groups of people. (De Chesnay, 2015)
elaborates vulnerability as a class of people with characteristics that are unable to protect their
own welfare and rights and should be therefore exempted as participants in research studies.
These include the mentally ill, children, aged people, among others.
Summary of the appraised papers

CRITICAL APPRAISAL OF RESEARCH METHODOLOGY
8
Strengths
The research studies had reliable sources of information that could highly contribute to
the accuracy of the results and findings of the research. Additionally, there was enough time for
data collection for the studies. This enabled the collection of data under a wide range of themes
or circumstances especially for the qualitative research (Bryman, 2015). Another strength of the
appraised quantitative research is that the findings can be generalized to a larger population
while maintaining the accuracy and viability of the research provided the data selection process
be well designed for study. Additionally, the data in the quantitative study can be very consistent,
reliable and precise. The data collected for the qualitative research is cost-effective according to
(Cox, 2018).
Weaknesses
In both the appraised research articles, the participants were voluntarily chosen thus not
randomly selected. The voluntary selection despite obeying the ethics codes of conduct, could
lead to some degree of biases in the findings unlike when the samples are randomly selected.
Another weakness of the studies is the size of the participants (Emerson, 2018, p.164). In both
the studies, small samples of the participants are used which could be inappropriate to project the
findings to other similar or related scenario to a larger population. Additionally, the data
collection process in the qualitative research of the second article is time-consuming and more
difficult to analyze since they do not fit into standard categories (Eriksson, 2015).

CRITICAL APPRAISAL OF RESEARCH METHODOLOGY
9
References
Bertram, T., Formosinho, J., Gray, C., Pascal, C., & Whalley, M. (2016). EECERA ethical code
for early childhood researchers. European Early Childhood Education Research
Journal, 24(1), 3-7.
Bromley, E., Mikesell, L., Jones, F., & Khodyakov, D. (2015). From subject to participant:
Ethics and the evolving role of community in health research. American Journal of
Public Health, 105(5), 900-908.
Bryman, A., & Bell, E. (2015). Business research methods. Oxford University Press, USA.
Cox, D. R. (2018). Analysis of binary data. Routledge.
De Chesnay, M. (Ed.). (2015). Caring for the Vulnerable. Jones & Bartlett Publishers.
Emerson, R. W. (2015). Convenience sampling, random sampling, and snowball sampling: How
does sampling affect the validity of research?. Journal of Visual Impairment & Blindness
(Online), 109(2), 164.
Eriksson, P., & Kovalainen, A. (2015). Qualitative methods in business research: A practical
guide to social research. Sage.
Fusch, P. I., & Ness, L. R. (2015). Are we there yet? Data saturation in qualitative research. The
qualitative report, 20(9), 1408-1416.
Gokgoz, E., & Subasi, A. (2015). Comparison of decision tree algorithms for EMG signal
classification using DWT. Biomedical Signal Processing and Control, 18, 138-144.
Greco, S., Figueira, J., & Ehrgott, M. (2016). Multiple criteria decision analysis. New York:
Springer.
Green, S. B., & Salkind, N. J. (2016). Using SPSS for Windows and Macintosh, Books a la
Carte. Pearson.
Liang, Z., Howard, P., & Wollersheim, D. (2017). Assessing the Competence of Evidence-
Informed decision-Making amongst health Service Managers. Asia Pacific Journal of
Health Management, 12(3), 16.

CRITICAL APPRAISAL OF RESEARCH METHODOLOGY
10
Madary, M., & Metzinger, T. K. (2016). Real virtuality: a code of ethical conduct.
Recommendations for good scientific practice and the consumers of VR-
technology. Frontiers in Robotics and AI, 3, 3.
Meys, E. M. J., Kaijser, J., Kruitwagen, R. F. P. M., Slangen, B. F. M., Van Calster, B.,
Aertgeerts, B., ... & Van Gorp, T. (2016). Subjective assessment versus ultrasound
models to diagnose ovarian cancer: A systematic review and meta-analysis. European
Journal of Cancer, 58, 17-29.
Petrova, E., Dewing, J., & Camilleri, M. (2016). Confidentiality in participatory research:
Challenges from one study. Nursing Ethics, 23(4), 442-454.
Rosenthal, M. (2016). Qualitative research methods: Why, when, and how to conduct interviews
and focus groups in pharmacy research. Currents in Pharmacy Teaching and
Learning, 8(4), 509-516.
Savic, M., Chur-Hansen, A., Mahmood, M. A., & Moore, V. M. (2013). ‘We don’t have to go
and see a special person to solve this problem’: Trauma, mental health beliefs and
processes for addressing ‘mental health issues’ among Sudanese refugees in
Australia. International Journal of Social Psychiatry, 37(4), 383-387.
Tam, N. T., Huy, N. T., Thoa, L. T. B., Long, N. P., Trang, N. T. H., Hirayama, K., &
Karbwang, J. (2015). Participants’ understanding of informed consent in clinical trials
over three decades: systematic review and meta-analysis. Bulletin of the World Health
Organization, 93, 186-198H.
Teherani, A., Martimianakis, T., Stenfors-Hayes, T., Wadhwa, A., & Varpio, L. (2015).
Choosing a qualitative research approach. Journal of graduate medical education, 7(4),
669-670.
Zeng, X., Zhang, Y., Kwong, J. S., Zhang, C., Li, S., Sun, F., ... & Du, L. (2015). The
methodological quality assessment tools for preclinical and clinical studies, systematic
review and meta‐analysis, and clinical practice guideline: a systematic review. Journal of
evidence-based medicine, 8(1), 2-10.

CRITICAL APPRAISAL OF RESEARCH METHODOLOGY
11
Appendix

CRITICAL APPRAISAL OF RESEARCH METHODOLOGY
12

CRITICAL APPRAISAL OF RESEARCH METHODOLOGY
13

CRITICAL APPRAISAL OF RESEARCH METHODOLOGY
14

CRITICAL APPRAISAL OF RESEARCH METHODOLOGY
15

CRITICAL APPRAISAL OF RESEARCH METHODOLOGY
16