Economic Evaluation Critical Appraisal

This article provides a critical appraisal of a study on the cost-effectiveness of femtosecond laser-assisted cataract surgery versus phacoemulsification cataract surgery. It discusses the methodology, results, and limitations of the study, as well as the process of economic evaluation and critical appraisal of healthcare interventions.

Running head: ECONOMIC EVALUATION CRITICAL APPRAISAL
Economic Evaluation Critical Appraisal
Name of the Student
Name of the University
Author note

1ECONOMIC EVALUATION CRITICAL APPRAISAL
Chosen study: Abell, R. G., & Vote, B. J. (2014) Cost-Effectiveness of Femtosecond Laser-
Assisted Cataract Surgery versus Phacoemulsification Cataract Surgery
Introduction
Economic evaluation is a process of systematic identification of the financial benefit,
measuring the significance and considering the viability and feasibility under a critical
quantitative analysis of an intervention. In healthcare intervention economic evaluation is
often use to measure how economically feasible the intervention procedure is, in terms of
affordability of the consumers, investing capability of the healthcare service provider and the
potential financial and non financial benefit from it. Hence, to be a health intervention plan
feasible for a particular scenario, it should be economically beneficial as well. The purpose of
this paper is to critically appraise an article which has been done a descriptive economic
evaluation of a particular intervention plan.
The critical appraisal of a study refers to the process of analysing the feasibility,
validity, value, attainability and tangibility of a research. In this paper to critically appraise a
chosen article on economic evaluation of a healthcare intervention, two types of assessment
will be used. Drummond’s ten-step checklist to structure with 10 questions will be used to
critically appraise the article. On the other hand, Quality of Health Economic Studies
instrument proposed by Offman will be used to to score the quality of the study more
formally. The scoring will be justified as well. For this critical appraisal the chosen article is
“Cost-Effectiveness of Femtosecond Laser-Assisted Cataract Surgery versus
Phacoemulsification Cataract Surgery”, a study conducted by Abell, R. G., & Vote, B. J. in
2014. The article appraisal will be presented in the following sections followed by the brief
description of the study.

2ECONOMIC EVALUATION CRITICAL APPRAISAL
Brief description of the study 300
The study was aimed performing a comparative cost-effectiveness analysis (CEA) of
femtosecond laser-assisted cataract surgery (LCS) with the conventional phacoemulsification
cataract surgery (PCS) based on a design called Retrospective CEA using computer-based
econometric modelling.
As methodology or experimental procedure used by this study was Hypothetical
cohort of patients who are undergoing cataract surgery, by collecting the data from the
current literature and another direct experience using LCS. A cost-effectiveness decision tree
model was developed. in order to analyse the cost-effectiveness of LCS compared with PCS.
Through reviewing the current literatures on complete the cohort of patients and outcomes,
the complication rates of cataract surgery were obtained in this study. The time trade-off
utility values converted from visual acuity outcomes was considered for more accurate
calculation. From these data the increase in quality-adjusted life-years (QALYs) was
calculate in a hypothetical cohort between 6 months and 1 year after cataract surgery. Apart
from that, the outcomes were combined with approximate costs in a cost-utility analysis
model to determine the incremental cost-effectiveness ratios (ICERs).
From the quantitative analysis conducted by this research through the simulated
complication rates of PCS and LCS and assuming resultant visual acuity it has been found
that the outcome improvement is 5% in uncomplicated cases of LCS where the cost-
effectiveness measured by the dollars spent per QALY was gained from LCS. It has been also
found that LCS was not cost-effective at $92 862 Australian Dollars. The total QALY gain
for LCS over PCS was 0.06 units. Moreover, it has been found that LCS is not cost efficient
relative to the present cost-effectiveness benchmarks and other medical procedures, including
PCS, irrespective of future changes in visual acuity results and complication rates. A
substantial decrease in the patient costs would improve the probability that LCS would be

3ECONOMIC EVALUATION CRITICAL APPRAISAL
seen as cost-effective (via decreased consumables / clication costs). It has been also found
that
Multivariate sensitivity analyzes have shown that LCS needs to considerably increase visual r
esults and complication rates across PCS, and also reduce patient costs in order to enhance ef
ficiency.
Checklist for the Economic Evaluation of Health Care Programmes 1500
1. Was a well-defined question posed in answerable form?
1.1. Did the study examine both costs and effects of the service(s) or programme(s)?
This study analysed both cost for the femtosecond laser-assisted cataract surgery
(LCS) and conventional phacoemulsification cataract surgery (PCS) as well as the resultant
effectiveness of the surgery through a collecting data from the health interventions based
literatures. However, the cost has been measured by the monetary unity and the effectiveness
has been measured by health improvement prevalence.
1.2. Did the study involve a comparison of alternatives?
This study is based on the comparison of two alternative method of cataract surgery in
terms of their cost and resultant benefits.
1.3. Was a viewpoint for the analysis stated and was the study placed in any particular
decision-making context?
The viewpoint of the study and the study conducting method were completely
relevant to each other. This study was about cost effectiveness evaluation and comparison of
two healthcare intervention, where the method of this study was exactly placed in this same

4ECONOMIC EVALUATION CRITICAL APPRAISAL
context.
2. Was a comprehensive description of the competing alternatives given (i.e. can you
tell who did what to whom, where, and how often)?
2.1. Were there any important alternatives omitted?
There are some other alternative intervention that has been omitted, since the study
was designed to compare the two chosen surgeries. Extracapsular cataract extraction (ECCE),
Intracapsular cataract extraction (ICCE), Manual small incision cataract surgery (MSICS) are
some of the other alternatives that were omitted in this study.
2.2. Was (should) a do-nothing alternative be considered?
In this case of cataract surgery the two chosen alternatives are two alternative
procedures of cataract surgery. Hence a healthcare intervention, especially surgical
intervention cannot be supplemented by doing nothing or Null Intervention. Hence, in this
case do-nothing should not be considered.
3. Was the effectiveness of the programme or services established?
3.1. Was this done through a randomised, controlled clinical trial? If so, did the trial
protocol reflect what would happen in regular practice?
This study did not use any primary experimental method. The cost of the chosen
interventions was analysed through cost determination tree and the effectiveness was
measured by reviewing several secondary researches.
3.2. Was effectiveness established through an overview of clinical studies?

5ECONOMIC EVALUATION CRITICAL APPRAISAL
Multiple clinical studies was reviewed to establish the effectiveness of both types of
surgery through comparing the increase in quality-adjusted life-years (QALYs). Some of the
study durations were 6 months and some of the study durations were 1 year after cataract
surgery.
3.3. Were observational data or assumptions used to establish effectiveness? If so, what are
the potential biases in results?
The assumption has been made in order to simulate a hypothetical cohort study based
on multiple conducted experimental studies on both PCS and LCS. Therefore, the potential
biasness is totally depends on the selection of the secondary resources. However, in this study
a significant number of literatures were reviewed that has successfully reduced the potential
biasness.
4. Were all the important and relevant costs and consequences for each alternative
identified?
4.1. Was the range wide enough for the research question at hand?
The cost was estimated through considering 6 possible complications in LCS .
However, for measuring the potential cost of PCS the considerations were not wide enough
for the accurate comparison. However, the range of effectiveness measurement is sufficient
for both PCS and LCS interventions. Moreover, the cost measurement range could be wider.
4.2. Did it cover all relevant viewpoints? (Possible viewpoints include the community or
social viewpoint, and those of patients and third-party payers. Other viewpoints may also be
relevant depending upon the particular analysis.)
The study did not consider all the possible viewpoint. To examine the cost of PCS
and LCS intervention the study should consider the viewpoint of the healthcare service

6ECONOMIC EVALUATION CRITICAL APPRAISAL
provider as well as the viewpoint of healthcare service consumer. However, this study mainly
considered the viewpoint of the healthcare service provider while assuming the consequent
relation of it with consumer perspective.
4.3. Were the capital costs, as well as operating costs, included?
The capital cost, operating cost and maintenance cost for the PCS and LCS
interventions were considered to formulate the total cost. However, the capital cost and the
operating cost were not project with details. When it comes to considering the inclusion of
both of these expenditures, the study did it correctly.
5. Were costs and consequences measured accurately in appropriate physical units
(e.g. hours of nursing time, number of physician visits, lost work-days, gained life
years)?
5.1. Were any of the identified items omitted from measurement? If so, does this mean that
they carried no weight in the subsequent analysis?
The capital cost, operating cost and maintenance cost for the PCS and LCS
interventions were considered without projecting any detailed description of the cost
distribution or weight. There was not subsequent analysis for this type of distributive cost
measurement as well.
5.2. Were there any special circumstances (e.g., joint use of resources) that made
measurement difficult? Were these circumstances handled appropriately?
The joint measurement of resources was used to measure the quality-adjusted life-
years (QALY) as the effectiveness unit of two alternative healthcare interventions. However

7ECONOMIC EVALUATION CRITICAL APPRAISAL
the rate of presented in percentage prevalence were presented properly in tabular form.
However, the QALY measurement was justified with a third ordered equation (y=0479x3
+0.191x2 – 0.4233x + 0.9128,) without providing any detailed derivation of that equation.
6. Were the cost and consequences valued credibly?
6.1. Were the sources of all values clearly identified? (Possible sources include market
values, patient or client preferences and views, policy-makers’ views and health
professionals’ judgements)
The sources of the values were identified with only the healthcare service provider’s
perspective. No other sources of values are identified, which is a major weakness of this
study.
6.2. Were market values employed for changes involving resources gained or depleted?
There was no variable considered to employ the market values involving potential
resource gained or depleted, which is another major weak point of this study.
6.3. Where market values were absent (e.g. volunteer labour), or market values did not
reflect actual values (such as clinic space donated at a reduced rate), were adjustments made
to approximate market values?
The market value evaluation was completely absent and there was n approximated
market value measurement present in this study.
6.4. Was the valuation of consequences appropriate for the question posed (i.e. has the
appropriate type or types of analysis – cost-effectiveness, cost-benefit, cost-utility – been
selected)?
This study considered the cost-effectiveness model to compare the valuations of

8ECONOMIC EVALUATION CRITICAL APPRAISAL
potential consequences of LCS and PCS intervention. Considering the question of this study,
this model is appropriate, since the benefit from a surgical intervention cannot be derived
from a financial calculation.
7. Were costs and consequences adjusted for differential timing?
7.1. Were costs and consequences that occur in the future ‘discounted’ to their present
values?
Considering the current intervention procedure of Cataract treatment and the possible
solutions of the potential complications, there will not be any major discontinuation of the
cost assumed as well as presented in this study. Hence, the present values can be considered
reliable and valid for next 10 to 15 years.
7.2. Was there any justification given for the discount rate used?
The discount rate was considered for the cost of glasses by 3%. However, there was
no justification regarding this consideration to support this fact. Lack of justification of
discount is another weakness of this study.
8. Was an incremental analysis of costs and consequences of alternatives performed?
8.1. Were the additional (incremental) costs generated by one alternative over another
compared to the additional effects, benefits, or utilities generated?
A Sensitivity Analysis was done using United States Cost Economics (Laser Cataract
Surgery [LCS] as Incremental Cost to US Public Cataract Cost to Determine Incremental
Cost-effectiveness Ratios [ICER].

9ECONOMIC EVALUATION CRITICAL APPRAISAL
9. Was allowance made for uncertainty in the estimates of costs and
consequences?
9.1. If data on costs and consequences were stochastic (randomly determined sequence of
observations), were appropriate statistical analyses performed?
The data on costs and consequences were stochastic and a sensitivity analysis was done as a
appropriate statistical method to determine the sequence of observation through incremental
estimation of cost with all possible consequences.
9.2. If a sensitivity analysis was employed, was justification provided for the range of
values (or for key study parameters)?
A Sensitivity Analysis was done using United States Cost Economics (Laser
Cataract Surgery [LCS] as Incremental Cost to US Public Cataract Cost to Determine
Incremental Cost-effectiveness Ratios [ICER]. However, the justifications were not
enough in this study to support all the considerations.
9.3. Were the study results sensitive to changes in the values (within the assumed range for
sensitivity analysis, or within the confidence interval around the ratio of costs to
consequences)?

10ECONOMIC EVALUATION CRITICAL APPRAISAL
As per the projected ratio of USD$/QALY in this study, some of the components
are sensitive and some of the components are not. Moreover, the result showed that the
values were less sensitive.
10. Did the presentation and discussion of study results include all issues of
concern to users?
10.1. Were the conclusions of the analysis based on some overall index or ratio of costs to
consequences (e.g. cost-effectiveness ratio)? If so, was the index interpreted intelligently or
in a mechanistic fashion?
To formulate the conclusion from the projected results the study used a ratio of cost
per effectiveness which was presented as USD$/QALY. However, the interpretation of this
ratios was more mechanical than intelligent. The major flaw of this study is the study
did not interpret the correct effectiveness of both interventions intelligently. Looking
over the numerical results could help the reader to determine the conclusive statement.
10.2. Were the results compared with those of others who have investigated the same
question? If so, were allowances made for potential differences in study methodology?
The conclusive results was not compared with other studies on this topic.
10.3. Did the study discuss the generalisability of the results to other settings and
patient/client groups?
The lack of discussion of generalisability of the results to other settings and
patient/client groups is another major weak point of this study.
10.4. Did the study allude to, or take account of, other important factors in the choice or

11ECONOMIC EVALUATION CRITICAL APPRAISAL
decision under consideration (e.g. distribution of costs and consequences, or relevant ethical
issues)?
The study has not taken the ethical factors and legal scope into account to present the
final verdict. The conclusion is directly result oriented which developed from the economic
perspective.
10.5. Did the study discuss issues of implementation, such as the feasibility of adopting the
‘preferred’ programme given existing financial or other constraints, and whether any freed
resources could be redeployed to other worthwhile programmes?
The discussion on feasibility of adopting LCS was done elaborately in the discussion section.
Quality of Health Economic Studies Instrument
Questions Point Score (Yes=point
and No=0)
1 Was the study objective presented in a clear, specific, and measurable
manner?
7 7
2 Were the perspective of the analysis (societal, third-party payer, etc.)
and reasons for its selection stated?
4 0
3 Were variable estimates used in the analysis from the best available
source (i.e., randomized control trial - best, expert opinion - worst)?
8 8
4 If estimates came from a subgroup analysis, were the groups pre-
specified at the beginning of the study?
1 1
5 Was uncertainty handled by (1) statistical analysis to address random
events, (2) sensitivity analysis to cover a range of assumptions?
9 9
6 Was incremental analysis performed between alternatives for
resources and costs?
6 0
7 Was the methodology for data abstraction (including the value of
health states and other benefits) stated?
5 5
8 Did the analytic horizon allow time for all relevant and important
outcomes? Were benefits and costs that went beyond 1 year discounted
(3% to 5%) and justification given for the discount rate?
7 7
9 Was the measurement of costs appropriate and the methodology for 8 0

12ECONOMIC EVALUATION CRITICAL APPRAISAL
the estimation of quantities and unit costs clearly described?
10 Were the primary outcome measure(s) for the economic evaluation
clearly stated and did they include the major short-term, long-term, and
negative outcomes?
6 6
11 Were the health outcomes measures/scales valid and reliable? If
previously tested valid and reliable measures were not available, was
justification given for the measures/scales used?
7 7
12 Were the economic model (including structure), study methods and
analysis, and the components of the numerator and denominator
displayed in a clear, transparent manner?
8 8
13 Were the choice of economic model, main assumptions, and
limitations of the study stated and justified?
7 7
14 Did the author(s) explicitly discuss direction and magnitude of
potential biases?
6 0
15 Were the conclusions/recommendations of the study justified and
based on the study results?
8 8
16 Was there a statement disclosing the source of funding for the study? 3 0
Total Points 100 73
As per the above scoring it can be seen that that this study has scored 73 out of 100 in
Quality of Health Economic Studies Instrument based assessment. The study has scored more
than 50% that indicates that this study has certain value as well as reliability. Above 70%
score also showed that this study is well developed and structured with a proper statistical
and evidence based support. However, this study is unable to reach above 80% score because
of its multiple drawbacks in many quality components.
Summary and Conclusion
As per the above discussion it can be concluded that the purpose of the chosen study
was to perform a comparative cost-effectiveness analysis (CEA) of femtosecond laser-
assisted cataract surgery (LCS) and conventional phacoemulsification cataract surgery (PCS).

13ECONOMIC EVALUATION CRITICAL APPRAISAL
From the Economic Evaluation of Health Care Programmes and Quality of Health Economic
Studies Instrument based assessments, it can be said that this study has significant value and
reliability in its field. At the same time, this study has multiple drawbacks in many quality
components. Moreover, the study could be more descriptive over assumption techniques and
conclusion formulation. After all, it can be concluded that this study is capable to clearly
shades light on current cost-effectiveness factors of femtosecond laser-assisted cataract
surgery (LCS) and conventional phacoemulsification cataract surgery (PCS).

14ECONOMIC EVALUATION CRITICAL APPRAISAL
References:
Abell, R. G., & Vote, B. J. (2014). Cost-effectiveness of femtosecond laser-assisted cataract
surgery versus phacoemulsification cataract surgery. Ophthalmology, 121(1), 10-16.,
doi: 10.1016/j.ophtha.2013.07.056
Drummond, M. F., Drummond, M. F., & McGuire, A. (2001). Economic evaluation in health
care: merging theory with practice. OUP Oxford. Retrieved from:
https://books.google.co.in/books?hl=en&lr=&id=_fJK15F75-
EC&oi=fnd&pg=PP2&dq=Economic+Evaluation+of+Health+Care+Programmes+Dr
ummond&ots=JUcHBiUB0H&sig=4-
x1VytlF8X7orBL0593pIxWfX8#v=onepage&q=Economic%20Evaluation%20of
%20Health%20Care%20Programmes%20Drummond&f=false
Jabbarvand, M., Hashemian, H., Khodaparast, M., Jouhari, M., Tabatabaei, A., & Rezaei, S.
(2016). Endophthalmitis occurring after cataract surgery: outcomes of more than 480
000 cataract surgeries, epidemiologic features, and risk
factors. Ophthalmology, 123(2), 295-301. , doi: 10.1016/j.ophtha.2015.08.023
Ofman, J. J., Sullivan, S. D., Neumann, P. J., Chiou, C. F., Henning, J. M., Wade, S. W., &
Hay, J. W. (2003). Examining the value and quality of health economic analyses:
implications of utilizing the QHES. Journal of Managed Care Pharmacy, 9(1), 53-
61., doi: 10.18553/jmcp.2003.9.1.53