Critical Analysis of a Humanitarian Intervention Evaluation Report

Running Head: CRITICAL ANALYSIS OF DFAT’S REPORT 0 CRITICAL ANALYSIS OF DFAT’S REPORT 15 A critical analysis of a Humanitarian Intervention Evaluation Report Student Name Introduction 2 Some of the other studies conducted by different organisations are 3 Purpose and rationale for the evaluation 4 Methodology used in the evaluation 4 Review Document 4 Key Informant Interviews 5 FGD (Focused Group Discussion) 5 Values and Principle 5 Strength and limitations 6 Strengths 6 Limitation 6 What Data

Running Head: CRITICAL ANALYSIS OF DFAT’S REPORT
0
A critical analysis of a Humanitarian Intervention Evaluation Report
Student Name

CRITICAL ANALYSIS OF DFAT’S REPORT
1
Table of Contents
Introduction................................................................................................................................2
Some of the other studies conducted by different organisations are......................................3
Purpose and rationale for the evaluation....................................................................................4
Methodology used in the evaluation..........................................................................................4
Document Review..................................................................................................................4
Key Informant Interviews......................................................................................................5
FGD (Focused Group Discussion).........................................................................................5
Values and Principle..................................................................................................................5
Strength and limitations.............................................................................................................6
Strengths.................................................................................................................................6
Limitation...............................................................................................................................6
What Data collected and its appropriateness.............................................................................7
NGO’s and other organisation involved in evaluation and their capacity.................................7
Criteria used to evaluate actions and achievements...................................................................8
How its approach differed from that suggested by other society’s framework.........................8
DFAT evaluation approaches.................................................................................................8
How it’s approach different from IFRS and RCS’s framework............................................9
Health featured in evaluation...................................................................................................10
Appropriateness of findings.....................................................................................................11
Conclusion................................................................................................................................12
References................................................................................................................................14

CRITICAL ANALYSIS OF DFAT’S REPORT
2
A critical analysis of a humanitarian intervention evaluation report
Introduction
Department of foreign affairs and trade released a document to evaluate the human
interventions provided and how well Australian government responded to that situation
during a cyclone pam hit Vanuatu Country. This article was published in 2017 by DFAT,
Canberra. On 13 March 2015 tropical cyclone struck many areas of Vanuatu with wind speed
of 270 km/h (Hong et al., 2018). According to Magee, Danielle, Verdon-Kidd, Kiem and
Royle (2015), it was the most disastrous cyclone ever faced in history. As discussed in DFAT
evaluation report this cyclone pam affected most of the population and damaged 90% of the
infrastructure. The destruction impacts all of the areas like agriculture, education, health
facilities, private property etc. It was estimated that around half of people were faced adverse
situation because of this cyclone.
Thousands of people displaced internally, high number people had no place to live
and were searching for shelter. At that time Vanuatu government had limited resources and
money to overcome this situation so they asked for help to all courtiers and they received
help from all over the globe, but Australia was the first nation responded quickly to support
people who were in trouble. According to news published by Ministry for Foreign Affairs
(2018) Julie Bishop (Minister of foreign affairs) announced to provide financial help of
$300,000 especially for people with disabilities. New Zealand announced financial help of
NZ$ 2.5 million (Hayward-Jones, 2015). Australian government and other departments
sanctioned more than $15 million to recover from negative impacts occurred due to cyclone.
Military assistance also provided relief services the people of Vanuatu (Bergin,
2015).Various other organisations like the Australian Red Cross, Red Crescent and other

CRITICAL ANALYSIS OF DFAT’S REPORT
3
private sectors also helped financially to support to Vanuatu government. This article
analysed effectiveness of services provided to the cyclone affected people and humanitarian
intervention evaluation report will be analysed critically. Methods, purposes, criteria used,
appropriateness of the findings and how different these were from services provided with
other organisation, will be discussed by reviewing the report of DFAT and other various
reports. It will also be analysed that whether the intervention mentioned in that document
were up to the mark or not.
Some of the other studies conducted by different organisations are:
UNICEF (2016) conducted a review article after six months of cyclone hit Vanuatu.
Some key findings of that study are:
 It was the biggest cyclone ever recorded.
 The pre-planned strategies to face cyclone were not enough in Vanuatu and Fuji.
 The interventions provided to the cyclone affected people were quick.
 Level 2 procedure were not used properly because of this the funding was not
processed on time and some of the opportunities missed.
 The management roles within UNICEF pacific were not clear.
 Strong relationship has been built by UNICEF with partner service providers.
A study conducted by Handmer and Iveson (2017) stated that people of Vanuatu
warned on time by using text , mobile phones etc. and it was found that text messages were
more effective to send warnings of cyclone pam within a population, and this becomes the
reason of low deaths occurred. Local people were alerted by youth (Ray, De, Leone, &
Gilbert, 2017).

CRITICAL ANALYSIS OF DFAT’S REPORT
4
According to an article published in Health Cluster bulletin by Ministry of health
Vanuatu (2015), most of the health facilities were affected, health services has been
decreased in all areas of health assistance, the availability of health staff including doctors
and nurses were very less at the time of cyclone hit Vanuatu, Shefa was the highly affected
area with nearly 88% damage of health facilities and in Tafea around 82 % health facilities
were affected. A news article published in BBC news (2016) just after 3 days of cyclone hit
Vanuatu reported that around 3300 people displaced and 24 died.
Study conducted by Care (2016) reveals that in there was less damage was reported in
Aniwa and compare to other areas like Erromango and Tanna where more than 60% structure
were damaged or destroyed.
Purpose and rationale for the evaluation
 To review the response of Australia to cyclone pam and how pacific island countries
can be supported better.
 To identify how DFAT can improve to facilitate pacific countries so that they can
prepare themselves to handle such disastrous situations in future, how well they can
respond and recover from such emergencies quickly.
 To get experience form recent cyclone tragedy and apply it in future for an instant
response.
 Various Organisation and NGOs worked to assist people, this study was done to
identify whether these organisations got enough funding and response from Australian
government or not.
 To evaluate the assistance provided by various government, private sector and NGO’s
were equally received by Vanuatu public or not.
 To identify how the funding distributed.

CRITICAL ANALYSIS OF DFAT’S REPORT
5
Methodology used in the evaluation
It was a cross a sectional study. Peoples from Vanuata and Fiji were interviewed
about the effectiveness of interventions provided at the time of cyclone. Focused Group
Discussion has been conducted among respondents.
Document Review
The documents were reviewed and coded to prepare questionnaires. Some
government organisation and DFAT provided these documents. Strategies, appeals and
reports, studies etc. mentioned in these documents mentioned.
Key Informant Interviews
The whole study was based on data collected from interviewed person. Thirty nine
participants were Australian government officials, 30 respondents from Vanuata government,
55 partner staff; ten were from private sector of Vanuata and Fiji and 5 stakeholders of
Australia. They were asked evaluation questions to acquire key information about the
assistance efficacy, effectiveness of assistance and recommendations. Detailed notes have
also been taken from each interview by audio recordings.
FGD (Focused Group Discussion)
The people from highly affected areas like Efate, Tanna and Erromango called to
participate in focused group discussion. The FGD were well designed and based on structured
guide. Some activities conducted among 14 groups of beneficiaries. In one activity they
asked participants to place pebble in the middle of the group and express their thoughts on a
particular situation. One more activity conducted by staff members was to assess the
understanding of participants about the importance of services. They asked to place a leaf or
pebble on an image showing the services they required just after the clone or some months
later. The pictures were about shelter, food medical services, water, transport, food etc. After

CRITICAL ANALYSIS OF DFAT’S REPORT
6
they placed their pebble on picture they were asked to explain the reason of placing leaf or
pebble on that particular image.
Values and Principle
The basic principle of this evaluation was to identify the quality and efficacy of the
interventions provided to the pacific island countries at the time of cyclone by Australian
government and department of foreign affairs and trade. It worked on a principle of getting
live comments from public by interviewing cyclone affected people; Australian and Vanuatu
officials to better understand the quality received by the public, to get recommendations and
suggestions for future interventions in such calamities. Face to face interaction was done to
get the feedback from the persons living in cyclone affected areas.
Strength and limitations
Strengths
 This study was evidence based study
 Face to face interviews collected the actual data and it supports the authenticity of
data.
 Study was done in both private and government sectors and among cyclone affected
people.
 It assessed the most needed requirements after cyclone.
 They reported both positive and negative aspects of the intervention provided by
DFAT
 Some of the activities conducted by them revealed the actual needs of Vanuatu’s
people

CRITICAL ANALYSIS OF DFAT’S REPORT
7
Limitation
 This study focused only on limited areas of services like health, shelter and education.
 Documentation handling was very poor.
 Coordination with other agencies was not proper
 The interviews have been conducted among government and private officials rather
than the public who actually faced the situation. The more focused were not on the
affected people.
 It does not focused on other basic needs like electricity, roads etc.
What Data collected and its appropriateness
The collected was the response of questionnaires designed. The response was taken in
audio format. Respondents of Australia and Vanuatu from both government and private
sectors were interviewed. Notes from every interview also collected. Data form Focused
group discussion also collected. The data collected after conducting some studies helped to
reach the confirmation that the services received by cyclone affected people was satisfying.
The collection by DFAT members followed a appropriate method of data collection in both
audio format and document format and the results gained at the end of the study proven that
the data collected were appropriate.
NGO’s and other organisation involved in evaluation and their capacity
Nongovernment organisation also participated in this study to gather local public of
affected areas for focussed group discussion. NGO’s and other organisations participated in
this evaluation were Australian Red Cross, Save the children, Care, Oxfam and World vision.
Australian Red Cross, New Zealand Red Cross and French Red Cross assisted around 40000

CRITICAL ANALYSIS OF DFAT’S REPORT
8
people and achieved 30% of total emergency services (Cullwick, 2016). They also helped to
prioritise identification and strength conclusion of capacity of local society person, recognise
discomfort and how cluster meetings has been conducted , they continued to support local
people by getting international response, they also helped to provide shelter, food and
emergency services.
Care organisation also involved in interventions provided to the affected people,
preparing them for cyclone seasons, to find out safe houses for homeless people, food and
water (Williams, 2017). Oxfam helped to bring Vanuatu Humanitarian team together, they
provided to clear the waste and to provide vouchers to farmers for agricultural needs (Oxfam,
2016). World vision and save the children provided services of food and security programs,
they helped to repair and maintain structure of collapsed schools and healthcare services also
provided to the cyclone affected (World Vision, 2015).
Criteria used to evaluate actions and achievements
Criteria used to analyse the actions and achievements of Australia’s intervention
program and office of development effectiveness. The main focus of this evaluation was to
analyse response of DFAT regarding how the departments led during the time of cyclone, the
experts coordinated to each other to deal with adverse situation. This evaluation covered only
two initial steps taken by Australian government to provide early recovery and response. The
main strategy behind this evaluation was to get feedback and advice from the local residence
and government and private officials.
Some of the question has been asked to analyse efficacy of intervention are:
 How effective the strategy behind rehabilitation services provided, the partners
selected and was the funding enough?

CRITICAL ANALYSIS OF DFAT’S REPORT
9
 What was gained with the interventions?
 How efficient was the Australian aid?
 Any improvement required making Australian aid effective?
 Response guided by which methods.
 Values related to Australian interventions.
 How DFAT coordinated with other groups and departments involved in interventions?
 How did Australian services helped in preparedness and disaster management?
 What improvements are required for quick response to emergencies?
How its approach differed from that suggested by other society’s framework
DFAT evaluation approaches
Evaluation has been done on the basis of policy frame work driven at the time of
cyclone, especially Humanitarian action policy 2011 and the protection in humanitarian
action framework 2013. Humanitarian action policy identifies saving lives and humanitarian
and emergency response. This policy saved people and assists them to achieve what they had
before the circumstances occurred it also supports Australia’s aid program purpose
(Australian Agency for international Development, 2011). Protection in humanitarian action
framework supported DFAT to achieve interventions goal it work on a policy of improving
safety measures of people faced natural disasters. It also facilitates people by providing
funding at the time of non-favourable conditions.
How it’s approach different from IFRS and RCS’s framework
IFRC worked on different strategies (Planning and Evaluation Department, 2011)
these are:

CRITICAL ANALYSIS OF DFAT’S REPORT
10
1. Follow code of conduct for global Red Cross and Red Crescent movements and
nongovernment organisations in disaster relief.
2. To provide relevant intervention which is needed mostly in a particular targeted
disaster affected group .it also considers the appropriateness of the intervention
provided.
3. Efficiency of the intervention also measured to know the cost effective services
delivered with good results
4. Effectiveness of the interventions also measured to understand which services can be
used in future .it also asses which intervention provided on time.
5. It evaluates the number of people assisted with the intervention and population not
covered. It gives an idea that which intervention is best suited to provide services to
large population in a short period of time.it also analysed that the assistance should be
provided to all equally
6. It examines the favourable and non-favourable impacts caused by particular
intervention by measuring the difference made.
7. It ensured that the intervention provided for long term and the recovery can be
achieved.
DFAT approaches and IFRC, RCS approaches are quite different. DFAT
concentrated on initial steps taken by donor, IFRC and RCS approaches concentrate on every
aspects of intervention for a long term effect.
World Vision (2018) reported in a report that complete recovery has not achieved yet
because most of the structure was damaged and still under process to reach to the goal, more
than 50,000 were provided food and around 47000 people received essential household items

CRITICAL ANALYSIS OF DFAT’S REPORT
11
with the help of other interventions providers like UNICEF, Ministry of Health and world
health organisation.
Health featured in evaluation
Department of foreign affairs and trade also evaluated the intervention provided by
Australian government in health care sectors. They found that the funding provided to the
government sector were very effective for fast recovery in health and after seven days of
cyclone there was only 2 % requirement for healthcare assistance. According to DFAT
evaluation report nearly 39 health facilities has been impaired by the cyclone pam and with
the help of local authorities and Australian interventions, 1341 patients has been cured with
the help of Vanuatu central hospital. The damaged health facilities have been repaired
quickly. Around 9,700 people with health issues were treated by twenty health teams, many
people received primary and secondary care, vaccination has been done to save children’s
from measles (World Health Organisation, 2016). Twenty Australian medical assistance
teams responded quickly to provide health services to the public affected with cyclone
(Coates, 2017). It has been evaluated by DFAT that 35000 $ has been sanctioned to Vanuatu
government in pharmaceuticals. It was stated in DFAT evaluation article that Australian
interventions focused in health sectors properly.
Appropriateness of findings
The key findings of DFAT’s evaluation report are:
 Nearly 188,000 people were affected due to cyclone hit on 13 March 2015.
According to an article published by United Nation Office for the Coordination of
Humanitarian Affairs (2015) nearly 166000 people has been affected by cyclone
among the population of 23, 4000.

CRITICAL ANALYSIS OF DFAT’S REPORT
12
 The cash released to help cyclone affected people would have been preferred form of
assistance to achieve recovery goals for selected, but not every group.
 Department of Foreign Affairs and Trade stated in that evaluation report Australian
financial help was appropriate, but according to article published in The Australian
(2018), it was not cleared that where the funding has been spent which was provided
by Australian government to the cyclone affected country. It was also stated in that
article that financial help provided for long term recovery was not enough.
 Small businesses affected by cyclone did not receive any help to overcome the
situation.
 According the report enough help has been provided to Crisis Response Team to
support Port Vila post. But the capacity of DFAT’s Port Villa post was not
satisfactory to balance competing priorities because of less available and untrained
staff however UNICEF did good job in health sector by treating more 10 thousand
children’s (UNICEF, 2015).
 Australian intervention providing groups concentrated on highly affected areas like
shelter, health and food. But on the other hand some studies studied that full recovery
of shelters still not achieved, according to a report published in Shelter Cluster (2015)
21 per cent households reported to require support for shelter and 29 % people living
with their family and friends because they had no place to live. Another study done by
UNOCHA (2015) stated that around 75000 people had no shelter.
 The report revealed that direct funding to government organisations was very
effective to achieve early recovery targets.
 It has been stated in evaluation report that Australia provided enough assistance.
 The evaluation founds that DFAT achieved the commitment of providing help in 48
hours.

CRITICAL ANALYSIS OF DFAT’S REPORT
13
 The communication between DFAT and other organisations was inappropriate. DFAT
itself gave low rating for the project stated that progress was not up to the mark (Klan,
2018).
 According to the study Australian interventions appropriately focused on highly
affected areas after cyclone.
Conclusion
After studying the whole report and reviewing the articles and reports it can be
concluded that the evaluation report published by department of Foreign affairs and trade was
based in evidence and appropriate hence it has some lacks. The negative aspects were also
discussed by DFAT in that evaluation , they stated that the communication between DFAT
and other assistance providers was not enough .the interventions provided by Australian
governments and private organisations was rapid and reached to the every person in need,
although some of the studies found that the money distributed to the assisting departments
were uneven . The methods used in the evaluation report were appropriate and the data
collected from the government and private organisation was collected appropriately. It has
been also found that some NGO’s were also involved in this study like Care, World vision,
save the children etc. Health assistance evaluation also included in this study and has been
done by organisations like UNICEF and WHO. Some of the studies found that the assistance
provided was not focused on the loss of small businessman’s. It only concentrated on most
affected area of Vanuatu. It was also found that study mostly focused on Australia’s initial
assistance. Focussed group discussion conducted by DFAT team members to collect the data
was appropriate and helpful to understand the need of cyclone affected people. It can also be
concluded that in this evaluation report most of the basic need were neglected to evaluate like
electricity and roads. Department of Foreign Affairs and trade itself gave them a now rating

CRITICAL ANALYSIS OF DFAT’S REPORT
14
and stated that the assistance provided to the Vanuatu public was not up to the mark and it
was less than expected. The evaluated findings of DFAT’s report were supported by most of
the studies supported. Some of the studies or articles criticised the evaluation report but major
studies were in favour of the findings concluded by DFAT.

CRITICAL ANALYSIS OF DFAT’S REPORT
15
References
Australian Agency for International Development (2011). Humanitarian action policy.
Retrieved from: http://dfat.gov.au/about-us/publications/Documents/ausaid-hap-dec-
11.pdf
BBC News (2016). Cyclone pam: UN confirms 24 dead and 3,300 displaced in Vanuatu.
Retrieved from: http://www.bbc.com/news/world-asia-31912305
Bergin, A. (2015). Vanuatu: the aftermath. Retrieved from:
https://www.aspistrategist.org.au/vanuatu-the-aftermath/
Care (2016). Does responsive disaster risk reduction make a difference when a category 5
cyclone strikes?. Retrieved from:
https://www.care.org.au/wp-content/uploads/2014/10/Vanuatu-DRR-Impact-Study-
Summary_12-Oct-2016.pdf
Coates, S. (2017). Rescue teams reach cyclone- hit Vanuatu islands, official toll lowered.
Retrieved from: https://www.reuters.com/article/us-weather-vanuatu-cyclone/rescue-
teams-reach-cyclone-hit-vanuatu-islands-official-toll-lowered-
idUSKBN0MC1ZY20150317
Cullwick, J. (2016). Tropical cyclone pan one year anniversary-Vanuatu Red Cross.
Retrieved from: http://dailypost.vu/news/tropical-cyclone-pam-one-year-anniversary-
vanuatu-red-cross/article_1d2ba05d-7cd4-5980-9cf2-7e6afd81cd5a.html
Handmer & Iveson (2017). Cyclone pam in Vanuatu: Learning from the low death tolls.
Australian journal of emergency management, 32(2), 63-64.
Hayward-Jones, J. (2015). Post cyclone aid to Vanuatu. Retrieved from:
https://www.lowyinstitute.org/the-interpreter/post-cyclone-aid-vanuatu

CRITICAL ANALYSIS OF DFAT’S REPORT
16
Hong, I., Pilarczyk, J. E., Horton, B. P., Fritz, H. M., Kosciuch, T. J., Wallace, D. J., Dike,
C., Rarai, A., Harrison, M. J., & Jockley, F. R. (2018). Sedimentological
charactersitics of the 2015 tropical cyclone pam over wash sediments from Vanuatu,
South pacific. Marine Geology, 396, 2016-214.
Magee, A. D., Verdon-Kidd, D. C., Kiem, A. S., & Royle, S. A. (2016). Tropical cyclone
perceptions, impacts and adaptation in the Southwest Pacific: an urban perspective
from Fiji, Vanuatu and Tonga. Natural hazards and earth system science, 16(5),
1091-1105.
Ministry for Foreign Affairs (2018). Humanitarian Assistance to Vanuatu. Retrieved from:
https://foreignminister.gov.au/releases/Pages/2018/jb_mr_180503a.aspx
Ministry of health Vanuatu (2015). Tropical cyclone pam: Vanuatu. Retrieved from:
http://www.wpro.who.int/healthclusterbulletin4.pdf
Morrison & Reid (2015). Vanuatu: cyclone pam, one month on. Retrieved from:
http://now.worldvision.org/story/vanuatu-cyclone-pam-one-month
Oxfam (2016). Oxfam in Vanuatu: Cyclone pam response. Retrieved from:
http://www.oxfam.org.hk/filemgr/2681/2016ip001cyclonepamoneyearreportfa.pdf
Planning and Evaluation Department (2011). IFRC framework for evaluation. Retrieved
from: http://www.ifrc.org/Global/Publications/monitoring/IFRC-Framework-for-
Evaluation.pdf
Rey, T., De, L. L., Leone, F., & Gilbert, D. (2017). An integrative approach to understand
vulnerability and resilience post-disaster: the 2015 cyclone pam in urban Vanuatu as

CRITICAL ANALYSIS OF DFAT’S REPORT
17
case study. Disaster prevention and management: an international journal, 26(3),
266.
Shelter cluster (2015). Shelter and settlements vulnerability assessment – final report;
cyclone Pam response. Retrieved from:
http://www.reachresourcecentre.info/system/files/resource-documents/
reach_vut_report_shelter_and_settlements_vulnerability_assessment_after_cyclone_p
am_may2015_1.pdf
The Australian (2018). Australia’s cyclone help ‘has not met expectations-DFAT. Retrieved
from: https://vanuatuindependent.com/2018/02/22/aus-cyclone-help-not-met-
expectations/
UNICEF (2015). Cyclone pam humanitarian situation report 13. Retrieved from:
https://www.unicef.org/pacificislands/SitRep_13_UNICEF_Pacific_Cyclone_Pam_2_
-_3_April_FINAL_(1).pdf
UNICEF (2016). Summary findings: UNICF after action review of cyclone pam. Retrieved
from:
https://reliefweb.int/sites/reliefweb.int/files/resources/unicef_after_action_review_of_
cyclone_pam_january_2016_2.pdf
UNOCHA (2015). Flash appeal launched for cyclone-affected Vanuatu. Retrieved from:
http://www.unocha.org/story/flash-appeal-launched-cyclone-affected-vanuatu
Williams, N. (2017). Cyclone Donna: Care prepares to respond in Vanuatu. Retrieved from:
https://www.care.org.au/media/media-releases/cyclone-donna-care-prepares-respond-
vanuatu/

CRITICAL ANALYSIS OF DFAT’S REPORT
18
World Health Organisation (2016). Public health risk assessment and interventions, tropical
cyclone pam: Vanuatu. Retrieved from:
http://iris.wpro.who.int/bitstream/handle/10665.1/13547/9789290617495-eng.pdf
World vision (2018). Cyclone pam update. Retrieved from:
https://www.worldvision.org.uk/our-work/emergency-response/vanuatu-cyclone-pam-
update/