Comprehensive Evaluation Report: Malawi Social Cash Transfer Programme

This report provides an in-depth evaluation of the Malawi Social Cash Transfer Programme (SCTP). It begins by outlining the theory of change underpinning the program and examining the impact of various indicators, such as schooling, feeding practices, and poverty levels, on individual behavior. The report then delves into the use of Randomized Control Trials (RCTs) to assess the program's effectiveness, including the selection of samples and the generation of balanced groups. The reasons for the lack of statistically significant differences between treated and control groups, as identified in the endline impact evaluation, are discussed, with reference to the baseline report. The report also highlights the use of the Difference-in-Differences (DID) method and the Inverse Probability Weighting (IPW) method to address attrition and selection bias, respectively. The Malawi case study, discussed in Chapter 3, is used to illustrate the SCTP's implementation and impact, including the role of government funding and community-based beneficiary selection. The report concludes by summarizing the key findings and implications of the evaluation, providing a comprehensive overview of the SCTP's impact and the methodologies used to assess it.

EVALUATION OF INDIVIDUAL
INTERVENTIONS

Table of Contents
INTRODUCTION...........................................................................................................................1
QUESTION 1...................................................................................................................................1
A. Theory of change presented in Malawi SCTP...................................................................1
B. Indicators and their impact................................................................................................2
QUESTION 2...................................................................................................................................3
A. Way of resolving problem.................................................................................................3
B. Evaluating RCT in case of Malawi....................................................................................3
C. Reason of no statistically significant differences in between treated and control groups. 4
D. Table in baseline report.....................................................................................................5
E. Country case discussed in Chapter 3..................................................................................5
QUESTION 3...................................................................................................................................6
A. Assumption of difference in difference method (DID).....................................................6
B. Four ways to test...............................................................................................................7
C. Why attrition could be a problem in the impact estimates................................................7
D. IPW method.......................................................................................................................8
CONCLUSION................................................................................................................................8
REFERENCES..............................................................................................................................10

INTRODUCTION
Present study is based on Malawi Social Cash transfer programme end line impact
evaluation report. It will explain theory of change presented in Malawi SCTP. Furthermore,
report will explain 3 types of indicators and their impact on final outcome. Study will explain
Randomised control trial (RCT) method with reference to Malawi. Assumption of difference in
difference method (DID) will be described in this research. In addition, Inverse probability
method for resolving problem of attrition will be discussed in this assignment.
QUESTION 1
A. Theory of change presented in Malawi SCTP
Cash transfer programs impact on the behaviour of individuals to a great extent. When
cash enters into the households then it makes changes in social perception. People make
economic and productive decisions that lead to alter in their future income (Handa and et.al,
2014). Each person has different needs and whenever cash comes to them, then they fulfil their
food, shelter, etc. with various requirements. Cash makes significant modifications in their
thoughts and they become able to accomplish their desires.
Difference between mediator and moderator
Mediators are such factors that work through systematic mechanism. These elements are
such as future expectation, attitude towards risk, information obtain by person. Whereas,
moderators are contextual features of local economy that impact on cash transfer mechanism.
Great examples of moderators are economic shocks, prices, infrastructure, etc (Abdoulayi and
et.al, 2014). Mediators are such elements in which cash directly impacts on the individual
behaviour. Moderators have indirect impact on the behaviour of individuals. Changes in norms
and infrastructure create problem in cash transfer. By this way, they get failed to fulfil their
desires. Environmental factors such as distance to school and poor health facilities put direct
impact on child’s growth. Moderators are qualitative or quantitative variables that have direct
impact on individual behaviour. On the other hand, mediators are such elements that describe
relationship in between two other variables. Mediation are straightforward component that
directly impacts on others (Austin and Stuart, 2015). Moderators are components that have
indirect impact on other dependent and independent variables.
1

Illustration 1: Framework and evaluation of Malawi SCTP
Source: (Malawi Social Cash Transfer Programme
Endline Impact Evaluation Report, 2016)
For example: when cash transfer occurs then household people invest their money in crop
production. This generates income for them; this money supports people in improving well-being
of kids and providing healthy nutrition to children. If services are not good then families will not
be able to provide good nutrition to their children (Mansournia and Altman, 2016). If there is
distance between school and home then it may create difficulty for the family members to
provide quality education to their kids. These moderators have high impact on the behaviour of
person. On the other hand, mediators have less impact on perception of individuals.
B. Indicators and their impact
The evaluation report of Social Cash Transfer in Malawai describes various indicators
that impact on behaviour of individual. Schooling and child labour is considered as one of the
essential indicators that impacts high. This enhances the participation of young children in school
(Ullah and et.al, 2015).
Feeding practices is one of the major components that impact on individual’s behaviour.
Some kids get solid eating material whereas others get average food. It is the variable that has
2

midline impact. On the other hand, poverty is another variable that has medium impact on an
individual’s behaviour. If per capita consumption of people is lower than their poverty line then
it means that the person in poor (Sommer, 2016). If person is poor then individual will not be
able to fulfil his basic needs. In such condition, person may face issues in their household living.
One of the most common impacts of poverty is reflected through food consumption. If person
has adequate monetary resources then they can consume food in excess whereas, if they have
limited funds then they have to compromise with their desires (Massucci and et.al, 2017).
One of the main mediators that impact on behaviour of individual is expectation towards
future. Each person has different perception for their future well-being. From the results, it can
be identified that this factor has significant impact on behaviour of persons. People always
expect better future and caregivers have positive outlook for their future. They always assume
that in the next two or three years, their life will be better. If cash transfer is appropriate then it
will change the perception of person positively (Marmar and et.al, 2015).
Preventive health care practices have immediate impact on the individual behaviour. If
babies are getting appropriate care in hospitals, then this may change the perception of their
family members. Whereas, if they are not getting correct treatment then it may make them
negative and change their behaviour as well (Shen and Björk, 2015).
QUESTION 2
A. Way of resolving problem
Randomization method is used to compare various versions of same programs. There are
many issues that are tried to be resolved by implementing various programs. This randomized
method is helpful in identifying the best method that has resolved issues properly. It compares
various programs and resolve issues (Difference in Difference, 2017). Randomised control trial
(RCT) method tries to analyse the method by making comparison with other programs. It
acknowledged standards and tried to generate high level of evidences for examining the results.
It resolves issues related to social science.
B. Evaluating RCT in case of Malawi
RCT designs have been used to evaluate studies and identifying their impact. It is
considered as set methodology “gold standard” that ensures validity of the results. In case of
Malawi, RCT method has been applied (Inverse probability weighting (IPW) for evaluating and
3

"correcting" selection bias, 2014). All samples have been selected randomly. Randomisation has
been done with support of GoM. RCT has helped in generating balance in between the groups.
This has helped in generating right answers. TA and VS were selected randomly, furthermore,
house hold selection was also made through the same process. The main motive of applying this
methodology is to identify whether programs have impact or not. Apart from this, it has been
applied to quantify how large impact has been made by this program (Shen and Björk, 2015). In
the present case, evaluators have selected some top priority questions and they have plans to get
answers. In order to evaluate the results, they have revisited the goal. They have applied theory
of change in this process. They have looked upon the key outcomes and expected pathway to
achieve the results. These results have been measured in the next phase. Evaluator has made
subdivision in terms of indicators. Evaluator has designed the questions and accordingly,
randomization method has been implemented (Webb and et.al, 2017). In order to evaluate the
entire program, person has taken into consideration various components such as fairness,
political feasibility, logistical feasibility, effects, sample size and power, etc. On the basis of all
these factors, evaluation has been done.
In case of Malawi, it has been identified that RCT method is quite effective and it has
worked. By this method, researcher has selected samples randomly. All these samples were able
to answer the research questions effectively without any biasness. Cluster randomised trial
method has been implemented (Austin and Stuart, 2015). Scholar has made clusters and on the
bases of clusters results have been examined. This has supported in identifying the impact of
various mediator and moderators on individual's behaviour. This has helped in gaining the
expected results. Sample size in Malawai study was appropriate and proper samples have been
involved in this research. This investigation has specified the characteristics of controlled and
treated group (Mansournia and Altman, 2016).
C. Reason of no statistically significant differences in between treated and control groups
By looking at health indicator self-reported health status indicator, it has been identified
that there is no statistical significant difference between treated group and controlled group. The
value of P is 0.48 which is less than 0.05 that reflects no significant difference in both variables.
Both these move in the same direction. It means that if health status is good then people will feel
happy and it will impact on their behaviour (Malawi Social Cash Transfer Programme Endline
Impact Evaluation Report, 2016). One of the main reasons of negative statistical relationship is
4

that value of P is <0.05 cutoff (Massucci and et.al, 2017). It reflects that two variables fall on
opposite sides of cutoff. Housing, use of durable goods and food consumption are different in
each family. Health indicator is moderator and it has no significant difference between treated
and controlled group members. It is identified that value of P is 0.86, fair aspect represents 0.73
value of P and good shows value 0.48. Very good variable shows 0.92, excellent 0.66. All these
factors that have been considered in statistical testing of these hypotheses have significant
difference between target and control group members. Then it is identified that value of p is less
than 0.05 that means there is no significant relationship in between both these variables. It
happens because both variables fall in different cutoff (Shen and Björk, 2015). By looking at
this indicator, it has been accessed that there is no statistical significant difference in between
treated and control group.
D. Table in baseline report
The table that has been used to identify no significant difference between treated and
controlled group is Appendix F” name: comparison of treatment and comparison groups-
baseline. This table clearly defines the relationship in between two variables. It has been
identified that there are many actors that have no significant difference (Webb and et.al, 2017).
In order to conduct this research, randomization process of sampling has been taken into
consideration. This method has helped in creating equivalent group at baseline. By this way
mean characteristic has been balanced between treatment and comparison groups. This RCT
method has worked in order to getting valid results (Austin and Stuart, 2015).
E. Country case discussed in Chapter 3
Malawi case has been discussed in chapter 3 of this research article. This SCTP program
has provided unconditional cash transfer facilities to households to ultra poor people. These
people consume low food and spend their income in fulfilling their consumption need. This
report has described that initial cash transfer will be done to fulfil basic needs of persons such as
food, cloth shelter etc. Once basic needs have been fulfilled then new cash transfer initiative will
be taken within household economy (Handa and et.al, 2014). In the second phase of cash transfer
government will pay attention on providing room for investments and implementing productivity
activities. By this way household people will be able to send their children in school because
they will have sufficient sources. This can enhance ability to free up older children and they will
attend school on regular basis. German government has signed agreement with GoM. The main
5

highlighting thing in this agreement was that GoM will provide funding for paying arrears. Later
on world bank has committed to launch government funded district so that it can be expended
into other districts as well (Abdoulayi and et.al, 2014).
Government has selected beneficiaries through community based approach. They have
not accepted experimental design evaluation on ethical concern. Their main objective was to
reduce poverty and enhance enrolment of children in the school. Government has given their
concerns for SCTP program, the main reason of getting government support to this research is
that to reduce poverty and hunger. Authorises were paying attention on increasing school
enrolment rate in ultra poor households (Zhai and Lafferty, 2017). In the year 2007-2008 impact
evaluation of pilot project has got positive results. After this study government has become able
to enhance food security, curative care seeking. The success of this project has created positive
impact in the mind of government and authorities have given their full support for SCTP project
as well.
It is very important for the researcher that to pay attention on ethical concern. As the
study was based on food consumption and reduction in poverty. In such condition if scholar has
taken assistance of experimental design then there would be high chances of exposing test results
(Massucci and et.al, 2017). In this type of design scholar would not be able to prevent
confidential details of respondents. Such type of experiments can hurt feelings and values of
specific community. Thus, researcher has selected longitudinal design. This method is applied in
studies in order to discover relationship between various variables. In this scholar study same
group of people for particular period of time. This type of researches are observational in nature.
As this design observe changes for extended period thus, it is better to observe results and find
out rout cause of problems (Shen and Björk, 2015).
QUESTION 3
A. Assumption of difference in difference method (DID)
DID is considered as quasi experimental design that take support of longitudinal data in
order to obtain appropriate counterfactual for estimating causal effects. It uses treatment and
control group members in order to find out answers. DID is effective method that assist in
estimating effects of intervention by examining changes in outcome over a period of time
between intervention and control group (Ullah and et.al, 2015). This method is based on
observations.
6

DID is generally applies in the investigation where scholar has to make observations
because exchange ability can not be assumed in such type of studies. It believes on less strict
exchange ability assumptions (Webb and et.al, 2017). Difference-in-difference method is more
suitable in the situation when randomization on individual is not possible. DID method removes
biases between treated and control group and try to find out real difference in both these group of
people over a period of time. Assumption on DID method are described as below:
 Exchange-ability
 Positivity
 Stable Unit treatment value assumption
It also emphasis on stable composition of intervention and control group. It ensures no
spillover effects. Parallel trend assumption is considered as most critical that helps in ensuring
internal validity of DID method (Mansournia and Altman, 2016). It ensures that difference in
control and treated group need to be constant over time. It believes that there is no need to make
statistical calculation because visual inspection is enough to find out results.
DID method is beneficial because it helps in getting intuitive interpretation. This is
helpful in obtaining causal effect by taking support of observational data. DID method can be
applied for individual and group level both. It accounts for changes that have been occurred over
extended period (Zhai and Lafferty, 2017).
B. Four ways to test
 In order to test assumption of DID method, various test can be applied. T- test can be
performed to test assumption of DID method. In this researcher can take average leverage
growth rate between treated and control group during pre-treatment era and can apply t-
test.
 Conditional mean estimation is another way to test assumption (Sommer, 2016).
 Regression analysis is another way to test assumption of this difference- In- difference
method.
 Simple observation can help in finding out accurate answers in the study.
C. Why attrition could be a problem in the impact estimates
One of the main benefit of implementing RCT method in research is that each group can
be balanced equally. Attrition can create problem in estimating impact in any research. Some
times' attrition creates bias. In this condition outcome over extended period can not be measured
7

properly. It is not a big issue, if it is up to 5% but when it goes beyond 20% then this may affect
overall impact of the study (Shen and Björk, 2015). As present study is longitudinal study and in
such type of researches some members or samples are lost to be analysed because of attrition.
Sample attrition can distort this comparison between treatment and control group members. If
researcher has selected different methods or patterns of attrition in different group of people then
their results will not longer be similar which may affect overall impact. In such condition
researcher may get failed to identify actual impact (Zhai and Lafferty, 2017). Some times
missing values affect accuracy of final results thus, impact get changed. This is problem in
research and impact on end outcome.
D. IPW method
Inverse probability method is generally applied in the type of statistical researches
whereas scholar has to resolve issue of missingness and selection bias. It is considered as
statistical tool that helps in making calculations to a population. This tool helps in removing
confounding. In this method researcher can build logistic regression model, this model assist in
estimating probability of exposure observed in individual (Inverse probability weighting (IPW)
for evaluating and "correcting" selection bias, 2014). This is the technique that aids in analysis
by weighting the observations. IPW method works on assumption that information of person can
be predict probability of inclusion. By this way researcher can make inferences about target
population. IPW model consider entire population and then it calculates non missing
information. After that scholar weight to each subject by inverse predicted probability.
Inverse probability weighting method create pseudo-population in order to remove
confounding. Weight procedure has been used by scholar in many previous studies. This method
makes the sample like population (Shen and Björk, 2015). Researcher can add larger weight to
people those who are under-represented and lower weight for the over-represented population.
By this way issue of missingness and bias can be minimized. This helps in enhancing validity
and accuracy of final results.
CONCLUSION
From the above report it can be concluded that SCTP has been conducted to reduce
poverty and hunger. This has helped in identifying statistical relationship between various
variables that lead to poverty in country. Randomised control trial (RCT) method try to analysis
the method by making comparison with other programs. DID is generally applies in the
8

investigation where scholar has to make observations because exchange ability can not be
assumed in such type of studies. IPW method works on assumption that information of person
can be predict probability of inclusion. By this way researcher can make inferences about target
population. IPW model consider entire population and then it calculates non missing
information. After that scholar weight to each subject by inverse predicted probability.
9

REFERENCES
Books and Journals
Abdoulayi, S. and et.al., 2014. Malawi Social Cash Transfer Program Baseline Evaluation
Report. North Carolina: Carolina Population Center. Retrieved May. 31. pp.2015.
Austin, P. C. and Stuart, E. A., 2015. Moving towards best practice when using inverse
probability of treatment weighting (IPTW) using the propensity score to estimate causal
treatment effects in observational studies. Statistics in medicine. 34(28). pp.3661-3679.
Handa, S. and et.al., 2014. Malawi Social Cash Transfer Program Baseline Evaluation Report.
Carolina Population Center, UNC.
Mansournia, M. A. and Altman, D. G., 2016. Inverse probability weighting. Bmj. 352. pp.i189.
Marmar, C. R. and et.al., 2015. Course of posttraumatic stress disorder 40 years after the
Vietnam War: Findings from the National Vietnam Veterans Longitudinal Study. JAMA
psychiatry. 72(9). pp.875-881.
Massucci, M. and et.al., 2017. A26Radically resected stage III colorectal cancer: sidedness and
prognosis. Annals of Oncology, 28(suppl_6).
Shen, C. and Björk, B. C., 2015. ‘Predatory’open access: a longitudinal study of article volumes
and market characteristics. BMC medicine. 13(1). pp.230.
Sommer, L., 2016, July. Systematic Evaluation of Moving Object Detection Methods for Wide
Area Motion Imagery. In Proceedings of the 2015 Joint Workshop of Fraunhofer IOSB
and Institute for Anthropomatics, Vision and Fusion Laboratory (Vol. 24, p. 153). KIT
Scientific Publishing.
Ullah, H. and et.al., 2015. Formulation and Application of Optimal Homotopty Asymptotic
Method to Coupled Differential-Difference Equations. PloS one. 10(4). p.e0120127.
Webb, H. J. and et.al., 2017. “Pretty Pressure” From Peers, Parents, and the Media: A
Longitudinal Study of Appearance‐Based Rejection Sensitivity. Journal of Research on
Adolescence. 27(4). pp.718-735.
10