Impact Evaluation of SMILES Program in Youth Justice System

This proposal aims to evaluate the impact of SMILES program in the youth justice system for children coming from families with mentally ill persons. The evaluation process makes use of non-specific design and pre-post survey method will be used as the data collection methods. The impact evaluation study also makes use of descriptive/ qualitative data analysis for the raw data collected.

Running head: IMPACT EVALUATION PROCESS OF SMAILES PROGRAM
PROPOSAL FOR IMPACT EVALUATION OF SMILES PROGRAM IN THE YOUTH
JUSTICE SYSTEM
Name
Institution
Course
Date

IMPACT EVALUATION PROCESS OF SMAILES PROGRAM
PROPOSAL FOR IMPACT EVALUATION OF SMILES PROGRAM IN THE YOUTH
JUSTICE SYSTEM
Table of Contents
Executive summary........................................................................................................2
Evaluation rationale.......................................................................................................3
Stake holders of the impact evaluation..........................................................................3
Evaluation aims and objectives......................................................................................4
Evaluation design...........................................................................................................4
Ethical responsibilities...................................................................................................6
Data collection method..................................................................................................7
The sample.....................................................................................................................7
The data collection instruments.....................................................................................7
Data analysis processes..................................................................................................8
The procedures...............................................................................................................8
Dissemination strategy...................................................................................................9
Timeline.......................................................................................................................10
Gantt chart....................................................................................................................10
References....................................................................................................................12
Executive summary
The research proposal is intended for an impact evaluation program by SMILES in the youth
justice center. The proposal is based on the continued challenges faced by children of 12-16 years of
age coming from families that have mentally ill persons. The study aims at carefully analyzing,
evaluating and identifying the main challenges faced by the children coming from families with
mentally ill patients. The study will thus impart the necessary skills needed by the children in
positively and easily copying to the environment. The evaluation process makes use of non-specific
design and pre-post survey method will be used as the data collection methods. The impact evaluation
study also makes use of descriptive/ qualitative data analysis for the raw data collected.

IMPACT EVALUATION PROCESS OF SMAILES PROGRAM
Evaluation rationale
Over the years, there has been a persistently increasing report regarding challenges faced by
children living with mentally ill relatives including fathers, mothers, siblings, brothers, and sisters
among others. These cases have been of varying consequences to the children, depending on the level
of mental illness of the child(ren). It is imperative and substantial that children of tender age do not
have sufficient skills of how to adapt to certain circumstances of the environment (Levy, et al., 2017,
p. 17). This is evidenced by the fact that, children from such affected families tend to have general
issues in their social life as well as cognitive development. Additionally, the relatives of the children
who are victims of mental illness have continued undergoing poor medical and mental health care due
to the incompetence and ignorance of the involved caretakers (Costello, Taylor, & O’Hara, 2015, p.
449).
In consideration of the preceding insight, the rationale for steering and executing SMILES
program (Simplifying mental illness + life-enhancing skills) is to help victimized children from
families having mentally affected relatives on how to cope and adjust to the environment in which
they live. The SMILES program will be implemented at the youth justice system centere. The youth
justice system is a foundation established to foster the empowerment of the young through the
provision of necessary support for development, both physically and mentally (Heslin, et al., 2017, p.
407). On the other hand, SMILES was formulated and invented in the late 1990’s by Erica Pitman. In
order to achieve effective implementation, the program was undertaken during the school breaks
when the children are on holidays (Flood, Minkler, Hennessey, Estrada, & Falbe, 2015, p.668)
Particularly, the programme helped in imparting skills and knowledge in children on how to easily
adapt and adjust to the challenges they face as a result of coming from families with persons having
mental illness. The programme facilitated care acclimatization and positive assimilation of the
depression caused by the mentally ill victims from the families (Ward et al., , 2018, p.70). It is
important to note that this program was designed specifically for children in the age group of 8-12 and
13-16 years of age.

IMPACT EVALUATION PROCESS OF SMAILES PROGRAM
Stake holders of the impact evaluation
It is typical that for the effective implementation and execution of any project
activities, there must be an efficient coordination of the stakeholders, both internal and
external. With such an insight, the stakeholders to be involved in the impact evaluation gram
will include the; the organizers (the SMILES support and employees, the youth justice system
support and the participants of the event (the children in the age of 12-16 years of age. The
program is also expected to attract some members of the health ministry and the community
members in which the youth justice system is located (Anwar et al., 2016, p.579)
Evaluation aims and objectives
As it was noted in the preceding section, the program was designed to categorically help the
children within the age of 8-12 and 13-16 years with mothers, brothers, sisters, and brothers who are
affected by mental illness. The proposed Impact evaluation thence is aimed at examining, analyzing
and evaluating the extent to which the smiles programme has impacted on the children coming from
families having mentally ill persons (Smith, Rissel, Shilton, & Bauman, 2017, p. 186). Since the
program centralized and specialized in helping children whose relatives are affected illnesses like
anxiety, bipolar disorder, depression and schizophrenia, the proposed evaluation program formulated
specific aims. The aims included but were not limited to the following;
a) Was the SMILES programme efficiently and effectively implemented in the intended
community
b) What extent did the SMILES program help in changing the attitude, altitude and
social life of the affected children?
c) What were the main barriers/ challenges faced during the implementation of the
program
d) What are the possible solutions to the potentially identified barriers and challenges
faced during the implementation of the programme?
e) Was the program positively welcomed by the intended population or community

IMPACT EVALUATION PROCESS OF SMAILES PROGRAM
Evaluation design
The impact evaluation regarding the implementation of the SMILES program in the youth
justice system will be executed using a well-defined designation manner, which is fundamental to the
extensive and comprehensive understanding in the effective implementation of the SMILES program
in the youth justice system. The study will make use of a non-specific design approach in evaluating
the program. This is because the design methods give chance for the incorporation of cross-sectional
methods post and multi-observational adjustment during the implementation process. The methods
will help in the measurement of the essential elements of the impact evaluation (Smylie et al., 2016,
p.143).
A series of other related programs have initially identified the essential elements of impact
evaluation, among which the elements include exposure analysis, satisfaction, delivery, and
participation. It is, however, important to note that content exposure helps in the determination and
measurement of efficacy test regarding the extent to which the program reaches the targeted group or
sample as well as the relevance of the program to the group. The participation focuses on the
identification and measurement of the satisfaction rate and representativeness of the group.
Additionally, the delivery methods are fundamental in the assessment of future dissemination and the
fidelity program while the contextual elements identify the processes in which the program engages
involved stakeholders as well as the participants of the programme. A holistic understanding of
appropriateness and quality of the impact evaluation can be obtained using the all the mention
methods (Levy, et al., 2017, p. 17)
The acknowledgment of other evaluation design is also important as it helps in the
identification of the strengths and weaknesses of the chosen design and further analyzing the basis
why they are not suited for the chosen impact evaluation process. For example, the randomized
controlled trials are well-thought to be appropriate for an extensive methodology firmness as it makes
use of focus and control groups that are arbitrarily allocated. This method helps in strengthening the
external rationality of the impact evaluation finding (Sallnow, Richardson, Murray, & Kellehear,
2016, p. 2011)

IMPACT EVALUATION PROCESS OF SMAILES PROGRAM
In the same way, the use of quasi-experimental design engages the use of naturally occurring
and predetermined groups. Nonetheless, the listed evaluation designs cannot be used in the impact
evaluation of the SMILES program due to the fact that they do not appropriately and effectively suit
the program (Boone, et al., 2017). There are also related ethical concerns that are associated with the
designs which further limits the design method.
Ethical responsibilities
The Australian law is deemed responsible for all the human related research studies in
relation to ethical responsibilities. It clearly defines the rights and limitation of the participants as well
as that of the researchers. The rules, rights, and responsibilities are based on the principle of fair
treatment. The ethical responsibilities of the study help in ensuring recommendable practices among
the involved parties in the impact evaluation processes. It also fosters the consideration of the likely
risks and benefits of fair treatment of the engaged parties or so, while considering the integrity of the
participants. The reflection of ethical responsibilities such as respect for one another, confidentiality
of some information gathered from the field, maintenance of justice among participants and research
merits helps in the promotion of an appropriate structure for molding a convenient impact and
evaluation environment. It, therefore, facilitates ethical quality, reciprocal responsibility, and trust
during the study process (Anwar et al., 2016, p.579)
Keeping in mind the study will engage human participants, it becomes imperative and
significant to maintain ethical practices. The approval of ethical responsibility will be approved and
attained by the in-charge authorities, specifically the community in which the program is being
conducted. A well-defined consent forms will be administered to the study participants through a
plain written statement. The forms will comprehensively explain the details of the study to be
conducted while highlighting the rights an responsibilities of respective stakeholder/ participants.
The possible/ probable risk, likely to arise during the impact evaluation will be highlighted
and given to the study participants in regard to the confrontational nature of the issues that will be
deliberated so as to have a clear understanding of the consequences of the study. This will help in the

IMPACT EVALUATION PROCESS OF SMAILES PROGRAM
mitigation of the anticipated risks by the participants (Heslin, et al., 2017, p. 407). Additionally, the
study participants will also be told the remedies of being involved in the study, for example,
explaining how getting engaged in the study will be beneficial to them.
Data collection method
Just like any other research study, the impact evaluation process makes use of well-defined
data collection methods while applying the most appropriate and applicable framework. This involves
the selection of a sampling method, data collection instruments, analysis and reporting as it cab be
evidenced in the subsequent sections.
The sample
The collection of sample participants will be done internally while considering the
participation zeal among the study population. It is typical that in an organization/ project, there exist
stakeholders with varying interest and participation. With this insight, the sampling method will
consider the more active and interested persons. The roles played by different individuals towards the
achievement of simplifying mental illness, and life enhancement skills’ program will also be a
significant factor to consider when selecting the sample participants (Gaughran et al., 2017, p.413). In
regard to the evaluation objectives, the sample will be selected that best fits the data collection
methods. In other words, the study will make use of purposeful sampling method to ensure the most
convenient and appropriate sample population is selected for the study.
The data collection instruments
For purposes of reliably and dependably collecting the impact evaluation data, suitable
instruments have been shortlisted in consideration to the SMILES aims and objectives. The data
collection will also engage the integration and triangulation of the sources of information/ raw data to
enhance the robustness of the study findings. Exposure and participation will be measured using a
pre-post survey. The survey will be done through the use of both close ended and open-ended
questions in a plane written/ typed form (Ward et al., , 2018, p.70)

IMPACT EVALUATION PROCESS OF SMAILES PROGRAM
Additionally, the study will make use of online platforms as a way of increasing/ widening
the response rate of the study from a comprehensive context. An analysis of the demographic
information regarding the participants will be done to assist in understanding the background of the
impact evaluation study these will include but not limited to age, sex, and culture among others
(White, Burns, Giglia, & Scott, 2016, p. 350). The demographic study will further facilitate the
understanding of the distribution of children facing the prevalent issue of mental illness environment.
The study will make use of focus groups in determining the extent to which the program impacts on
the targeted community. The use of focus groups acts as control groups in determining study the
influence of the study. A unified time interval assessment will be conducted by the educators to
analyze and evaluate the program context and fidelity. It will be accompanied by the use of checklists
and attendance of the participants. The identification of challenges, barriers and pitfalls will further be
attained through the discussions among focus groups in the impact evaluation process (Moore, et al.,
2015, p.350).
Data analysis processes
The data analysis stage will engage the identification and summarization of the compiled raw
data and themes while making use of relevant and applicable statistical tools of analysis. The analysis
will also involve the use of the descriptive method to interpret the qualitative data. In other words, the
data analysis will engage both quantitative and qualitative analysis. It will assist in striking a balance
between objectivity and subjectivity of the data collected. Different research scholars have
recommended different analysis method as being the most reliable and dependable. It is, however,
important to note that the mode of analysis is solely determined by the study mode. Thence, since the
impact evaluation process will gather both statistical and descriptive data, it makes it imperative to
use both/ mixed method (Ward et al., , 2018, p.70).
The procedures
The study will follow a well-defined procedure, right from the initiation to the dissemination
of the evaluation impact. This will be done as follows.

IMPACT EVALUATION PROCESS OF SMAILES PROGRAM
a) Submitting the study aims and objectives of the project's sponsors in order to lobby
for funds
b) Submission of application for the ethical approval from the responsible authorities
prior to the kick-off of the project
c) Invitation of the project participants to participate in the impact evaluation study
d) Admission of the pre-program survey to the potential participants of the study wit in a
period of 2 weeks
e) Observation of all the sessions from week two to week six of the study by the
educator, accompanied by the end of session discussion.
f) Administering the post program, the survey will be conducted in the seventh week.
g) Conducting of the focus group will be done on the seventh to eight weeks. This will
be done in relation to the number of participants available for the study
h) On the 9th week, the tallying and analysis of data using statistical tools will be
undertaken
i) Five months after the impact evaluation of the SMILES program, a post-survey will
be undertaken to ensure the effectiveness of the programme.
j) Re-evaluation of the study to identify the gaps in the findings of the study
k) Use of strategies identified as most recommendable from the dissemination section to
disseminate the final reports/ findings.
Dissemination strategy
As a way of ensuring maximum and optimistic realization of the impact evaluation study
findings, the communication of the carefully analyzed findings, as well as the recommendations, will
be undertaken using reactive and proactive channels. This will be made in association with the
specified age group of 12 to 16 yeas as initially highlighted by the study age group target. The
dissemination will also involve the engagement of the relevant stakeholders when addressing the
identified gaps in the impact evaluation study (Ward et al., , 2018, p.70).

IMPACT EVALUATION PROCESS OF SMAILES PROGRAM
Documentation of the report compiled from the impact evaluation will be made using a
format of 1:3:25. The final document will be sent to the important stakeholders of the project,
including schools catering to children having mentally ill relatives in their homes. Additionally, the
report will also be established in public journals to improve and strengthen on the evidence-based
practices regarding the program.
Timeline
Task Name Duration Start Finish Predecessors
Submitting the study aims and
objectives of the project's sponsors in
order to lobby for funds
Seven
days Sun 7/1/18 Mon 7/9/18
Submission of application for the
ethical approval Ten days Wed 8/1/18 Tue 8/14/18 1
Invitation of the project participants
to participate 15 days Wed 8/15/18 Tue 9/4/18 2
Admission of the pre-program survey
to the potential participants of 21 days Thu 9/6/18 Thu 10/4/18 3
Observation of all the sessions from
week two to week six of 40 days Mon 10/8/18 Fri 11/30/18 4
Administering the post-program,
survey 20 days Fri 11/30/18 Thu
12/27/18 5
Conducting of focus group 15 days Fri 12/28/18 Thu 1/17/19 6
tallying and analysis of data using
statistical tools 24 days Fri 1/18/19 Wed
2/20/19 7
evaluation of the SMILES program 22 days Thu 2/21/19 Fri 3/22/19 8
Re-evaluation of the study to identify
the gaps 54 days Tue 3/26/19 Fri 6/7/19 9
dissemination 15 days Tue 6/11/19 Mon 7/1/19 1,10
Gantt chart

IMPACT EVALUATION PROCESS OF SMAILES PROGRAM

IMPACT EVALUATION PROCESS OF SMAILES PROGRAM
References
Costello, M., Taylor, J., & O’Hara, L. (2015). Impact evaluation of a health promotion-focused
organisational development strategy on a health service’s capacity to deliver comprehensive
primary health care. Australian journal of primary health, 21(4), 444-449.
Flood, J., Minkler, M., Hennessey Lavery, S., Estrada, J., & Falbe, J. (2015). The collective impact
model and its potential for health promotion: overview and case study of a healthy retail
initiative in San Francisco. Health Education & Behavior, 42(5), 654-668.
Smith, B. J., Rissel, C., Shilton, T., & Bauman, A. (2017). Advancing evaluation practice in health
promotion. Health Promotion Journal of Australia, 27(3), 184-186.
Smylie, J., Kirst, M., McShane, K., Firestone, M., Wolfe, S., & O'Campo, P. (2016). Understanding
the role of Indigenous community participation in Indigenous prenatal and infant-toddler
health promotion programs in Canada: A realist review. Social Science & Medicine, 150, 128-
143.
Levy, D. T., Cummings, K. M., Villanti, A. C., Niaura, R., Abrams, D. B., Fong, G. T., & Borland, R.
(2017). A framework for evaluating the public health impact of e‐cigarettes and other
vaporized nicotine products. Addiction, 112(1), 8-17.
Boone, P., Eble, A., Elbourne, D., Frost, C., Jayanty, C., Lakshminarayana, R., ... & Reddy, P. (2017).
Community health promotion and medical provision for neonatal health—CHAMPION
cluster randomised trial in Nagarkurnool district, Telangana (formerly Andhra Pradesh),
India. PLoS medicine, 14(7), e1002324.
Sallnow, L., Richardson, H., Murray, S. A., & Kellehear, A. (2016). The impact of a new public
health approach to end-of-life care: A systematic review. Palliative medicine, 30(3), 200-211.
Anwar-McHenry, J., Donovan, R. J., Nicholas, A., Kerrigan, S., Francas, S., & Phan, T. (2016).
Implementing a Mentally Healthy Schools Framework based on the population wide Act-

IMPACT EVALUATION PROCESS OF SMAILES PROGRAM
Belong-Commit mental health promotion campaign: A process evaluation. Health
Education, 116(6), 561-579.
Heslin, M., Patel, A., Stahl, D., Gardner-Sood, P., Mushore, M., Smith, S., ... & Ohlsen, R. (2017).
Randomised controlled trial to improve health and reduce substance use in established
psychosis (IMPaCT): cost-effectiveness of integrated psychosocial health promotion. BMC
psychiatry, 17(1), 407.
Gaughran, F., Stahl, D., Ismail, K., Greenwood, K., Atakan, Z., Gardner-Sood, P., ... & Lowe, P.
(2017). Randomised control trial of the effectiveness of an integrated psychosocial health
promotion intervention aimed at improving health and reducing substance use in established
psychosis (IMPaCT). BMC psychiatry, 17(1), 413.
Moore, G. F., Audrey, S., Barker, M., Bond, L., Bonell, C., Hardeman, W., ... & Baird, J. (2015).
Process evaluation of complex interventions: Medical Research Council guidance. bmj, 350,
h1258.
White, B. K., Burns, S. K., Giglia, R. C., & Scott, J. A. (2016). Designing evaluation plans for health
promotion mHealth interventions: a case study of the Milk Man mobile app. Health
Promotion Journal of Australia, 27(3), 198-203.
Ward, M., Schulz, A. J., Israel, B. A., Rice, K., Martenies, S. E., & Markarian, E. (2018). A
conceptual framework for evaluating health equity promotion within community-based
participatory research partnerships. Evaluation and program planning, 70, 25-34.