Quasi-Experimental Study on the Effectiveness of 360 Literacy Program

This report presents a quasi-experimental study designed to evaluate the effectiveness of the 360 Literacy program on at-risk high school students. The study investigates the program's impact on academic achievement (measured by GPA), school attendance, and classroom behavior (measured by teacher behavior referrals, in-school suspensions, and out-of-school suspensions). The research questions explore the extent to which the 360 Literacy program influences these factors, and hypotheses are formulated to test for significant differences before and after participation in the program. The methodology involves a quantitative approach with a pretest-posttest design, comparing data collected before and after the implementation of the 360 Literacy program at Arts Magnet High School of South Carolina. The study aims to provide insights into the program's effectiveness and inform funding decisions for similar interventions.

Running head: RESEARCH METHODOLOGY
A Quasi Experimental Study on the Effectiveness of the 360 Literacy Program at a Title 1
High School
Name of the Student:
Name of the University:
Author note:

1
RESEARCH METHODOLOGY
Chapter 3: Research Methodology
1. Introduction
After school programs (ASPs) are those organized programs or activities that the school
students can participate outside the conventional school hours. In many cases, these programs are
organized or run by the primary or secondary school itself, while sometimes externally funded
commercial or non-profit organizations run such programs for the personality and skill
development of the students. As stated by Check (2013), the ASPs are meant for the children and
youth communities of all ages and these include a wide range of focus areas. These areas
includes the academic support, youth development, coaching and mentoring, sports, arts and
recreation. The author also stated that the activities after the school hours can help the children
and the youth are quite significant in their personality development. A similar view was put
forward by Posavac (2015). It is said that the high standard after school programs result in
positive outcomes for the students in terms of enhanced academic performance, interpersonal
skills, classroom behavior, nutrition and health. It is also beneficial for the businesses and the
communities when the youth and the children enjoy safe as well as productive environment and
methods of learning and recreation while their parents are at work (Mayer & Jimerson, 2019).
Thus, there are various ASPs that are supported by the federal government for the benefit of the
young generation of the nation.
These ASPs are designed for children and youth of the communities and hence a detailed
planning is required for designing different types of beneficial ASPs. It is a matter of concern for
the educators, parents, researchers and policy makers. Hence, various ASPs are designed with
diverse purposes, such as, academic skills enhancement, cultural enrichment, development of
discipline, punctuality and other interpersonal skills (Schargel & Smink, 2014). Thus, many

2
RESEARCH METHODOLOGY
schools offer after school care services, which not only beneficial for the students to learn many
things, but also for the working parents. A large number of students who have working parents
return to an empty home after school, which is not beneficial for the mental health of the
children. Thus, both the parents and the school authorities encourage the after school activities to
keep the students engaged in constructive work (Encyclopedia.com, 2019). It has been found that
these activities help the children to stay in a safe, protected and structured environment, with
skill enhancing activities. As per the experts, the ASPs enhance academic performance, improve
the self-confidence and social skills, self-care skills, and many other personality development
aspects.
Two types of categories can be mentioned in case of ASPs namely, stand-alone programs
and broad based programs. The stand alone programs refer to the after school programs which
focus on a single type of programs or service primarily. The broad based services refer to the
combination multiple activities. These activities can be fee based, academic programs,
community learning activities, or sports and recreational activities (Bernhardt & Bernhardt,
2013). However, despite following different structures, these activities are aimed at improving
the personal profile of the children and youth.
It has been found that the USA federal government spent $1.21 billion in the year 2017-
2018 (Alliance, 2018). This is quite a substantial amount of investment in designing and
implementing the ASPs across the elementary and high schools across the country. However,
according to Posavac (2015) it has been found that there are many cases where the programs are
not designed for the children and youth with special needs. They require all the more attention
and care and programs for self-development. Lack of trained guides and teachers often
contributes in this problem. Lack of awareness is also a major problem when it comes to the

3
RESEARCH METHODOLOGY
needs of the special children. Thus, there is a waste of funding from the federal government for
the ASPs, which are meant for all children and youth of the country. Another information stated
by Karner (2017) in his book that the ASPs implemented in the schools did not improve the
student’s behavior, rather the impact was worse as the planning and design of the programs were
not suitable for the actual requirement of the students. The 21st century programs also include
food, recreation and other developmental activities, but those also failed to improve the academic
performance of the students. Varenne (2018) also put forward a similar point of view. It has been
stated that till date $12 billion of the federal tax money was spent on the education system and
designing the ASPs, however, it did not bring the desired outcomes in terms of improving the
performance and skills of the children. Sometimes, the fees charged by the schools for after
school care is quite high and many families cannot afford that, resulting in failure of the purpose.
The purpose of this research study is to explore and evaluate the success of the 360
Literacy program for enhancing the academic achievement of the students, school attendance and
reducing the externalizing of the behavior among the high school students by applying the quasi-
experimental studies. If the program is found to be effective, the recommendations will be made
to the schools and to the district officials regarding funding of the programs.
2. Research questions
RQ1. To know what extent does the 360 Literacy program impact the academic achievement as
measured by GPA of at-risk high school students who attend afterschool programs regularly?
RQ2. To know what extent does the 360 Literacy program impact the school attendance of at-
risk high school students who attend afterschool programs offered regularly?

4
RESEARCH METHODOLOGY
RQ3. To know what extent does the 360 Literacy program impact the classroom behavior as
measured by the number of teacher behavior referrals, in school suspensions (ISS), and out of
school suspensions of at-risk high school students who attend afterschool programs offered
regularly?
3. Research hypotheses
Hypothesis 1
H10. There is no significant difference in the academic achievement as measured by GPA of at-
risk students before and after participation in the 360 Literacy program.
H1a. There is a significant difference in the academic achievement as measured by GPA of at-
risk students before and after participation in the 360 Literacy program.
Hypothesis 2
H20. There is no significant difference in the school attendance of at-risk high school students
before and after participation in the 360 Literacy program.
H2a. There is a significant difference in the school attendance of at-risk high school students
before and after participation in the 360 Literacy program.
Hypothesis 3
H30. There is no significant difference in the behavior of at-risk high school students before and
after participation in the 360 Literacy program.
H3a. There is a significant difference in the behavior of at-risk high school students before and
after participation in the 360 Literacy program.

5
RESEARCH METHODOLOGY
This chapter will present an in-depth description of the research methodology and designs
to be followed by the researcher to conduct the study. Research methodology refers to the set of
assumptions and the plans and actions to be undertaken to address the research questions in the
most logical manner and it explains the process of data collection and analysis.
4. Research Methodology and Design
In this given study on the effectiveness of the 360 Literacy Programs on the academic
achievement, school attendance and externalizing behaviors of the at-risk high school students,
quantitative quasi-experimental study design will be adopted. This type of study represents a
form of experimental research majorly used in the psychology and social sciences. Letourneau et
al. (2015) highlights about quantitative quasi-experimental study represents empirical
interventional study, which is used for estimating the causal effect of an intervention on a target
population or sample without having any random assignment. It manipulates the independent
variable but the participants are not chosen randomly and assigned the orders of the conditions
randomly (Campbell & Stanley, 1963). As the independent variable is manipulated before
measuring the dependent variable, this study design helps in eliminating the directionality
problem of the study. However, as the participants are not assigned randomly, the quasi-
experimental study does not eliminate the problem of confounding variables. Thus, it can be said
that, regarding internal validity, the study of quasi-experiment lie in between the true
experiments and correlational studies (Hartas, 2015).
As stated by Krishnan & Sitaraman (2013), the quasi-experiments are generally
conducted in the field settings in which random assignment is not possible. This type of study is
chosen for evaluating the effectiveness of any treatment or programs, such as, any psychological
therapy or educational intervention, as seen in the given case. Under this research design, the

6
RESEARCH METHODOLOGY
comparison between the pre-test and post-test design occurs. Nielsen, Randall & Christensen
(2017) stated that in the pretest-posttest design, the dependent variable of the study is measured
once before the intervention treatment is imposed and once after it was implemented. Thus, the
independent variable is controlled in a manner that is non-random before and after the
implementation of the intervention treatment. It generates within subject experiment, but the
order of the conditions is not counterbalanced as it is typically not possible to test the participants
under the treatment or intervention condition before the untreated condition (Cook & Campbell,
1979). If the posttest score is better than the pretest score, then it is concluded that the treatment
or intervention is impactful on the dependent variable. However, there are other factors also that
can cause the improvement of the posttest score, such as, exposure to historical information or
maturity of the participants, which might help them to behave differently during pretest and
posttest situation (D’onofrio et al., 2013).
It can be elaborated as the pre test data on the academic performance and other factors
will be compared with the post test data, that is, the information collected after the imposition of
the test condition, that is, the 360 Literacy programs, and the findings will be helpful in
understanding the impact of the intervention program on the dependent variables. As in the
given study, quail-experimental study will be applied with the pretest and posttest condition for
the research. As the impact of the 360 Literature Programs as an intervention for the
improvement of academic performance, school attendance and externalizing the behavior of the
at-risk students of the Arts Magnet High School of South Carolina will be evaluated, the
application of quasi experimental study will be most suitable. This study design allows the
researcher to examine the impact of the intervention on the participants in a controlled manner,
which is not random and also get the findings of the research questions by comparing the pretest

7
RESEARCH METHODOLOGY
and posttest information (Shadish, Cook & Campbell, 2002). This helps to get precise findings
about the intervention treatment on the participants. As numerical data will be collected and
analyzed, quantitative methods of study will be applied.
Different types of quantitative research study designs other than quasi-experimental are
descriptive, correlational, and experimental. Quantitative research designs are applied to evaluate
the relationship between the variables that deals with numbers and the statistics for explaining
and analyzing the findings using statistical tools (Turner, Balmer & Coverdale, 2013). Under
descriptive research design, the researcher describes the current status of the research
phenomenon and this method does not require a research hypothesis to begin with. The analysis
is obtained from the existing data (Chudleigh & Smith, 2015). Correlational research design is
used to find out any type of association between the dependent and independent variables
through statistical study and the population can be chosen randomly and there is no intervention
or treatment, for which pretest and posttest conditions are present (Curtis, Comiskey &
Dempsey, 2016). The internal validity is lower in correlational study in comparison to quasi-
experimental study. Lastly, under experimental study, the experiment is conducted with random
assignments. In other words, the aspect of randomness is more in the outcome and the variables
are not manipulated. In other words, in the true experiment, the participants are assigned
randomly to the treatment or to the control group, which does not happen in case of quasi-
experiment, in which the assignment does not happen randomly. The interval validity is more in
case of experimental study than for the quasi-experimental study.
Thus, it can be said that, quasi experimental study is more suitable for educational
research as the students or the teachers are not usually able to be assigned randomly to the

8
RESEARCH METHODOLOGY
schools, classrooms or districts only for the research purpose. Thus, in the given study, the
research questions will be best answered with the help of quasi experimental study.
5. Population and sampling
In the given study, the study will be performed at Arts Magnet High School in South
Carolina, where there are 1413 students from grade 9 to 12 and 108 teachers in 2017, as per the
data of South Carolina Department of Education. Thus, the population of the study consists of all
the high and middle school students off Arts Magnet High School of South Carolina, who
attended the afterschool program, 360 Literacy, at the high school in the year 2016 and 2017.
More than 100 students attended this program, however, only 30 students are chosen for the
study, who did not attend the program in 2016 but attended in 2017. The population is relevant
as the students who have not undergone the ASP, cannot produce accurate test result for the
study.
Convenience sampling technique is used for choosing the sample of 30 students.
Sampling techniques are of two types, namely, probability and non-probability. Under
probability sampling, the selection of the sample is random and each has equal chance of being
selected, while under non-probability sampling, the selection of each sample is not equal and
random (Acharya et al., 2013). Convenience sampling technique is a non-probability sampling,
in which the subjects or the samples are chosen on the basis of their proximity and convenient
accessibility to the researcher. Thus, as stated by Etikan, Musa & Alkassim (2013), in the
convenience sampling, no inclusion criteria is usually identified or applied for selecting the
subjects. In the given study, the researcher applied convenience sampling technique to choose 30
students to test the impact of the testing intervention, that is, the 360 Literacy program in the
high school. This sample will feature the same characteristics as the population for the study and

9
RESEARCH METHODOLOGY
hence, will produce precise outcome to the research questions. The number of subjects, that is,
30 is decided on the basis of convenience of accessibility. All the 30 students will be contacted in
their school and will be asked to take part in the quasi experimental study.
6. Materials/Instrumentation
The materials of the study consist of the data on the 30 chosen students are obtained from
the high school. The GPA of the academic performance, discipline and the attendance records of
the students are secondary data and those will be obtained from the school guidance office. For
this data, no survey will be conducted and no students and teachers will be involved. Hence, as
primary data will not be required, hence no instruments for primary data collection, that is,
survey questionnaire, interviews, tests, or observations will be used. As the secondary data will
be collected from authentic school records, thus, the reliability and validity test will not be
conducted. Since, primary data is not involved, no pilot or field testing is required.
7. Operational Definitions of Variables
Regular attendance to 360 Literacy- a student is a regular attendee to the program if they
attend all 4 days of the program.
360 Literacy- this is an after school program (ASP) implemented by the Arts Magnet
High School, South Carolina for the high and middle school students to improve their academic
performance, attendance and behavior of the students.
Behavior write-up- this refers referral written by a teacher or staff on a student as a result
of their disrupting or inappropriate behavior in and out of the classroom.

10
RESEARCH METHODOLOGY
Regular school Attendance- according to the School Attendance Law of South Carolina
and the school year consists of 180 days and to receive credit, a student must attend at least 170
days out of 180 days (Sansani, 2015). Lawful absence is granted on the basis of documentation
for chronic illness or accidents or emergencies.
Externalizing behavior- this refers to the problem behavior of the students directed
towards the external environment in the school. This includes physical aggression, destruction of
the property, disobeying rules, cheating etc.
These variables are included in the research questions and hypotheses and all the
observations before the test or intervention is the independent variables, while those becomes
dependent posttest. 360 Literacy plays the role of mediator in these hypotheses.
8. Data Collection and Analysis
As stated earlier, secondary data will be collected from the School Guidance office and
the pretest and posttest information will be compared using quantitative method of analysis.
Statistical functions will be applied on the datasets for the year 2016 and 2017. Frequency
analysis, comparison of means, such as, paired t-test, cross tabulation and correlation functions
will be applied on the datasets to find out the difference between the scores of GPA, school
attendance and behavior of the students who attended the 360 Literature Program in 2017 but not
in 2016.
9. Assumptions
It is assumed that there will be difference in the scores and their means for the students
for the year 2016 and 2017 due to their enrolment in the ASP that is 360 Literacy. It is also

11
RESEARCH METHODOLOGY
assumed that primary data will not be suitable as perspective of the students will vary regarding
the test variables and they are not mature enough to understand the implications of the ASP so as
to report truthfully and accurately. Thus, secondary data will be collected from the School
Guidance Office, that is, from an authentic source so that the data is valid and reliable.
10. Limitations
Time is the limitation for this cross sectional study and to mitigate the limitation, the
researcher has chosen to collect only the secondary data for only 2 years that is 2016 and 2017.
Although the program is in-effect for quite a few years due to time constraints primary data and
data for few more years will not be collected and analyzed, which could have resulted in more
accurate outcome.
11. Delimitations
The research objectives, population of the study, variables taken, and analysis process are
the delimitations for the study. The objectives are set in a manner to narrow down the research
horizon, and conduct the study in a concise manner and controlled direction so that it does not
become robust. The population was chosen as per the subject of the study, that is, the students
who have attended the ASP for the year 2016 and 2017. The variables are also chosen according
to the purpose of the study and accordingly the quantitative analysis process is chosen.
12. Ethical Assurances
The researcher will collect approval from the University’s Institutional Review Board
(IRB) prior to the data collection. Anonymity will be maintained strictly as no personal data of
the students will be collected while collecting the data on the academic GPA, attendance and

12
RESEARCH METHODOLOGY
behavioral record. The data will be used only for academic purpose and will be stored with the
University as per the IRB standard. The researcher will conduct the study without any biasness
towards the any of the variables.
13. Summary
From the above discussion, it can be said that, to conduct the study on the impact of ASP
on the academic performance, behavior and attendance of the students of Arts Magnet High
School, quantitative quasi-experimental research design will be chosen and by applying
convenience sampling, 30 students will be chosen from the population of all students from the
high and middle school. Secondary data on the GPA and other variables will be collected for
these 30 students from the school guidance office and quantitative analysis will be performed to
get the desired outcome and fulfill the research objectives.

13
RESEARCH METHODOLOGY
References
Acharya, A. S., Prakash, A., Saxena, P., & Nigam, A. (2013). Sampling: Why and how of
it. Indian Journal of Medical Specialties, 4(2), 330-333.
Bernhardt, V. L., & Bernhardt, V. (2013). Data analysis for continuous school improvement.
Routledge.
Campbell, D. & Stanley, J. (1963). Experimental and quasi-experimental designs for research.
Chicago, IL: Rand-McNally.
Check, K. (2013). Comprehensive School Physical Activity Programs A Guide for Schools.
Chudleigh, J., & Smith, J. (2015). Research essentials. Nursing Children and Young People
(2014+), 27(3), 12.
Cook, T. D., & Campbell, D. T. (1979). Quasi-experimentation: Design and analysis issues for
field settings. Boston, MA: Houghton Mifflin Company.
Curtis, E. A., Comiskey, C., & Dempsey, O. (2016). Importance and use of correlational
research. Nurse researcher, 23(6).
D’onofrio, B. M., Lahey, B. B., Turkheimer, E., & Lichtenstein, P. (2013). Critical need for
family-based, quasi-experimental designs in integrating genetic and social science
research. American journal of public health, 103(S1), S46-S55.
Encyclopedia.com. (2019). After-School Programs. Retrieved from
https://www.encyclopedia.com/children/applied-and-social-sciences-magazines/after-
school-programs

14
RESEARCH METHODOLOGY
Etikan, I., Musa, S. A., & Alkassim, R. S. (2016). Comparison of convenience sampling and
purposive sampling. American journal of theoretical and applied statistics, 5(1), 1-4.
Hartas, D. (Ed.). (2015). Educational research and inquiry: Qualitative and quantitative
approaches. Bloomsbury Publishing.
Karner, C. (2017). The urban school: A factory for failure. Routledge.
Krishnan, S. S., & Sitaraman, R. K. (2013). Video stream quality impacts viewer behavior:
inferring causality using quasi-experimental designs. IEEE/ACM Transactions on
Networking, 21(6), 2001-2014.
Letourneau, N., Secco, L., Colpitts, J., Aldous, S., Stewart, M., & Dennis, C. L. (2015). Quasi ‐
experimental evaluation of a telephone‐based peer support intervention for maternal
depression. Journal of advanced nursing, 71(7), 1587-1599.
Nielsen, K., Randall, R., & Christensen, K. B. (2017). Do different training conditions facilitate
team implementation? A quasi-experimental mixed methods study. Journal of Mixed
Methods Research, 11(2), 223-247.
Posavac, E. J. (2015). Program evaluation: Methods and case studies. Routledge.
Sansani, S. (2015). The differential impact of compulsory schooling laws on school quality in the
United States segregated South. Economics of Education Review, 45, 64-75.
Schargel, F., & Smink, J. (2014). Strategies to help solve our school dropout problem.
Routledge.

15
RESEARCH METHODOLOGY
Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental
designs for generalized causal inference. Boston, MA: Houghton Mifflin.
Turner, T. L., Balmer, D. F., & Coverdale, J. H. (2013). Methodologies and study designs
relevant to medical education research. International Review of Psychiatry, 25(3), 301-
310.
Varenne, H. (2018). Successful failure: The school America builds. Routledge.
MAYER, M. J., & JIMERSON, S. R. (2019). THE IMPORTANCE OF SCHOOL SAFETY AND VIOLENCE
PREVENTION.
Alliance, E. (2018). Federal afterschool funding update, June 27. Retrieved from
https://www.afterschoolalliance.org/afterschoolsnack/Federal-afterschool-funding-
update-June-27_06-27-2018.cfm