Experimental Design Methods

This article discusses the Pretest-Post-test design and Factorial design in Experimental Design Methods. It explains the importance of experimental design in scientific research and how it can be used to investigate the effectiveness of research in various fields. The article also discusses the threats that may affect the internal validity of an experiment and the statistical analysis of data. Hypotheses for a study on the effectiveness of the Inquiry method of teaching in enhancing Inductive reasoning among primary school children are also formulated.

Experimental Design Methods

Introduction
The Pretest-Post-test design is an experimental design that has subjects being randomly assigned
to either the experimental or the control groups 1. Pretest is conducted for both groups on the
independent variable. Treatment is applied on the experimental group while the control group
receives no treatment. Both the experimental and the control groups are post-tested to evaluate
the impact of manipulation of the independent variable on the dependent variable.
However, an advanced form of this design which allows for testing of two or more hypotheses is
the factorial design 2. The factorial design allows for manipulation of two or more independent
variables to elicit response on the dependent variable. Therefore, two or more hypotheses can be
tested on a single experiment.
The experimental design for this task is therefore a pre-test-post-test design since it has only one
factor. The factor involved is the groups (experimental or control). This method can be applied in
scientific research to help in determining how effective the research would be 3. It can be seen as
a clear way of knowing the accuracy of the question at hand. The experimental design method is
useful in the investigation of the effectiveness of research in many fields such as in agriculture,
medical, and education. In education, researchers can use the method to help them in the
observation of a particular occurrence and recommend the course of action to be taken in
consideration to the result from the research carried out 4.
The table indicates the pre-test-post-test control group method where subjects have been
randomly assigned to the treatments and control groups. Pre - test analysis on the independent
variable for both the experimental and control groups while post - test analysis is only carried out
on the experimental group since the control group is not subjected to treatment.
1
Reason why the above design is a good experimental design
1 Hall Ashley, Experimental Design: Design Experimentation (2011
2 Siddiqui, Adaptive sequential experimentation technique for 3³ factorial designs based on revised simplex search
(2010)
3 Wang, Applied scientific research (2012).
4 Ping, Jian, Data Processing in Scientific Research Performance Evaluation of University Based on DEA and
Supervision Measures: Advanced Materials Research (2014).

The Pre- Test Post-Test design is a good experimental design for investigating the effectiveness
of the inquiry method in enhancing Inductive reasoning among primary school children. This is
because the Pre- Test Post- test experimental research design allows for the random selection of
groups 5. In this design, pupils will be randomly selected and put into either the experimental
group or control group. The pupils who fall in the experimental group are subjected to the
treatment, which is the Inquiry method of teaching. However, pupils in the control group are not
taught using the Inquiry method.
Pre- test analysis is carried out on the experimental and control groups to assess their
performance. Post- test analysis is also carried out to investigate the effect of the Inquiry method
of teaching on pupils’ academic performance.
The Pre- Test Post- test design eliminates the possibility of bias in selection of subjects into
groups due to the random assignment of pupils to the Inquiry method of teaching. This type of
research design also eliminates the effects of confounding since only one factor is involved 6.
The effect of only the Inquiry method of teaching in enhancing inductive reasoning among
Primary school children is evaluated.
Hypotheses for the study;
Since the study tries to investigate the effectiveness of the Inquiry method of teaching in
enhancing Inductive reasoning on primary school children, we compare the post- test results of
the experimental group and those of the control group. We therefore formulate the following
Hypotheses:
1. H0: The distribution of responses is the same across the 4 weeks-time period.
2. H0: Pupils who are subjected to the Inquiry method of teaching do not perform differently
from pupils who are not subjected to the inquiry method of teaching in terms of inductive
reasoning.
2
Threats that may affect the internal validity of the experiment
25 Hildebrand, Martin, Random walks on random simple graphs (2016).
6 Cinelli, Francesco, Confounding in field experiments: (2012).

There are several threats that may affect the internal validity of a Pre- Test Post- test experiment.
Some of these threats are as discussed below;
i. The experimental design could be faced with assignment bias. Although subjects are
randomly selected into the experimental and control groups, the effect of individual
differences within the experimental and control groups is not explored 7. When
subjects are selected for research in a biased manner, there are high chances that the
results will not be accurate. Therefore, if half of the subjects are supposed to be
selected, it will be in order to select more than 25% of the subjects so that majority of
the subjects with diverse characteristics are included in the experiment. Failure to rule
out assignment bias could result to a weak statistical power.
When evaluating the effect of a treatment on a sample of subjects in an experimental
study, it is important that the experimental and control groups be as similar as
possible. The groups should only differ in respect of the intervention of interest to be
applied to the experimental group. A common technique to ensure that subjects are as
similar as possible for the experimental and control groups is subject selection by
randomization. As long as the sample sizes are large enough, the effect of individual
differences on the intervention applied is ruled out.
An alternative option of ruling out the effect of individual differences is by applying
more than one treatment to all subjects both in the control group and the experimental
group 8.
ii. Another threat that may affect the internal validity of the experiment is
instrumentation. Instrumentation refers to the change in the characteristics of the
measuring instrument over time 9. When pupils are used to evaluate the effectiveness
of a teaching method, they gain knowledge over time or change their attitudes
towards that method. Interpretation of results based on the Pre - test Post- test
experimental study design might therefore not always be accurate.
When a good instrument is used in the test, accurate results will be achieved during
the pre-test and post-test. More importantly, using an instrument which is not a
treatment in the post-test will give a better performance than when the instrument is
used as a treatment.

Statistical analysis of data
It is necessary to conduct statistical analysis of the data in order to draw meaningful
inference from the data. Statistical analysis is a combination of statistical operations, both
descriptive and inferential in nature 10.
For our study, it is paramount to conduct statistical analysis in order to investigate whether
the intervention (treatment) had a significant effect on the dependent variable. For the case of
our study, the treatment applied was the inquiry method of teaching. Statistical analysis is
carried out to investigate the effectiveness of the inquiry method of teaching. Before
undertaking statistical analysis, a null hypothesis is proposed. Statistical analysis results
provide a justification on whether or not to reject the null hypothesis.
Statistical analysis of data can also be used to check for presence of outliers and for checking
of normality of data. The results could therefore lead to the rejection or non-rejection of the
hypothesis of non-normality of data.
However, deciding which statistical analysis to carry out depends upon the type of data being
analyzed and the distribution of the data. Appropriate hypotheses are formulated based on
these. Statistical significance levels are set to guide the rejection points and determine the
statistical significance of data.
Rejection of a null hypothesis implies that the test applied there lacks sufficient evidence to
support the null hypothesis.
3
References
Hall Ashley, Experimental Design: Design Experimentation (2011).
37 Middleton, Joel A, Bias of the regression estimator for experiments using clustered random assignment (2012).
8 Sebastian, Testing for heterogeneous treatment effects in experimental data: false discovery risks and correction
procedures: Journal of Development Effectiveness (2014).
9 W. Eberhardt, New developments in experimental instrumentation (2011).
10 Wegman, Edward J, Special issue of statistical analysis and data mining (2012).

Siddiqui, Adaptive sequential experimentation technique for 3³ factorial designs based on revised
simplex search (2010).
Wang, Applied scientific research (2012).
Ping, Jian, Data Processing in Scientific Research Performance Evaluation of University Based on DEA
and Supervision Measures: Advanced Materials Research (2014).
Hildebrand, Martin, Random walks on random simple graphs (2016).
Cinelli, Francesco, Confounding in field experiments: direct and indirect effects of artifacts due to the
manipulation of limpets and macroalgae (2012).
Middleton, Joel A, Bias of the regression estimator for experiments using clustered random assignment
(2012).
Sebastian, Testing for heterogeneous treatment effects in experimental data: false discovery risks and
correction procedures: Journal of Development Effectiveness (2014).
Eberhardt, New developments in experimental instrumentation (2011).
Wegman, Edward J, Special issue of statistical analysis and data mining (2012).