Critical Thinking | Test Manual of WCTT

1. Reliability of a test implies that the test produces consistent results when taken over
different periods of time, across different researchers, and across items. In the present
case of the test manual of WCTT, reliability can be checked through the results of
Study 1. The same group of participants took the test twice at an interval of 16 days.
Here, the correlation was found to be 0.89. As a norm, correlation of +0.80 or higher
is a good indicator of reliability. Reliability in WCTT implies that the students who
scored high on critical thinking in week one would be tested high on critical thinking
after two weeks or two months as well. Critical thinking as a competency is largely a
variable like personality factors or intelligence that would be consistent over time
unlike moods or emotions that may vary across the time dimension. Hence, a high
reliability is to be expected in such a test. A study conducted at a different institute in
Study 4 yielded Cronbach alpha coefficient of 0.78 which is a good indicator of
internal consistency and reliability. In social science research, such as the present
one, a Cronbach alpha coefficient of 0.70 or higher is considered acceptable, when
there are dichotomous questions that is questions with two acceptable answers. This
measure of reliability indicates that the scale measures what it is supposed to measure,
which is critical thinking in this case. This measure is also useful in ensuring
reliability of latent or hidden variables.
2. Along with reliability, it is important to measure the validity of a test which is a
measure of whether the test measures what it truly represents. Content validity
measures the extent to which the tool includes the concept in its entirety. WCTT
includes four indicators to include critical thinking. Content validity is assessed by
measuring the tool against the conceptual definition. In the case of WCTT, there is
not much evidence suggesting that Wagner did a content validity of his tool. A good
way to measure content validity of WCTT would have been to consult several subject
matter experts to gauge whether the test covered all essential measures of critical
thinking. In the present case, Wagner based his test manual on a review of extant
literature and his judgment, but the efforts taken for ensuring content validity have not
been mentioned in the case. Content validity is an essential component in
determining the validity of an instrument since it ensures that the test measures what it
intends to inn its entirety. For example, a panel of experts may review the test and
find that a component such as maybe ‘problem solving’ is missing in the test manual
and that would make it difficult to attain generalizability of the WCTT. Content
validity is also necessary to ensure that there is no bias in the measurement and that

all indicators are measuring the relevant aspects of critical thinking. It can be
understood as a more formal measure than that of face validity.
3. Criterion validity is a measure of well the test scores are correlated with other
measures that may be known to be related to measure. For example, one who scores
higher on critical thinking may be expected to score high on motivation to learn. In
the case of WCTT, Study 2 measured the scores of critical thinking along with several
other related criteria or measures. It can be inferred that WCTT has a moderate
correlation with the Watson-Glaser Critical Thinking Appraisal (r=.61, p<.001) and
the Peak Verbal Ability Test (r=.43, p<.001) while a low or weak correlation with no
statistical significance with Need for Achievement (r=0.6, ns). This indicates that
score on critical thinking can be used to some extent to predict performance on
Watson-Glaser Critical thinking. It also can be interpreted that someone with a high
score on WCTT would have a high score on the test of their verbal ability. If critical
thinking measured through WCTT was being used as a predictor for one’s need for
achievement, the score and absence of statistical significance would indicate that it
would not be valid. In criterion related validity, it is important to test for those
measures or criteria that would be expected to be correlated to the construct of
interest. For example, critical thinking score may have a high correlation with high
IQ but it may not necessarily imply that one is predictive of the other. This measure
indicates how well the construct of interest has been operationalized by the researcher
by measuring it against already established criteria.
4. Construct related validity of WCTT would essentially indicate the extent to which the
test can be generalized that is how much the sample selected would be an accurate
representation of the population. The WCTT is supposed to measure Critical
Thinking. Through the measurement of different types of validity such as face and
content validity, and criterion-related validity (measured through predictive,
concurrent, convergent, and divergent validity), construct validity can be established.
In addition to the measures stated above, the factor analysis shows weak loadings of
factors 3 and 4 that is IEA and AIC. There could be various reasons attributed to this.
Additionally, Wagner used simple language structures in his instrument which may
appeal to taking the test accessible across cultures. The evidence is incomplete to
measure construct validity. The data from confirmatory factor analysis is a good
indicator of the nature of the construct being measured. The present evidence lacks
important measures of goodness of fit, comparative fit index, and Chi-squared test.