Evaluating Orthodontic Awareness Study

This assignment critically evaluates a research study examining orthodontic awareness among preadolescents. The analysis focuses on the study's methodology, particularly the sampling techniques and statistical analyses (ANOVA and regression). The critique highlights potential biases in school selection and volunteer participation, suggesting improvements such as two-way ANOVA and ANCOVA for enhanced statistical rigor.

THE SCIENTIFIC METHOD
In this article we critique a journal article related to application of statistical tests, typically t-
test, analysis of variance(ANOVA) and regression analysis. The critique is performed with
respect to the strengths and weaknesses of the scientific research and whether the conduct
and findings of the research have been communicated effectively using the scientic jargon.
Keywords:t-test, regression, anova, statistics
INTRODUCTION
The scientific method is the name given to a series of procedures for obtaining new knowledge or
updating existing knowledge about the varied phenomena which typically involves empirical and
measurable or verifiable source of observation. The father of scientific methodology is said by some
to be the Arab polymath Ibn al-Haytham who first argued the importance of questioning phenomena
and validating them through formally testing them. The principal steps involved in the scientific
method are:(i)detailed observation of events or phenomena, (ii) asking relevant questions about the
cause and/or effect of the phenomena, (iii) formulating testable hypotheses based on the
observations (iv) develop testable predictions based on the hypotheses framed (v) testing the
validity of the predictions using empirical and falsifiable testing procedures (vi) formulating
effective theories, if the tests in the previous step are positive/ reformulate hypiotheses and
predictions and retest those. Thus, we see that scientific method is more of a cyclic and continuous
process over time.(Science Made Simple. 2016). The scientific method, at least since the time of
Galileo, has produced innumerable advancements in human knowledge and thinking, the fruits of
which we are enjoying at the present moment. In this project, though, we look and critique at the
scientific method with reference to the statistical perspective in a typical experimental setup, taking
a published journal article for our reference.
t-TEST, ANOVA AND REGRESSION ANALYSIS
The three most prominent statistical terms that are being used in practice in statistical analyses,
and are being critically examined with respect to their application in this report are: the t-test,
analysis of variance(ANOVA) and regression analysis. It is important to know the details of these
three terms before we try to critique the journal article. In a sense, all the three terms can be
included under the broad category of regression analysis, which is used to compare the statistical
difference between two or more different samples or variables.(Paret, M., 2016) , (Lomax, R. G.
2007)Typically, the analysis is used to find out the relationship(s) between one dependent and one
or more independent variables in a population. In regression analysis, the dependent variable is
assumed to be a function of the independent variables with respect to certain parameters, the
number of which depends of the type of regression we use, like linear or non-linear or multilinear.
In mathematical terminology, we can say that regression analysis hinges on two things:
(i)Correlation, or the problem of finding the form of the function f where
Y =f ( Xi , βi )(1)
where Y is the dependent variable and Xiare dependent variables, βibeing the parameters which is
usually determined in linear and general linear cases by a factor known as correlation coefficent

which is a ratio of product of covariances(deviation from means) to the product of standard
deviations In this step, an additional terminology used is the error in prediction, denoted by ϵiwhich
is minimized by a method commonly known as method of least squares.
(ii) The testing of statistical significance of the above estimated function by calculating the
conditional probability of the model being false and comparing the obtained value with that of a
known standard probability distribution. Two additional terminologies used in this step are the null
and alternative hypothesis. The null hypothesis is the assumption that the dependency of the
dependent variable on the independent variable is not real but accidental and the alternative
hypothesis is that the dependency is actual. In this context, the two most widely used probability
distributions are the t-distribution and F-distribution. When the t-distribution is used, the statistical
test is said to be a t-test, and when the F-distribution is used, the test is said to be an F-test.
The analysis of variance(ANOVA) is determining the interdependence and statistical difference
between a set of more than two variables using an F-test applied to the ratio of variance between the
group of interdependent variables and variance within of group of variables. Note that the statistical
tests like t-test and F-test are not limited only to regression analysis, rather can also be performed on
sample statistics which are assumed to follow the Student’s-t or Snedecor-F distribution.(NLREG.,
2017), (Montgomery, D. C. 2012)
In addition, for the analysis that we will use in this article, we also need some terminology from the
sampling theory. The most common sampling methods used in experiments are: simple random
sampling, stratified random sampling and multistage random sampling. In simple random sampling,
each and every unit in the population is equally likely to be a part of the sample chosen. In stratified
random sampling, the population is stratified, or classified beforehand into strata or classes based on
certain parameters and then, the sample is constructed by simple random sampling from amongst
the strata. In multistage random sampling, the sample is chosen in multiple steps. This type of
sampling is quite useful in cases where location based sampling is essential. In this type of sampling
the population is first divided into regions from which the first simple random sampling is done.
The second stage consists of simple random sampling from dividing the hitherto divided regions
into further smaller regions, and so on. Thus, in this type of sampling, we are more localising the
population to be sampled.(Yale University ., 2017)
THE ARTICLE CHOSEN FOR CRITICAL ANALYSIS
The article that is chosen for this critical report is Pandey, M., Singh, J., Mangal, G., Yadav, P.,
(2014),Evaluation of awareness regarding orthodontic procedures among a group of
preadolescents in a cross-sectional study, Journal of International Society of Preventive and
Community Dentistry,4(1),44-47. The article is an open access article, available on PubMed Central
with PMCID:PMC4015160. In the article, statistical analysis is done regarding the awareness and
know-how of orthodontic and other dental procedures among a group of preadolescents from rural
and urban areas in Bilaspur district, Chattisgarh State, India. The objective of the paper says that the
study was conducted as there was a high prevalence of malocclusion, which is a dental anomaly
characterized by large abnormalities in the tooth position. The method employed by the authors of
the paper was a cross-sectional study of about 1010 subjects(students) with average age somewhere
around 13.02 years with standard deviation of 2.1 years using a questionnaire consisting of

questions pertaining know how of orthodontical procedures. The questionnaire was validated by
using a pilot study consisting of nine items. Finally, to test the statistical significance, t-test and
ANAOVA were conducteed at a significance level of 0.05. The broad results of the study were that
students aged 14 years were more aware than other students, that girl students were significantly
more aware of the orthodontic procedures and that students from urban background were similarly
more aware than their rural counterparts.
THE EXPERIMENTAL DESIGN USED IN THE ARTICLE
The method of design of the study used by the authors requires some mention. The Methods section
of the article says that first, an epidemological survey was taken in the Bilaspur district during the
period between September and December, 2013; and then later a multistage random sampling was
done to select six schools consisting of three in urban and three in rural areas in the same district.
We see that preliminary epidemological survey, done in this case is quite important, as it helps to
determine the local variations amongst the population effectively. Sampling before assessing the
local properties of the population may lead to enormous difficulties in proper data sampling and
may even lead to biased sample. The next thing we note is the use of multistage random sampling to
select the six schools. Note that this is a very crucial part of the reseearch. If simple random
sampling or, stratified random sampling would have been made, there could have been drastic
changes in the result. We note that the method adopted by the authors is well suited to the purpose,
as the population that is used to study is a very large one, and simple random sampling could be
representative of a very small part of population. Stratified sampling, though better than simple
random sampling in this case, gives the problem of stratification of the population, which is another
complex issue having various parameters. Thus, the via media solution is to employ multistage
random sampling that effectively tries to capture the population of Bilaspur district. We also note
that a dual stratification has also been done by the authors by including equal samples from urban
and rural areas. This is because, the urban students, having good access to modern information, may
be well equipped with orthodontic procedures, whereas, those with a rural background may not be
as aware as their urban counterparts. Thus, to ensure evenness, equal samples were selected from
both the urban and rural areas. We also note that pilot study was conducted to validate the
questionnaire beforehand. Pilot study is a small scale reliability test of the sample to evaluate the
feasibility, time, cost, adverse events, and statistical variability of the sample. This is quite essential,
because, in spite of the precautions taken while choosing the sample, like multistage sampling and
stratification into urban and rural parts, it may so happen that the sample is thoroughly biased, for
example, it may so happen that a particular school chosen may be the only one where the students
are exceptionally educated as regards dental procedures due to the fact of there being a qualified
dentist and better infrastructural facility in the school and hence report to the questionnaire more
faithfully and correctly in comparison with other schools, where the students may just randomly
guess the answer to questionnaires based on their whims and fancies. To ensure fairness, a pilot
study is essential before embarking to the full detailed questionnaire. The article states that the
intra-examiner reliability factor, κ =0.87. We note that there is a good intra-examiner reliability, as
the cohen’s kappa is close to 1.(Sapiens 2010). Now, the authors state that after the pilot study was
performed amongst the chosen samples, the main sample was chosen excluding those undergoing
orthodontic treatments. This latter step is added to ensure unbiasedness, as those students would be
naturally more aware of orthodontic procedures.

STATISTICAL ANALYSIS USED IN THE ARTICLE
The article states that after the questionnaire was successfully answered by the volunteer students,
the statistical analysis of the scored data was done using SPSS 16.0 software. The details that were
recorded were descriptive statistics including mean percentage scores, standard deviations,
frequency scores, student’s t-test and ANOVA to test the statistical significance of means, and
finally multiple linear regression. Here, we note the authors are using a complete approach to
statistically analysing the data. The descriptive statistics are as much important to analyse the data
as much as the quantitative tests used. The descriptive statistics proivide us a qualitative description
of the data, which is quite useful to understand a priori as to which statistical test to apply to
investigate the data more thoroughly. Now, we see the results obtained. We see that amongst the
1010 students selected for the study, 556 were boys and 454 were girls, 606 were from urban areas
and 404 from rural areas and the ages varied from 12 to 15 years. We observe that here the
dependent variable is the nominal binary awareness of the orthodontic procedure, whereas the
independent variables are age, location(urban/rural) and gender. From Table1 in the article, we can
also find out that the number of 12 year olds is 254, 13 year olds 200, 14 year olds 252 and 15 year
olds 304. Thus, we see that there are roughly equal proportion of students in each of subcategories
of each of the independent variables. Thus, the sample chosen is well suited to the variables
chosen. The study also revelead that the overall awareness of orthodontic procedures among the
students is about 45.1%. The mean scores of awareness of orthodontic procedures amongst
girls(4.46) with a standard deviation of 1.71 was found to be significantly higher than that of boys
(4.00) with standard deviation of 1.489 using t-test. We need to understand here that the awareness
as a whole gave a rough account of know how of the orthodontic procedures among the students,
whereas the mean scores calculated on the basis of grouping the sample gives us a better command
over the distinction of the same over the various sub-categories of the sample. In addition, the t-test
also gives us conclusive evidence that the mean scores are significantly different and not due to
pure chance alone, which is seen from Table 2 in the article, where p-value is seen to be 0. Though
theoretically p-value cannot be equal to 0, but the zero value is indicative of a very low value in the
SPSS software. What this implies in the context of t-test is that the probability of the difference of
means of the two subcategories in the sample(boys and girls in case of Table 2) being zero
assuming the null hypothesis that they are equal is close to zero. We need to note that the level of
significance of the test, or the threshold value of the value of Student’s-t distribution chosen before
the study began was 0.05. But, the observed value of probability is found to be close to 0, which is
indicative that the original assumption of the null hypothesis that the means of the genders are equal
is false and needs to be rejected. This simple test shows the power of statistical logic, which is not
to be obtained in a casual prima facie study. Similarly, the t-test applied to subcategories according
to location gave significant difference amongst the mean scores in urban(4.43 with standard
deviation 1.606) and rural locations(4.00 with a standard deviation 1.578), with a p-value close to
zero or exactly zero in the SPSS software. The next observation in the results section we need to
consider is the ANOVA table. Here, in addition to the p-value, we have additional column showing
the F-value. What the F-value actually shows is the ratio of between the group variances(the age

groups in Table 1) and within the group variances(variance of awareness in a single age group). We
see that if the null hypothesis that the mean of the groups according to ages are equal, which is
assumed, were true, then the between the group variances would be dominated by or equal to the
within the group variances, thus giving us an F-value of close to 1 or less than 1. But, that this is not
the case is shown by the high value of F-value in all the four subcategories of age parameter, being
well above 1 in all the four cases(seen to be 57.61, 81.48, 69.21, 51.81 respectively for 12, 13, 14
and 15 year olds). This, combined with the p-values being close to zero, proves the significance of
the difference in the mean scores of the different age groups between different, with the observed
mean score of 5.20 with a standard deviation of 0.747 for 14 year olds to be the highest . The last
analysis we take up is the regression analysis between the three different variables of age, location
and gender. We note that the t-test is a subset of F-test(ANOVA) in the two dimensional case, or in
other words, when we compare only two variables or subgroups within a sample, the F-test is
nothing but the t-test as, the Snedecor-F distribution with 1 degree of freedom is the Student’s-t
distribution. Again, the ANAOVA is nothing but the regression analysis of the variances. Thus, in
ANOVA and t-tests done, the authors compared the subcategories in the independent variables of
Age(12 to 15 year olds), Gender(Boys and Girls) and Location(Urban and Rural), in the final
regression analysis they are comparing the mean scores in the three main categories of variables
themselves. Thus, ANOAVA and t-tests were a regression analysis done on the subcategories
within the independent variables. In the regression analysis, we find that age, gender and location,
in that order of preference affect the dependent variable of awareness of orthodontic procedures.
This is ensured by comparing the R2value, which is nothing but the multiple correlation coefficient
computed by taking into account the individual correlation coefficients involving the individual
independent variables with the dependent variable as in a correlation matrix. The R2scores obtained
keeping only age constant is less than that obtained keeping both age and gender constant which is
less than that obtained keeping all the three constant. The F-values being significantly greater than
1, and p-values being close to zero, gives us the validity of the differential effect of the three
independent variables on the dependent variable.
CONCLUSION AND SUMMARY
We see that the authors of the journal article have made a thorough statistical analysis of the
awraeness of orthodontic procedures amongst the student in an attempt to better understand the
prevalence of malocclusion. The way the sample was chosen, the way the validation was conducted,
the statistical analyses conducted and the results and conclusions drawn, were, from a statistical and
analytical perspective, quite appreciable. But, as in statistics, there are bound to errors in the study.
The first error might have crept in from the choice of schools and the volunteers that participated. In
spite of wisdom in sampling and conducting the pilot study, there might have been biases in the
selection of schools due to political, or, similar reasons, like better infrastructure in schools and
enthusiasm in students. It might be the case that within the group variances in the ANOVA and
regression analyses be very small and insignificant as compared to the between group variances
just because of the reason that the volunteers were of a homogenous nature, thus giving ahigh F-
value, in spite of there being no significant difference in the means. Again, this has to do with
sample selection. We think that it might have improved the statistical analysis if two way ANOVA
be done to improve the test and an ANOCOVA(Analysis of Covariance) be performed. (Kass, R.

E . 1 February 2011). Overall, the study made by authors can be a suitable model which could be
implemented in statistical studies.
REFERENCES
Pandey, M., Singh, J., Mangal, G., Yadav, P.,(2014),Evaluation of awareness regarding orthodontic procedures among
a group of preadolescents in a cross-sectional study, Journal of International Society of Preventive and Community
Dentistry,4(1),44-47.
Kass, R. E (1 February 2011). "Statistical inference: The big picture". Statistical Science. Institute of Mathematical
Statistics. 26 (1): 1–9.
Montgomery, D. C.(2012). Design and analysis of experiments (8th Ed.). John Wiley & Sons.
Science Made Simple(2016),The Scientific Method,[ONLINE] Available at
http://www.sciencemadesimple.com/scientific_method.html [Accessed 16/10/2017]
Frost, J. (2016), Understanding Analysis of Variance (ANOVA) and the F-test,[ONLINE] .Avilable at
http://blog.minitab.com/blog/adventures-in-statistics-2/understanding-analysis-of-variance-anova-and-the-f-
test[Accessed 16/10/2017]
NLREG. (2017), Understanding the Results of an Analysis,[ONLINE] Available at http://www.nlreg.com/results.htm
[Acccessed 16/10/2017]
Sapiens(2010).Action without change? On the use and usefulness of pilot experiments in environmental management
[ONLINE] Available at http://sapiens.revues.org/979 [Accessed 16/10/2017]
Paret, M., (2016), Regression versus ANOVA: Which Tool to Use When, [ONLINE] Available at
http://blog.minitab.com/blog/michelle-paret/regression-versus-anova%3A-which-tool-to-use-when [Accessed
16/10/2017]
Lomax, R. G. (2007). Statistical Concepts: A Second Course. p. 10.
McLugh, M.L. (2012), Interrater reliability: the kappa statistic, Biochem Med (Zagreb). 2012 Oct; 22(3): 276–282.
Yale University (2017), Sampling, [ONLINE] Available at http://www.stat.yale.edu/Courses/1997-98/101/sample.htm
[Accessed 16/10/2017]