University Exam 4PS201: Scientific Methodology in Statistics

This document presents a comprehensive solution to an exam on scientific methodology in statistics. The exam covers key topics such as estimating replicability through various methods, including direct replication, statistical power, and the z-curve. It also addresses the reasons why scientific studies may be false, based on the work of Ionnidis, and discusses the role of effect sizes, flexibility in research design, and potential conflicts of interest. Furthermore, the solution examines statistical concepts like Type I errors, alpha, sample size, and effect size, as defined by Cohen. It explores the contributions of Fisher and Pearson to statistics, including their methods and the controversy between them. The exam also includes questions on research design, specifically focusing on the impact of globalization on the labor market, with a quasi-experimental design, hypothesis formulation, and the identification of indicators such as trade, foreign investment, and technological advances. The solution also addresses validity threats, including history, participant changes, and instrument changes. Finally, the document concludes with an analysis of the impact of globalization on labor demand and wages, with supporting arguments and indicators.

Running Head: Scientific Methodology 1
Scientific Methodology in Statistics
Name of Student
Name of Professor
Date

Scientific Methodology 2
Question 1
The most direct way of estimating replicability is by conducting the research while using the
original methods and then comparing the new and original results. The data could also be
assessed for indicators of bias, errors and outliers. This can be done through comparing the
reported data against a database that had previously reported values (Roediger, 2012). This can
be used to show outlying data and thus encourage additional investigation to find the cause of
such differences in data. Whilst it is possible to replicate research using the same methods has
the disadvantages of being subject to human error and the researcher’s personal bias.
Another method of estimating replicability involves using the idea of statistical power. From
Fisher’s analysis that a good experiment seldom produces a non-significant result when the null-
hypothesis is not true and the proposal of Cohen that any good experiment should have a power
of at least 80% which means out of every five replications, at least four should show true
outcomes with only one failing to reject the false null hypothesis (Killeen, 2005). The four main
methods of estimating replicability involve using the proof against the null hypothesis in the
original studies to estimate the average power. The simplest scenario involves using the same
power for the replicability. A real world but more complex scenario involves the studies having
different powers. In this case, replicability is found through the average power.
The z-curve can also be used to estimate replicability by following the normal analysis of
converting p-values to Z-scores and using this as a common measurement for the integration of
the results from the original research. This makes it possible to fit one function to the p-values
that come from a variety of methods and tests in statistics. The main component of this method is
the simplicity and ease of manipulation of power in the one tailed z-test. Here, the distribution of
the test statistics in the alternative hypothesis is shifted normally by a quantity denoted by m.
The maximum likelihood estimator can also be used to estimate replicability through finding the
value of unknown measures that make the outcome of the research most probable (Sohn, 1998).
Thus, if there are many parameters that are to be estimated, the maximum likelihood estimator
yields the accurate parameter estimates when compared to the other methods.

Scientific Methodology 3
Question 2
According to Ionnidis, most scientific studies are false because they are usually small and
therefore cannot represent all aspects of the topic in question. If the effect sizes, i.e. the strength
of the relationship between the two variables is small, then the research findings are also more
likely to be false. He also says that if the probability of the findings to be true is very low before
the study, then there is a higher chance for the findings to be also false. The flexibility of the
research design, defined outcomes and ways of analysis also impact whether the findings will be
true or false. The higher the flexibility, the more the chances of the findings being false.
Therefore the results and topics should be commonly agreed upon by all for the findings to be
considered true. Ionnidis also suggests that if there are interests apart from achieving the findings
such as economic gain among others, then there is a higher chance of the findings to be false
because of conflict of interest.
False positives are mainly regarded as Type I errors. They occur when it is concluded that an
outcome happened when in reality it did not. They are very costly when reported and in any
research paper (Ionnidis et al., 2011). They may lead to erroneous conclusions which may persist
because of the difficulty of challenging their findings by later research over the same matter.
There is a higher chance of a researcher to have false findings on the existence of an effect than
to correctly prove that the effect does not exist (Sink, 2010). This can be mainly attributed to the
dependent variables, sample size, covariates and the reporting subsets of the conditions during
the research.
Question 3
According to Cohen,
Alpha represents the maximum risk of accidentally rejecting the null hypothesis which is also
referred to Type I error.
Sample size. This is the necessary size of the sample that enables the researcher to obtain the
required power for a given alpha and hypothesized effect size.

Scientific Methodology 4
Effect size. This is the extent by which the null hypothesis is false obtained by finding the
difference between the null hypothesis and the alternate hypothesis.
The power represents the long-term probability of rejecting the null hypothesis given the effect
size, sample size and alpha.
The sample size is directly proportional to the power desired. It is however inversely
proportional to both the effect size and alpha.
Question 4
a) d=0.35
b) power=95 %
c) n=70.83
d) Type II error=0.57
e) Type I error =0
Question 5
Fishers advocated for use of significance tests as a means of data testing. The process involved
calculating the theoretical probability of research results after setting up the null hypothesis with
data closer to the mean of the distribution highly probable to happen in the null distribution. He
proposed the use of significance tests in identification of valid results. He also published a paper
on the method of the maximum likelihood estimator where he proposed that to get the estimates
of the parameters in a statistical model when the observations are known, one had to maximize
the likelihood of the observations with the parameters (Jackson, 2003). He also found the
existence of a sample mean which is not the same as the population mean. He is also credited
with the introduction of the concept of variance in statistics.
Because of Fisher’s analysis of variance, certain limitations in the designs of experiments came
up. He thus changed the design of experiments forwarding it as a science and explained clearly
the principles of randomization, replication and local control.
Pearson on the other hand came up with the method of measuring the relationship between two
continuous variables using the product-moment correlation. He also introduced other methods of
correlation among the tetrachoric correlation and the phi-coefficient for dichotomous variables.

Scientific Methodology 5
Question 6
The feud between was generated when Fisher started to solve a statistical problem that Pearson,
who was by then a well-respected statistician, had failed despite worked on it for a very
longtime. The problem was focused on estimating the correlation between different variables
with Pearson only examining a few cases because of the difficulty in the calculations. Fisher
solved all the cases involved in the problem within a week (Fienberg, 1992). Pearson however,
initially declined to publish Fisher’s findings because he could not understand them and when he
did publish them, he printed it as a footnote for a larger article that he had written. This infuriated
Fisher and thus a lifelong feud was formed.
Fisher also published two theoretical articles in 1922 that derived from Pearson but corrected the
mistakes of Pearson’s original work (Lehmann, 2003). This made Pearson angry and thus it is
reported that he sabotaged Fisher’s attempts to publish another paper on χ ²with editors of the
Journal. Pearson also condemned Fisher’s publication finding fault with his ideas on the t and χ²
topics. However, Fisher was adamant that the paper was substantive and was later proven to be
right.
Question 7
Research topic:
Effect of globalization on the labor market
Hypothesis:
Increased globalization has led to increased elasticity in the demand for labor.

Scientific Methodology 6
Question 8
IIn
Migration
Globalization
Spread of
Technology
Foreign
Investment
Trade
Wage
Convergence Elasticity of
Labor

Scientific Methodology 7
Foreign direct investment (FDI) will raise the productivity of workers in countries where capital
is scarce and therefore their wages because of the transfer of management skills, capital and
technology.
Trade promotes the convergence of wages because as countries with export goods that require a
lot of labor to be produced will in turn induce higher wages to rise relative to the rich countries
with low labor but a lot of capital.
Confounding can be regarded as mixing of effects thus a confounder has an effect and is
imbalanced between the two variables that form the basis of the research.
Question 9
The purpose of the study is to find out whether globalization has a causal effect on the elasticity
of labor demand in a country. The indicators for the link between the two are identified as trade,
foreign investment, technological advances and migration. Each of these indicators is checked
independently to find the one with the biggest effect on labor demand. Therefore, scenarios are
observed where only one of the indicators is prominent with the others being negligible.
The hypothesis of the research is to find if an increase in globalization will lead to a
corresponding increase in the elasticity for the demand for labor.
The research design is the quasi-experimental design thus the study has a pre-test and post-test.
Thus the study was conducted in developed countries before and after they became developed.
Countries that have been developed for a long time were used as the control group.
This design was considered the most appropriate for this study because it is the most suitable
design to determine the cause and effect relationship between globalization and the elasticity of
demand for labor. The structure and rules of design also allow for the isolation of single factors
in this case, trade, migration, foreign investment and spread of technology and thorough
examination of their associations and effects in a way that is challenging in real world scenarios.

Scientific Methodology 8
.
The population of the study were taken from worker in the industrial, agricultural and service
sectors of the countries under study. A total of 400 workers were sampled to be used in the study
with the experimental group mainly from the service industry in the developed countries.
Validity refers to the extent in which a researcher can be sure that the conclusions from the
research findings are accurate and reliable. Internal validity refers to the factors that can only
affect the dependent variable. It represents the link between the dependent and independent
variables. External validity assures of the causal relationship between the two variables.
Statistical validity refers to the inference in which the researcher is focused on the connection
between the cause and effect. This is achieved by performing the necessary tests of significance
upon the predicted hypothesis. Construct validity is where the construct is used to predict the
relationship of the dependent variable.
The main threats to the validity of the research findings include history which in this case refers
to the time between the pre-test and post-test where specific events such may occur to change the
rate of development of the countries in question. It is also possible for the participants to have
changed roles during the period of the study thus impacting on its validity.
Statistical regression might also pause a danger to the study because choosing only extreme
participants for the study might cause the finding not to reflect the actual population and thus
become invalid. Changes in the instruments used for the study might also affect its validity. The
participants might also stop undertaking in the study and thus affect the reliability of the results.
The effect of giving out the pre-test might also affect the second test especially in the responses
of the participants during the second test.
The workers might also suffer from the John Henry Effect. John Henry was a worker who
outperformed a machine because he knew that his output was being measured against that of the
machine. The workers might therefore undergo such an effect after the pre-test thus affecting the
validity of the study during the post-test.
Question 10

Scientific Methodology 9
Increase in globalization has led to a higher elasticity in demand for labor because of the effect it
has on employment and wages. Due to globalization, the relative wages of skilled to unskilled in
the long run have been increasing. The demand for the skilled labor has also increased over the
same period.
The construct was quantified by observing the movement of shares of the Gross Domestic
Product (GDP) that moved from agriculture to the service sector. This is a marker for
globalization. The service-sector economy is therefore compared to the agriculture economy to
show the relationship between the two sectors and hence as a result the extent of globalization of
the country.
The measure was chosen because it easily gives a picture of the trend of globalization and can
therefore be used to find the causal effect with its independent variables (Bhorat and Lundall,
2004). The measure was quantified by taking percentages of the GDP that had changed in the
past decade in relation to the agricultural and service sector of the selected countries.
Yes, there were modifications after data selection to cater for the changes of the GDP for
selected countries. The data was then modified in accordance with the effects of the channels of
globalization specific and unique for each country. Looking at the GDP share according to
economic activities, the industrialized benchmark of increasing shifts in sectors are being
replicated to a lesser extent in developing countries. It is noticeable that the trends in the
agriculture, mining and construction sector mirror that of developed countries from two decades
ago.

Scientific Methodology 10
References
Bhorat, H., & Lundall, P. (2004). Employment and labour market effects of globalization: Selected
issues for policy management. Employment Analysis Unit, Employment Strategy Department,
International Labour Office.
Cohen, J. (1992). Statistical power analysis. Current directions in psychological science, 1(3), 98-101.
Fienberg, S. E. (1992). A brief history of statistics in three and one-half chapters: a review essay.
Gribbons, B., & Herman, J. (1996). True and quasi-experimental designs. Practical assessment,
research, and evaluation, 5(1), 14.
Irwin, R. J. (2009). Equivalence of the statistics for replicability and area under the ROC curve. British
Journal of Mathematical and Statistical Psychology, 62(3), 485-487.
Ioannidis, J. P., Tarone, R., & McLaughlin, J. K. (2011). The false-positive to false-negative ratio in
epidemiologic studies. Epidemiology, 450-456.
Jackson, D. L. (2003). Review of The Lady Tasting Tea: How Statistics Revolutionized Science in the
Twentieth Century, by David Salsburg. Structural Equation Modeling, 10(4), 651-655.
Killeen, P. R. (2005). Replicability, confidence, and priors. Psychological Science, 16(12), 1009-1012.
Lehmann, E. L. (1993). The Fisher, Neyman-Pearson theories of testing hypotheses: one theory or
two?. Journal of the American statistical Association, 88(424), 1242-1249.
Roediger III, H. L. (2012). Psychology’s woes and a partial cure: The value of replication. APS
Observer, 25(2).
Sink, C. A., & Mvududu, N. H. (2010). Statistical power, sampling, and effect sizes: Three keys to
research relevancy. Counseling Outcome Research and Evaluation, 1(2), 1-18.
Sohn, D. (1998). Statistical significance and replicability: Why the former does not presage the
latter. Theory & Psychology, 8(3), 291-311.