COMP640 - Forecasting: Technological Changes & Student Effects

This capstone project, submitted to the University of Potomac for COMP640 - Forecasting and Management Technology, investigates the multifaceted effects of technological changes on students. The project begins by outlining the problem of technology overuse and its impact on student dependency, then proceeds to explore the research question, hypothesis, purpose, and significance of the study. A thorough literature review provides a foundation for the methodology, findings, and conclusions. The study examines the integration of technology in education, including the use of AI, e-learning, and ICT, and their influence on student achievement and cognitive abilities. The report includes a table of contents, acronyms, and a list of references, providing a comprehensive analysis of the topic. The problem statement highlights the increasing dependency on technology in modern life and its impact on students, setting the stage for an in-depth analysis of the role of technology in the classroom and its effects on learning outcomes. The research aims to determine the impact of technology on students and suggests that the overuse of technology is increasing and affecting the learning ability of students. The study's significance lies in understanding the effects of technology on students' lives and learning, guiding educators, policymakers, and students in making informed decisions about technology integration.

ABHANDL UNGEN
https://doi.org/10.1007/s11577-019-00600-2
Köln Z Soziol (2019) (Suppl 1) 71:75–97
Internationally Comparative Research Designs in the
Social Sciences: Fundamental Issues, Case Selection
Logics, and Research Limitations
Achim Goerres · Markus B. Siewert · Claudius Wagemann
Published online: 29 April 2019
© Springer Fachmedien Wiesbaden GmbH, ein Teil von Springer Nature 2019
Abstract This paper synthesizes methodological knowledge derived from com-
parative survey research and comparative politics and aims to enable researches to
make prudent research decisions. Starting from the data structure that can occur in
international comparisons at different levels, it suggests basic definitions for cases
and contexts, i. e. the main ingredients of international comparison. The paper then
goes on to discuss the full variety of case selection strategies in order to high-
light their relative advantages and disadvantages. Finally, it presents the limitations
of internationally comparative social science research. Overall, the paper suggests
that comparative research designs must be crafted cautiously, with careful regard to
a variety of issues, and emphasizes the idea that there can be no one-fits-all solution.
Keywords International comparison · Comparative designs · Quantitative and
qualitative comparisons · Case selection
A. Goerres ( )
Fakultät für Gesellschaftswissenschaften, Institut für Politikwissenschaft, Universität Duisburg-Essen
Lotharstr. 65, 47057 Duisburg, Germany
E-Mail: [e-mail]
M. B. Siewert · C. Wagemann
Fachbereich Gesellschaftswissenschaften, Institut für Politikwissenschaft, Goethe-Universität
Frankfurt
Theodor-W.-Adorno Platz 6, 60323 Frankfurt am Main, Germany
M. B. Siewert
E-Mail: [e-mail]
C. Wagemann
E-Mail: [e-mail]
K

76 A. Goerres et al.
International vergleichende Forschungsdesigns in den
Sozialwissenschaften: Grundlagen, Fallauswahlstrategien und Grenzen
Zusammenfassung Dieser Beitrag bietet eine Synopse zentraler methodischer
Aspekte der vergleichenden Politikwissenschaft und Umfrageforschung und zielt
darauf ab, Sozialwissenschaftler zu reflektierten forschungspraktischen Entschei-
dungen zu befähigen. Ausgehend von der Datenstruktur, die bei internationalen
Vergleichen auf verschiedenen Ebenen vorzufinden ist, werden grundsätzliche Defi-
nitionen für Fälle und Kontexte, d. h. die zentralen Bestandteile des internationalen
Vergleichs, vorgestellt. Anschließend wird die gesamte Bandbreite an Strategien zur
Fallauswahl diskutiert, wobei auf ihre jeweiligen Vor- und Nachteile eingegangen
wird. Im letzten Teil werden die Grenzen international vergleichender Forschung in
den Sozialwissenschaften dargelegt. Der Beitrag plädiert für ein umsichtiges Design
vergleichender Forschung, welches einer Vielzahl von Aspekten Rechnung trägt;
dabei wird ausdrücklich betont, dass es keine Universallösung gibt.
Schlüsselwörter Internationaler Vergleich · Vergleichende Studiendesigns ·
Quantitativer und qualitativer Vergleich · Fallauswahl
1 Introduction
This article deals with the challenges and pitfalls that researchers frequently have
to face when engaging in cross-national comparative analyses. Such a discussion is
not an easy task. Both methodologists and practitioners conducting cross-national
analyses at the macro level use different terminologies and emphasize different
criteria of comparison than their colleagues who work at the individual level. This
is complicated even further by similar communication deficits across qualitative
and quantitative methods (Brady and Collier 2004, 2010; Goertz and Mahoney
2012; King et al. 1994). Against this backdrop, we seek to inform a heterogeneous
readership about the terminology and various strands of argumentation, as well as of
potentials and pitfalls related to carrying out cross-case international comparisons.
We take a pluralistic stance on methods by bringing together insights from var-
ious strands of methodological schools of thought on how to design and conduct
comparative research. We hence present a concise summary concerning the state-
of-the-art of doing comparisons in the social sciences, but most certainly do not
seek to propose a specific recipe for how to carry out cross-country comparisons,
or multilevel research. This article is a cookbook with many recipes fitting different
occasions rather than just one recipe. This also means that we do not take sides on
the methodological debates or propose a fixed set of rules in terms of what compar-
ative research should look like. Instead, we would rather start from the assumptions
(i) that the application of methods should be question driven (Shapiro 2002), (ii)
that a research design can, and even must, undergo necessary adjustments during
the course of research (Schmitter 2008), (iii) and that, at the end of the day, ev-
ery researcher should be her/his own methodologist (Wright Mills 1959, p. 224).
The overall goal of the article is, therefore, to provide orientation about the state
K

Internationally Comparative Research Designs in the Social Sciences: Fundamental Issues,... 77
of important debates and discussions in the field of comparative research without
ascribing a higher value to one specific approach.
Our focus lies on international comparisons. Not every comparison necessarily
has to be internationally oriented, since we can also compare city structures, parties,
social movements, government action, etc., within a single country, using similar
lines of logic. However, cross-country comparisons usually show certain compli-
cations, compared to an otherwise similar mono-country project: concepts need to
be applicable across different cases; analytically important differences need to ex-
ist across cases to be explained; practical problems can come to the fore, such as
planning fieldwork in a foreign country or experiencing a language barrier. In short,
the cross-national perspective poses challenges and pitfalls which are different from
comparisons within the same country context (see Snyder 2001 on the issue of sub-
national comparison). This means for our purpose that we deal with comparison as
such, but always with an eye to the specific challenges for international comparison.
The article is structured as follows: first, we place the internationally comparative
design into a broader methodological perspective, discuss different data structures,
and then elaborate what they mean for a project, before defining cases and contexts
as the basic concepts. Second, we give a comprehensive overview with guidelines on
different selection strategies for international cases. Third, we discuss the limitations
of the internationally comparative design before, fourth, concluding the paper.
2 The Basics of Comparative Analysis: Cases, Contexts, and Data
Structure
2.1 Comparative Research in the Social Sciences
The etymological origin of the word “comparison” comes from Latin and points
to the identification of similarities and differences, shaping the labels of scien-
tific subdisciplines such as comparative macro-sociology or comparative politics
(Goldthorpe 1997; Powell et al. 2014). At the same time, the term has also had
a methodological career, most famously through Arend Lijphart’s (1971) seminal
article on the “Comparative Method”, which seemed to identify a whole study field
with a method—or, as we would say, a design. However, reading Lijphart carefully,
one detects a clear rank order of methodological approaches that still holds today
(see Lijphart 1971, p. 684 et seqq.). First and foremost, the experimental study con-
tinues to be the gold standard due to the possibility for the researcher to manipulate
the values of the independent variable while controlling for possible moderating
factors. Lijphart defines the “statistical method” as the weaker variant of the ex-
periment, keeping in place at least one of the central principles of experiments,
namely to select cases randomly. Finally, the “comparative method” is presented
as the weakest variant and “a very imperfect substitute” (Lijphart 1971, p. 685) of
experimental and statistical methods. It is notable at this point that Lijphart identi-
fies the comparative method with a small-N analysis, i. e. an analysis of just a few
cases. This then subsequently implies the main limitation: “The number of cases it
K

78 A. Goerres et al.
deals with is too small to permit systematic control by means of partial correlations”
(Lijphart 1971, p. 684).
Comparative research designs are hence not free from criticism. If we compare
countries, the number of available cases is often not only limited for the desired
sample, but also for the theoretical reference population. Applying specific theo-
retical lenses creates research situations where only a limited range of countries
are available—the often labeled “theories of the middle range” (Merton 1957) from
a perspective of research design. But when we study, for instance, industrialized
advanced economies or countries’ responses to natural disasters, we usually end up
with numbers which do not allow for the application of standard statistical tech-
niques, given that since basic assumptions, such as questions of distribution, unit
homogeneity, or causal independence (see also King et al. 1994), are not met.
Quantitative researchers are quick to worry about an indeterminate research de-
sign when comparing countries, i. e. that there are more variable constellations than
observations. This perspective reflects one of the reasons for not trying to engage
in international comparisons since the luxury of having enough observations at the
international level is rarely found in the available data. Apart from having such a rare
abundance of international data, only the quasi-experimental design is not subject
to the problem since it is based on an ex post construction of artificial treatment and
control groups of international cases (see below).
One proposed way to circumvent this is to engage in small-N comparisons with
only two, four, or a few more cases under observation (Mahoney 2003; Skocpol and
Somers 1980). Others also subsume longitudinal designs within a case over time
(often marked through historical ruptures and embedded in temporal sequences)
as a “comparison” (Gerring 2007, p. 28). More recent techniques such as Quali-
tative Comparative Analysis (QCA) even allow one to work on designs focusing
on a mid-sized number of cases through the use of set-theoretic relations (Ragin
2008; Schneider and Wagemann 2012). What all these proposals have in common
is that they, first, do not reach the case numbers which are typical for most surveys
and other quantitative approaches, can therefore, second, not rely on probabilistic
approaches or techniques which are based on randomization, and are, third, accused
of not meeting the standards for scientific inference which are typical of quantitative
approaches (see e. g. Brady and Collier 2004, 2010; Goertz and Mahoney 2012).
When speaking about comparative research, we thus quickly touch upon the
debates between qualitative and quantitative methods, or more specifically between
macro-level comparativists using (comparative) case study logics versus quantitative
researchers who apply the large-N logic of individual analysis to the country level
(see e. g. Brady and Collier 2010; Collier 2014; della Porta and Keating 2008; Goertz
and Mahoney 2012; Mahoney and Goertz 2006; Ragin 2004). This bifurcation within
the methodological world has however engendered various strands of literature that
are virtually or even completely isolated from each other. Just think of the proposals
from the (comparative) case study design literature (Blatter and Haverland 2012;
Gerring 2007; Ragin 2008; Rohlfing 2012), or the methodological pieces about
complex survey studies with international survey data (Steenbergen and Jones 2002),
which ignore each other to put it mildly. However, both approaches are intended for
comparisons at the international level.
K

Internationally Comparative Research Designs in the Social Sciences: Fundamental Issues,... 79
2.2 International Comparison at Different Levels of the Data Structure
Let us start by locating where the international component can be found in the data
structure. The simplest data structure in terms of international comparison is non-
hierarchical, as is illustrated in scenario I in Tab. 1. Only one level of variance exists
here, namely, country cases. Researchers focus completely on one level of compar-
ison, and only strive to make statements at one level of international analysis. Such
an international analysis would not be considered to warrant any kind of multilevel
modelling strategy due to the nonhierarchical nature of the data. A prominent exam-
ple of such a design can be found in a volume edited by Robert Dahl (1966), which
entails contributions comparing political oppositions in Western democracies with-
out any further hierarchy in the data. Country-level national oppositions in a given
region (here: Western Europe) are considered equally, without any reference to levels
above (such as supranational regions) or below (such as subnational oppositions).
Once we have hierarchical data—i. e. a data structure with different levels of
aggregation—international comparisons can be a part of the overall design which
targets different levels. Scenario II in Tab. 1 depicts this situation where the inter-
national comparison is at the highest level of aggregation, with the actual units of
analysis being nested in country contexts. A typical design in this respect is inter-
nationally comparative survey studies, where individuals are the units of analysis,
and the contexts in which individuals are embedded are subject to an international
comparison. An example of such a data structure can be found in Achim Goerres
and Markus Tepe (2010), who examine in which country contexts older people are
supportive of state structures providing public childcare. In this study, individuals
as cases are grouped in country contexts that stand for different political, socioeco-
nomic and cultural characteristics with regard to both societal and political aspects.
Based on the analysis of surveys in twelve countries, the authors then identify direct
as well as moderating effects from the macro towards the micro level. In such a data
scenario, researchers must take at least two decisions for case selection (see also
below): one for the country comparison at the top and one for the units of analysis
within the country contexts. Researchers also strive to make analytical statements
about the meaning of the international variance for the unit of analysis (macro–micro
Table 1 Three forms of international comparison in the data structure (authors own work)
Scenario I: No hierarchy Scenario II: Hierarchical data,
international comparison at the
highest level of aggregation
Scenario III: Hierarchical data, inter-
national comparison at more than one
level in the data structure
K

80 A. Goerres et al.
effects) and about the contextualization (or moderation) of subnational effects (here
micro-level effects) through the macro-level effects.
Scenario III in Tab. 1, finally, illustrates a data structure in which the international
comparison comes in on several layers. The work carried out by Gary Marks et al.
(2006) is a prime example of such a multilayered design. Investigating patterns of
national party competition across Eastern and Western Europe, their cases are polit-
ical parties that are nested in countries, while the countries again are nested in the
country groups of Eastern and Western Europe with different historical traditions.
Country cases are thus combined further in analytical groups. In the example, the
selection of countries is justified with reference to the additional country groups that
are relevant for the project. The two upper levels of international comparison are
integrated with one another. It is possible to have one logic for comparison for the
supranational country group and a second logic for the actual country cases, and fur-
ther ones for the subnational units. Researchers thus have at least three opportunities
for selection and can make inferences about the impact of the supranational region
on their subnational unit of analysis—here: parties—of the national context on the
subnational, of the supranational on the national, and of all these causal arrows as
contextualizing factors in moderation analysis.
Researchers who are confronted with the question of how to define their research
design will have an easy choice between scenario I on the one hand or scenarios II
or III on the other. Scenario I does not entail any interest in subnational variation,
thus making the data structure and design decisions on international aspects less
complicated. If researchers are not interested in scenario I, they can thus choose
between scenarios II and III. They should opt for scenario III if the number of
country cases is sufficiently large to warrant further grouping in country groups, and
if they have theoretical reasons to argue for a supranational layer of causal dynamics.
Only scenarios II and III allow the modelling of causal relationships between
different levels of aggregation. There are many effects on individuals from the
country contexts, and individuals as a whole can influence the country context. There
are also macro–macro causal relationships, such as the diffusion of environmental
problems across states and its subsequent influence on individuals.
2.3 Cases
We have already used a key term, namely “case”, that we need to define properly.
The terminological clarification of what a case is starts with a confusion: if we com-
pare internationally, it seems quite clear that a country constitutes a case. However,
other terms are often used synonymously, such as “unit of analysis” or “unit of
observation” (for some examples, see Gerring 2007, p. 17; 19 et seqq.; Seawright
and Collier 2010a, p. 315, p. 357), even though their meaning is not always unam-
biguous. In order to be more illustrative, one could say that the discussion of what
a case is can be abbreviated as the need to describe the entities which define the
rows in a spreadsheet. In an international comparison, cases are most prominently
identical to countries and other geographical entities, but also to societies, markets,
organizations (e. g. political parties, unions, businesses, schools), events (e. g. wars,
natural disasters, scandals), processes (democratization, deprivation, mobilizations,
K

Internationally Comparative Research Designs in the Social Sciences: Fundamental Issues,... 81
radicalization), etc. Depending on the level at which we operate, even individuals
between whom we might want to further differentiate, e. g. according to their various
life phases, gender, etc., can qualify as cases.
The discussion of what a case actually is becomes relevant due to two important
implications: first, the definition of what constitutes a case also comprises the ques-
tion of what it is a case of, i. e. to which reference population it can be attributed
(Collier and Mahoney 1996, p. 4, 38; Ragin 2000, p. 43 et seqq., 2004). It is in-
dispensable to render the reference population explicit, since inferences can only be
made to that reference population, if at all. In contrast to standard statistical tech-
niques, the problem of comparative research operating with small or midsized case
numbers is not only (or perhaps not even so much) the question of case selection or
sampling, but that of the researcher carefully defining the population (Mahoney and
Goertz 2004).
Second, and connected to the first issue, is the discussion of “casing” (Ragin
and Becker 1992; see also Rohlfing 2012, p. 23–28). The issue of what constitutes
a case is usually not naturally given, but rather needs creative construction on the
part of the researcher. For instance, while country borders might lend themselves as
natural identifiers of countries as cases, the endeavor is made more difficult if the
units of observation are organizations. The more formalized organizational structures
are, the better defined are their borders, and the easier it is to define it as a case.
However, when comparing, for instance, organizational fields in a given economic
sector, the establishment of where the field starts and where it ends is anything
but trivial. The same holds, for instance, for the social movement organizations
which are characterized by fluid structures and memberships. Before comparing
social movements, scholars therefore have to define what a movement is. Note that,
although the definition of “country cases” seems to be clearcut, the problem of
casing can also occur at the country level. While just three years separate Germany
in 1988 from Germany in 1991—which is the same time distance as between 1978
and 1981—only few would suggest that Germany after the fall of the Berlin Wall
constitutes the same case as preunification Germany. This change was certainly
also accompanied by territorial changes (growth) and a new legal situation (full
sovereignty), which might also have led to a different country in structural terms
(despite the continuation of the Basic Law [Grundgesetz] and the main institutional
structures). Examples of such temporal “before–after” gaps constituting new cases
abound in the social sciences—another illustrative examples is the world pre-9/11
and post-9/11.
“Casing”, however, draws our attention to a further special asset of “defining
what a case is”, going back to the fact that cases can also be seen as configura-
tions of their properties—a perspective which is largely inspired by the works of
Charles Ragin (2000, p. 64 et seqq.), but also by Paul Lazarsfeld’s (1937) idea of
a property space. Depending on the actual research question, different aspects of
cases might be analytically important. Studying the United States of America from
the perspective of the migration research means that the researcher understands the
U.S. case differently, as if (s)he were studying religious pluralism, resistance to
welfare reform, executive politics, or the polarization of politics. The more vaguely
defined a case is (i. e. the less clear its borders are), the more room for “casing”
K

82 A. Goerres et al.
opens up—think about such creative concepts as the idea of a “European society”.
While vaguely defined concepts have the advantage of allotting considerable scope
to the individual researchers’ decisions with regard to casing, they usually come at
the price of ambiguous conceptual definitions (Collier and Adcock 1999; Goertz
2006; Sartori 1970).
In fact, cases can only be compared if they share at least enough characteristics in
order to belong to the same group of research objects. While Germany is a country,
San Francisco is a city, which means that Germany and San Francisco should not be
compared if this fundamental difference in territorial constitution is relevant for the
research interest. If we compare, for instance, Liechtenstein and Würselen, the city
in which the 2017 SPD candidate for Chancellor, Martin Schulz, was Mayor before
starting his EU career, we will see that both territories have a more or less similar
number of inhabitants, varying between 35,000 and 40,000. If we are only interested
in structures of social networks in communities of that size, then the two settings
might be comparable, but otherwise not. We can see that this again takes up the issue
of casing from above: the comparability strongly depends on the properties at which
we look when we execute the comparison. Liechtenstein and Würselen might not
be comparable in many respects, but they are comparable in terms of the population
size. So, if the population size is a decisive category, and if we can make convincing
arguments that all the other differences between the two cases do not influence or
are not relevant to the phenomena that we want to study (something which is hard
to imagine for this example), then a comparison of these two units can make sense.
Casing is thus closely connected to the idea of case properties. Comparability
is ensured through a configuration of case properties in which some properties are
held constant in such a way that they form a species (in the sense of a higher-order
concept), while other properties are defined as being irrelevant. If we understand
every case as a configuration of its properties, then comparability is ensured by
having sufficient subsets of shared properties. The old idea of genus et differentiam,
which is used for defining concepts, comes back in here: while much has to be
equal, or at least sufficiently similar, between two (or more) cases so that the same
genus can be ascertained, other properties must be different so as not to compare
two equal cases.
Note that cases in international comparison are more often than not dependent on
one another, and arguably increasingly so. This certainly also has repercussions on
questions of inference which will be addressed later. Indeed, the independence of
country cases should not be taken for granted and is difficult to achieve in our current
times of international exchanges of knowledge and experiences—an issue which is
usually referred to as Galton’s problem. For instance, the spatial dependency of
countries can lead to the diffusion of policy ideas that can be traced through interna-
tional policy diplomacy, i. e. policy experts travel to the neighboring country to learn
about public policy issues and can then try to implement their insights back home
(Simmons and Elkins 2004). Another example refers to the Arab Spring, which was
strongly characterized by the spill-over and imitation processes. This mutual depen-
dency can also arise out of temporal dependency between geographically, culturally,
or otherwise close countries (Jahn 2006). In some studies, this mutual dependency
of cases is captured in an analysis of the relationship between international units
K

Internationally Comparative Research Designs in the Social Sciences: Fundamental Issues,... 83
themselves. To address these issues, Lundsgaarde et al. (2010), for instance, employ
dyadic data of foreign aid and trade flows to directly estimate the mutual influences
of countries and money flows. What remains a task for all researchers is to identify
and take into an account possible dependencies between cases in an international
comparison.
2.4 Contexts
In order to systematically study the dependency of cases, the concept of contexts is
relevant. We understand contexts as those environmental conditions into which cases
are embedded, i. e. cases are sorted in groups whose characteristics can be analyti-
cally described. Cases belonging to certain contexts share elements of the context,
and because of this they are similar and thus more comparable than if we worked
with random samples from a universe of cases. For example, Germany belongs to
the context of rich countries (defined through the GDP level, for example), and be-
ing embedded in such a context renders Germany different from those cases which
are not embedded in the same context. Attention must be paid to this similarity of
cases that are embedded in contexts and it can be explicitly used in the international
comparison.
The similarity of cases within a context is usually connected to the characteristics
of data collection. For instance, in international surveys with random samples in each
country, two randomly drawn respondents from one country are more similar to each
other than two randomly drawn individuals from two countries. The embeddedness
of cases in a context can be addressed by using variables to describe the contextual
characteristics at the case level, thus bringing the context dependency to the level
of the case. For instance, in the volume edited by Cees van der Eijk and Mark
Franklin (1996), the contributors pool international survey data and measure all
country characteristics as individual-level variables. Yet, going back to Coleman’s
concern with different levels of causal paths and problems of aggregation in his
bathtub heuristic (Coleman 1990), one might wish to explicitly model the differences
between a case and its contexts, as these are set at different levels of aggregation
and rely on different causal mechanisms.
3 Selecting Cases for Comparative Research
It should have become clear that choosing the right cases for each level is a crucial
task for any comparative research design. We therefore next address different logics
of sampling, as most users of quantitative individual-level techniques would say that
case selection has become the central term in the comparative case literature. We start
by describing the very low-key logic of contrasting empirics from different countries.
We then address quasi-experimental logics of selecting country contexts. After that,
we talk about random selection of country cases, and finally, and most extensively,
about theoretical sampling. Table 2 provides an overview of the identified case
selection logics and summarizes their defining features, as well as highlighting both
potentials and pitfalls.
K

84 A. Goerres et al.
Table 2 Selection logics for comparisons: potentials and pitfalls (Based on the authors’ on compilation)
Sampling logic Defining feature Potentials Pitfalls
Contrasting At least two country
cases are used in order
to describe cases analyti-
cally
Typically, very low
demand on selection;
some variance suffices
Inferring from the re-
sults is not possible; no
known application in the
multilevel world
Census All countries in a theo-
retically defined universe
are contained in the sam-
ple; limitations only
arise from a lack of data
availability
More data is always
better if high quality is
assured; no uncertainty
due to sampling
High demand on data
availability and data
quality; any census can
be seen as a sample from
a theoretical superpop-
ulation that needs to be
defined
Quasi-experi-
ments
Country-period cases are
compared with them-
selves or other country-
period cases in order to
evaluate the impact of an
ex post constructed treat-
ment with an artificial
control group
Gives high leverage on
causal effects of the
treatment; can be com-
bined with hierarchical
data modeling
High demand on avail-
ability of comparison;
main variable of inter-
est must be identifiable
and constructed as an
exogenous factor
Random sam-
pling of country
contexts
Countries are sampled
randomly, usually in
a very small N, in order
to collect further data in
a resource-rich manner
The resource-rich data
collection is white-
washed as to antecedent
factors, and is thus unbi-
ased
Data analysis that places
high emphasis on coun-
try-level effects will be
influenced by problems
of inference with small N
Theoretical
case selection
Various substantiated
reasons, often derived
from theory or previous
empirical research, are
used in order to arrive at
a purposeful set of cases
Relevant factors can be
identified more easily;
middle-range expla-
nations are possible;
explanatory narratives
can be achieved more
easily
Generalizability is lim-
ited; difficult to find the
“correct” rule for selec-
tion
3.1 Contrasting Cases
On the simplest level, an international comparison can just be an exercise in contrast-
ing two different case experiences of the phenomenon in question. It is a relatively
shallow design as far as the international selection strategy is concerned but is ap-
plied relatively frequently in the published work. As mentioned earlier, international
comparisons usually involve countries as cases for which researchers then explore
differences and/or similarities between them. Analytically, such exercises have a very
low-hung goal, namely to demonstrate that there is variance across countries—or
that there is no such variance—and to use this insight in order to enhance the ana-
lytical description of what is happening in the various settings. There are numerous
examples of such a contrasting approach. For example, Weisskopf (1975) contrasts
the ways in which political leadership dealt with issues of economic development
in India and China without being very explicit about why he chose these countries.
If anything, two country cases suffice in order to show similarities or differences.
In principle, such comparative designs are not restricted to two cases, but can involve
several cases. Researchers who have a main interest in analytically describing one
K

Internationally Comparative Research Designs in the Social Sciences: Fundamental Issues,... 85
case—maybe because it is the context of a follow-up study—could use this technique
of contrasting in order to analytically describe their main case in comparison with
another one. In most comparative research projects, however, it seems to make
more sense to select by theoretical sampling or to create a census of all available
international cases in a theoretical universe. The contrasting approach usually does
not have a very nuanced strategy for case selection but is likely to refer to a general
argument of “these are interesting countries” and/or “we know them well”.
3.2 A Census of Cases
Another relatively simple rationale in terms of selection logic is to opt for a full
census of cases, given a certain theoretical definition (Berk et al. 1995). For instance,
the Comparative Party Manifesto Project is a data collection for all political parties
in any political system since 1945. This project has been ongoing since 1979 and
successively extended the scope of available countries and years across four decades
with a full census (e. g. Merz et al. 2016).
Researchers should always opt for this selection logic if they have a reasonable
chance of actually realizing this census and if the data quality is similarly high across
all cases and points in time. When applying this kind of logic, researchers should
ask themselves whether their universe is in fact not a sample from a theoretical
superpopulation. The data for countries are always restricted to a certain time period,
leading to the question as to what the data for these country-period cases mean for
other periods of the same countries. Some social scientists thus suggest that statistical
analyses of census data should still include uncertainty measures in order to reflect
that kind of inference about a theoretical superpopulation (Behnke 2005; Broscheid
and Gschwend 2005).
3.3 Quasi-experimental Logic
A rather demanding way of conceptualizing a comparative design is to follow quasi-
experimental logic. This means that cases are selected that have experienced some
kind of treatment, i. e. an exogenous variable exerting a certain effect on them.
A “sibling” case is then chosen for each treated country that mirrors the first case
“as if” the treatment had not occurred.
We describe two variants of this approach. Carporaso and Pelowski (1971) con-
ducted an analysis of the effects of membership in the European Community in
its early phase. They applied interrupted time-series analysis in order to compare
countries with themselves before and after significant changes in EC membership
regulation. The change in various outcome variables is compared against the hypo-
thetical value of Y that is estimated based on the past trend. In another example,
Sebastian Galiani et al. (2017) compare countries against themselves, once shortly
before they cross an external set threshold for receiving foreign aid by the Interna-
tional Development Association (the development aid agency of the World Bank)
and once shortly after. Thus, a country’s economic development is compared with
receiving aid and without receiving aid.
K

86 A. Goerres et al.
This quasi-experimental logic is very powerful in terms of causal inference, as
it comes close to an experimental study. There are, however, many circumstances
in which such a design is not feasible, as cases of the artificial control group are
not available in such a comparison, or because there is no pattern that can be
operationalized as a clear treatment. It is the only international comparative design
in which there is no danger of an indeterminate research design, i. e. where there are
too many country-level variables and too few observations at the international level.
3.4 Random Sampling
Random selection has two general advantages. It allows the use of classic frequentist
statistics in order to make inferences about the population from which the random
sample was drawn. This feature is not relevant for an international comparison
since the population of feasible countries or country-time points is typically not
that big. Moreover, random selection blurs any differences between elements that
come into the sample and those that are not drawn into the sample. No antecedent
factor determines which element gets in and which one does not. That latter feature,
in contrast, is very helpful. Researchers who are mainly interested in subnational
units and have limited resources might choose a random sample of countries with
a relatively small N because they do not want their resource-intensive research at
the subnational unit to be distorted by the preselection of countries. For example,
Franklin (2008) studies the reaction of governments to challenges of their human
rights violations in the media. Since he uses extensive media sources in each country
to identify episodes of human rights violations and reactions or nonreactions in the
public media, he drew a random sample of seven Latin American countries, so
that his findings are unbiased as to country characteristics. The fact that he draws
inferences from a random sample of n = 7 is of no relevance to him.
The more common usage of random selection (Fearon and Laitin 2008), also with
regard to large-N scenarios, takes place in numerous comparative survey studies,
sometimes with surprisingly practical implications. An international consortium of
researchers very often defines a country sample here (usually with some rough
definitional characteristics such as liberal democracies), and then negotiates with
country teams and national funding agencies as to who gets in and who stays out.
Random selections of respondents are then executed within each country that allow
for inferences about the population with regard to each country context. Researchers
confronted with such a design have to be aware that—at the country level—the
sample is not random (but typically a theoretically defined sample that is furthermore
subject to feasibility aspects), and that they have at their disposal a series of equal
random samples from countries for which classic frequentist techniques can be
applied. Researchers very often apply random-effects models to such data sets where
the statistical technique actually assumes that the country sample is also a random
sample. There are some more recent methodological studies that explore how to best
apply statistics in such a context (please, see also other articles in special issue).
K