Meta-Analysis: Integrating Mobile Devices in Education - COMP640

This report presents a meta-analysis and research synthesis on the effects of integrating mobile devices in teaching and learning, based on 110 experimental and quasi-experimental journal articles published between 1993 and 2013. The study found a moderate mean effect size of 0.523 for the application of mobile devices in education. It analyzes the impact of moderator variables and synthesizes the advantages and disadvantages of mobile learning across different levels of these variables, drawing on content analyses of individual studies. The research examines the overall effectiveness of mobile technology, the influence of moderator variables on learning achievement, and provides an overview of the use of mobile devices in educational experimental studies, including the types of devices, subjects taught, software used, and duration of interventions. The results and their implications for research and practice are discussed, providing insights into the evolving role of mobile devices in education and their potential to enhance student learning outcomes. The study also acknowledges limitations of previous qualitative reviews and the need to consider diverse age groups and devices in mobile learning research.

The effects of integrating mobile devices with teaching and
learning on students'learning performance: A meta-analysis
and research synthesis
Yao-Ting Sunga
, Kuo-En Changb
, Tzu-Chien Liua, *
a Department of Educational Psychology and Counseling,National Taiwan Normal University,Taiwan,ROC
b Grad.Institute of Information and Computer Education,National Taiwan Normal University,Taiwan,ROC
a r t i c l e i n f o
Article history:
Received 12 August 2015
Received in revised form 17 November 2015
Accepted 19 November 2015
Available online 23 November 2015
Keywords:
Evaluation methodologies
Pedagogical issues
Teaching/learning strategies
a b s t r a c t
Mobile devices such as laptops, personal digital assistants,and mobile phones have
become a learning toolwith great potential in both classrooms and outdoor learning.
Although there have been qualitative analyses of the use of mobile devices in education,
systematic quantitative analyses of the effects of mobile-integrated education are lacking.
This study performed a meta-analysis and research synthesis of the effects of integrated
mobile devices in teaching and learning, in which 110 experimental and quasiexperimental
journal articles published during the period 1993e2013 were coded and analyzed. Overall,
there was a moderate mean effect size of 0.523 for the application of mobile devices to
education.The effect sizes of moderator variables were analyzed and the advantages and
disadvantages of mobile learning in different levels of moderator variables were synthe-
sized based on content analyses of individual studies.The results of this study and their
implications for both research and practice are discussed.
© 2015 The Authors. Published by Elsevier Ltd.This is an open access article under the CC
BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/).
1. Introduction
1.1.Integrating mobile devices with learning and instruction
Mobile computers have gradually been introduced into educational contexts over the past 2 decades. Mobile technolog
has led to most people to carry their own individual small computers that contain exceptional computing power,such as
laptops,personaldigital assistants (PDAs),tablet personalcomputers (PCs),cell phones,and e-book readers.This large
amount of computing power and portability, combined with the wireless communication and context sensitivity tools, ma
one-to-one computing a learning tool of great potential in both traditional classrooms and outdoor informal learning.
With regard to access to computers,large-scale one-to-one computing programs have been implemented in many
countries globally (Bebell & O'Dwyer, 2010; Fleischer, 2012; Zucker & Light, 2009), such that elementary- and middle-sch
students and their teachers have their own mobile devices.In addition,in terms of promoting innovation in education via
information technology, not only does mobile computing support traditional lecture-style teaching, but through convenien
* Corresponding author.Department of Educational Psychology and Counseling,National Taiwan Normal University, 162,HePing East Road,Section 1,
Taipei,Taiwan,ROC.
E-mail addresses: [e-mail] (Y.-T.Sung),[e-mail] (K.-E.Chang),[e-mail] (T.-C.Liu).
Contents lists available at ScienceDirect
Computers & Education
j o u r n a lhomepage: w w w . e l s e v i e r . c o m / l o c a t e / c o m p e d u
http://dx.doi.org/10.1016/j.compedu.2015.11.008
0360-1315/© 2015 The Authors.Published by Elsevier Ltd.This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/
licenses/by-nc-nd/4.0/).
Computers & Education 94 (2016) 252e275

information gathering and sharing it can also promote innovative teaching methods such as cooperative learning (Lan, Sun
& Chang,2007; Roschelle et al.,2010),exploratory learning outside the classroom (Liu,Lin, Tsai,& Paas,2012),and game-
based learning (Klopfer,Sheldon,Perry,& Chen,2012).Therefore,mobile technologies have great potential for facilitating
more innovative educationalmethods.Simultaneously,these patterns in educationalmethods will likely not only help
subject content learning,but may also facilitate the development of communication,problem-solving, creativity,and other
high-level skills among students (Warschauer,2007).
However,despite the proposed advantages ofusing mobile computing devices for increasing computer accessibility,
diverse teaching styles, and academic performance, currently researchers found mixed results regarding the effects of mo
devices (e.g., Warschauer, Zheng, Niiya, Cotten, & Farkas, 2014), and very few studies have addressed how best to use m
devices,and the effectiveness of doing so.
1.2. Review of the research into integrating mobile devices with teaching and learning
There are seven studies which reviewed the research into integrating mobile devices with teaching and learning and ca
be divided into two types according to the devices they focused on: (1) those focused on how laptops are used in schools a
(2) those focused on the applications of various types of mobile device in education (see Appendix A).
Regarding the review of laptop-based programs,Zucker and Light (2009) believed that schoolprograms integrating
laptops into schools have a positive impact on student learning. However, they also believed that laptop use did not achie
the goals of increasing higher-level thinking and transformation of classroom teaching methods.Penuel (2006) reviewed
30 studies that examined the usage of laptops with wireless connectivity in one-to-one computer programs. Those studies
found that students most often used the laptops to do homework,take notes,and finish assignments.General-purpose
software such as word processors, web browsers, and presentation software were relatively common. Bebell and
O'Dwyer (2010) examined four different empirical studies of laptop programs in schools.They discovered that in most
schools participating in one-to-one programs there were significant increases in grade-point averages or standardized test
of student achievement, relative to schools that did not provide such programs. In addition, they found that most students
used their laptops to write,browse the Internet,make presentations,do homework,or take tests.Furthermore,teachers
made more changes to their teaching methods when they had increased opportunities to use laptops.Students partici-
pating in one-to-one programs also had a deeper engagement with what they were learning when compared to control
groups.
Fleischer (2012) conducted a narrative research review of 18 different empirical studies on the usage of laptops.These
studies found a large range in the number of hours that students used laptops, from a few days to as little as 1 h per week
most frequently used computer functions were searches, followed by expression and communication. In most studies it wa
found that students had a positive attitude toward laptops,and felt that they were more motivated and engaged in their
learning,and it was further believed that teachers conducted more student-centered learning activities.Moreover,consid-
erable differences in classroom educational practices arose from the diversity of teachers'beliefs about the usefulness of
laptops.Fleischer (2012) also found severalchallenges regarding the use oflaptops in classrooms,such as encouraging
teachers to change their previous beliefs and teaching methods (e.g., teacher-centered lectures) in response to their stude
greater flexibility and autonomy; how to reconcile the conflict between the students' desire for independent study and the
need for teachers' guidance; and how to facilitate teachers' competence by designing an appropriate curriculum and teach
models for laptop usage programs.
With respect to the research on the use of mobile technology in education,Hwang and Tsai(2011) provided a broad
discussion of studies on mobile and ubiquitous learning published in six journals between 2001 and 2010. In their review o
154 articles, they discovered that the use of mobile and ubiquitous learning accelerated markedly during 2008; researcher
mostly studied students of higher education,and the fields most often researched were language arts,engineering,and
computer technology.Frohberg,Goth,and Schwabe (2009) categorized 102 mobile-learning projects,and discovered that
most mobile-learning activities occurred across different settings,and took place within a physical context and an official
environment, such as a classroom or workplace. Regarding the pedagogical roles that mobile devices play in education, m
research has used mobile devices primarily as a sort of reinforcement tool to stimulate motivation and strengthen engage-
ment, and secondarily as a content-delivery tool. Few projects have used mobile devices to assist with constructive thinkin
or reflection. Furthermore, most learning activities using mobile devices have been controlled by the teacher, with there b
only a handful of learner-centered projects in existence.Concerning the communication functions,very few projects have
made any use of cooperative or team communication.Moreover,the vast majority of studies have made use of novice par-
ticipants; little research has involved experienced participants. When sorted according to educational goals, it was found t
the vast majority of research has focused on lower-level knowledge and skills, and ignored higher-level tasks such as analy
and evaluation. Wong and Looi (2011) investigated the influence of mobile devices on seamless learning. Seamless learnin
refers to a learning model that students can learn whenever they want to learn in a variety of scenarios and that they can
switch from one scenario or one context to another easily and quickly (Chan et al., 2006; Wong & Looi, 2011). Wong and L
(2011) selected and analyzed a sample of 54 articles on the use of mobile devices to facilitate seamless learning,and found
that all 54 articles contained 10 features,including formaland informal learning,personalized and sociallearning,and
learning across multiple durations and locations.
Y.-T.Sung et al./ Computers & Education 94 (2016) 252e275 253

1.3. Purposes of this study
While analyzing the overall effectiveness of using mobile devices in education, the review research described above ha
two major limitations. First, all of the reviews adopted a qualitative approach, which may be able to describe and summar
how related studies were conducted and the problems encountered during their execution,but this makes it difficult to
evaluate the effects actually produced by the mobile devices in general and the specific moderator variables. Second, mu
the previous review research has focused on the usage of laptop computers as the subject of their investigation (e.g., Pen
2006), and most of the research participants in those reviewed articles were in primary and secondary schools. However,
many new developments in mobile hardware have meant that diverse age groups now use different devices. Therefore, m
different moderators need to be accounted for when attempting to determine whether or not intervening variables have a
effect.
In the context of this background, the primary goal of this study was to perform a meta-analysis and research synthes
the research on the usage of mobile devices in education published in the last 2 decades.Specifically,the purposes of this
study were as follows:
1. To provide an overview of the status of the use of mobile devices in educational experimental studies,including who is
using them, which domain subjects are being taught, what kinds of mobile device and software are being used, where
programs take place,how the devices are used in teaching,and the duration of the interventions.
2. To quantify the overall effectiveness of integrating mobile technologies into education on student learning achievemen
3. To determine how the moderator variables influence the effects of mobile devices on learning achievement.
4. To synthesize the advantages and disadvantages of mobile learning in levels of moderator variables based on the cont
analysis of articles related with moderator variables.
2. Method
2.1. Data sources and search strategy
Journal articles published during the period 1993e2013 were searched electronically and manually, and via reference-l
checking to retrieve the relevant literature.For electronic searches,the main databases were the Education Resources In-
formation Center (ERIC) and the Social Sciences Citation Index database of the Institute of Science Index (ISI).Two sets of
keywords were searched: (1) mobile-device related keywords,including mobile,wireless,ubiquitous,wearable,portable,
handheld, cell phone, personal digital assistant, PDA, palmtop, pad, web pad, tablet PC, tablet computer, laptop, e-book,
pen, pocket dictionary,and classroom response system; and (2) learning-related keywords,including teaching,learning,
training, and lectures. The two sets of keywords were combined when searching the electronic databases. Manual search
included the major journals in educational technology and e-learning,such as the Australian Journal of Educational Tech-
nology,British Journal of Educational Technology,Computers & Education,Computer Assisted Language Learning,Educa-
tional Technology Research and Development, Journal of Computer Assisted Learning, Language Learning & Technology,
ReCall.
After collating all of the related literature, another round of searches was conducted using the reference lists found in t
literature yielded by the electronic search to find any omitted but relevant works.
2.2. Search results
2.2.1.Initial screening
The initial search yielded 4121 abstracts published between 1993 and 2013 (1718 in ERIC and 2403 in ISI) that were re
to mobile learning. Two authors read each abstract of the article and judged whether or not the article was related to teac
and learning with a mobile device,which resulted in the selection of 925 articles.
2.2.2.Screening for experimental and quasiexperimental research
In the second stage,the studies were screened according to the research method.Experimental studies (including the
pretest-posttest equivalent-group,posttest-only equivalent-groups,and randomized matched subjects and posttest-only
control-group designs) and quasi-experimental studies (including the pretest-posttest nonequivalent-groups and counter-
balanced designs; see Ary, Jacobs, & Razavieh, 2002; Best & Kahn, 1998, for a reference) were included. Conceptual anal
research reviews, case studies and qualitative research,survey research, and pre-experimental studies were all excluded at
this stage.At the completion of this stage there remained 182 articles.
2.2.3.Application of inclusion and exclusion criteria for the meta-analysis
Studies were eligible for inclusion in the meta-analysis if they conformed with the following three criteria:
Y.-T.Sung et al./ Computers & Education 94 (2016) 252e275254

1. The application of mobile devices was the key variable of the study.The experimental group had an intervention that
used mobile devices,and was compared with a control group that used traditional learning.If both the experimental
and control groups used mobile-device interventions,and only the teaching methods were compared,then the study
was excluded (e.g.,Hsu, Hwang, Chang,& Chang,2013; Jeong & Hong,2013; Li,Chen,& Yang,2013; Ryu & Parsons,
2012).
2. Sufficient information was presented to calculate effect sizes, such as means, standard deviations, t, F, orc 2 values, or the
number of people in each group. Articles in which the sample sizes of each group were not cited,lacked any inferential
statistical results,or had inferential statistical results but were still inadequate for calculating an effect size according to
Lipsey and Wilson (2000) were excluded (e.g.,Gleaves,Walker, & Grey, 2007; Langman & Fies,2010; Purrazzella &
Mechling,2013; Yang et al.,2013).
3. Experimental results were presented with learning achievement as a major dependent variable measured by standardiz
or researcher-constructed tests. Studies for which the results were related to affective variables (e.g., learning attitude
learning motivation) or interaction between peers but without learning achievement were excluded (e.g., Jian,Sandnes,
Law, Huang,& Huang,2009; Lan et al.,2007; Mouza, 2008; Siozos, Palaigeorgiou,Triantafyllakos,& Despotakis,2009).
Application of these criteria yielded 110 articles that were acceptable for inclusion in the meta-analysis. For a complete
of these references,please see our online supplemental archive.
2.3. Selection and coding of the outcome variables
One of the most used framework for representing the research content and dimensions is the activity theory (AT), which
uses activity as a unit for analyzing human practices (Bakhurst,2009).Recently,several researchers have used the AT as a
theoretical basis for analyzing mobile learning studies (e.g.,Frohberg et al.,2009; Sharples,Taylor,& Vavoula,2007) or for
designing mobile learning scenarios (e.g.,Zurita & Nussbaum,2007).This study used six major components of AT to select
moderator variables and analyze mobile learning: (a) Subjects: which involve all the people who may be involved in learnin
curriculums through mobile devices,such as students ofdifferent age levels or teachers ofdifferent levels ofteaching
expertise.(b) Objects (or objectives) of the mobile learning,which focus on the goal such as acquiring cognitive skills or
enhancing learning motivation through mobile devices. (c) Tools/instruments in the mobile learning, which may be artifact
(e.g.,hardware and software) or learning resources (e.g.,tutors).(d) Rules/control for the activity,which are norms or reg-
ulations that circumscribe the mobile activities, such as the procedure in teaching scenarios designed for the learning pac
styles designated. (e) Context of the activity, which refers to the physical (e.g., classroom or museum) or social (e.g., amb
of learning in a group) environments for conducting mobile learning.(f) Communication/interaction,which refers to the
method of interaction between users and mobile technologies (such as the process teachers' adaption to mobile devices) o
the communications styles among learners.
2.3.1.Research name
This refers to the first author's name,the year of publication,and the article title.
2.3.2.Research participants
In this review,for all the reviewed articles,the research participant corresponded to the “subject” of the AT framework,
and was coded by their learning stages,including kindergarten,elementary school,middle school,(senior) high school,
university, graduate school,teachers,adults,and mixed.
2.3.3.Treatments
The treatments of the reviewed articles corresponded to the “tools” component (e.g.,the hardware and software),the
“rules/control” component (e.g., the teaching methods and domain subjects), and the “context” component (e.g., interven
settings, intervention duration).The description for each of these treatment variables are as follows:
1. Hardware: Different types of mobile hardware, which comprised PDAs, laptops, tablet PCs, cell phones, iPods, MP3 playe
e-book readers, pads, digital pens, pocket dictionaries, and classroom response systems (CRSs), or any mixture of there
2. Software:Different types ofsoftware,which encompassed general-purpose software and learning-oriented software
(Sung & Lesgold,2007),the former referring to commercialsoftware currently in circulation that was not designed
especially for teaching and learning (e.g.,word processors or spreadsheets),and the latter having been designed specif-
ically for educational programs or goals.
3. Teaching method: Different teaching methods, including lectures, cooperative learning (students were divided into grou
and completed learning tasks collaboratively,e.g.,Chang,Lan, Chang,& Sung,2010; Huang,Liang,Su, & Chen,2012),
inquiry-oriented learning (using problem-, project-, or inquiry-based methods with mobile devices for learning, e.g., Che
2010; Lowther, Ross, & Marrison, 2003), self-directed study (teachers/researchers did not designate or implement speci
teaching scenarios for students to follow, students use mobile devices for self-paced learning, e.g., Chen & Li, 2010; Ch
Y.-T.Sung et al./ Computers & Education 94 (2016) 252e275 255

Tan,& Lo, 2013),computer-assisted testing/assessment (using mobile devices for formative assessment or quizzes in
classroom or outdoors,e.g., Agbatogun, 2012),and mixed methods thereof.
4. Domain subject:Domain subjects were analyzed to establish the relative effectiveness of mobile devices for teaching
different subjects,including language arts,social studies,science,mathematics,multidisciplinary (if the mobile devices
were used in several subjects, but measurement of the achievement was presented as a whole instead of separately, t
was coded as multidisciplinary),specific abilities (e.g.,spatial ability or creativity),health-care programs,education,
psychology,and computer and information technology.
5. Implementation setting: Implementation settings were included to establish whether the impact of mobile devices on
learning differed according to the environment in which they were used, which included classrooms, outdoors (e.g., zo
campus gardens),museum, laboratory,workplaces, and unrestricted settings (devices may be used anywhere).
6. Intervention duration: Different periods of time for the intervention,including periods no more than four hours (4 h),
between five and 24 h (>4 and 24 h), between one day and seven days (>1 day and 7 days), between one week and f
weeks (>1 week and 4 weeks),between one month and six months (>1 month and 6 months),and more than six
months (>6 months).
2.3.4.Dependent variables
The dependentvariables corresponded to the “Objective” ofthe AT model, including two categories:the learning
achievement dependent variables refer to measurement of cognitive outcomes such as knowledge application,retention,
problem solving…etc.The affective variables refer to measurement of motivation,interest,participation…etc.
2.4. Data analysis
2.4.1.Calculating the effect size
The following meta-analysis steps recommended by Borenstein, Hedges, Higgins, and Rothstein (2009) were employed
this study: (a) determine the effect sizes of each article,(b) determine the weighted mean effect size across articles,(c)
calculate the confidence interval for the average effect size, and (d) determine whether the effect size of any particular g
was influenced by a moderator variable based on a heterogeneity analysis (QB).
Two formulae were used to calculate the effect sizes of the studies. Cohen's d formula (Cohen, 1988) was used to dete
the effect size for the experimental research with random assignment and without a pretest:
d ¼ X1 X 2
ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
ðn11Þs 2
1þðn21Þs 2
2
ðn1þn22 Þ
r (1)
where X1 and X2 represent the mean scores, n1 and n2 represent the sample sizes, and s2
1 and s2
2 represent the variances of the
experiment and control groups, respectively.
For experimentalor quasiexperimentalresearch with pretests,it was proposed that the pretest should be taken into
consideration instead of using the posttest in order to mitigate possible selection bias (Furtak, Seidel, Iverson, & Briggs, 2
Morris, 2008). Hence, the formula developed in Comprehensive Meta Analysis (version 2.0) was used to obtain effect size
research with pre- and posttests:
ESPre=Post Test Two Groups¼ X1 Post X 1 Pre X2 Post X 2 Pre
SDPost
(2)
where X1 Pre and X1 Post represent the mean scores of the experimental group for the pretest and posttest, respectively, an
X2 Pre and X2 Post represent the mean scores of the control group for the pretest and posttest,respectively.SDPost can be
calculated as follows:
SDPost ¼
ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
n2 Post 1 s2
2 Post þ n1 Post 1 s2
1 Post
n2 Post þ n1 Post 2
s
(3)
where n1 Post and n2 Post represent the sample sizes of the experimental and control groups,respectively,for the posttest,
while s2
1 Post and s2
2 Post represent the variances of the experimental and control groups, respectively.
The two types of effect sizes were calibrated using the sample weights to calculate a Hedges'g according to
g ¼ 1 3
4df 1 d (4)
Y.-T.Sung et al./ Computers & Education 94 (2016) 252e275256

2.4.2.Evaluating publication bias
The fail-safe N (i.e.,classic fail-safe N) of Rosenthal(1979) was used to estimate how many insignificant effect sizes
(unpublished data) would be necessary to reduce the overall effect size to an insignificant level. The comparison criterion w
5nþ10, where n is the number of studies included in the meta-analysis. If the fail-safe N is larger than 5nþ10, it means tha
estimated effect size of unpublished research is unlikely to influence the effect size of the meta-analysis.Moreover,the
present study also adopted Orwin's fail-safe N (Orwin, 1983) to estimate the number of missing null studies that would be
required to bring the mean effect size to a trivial level.
3. Results and discussion
3.1. Descriptive statistics information
Table 1 presents the distribution of moderator variables and their corresponding effect sizes (g).In total there were 110
articles, 419 effect sizes, and 18 749 participants. The largest proportion of studies involved the college-student-level learn
stage (38.4%); the next largest group was elementary-school students (33.9%). More studies used learning-oriented softw
(62.7%) than general-purpose software (34.5%).Handheld devices (including PDA,cell phone,iPod, MP3 player,digital pen,
pocket dictionary,and classroom response system) were the most widely studied ofthe hardware (72.7%),followed by
laptops (21.8%, including laptop, pad, tablet PC, and e-book reader). The largest proportion of studies were set in the class
(50.0%), followed by outdoors (15.5%) and unrestricted settings (16.4%). For teaching methods, self-directed study (30.9%
the most frequently researched,and the most frequently studied intervention duration was >1 month and 6 months
(32.7%),followed by > 1 week and 4 weeks (25.5%) and 4 h (20.9%).Finally,language arts were the most often studied
domain subject (34.7%),followed by science (22.9%).
In addition,among those moderating variables,the evolution of hardware used,implementation setting,and domain
subjects may have seen the greatest amount of change during 1993e2013. The trends of those moderating variables durin
the two decades are shown in Figs. 1e3.Fig. 1 shows the evolution of the use of different mobile devices.Compared with
laptop and mixed categories,handheld devices (e.g.,cell phone) had been used more since 2009e2013 and showing an
obviously rising trend. Moreover, Fig. 2 shows the evolution of the use of different implementation settings. Compared with
informal settings (e.g.,museums; outdoors) and unrestricted categories,formal settings (e.g.,classroom; laboratories) had
been set more since 2004e2008 and showing an obviously rising trend.Finally,Fig. 3 shows the evolution of the domain
subject.Compared with other domain subjects,language arts had been studied more since 2009e2013 and showing an
obviously rising trend.
3.2. Overall effect size for learning achievement
The distribution of the effect sizes of the 110 articles is shown in Fig.4. The forest plot of effect sizes and the 95%
confidence interval of the 110 articles are shown in Appendix B. There were two unusually large effect sizes, g ¼ 4.045 (H
& Lee, 2011) and g ¼ 3.050 (Wu,Sung,Huang,Yang,& Yang, 2011),which were larger than the average effect size for the
entire collection of 110 articles (g ¼ 0.628) more than three standard deviations, and so these were not included in furthe
analyses (Lipsey & Wilson,2000). Using the procedure ofLipsey and Wilson (2000) with a random-effects modelto
integrate the effect sizes of the 108 articles, there was an overall moderate mean effect size of 0.523, with a 95% confiden
interval of 0.432e0.613.Researchers (e.g.,McMillan, Venable, & Varier, 2013; Van der Kleij, Feskens, & Eggen, 2015) have
proposed that Hattie's (2009) criterion is appropriate for evaluating the effect sizes in educational contexts. Therefore, we
adopt Hattie's (2009) criterion to interpret the effect size of our research,in which an effect size of 0.60 is high,around
0.40 is medium, around 0.20 is low, and <0.20 is with little significant meaning. In this study it was found that using mobil
devices in education had a medium effect size for learning achievement; in other words, 69.95% of learners using a mobile
device performed significantly better in dependent variables related with cognitive achievement than those not using
mobile devices.
The Q statistics show that the effect sizes in the meta-analysis were heterogeneous (Qtotal ¼ 626.302, z ¼ 11.315, p < .001),
which indicates that there are differences among the effect sizes resulting from factors other than subject-level sampling
error,such as the diversity of the learning stage,the hardware used,and the teaching methods.
Furthermore,we also conducted an analysis for the studies related to the affective variables (such as motivation,
engagement, attitude, satisfaction, preference). The overall mean effect size of the 22 articles was 0.433 (z ¼ 6.148, p ¼
with a 95% confidence interval of 0.295e0.570.According to Hattie's criterion,there is a medium effect size for affective
variables when using mobile devices in educational context.
The overallmean effect size for learning achievement in this meta-analysis was 0.523,meaning that learning with
mobiles is significantly more effective than traditional teaching methods that only use pen-and-paper or desktop com-
puters. Compared to past comparisons of effects between using computers and not using computers in education, the effe
size of using mobile devices reported herein seems larger than those found in meta-analysis into desktop-computer-based
instruction, such as in the studies of Kulik and Kulik (1991) and Tamim, Bernard, Borokhovski, Abrami, and Schmid (2011),
who found mean effect sizes for computer-based instruction of 0.30 and 0.35,respectively.One of the reasons for the
different effect sizes may be differences in the features of desktops and mobile devices; however,there are alternative
Y.-T.Sung et al./ Computers & Education 94 (2016) 252e275 257

explanations,including differences in the meta-analysis methodology,dependent variable measurements,or software
employed. Whether computer-based instruction would be able to enhance students'learning motivation remained
equivocal (e.g.,Jabbar & Felicia,2015; Wouters,van Nimwegen,van Oostendorp,& van der Spek,2013).Our study found
that mobile learning was able to facilitate students' affective learning outcomes, which provides more convergent eviden
for the effects of using computers in learning and teaching.Possible reasons may include that mobile learning integrated
Table 1
Categories and learning achievement effect sizes for 110 articles.
Variable Category Number of studies
(k)
Number of effect
sizes
Proportion of
studies
Proportion of effect
size
Effect size
(g)
Learning stage 1. Kindergarten 1 2 0.009 0.005 0.103
2. Elementary school 38 97 0.339 0.232 0.654
3. Middle school 10 47 0.089 0.112 0.512
4. High school 10 47 0.089 0.112 0.390
5. College 43 128 0.384 0.305 0.599
6. Adults 2 4 0.018 0.010 2.474
7. Mixed 8 94 0.071 0.224 0.084
Intervention
duration
1. Not mentioned 7 23 0.064 0.055 0.782
2. 4 h 23 86 0.209 0.205 0.521
3. > 4,24 h 2 18 0.018 0.043 0.385
4. >1, 7 days 5 9 0.045 0.021 0.369
5. >1 week,4 weeks 28 95 0.255 0.227 0.643
6. >1 month, 6 months 36 100 0.327 0.239 0.630
7. >6 months 9 88 0.082 0.210 0.290
Hardware used 1. Not mentioned 2 8 0.018 0.019 1.421
2. Handhelds 40 87 0.364 0.208 0.743
3. Laptop 14 109 0.127 0.260 0.276
4. Tablet PC 8 19 0.073 0.045 0.615
5. Cell phone [phone].676
6. iPod or MP3 player 5 16 0.045 0.038 0.524
7. E-book reader 2 41 0.018 0.098 0.693
8. Digital pen 1 1 0.009 0.002 0.217
9. Pocket dictionary 2 11 0.018 0.026 0.160
10. Classroom response systems 8 31 0.073 0.074 0.369
11. Mixed 4 12 0.036 0.029 0.273
Software used 1. Not mentioned 3 29 0.027 0.069 0.355
2. General purpose 38 223 0.345 0.532 0.494
3. Learning-oriented 69 167 0.627 0.399 0.626
Implementation
setting
0. Not mentioned 2 3 0.018 0.007 0.700
1. Classroom 55 242 0.500 0.578 0.487
2. Museum 4 13 0.036 0.031 0.833
3. Laboratory 3 12 0.027 0.029 0.329
4. Outdoors 17 27 0.155 0.064 0.760
5. Unrestricted 18 94 0.164 0.224 0.480
6. Workplaces 3 14 0.027 0.033 0.247
7. Mixed 8 14 0.073 0.033 1.032
Teaching method 1. Not mentioned 9 84 0.082 0.200 0.186
2. Lectures 13 45 0.118 0.107 0.556
3. Discovery and exploration 13 25 0.118 0.060 0.920
4. Cooperative learning 9 60 0.082 0.143 0.261
5. Problem-solving 10 32 0.091 0.076 0.572
6. Game-based learning 4 7 0.036 0.017 0.404
7. Self-directed study 34 122 0.309 0.291 0.521
8. Podcasting 1 6 0.009 0.014 0.153
9. Computer-assisted testing 6 8 0.055 0.019 0.660
10. Project-based learning 1 7 0.009 0.017 2.551
11. Mixed 10 23 0.091 0.055 0.847
Domain subject 1. Language arts 41 169 0.347 0.403 0.593
2. Social studies 5 10 0.042 0.024 0.776
3. Science 27 78 0.229 0.186 0.578
4. Mathematics 12 41 0.102 0.098 0.338
5. Multidisciplinary 1 6 0.008 0.014 0.333
6. Specific abilities 5 24 0.042 0.057 0.103
7. Health-care programs 7 18 0.059 0.043 0.535
8. Education 3 6 0.025 0.014 0.381
9. Psychology 3 7 0.025 0.017 0.467
10. Computer and information
technology
14 60 0.119 0.143 0.716
Y.-T.Sung et al./ Computers & Education 94 (2016) 252e275258

more diverse type of teaching/learning strategies and involved more different learning scenarios in different situations (se
next section for more descriptions). However, because many of the articles included in our study used teaching programs
lasted for very short-term durations (see next section), the effect of novelty for technology should be taken into
consideration.
Fig. 1. Histogram of the hardware used in mobile devices assisted learning across time.
Fig. 2. Histogram of the implementation setting in mobile devices assisted learning across time.
Fig. 3. Histogram of the domain subjects in mobile devices assisted learning across time.
Y.-T.Sung et al./ Computers & Education 94 (2016) 252e275 259

3.3. Effect sizes of learning achievement for moderator variables
To learn more about the effects of moderating variables on mobile devices with teaching and learning,this study con-
ducted analyses for the effects of learning achievement with moderator variables. Because there were only 22 studies wh
related to affective dependent variables can calculate effect size, which is not comprehensive enough to cover different le
of moderating variables, the moderator analyses did not include the affective effects.
As indicated in Table 1,some levels of the moderator variables included small samples,and so a few of the levels were
merged within some moderator variables. For the learning stage, kindergarten and elementary school were combined int
“young-children” category; middle schools and high schools were combined into “secondary-schoolers;” and college and
graduate students,teachers,and working adults were combined into “adult users.” With respect to the hardware,laptops,
tablet PCs, and e-book readers were combined into a “laptops” category, while PDAs, iPods, MP3 players, cell phones, dig
pens,dictionaries,and classroom response systems were bundled together to form one “handheld” category.In terms of
function, digital pen is different from other handheld devices, such as iPod, PDA, and smart phone. Also, there was only o
study on digital pen. Therefore, it was excluded in our moderator analysis. In terms of the settings, classrooms, laboratori
and workplaces were combined into “formal learning environments,” while museums and outdoors were combined into
“informal learning environments” (Dierking, Falk, Rennie, Anderson, & Ellenbogen, 2003). Intervention durations were als
combined, with 4 h, > 4 and 24 h, and >1 day and 7 days becoming “ 1 week.” For domain subjects, specific abilities and
multidisciplinary were combined into “domain-general subjects.” In addition, health-care programs, education, psycholog
and computer and information technology were combined into “professional subjects.” For teaching methods, discovery a
exploration,problem-solving,and project-based learning were combined into “inquiry-oriented learning.” Moreover,the
learning methods of self-directed study and podcasting were combined into “self-directed study.” Table 2 list the effect si
for the moderator variables.
3.3.1.Learning stage
Table 2 indicate that young children had a high effect size on learning achievement (g ¼ 0.636, z ¼ 8.000, p < .001),
adults (g ¼ 0.552, z ¼ 7.360, p < .001) and secondary-schools (g ¼ 0.451, z ¼ 4.274, p < .001) had medium effect sizes
Mixed (g ¼ 0.086, z ¼ 0.503, p ¼ .615) did not show significant effect sizes. The QB achieved significance (QB ¼ 9.226, p ¼ .026),
meaning that the mean effect size different significantly between the categories.
The results indicated that mobile-assisted learning/instructions were not effective for groups with mixed-age students.
The possible reason may be that it is difficult to design appropriate teaching method or material for students with differen
needs and competence in the same group.
3.3.2.Hardware used
Table 2 gives the effect sizes for the usage of different types of hardware in mobile learning.While ignoring the “not
mentioned” category,handheld devices (g ¼ 0.591,z ¼ 10.992,p < .001) were associated with a medium effect size,while
laptops (g ¼ 0.309, z ¼ 3.350, p ¼ .001) were associated with a low effect size. The QB was significant (QB ¼ 18.426, p < .001),
indicating that the effect sizes differed significantly among the various categories.The R2 was 7%,meaning that 7% of total
between-study variance in effects can be explained by hardware used.
The positive learning outcomes of implementing handhelds could be attributed to their features. For example, to make
of the portability and communication functionality of cell phones, the short message service were used to help teach fore
language vocabulary (e.g., Bas¸oglu & Akdemir, 2010; Lu, 2008; Saran, Seferoglu, & Cagıltay, 2012), and because the messages
Fig. 4. Histogram of the effect sizes of the 110 articles.
Y.-T.Sung et al./ Computers & Education 94 (2016) 252e275260

were short, students could efficiently use their short periods of spare time to take small “bites” out of the material. Anothe
example is the use of cell phones to communicate, make records, and give and receive feedback. These functions can rem
students about their learning schedule,and promote self-awareness (Liu,Tao,& Nee,2008; Runyan et al.,2013) and self-
regulation (Kondo et al.,2012).The aforementioned advantages of the handhelds created the environment for seamless
learning,which should be able to prompt better learning outcomes.
According to the analysis result,the implementation of handhelds induced higher learning outcomes than the imple-
mentation of laptops. It is perhaps due to the fact that studies with handhelds tend to integrate innovative teaching metho
(Lu, 2012).Among the handheld research,there was 31.6% employing teaching methods,such as inquiry-oriented and
cooperative learning (Table C1 of Appendix C). In contrast, in a large portion of the laptop-related studies (50.0%, Table C1
Appendix C),the computers were placed into the classroom and used simply for lectures,self-directed study,or with no
specific teaching methods.
It is important to note here that most ofthe research on handhelds in education has involved only short-term in-
terventions,with 29.1% (Table C2 of Appendix C) testing their effectiveness within 1 week.These users of handhelds also
probably experienced a transient effect because of their novelty (Kulik & Kulik, 1991).In contrast,most of the research on
laptops involved long-term use, with 25.0% (Table C2 of Appendix C) being used for > 6 months. Long-term laptop use wit
appropriate supporting logistics may reduce both the students'level of commitment and the teachers'willingness to use
computers to integrate their teaching with the students' learning (Drayton, Falk, Stroud, Hobbs, & Hammerman, 2010; Ina
Lowther,2010; Penuel,2006).
Table 2
The learning-achievement effect sizes of categories and their related moderator variables.
Category k g z 95% CI QB R2
Learning stage 9.226* 0%
1. Young children 39 0.636 8.000*** [0.480e0.791]
2. Secondary-schoolers 20 0.451 4.274*** [0.244e0.658]
3. Adults 43 0.552 7.360*** [0.405e0.700]
4. Mixed 8 0.086 0.503 [0.248 to 0.419]
Hardware used 18.426*** 7%
1. Not mentioned 2 1.416 4.491*** [0.798e2.033]
2. Handhelds 78 0.591 10.992*** [0.485e0.696]
3. Laptops 24 0.309 3.350** [0.128e0.490]
4. Mixed 3 0.044 0.173 [0.460 to 0.548]
Software used 3.025 0%
1. Not mentioned 3 0.347 1.262 [0.192 to 0.886]
2. General purpose 37 0.429 5.407*** [0.273e0.584]
3. Learning-oriented 68 0.590 9.699*** [0.471e0.709]
Implementation setting 7.993* 8%
1. Not mentioned 2 0.701 2.069* [0.037e1.365]
2. Formal settings (classroom,laboratory,hospital) 60 0.430 7.328*** [0.315e0.545]
3. Informal settings (museum, outside) 21 0.768 7.096*** [0.556e0.980]
4. Unrestricted 25 0.550 5.887*** [0.367e0.734]
Teaching method 26.744*** 12%
1. Not mentioned 9 0.186 1.369 [0.080 to 0.452]
2. Lectures 12 0.394 3.120** [0.146e0.641]
3. Inquiry-oriented learning 24 0.844 8.400*** [0.647e1.041]
4. Cooperative learning 9 0.261 1.673 [0.045 to 0.566]
5. Game-based learning 4 0.407 1.922 [0.008 to 0.822]
6. Self-directed learning 34 0.440 5.492*** [0.283e0.597]
7. Computer-assisted testing 6 0.656 3.661*** [0.305e1.006]
8. Mixed 10 0.839 5.702*** [0.550e1.127]
Intervention duration 4.924 0%
1. Not mentioned 7 0.770 4.181*** [0.409e1.130]
2. 1 week 30 0.479 5.175*** [0.298e0.661]
3. >1,4 weeks 27 0.552 5.644*** [0.360e0.743]
4. >1 month,6 months 35 0.566 6.870*** [0.405e0.728]
5. >6 months 9 0.287 1.942 [0.003 to 0.577]
Domain subjects 9.108 0%
1. Language arts 39 0.473 6.352*** [0.327e0.619]
2. Social studies 5 0.768 3.682*** [0.359e1.177]
3. Science 27 0.565 6.397*** [0.392e0.738]
4. Mathematics 12 0.337 2.628** [0.086e0.588]
5. General 6 0.151 0.868 [0.190 to 0.491]
6. Professional subjects 27 0.592 6.808*** [0.422e0.763]
Note.CI ¼ confidence interval.
*p < .05; **p < .01; ***p < .001.
Y.-T.Sung et al./ Computers & Education 94 (2016) 252e275 261

3.3.3.Software used
The data given in Table 2 indicated that the effect sizes for learning-oriented software (g ¼ 0.590,z ¼ 9.699,p < .001)
approached high effect size, and general-purpose software (g ¼ 0.429, z ¼ 5.407, p < .001) had medium effect size. TheB did
not achieve significance at the p < .05 level (QB ¼ 3.025,p ¼ .220),which means that the average effect size did not differ
significantly between the two categories.
According to the survey results,after 1990 most of the software that the teachers used was actually made for general
purposes (e.g.,word processors,spreadsheets,and web browsers) (Becker,1991,2001; Drayton et al.,2010),instead of
learning-oriented software tailored for teaching and learning tasks. This made it difficult for most teachers to achieve the
of greater efficiency and effectiveness in education using the technology-adapted instruction that they applied (Sung &
Lesgold,2007; Weston & Bain,2010).The present study indicates that the aforementioned shortage of learning-oriented
software has improved,with software specifically designed for teaching and learning goals or activities being used in
62.7% of the research,and only 34.5% of the studies using general-purpose software.
Even though there was no significant difference between learning oriented software and general-purpose software in o
research,learning oriented software showed interesting features in mobile based learning.First,the software and the cur-
riculum were closely integrated.As an example,Looi and colleagues (Looi et al.,2011; Zhang et al.,2010) combined educa-
tional software with cell phones to make a mobilized curriculum for elementary-level natural science,which was able to
implement seamless learning in classrooms, outdoors, and in the home. Their designs were not only based on the pedago
inquiry learning, but also promoted formative assessment, cooperative learning, and social interaction in teaching tasks.
second feature of learning-oriented software is that it provides diverse educational activities. Within the studies included
this research, those in which learning-oriented software was used implemented various educational methods, most of wh
were related to inquiry,cooperation,game-based learning,problem-solving, and formative assessment. On the other hand,
for those studies using general-purpose software, lectures and self-directed study were implemented. Moreover, among t
37 studies with the general-purpose software, 6 of them did not mention the teaching methods (Table C1 of Appendix C).
third feature of learning-oriented software is its ability to enable elaborate and efficient designs for teaching strategies an
learning scenarios. The steps and procedures of the aforementioned teaching strategies, such as inquiry, cooperation, ga
based learning,and problem-solving,were all fairly complex.Learning-oriented software allowed teachers with no pro-
gramming skills to flexibly and efficiently implement mobile-assisted education. For example, Lan et al. (2007, Lan, Sung,
Chang,2009) designed an English foreign-language learning model based on cooperative learning and reciprocal teaching
Procedures related to reciprocal teaching,such as reading text,questioning and probing,answering and feedback,were all
designed for specific modules that could be further arranged according to the needs of different teaching situations. Teac
could substitute their own material, or even completely customize their program. In addition, the research of Roschelle et
(2010) on cooperative learning set out three stages of design and implementation for modules, modules for experiments
classroom tryouts,and modules for classroom implementation.After 2 years of designs,tryouts,and revisions,their PDA-
based cooperative learning modules were able to integrate the mathematics content,cooperative learning procedures,and
teacher-training programs for efficient use in the classroom.
3.3.4.Implementation settings
As indicated in Table 2, when the “not-mentioned” category is ignored, informal settings had a high effect size (g ¼ 0.
z ¼ 7.096,p < .001),while unrestricted settings (g ¼ 0.550,z ¼ 5.887,p < .001) and formal settings (g ¼ 0.430,z ¼ 7.328,
p < .001) had medium effect sizes. The effect size of informal setting was larger than that of the formal setting, as the 95
confidence intervals of the two effect sizes did not overlap.The QB was significant (QB ¼ 7.993,p ¼ .046),showing that the
average effect size differed significantly with the category. The R2 was 8%, meaning that 8% of total between-study variance in
effects can be explained by implementation settings.
As found in the present study,the effect size was larger for using mobile devices in the outdoors and informal locations
than for using them in more formal places. Some observations on the use of mobile devices in informal places may be he
for explaining this phenomenon.First, this could be due to the motivation induced by the novelty of the technology and
activities. Students are keen to go outside or to museums to learn, and combining this with the use of novel learning tool
facilitate learners'motivation (e.g.,Zhang,Sung,Hou, & Chang,2014).The second is that most of the informal educational
models,software functionality,and hardware characteristics were closely integrated in the included research,and this
probably improved the learning effects.In the present study,77.9% of informallearning-oriented software was specially
designed for specific learning scenarios in specific settings (Table C3 ofAppendix C).These more elaborately designed
teaching procedures allow educational effects to become more apparent. For instance, when learning in museums, one o
important issues is how to guide learners' attention to exhibitions through an appropriate learning process, and informati
and interesting activities to promote interaction among visitors, computers, and the historical contexts (e.g., Hsi, 2003; S
Hou, Liu, & Chang,2010).Several of the studies included in our research combined the models of role-playing games and
problem-solving to immerse learners in the historical events, engaging them to observe and learn target exhibits more de
(e.g.,Huizenga,Admiraal,Akkerman,& ten Dam,2009; Sung,Chang,Hou, & Chen,2010).Similarly,researchers are also
concerned with how to make the fieldwork involved in the natural and social sciences structuralized, focused, and efficien
rather than loose, absent-minded, and ineffective. In several studies (e.g., Hwang, Chu, Lin, & Tsai, 2011; Liu, Tan, & Chu
the researchers tried to make observations,note-taking,problem-solving,information exchanges,and discussion more
Y.-T.Sung et al./ Computers & Education 94 (2016) 252e275262

structured,and to sharply focus the students'learning process by integrating mobile devices with other peripheral devices
such as camcorders,positioning functions,and measuring facilities.
3.3.5.Teaching methods
The data regarding the effect size for different teaching methods are given in Table 2. Three high effect sizes were foun
inquiry-oriented (g ¼ 0.844,z ¼ 8.400,p < .001),mixed methods (g ¼ 0.839,z ¼ 5.702,p < .001),and computer-assisted
testing (g ¼ 0.656,z ¼ 3.661,p < .001).Lectures (g ¼ 0.394,z ¼ 3.120,p ¼ .002) and self-directed study (g ¼ 0.440,
z ¼ 5.492,p < .001) were around medium effect sizes.However,cooperative learning (g ¼ 0.261,z ¼ 1.673,p ¼ .094) and
game-based learning (g ¼ 0.407,z ¼ 1.922,p ¼ .055) did not show significant effect sizes.The QB achieved statistical sig-
nificance (QB ¼ 26.744, p < .001), indicating that the average effect sizes differed significantly among the various categorie
The R2 was 12%, meaning that 12% of total between-study variance in effects can be explained by teaching methods.
The unique features of mobile devices can enhance the essential functionalities of certain specific teaching methods, an
thus promote educational outcomes.Because each student has his own mobile device,this “individuality” combined with
wireless communication enabled more accessible self-paced and self-directed study. Combining the features of individualit
and instant message delivery resolves the past difficulties of putting instant formative assessment into the classroom (e.g
Chen & Chen,2009),such that these assessments can even be performed outdoors with equal ease (e.g.,Shih,Kuo, & Liu,
2012).Another feature that empowers the teaching and learning process is the portability and context awareness of mo-
bile devices. These two features allow learners to exploit the information in the environments in which they are situated, a
to retrieve, record, and react to the data needed to resolve their learning issues by traversing multiple learning environme
such as fieldwork and museums (e.g.,Tan,Liu, & Chang,2007).
It is note-worthy that although researchers (Kukulska-Hulme & Shield, 2008; Roschelle & Pea, 2002) have proposed tha
conveying information and giving feedback via mobile devices can help to keep learners in touch with their peers, promote
discussions, and to facilitate the effects of cooperative learning, our study found that in general theses features did not he
enhance cooperative learning outcomes.The researchers ofcooperative learning used mobile devices'features of in-
dividuality and sharing coupled with mechanisms for enhancing social interaction, such as co-constructing concept maps (
& Wu, 2006),peer evaluation (Lan et al.,2007; Roschelle et al.,2010),and building consensus (Zurita & Nussbaum,2004).
Interestingly,perhaps these methods had facilitated the positive interactive relationships among team members (e.g.,Lan
et al., 2007; Zurita & Nussbaum, 2004), however, these teaching methods did not enhance the learning outcomes
compared with the cooperative scenarios without using mobile devices. There are at least two possible reasons for the res
Firstly, the cooperative learning tasks in those studies, when coupled with mobile devices, may be helpful for increasing th
interactive behaviors and socialcohesions among team members.However,the increased socialcohesion may not be
powerful enough to enhance learning achievement. As Slavin (2012) proposed, whether higher social cohesion is related w
higher learning achievement is not conclusive.Those methods used in the above-noted research may be insufficient to
empower the cognitive elaboration processes imperative for enhancing students' learning. In those studies students in bot
the control and treatment groups received cooperative treatments: The only difference was mobile-device usage. Thus,the
inherent effects of mobile devices may not go much beyond sharing,communicating,and consensus building.Therefore,
elaborate design of learning scenarios, such as mechanisms for prompting questioning and explanatory strategies (Byun, L
& Cerreto, 2014; Gillies & Haynes, 2011) specifically related with the learning content, may be needed to be incorporated
the mobile-device based activities in order to enhance students' cognitive elaboration processes and outcomes. The secon
possible reason is that the intervention durations of the mobile-based cooperative learning programs were not long enoug
to produce positive effects.Researchers have proposed that severalweeks of duration is helpfulfor producing positive
learning outcomes in cooperative learning (Slavin, 1993), as sufficient time is important for learners to get familiar with tea
members,tasks,and required procedure (Slavin, 1977).Time for familiarization may be even more important for mobile-
devices based cooperative learning because learners need time to get familiar not only with members,tasks,and proce-
dure,but also with the hardware and software.Most of the research included in our study lasted for less than one month,
which may be too short for the programs to produce sound effects.
Another note-worthy finding is that game-based learning did not achieve a significant overall effect in mobile learning,
either. The major reason may be that most of the studies (e.g., Ketamo, 2003; Kim et al., 2011; Riconscente, 2013) focuse
using the mobile devices to provide learners with a handy and individualized game-based environment to enhance their
motivation and engagement. However, the relationships between the concepts to be learned and the content of the game
not have been closely integrated,and therefore the effects of learning might not have been illustrated.
Researchers have pointed out that computer interventions in education have not yet led to practical implementations o
innovative educational methods (Ertmer & Ottenbreit-Leftwich, 2010; Gerard, Varma, Corliss, & Linn, 2011). Contrarily, it w
found in the present study that mobile devices seemed to elicit much more diverse and innovative educational methods fr
researchers.
3.3.6.Intervention duration
When the “not-mentioned” category is ignored, interventions of >1 month and 6 months duration (g ¼ 0.566, z ¼ 6.87
p < .001), those of >1 week and 4 weeks duration (g ¼ 0.552, z ¼ 5.644, p < .001), and those 1 week had medium effec
sizes (g ¼ 0.479, z ¼ 5.175, p < .001). Interestingly, interventions conducted for durations of >6 months had a non-signifi
Y.-T.Sung et al./ Computers & Education 94 (2016) 252e275 263