Research Limitations: Honesty, Common Sense in Learning Environments

This document, an editorial from the journal "Interactive Learning Environments," addresses the crucial need for honesty and common sense when conducting and interpreting research in educational technology. The author highlights the prevalence of technology integration in education and the associated research, emphasizing the importance of acknowledging research limitations. The editorial discusses various limitations, including issues of internal and external validity, response bias, and the Hawthorne effect. It stresses the importance of researchers being aware of potential biases and including statements of limitations in their work. The author also encourages researchers to consider the learner's experience and to pilot new designs before implementing them on live learner cohorts. Finally, the editorial underscores the role of peer reviewers in identifying limitations and promoting rigorous research practices, ultimately advocating for cautious statements, clear identification of limitations, and a focus on future research directions.

See discussions, stats, and author profiles for this publication at: https://www.researchgate.net/publication/325761416
Research limitations: the need for honesty and common sense
Article in Interactive Learning Environments · July 2018
DOI: 10.1080/10494820.2018.1486785
CITATIONS
3
READS
2,097
1 author:
Some of the authors of this publication are also working on these related projects:
BSc Professional Development in Business - blend but mostly onlineView project
Teaching Research Methods in HEView project
Sue Greener
University of Brighton
71PUBLICATIONS234CITATIONS
SEE PROFILE

Interactive Learning Environments
ISSN: 1049-4820 (Print) 1744-5191 (Online) Journal homepage: http://www.tandf
Research limitations: the need for hone
common sense
Sue Greener
To cite this article: Sue Greener (2018) Research limitations: the need for hon
sense, Interactive Learning Environments, 26:5, 567-568, DOI: 10.1080/104948
To link to this article: https://doi.org/10.1080/10494820.2018.1486785
Published online: 13 Jun 2018.
Submit your article to this journal
Article views: 15
View Crossmark data

EDITORIAL
Research limitations:the need for honesty and common sense
A quick look through the articles in this issue offers a handy instant view of the focus of curren
research into learning with technologies.Educationalresearchers are overwhelmingly keen on
using technology to flip learning, to bring context into the classroom through Virtual and Augm
Reality,and to use enquiry or problem-based scenarios and game-based activities for collabora
and personal learning engagement.The concerns of teachers continue to include the development
of criticalthinking,the impact or not of learning preferences,cognitive load,peer assessment and
what constitutes learner presence.
At a time when the Pandora’s Box of the world wide web is shaken by contrasting notions a
cations of data privacy, it is not surprising to see Information Security, Data Management and
ance, and Data-Enabled Institutional Culture as three of the top ten IT issues for 2018 raised in
Review (Grajek, 2018). Leading educational researchers are aware that Higher Education is be
cally reshaped by digital technology, and this journal can provide evidence of debates within H
Education around the student’s and the organisation’s experience of digital enhancements for
ing. This issue includes two calls for papers for future Special Issues relating to learning analyt
use of “big data”, and the application of crowd-sourcing within education.
But as we press forward with research-led and practitioner-led exploratory and explanatory
studies in this field,we should perhaps take a momentto considerhow potentially misleading
such studies may easily be.Otherwise,we may fallinto the trap that Higher Education teachers
constantly warn their students about:that ofaccepting data,especially self-report but also other
quantitative data,at face value.While it is unusualfor this journalto publish smallstudies
founded on self-report data alone,particularly where a new application or course design has been
tested on only one learner cohort,it is not difficult to find reviews ofliterature in papers which
cite such work uncritically.A recent study ofthis behaviour in the tourism research field (Yüksel,
2017)found considerable response bias ofvarying kinds in the papers reviewed,but also found
that such papers could find further airing through citation.It is incumbent upon journals such as
this to emphasise the need for cautious statements where data may include bias,and for papers
always to include a statementof limitations,whetherrelated to research design,methodology,findings or the conclusions drawn.Such limitations may be included in the methodology section
or the conclusions,but without them we cannot be sure that authors have genuinely had regard
to potentialareas of exclusion or bias which affect the results they report.It is unfortunately often
the case that it falls to peer reviewers to point out the need for limitations to be included.
Limitations generally fall into some common categories, and in a sense we can make a chec
authors here.Price and Murnan (2004) gave an excellent and detailed summary of possible rese
limitations in their editorial for the American Journal of Health Education.They discussed limitations
affecting internal and external validity; internal validity relating to limitations of the study des
its internal integrity,external validity relating to the outward generalisability of the reported resu
The construction of questions within a survey instrument, the layout and type of questions in s
the levelof understanding of the respondents,an over-dependence on convenience sampling and
generalaccessdifficulties,confounding variables,statisticalinadequacies,low response rates,
absence ofor impossibility ofcontrolgroups,these allofferpotentiallimitations in educational
research. But perhaps the most common issue for papers submitted to this journal is the “Haw
effect”, the notion that an innovation or intervention may produce positive results not related
© 2018 Informa UK Limited,trading as Taylor & Francis Group
INTERACTIVE LEARNING ENVIRONMENTS
2018,VOL.26,NO.5,567–568
https://doi.org/10.1080/10494820.2018.1486785

innovation but to the presence of an innovation and that the respondents are given the focus
attention in a research study.All such limitations and more should of course be wisely considered
prior to the execution of research,but at very least must form part of the presentation of a study
for publication.
This is not just about being self-critical or particularly humble in presenting our research. Id
ing limitations, and explaining to the reader what impact these limitations have on the study r
not only demonstrates rigour but also gives the authors a chance to identify clear directions fo
research. In fact this section or paragraph can be one of the most exciting parts to read in a p
we get a sense ofthe challenges faced in the study and the forward thinking about prospective
improvements for further research which can be a rich seam of opportunity for a range of auth
As they mine this seam for research opportunities,we should also remember that we are discussing
learning here.Despite the plethora ofexperimentaland quasi-experimentaldesigns we see,the
learner often has only one “go” at learning at a time.If we start to experiment with live learner
cohorts we can let them down.Researchers in the field of digitaland interactive learning should
perhaps take to heart some lessons from early medical research and experiment first on them
Learning communities of teachers,designers and trainers can espouse beta designs readily,and can
pilot or pioneer new designs before they lead to serious limitationsof applicabilityand
generalizability.
The papers in this issue demonstrate clear and thoughtful understanding of research limitat
an attribute and research practice which we, as editors, strongly recommend to potential cont
to the journal.
References
Grajek,S.(2018).Technology and the remaking of higher education:A longer view.Educause Review,53(1),11–59.
Price,J. H.,& Murnan,J. (2004).Research limitations and the necessity of reporting them.American Journalof Health
Education,35(2),66–67.
Yüksel, A. (2017, April). A critique of “Response Bias” in the tourism, travel and hospitality research. Tourism M
59,376–384.
Sue Greener
University of Brighton,Brighton,England
[e-mail]
568 EDITORIAL
View publication statsView publication stats