BIG DATA CHALLENGES IN IOT AND CLOUD

This document provides an overview of big data challenges in IoT and cloud computing, including security and privacy issues, data mining with big data, integrity assurance solutions for big data computing applications, and more. It covers 48 references from reputable sources, including IEEE Cloud Computing, International Journal of Advances in Applied Sciences, and ACM Proceedings.

Running head: BIG DATA CHALLENGES IN IOT AND CLOUD
Big Data Challenges in IoT and
Cloud
Executive Summary
The purpose of this report is to focus on IoT
and its applications in various fields. The
application of IoT in Big Data is presented in this
report to evaluate Big Data challenges in IoT and
Cloud. The various technologies IoT has been
discussed in this report. The scope of this report
presents evaluation of Big Data challenges to
propose appropriate solutions and identify different
IoT technologies. The evaluation of project is done
through agile project methodology. The report
results in possible solutions for the challenges of
Big Data through various IoT technologies. The
report recommends possible strategies to be
undertaken while evaluating the challenges. Finally
the report concludes future opportunities of Big
Data in IoT and Cloud with possible measures
undertaken.
Table of Contents
Big Data Challenges in IoT and Cloud....................0
Executive Summary.................................................0
1. Introduction.........................................................0
1.2. Problem Statement........................................2
2. Literature Review............................................2
2.1. Introduction..................................................2
2.2. Integration of big data, IoT and Cloud.........2
2.3. Challenges of big data processing and
platforms for analytics.........................................3
2.3. Big data challenges in IoT............................3
2.4. Big data challenges in Cloud........................4
2.5. IoT technologies and applications................5
3. Issues and Solutions............................................5
4. Future Research...................................................7
5. Advantages and disadvantages............................7
6. Conclusion...........................................................8
References...............................................................9
1. Introduction
The big data, Cloud and Internet of Things are the
emerging trends in this digital world. These
technologies are not only the required technology
but also the necessity in every sector and
organization. These technologies although emerged
individually however, they have been intertwined
(Aly, Elmogy & Barakat, 2015). The increasing
digital transformation has resulted in interdependent
of these three technologies. The intertwining of
these technologies is described as huge demand for
big data has resulted in adoption of both Internet of
Things and Cloud platforms (Sun et al., 2016). The
organization uses big data and are rapidly
increasing. The big data along with Internet of
Things and Cloud has found many applications in
various industries and they are business, healthcare,
banking, financial companies and other companies
(Fazio et al., 2015). The big data is increasingly
day-by-day, it is characterized by three factors, and
they are numerous data, data is complex and data
are rapidly generated, captured and quickly
processed. Cloud Computing is the technology,
which is absorbing the bid data aspects to provide a
better integration for data and information. The
Internet of Things is a valuable technology which
visualizes, uncovers insights from various types of
data such as structured, semi-structured and
unstructured (Marjani et al., 2017). The Big Data
produces huge number of data and they are
complex to identify and evaluate. The challenges
related to Big Data in Internet of Things are making
sense of the complex data, identifying consumed
data and taking actions on the data (Da Xu, He &
Li, 2014). The challenges of big data briefly
describing in Internet of Things are as follows. The
first challenge is representation of data that contains
datasets and have certain levels, semantics,
structure, granularity, industry and accessibility (Yi,
Li & Li, 2015). The representation of data should
be proper, as improper representation of data will
minimize the effective data analysis. Hence,
representation of data is necessary to provide a
well-analyzed data for future operations. The
second challenge is reduction in data redundancy
and compression of data (Aly, Elmogy & Barakat,
2015). Generally, the datasets has high level of
redundancy that needs to be compressed. The data
after reducing redundancy is filtered and
compressed to get the actual data that is relevant to
the organization. The third challenge is

1BIG DATA CHALLENGES IN IOT AND CLOUD
management of data life cycle that is necessary to
ensure quality of data (Hashem et al., 2015). The
aspects related to big data are storage systems,
sensors and computing that poses challenges if not
managed properly. The big data has hidden values
that are dependent on freshness of data. The fourth
challenge is analytical mechanism where big data
processes huge amount of data within a limited
period (Sun et al., 2016). The traditional database
systems lack scalability and flexibility. The big data
poses challenges when intermixed with traditional
database. The fifth challenge is confidentiality of
data, which poses the challenge of maintaining and
handling of data (Hashem et al., 2015). The data are
huge sets and hence service providers are unable to
handle the data sets due to their limited capacity of
managing data. The sixth challenge is energy
management, which shows that big data in Cloud
and Internet of Things absorbs high energy
(Hassanalieragh et al., 2015). This cause effect on
environment and economy perspective. The seventh
challenge is scalability, which shows that big data
must support current and future datasets that may
change.
This shows that big data poses challenges in
Internet of Things and Cloud. The challenges are
several and they need focus and evaluation for
future prospective (Sajid, Abbas & Saleem, 2016).
However, there are several advantages of big data
analytics in Cloud and Internet of Things (Liu et al.,
2015). The advantages of big data in Internet of
Things and Cloud are reduced cost, virtualization
and instant access to infrastructure in cloud.
The big data is influencing organizations
through use of Internet of Things and cloud and
they are generating at high rates (Liu et al., 2015).
The future of big data depends on Internet of
Things and Cloud. This can be shown as follows.
The Internet of Things is the future of digital world
and it is nothing nut networks that are
interconnected to provide network services for data
(Wang & Ranjan, 2015). The cloud is a reliable
technology that helps to provide infrastructure to
the organization for storing data.
The big data and Internet of Things both
provide services as follows. The Internet of Things
provides connection of machines and sensors to be
used and big data to enable the move of network
from virtual world to real world. The smart and
connected devices give perfect explanation of
adopting big data and Internet of Things (Terzi,
Terzi & Sagiroglu, 2015). This explains that
Internet of Things and big data analytics consists of
various Internet of Things data. The big data
analytics has various levels and each level are as
follows. The real time level is used for analysing
huge data through the sensors using Greenplum and
Hana (Sajid, Abbas & Saleem, 2016). The offline
level is used for applications where requirement for
response time is not high. The existing
architecture/tools for this level are Scribe, Kafka,
Timetunnel and Chukwa (Da Xu, He & Li, 2014).
The memory level is used for data where the
volume is smaller than the cluster having maximum
memory. The existing tool for this level is
MongoDB (Terzi, Terzi & Sagiroglu, 2015). The
business intelligence level is used when the
memory level is surpassed by scale using the tool
data analysis plans. The massive level is used when
data totally surpasses the business intelligence
capacity and databases. The existing tool for this
level is MapReduce.
The Internet of Things and Cloud has
various applications in various industries and the
applications are as follows. The first domain is
transportation consisting of smart parking, driving
through 3D assistance (Sajid, Abbas & Saleem,
2016). The second domain is Smart environments
domain consisting of smart water supply, smart
home and offices. The third domain is Healthcare
domain consisting of health tracking and
pharmaceutical products. The fourth domain is food
sustainability and fifth domain is futuristic
applications domain consisting of robot taxi and
city information model (Hashem et al., 2015).
There are challenges related to Internet of Things
for future direction. The challenges are architecture,
environment innovation, technical, hardware,
privacy and security, standard, business,
development and data processing including
heterogeneous data, noisy data and massive-
intensive data. The big data issue in Cloud is based
on infrastructure provided by cloud for big data
storage (Suciu et al., 2015). This issue varies from
one organization to another organization. The large
unstructured data generated are dealt through big

2BIG DATA CHALLENGES IN IOT AND CLOUD
data analytics however; data storage in cloud poses
challenges as cloud provides various infrastructures
(Perera et al., 2015). The one technology, which is
currently being used and widely accepted, is fog-
computing technology. The fog computing provides
several benefits. The benefits are bringing data near
to the user, creation of dense geographical
distribution, true support to Internet of Things (Yi,
Li & Li, 2015). There are various Internet of Things
technologies are discussed in this report. These
technologies are near field communication, radio
frequency identification, and low energy Bluetooth
and wireless, radio protocols, LTE-A and WiFi-
Direct (Perera et al., 2015). These technologies
correlated with big data and cloud on a big platform
to provide large future opportunities. This
correlation also poses challenges in the real world
environment if not managed properly and carefully
(Aly, Elmogy & Barakat, 2015). The big data is
rapidly growing and to accommodate big data in
Internet of Things and Cloud there are various
technologies and techniques that will be discussed
in further sections. The problem discussed in this
report is how big data poses challenges in Internet
of Things and cloud.
The above are the concerns and problems of
the topic discussed and the topic will be elaborated
in further sections through prior researches and
methodology (Aly, Elmogy & Barakat, 2015). The
purpose of this report is to focus on the challenges
of big data in Internet of Things and Cloud to
evaluate various existing and developing
technologies for Internet of Things. The report also
focuses on the different applications of Internet of
Things along with focus on big data and cloud in
real world environment (Marjani et al., 2017). The
report presents prior researches on big data
challenges in Internet of Things and cloud to
evaluate the technologies. The report provides
issues of the challenges and the solution for these
challenges with focus on future research. The report
presents the methodologies through which the
Internet of Things, big data and cloud are evaluated
in this report. The methodologies used in the
literature review section are described and
compared to choose the best-suited methodology
for this report. The report also presents tables and
graphs to illustrate more about this topic and
support the justifications made in this report.
The outline of this report is as follows. The
first section is literature review on past and present
technologies related to big data challenges in
Internet of Things and Cloud. This section briefly
describes the researches done on big data, Internet
of Things and Cloud to identify the Internet of
Things technology. The given section covers the
major part of the report that provides brief details of
the problems and challenges of the topic discussed
in this report. The second section is methodology
section and it contains past and current
methodologies used in the literature review section.
This section also consist comparison of
methodologies from different perspectives such as
simplicity, efficiency, cost and time saving,
connectivity and feasibility. This section then
provides the best methodology or combination of
methodology to suit the purpose of this report based
on several factors. The report then provides issues
along with solutions for the future research
perspective.
1.2. Problem Statement
The challenges of Big Data in IoT and cloud
have posed various difficulties in adoption of Big
Data when used along with IoT. These challenges
are important to solve due to data storage, data
security and integrity of data related with Big Data.
The Big Data poses problems such as huge and
complex data and data are biased. There are several
Internet of Things technologies which help to solve
challenges of Big Data such as Hadoop and Spark.
There are several methods that have been presented
to tackle the challenges of Big Data and the most
important method is use of Hadoop. The Big Data
challenges in IoT and Cloud need to be solved for
future purposes.
2. Literature Review
2.1. Introduction
This section focuses on the challenges of big
data in IoT and Cloud computing through prior
researches done on this topic. Big data is
intertwined with IoT and Cloud to provide a better
platform for managing data (Da Xu, He & Li,

3BIG DATA CHALLENGES IN IOT AND CLOUD
2014). The IoT, Big data and Cloud provide better
integration for every sector however, there are
challenges with these integration. These challenges
are focused in the following sections.
2.2. Integration of big data, IoT and Cloud
The attraction and interest of big data in
every industry is increasing rapidly. The industries
from social networks to multimedia to business
transactions all has adopted or on a verge of
adopting these technology. There are 4 Vs related
big data, volume, variety, velocity and value that
poses challenges for researchers (Ahmed et al.,
2017). The storage of and processing of data is a
challenge for every sector. The cloud computing is
the recent technology that has emerged to solve the
big data problems by providing cost effective data
center to store huge data. However, there are
concerns related to service qualities of cloud such
as privacy of user and data security (Da Xu, He &
Li, 2014). The security concern is the top most
priority regarding the use of cloud. IoT is a
platform for next-generation computing which is
integrated in every sector from an individual’s life
to an organization. This platform works in real-
timed fashion in every sector and has large datasets
that is generated over a particular time ((Ahmed et
al., 2017). The prior researches shows that there are
areas associated with big data security in cloud and
IoT which needs to be looked upon for future
researches (Liu et al., 2015). The areas are
efficiency including communication and storage
and computation time, security and scalability.
2.3. Challenges of big data processing and
platforms for analytics
The various platforms big data platforms
poses challenges in IoT and cloud and they are
discussed as follows.
Challenges of Apache Hadoop: lack of
encryption at network and storage level, unsuitable
for small data, limited flexibility and high input-
output overhead (Ahmed et al., 2017).
Challenges 1010data: data extraction, data
loading and data transformation.
Challenges of Cloudera, a Hadoop based
framework, are that there is no software and
hardware systems of its own and relies on third
parties.
Challenges of SAP-Hana are: all data must
be read that is in a row even though a few columns
of data are required for accessing.
Challenges of Hadoop Autonomy Vertica
Enterprise security (HAVEn): a large database is
generated by an increment in tenants where all
operations related to release processes and lock
holding are decelerated (Marjani et al., 2017).
Challenges of Hortonworks: it cannot
minimize the node-groups in the cluster generated
by the system.
Challenges of pivotal big data suite: it has
several unresolved issues that hinders its adoption
in industries.
Challenges of Infobright: Infobright
optimizer cannot optimally answer all the queries
(Ahmed et al., 2017).
Challenges of MapR: it has larger
complexity as compared to Hadoop.
2.3. Big data challenges in IoT
The big data challenges in IoT are as
follows that arise due to introduction of big data in
IoT. The challenges are as follows that are
discussed here.
 Privacy issue- The issue of privacy arises
when there is any compromise of system for
restoring sensitive data using tools of big
data analytics. There is a problem regarding
data mining where the privacy issue is high
(Marjani et al., 2017). The users find it
difficult to trust on usage of big data in IoT
due lack of proper service level agreement
for sensitive data misuse use such personal
information. The other issues is security
risk, related with IoT data, in heterogeneity
of using devices and data generated such as
data types, communication protocols and
raw devices (Ahmed et al., 2017). The other
challenges emerges due to generated data
through IoT are as follows. The challenges
are timely update of systems, managing
identification of traffic patterns from
suspicious and legitimate ones,
interoperability and protocol convergence
(Pfarr, Buckel & Winkelmann, 2014).

4BIG DATA CHALLENGES IN IOT AND CLOUD
However, true and accurate open ecosystem
having standard APIs is required to prevent
problems of reliability and interoperability.
The other measures to tackle the privacy
issue is protection of devices during peer-to-
peer interaction (Da Xu, He & Li, 2014).
The other measure is to hardcode the
devices to protect it from common privacy
and security issues.
 Data mining issue- There are primary
challenges for data mining where data
exploration and information extraction
poses challenges for big data in IoT (Liu et
al., 2015). The challenges associated with
processing of data and data mining are
exhaustive data writes/reads. This provide
challenges due to data exploration,
heterogeneous communication, extraction
processes and integration (Lee & Lee,
2015). The other issue is to extract actual
and knowledgeable data from the large pool
of data. The heterogeneity and size of big
data also poses challenge in data mining.
The large data sets has more complexity and
ambiguity that leads to challenges and
require additional steps for data mining
(Terzi, Terzi & Sagiroglu, 2015). Hence, to
minimize these challenges, parallel
associated rule mining procedures and
parallel k-means algorithm has been
introduced.
 Visualization issue- The visualization is
difficult to achieve for big data, which is
large and enormous. Visualization poses
challenge for big data, to work on diverse
and heterogeneous data. The visualization
design for big data framework is difficult to
achieve (Terzi, Terzi & Sagiroglu, 2015).
The response time is also a big data
challenge in IoT. The data are different and
hence visualization of these data is also
different. Thus, this poses a challenge to
visualize data in IoT. The enormous
parallelization of data is difficult in a
scenario where big data is growing rapidly.
The other issues are visual noise,
information loss, large image observation,
changing of image frequently and
requirement of high performance (Pfarr,
Buckel & Winkelmann, 2014). However,
there are several guidelines introduced to
minimize these issues and they are data
awareness, data quality, meaningful results
and interactive visualization tools.
 Integration issue- The issue of integration
of data is an issue that should be looked
upon. These issue poses challenges as data
is of different types (structured, semi-
structured and unstructured) and generated
from different sources (Liu et al., 2015).
These shows that integration of data is
difficult and complex. The challenges
related with data integration are overlapping
of similar data, increased performance,
scalability and access of real-time data is
enabled (Marjani et al., 2017). The other
challenges are adjusting of unstructured and
semi-structured data prior to integration and
analysing. However, these issues can be
addressed through text mining and other
techniques of extracting.
2.4. Big data challenges in Cloud
The challenges of big data in cloud are
given as follows.
 Data quality issue- Big data poses
challenge if data is not accurate and timely
available (Balachandran & Prasad, 2017). If
there is no guarantee of data quality when
implemented through process of information
management.
 Data storage issue- The data are enormous
and growing rapidly with requirement of big
storage to adjust the storage of big data
(Pfarr, Buckel & Winkelmann, 2014). This
poses challenges for cloud computing to
provide big storage capacity in competitive
environment.
 Privacy and security issue- The security
and privacy are the major concern for big
data. The organizations in every sector
require to establish policies for security and
privacy of data (Pfarr, Buckel &
Winkelmann, 2014). However, this poses
challenge as data are not limited to be
suitable for the policies.

5BIG DATA CHALLENGES IN IOT AND CLOUD
 Hacking and several attacks- There is high
risk of attacks and data breaches related
with data in cloud (Litchfield & Althouse
2014). This is because even if only one part
of cloud infrastructure is attacked, all the
clients using the cloud infrastructure will be
affected.
 Delivery of service and billing issue-
There is lack of proper service level
agreement (SLA) given by the cloud
provider for data storage that guarantees
data scalability and availability (El-Seoud et
al., 2017). The budget is also an issue as
cloud provides on-demand service and it is
difficult to assess whether it is justifiable or
not.
 Portability businesses and
interoperability issue- The migration of
services in and out should be smooth and
without lock-in period (El-Seoud et al.,
2017). This poses problem if proper
migration is not done as it can lead to data
loss.
 Data availability and reliability issue- The
proper monitoring of internal and third party
tools, supervise usage, SLAs, robustness,
business dependence and performance.
2.5. IoT technologies and applications
The below IoT technologies helps to solve
the big data challenges in IoT and Cloud.
RFID
The automatic identification and capture of
data is allowed through radio frequency
identification by using a reader, waves and a tag.
There are three tags namely passive, active and
semi-passive which provides identification for data
in the system (Balachandran & Prasad, 2017). The
applications of RFID are IT asset tracking, race
timing, e-passport and transportation payments,
logistics and supply chain, materials management,
library and laundry management.
WSN
This technology consists of sensor-equipped
devices that are spatially distributed to monitor and
manage physical or environmental situations. The
technology cooperate and coordinate with RFID
technology to track all the movement and status of
entities like temperature, location and others
(Litchfield & Althouse, 2014). It provides low cost
and low power for device usage. The applications
of WSN are in military, health, environmental,
home, commercial and industrial monitoring.
Middleware
The communication and I/O is performed
easily by software developers through middleware
which is a software layer. This software layer is
interposed between the software applications. It has
features of hiding different technologies details that
is fundamentally free IoT developers from using
software services (Litchfield & Althouse, 2014).
The example of middleware is Global Sensor
Networks (GSN) that is an open source.
Cloud computing
The cloud computing technology provides
accessibility for on-demand access of configurable
resources to a shared pool which can be conducted
as SaaS or IaaS. This technology provides solution
in back-end to manage large data streams and data
processing for infinite number of humans and IoT
devices in real-time scenario (El-Seoud et al.,
2017). The applications of cloud computing are in
social networking, big data analytics, chatbots,
education, healthcare and banking industry.
Internet of Things software
The development of countless user-specific
and industry-oriented application of IoT are
facilitated by IoT. The devices integrated with IoT
required to ensure proper action taken on received
data in a timely manner (Litchfield & Althouse,
2014). The applications of this technology are in
airline, pharmaceuticals, manufacturing, media and
entertainment, insurance and business services.
The applications of IoT are in indistries such
as logistics and transportation where the status of
goods that are transported are monitored
(Balachandran & Prasad, 2017). This monitoring of
goods are done continuously to take appropriate
actions in case of emergency. The other application
is in healthcare where simultaneous monitoring and
reporting, tracking of data and medical assistance in
remote areas (Balachandran & Prasad, 2017). The
other applications are in retail, smart home, supply
chain and connected car and wearable devices.

6BIG DATA CHALLENGES IN IOT AND CLOUD
3. Issues and Solutions
The issues identified in this research paper are
given as follows. The big data poses various
challenges in Internet of Things and Cloud that are
described in this research paper. The major
challenge is related to security of data. The issues
are related to privacy, security, data mining,
visualization, integration, data quality, data storage,
hacking and several attacks (Assunção et al., 2015).
The other issues are service delivery and billing,
portability businesses and interoperability, data
availability and reliability. The big data challenges
processing and platforms are also described in the
research paper. These issues should be addressed
with proper methods to minimize the challenges in
Internet of Things and Cloud.
The paper will now focus on big data challenges
in Internet of Things and Cloud along with Internet
of Things technologies that can solve these
challenges. The past and present methodologies to
solve the problems of this research paper. They are
discussed as follows.
 Hadoop- The big data is collected and
managed by Hadoop, which is managed by
the Apache Software Foundation. It is an
open source platform to manage big data.
Hadoop provides data processing in
parallelized form to allow quick
computations and to hide latency (Bagheri
& Shaltooki, 2015). The two main elements
for Hadoop are Map Reduce engine and
Hadoop Distributed File System (HDFS).
The enormous data is stored in Hadoop
Distributed File System. This data is
reproduced to the client application at a very
high bandwidth. On the other hand,
MapReduce is a framework that is used for
data processing in a more distributed way
via several machines. Hadoop handles 3Vs
related to big data, variety, volume and
velocity (Jaseena & David, 2014). Hadoop
deals with big data variety by providing
storage of every type of data whether
structured or unstructured. Hadoop handles
volume of data by scaling out to provide
more storage. Hadoop manages velocity of
big data by loading raw data in the
Information System and after that it can be
viewed as per need (Yin & Kaynak, 2015).
The flexibility of the system helps to
smoothly load data without any congestion
and changing data can also be
accommodated with easy integration.
 Map Reduce- Map Reduce is defined as a
paradigm for broad programming. The
original actions were parallel execution,
load balancing, data manipulation and fault
tolerance (Chen & Zhang, 2014). The Map
Reduce is so called because it has two
abilities from that of existing functional
computer aspects and languages: Map and
then reduce. The Map Reduce is a
framework that helps to collect data and
gather them from records with a common
key and then together joins them. This
shows that for each different key that is
produced acquire formation of single group
(Wu et al., 2014). The Map Reduce is
considered as a technology however, it is
only an algorithm that provides how the data
will fit into the system. The Map Reduce
being just an algorithm has limited scope to
manage big data however, it can be utilized
for best to manage data if combined with
other technologies. The Map Reduce is used
in two phases to solve big data challenges
and they are Map phase and Reduce phase
(Reyes-Ortiz, Oneto & Anguita, 2015). The
map phase helps to do functional operation
on datasets to emit the key which is mapped
and generate pairs for the output of this
phase. The reduce phase helps to collect
data from nodes and combines them in such
a way that it comes out as expected output.
 Hive- Hive can be determined as an
essential part of Hadoop system that can be
viewed as a principle aspect for the data
warehouse in the system (Wang et al.,
2016). The tools that are already deployed
for data warehouse in the system are not
able to adjust in the events where
accessibility of data is everywhere and
sometimes operated privately (Babiceanu &
Seker, 2016).). Map Reduce provides

7BIG DATA CHALLENGES IN IOT AND CLOUD
tracking of reusable code characteristics that
is difficult from business prospective
(Zheng et al., 2015). On the other hand,
Hive is a technology that cannot treat with
transaction and applications for real-time
analysis due to some complicated technique.
Hive is used in conjunction with Hadoop
and Map Reduce where several challenges
of big data is overcome by this technology
(Bagheri & Shaltooki, 2015). It has three
main function and they are running of data
queries, summarizing of data and big data
analytics.
 Mahout- This technology is built on an
Apache library which is an open-source and
able to scale up and down as per the
requirement. This technology also helps to
manage huge volume of data (Suthaharan,
2014). There are different segments that rely
upon three particular machine learning
aspects on which Mahout currently operates.
These machine learning aspects are
collaborative, clustering and classification.
It is a library of various scalable algorithms
for machine-learning that is implemented on
top of Hadoop (Wang, 2016). Mahout is a
recently introduced technology that helps to
provide better integration for big data
challenges. The machine learning
technology helps to analyse data that are
huge and repetitive, flowing in Internet of
Things (Allen, 2017). This technology has
already adopted by some sectors such as
social media who are enjoying the leverage
of this technology. Machine learning
technology helps to analyse huge data and
separates the unmatched data which is not as
per the norms of data.
 GFS- GFS is developed by Google Inc. and
it is defined as distributed file system. The
enhancement of GFS is for Google’s central
data storage and requirements for usage that
can enable huge quantities of data which
requires recalling (Kim, Trimi & Chung,
2014). GFS technology is used for various
purposes and they are scalability,
performance, availability and reliability
(Suthaharan, 2014). These purposes of the
distributed file system is manipulated
through technological environment and
application workloads.
Thus, above are the methods and these can be
used for to solve big data challenges in Internet of
Things.
The methodologies are compared in terms of
aspects that are described below.
 Hadoop is efficient, complex to use, can be
extended to other applications as well. It is
time saving and cost saving with flexibility
to use (Zheng et al., 2015). However, issues
are complexity and availability.
 Map Reduce is quite efficient, simple to use
that can be extended to other applications
also. It is time saving and costly with
flexibility to use. However, issues are not
suitable for small data and speed is slow.
 Hive is efficient, quite simple to use that can
be extended to other application also. It is
time saving and cot saving with flexibility in
its structure (Cevher, Becker & Schmidt,
2014). However, it cannot deal with real-
time analysis.
 Mahout is efficient, complex to use that ca
be extended to other applications as well. It
is also time saving and cost saving with
flexibility to use. However, it cannot be
applied on theoretical problems.
 GFS is efficient, simple to use that can be
extended to applications. It is time saving
and cost saving with flexibility. However,
issues are cannot write randomly and not
suitable for small sized data.
Therefore, from above points it can be shown
that Hadoop along with Map Reduce, Hive and
Mahout is the best methodology for big data
challenges in Internet of Things and Cloud.
4. Future Research
The big data, Internet of Things and Cloud
are a big boon for every industry, if used conjointly.
The future of these three technologies are great and
it can have major impact on every sector. The big
data is increasingly adopted by the organizations
and its adoption has generated the need to adopt
cloud computing and Internet of Things (Kim,
Trimi & Chung, 2014). The Internet of Things, big

8BIG DATA CHALLENGES IN IOT AND CLOUD
data and cloud will transform businesses where they
will be able to extract valuable data from pool of
data that can maximise their business (Kim, Trimi
& Chung, 2014). The Healthcare industry and
banking industry are the other two major sectors
which will be getting benefit from big data, Internet
of Things and cloud computing. Healthcare industry
will use these technologies to improve patients care
and make them safer and secure in the healthcare
environment (Wang et al., 2016). The banking
sector will also be able to improve their
performance by making a safer environment for
customers so that they feel secure. The payment and
transaction will become safer helping bank
employees and customers to transact or pay
securely without any fear of losing sensitive data
due to any attack such as hacking or data breach
(Skarmeta, Cano & Iera, 2015). The big data,
Internet of Things and cloud will not only affect the
industries but it will also help people to use for their
individual purposes to do basic functions and
operations (Durgude & Yalij, 2015). The accurate
real-time analysis which I the most important stage
in the current scenario for every industry and an
individual will also be achieved due to these three
technologies use.
5. Advantages and disadvantages
The advantages and disadvantages of big
data, Internet and Things and Cloud computing,
with methodologies used, are given as follows.
Advantages
 Cost reduction- The cost will be drastically
reduced with the use of these three
technologies and this will enhance more
frequent adoption and deployment of these
technologies in various sectors (Bossé &
Solaiman, 2016).
 Availability- The big data, Internet of
Things and cloud are readily available to be
used for services in any industry whether
on-demand or on premise (Xu et al., 2015).
The technologies are timey update
automatically which also reduce manual
work for every industry.
 Scalability and elasticity- The big data is
scalable and has property of elasticity which
expands out and expands in to adjust big
data in the storage system to be available for
every industry when required.
 Increased storage capacity- The storage can
be increased as per the requirement of data
which helps to store enormous data that are
generated on a daily basis (Skarmeta, Cano
& Iera, 2015). These technologies has
capacity to increase their storage as
maximum as it can.
 Fraud detection- These technologies when
used conjointly, detect frauds easily and
eliminate risks by taking proper steps and
also helps to pre-plan for these risks in case
of disasters.
 On-demand service- the service can be used
on-demand which helps to minimize the
concern of having physical data storage
systems and provide more benefits than
traditional data storage systems (Zheng et
al., 2015).
Disadvantages
 Not compatible sometimes- The
technologies are sometimes not compatible
to benefit the organizations (Aly, Elmogy &
Barakat, 2015). Sometimes poses challenges
to manage these technologies together.
 Possible failure- These technologies can
sometimes fail if not properly managed and
maintained (Bossé & Solaiman, 2016). An
expert management and maintenance is
required where there will be no chance of
failure.
 Privacy and security- The privacy and
security are the major concerns that are
important part of these technologies (Bossé
& Solaiman, 2016). The privacy and
security should be taken into account as
these are the major aspect which can lead to
failure of businesses in any industry.
6. Conclusion
Therefore, above discussions it can be
concluded that, there are big data challenges in
Internet of Things and Cloud. The big data
challenges are given in this research paper to
analyse its impact in Internet of Things. There are
various challenges described in this research paper
that are privacy and security issues, data mining

9BIG DATA CHALLENGES IN IOT AND CLOUD
issues and data integration issues and data storage
issues. This paper discusses various technologies
associated with the big data, cloud and Internet of
Things that are important for the challenges to
overcome. There are various applications of these
technologies when conjoint together. The
applications are discussed in this report. The report
shows that there are challenges which needs to be
looked upon with proper emphasis on each
technology and tools. The methodologies to solve
these challenges has been discussed in this report to
propose a solution for the big data challenges. The
big data challenges in Internet of Things and Cloud
shows that the challenges related to Internet of
Things and Cloud that are essential to be taken into
account. The possible strategies in the research
paper shows that there are various technologies that
have been previously deployed and are on a verge
of deploying technologies. The methodologies are
helpful to support big data challenges in Internet of
Things. The best methodologies are combination of
Hadoop along with Map Reduce, Hive and Mahout
is necessary to minimize big data challenges in
Internet of Things and Cloud. This combination is
the possible and best solution for the present and
future scenario with adding up of more advanced
and new technologies. These methodologies has
several advantages which are beneficial for big data
challenges in Internet of Things and Cloud. These
methodologies provide various benefits such as
reduction in cost for future opportunities, scalability
and flexibility of methodology to improve big data
challenges in Internet of Things and Cloud.
However, there are advantages and disadvantages
of these three technologies that is essential to be
taken into account before and after adopting these
technologies. The several drawback of big data
challenges in Internet of Things are compatibility
issues and power failure in events of disruption.
Hence, it can be concluded that the challenges can
be minimized with proper use of these
methodologies. The future research shows that if
these technologies maintained properly then it can
have major boost in performance of every industry
using combination of big data, Internet of Things
and Cloud. Therefore, big data challenges in
Internet of Things and Cloud are the factors that
needs central attention and valuable strategies to
manage and maintain the technologies.

10BIG DATA CHALLENGES IN IOT AND CLOUD
References
1. Ahmed, E., Yaqoob, I., Hashem, I. A. T.,
Khan, I., Ahmed, A. I. A., Imran, M., &
Vasilakos, A. V. (2017). The role of big
data analytics in Internet of
Things. Computer Networks, 129, 459-471.
2. Allen, T. (2017). SAPVoice: How To Solve
IoT's Big Data Challenge With Machine
Learning. Forbes. Retrieved 13 April 2018,
from
https://www.forbes.com/sites/sap/2017/02/0
2/how-to-solve-iots-big-data-challenge-
with-machine-learning/#7f1d64c723d5
3. Aly, H., Elmogy, M., & Barakat, S. (2015).
Big Data on Internet of Things:
Applications, Architecture, Technologies,
Techniques, and Future Directions. Int. J.
Comput. Sci. Eng, 4, 300-313.
4. Aly, H., Elmogy, M., & Barakat, S. (2015).
Big Data on Internet of Things:
Applications, Architecture, Technologies,
Techniques, and Future Directions. Int. J.
Comput. Sci. Eng, 4, 300-313.
5. Assunção, M. D., Calheiros, R. N., Bianchi,
S., Netto, M. A., & Buyya, R. (2015). Big
Data computing and clouds: Trends and
future directions. Journal of Parallel and
Distributed Computing, 79, 3-15.
6. Babiceanu, R. F., & Seker, R. (2016). Big
Data and virtualization for manufacturing
cyber-physical systems: A survey of the
current status and future outlook. Computers
in Industry, 81, 128-137.
7. Bagheri, H., & Shaltooki, A. A. (2015). Big
Data: challenges, opportunities and Cloud
based solutions. International Journal of
Electrical and Computer Engineering, 5(2),
340.
8. Bagheri, H., & Shaltooki, A. A. (2015). Big
Data: challenges, opportunities and Cloud
based solutions. International Journal of
Electrical and Computer Engineering, 5(2),
340.
9. Balachandran, B. M., & Prasad, S. (2017).
Challenges and Benefits of Deploying Big
Data Analytics in the Cloud for Business
Intelligence. Procedia Computer
Science, 112, 1112-1122.
10. Bossé, É. & Solaiman, B.
(2016). Information fusion and analytics for
big data and IoT. Artech House.
11. Cevher, V., Becker, S., & Schmidt, M.
(2014). Convex optimization for big data:
Scalable, randomized, and parallel
algorithms for big data analytics. IEEE
Signal Processing Magazine, 31(5), 32-43.
12. Chen, C. P., & Zhang, C. Y. (2014). Data-
intensive applications, challenges,
techniques and technologies: A survey on
Big Data. Information Sciences, 275, 314-
347.
13. Da Xu, L., He, W., & Li, S. (2014). Internet
of things in industries: A survey. IEEE
Transactions on industrial
informatics, 10(4), 2233-2243.
14. Da Xu, L., He, W., & Li, S. (2014). Internet
of things in industries: A survey. IEEE
Transactions on industrial
informatics, 10(4), 2233-2243.
15. Díaz, M., Martín, C., & Rubio, B. (2016).
State-of-the-art, challenges, and open issues
in the integration of Internet of things and
cloud computing. Journal of Network and
Computer Applications, 67, 99-117.
16. Durgude, D. M., & Yalij, N. S. (2015). Big
Data Analysis: Challenges and
Solutions. International Journal of Scientific
Research and Management, 3(2).
17. El-Seoud, S. A., El-Sofany, H. F.,
Abdelfattah, M. A. F., & Mohamed, R.
(2017). Big Data and Cloud Computing:
Trends and Challenges. International
Journal of Interactive Mobile Technologies
(iJIM), 11(2), 34-52.
18. Fazio, M., Celesti, A., Puliafito, A., &
Villari, M. (2015). Big data storage in the
cloud for smart environment
monitoring. Procedia Computer Science, 52,
500-506.
19. Hashem, I. A. T., Yaqoob, I., Anuar, N. B.,
Mokhtar, S., Gani, A., & Khan, S. U.
(2015). The rise of “big data” on cloud
computing: Review and open research
issues. Information Systems, 47, 98-115.

11BIG DATA CHALLENGES IN IOT AND CLOUD
20. Hassanalieragh, M., Page, A., Soyata, T.,
Sharma, G., Aktas, M., Mateos, G., ... &
Andreescu, S. (2015, June). Health
monitoring and management using Internet-
of-Things (IoT) sensing with cloud-based
processing: Opportunities and challenges.
In Services Computing (SCC), 2015 IEEE
International Conference on (pp. 285-292).
IEEE.
21. Jaseena, K. U., & David, J. M. (2014).
Issues, challenges, and solutions: Big data
mining. Computer Science & Information
Technology (CS & IT), 131-140.
22. Kim, G. H., Trimi, S., & Chung, J. H.
(2014). Big-data applications in the
government sector. Communications of the
ACM, 57(3), 78-85.
23. Lee, I., & Lee, K. (2015). The Internet of
Things (IoT): Applications, investments,
and challenges for enterprises. Business
Horizons, 58(4), 431-440.
24. Litchfield, A., & Althouse, J. (2014). A
systematic review of cloud computing, big
data and databases on the cloud.
25. Liu, C., Yang, C., Zhang, X., & Chen, J.
(2015). External integrity verification for
outsourced big data in cloud and IoT: A big
picture. Future Generation Computer
Systems, 49, 58-67.
26. Liu, C., Yang, C., Zhang, X., & Chen, J.
(2015). External integrity verification for
outsourced big data in cloud and IoT: A big
picture. Future Generation Computer
Systems, 49, 58-67.
27. Marjani, M., Nasaruddin, F., Gani, A.,
Karim, A., Hashem, I. A. T., Siddiqa, A., &
Yaqoob, I. (2017). Big IoT data analytics:
Architecture, opportunities, and open
research challenges. IEEE Access, 5, 5247-
5261.
28. Marjani, M., Nasaruddin, F., Gani, A.,
Karim, A., Hashem, I. A. T., Siddiqa, A., &
Yaqoob, I. (2017). Big IoT data analytics:
Architecture, opportunities, and open
research challenges. IEEE Access, 5, 5247-
5261.
29. Perera, C., Ranjan, R., Wang, L., Khan, S.
U., & Zomaya, A. Y. (2015). Big data
privacy in the internet of things era. IT
Professional, 17(3), 32-39.
30. Pfarr, F., Buckel, T., & Winkelmann, A.
(2014, January). Cloud Computing Data
Protection--A Literature Review and
Analysis. In System Sciences (HICSS), 2014
47th Hawaii International Conference
on (pp. 5018-5027). IEEE.
31. Reyes-Ortiz, J. L., Oneto, L., & Anguita, D.
(2015). Big data analytics in the cloud:
Spark on hadoop vs mpi/openmp on
beowulf. Procedia Computer Science, 53,
121-130.
32. Sajid, A., Abbas, H., & Saleem, K. (2016).
Cloud-assisted iot-based scada systems
security: A review of the state of the art and
future challenges. IEEE Access, 4, 1375-
1384.
33. Skarmeta, A. F., Cano, M. V. M., & Iera, A.
(2015). Guest Editorial: Smart Things, Big
Data Technology and Ubiquitous
Computing solutions for the future Internet
of Things. JoWUA, 6(1), 1-3.
34. Suciu, G., Suciu, V., Martian, A.,
Craciunescu, R., Vulpe, A., Marcu, I., ... &
Fratu, O. (2015). Big data, internet of things
and cloud convergence–an architecture for
secure e-health applications. Journal of
medical systems, 39(11), 141.
35. Sun, Y., Song, H., Jara, A. J., & Bie, R.
(2016). Internet of things and big data
analytics for smart and connected
communities. IEEE Access, 4, 766-773.
36. Suthaharan, S. (2014). Big data
classification: Problems and challenges in
network intrusion prediction with machine
learning. ACM SIGMETRICS Performance
Evaluation Review, 41(4), 70-73.
37. Terzi, D. S., Terzi, R., & Sagiroglu, S.
(2015, December). A survey on security and
privacy issues in big data. In Internet
Technology and Secured Transactions
(ICITST), 2015 10th International
Conference for (pp. 202-207). IEEE.
38. Terzi, D. S., Terzi, R., & Sagiroglu, S.
(2015, December). A survey on security and
privacy issues in big data. In Internet
Technology and Secured Transactions

12BIG DATA CHALLENGES IN IOT AND CLOUD
(ICITST), 2015 10th International
Conference for (pp. 202-207). IEEE.
39. Wang, H., Xu, Z., Fujita, H., & Liu, S.
(2016). Towards felicitous decision making:
An overview on challenges and trends of
Big Data. Information Sciences, 367, 747-
765.
40. Wang, L. (2016). Machine learning in big
data. International Journal of Advances in
Applied Sciences, 4(4), 117-123.
41. Wang, L., & Ranjan, R. (2015). Processing
distributed internet of things data in
clouds. IEEE Cloud Computing, 2(1), 76-80.
42. Wang, Y., Wei, J., Srivatsa, M., Duan, Y.,
& Du, W. (2016). IntegrityMR: Exploring
Result Integrity Assurance Solutions for Big
Data Computing Applications. International
Journal of Networked and Distributed
Computing, 4(2), 116-126.
43. Wu, X., Zhu, X., Wu, G. Q., & Ding, W.
(2014). Data mining with big data. IEEE
transactions on knowledge and data
engineering, 26(1), 97-107.
44. Wu, X., Zhu, X., Wu, G. Q., & Ding, W.
(2014). Data mining with big data. IEEE
transactions on knowledge and data
engineering, 26(1), 97-107.
45. Xu, X., Sheng, Q. Z., Zhang, L. J., Fan, Y.,
& Dustdar, S. (2015). From big data to big
service. Computer, 48(7), 80-83.
46. Yi, S., Li, C., & Li, Q. (2015, June). A
survey of fog computing: concepts,
applications and issues. In Proceedings of
the 2015 Workshop on Mobile Big Data (pp.
37-42). ACM.
47. Yin, S., & Kaynak, O. (2015). Big data for
modern industry: challenges and trends
[point of view]. Proceedings of the
IEEE, 103(2), 143-146.
48. Zheng, Z., Wang, P., Liu, J., & Sun, S.
(2015). Real-time big data processing
framework: challenges and
solutions. Applied Mathematics &
Information Sciences, 9(6), 3169.