Big data Challenges in IOT and Cloud Assignments PDF

Running head: BIG DATA CHALLENGES IN IOT AND CLOUD
Big Data Challenges in IoT and
Cloud
Executive Summary
The purpose of this report is to focus on IoT
and its applications in various fields. The
application of IoT in Big Data is presented in this
report to evaluate Big Data challenges in IoT and
Cloud. The various technologies IoT has been
discussed in this report. The scope of this report
presents evaluation of Big Data challenges to
propose appropriate solutions and identify different
IoT technologies. The evaluation of project is done
through agile project methodology. The report
results in possible solutions for the challenges of
Big Data through various IoT technologies. The
report recommends possible strategies to be
undertaken while evaluating the challenges. Finally
the report concludes future opportunities of Big
Data in IoT and Cloud with possible measures
undertaken.
Table of Contents
Big Data Challenges in IoT and Cloud....................0
Executive Summary.................................................0
1. Introduction.........................................................0
1.2. Problem Statement........................................2
2. Literature Review............................................2
2.1. Introduction..................................................2
2.2. Integration of big data, IoT and Cloud.........2
2.3. Challenges of big data processing and
platforms for analytics.........................................3
2.3. Big data challenges in IoT............................3
2.4. Big data challenges in Cloud........................4
2.5. IoT technologies and applications................5
3. Issues and Solutions............................................5
4. Future Research...................................................7
5. Advantages and disadvantages............................7
6. Conclusion...........................................................8
References...............................................................9
1. Introduction
The big data, Cloud and Internet of Things are the
emerging trends in this digital world. These
technologies are not only the required technology
but also the necessity in every sector and
organization. These technologies although emerged
individually however, they have been intertwined
(Aly, Elmogy & Barakat, 2015). The increasing
digital transformation has resulted in interdependent
of these three technologies. The intertwining of
these technologies is described as huge demand for
big data has resulted in adoption of both Internet of
Things and Cloud platforms (Sun et al., 2016). The
organization uses big data and are rapidly
increasing. The big data along with Internet of
Things and Cloud has found many applications in
various industries and they are business, healthcare,
banking, financial companies and other companies
(Fazio et al., 2015). The big data is increasingly
day-by-day, it is characterized by three factors, and
they are numerous data, data is complex and data
are rapidly generated, captured and quickly
processed. Cloud Computing is the technology,
which is absorbing the bid data aspects to provide a
better integration for data and information. The
Internet of Things is a valuable technology which
visualizes, uncovers insights from various types of
data such as structured, semi-structured and
unstructured (Marjani et al., 2017). The Big Data
produces huge number of data and they are
complex to identify and evaluate. The challenges
related to Big Data in Internet of Things are making
sense of the complex data, identifying consumed
data and taking actions on the data (Da Xu, He &
Li, 2014). The challenges of big data briefly
describing in Internet of Things are as follows. The
first challenge is representation of data that contains
datasets and have certain levels, semantics,
structure, granularity, industry and accessibility (Yi,
Li & Li, 2015). The representation of data should
be proper, as improper representation of data will
minimize the effective data analysis. Hence,
representation of data is necessary to provide a
well-analyzed data for future operations. The
second challenge is reduction in data redundancy
and compression of data (Aly, Elmogy & Barakat,
2015). Generally, the datasets has high level of
redundancy that needs to be compressed. The data
after reducing redundancy is filtered and
compressed to get the actual data that is relevant to
the organization. The third challenge is

1BIG DATA CHALLENGES IN IOT AND CLOUD
management of data life cycle that is necessary to
ensure quality of data (Hashem et al., 2015). The
aspects related to big data are storage systems,
sensors and computing that poses challenges if not
managed properly. The big data has hidden values
that are dependent on freshness of data. The fourth
challenge is analytical mechanism where big data
processes huge amount of data within a limited
period (Sun et al., 2016). The traditional database
systems lack scalability and flexibility. The big data
poses challenges when intermixed with traditional
database. The fifth challenge is confidentiality of
data, which poses the challenge of maintaining and
handling of data (Hashem et al., 2015). The data are
huge sets and hence service providers are unable to
handle the data sets due to their limited capacity of
managing data. The sixth challenge is energy
management, which shows that big data in Cloud
and Internet of Things absorbs high energy
(Hassanalieragh et al., 2015). This cause effect on
environment and economy perspective. The seventh
challenge is scalability, which shows that big data
must support current and future datasets that may
change.
This shows that big data poses challenges in
Internet of Things and Cloud. The challenges are
several and they need focus and evaluation for
future prospective (Sajid, Abbas & Saleem, 2016).
However, there are several advantages of big data
analytics in Cloud and Internet of Things (Liu et al.,
2015). The advantages of big data in Internet of
Things and Cloud are reduced cost, virtualization
and instant access to infrastructure in cloud.
The big data is influencing organizations
through use of Internet of Things and cloud and
they are generating at high rates (Liu et al., 2015).
The future of big data depends on Internet of
Things and Cloud. This can be shown as follows.
The Internet of Things is the future of digital world
and it is nothing nut networks that are
interconnected to provide network services for data
(Wang & Ranjan, 2015). The cloud is a reliable
technology that helps to provide infrastructure to
the organization for storing data.
The big data and Internet of Things both
provide services as follows. The Internet of Things
provides connection of machines and sensors to be
used and big data to enable the move of network
from virtual world to real world. The smart and
connected devices give perfect explanation of
adopting big data and Internet of Things (Terzi,
Terzi & Sagiroglu, 2015). This explains that
Internet of Things and big data analytics consists of
various Internet of Things data. The big data
analytics has various levels and each level are as
follows. The real time level is used for analysing
huge data through the sensors using Greenplum and
Hana (Sajid, Abbas & Saleem, 2016). The offline
level is used for applications where requirement for
response time is not high. The existing
architecture/tools for this level are Scribe, Kafka,
Timetunnel and Chukwa (Da Xu, He & Li, 2014).
The memory level is used for data where the
volume is smaller than the cluster having maximum
memory. The existing tool for this level is
MongoDB (Terzi, Terzi & Sagiroglu, 2015). The
business intelligence level is used when the
memory level is surpassed by scale using the tool
data analysis plans. The massive level is used when
data totally surpasses the business intelligence
capacity and databases. The existing tool for this
level is MapReduce.
The Internet of Things and Cloud has
various applications in various industries and the
applications are as follows. The first domain is
transportation consisting of smart parking, driving
through 3D assistance (Sajid, Abbas & Saleem,
2016). The second domain is Smart environments
domain consisting of smart water supply, smart
home and offices. The third domain is Healthcare
domain consisting of health tracking and
pharmaceutical products. The fourth domain is food
sustainability and fifth domain is futuristic
applications domain consisting of robot taxi and
city information model (Hashem et al., 2015).
There are challenges related to Internet of Things
for future direction. The challenges are architecture,
environment innovation, technical, hardware,
privacy and security, standard, business,
development and data processing including
heterogeneous data, noisy data and massive-
intensive data. The big data issue in Cloud is based
on infrastructure provided by cloud for big data
storage (Suciu et al., 2015). This issue varies from
one organization to another organization. The large
unstructured data generated are dealt through big

2BIG DATA CHALLENGES IN IOT AND CLOUD
data analytics however; data storage in cloud poses
challenges as cloud provides various infrastructures
(Perera et al., 2015). The one technology, which is
currently being used and widely accepted, is fog-
computing technology. The fog computing provides
several benefits. The benefits are bringing data near
to the user, creation of dense geographical
distribution, true support to Internet of Things (Yi,
Li & Li, 2015). There are various Internet of Things
technologies are discussed in this report. These
technologies are near field communication, radio
frequency identification, and low energy Bluetooth
and wireless, radio protocols, LTE-A and WiFi-
Direct (Perera et al., 2015). These technologies
correlated with big data and cloud on a big platform
to provide large future opportunities. This
correlation also poses challenges in the real world
environment if not managed properly and carefully
(Aly, Elmogy & Barakat, 2015). The big data is
rapidly growing and to accommodate big data in
Internet of Things and Cloud there are various
technologies and techniques that will be discussed
in further sections. The problem discussed in this
report is how big data poses challenges in Internet
of Things and cloud.
The above are the concerns and problems of
the topic discussed and the topic will be elaborated
in further sections through prior researches and
methodology (Aly, Elmogy & Barakat, 2015). The
purpose of this report is to focus on the challenges
of big data in Internet of Things and Cloud to
evaluate various existing and developing
technologies for Internet of Things. The report also
focuses on the different applications of Internet of
Things along with focus on big data and cloud in
real world environment (Marjani et al., 2017). The
report presents prior researches on big data
challenges in Internet of Things and cloud to
evaluate the technologies. The report provides
issues of the challenges and the solution for these
challenges with focus on future research. The report
presents the methodologies through which the
Internet of Things, big data and cloud are evaluated
in this report. The methodologies used in the
literature review section are described and
compared to choose the best-suited methodology
for this report. The report also presents tables and
graphs to illustrate more about this topic and
support the justifications made in this report.
The outline of this report is as follows. The
first section is literature review on past and present
technologies related to big data challenges in
Internet of Things and Cloud. This section briefly
describes the researches done on big data, Internet
of Things and Cloud to identify the Internet of
Things technology. The given section covers the
major part of the report that provides brief details of
the problems and challenges of the topic discussed
in this report. The second section is methodology
section and it contains past and current
methodologies used in the literature review section.
This section also consist comparison of
methodologies from different perspectives such as
simplicity, efficiency, cost and time saving,
connectivity and feasibility. This section then
provides the best methodology or combination of
methodology to suit the purpose of this report based
on several factors. The report then provides issues
along with solutions for the future research
perspective.
1.2. Problem Statement
The challenges of Big Data in IoT and cloud
have posed various difficulties in adoption of Big
Data when used along with IoT. These challenges
are important to solve due to data storage, data
security and integrity of data related with Big Data.
The Big Data poses problems such as huge and
complex data and data are biased. There are several
Internet of Things technologies which help to solve
challenges of Big Data such as Hadoop and Spark.
There are several methods that have been presented
to tackle the challenges of Big Data and the most
important method is use of Hadoop. The Big Data
challenges in IoT and Cloud need to be solved for
future purposes.
2. Literature Review
2.1. Introduction
This section focuses on the challenges of big
data in IoT and Cloud computing through prior
researches done on this topic. Big data is
intertwined with IoT and Cloud to provide a better
platform for managing data (Da Xu, He & Li,

3BIG DATA CHALLENGES IN IOT AND CLOUD
2014). The IoT, Big data and Cloud provide better
integration for every sector however, there are
challenges with these integration. These challenges
are focused in the following sections.
2.2. Integration of big data, IoT and Cloud
The attraction and interest of big data in
every industry is increasing rapidly. The industries
from social networks to multimedia to business
transactions all has adopted or on a verge of
adopting these technology. There are 4 Vs related
big data, volume, variety, velocity and value that
poses challenges for researchers (Ahmed et al.,
2017). The storage of and processing of data is a
challenge for every sector. The cloud computing is
the recent technology that has emerged to solve the
big data problems by providing cost effective data
center to store huge data. However, there are
concerns related to service qualities of cloud such
as privacy of user and data security (Da Xu, He &
Li, 2014). The security concern is the top most
priority regarding the use of cloud. IoT is a
platform for next-generation computing which is
integrated in every sector from an individual’s life
to an organization. This platform works in real-
timed fashion in every sector and has large datasets
that is generated over a particular time ((Ahmed et
al., 2017). The prior researches shows that there are
areas associated with big data security in cloud and
IoT which needs to be looked upon for future
researches (Liu et al., 2015). The areas are
efficiency including communication and storage
and computation time, security and scalability.
2.3. Challenges of big data processing and
platforms for analytics
The various platforms big data platforms
poses challenges in IoT and cloud and they are
discussed as follows.
Challenges of Apache Hadoop: lack of
encryption at network and storage level, unsuitable
for small data, limited flexibility and high input-
output overhead (Ahmed et al., 2017).
Challenges 1010data: data extraction, data
loading and data transformation.
Challenges of Cloudera, a Hadoop based
framework, are that there is no software and
hardware systems of its own and relies on third
parties.
Challenges of SAP-Hana are: all data must
be read that is in a row even though a few columns
of data are required for accessing.
Challenges of Hadoop Autonomy Vertica
Enterprise security (HAVEn): a large database is
generated by an increment in tenants where all
operations related to release processes and lock
holding are decelerated (Marjani et al., 2017).
Challenges of Hortonworks: it cannot
minimize the node-groups in the cluster generated
by the system.
Challenges of pivotal big data suite: it has
several unresolved issues that hinders its adoption
in industries.
Challenges of Infobright: Infobright
optimizer cannot optimally answer all the queries
(Ahmed et al., 2017).
Challenges of MapR: it has larger
complexity as compared to Hadoop.
2.3. Big data challenges in IoT
The big data challenges in IoT are as
follows that arise due to introduction of big data in
IoT. The challenges are as follows that are
discussed here.
 Privacy issue- The issue of privacy arises
when there is any compromise of system for
restoring sensitive data using tools of big
data analytics. There is a problem regarding
data mining where the privacy issue is high
(Marjani et al., 2017). The users find it
difficult to trust on usage of big data in IoT
due lack of proper service level agreement
for sensitive data misuse use such personal
information. The other issues is security
risk, related with IoT data, in heterogeneity
of using devices and data generated such as
data types, communication protocols and
raw devices (Ahmed et al., 2017). The other
challenges emerges due to generated data
through IoT are as follows. The challenges
are timely update of systems, managing
identification of traffic patterns from
suspicious and legitimate ones,
interoperability and protocol convergence
(Pfarr, Buckel & Winkelmann, 2014).