Big Data Challenges in the IoT and Clouds

Big data Challenges in the IoT and Clouds
[Name of the Author]

ABSTRACT – The report mainly discusses about the various challenges that is faced by big data challenges in the cloud and
IoT. IoT or internet of things can be stated as the system, which consists of interrelated computing devices, mechanical and
digital machines, objects and many more along with a unique identifier. The IoT devices have the capability of transferring data
over the network without any type of human-to-human or human-to-computer interactions. This report also defines the
problems that might arise while handling big data in the cloud and the IoT. The report provides a brief discussion about the
various problems faced by big data in the IoT and the clouds. The report also consists of the literature review about the various
past and present work with this technology.
1. INTRODUCTION (15 MARKS)
The report discusses about the various problems that
might arise while using the big data in the cloud or the IoT.
IoT has evolved due to the convergence of the various
wireless technologies, micro electromechanical systems,
micro services and the internet. This convergence has helped
a lot in tearing down of the silo walls that exists between the
operational technology and the information technology [1].
This has initially allowed the analysis of the unstructured data
generated by the machine for the insights to drive the
improvement process. IoT devices are generally associated
with providing of large amount of data and if the data
produced, is analyzed in a proper way then those data can be
extremely valuable. The various challenges faced by the big
data in the IoT and the clouds have been described in the
following section of the report.
IoT or internet of things has many benefits but still there
exists some challenges regarding the big data. Big data and
IoT can be considered as the outputs of the demands of
business, which is responsible for driving the changes in an
application. Firstly the thing which comes in mind is that
when big data and IoT are used together the volume of data
increases which gives rise to the challenges regarding the
storage of the data. In addition, this can be tackled by making
use of additional data centers, which will handle the extra
load. So taking into consideration the enormous Impact of IoT
on the data storages the total system can be moved into the
usage of platform-as-a-service, which is a cloud based
solution. Along with this, there exists security challenges like
breach of data, loss of data many more. As Iot is an emerging
technology so it can be stated that it is still a new form of
technology for the security professionals and they lacks the
experience for handling of the security threats. Because of this
reason the risk increases and along with this the risk is
regarding not only the data but also it can also damage the
devices, which are connected to the network. There is a need
of making crucial changes in the security landscape of the IoT
technology.
2. MAJOR CHALLENGES OF BIG DATA IN THE IOT AND
CLOUDS
There exists various types of challenges, which are faced
by the big data in the IoT and clouds. Some of the major
challenges are listed below:
2.1 Data challenges
2.1.1 Massive amount of data
The IoT system consists of several devices and
they are associated with providing of huge
amount of information. The huge amounts of data
produced by the IoT devices acts as one of the
major reason behind the problem of transmission,
storage and processing of the data. The volume of
data is increasing day by day as many objects
present in today’s world is tracked and recorded.
It has been seen that the large data or the big data
is being stored but the processing of such huge
datasets is very much difficult [26]. This is
ultimately resulting in the decreased amount of
processed data, which is responsible for the
forming of blind zones.
2.1.2 Various Forms of Data Collected
Data is collected from various sensors, this
sometimes combines with the other unstructured
data and for this reason, and there exists a strong
relationship between the sensors and the other
unstructured data. The big data is responsible for
storing of various forms of data collected, which
might include the structured data, Semi-structured
data and the unstructured data. Only 20% of the
data is processed amongst the total amount of
data collected [27]. The rest 80% of the data
cannot be analyzed and processed by making use
of the traditional methods. This ultimately helps
in concluding to the fact that most of the data
collected cannot be used for the purpose of
making decisions.
2.1.3 Data Transmission Speed
The speed at which the data gets transmitted in
the internet can also be termed as velocity. The
exploring and acquisition of insights about the
data is only possible if analysis of the data is done
at high speed and on a real-time basis [28]. The
software’s which are existing in today’s world is
capable of generating data streams at very high
speed which initially results in the formation of
various types of difficulties during analysis on a
real-time basis.
2.1.4 Time Series for Data Analysis
The sensors are associated with capturing
events in a specific point of time but it has been
seen that this capturing is sometimes useless. But
in case of serious situations it is necessary to
record and address. This analysis is very much
difficult for most of the companies.

2.1.5 Security and Privacy
Transferring of the data is done by making use
of the wireless medium in the IoT. Therefore, it is
very much essential to ensure the security and the
privacy of the information. There could exists a
number of problems, which might include the
physical attack or the wireless information attack
[29]. This can have a very devastating effect over
the security and the authenticity of the
information, which is being transmitted. It is
possible for the attackers to attack the IoT devices
physical and steal the information while the data
is being transmitted. Along with this, there exists
various IoT devices, which do not accept the
security packages, and this is the main reason
behind low self-defense.
2.2 Process Challenges
2.2.1 Selective Data Acquisition
The data are generally acquired by making use
of the sensors but not all the data that is collected
is important. Due to this reason, there is a need of
filtering and compressing this data. These filters
are associated with determining the data that are
to be accepted and discarded [30]. The task of
designing a smart filter is very much complicated
and challenging.
2.2.2 Data Extraction
The data that is collected is of different forms.
Therefore the utilization of this data can be done
in an effective manner if they are converted into a
single structured format [31]. So there exists the
need of designing a new process for the
extraction of the required information and this
designing of the new processes is very much
challenging.
2.2.3 Data Heterogeneity
Data is generally collected from different types
of sources so they are heterogeneous in nature.
Therefore it can be stated that processing of the
data is not a straight forward process. This
happens due to the reason that finding,
identifying and understanding the information is
very difficult for the type of data which cannot be
integrated seamlessly [32]. It becomes very much
difficult to analyse the heterogeneous data
because they are having different structures and
different schematics as well. The integration of
the heterogeneous data in order to process then on
a real time basis acts as a major challenge.
2.3 Analytic challenges
2.3.1 Analytics Challenges over Unstructured Data
The analysis of the unstructured data is very
difficult. Transformation of the large volumes of
data into meaningful abstracts, which is
associated with supporting cue-based decision-
making, is very much challenging. Another major
challenge faced by video analytics includes the
enormity of the video data [33]. The major
challenge faced by social analytics is its data-
centric nature and it’s indisciplinary research
2.3.2 Visualization Challenges
The challenges of visualization includes its
applicability for the large volumes of data, along
with this the possibility of visualization of the
data that is being presented in different types of
data format, speed and the effectiveness of the
data presentation.
2.4 Schematic data challenges
2.4.1 Data Semantics Challenges
Data schematics acts as major element of the
data analysis process and it is very much
challenging to deal with the different structure of
the data and the different types of information
and analysis of the data as the structure of the
information is very much complex [34]. The
integration of various heterogeneous collections
of data has become a colossal issue. This has
mainly happened because data sources are very
sparse and incomplete. For this reason, it is an
onerous task to find a logical connection between
the data.
2.4.2 Data Scalability Challenges
The data engineers are associated with facing
challenge while creating a domain knowledge
models and schematic annotation frameworks
which can be used so as to describe the huge
number of devices in the IoT. It is essential that
the domain knowledge is associated with the
sematic description of the data and this happens
due to the fact that IoT devices are capable of
referring with the separate phenomenon.one of
the major challenge is regarding the description
of the granularity which means that the terms and
concepts are very specific and the domain
knowledge is extensive [35]. However, the
challenges in handling semantic data are the scale
of data developed by IoT resources, the changing
status of resources and data, and the volatility of
the IoT environment.
2.4.3 Data Fusion Challenges
Data fusion is generally associated with
improving the quality of the data. The schematic
data fusion is very much challenging due to the
fact that data is generally collected from different
sources and along with this different types of
algorithm is used for the purpose of improving
the quality of the service as well as the accuracy
[36].
2.4.4 Data Discovery Challenges
Handling of the data and the storage in an
efficient way is becoming more and more
difficult and along with time, the volume of the
data and the schematic description is increasing
[37]. The major challenge included in this are the