Big Data Challenges in the IoT and Clouds

This report discusses the challenges faced by big data in the IoT and Clouds. It covers issues in data management, storage, processing, and security. The report also provides solutions to overcome these challenges.

Big data Challenges in the IoT and Clouds
[Name of the Author]

ABSTRACT – The entire report is having the aim of discussing the different kind of problems, which are encountered by the
big data which are handled by the Clouds and the IoT or the Internet of Things. IoT or internet of things is one of the emerging
technologies which is seen to consisting of of interrelated computing devices along with mechanical as well as digital
machines, objects and many more and all of this are consisting of an unique identifier. In addition to all this the IoT devices are
also seen ti be having different kind of capabilities related to the data transferring over the network without involving any kind
of human-to-human or human-to-computer interactions. Cloud computing generally refers to the on-demand availability related
to the different computer system resources mainly storage and the computing power without the involvement of any kind of
human user involvement. This term is generally used of the purpose of defining the data centers, which are available to
numerous users over the internet. This report is aimed at defining the problems, which are encountered by the Internet of Things
and the Cloud computing while handling the big data. The report is also associated with discussing about the different present
and past work conducted regarding the big data challenges as well along with providing an appropriate methodology for
overcoming the challenges that are faced.
1. INTRODUCTION (15 MARKS)
IoT or the Internet of things is the technology which ahs
been seen to have evolved because of the convergence that
took place between the different kind of the wireless
technologies, micro electromechanical systems, micro
services and the internet. The convergence has been
associated with tearing up of the barriers, which were seen to
be existing between the operational technology and the
information technology [1]. This in turn has been responsible
for allowing the analysis of the data which are unstructured
and are generated by the machine for the insights to drive the
improvement process. Digital transformation has been
associated with making the technologies like the big data
cloud and the IoT to become the top most agenda which
almost each and every business is having. It can be stated that
the each of these technologies are having their own
necessities. Despite of the fact that the Big data, IOT and the
cloud storages have evolved in an independent way still they
are associated with showing certain persistent coverage’s
which has been associated with helping in the creation of
more capabilities related to this techniques which are the best.
The aspects of the big data, IoT and Clouds are seen to be
trending throughout the entire globe and along with the IT
industry is seen to be in a time of transition with the rapid
growth of the data at an unconstrained and in an exponential
rate. The organization of todays world are seen to be more
data centric [2]. The main aim of this paper is discuss the
major challenges which are faced by the IoT and the clouds
while handling the big data.
2. LITERATURE REVIEW : MAJOR CHALLENGES OF BIG
DATA IN THE IOT AND CLOUDS
Despite of the convergence of the Big data with the cloud
and the IoT so as to act as a solution of the next generation it
is attributed towards the various kind of technical challenges
which are seen to be acting as disadvantageous for the users.
Multiple issues are seen to be existing in the process of big
data analytics by making use of the clouds and the IoT which
if ignored would be associated with causing a significant harm
to the entire business [3]. The increased overflow of the big
data in the era of big data has been associated with brining of
enormous amount of challenges. Resolving of the big data
problems is possible by depending upon the modern means
and cloud computing technology in the figure provided below
some of the difficulties which are faced by the cloud based big
data analytics have been provided.
2.1. Issues in the BIG data and the IoT:
Big data along with providing of opportunities is also
associated with bringing a lot of challenges for the users, this
in turn is associated with implication of the fact that there is
an essential need of overcoming the challenges [4]. In order to
gain valuable insights from the big data, one of the major
issue, which is seen to be associated with the dig data, is the
heterogeneity. The production of the different type of data
from the multiple sources is associated with making this
become more complicated. The reason behind this is the
production of data in raw form as well as in structure, semi-
structures and unstructured formats. Besides this some level of
heterogeneity is also seen to be existing in the in type,
structure, semantics, organization, granularity, and
accessibility of the datasets, which initially leads to variety
and heterogeneity.
Another major issue is related to the storage, the rapid
growth of the data has been associated with restricting the
existing storages capacity [5]. The traditional storage systems
generally relied upon the RDBMS but still each of the storage
systems are seen to having its own limitations in terms of the

data storage and management acting as an disadvantageous
aspect.
Additionally the big data is also associated with leading to
the issues of data transportation. This particular challenge is
associated with estimating the fact that the time taken for the
purpose of transmitting the data from starting to end point is
less than the time that is taken for processing the data, the disk
technology is currently restricted to 4 terabytes per disk. This
in turn is associated with implication of the fact that around
1Exabyte would be associated with needing around 25000
discs [6]. Besides this, an Exabyte of the data can also be
processes in a single computer system. But for doing this it
would no longer be capable of attaching the number of disks
directly that are required.
Followed by the above issues is the issue related to data
management. Big data along with the big data management is
seen to be working in an simultaneous way so as to achive
success. Big data is generally associated with dealing with the
data of high volume; high velocity, which are coming from
different, sources and is multi-structured in nature. The
discipline is mainly associated with covering the different
kind of data fields, which includes the data warehousing
integration of data, management of content, processing of
events and administration of database [7]. The aspect of
management of the big data is one of the challenging factors.
Some of the other significant challenges are associated with
including the access, metadata, utilization, updating,
governance, and reference.
Followed by all this one of the major issue that is faced is
the issue related to data processing and data security. There
exists numerous new analytic algorithms along with extensive
parallel processing which are necessary for having an
effecting processing of the Exabyte’s of the data. This in turn
is associated with making sure of the timely and actionable
information. Whenever a query related to big data is
processes, the most significant demand is the speed [8].
However, this might be associated with costing a huge amount
of time and the reason behind this is that it is not possible to
traverse the entire database in a short time period. For this
type of situations, the index would be acting as the optimal
and the desirable solutions, which would be including the
appropriate index for the big up data and the updated data
preprocessing technology. Because of the challenge related to
limited capacity in maintaining and analyzing the massive
datasets, the service providers along with the business owners
are seen to be relying upon the professional and tools for the
purpose of analyzing the data. This type of reliance is
associated with increasing the potential safety as well as the
risks for the data [9]. It is essentially to be made sure that the
analysis of the big data is necessarily being delivered to the
third party depending upon the conditions that is there exists
proper protective measures, which are aimed at the protection
of the sensitive data.
Some of the other issues, which are arising because of the
big data, includes the scalability and the time of response.
Management of the data which are increasing at a rapid rate
and are large in size turns out to be one of the most
challenging issue, scalability is generally expressed in three
different aspects which is associated with including the
volume of data, size of the hardware and the concurrency. The
analytical system, which the big data is having, must be
associated with supporting the existing as well as the future
datasets.
2.2. Issues in the clouds:
The fact that the users are still skeptical regarding the
authenticity of the clouds is the reason responsible for
restricting the adaptation of the cloud computing technology.
This turns out to be a major challenge, which is being faced
by the cloud computing technology [10]. Some of the major
problems or reasons, which are responsible for restricting the
cloud efficiency, is associated with including the security,
costing and charging models, service level agreements and
cloud interoperability. The issue related to the security aspect
has been associated with limiting the acceptance of the cloud
computing. There exists numerous security concerns which
are seen to be linked with the cloud computing. Some of the
issues are also related to the additional cost that is required for
the transferring of the organizations data between the different
stakeholders by making use of the clouds, which is estimated
to eb very high. This acts as a major challenge especially for
the customers who are associated with the usage of the hybrid
cloud deployment model in which the distribution of the data
is done amongst the different type of clouds.
2.3. Technological issues kept unsolved after the
convergence of the three technologies:
Despite of the fact that many of the users have greatly been
associated with the acceptance of the cloud computing and the
research conducted upon the big data present in the cloud iss
seen to be have been remaining in the early stage. There exists
different kind of issues, which are still unsolved and have
been seen to be having an continuous emergence by means of
application of these particular technology [11]. The
convergence of the three technologies have been associated
with solving most of the problems but still there exists some
portion of the issues which have been left unsolved even after
the convergence. Some of the issues are mainly consisting of
the following:
 Heterogeneity which is associated with bringing of
challenges like the lock-in of vendors, along with the
issues related to licensing;
 integrity of the data is a crucial aspect of the data
security which is associate with stipulation of the data
which should only be modified by the parties who are
authorized to do so in order to prevent any kind of
misuse [12];
 quality of the data is still an issue because of the origin
of the data from different sources which are not
possible verify;
 the privacy and the reason behind this is the exposure
of the information of individuals for scrutiny, leading
to putting forward of different kind of issues such as
the profiling, stealing and loss of control of the data.
 Governance in which the policies along with the
principles and eth frameworks, which are associated

with striking the stability existing between the risk and
the value, associate with the creation of huge
challenges.
 Legal issues because of the existence of different laws
in different countries and regulations associated with
helping in achievement of privacy and protection [13].
Still no laws and regulations are associated with
providing of adequate protection for the data of an
individual. [7].
Annotated Bibliography:
Cai, H., Xu, B., Jiang, L., & Vasilakos, A. V. (2016). IoT-based big data
storage systems in cloud computing: Perspectives and challenges. IEEE
internet of things journal, 4(1), 75-87.
This paper mainly idcusses about the different IoT applications which has
emerged out to be an important field for the engineers as well as for the
researchers. This in turn has been associated with reflecting the magnitude
and the impact of the data related problems which are to be solvedin the
contemporary businesses, besdes this the paper also analyzes the current
research upon the IoT applications after providing of the functional
framework which is associated with identifying the acquisition, management,
processing and mining areas of IoT big data, and several associated technical
modules.
Marjani, M., Nasaruddin, F., Gani, A., Karim, A., Hashem, I. A. T., Siddiqa,
A., & Yaqoob, I. (2017). Big IoT data analytics: architecture, opportunities,
and open research challenges. IEEE Access, 5, 5247-5261.
This paper is mainly associated with investigating the state-of-art research
efforts which are directed towards the big data analytics of the IoT. Besides
this paper also dicusses about the relationship that exists between the big data
and the IoT. The study is also associated with addition of values by proposing
a new architecture for the Big IoT data analytics. In addition to this the paper
discusses about the types of big data IoT analytics along with the methods and
the technologies used for big data mining.
Jin, X., Wah, B. W., Cheng, X., & Wang, Y. (2015). Significance and
challenges of big data research. Big Data Research, 2(2), 59-64.
This paper is associated with depicting the fact that the radip developments in
the internet, as well as in the internet of things and the cloud computing have
been associated with leading to an explosive growth in the data in almost
every indsurtry as well as business area. The paper also depicts the fact that
big data is one of the hot topics which has been aassociated with attracting the
attention of academia, industry, and governments around the world. This
paper briefly presents forward the concept of big data. The grand challenges
faced by big data has also been presented in this paper along with the possible
solutions so as to address the challenges.
Yang, C., Huang, Q., Li, Z., Liu, K., & Hu, F. (2017). Big Data and cloud
computing: innovation opportunities and challenges. International Journal of
Digital Earth, 10(1), 13-53.
This paper has been associated surveying two frontiers which includes the big
data and the cloud. Besides this the paper has also resented reviews along
with the advantages and the consequences of the utilization of the cloud
computing so as to tackle the big data in the digital world and all other
relevant science domains. This has been by putting forward of a general
introduction folloowed by the sources, challenges status of the technolohy and
the research opportunites. The review of this paper has also been associated
with introducing the future innovations and the research agenda related to
cloud computing.
Stergiou, C., Psannis, K. E., Kim, B. G., & Gupta, B. (2018). Secure
integration of IoT and cloud computing. Future Generation Computer
Systems, 78, 964-975.
According to this paper the mobile cloud computing is one of the new
technology which generally refers to the infrastructure in which both the data
storage and the data processing operates outside of the mobile device.
Another recent technology is associated with including the Internet of things
and this is growing at a rapid rate in the filed of telecommunication. The
paper is associated with representing asurvey of the IoT and the cloud
computing by having a focus upon the different kind of security issues of both
of this technology. Two after mentioned technologies have been combined
soa sto examine the common features so as to dicover the different benefits
of the integration.
Díaz, M., Martín, C., & Rubio, B. (2016). State-of-the-art, challenges, and
open issues in the integration of Internet of things and cloud
computing. Journal of Network and Computer applications, 67, 99-117.
The paper is associated with defining the fact that the IoT is a new paradigm
which id dependent upon the internet which is seen to be compromising of
different kind of technologies such as the RFID. This paper has mainly been
associated with presentinga survey related to the integration of the
components which includes the cloud paltforms, cloud infrastructure and the
IoT middleware. Additionally some of the integration proposals along with
the data analytics techniques have also been surveyed along wuth the different
kind of challenges and the opern research issues.
3. Methodology:
In todays era of big data hardware is no longer associated
with acting as the limiting factor in the process of acquisition
of the applications but certainly acts as the management of the
data acquired. The usage of the cloud storage is very simple
and cloud computing resources which are available are
responsible for the creation of the single aggregation point for
the data which are seen to be coming in from huge number of
devices which have been embedded and are associated with
providing of access to that particular data form any of the
group [15]. The convergence of the IoT and the cloud
computing is one of the best solution for tackling the various
kind of problems which are faced while handling the big data.
In case of big data the collection of the data is one of the
major concern and the IoT is capable of playing an important
role in the process of data collection and sharing of the data.
The cloud is associated with offering of everything as service
business model for the IoT and the Big data. The usage of the
different big data approaches along with the clouds is possible
for the purpose of storing and analysing the various IoT data
so as to improve the scalability along with the availability
[16]. This would be requiring billions of devices envisaged in
the IoT. This is also necessary for enabling the WSNs so as to
become extensions of the Internet Infrastructure and for taking
of full advantage of the cloud as well as the big data services.
The availability of the increased storage along with increased
processing power at a cost which is low along with a greater
bandwidth would be associated with enabling of the range of
cloud computing services [17]. In terms of the IoT this would
be associated with allowing more number of sources of the
data to be collected and for holding the data for a longer time
along with providing the capability of being processed by
making use of the powerful cloud computing based
applications and the big data techniques such as the HBase
and MapReduce.
Big data storage along with the processing are considered
to be of the major application of the cloud computing systems.
In addition to this the development of the IoT paradigms is
also seen to have advanced in the research upon the Machine
To Machine or M2M communications and has been associated
with enabling of the novel tele-monitoring architectures. Still
there is an essential need of converging the existing
decentralized cloud systems along with the general software
so as to process the big data and the IoT systems [18]. Most of
the IoT applications are seen to be based upon the M2M

communication protocols which exists between the large
number of heterogeneous and the geographically dispersed
sensors [19]. For this reason they are seen to be having the
need of handling the various kind of streams from the sensors
and this cloud be associated with acting as a direct benefit
from the immense distributed storage capacities of the cloud
computing infrastructures. In addition to this the cloud
computing infrastructures are capable of boosting up of the
computational capabilities of the IoT applications,
The Hbase storage a type of storage which is associated
with taking the responsibility of scaling and storing the
clusters of data in the multiple number of storage nodes [20].
In addition to all this there exists several IoT services which
turns out to be beneficial in terms of the utility based delivery
paradigm which is mainly associated with emphasizing upon
the on-demand establishment and the delivery of the IoT
applications over a cloud based infrastructure.
4. Findings and Conclusion:
The aspects of the big data along with the cloud computing
and IoT has seen to have emerged in the last few years which
has been associated with providing of more data along with
opportunities which have been associated with enabling of the
research as well as the decision support applications in almost
each and every filed. The convergence of the three
technologies is also responsible for the creation of new
opportunities however it had also been associated with
resulting in putting forward of different kind of challenges.
The discussion conducted above helps in concluding to the
fact that there are several challenges which are faced by the
cloud computing and the IoT while handling the big data. The
report has been associated with highlighting the various kind
of problems which are generally faced in a brief manner and
this brief discussion has been conducted so as to understand
the potential impact of the big data upon the two emerging
technologies. Some of the major challenges which are faced
includes the privacy and security issues, data mining issues
and data integration issues and data storage issues. This report
is also aimed at highlighting certain technologies which are
associated with helping in overcoming the major challenges
that are being faced while handling the big data. The report
has also been associated with discussing the methodologies
for solving the major challenges faced while handling the big
data. The possible strategies for tackling the problems has also
been discussed in this paper and there exists numerous
technologies which has been deployed previously so as to
tackle all this problems. One of the best methodology for
solving the problem is by combining the Haddop with the
MapReduce, and in addition to this the Hive and the Mahout
are also some of the technologies which are used for tackling
the challenges. This combination is considered to be the best
and the possible solution for the existing as well as for the
future scenarios.
References:
[1] Cai, H., Xu, B., Jiang, L., & Vasilakos, A. V. (2016). IoT-based big data
storage systems in cloud computing: Perspectives and challenges. IEEE
internet of things journal, 4(1), 75-87.
[2] Marjani, M., Nasaruddin, F., Gani, A., Karim, A., Hashem, I. A. T.,
Siddiqa, A., & Yaqoob, I. (2017). Big IoT data analytics: architecture,
opportunities, and open research challenges. IEEE Access, 5, 5247-
5261.
[3] Jin, X., Wah, B. W., Cheng, X., & Wang, Y. (2015). Significance and
challenges of big data research. Big Data Research, 2(2), 59-64.
[4] Yang, C., Huang, Q., Li, Z., Liu, K., & Hu, F. (2017). Big Data and
cloud computing: innovation opportunities and challenges. International
Journal of Digital Earth, 10(1), 13-53.
[5] Wang, L., & Ranjan, R. (2015). Processing distributed internet of things
data in clouds. IEEE Cloud Computing, 2(1), 76-80.
[6] Stergiou, C., Psannis, K. E., Kim, B. G., & Gupta, B. (2018). Secure
integration of IoT and cloud computing. Future Generation Computer
Systems, 78, 964-975.
[7] Díaz, M., Martín, C., & Rubio, B. (2016). State-of-the-art, challenges,
and open issues in the integration of Internet of things and cloud
computing. Journal of Network and Computer applications, 67, 99-117.
[8] Sivarajah, U., Kamal, M. M., Irani, Z., & Weerakkody, V. (2017).
Critical analysis of Big Data challenges and analytical methods. Journal
of Business Research, 70, 263-286.
[9] Bello-Orgaz, G., Jung, J. J., & Camacho, D. (2016). Social big data:
Recent achievements and new challenges. Information Fusion, 28, 45-
59.
[10] Najafabadi, M. M., Villanustre, F., Khoshgoftaar, T. M., Seliya, N.,
Wald, R., & Muharemagic, E. (2015). Deep learning applications and
challenges in big data analytics. Journal of Big Data, 2(1), 1.
[11] Jin, X., Wah, B. W., Cheng, X., & Wang, Y. (2015). Significance and
challenges of big data research. Big Data Research, 2(2), 59-64.
[12] Yin, S., & Kaynak, O. (2015). Big data for modern industry: challenges
and trends [point of view]. Proceedings of the IEEE, 103(2), 143-146.
[13] L’heureux, A., Grolinger, K., Elyamany, H. F., & Capretz, M. A.
(2017). Machine learning with big data: Challenges and
approaches. IEEE Access, 5, 7776-7797.
[14] Hilbert, M. (2016). Big data for development: A review of promises and
challenges. Development Policy Review, 34(1), 135-174.
[15] Yang, C., Huang, Q., Li, Z., Liu, K., & Hu, F. (2017). Big Data and
cloud computing: innovation opportunities and challenges. International
Journal of Digital Earth, 10(1), 13-53.
[16] Cai, L., & Zhu, Y. (2015). The challenges of data quality and data
quality assessment in the big data era. Data Science Journal, 14.
[17] Hashem, I. A. T., Yaqoob, I., Anuar, N. B., Mokhtar, S., Gani, A., &
Khan, S. U. (2015). The rise of “big data” on cloud computing: Review
and open research issues. Information systems, 47, 98-115.
[18] Cai, H., Xu, B., Jiang, L., & Vasilakos, A. V. (2016). IoT-based big data
storage systems in cloud computing: Perspectives and challenges. IEEE
internet of things journal, 4(1), 75-87.
[19] Yu, S. (2016). Big privacy: Challenges and opportunities of privacy
study in the age of big data. IEEE access, 4, 2751-2763.
[20] Marjani, M., Nasaruddin, F., Gani, A., Karim, A., Hashem, I. A. T.,
Siddiqa, A., & Yaqoob, I. (2017). Big IoT data analytics: architecture,
opportunities, and open research challenges. IEEE Access, 5, 5247-
5261.