INFS 5095 SP2 2018: Technology Review on Big Data for IoT Data

This report provides a technology review of using big data for managing and exploiting Internet of Things (IoT) data. It begins by explaining the nature of IoT data, emphasizing the increasing data consumption from connected devices and its impact on businesses. The report discusses storage and data security in big data, highlighting the shift towards cloud-based models and technologies like HDFS and NoSQL databases. It also outlines the benefits of using big data for IoT data, such as pre-processing data, filtering unwanted data, and real-time analysis for predicting trends. Limitations, including managing diverse data types and ensuring data security against cyber-attacks, are also addressed. The report concludes by explaining how organizations can use IoT data for analytics, modeling, and creating new services.

Big Data Basics
INFS 5095
2018 SP2
Student's Assignment Guide
Assignment 1 – Technology Review
(Internal and External/Online)
Technology Review for managing
or exploiting IoT Data
[Your Name]
[Date]
Page 1 of 10

Page 2 of 10

Contents
Executive Summary...................................................................................................................4
About Internet of Things (IoT) Data..........................................................................................5
About Storage and data security in big data...............................................................................6
Benefits of Using Storage and data security of Big Data with IoT Data...................................7
Limitations to Using Storage and data security of Big Data with IoT Data..............................8
How organisations will find IoT Data useful.............................................................................9
References..................................................................................................................................9
Page 3 of 10

Executive Summary
The following report discusses the use of big data in Iot and how the technology is used to
store user information and data securely. The report discusses about the Iot Devices and its
impact on modern businesses. The report discusses several features of big data as well in
relation to data security and storage techniques .Several limitations has been discussed as
well.
Page 4 of 10

About Internet of Things (IoT) Data
With the exponential increase in the number of devices connected to the internet, the
excessive data consumption from these devices reflect the importance of Big data in IoT
devices. The web of devices such as actuators, sensors. Software, embedded electronics,
home appliances and vehicles are collectively referred as the Internet of Things or IoT. Each
device has its own unique computing system that enables it to communicate with the other
devices which are connected to the internet. This direct integration of the devices facilitates
accuracy and efficiency of data transfer among the devices. The organizations which are
delivering the IoT devices have to analyse and collect data from these devices simultaneously
allowing a direct integration of computer systems with the physical worlds (Mulani and
Pingle 2016). The data from the IoT devices can range from camera streaming services,
biochip transponders to heart monitoring implants. The data from the devices are
continuously exchanged among them leading to an amalgamation of software, service, data
and hardware.
According to a research paper from Gartner, estimates state that the revenue that will be
generated from IoT data by the year 2020 will exceed more than 300 billion dolars (Cui
2016). The data that will be generated will be vast and will need well analysis method for
accurate representation. This will impact other companies which will be forced to turn to data
analysis tools and big data for accommodating this massive influx of user information from
these IoT devices.
About 73% of the data that are retrieved from IoT devices are used to improve businesses.
The data is used for reducing downtime and maintenance, reducing the operational costs,
improvement in the decision making and product performance (Wortmann and Flücht 2015).
The data are also used to build business cases and strategies and how they are affecting the
Page 5 of 10

business from their traditional counterparts. Internet of things data also enables organizations
to invest in technical consultation, deployment and implementation of strategies and
planning.
About Storage and data security in big data
The influx of data from the IoT devices have to be handled by the data centres who have built
their respective data storage facilities. The data centres need to be capable of handling such a
vast amount of unstructured data. Organizations are slowly shifting towards PaaS service
models (cloud based models) instead of developing their own storage solutions. Unlike the
traditional data storage systems, these models need to be constantly upgraded to handle the
continuous overflow of data.
The storage of data with big data helps to potentially transform businesses and society across
sectors. The advanced analystics that are offered by big data is still unrivalled. Then big data
storage technologies are capable of idetofying various trends in the data and has the
capability to handle huge amount of data. HDFS or Hadoop File systems are used to store
data which are unstructrured in a proper way (John Walker 2014). It is an integral part of the
Hadoop Framework and is useful for bulk processing and ingesting data. Another storage
technology used for big data is NoSQL databases. It is a model that does not rely on
transactional properties such as durability, isolation, consistency and atomicity (Chen et al.
2014).
A new relational database known as NewSQL is also used for compare scalability while
maintain the transactional features of NoSQL. Some technologies provide more query
facades to the existing NoSQL database. It provides a high level interface while achieving a
low latency. Other technologies has been developed for increasing the data security of the
Page 6 of 10

storage systems. Hive is such a platform that has been developed over HDFS system. By
translating the query via Map Reduce it helps to execute the queries at high latency. This is
made possible by validating the data every single time during the query time. Other storage
facilities used for big data analytics is Cloud storage. This system is used for onlinen backups
rather than hardware storages and offers a cheaper alternative to the enterprise environment.
Benefits of Using Storage and data security of Big Data
with IoT Data
The benefits of using big data for storage and data security with IoT Data is immense and is
necessary for the two technologies to co-exist. The huge amount of data cannot be handled by
traditional databases as they are not built for handling data at a high rate. Big data helps to
utilize the edge data analytics to pre-process data before they are put into the data servers for
storage. Some data are filtered away and useless data are monitored and removed
accordingly. With the growing number of unstructured data, companies delivering IoT
products need big data to analyse what data is coming in and going out of the company (Lee
et al. 2013). Relevant and vital data are collected from the IoT devices and analysed for their
importance accordingly. Parameters are set which rules out any type of unwanted data that
are coming from the IoT devices. This helps to reduce the amount of data that are used for
storage purposes. Unnecessary data is filtered out to prevent excessive data from
accumulating in the data storages.
Another benefit of using big data for storage and security purposes with IoT is the usage of
different communication protocols such as WiFi, Bluetooth, Zigbee, MQTT and CoAp by the
IoT devices. Understanding all these protocols and analysing the data accordingly is a major
challenge for many businesses. IoT interoperability with big data can open up new avenues
which can be used by businesses for better efficiency. The data needs to be correlated,
Page 7 of 10

received from several domain sources and protocols and analysed with the help of big data to
make it structured and usable.
The vast amount of data coming from several Iot devices need to be analysed in real so that
the data makes sense and stays usable. Big data analytics helps to analyse the data in real time
and predict certain trends that may have existed from a long time. Such insights helps the
business to make decisions based on actual facts rather than gut feelings (Marz and Warren
2015). The usability of big data with IoT devices for storage has other benefits such as
anticipating failures which cannot be tracked by any other analytics. As the embedded
sensors in the IoT devices transmit data 24x7 , with real time analysis it can detect real time
dangers and help improve customer experience.
Limitations to Using Storage and data security of Big
Data with IoT Data
With several IoT devices in the market, the data can include database data, transactional data,
ERM as well as CRM data and other unstructured data such as emails. This massive types of
data from several sources can be managed up to an extent with the help of pre-processing but
development is needed for efficient services. Another limitation of the data that is stored is
based on data security. With the rising levels of cyber-attack sin the recent decade, no
information can be termed as 100% secure (Gubbi et al. 2013). Strong user authentication,
automatic encryption and intrusion protection are necessary for protecting this data but that
area is still being researched. Proper tools need to be deployed for better security controls
such as protecting log files and tools used for analytics inside the data platform.
Another limitation is the improper encryption of output data from IoT devices. The data that
are processed by big data analytics tools are often showed via dashboards, reports and
Page 8 of 10

appluications. Although efforts are put in encrypting the input data sources properly, not
much work is done to encrypt the output data as well whuch can be used for intrusion.
How organisations will find IoT Data useful
The IoT data is used for gaining analytics information of consumers and used as a source of
modelling. The data is used to make modules for an organisation where new modules are
added based on their necessity. New services such as customer care centres can be added by
an organisation if the need to become more customer centric arises in the organisation based
on the data. Various vehicles use this data to decide their telemetry (Wu et al. 2014). A
temperature drop in a certain region or excessive traffic in an area allows the car to navigate
through an entirely different path. This rapid analysis of user information is only possible
with real life analysis with the help of big data. It can help the organisation to create new time
series data which allows the device to create a record of all the real time events.
Page 9 of 10

References
Chen, M., Mao, S. and Liu, Y., 2014. Big data: A survey. Mobile networks and
applications, 19(2), pp.171-209.
Cui, X., 2016. The internet of things. In Ethical Ripples of Creativity and Innovation (pp. 61-
68). Palgrave Macmillan, London.
Gubbi, J., Buyya, R., Marusic, S. and Palaniswami, M., 2013. Internet of Things (IoT): A
vision, architectural elements, and future directions. Future generation computer
systems, 29(7), pp.1645-1660.
John Walker, S., 2014. Big data: A revolution that will transform how we live, work, and
think.
Lee, G.M., Crespi, N., Choi, J.K. and Boussard, M., 2013. Internet of things. In Evolution of
Telecommunication Services(pp. 257-282). Springer, Berlin, Heidelberg.
Marz, N. and Warren, J., 2015. Big Data: Principles and best practices of scalable realtime
data systems. Manning Publications Co..
Mulani, T.T. and Pingle, S.V., 2016. Internet of things. International Research Journal of
Multidisciplinary Studies, 2(3).
Wortmann, F. and Flüchter, K., 2015. Internet of things. Business & Information Systems
Engineering, 57(3), pp.221-224.
Wu, X., Zhu, X., Wu, G.Q. and Ding, W., 2014. Data mining with big data. IEEE
transactions on knowledge and data engineering, 26(1), pp.97-107.
Page 10 of 10