Managing IoT Data with Event Stream Processing: Technology Review

This report discusses the importance of big data and its utility in bringing about changes in organizations. It identifies a technology, Event Stream Processing, and discusses its benefits of using with IoT data in depth. The paper also sheds light on the limitations of using Event Stream Processing technology using IoT data. The paper also details about the usefulness of this technology in organization providing an in-depth analysis of this technology.

Big Data Basics
INFS 5095
2018
Student's Assignment Guide
Assignment 1 – Technology Review
(Internal and External/Online)
Technology Review for managing
or exploiting IoT Data
[Your Name]
[Date]
Page 1 of 8

Contents
Introduction...........................................................................................................................................3
About Internet of Things (IoT) Data.......................................................................................................3
About Event stream Processing.............................................................................................................4
Benefits of Using Event Stream Processing with IoT Data.....................................................................5
Limitations to Using Event Stream Processing with IoT Data................................................................6
Using Event Stream Processing with IoT Data.......................................................................................6
References.............................................................................................................................................8
Page 2 of 8

Introduction
Nowadays, organizations have access to and deal with large volume of data (Wu et al. 2014).
The organizations have realized that to become successful and in order to maximalise the advantage,
they need to harness data and utilize it for identification of new opportunities. The report discusses
the importance of big data and its utility in bringing about changes in organisation (George , Haas
and Pentland 2014). The use of Big Data analytics have many advantages in an organisation that
includes cost reduction of the company, faster and better decision making and creation of new
products and services .The report discusses about Internet about things(IoT) data and its
importance (Gubbi et al. 2013) . The report identifies a technology, Event Stream Processing and
discusses its benefits of using with IoT data in depth. The paper also shed light on the limitations of
using Event Stream Processing technology using IoT data. The paper also detail about the usefulness
of this technology in organization providing an in-depth analysis of this technology.
About Internet of Things (IoT) Data
IoT is an acronym for Internet of Things. By Internet of Things, it relates to the ever-
increasing network of all the physical objects that has IP address for connectivity of internet and it
relates to the communication that takes place among these entities and various other devices and
system that are internet-enabled (Lee and Lee 2015). The Internet of Things expands the internet
connectivity beyond the conventional devices, which are usually used like smart phones, tablets,
desktop computers and laptop to varying range of devices, and daily used things that make use of
embedded technology to make communication and interaction with the external environment all
through the help of internet. The components of Internet of Things (IoT) are smart device, sensors,
gateway, cloud, analytics and user interface.
Internets of Things (IoT) are revolutionizing products, it is considered as the major frontier
that can improve our lives in many aspects. The devices that were not earlier been able to connect
and networked now can be connected with the help of IoT. Thus, IoT is all set to bring about a
transformation to the world completely. It is estimated that by 2020 almost Fifty million devices
would be connected to the internet (Singh, Tripathi and Jara 2014).
Figure 1: Future of Internet of things
(Source : Singh, Tripathi and Jara 2014)
Page 3 of 8

The benefits of Internet of Things in an organization’s business are as follows:
 Efficient utilization of resources is one of the benefits of internet of Things.
 Reduced efforts of human
 Lowering of cost and increase in productivity
 Real-time marketing
 Decisions analytics (Wagner et al. 2014)
 Better customer experiences
 Superior quality data
IoT is proving to be a modern platform, to provide organized customer service by building real
time interaction and it provides interactive engagements with the customers.
Figure 2: Concept of
Internet of Things (IoT)
(Source : Lee and Lee 2015)
About Event stream Processing
An event stream is a succession of event objects aligned in an organised manner, generally
by time. Event Stream Processing (ESP) is a type of computing that is performed on data that are
performed on the data on these event objects (Cammert et al. 2016). The objective of Event stream
processing is Stream data integration or Stream data analytics which are also called Complex Event
processing (CEP)( Cugola et al.2015).The Stream data analytics can be performed and executed in
three instances .First ,when the new data arrives using the ESP platform software which are event-
driven. Secondly, shortly after the data has arrived using the queries, which are real time and on -
demand .Thirdly, after the data has arrived long and stored using on demand queries of historic
data.
While the conventional analytics processes the data after it is stored, event stream
processing changes the order of procedure of analytics, which allows faster reaction time and
Page 4 of 8

provides an opportunity to take proactive measures before the situation gets over. By using this
technique of data processing, it brings about many advantages, as the system does not have to
retain various events as a result it uses very less memory.
Benefits of Using Event Stream Processing with IoT
Data
Event Stream Processing provides many smart solutions to different challenges .The benefits of
using Event Stream Processing includes:
 It processes huge volume of stream events.
 It enables to analyse huge volume of high velocity big data while it is in the state of motion
enabling the users to aggregate, categorise filter and cleanse the data even before it is
stored.
 It enables to monitor data and make interaction continuously (Malek et al. 2017).
 Scale the data according to data volumes.
 Identify and detect interesting patterns and relationships
 It enables the users to handle the issues as they arises
With the benefits of Event Stream processing, the companies have more access to streaming of
data from both internal and external sources. The internal sources include corporate websites,
transactional application, control system, sensors and meters. The external sources include the
government agencies, business partners, data brokers, social computing platforms, news and
weather feeds. The Event Stream Processing technology is growing rapidly and it will be adopted by
multiple departments various companies. ESP will reach new heights in next five years making use of
SaaS solutions and off-the-shelf packaged applications.
Page 5 of 8

Figure 3: Event Stream Processing Technology Model
(Source: Cammert et al. 2016)
Limitations to Using Event Stream Processing with IoT
Data(ESP) or Event Stream processing has wide range of application and usability. In today’s
world of Internet of things, the event stream processing finds its importance and applications in
many fields. There are hardly any drawback and limitation associated with the Event Stream
Processing with IoT data. However, this report identifies a limitation or drawback of Event Stream
Processing with IoT. The disadvantage of Event Stream Processing is that stream processors only
retain sufficient data to fulfil the criteria of window based queries, which are presently active in the
system (Cao et al. 2013). On the other hand, windows can retain data of several months. Most of the
stream processing platforms should provide integration associated with storage platforms, both for
persistence of stream, and provide static table or data stream joins
Using Event Stream Processing with IoT Data
Processing of event stream data is one of the major entities that enable continuous
intelligence and other facets of digital business. It has brought about complete transformation of
markets of finance and it has become an essential to various smart electrical grids, marketing, which
are location based, supply chain, fleet management and other operations of transport. Majority of
the growth and development in Event Stream Processing usage for the coming ten years will come
from mainly three areas where it is already somewhat established various entities like IoT,
management of customer experience and applications related to detection of frauds.
The few ways by which Event Stream Processing is associated and plays very significant role in
Internet of Things (IoT) are as follows:
 Detection of events of importance that spark apt action-Processing by Event stream
identifies complex arrangements in the real time that are generated by a behavior of the
individual on their devices (Tsimelzon et al. 2013). The technologies associated with Event
Stream processing are also used for detection of opportunities and potential frauds .This
opportunities can be used to send marketing offers, which are personalized and can be used
for propagation of information to dedicated system for prompt action.
 Aggregation of data for checking – The Processing by Event stream technology can be used
for continuously monitoring of data of sensor from devices and sources of other
equipments , it is used for monitoring for progression, interactions and point of thresholds
that indicates a problem that alerts the operator so that they can take immediate action
before chances of any damage.
 Sensor cleansing of data and validation –There may be a condition when data of sensor
sometimes are not complete or may contain any irregular values. There are many
technologies, which can be directly embedded into streams of data (Gschwandtner et al.
2014).This addresses the sensor quality of data. These methods involve getting hold of few
incidents of past that is required to examine the faulty issues of data quality.
 Operations that predict real time data and optimization of data – Processing of data that
are mainly event streamed sometimes is not sufficient to legitimize making of decisions,
which are done in real-time. Streaming of data should always involve power of analytics that
enables to comprehend and comprehend complex structures which provides characteristic
Page 6 of 8

figures. It is already understood that the obtained value in Internet of Things (IoT) data will
come from analysis, of what the current conditions are and any proactive actions, which
could be undertaken. In the real world, this would signify that the data present ion the
transit of the arriving train would be obtained through a chain or series of calculations,
which would resolve how the arrival of the train could immensely influence various other
vehicles. The calculations of data that are done in real time environment can benefit to
curtail the influence on the travellers by postponing, making the train wait at next junction
so that the people do not fail to oversight their connections. This also can make the bus
driver wait by alerting them so that they can wait for a sometime more so that to avert
isolation of passengers at the station.
Page 7 of 8

References
Cammert, M., Heinz, C., Krämer, J. and Riemenschneider, T., Software AG, 2016. Systems and/or
methods for forecasting future behavior of event streams in complex event processing (CEP)
environments. U.S. Patent 9,286,354.
Cao, J., Xiao, Q., Ghinita, G., Li, N., Bertino, E. and Tan, K.L., 2013, March. Efficient and accurate
strategies for differentially-private sliding window queries. In Proceedings of the 16th International
Conference on Extending Database Technology (pp. 191-202). ACM.
Cugola, G., Margara, A., Matteucci, M. and Tamburrelli, G., 2015. Introducing uncertainty in complex
event processing: model, implementation, and validation. Computing, 97(2), pp.103-144.
George, G., Haas, M.R. and Pentland, A., 2014. Big data and management.
Gschwandtner, T., Aigner, W., Miksch, S., Gärtner, J., Kriglstein, S., Pohl, M. and Suchy, N., 2014,
September. TimeCleanser: A visual analytics approach for data cleansing of time-oriented data.
In Proceedings of the 14th international conference on knowledge technologies and data-driven
business (p. 18). ACM.
Gubbi, J., Buyya, R., Marusic, S. and Palaniswami, M., 2013. Internet of Things (IoT): A vision,
architectural elements, and future directions. Future generation computer systems, 29(7), pp.1645-
1660.
Lee, I. and Lee, K., 2015. The Internet of Things (IoT): Applications, investments, and challenges for
enterprises. Business Horizons, 58(4), pp.431-440.
Malek, Y.N., Kharbouch, A., El Khoukhi, H., Bakhouya, M., De Florio, V., El Ouadghiri, D., Latré, S. and
Blondia, C., 2017. On the use of IoT and big data technologies for real-time monitoring and data
processing. Procedia computer science, 113, pp.429-434.
Singh, D., Tripathi, G. and Jara, A.J., 2014, March. A survey of Internet-of-Things: Future vision,
architecture, challenges and services. In Internet of things (WF-IoT), 2014 IEEE world forum on (pp.
287-292). IEEE.
Tsimelzon, M., Sanin, A., Motwani, R., Seidman, G.R. and Patel, G., Sybase Inc, 2013. Continuous
processing language for real-time data streams. U.S. Patent 8,396,886.
Wagner, S., Brandt, T., Kleinknecht, M. and Neumann, D., 2014. In free-float: How decision analytics
paves the way for the carsharing revolution.
Wu, X., Zhu, X., Wu, G.Q. and Ding, W., 2014. Data mining with big data. IEEE transactions on
knowledge and data engineering, 26(1), pp.97-107.
Page 8 of 8