Big Data: Characteristics, Technologies, and Applications Report

This report provides a comprehensive overview of big data, defining it as a technique for managing large datasets and addressing challenges in analysis, curation, and privacy. It explores the history of big data, tracing its roots in data analytics and the evolution of data processing technologies. The ...Read More

BIG DATA
1
Big Data

BIG DATA
2
Table of Contents
Introduction...........................................................................................................................................2
History of big data.................................................................................................................................2
Overview of big data.............................................................................................................................3
Characteristics of big data.....................................................................................................................4
Volume..............................................................................................................................................4
Variety...............................................................................................................................................4
Velocity..............................................................................................................................................4
Variability..........................................................................................................................................4
Veracity.............................................................................................................................................5
Complexity.........................................................................................................................................5
Technologies..........................................................................................................................................5
Applications of big data.........................................................................................................................6
Government......................................................................................................................................7
International development................................................................................................................7
Manufacturing...................................................................................................................................7
Cyber-physical models.......................................................................................................................7
Advantages of big data..........................................................................................................................8
Disadvantages of big data......................................................................................................................8
Challenges faced by big data.................................................................................................................8
Dealing with data growth..................................................................................................................8
Generating insights in a timely manner.............................................................................................9
Integrating disparate data resources.................................................................................................9
Security and privacy-related issue.....................................................................................................9
Recommendation..................................................................................................................................9
Conclusion...........................................................................................................................................10
References...........................................................................................................................................11

BIG DATA
3
Introduction
Big data is defined as a technique which is used to control and manage a large
amount of data set. Many organization contains very large data but they are not able to
handle some challenges such as analysis, data curation, sharing, and data visualization
and information privacy (Chen, & Zhang, 2014). To avoid such kind of problems,
information and communication technology has established a new method that is big
data. The main goal of this report is to appreciate the fundamental idea of big data with
their advantages and disadvantages. Today most organization analyse the data with the
help of big data technology because it has the ability to control and monitor huge data.
The word big data refer to the usage of prognostic analytics and this technology is more
efficient for the visualization of data. With the help of this technology consumers can
handle data or information in a computer system and it is observed that the rate of big
data has increased by 40% in the last five years.
History of big data
The fundamental concept of big data technique is completely based on the data
analytics approach. Many organizations understand that if they detect the data which
streams into their commercial then they can apply this process for analysis. Big data is
an advanced technology that comprises data and it also used to store the consumer’s
data. The term big data was identified by Roger Douglas in the year 2005 and the
application of this technology is something that has been in reality for a long duration.
The colossus is in one of the first data processing technique that was introduced by the
British in the year 1943 (Chen, Chiang, & Storey, 2012). This machine was used for
searching for the data pattern and record user's data at the rate of five thousand
characters per second. From this investigation in the year 1952 the United States
created a national security agency that uses a machine learning program to handle a
large amount of data. The first data centre was developed in the year 1965 for storing
billions of tax and fingerprint sets.
During the generation of big data yahoo developed an open source Hadoop for
indexing the complete World Wide Web. At that time the use of social media was
increasing very fast and a large amount of data were detected on daily basis (Chen, Mao,

BIG DATA
4
& Liu, 2014). Many government communities and business established big data
projects like in the year 2009 the biggest biometric database created and governments
analysed and stored fingerprint and iris scans of all consumers. In the last few years,
many organizations have introduced a new approach to big data, for example, HCL,
Google and many more. In the year 2017, the rate of this technique has grown by 30%
due to this increment information and technology developed many processes to handle
human data. 80% of the available data has been developed in the last 2 years and the
use of big data process is growing very quickly because many consumers use social
media for communication.
Overview of big data
Big data technology helps business communities to harness their data or
information and use this technique to produce a new opportunity for analysis. This
technology is making a new generation of decision support data management and many
industries are identifying the potential value of this data and putting into a technique.
Information and technology developed many approaches for maintaining data for
example simulation, machine education, and regression analysis but these are not much
efficient. For which big data technology has been created that has the ability to analyze
both structured and unstructured data and a recent study indicated that in the year
2015 big data provided almost 4 million jobs globally (Fan, & Bifet, 2013).
It is also making a huge demand for consumers and organization’s that can store
and analyse big data technology. Recent investigation shows that in the year 2018 the
United States faces a shortage of 140,000 to 190,000 individuals with deep analytics
(Gandomi, & Haider, 2015). Big data technology represent a blog of records where data
of every user can be analysed and it is a correct statistically that provide a clear
understanding of pictures, documents and data. It has a volume which needs a parallel
processing and a special model to store human data and it is not possible to access one
computer. The main reason for making big data technology was to store and analysis
the huge amount of data and it has the capability to recognize the customer data. This
technology has numbers of resources, for example, every mouse clicks on a web link
that can be stored in web log files and analysed to understand the behaviour of
consumers. This technology involves data sets that store, manage and capture data. This

BIG DATA
5
technology is very popular and it can identify the most efficient platform to improve the
business. Machine learning and digital footprint both are very common big data
technologies that use for analysing consumers data. It is high velocity, high variety and
high volume that need latest forms of processing to increase decision making and
process optimization and a new veracity has been added by some communities to
describe the concept of big data technology.
Characteristics of big data
There are many characteristics of big data technique which are described below:
 Volume
 Velocity
 Variability
 Veracity
 Variety
 Complexity
Volume
The amount of data which is produced is most significant in this context and the
main role of this step is that it identifies the value and potential of the data. It is also
called the size of the data and organization can analysis a huge quantity of data at a
time.
Variety
The next step of big data technology is that variety that defines the category of
data where information divided into subparts. This provides a platform to help the
consumers that are very closely analysing the data and are linked with it.
Velocity
The term velocity is defined as the speed of generation of data or in another
word it identifies how debauched the data is shaped and handled to meet the
requirement of the consumers (Gantz, & Reinsel, 2012).

BIG DATA
6
Variability
This is in one of the serious factors that can be an issue for those organizations
use big data for analysis purpose. It is defined as the inconsistency that can be indicated
through the data or information at eras thus hindering the procedure of existence bright
to control and achieve the data more effectively.
Veracity
The value of the statistics being apprehended can vary from person to person
and it is observed that the precision of study completely depended upon the veracity of
the source facts.
Complexity
The management of facts is very complex and crucial especially for a huge
quantity of data and information comes from numbers of resources. These consumers’
data require be connecting and correlating in order to grasp the data which is
hypothetical to be taken through these numbers sets. This condition is called the
complexity of the information set.
There are main 5C systems used in the cyber-physical systems which are described
below:
 Connections (networks and sensors)
 Content (memory and model)
 Community (teamwork)
 Customization (value and personalization)
 Cloud (calculating and data on request)
Technologies
There are numbers of technologies involve in big data, for example,
crowdsourcing, integration, machine learning, data fusion, A/B testing, signal
processing, visualization and time series algorithm. The massively parallel processing is
an advanced technology that has the capability to handle and store a petabyte of data
(Hashem, et al., 2017). DARPA technology data analysis approach pursues the essential
construction of enormous data circles and in the year 2008 ayas has launched this
technology and now most of the organizations are using this for analysis of data.

BIG DATA
7
Cloud computing is a modern technology which is used to store and monitor the
consumer's data and it is also used as a backup plan by which the problem of cyber-
attacks. This kind of technique generally used in big data to improve the scalability, for
example, organisations can add more than 15 computers in several clicks (Kitchin,
2014).
Hadoop is an approach which is used to distribute a large amount of data from
one location to another. The main role of this process is that it breaks a huge chunk into
very small pieces on different computer networks. HDFS is the part of Hadoop system
that has the ability to simulate multiples files at a time and it can be used to address the
large data files.
Apache Spark is a type of technique which is used for parallel data processing
that makes a real-time analytics. The main advantage of this process is that
organizations can easily find the nature and behaviours of consumers by which they can
improve their productivity and performance. There are many advantages to this
technology first of all; it gives a comprehensive outline to manage big data technology
with a variety of database. It also enables Hadoop clusters to simulate 100 times faster
in memory and 10 times faster when it runs on disk. It can be written in any language
like java, python, and scale and it can be used to balance the consumer’s data. Therefore,
many organizations use this technology for analysis and manage the user’s data and it is
observed that in the last two years the use of apache spark has increased by 30%.
Applications of big data
Big data technology has enhanced the demand for information security
specialists in many organizations such as IBM, EMC, HP, AG, Oracle Corporation, and
dell. All these business communities have spent around $15 billion on software to
manage and control the human data or information (Labrinidis, & Jagadish, 2012). In
the year 2011, this organization’s spent around %100 billion and they were increased
their share of more than 10%. It is investigated that 1/3rd of the data is deposited in the
procedure of alphanumeric transcript which is a very most common method to manage
huge data. There are lot of applications of big data technology which are described
below:

BIG DATA
8
Government
This is one of the best applications of a big data process that allows a way to
access and control the consumer’s data in terms of cost, innovation and productivity.
The recent event shows that this technology does not come without their flaws and data
analysis need numbers of the central government to work in collaboration and produce
new process. In the year 2012, the United States developed big data investigation and
development to address the issues faced by the government during the data managing
process. Initially, they composed big data into 6 departments and produce around 84
big data programmes (LaValle, et al., 2011).
International development
The actual use of info technology indicates that big data process can make a
significant contribution to managing the consumer's data. It also provides the gainful
chances to enhance executive approach in many growth sectors, for example,
employment, economic productivity, resource management, and health care (Lazer,
Kennedy, King, & Vespignani, 2014). The main challenge faced by the big data is security
because hackers use malicious software to access consumer’s computer networks.
Manufacturing
According to a recent investigation big data process provides a way to improve
the performance and efficiency in manufacturing. It delivers an infrastructure for
transparency in the field of manufacture industries which is the capability to avoid the
problem of uncertainty and inconsistency in the performance (McAfee, et al., 2012).
Cyber-physical models
Modern PHM process implemented data analytics code which can perform more
effectively when large data or information involves in machine learning. To control and
manage the machine life cycle a new approach is needed for which information and
communication technology developed big data technique that controls data more
effectively (Madden, 2012). In private sectors this technology plays a significant role, for
example, Wal-Mart organization handles around 1 million consumers every hour which
is done by using big data technology. Numbers of retail banks use big data technology
because they communicate with many users in a day for which they analysis data by
using this technology.

BIG DATA
9
Advantages of big data
 It helps to understand and target consumers and can be used to optimize
business process
 It provides a way to improve science and research
 More effective approach to analysis data
 Take very less time
 One platform carries numbers of information (Murdoch, & Detsky, 2013).
Disadvantages of big data
There are many drawbacks of this technology which are described below:
 Very costly process
 Unstructured process for a large amount of data
 Very less secure and private
 Can increase social stratification
 It is not useful for short networks
Challenges faced by big data
There are many challenges faced by these technologies which are described
below:
Dealing with data growth
The most common challenge occurs in big data process is storing and analysis
data or information. It is estimated by the digital universe that the quantity of data
stored in the information technology system is doubling every 2 years (Provost, &
Fawcett, 2013). In which the rate of unstructured data is very high that means it does
not reside in the computer database. It is very difficult to store and control audio, video,
and other unstructured information and this problem are increasing very fast because
many organization dealing with huge unstructured data.
Generating insights in a timely manner
Many organizations use this technology to achieve their business goals but they
also used to store their big data which is also a crucial problem. This technology was
developed to control and manage big data not to store human data for which

BIG DATA
10
information and communication technology developed cloud computing to store
business data (Raghupathi, & Raghupathi, 2014). Every business industries want to
improve their decision-making process for which they adopt third-party servers due to
which the problem of insight occurs.
Integrating disparate data resources
Data integration is also a very common issue faced by big data technology and it
is observed that big data comes from various places such as social media, emails, and
enterprise and employees documents. To combine all these data and produce a new
report that can increase the difficulty in big data for which vendors provided many
software’s and tools but many organizations say that they have not resolved the issue of
data integration yet.
Security and privacy-related issue
Security of information is one of the crucial problems for every technology and
big data is very less secure. In the last few years the issue of security is growing very
fast because hackers use malicious software to hack human data (Swan, 2013). They
produce traffic and unwanted signals and transfer to the organization computer
network by which they easily enter into computer devices and block user’s personal
information (Wu, Zhu, Wu, & Ding, 2014). There are various kinds of security threats
occur in big data, for example, denial of service attack, malware, and ransomware and
wanna cry attack and data breach. In the last two years around 40% of organizations
were suffered from the issue of ransom ware and DOS attack.
Recommendation
It is recommended that the issue of security can be resolved by making security
strategies and policies and information and technology developed advanced analysis
approach like apache spark that can be adopted. Generally, hackers share fraud emails
and unauthentic signals to employees accounts that are developed by malicious
software and by which they encrypt their private details (Zikopoulos, & Eaton,
2011). To avoid such kinds of problems organization can adopt a firewall, encryption,
and antivirus and cryptography technology. Encryption is a very popular technology
that converts data into the form of code which cannot be easily read by hackers and

BIG DATA
11
transferred from one place to another. By using this technology people can address the
problem of a data breach in big data.
Conclusion
Big data is a very popular innovation in the field of information technology which
is used by many organizations to control and monitor their large data. By using this
approach people can analyse and handle huge data like audio, video, pictures and other
unstructured data. This report is completely based on the summary of big data and with
the help of this report readers can enhance their knowledge in the area of technology.
There are main five characteristics of big data such as volume, variety, veracity, velocity
and variability all these are described in this report. This research report explained the
fundamental concept of big data technology with their advantages and disadvantages
and also evaluated the challenges faced by this technique. People should ensure that
they use only authentic servers and they can adopt security plans to avoid security-
related issues such as firewall, encryption and biometric recognition.

BIG DATA
12
References
Chen, C. P., & Zhang, C. Y. (2014). Data-intensive applications, challenges, techniques
and technologies: A survey on Big Data. Information Sciences, 275, 314-347.
Chen, H., Chiang, R. H., & Storey, V. C. (2012). Business intelligence and analytics: from
big data to a big impact. MIS Quarterly, 18(2), 1165-1188.
Chen, M., Mao, S., & Liu, Y. (2014). Big data: A survey. Mobile networks and
applications, 19(2), 171-209.
Fan, W., & Bifet, A. (2013). Mining big data: current status, and forecast to the
future. ACM sIGKDD Explorations Newsletter, 14(2), 1-5.
Gandomi, A., & Haider, M. (2015). Beyond the hype: Big data concepts, methods, and
analytics. International Journal of Information Management, 35(2), 137-144.
Gantz, J., & Reinsel, D. (2012). The digital universe in 2020: Big data, bigger digital
shadows, and biggest growth in the far east. IDC iView: IDC Analyze the
future, 2007(2012), 1-16.
Hashem, I. A. T., Yaqoob, I., Anuar, N. B., Mokhtar, S., Gani, A., & Khan, S. U. (2015). The
rise of “big data” on cloud computing: Review and open research
issues. Information systems, 47, 98-115.
Kitchin, R. (2014). The real-time city? Big data and smart urbanism. GeoJournal, 79(1),
1-14.
Labrinidis, A., & Jagadish, H. V. (2012). Challenges and opportunities with big
data. Proceedings of the VLDB Endowment, 5(12), 2032-2033.
LaValle, S., Lesser, E., Shockley, R., Hopkins, M. S., & Kruschwitz, N. (2011). Big data,
analytics and the path from insights to value. MIT Sloan management
review, 52(2), 21.
Lazer, D., Kennedy, R., King, G., & Vespignani, A. (2014). The parable of Google Flu: traps
in big data analysis. Science, 343(6176), 1203-1205.

BIG DATA
13
Madden, S. (2012). From databases to big data. IEEE Internet Computing, 4(3), 4-6.
McAfee, A., Brynjolfsson, E., Davenport, T. H., Patil, D. J., & Barton, D. (2012). Big data:
the management revolution. Harvard business review, 90(10), 60-68.
Murdoch, T. B., & Detsky, A. S. (2013). The inevitable application of big data to health
care. Jama, 309(13), 1351-1352.
Provost, F., & Fawcett, T. (2013). Data science and its relationship to big data and data-
driven decision making. Big data, 1(1), 51-59.
Raghupathi, W., & Raghupathi, V. (2014). Big data analytics in healthcare: promise and
potential. Health information science and systems, 2(1), 3.
Swan, M. (2013). The quantified self: Fundamental disruption in big data science and
biological discovery. Big data, 1(2), 85-99.
Wu, X., Zhu, X., Wu, G. Q., & Ding, W. (2014). Data mining with big data. IEEE transactions
on knowledge and data engineering, 26(1), 97-107.
Zikopoulos, P., & Eaton, C. (2011). Understanding big data: Analytics for enterprise-class
Hadoop and streaming data. McGraw-Hill Osborne Media.