Critical analysis of Big Data challenges

Abstract
Big data have become increasingly popular among businesses. In the report, the big data
technology has been explored from different perspectives. Following the discussion, the impacts
of big data on businesses have been explored. Even though big data has various benefits to
businesses, it also have some serious issues to consider.
1

Table of Contents
Abstract.......................................................................................................................................................1
Introduction.................................................................................................................................................3
What is Big Data?........................................................................................................................................3
3V’s of Big Data......................................................................................................................................3
Big Data Analytics..................................................................................................................................4
Conventional BI tools Vs. Big Data Analytics........................................................................................4
Data Warehouses and ETL......................................................................................................................5
Big Data Ecosystem................................................................................................................................6
Business Impacts of Big Data......................................................................................................................7
Positive Impacts......................................................................................................................................7
Negative Impacts.....................................................................................................................................8
Conclusion...................................................................................................................................................8
References...................................................................................................................................................9
2

Introduction
The aim of this paper is to discuss big data and its business impacts. A high volume of data flows
through the Internet. Different organizations capture and analyze the data to find out meaningful
information. This huge amount of data comes from various sources like social media, businesses,
and sensor networks, IoT or the Internet of Things and so on. The streaming data sets are highly
important for enterprises and organizations. It helps them to understand their market, customers,
products, and the level of competition in the market. As the digital economy has made the
business world more competitive so all businesses want more information for decision making
and to win competitive advantages (John Walker, 2014).
In the following sections of the report, there is a detailed discussion of the big data technologies
and analytics and how it impacts businesses in decision making.
What is Big Data?
In computing parlance, big data refers to the voluminous data sets from heterogeneous data
sources. Such data can be structured, unstructured, or semi-structured. Businesses use bog data
daily for their business processes. Even though the volume of big data is an important
characteristic, there are other characteristics that define it. Businesses are more interested in the
value of information that can be taken out from the processing and analysis of such high volumes
of data in real-time (Wamba, Akter, Edwards, Chopin, & Gnanzou, 2015).
Traditional database management systems are no longer sufficient to help a business in data
processing and analysis for decision making. Such database management systems are suitable for
storing and processing structured data. Big data comprises mostly unstructured and semi-
structured data. Other than volume, the complexity of big data stems from the characteristics
such as high velocity, and variety. When a high volume of data is generated in real-time by
heterogeneous sources and in different formats, then it becomes a difficult task to process and
analyzes the data to find out reliable and meaningful information from that data. Hence,
businesses face challenges in every step of big data processing and analysis. Some of the
challenges are related to capturing, storing, analyzing, searching, sharing, transferring, querying,
visualizing, updating, and retrieving information and to ensure the reliability, protection, and
privacy of data (Wamba et al., 2015).
3V’s of Big Data
The three main characteristics of big data are termed as 3V's of big data. The characteristics are
Volume, Velocity, and Variety. There are other attributes too. For example, value, and veracity.
The volume of big data refers to the terabytes of data being generated and stored. The volume of
big data is very important as it plays an important role in determining the value of the data.
Structured data are usually the data bound by some predefined data structures (Wamba et al.,
2015). For example, data collected from relational databases. Semi-structured data may have
some structure in it but usually, don't have any data model associated with it. And unstructured
data don't have any predefined data model or data structure. Big data can have different types of
nature. For example, there may be structured, unstructured or semi-structured data. At the same
time, there can be data of different formats like text, audio, video, XML etc. The velocity of big
3

data refers to the speed at which data gets generated and processed. Very often, big data is
available in real-time or the value of big data can be obtained only if the data is processed in
real-time. Variability is another characteristic of big data. The inconsistency found in big data
collected from different sources is known as the variability of big data. Variability affects the
efficiency of the process of handling and managing big data.
Data quality is important. Big data captured from different sources do not have the same data
quality. The reliability and accuracy of results of various big data analytics depend on the quality
of big data (John Walker, 2014).
Big Data Analytics
Businesses use various bog data analytics tools to examine large and varied datasets. These tools
help to uncover various business insights, patterns, correlations etc. Businesses use that outcome
to understand their customers, preferences of the customers, market trends, and forecasts and so
on. Big data analytics give the businesses the power to make informed business decisions
(Loebbecke & Picot, 2015).
Big data analytics use specialized systems and software. Businesses can leverage the power of
big data to generate more revenue, exploring new possibilities, running more effective marketing
campaigns, delivering better customer services, improving operational efficiency and by winning
competitive advantages over their rivals (George, Haas, & Pentland, 2014).
Statistician, predictive modelers, data scientists, and other big data professionals work on the
analysis of big data. The growing volume of unstructured data cannot be processed efficiently by
traditional business intelligence or BI tools. Big data analytics tools are capable of working on
semi-structured, unstructured, and structured data. Some examples of semi-structured and
unstructured data are logs from various web servers, clickstream data, content from social media,
responses collected from the various online survey, mobile call and message records, data
streams from IoT sensors and so on (Loebbecke & Picot, 2015).
Conventional BI tools Vs. Big Data Analytics
Conventional business intelligence tools allow businesses to run various queries on their data. It
helps businesses to improve their business operations and performance of the supply chain
management. On the other hand, big data analytics tools and techniques offer more advanced
data analysis processes using complex analytics tools. For example, predictive models based big
data analytics tools can help to understand forecasts, statistical methods help to understand the
scenarios of various what-if analysis (George et al., 2014).
The growth of big data is enormous. It is being generated and collected continuously from
various information sensing IoT systems and devices. Some examples are aerial remote sensing,
mobile devices, software logs, RFID or Radio Frequency Identification-based systems and so on
(Schmarzo, 2013). Conventional BI tools are often integrated with relational database
management systems and various statistical analysis tools that can run on stored data. On the
other hand, big data usually requires parallel processing capabilities to work on massive data sets
stored across tens to thousands of servers connected over a network or the Internet. To realize the
value of big data, users need to have high performing big data analytics tools.
4

The conventional BI based approach is more structured and repeatable in nature. Business users
use the tools for determining what kind of questions must be asked. On the other hand, big data
analytics are used to deliver a platform to the business and information technology professionals
so that the users can find useful insights from data (Schmarzo, 2013). BI tools are usually limited
to the structured data. On the other hand, big data analytics tools can handle structured, semi-
structured, and unstructured data. Some of the BI-related technologies like in-memory databases,
column-based databases, and data warehouses are capable of working with big data analytics
also. However, when it comes to the velocity of big data, the frequency of generating data or the
frequency of delivery of the results, data warehouses may not perform well with high-velocity
data streams. The reason is, data warehouses work mostly with batch-oriented data models and
data flows. On the contrary, big data needs solutions that can handle real-time or quasi-real-time
data streams and more agile processes to work on the data streams.
BI and big data are not a replacement for one another. Rather a business can use both efficiently
to improve their business processes and decision-making activities.
Data Warehouses and ETL
Big data requires data warehouses that are fault tolerant, and can handle time-sensitive data
streams and can work well with various big data analytics. The primary goal is to allow
businesses in making decisions quick (Schmarzo, 2013).
Usually, traditional data warehouses, use temporary locations to collect data from different
sources. Then it loads the collected data into the data warehouse. The temporary areas are called
staging areas. This solution is not suitable for big data as it adds a delay in processing data and
not suitable for real-time data streams. So, the staging areas are replaced using some suitable big
data file system. It allows new data sources having unstructured and semi-structured data to be
loaded on the data warehouse faster. Metrics are created for loading such data on the data
warehouse. Unstructured data are more important to find out new ideas and business insights. It
also provides detailed explanations to various business queries.
ETL stands for Extract- Transform- Load. ETL processes are used by businesses to extract data
from heterogeneous sources, transforming the data so that it becomes suitable for further analysis
and then loading the same to desired business intelligence or big data analytics platforms. Data
may be stored in various data marts during the extraction phase. The MapReduce platform works
well for the transformation phase in a big data analysis process. It provides a parallel and
scalable solution for transforming data. The outcome of the transformation process provides data
in some consumable format so that the big data analytics tool can consume and process the data
faster. Rather than using ETL scripts, MapReduce scripts are used for ETL processes for big data
processing.
In traditional data warehouses and BI tools based applications, the whole execution part takes
place on different servers. It requires to move data from one server to another and takes longer
time. On the other hand, MapReduce scripts allow data warehouses to do the execution part
where the data resides. It cuts down the execution time and saves bandwidth by reducing the
unnecessary data flows between servers.
5

Big data technologies like Hadoop allows enforcing different methods to make the queries more
meaningful and find more insights from data sets. MapReduce follows the batch-processing
method and direct querying method on the installed big data file system. It also offers in-memory
stream processing to support real-time query processing for big data.
The primary objective of a data warehouse is to provide consolidated information to businesses
so that they can use the information for the business processes, create reports and so on. On the
other hand, the primary objective of an ETL tool is to transform data from one format to another
as required by the data warehouse and other analytics tools. An intermediate location is used for
the ETL transformation process before loading the data into a data warehouse (Kwon, Lee, &
Shin, 2014).
The three primary functions of an ETL process are,
 Extracting data from data sources.
 Converting the format of the extracted data sets into a suitable format according to the
targeted database or data warehouse. The rules of the transformation are set by the users.
 Writing down the data into the target database or the data warehouse.
With the emergence of big data technologies, ETL processes also have improved. It can support
and integrate traditional data warehouses with other operational and transactional systems being
used by a business, cloud platforms, Hadoop platforms, various MDM hubs and BI platforms.
Hence, ETL is now capable of extracting, transforming, and loading data for big data solutions
(Kwon et al., 2014).
Various software tools are used during various intermediate phases to clean, profiling and to
audit data. ETL tools can be extended further to add various data mapping, data quality checking,
data cleaning tools. Advanced ETL supports massive parallel data reading and writing activities
also.
Big Data Ecosystem
Big data ecosystem consists of various techniques to analyze big data, big data visualization, and
other big data-related technologies. For example, A/B testing, natural language processing, and
machine learning based applications are majorly used for analysis of big data. Whereas, business
intelligence tools, cloud computing platforms, and other database technologies are used to
implement a big data solution. Visualization is an important part of any big data-based solution.
It helps to deliver results to users using visual content like graphs, charts and so on (Kaisler,
Armour, Espinosa, & Money, 2013).
Tensor-based computations are suitable for big data processing. A tensor is a representation of
multidimensional big data. An example of tensor-based processing is multi-linear subspace
learning. Other technologies like MPP or massively parallel processing databases, data mining,
search-based applications, distributed databases etc. are used for various big data processing
tasks.
6

Cloud computing based solutions and advanced storage solutions also help to improve the
parallel processing tasks. For example, big data analytics use shared storage architectures like
NAS or Network Attached Storage, SAN or Storage Area Network etc. (Kaisler et al., 2013).
Business Impacts of Big Data
Businesses are using various analytics and technologies to gain the advantages of big data.
However, there are both positive and negative impacts of big data on businesses. Some of those
impacts have been presented below.
Positive Impacts
Big data can bring various positive impacts to a business. Some of those are,
 Big data helps business in decision making. It makes the whole decision-making process
more data-driven and agile. The business insights and information found from various big
data analytics implemented for a business can help in fine-tuning the business strategies
of a business.
 Big data supports other cutting-edge technologies like cloud computing, machine
learning, natural language processing etc. Businesses can develop more intuitive and
user-friendly customer relationship management and feedback systems (Sharma, Mithas,
& Kankanhalli, 2014).
 Predictive analysis and other big data based analytic tools can help to identify risks
related to market, competition, and products or services early and can provide more
information so that a business can take a well-informed strategic move to mitigate the
risks.
 Big data technologies can also help to improve the operational efficiency of business
processes. The ETL processes and data warehouses allow to move data at various
intermediate storage areas and to process data there. It reduces the processing time and
improves the business operations. The integration of data warehouse and other big data
technologies also allows focussing on important data faster (Erevelles, Fukawa, &
Swayne, 2016).
 Big data tools like Hadoop, other open source solutions, and various cloud-based big data
analytics also help in saving the operational cost of a business. These tools are capable of
working efficiently for larger data sets (Sharma et al., 2014).
 In-memory processing and in-memory analytics tools help to identify new data sources
quickly and can add that in data processing activities. Thus, the businesses can always
make decisions based on the current data available.
 Businesses can explore the results found from various customer-centric big data analytics
tools. The results from the applications of these tools can help to create and deliver
customer-focused products and services. Thus, big data can help businesses to develop
new products that can meet the preferences and demands of the customers (Erevelles et
al., 2016).
 Analysis of big data helps to understand various conditions of targeted markets.
Businesses can use the information to go ahead of the competitors.
7

 Sentiment analysis of the customers can be done using suitable big data analytics tools.
Online reputation of a business can be maintained using various big data tools and
analysis processes. Further, they can improve and monitor their online presence
(Erevelles et al., 2016).
Negative Impacts
There are some negative impacts of big data on a business. Some of the impacts are,
 Big data consists of GPS information, social media activities, data collected by IoT
devices and sensors and so on. Most of these data are user-centric and may contain
private information. Businesses collect these digital traces to deliver more personalized
services and products. However, there are security and privacy issues (Kaisler et al.,
2013). Most of the users do not share the data intentionally for a business. For example,
the Facebook posts of a user may not be intended for a business. But the business may
find the information and consider the person to be a prospective customer. Then they can
send customized offers to the user that the user is not willing to receive (Kshetri, 2014).
 The storage, extraction, and processing of sensitive personal information of individual
increases the risks of unauthorized access to data, attacks from the cyber world, and so
on (Kaisler et al., 2013).
 While extracting data from heterogeneous sources, it may collect low-quality data
having reliability issues, or personally identifiable information or intellectual properties.
In that cases, the results produced from the analysis of these data may be unusable or not
reliable enough to make decisions. It may lead to legal consequences also (Kaisler et al.,
2013).
Conclusion
In this report, big data technologies and the impacts of big data on businesses have been covered
briefly. Big data has become common for businesses. Businesses from different industries are
implementing various solutions based on big data and analytics to win competitive advantages,
to accelerate business processes, to make well-informed business decisions and so on. Big data
also helps businesses to decide new strategies and to win the competition in the market. The
readers can understand the context of big data on businesses and about the technologies.
8

References
Erevelles, S., Fukawa, N., & Swayne, L. (2016). Big Data consumer analytics and the
transformation of marketing. Journal of Business Research, 69(2), 897-904.
George, G., Haas, M. R., & Pentland, A. (2014). Big data and management. Academy of
Management Journal, 57(2), 321-326.
John Walker, S. (2014). Big data: A revolution that will transform how we live, work, and think.
In: Taylor & Francis.
Kaisler, S., Armour, F., Espinosa, J. A., & Money, W. (2013). Big data: Issues and challenges
moving forward. Paper presented at the System Sciences (HICSS), 2013 46th Hawaii
international conference on.
Kshetri, N. (2014). Big data׳ s impact on privacy, security and consumer welfare.
Telecommunications Policy, 38(11), 1134-1145.
Kwon, O., Lee, N., & Shin, B. (2014). Data quality management, data usage experience and
acquisition intention of big data analytics. International Journal of Information
Management, 34(3), 387-394.
Loebbecke, C., & Picot, A. (2015). Reflections on societal and business model transformation
arising from digitization and big data analytics: A research agenda. The Journal of
Strategic Information Systems, 24(3), 149-157.
Schmarzo, B. (2013). Big Data: Understanding how data power big business: John Wiley &
Sons.
Sharma, R., Mithas, S., & Kankanhalli, A. (2014). Transforming decision-making processes: a
research agenda for understanding the impact of business analytics on organizations.
European Journal of Information Systems, 23(4), 433-441.
Wamba, S. F., Akter, S., Edwards, A., Chopin, G., & Gnanzou, D. (2015). How ‘big data can
make big impact: Findings from a systematic review and a longitudinal case study.
International Journal of Production Economics, 165, 234-246.
9