Big Data Systems Enterprise Deployment, Integration, Scalability and Security Issues

This paper discusses the challenges and solutions for deploying, integrating, scaling and securing big data systems in enterprises. It focuses on SliceDice's hybrid model and the use of Hadoop and NoSQL. The paper also covers data visualization and security suggestions. The subject is Big Data Systems and the course code is not mentioned. The paper is submitted to a university and the names of the authors and tutor are not provided.

BIG DATA SYSTEMS ENTERPRISE DEPLOYMENT, INTEGRATION, SCALABILITY
AND SECURITY ISSUES
Big data systems enterprise deployment, integration, scalability and security issues
University's Name
Submitted by Names:
Tutor:
Date of submission

BIG DATA SYSTEMS ENTERPRISE DEPLOYMENT, INTEGRATION, SCALABILITY
AND SECURITY ISSUES 2
Introduction to Company
The existence of systems used to host databases is in line with reporting as well as the
sharing of the information in organisations. Slicing Dice is offers data warehouse and analytic
database extensive services and will be the focus of this paper. With the necessity to store, load,
query as well as visualize data among engineers, management of composite base of data is
always mind-boggling. Therefore, SliceDice came in to suffice the gap by allowing companies to
simply put in and question a wide array of real-time, historical, or time-series data eliminating
the need for server management. The aforesaid follows a contemporary increased need by users
for an instant, effective as well as secure modes. An absolute amount of information amongst the
enterprise, retrieving data in an efficient manner mandate for aligned attempts between existent
systems.
Since the emergence of Enterprise Data Warehouses about 30 years ago, it not only
became popular but an essential facet for SliceDice operations of business intelligence.
Therefore, the technology is credited for creating a real impact on the general accomplishments
of the organization. In essence, EDW, as implemented in the company, is a database that lays in
all data related to the company. Similarly, it provides information to all cleared users, as well as
offers support in terms of thorough analytical thinking. Through a detailed and approachable
report system, SliceDice implemented warehouse can be used differently by third-party
organizations. However, due to the changing needs of the company, consideration is made to
integrate big data through Hadoop into the existing EDW architecture.

BIG DATA SYSTEMS ENTERPRISE DEPLOYMENT, INTEGRATION, SCALABILITY
AND SECURITY ISSUES 3
Data Flow
The move looks forward to creating a relationship between the company's data warehouse
with big data, hence, creating a hybrid model. The latter will ardently emerge as extremely
structured, and optimized functional data. The latter is intended to be left at the strictly
manipulated data warehouse whereas the Hadoop-based infrastructure will have real-time control
all data that is subject to alteration as well as extremely distributed data. SliceDice comprehends
that it is posed with a business necessity for combining conventional data warehouses, the
historic business sources of data as well as a lesser integrated source of big data. Therefore, it
resolute to the hybrid model since it will support orthodox and sources of big data in the bid to
meet their daily goals.
Figure 1 Big Five Vs of Big Data
Retrieved from:

BIG DATA SYSTEMS ENTERPRISE DEPLOYMENT, INTEGRATION, SCALABILITY
AND SECURITY ISSUES 4
SliceDice is designed to provide data querying assistance to all companies implementing
a web presence as to be mentioned. First, it seeks to bolster clients' financial health so that the
relevant executives including CFO, CEO and CIO cannot be extravagant. Limitless as well as
free storage of data are offered at different price models which are determined by the volume of
inserted data. Similarly, it offers a simple and sheer price which allows clients to select between
cheap price models including wage-per-column or wage-per-gigabyte. The platform provides an
entirely free test for databases through a live demo runnable through a third-party website.
Lastly, SliceDice can easily liberate a client from committing to its website by simply
terminating an account as well as instant exporting of the data.
Reckoning the engineers need for peace of mind, SliceDice offers handy features that
allow them to focus on insight giving and creativity. First is the entire lack of server management
as a solution levying of all infrastructure worries. Secondly, a high presence and backup are
enabled through a built-in redundancy having three independent data centres. They provide
independent and simultaneous configuration allowing for continuous servicing and support
through clients' insertion and query of data. Thirdly, SliceDice poses nil sustenance requirements
since one only needs to insert their data and query through SQL or the sites REST API.
Therefore, it offers trivial needs for tuning or tweaking databases. A lucid increase in space is
also supported without the need of any concern. Set up client libraries are provided hence,
making it easy for clients to integrate SliceDice with other frameworks. Creation of native users
made from popular languages including Java, Python, JavaScript, PHP, Arduino, Ruby and .NET
(Sharif and Cooney 2015).
Another reason for structuring the hybrid data warehouse into SliceDice includes the
need for an assured backward compatibility. The move ensures that any use of the API will

BIG DATA SYSTEMS ENTERPRISE DEPLOYMENT, INTEGRATION, SCALABILITY
AND SECURITY ISSUES 5
function despite the undoing by companies to upgrade code. Being an entirely servicing
framework, SliceDice doesn't delegate responsibilities which require cooperating with numerous
companies so as to link dissimilar technologies in the bid to achieve objectives. Provision of
export transform load (ETL), data warehouse as well as visualization capacities. Similarly,
without incurring additional costs data can easily be exported from SlicingDice at any time.
Figure 2 Enterprise data architecture using combined EDW/Big data environment
Retrieved from: https://www.infoq.com/articles/HadoopSecurityModel

BIG DATA SYSTEMS ENTERPRISE DEPLOYMENT, INTEGRATION, SCALABILITY
AND SECURITY ISSUES 6
Suitable Platform for Big Data
The intended use of big data also referred to as NoSQL or Hadoop and boosted analytics
is caused by the equivalent advent of the web, social networking, and mobile gadgets. The latter
have been causal factors to an essential alteration of data's nature. Big data, unlike the
conventional corporate data, is extremely distributed, lightly structured and increasingly huge in
volume. The method offers a number of ways to process and analyze big data despite many
having similar characteristics. Capitalizing on good hardware will enable scaling out,
concurrency in processes; employment of incomparable data storage capacities to litigate both
unstructured and semi-structured data. Similarly, the method will apply boosted analytics as well
as data visualization technology to big data in the bid to pose insights to clients. The data
approaches intended to transform SliceDice company business analytics as well as data
management will include Apache Hadoop and NoSQL.
The coexistent strategy of Big Data and EDW on SliceDice architecture will involve
several steps led by determining whether and where Hadoop can fit in the existing data
warehouse. Being a family of merchandise having numerous capabilities, Hadoop can contribute
to a couple of areas in EDW. It is most obliging as a data framework for catching and laying in
big data across a diverse data warehouse surrounding as well as appending to the process of data
analysis on other platforms (Demchenko 2014). The mentioned approach will enable the firm to
secure its investment in EDW framework as well as stretch into accommodating the Big data
surrounding. Following the integration, Hadoop will conform to a number of roles within the
EDW architecture as mentioned in the following part of this paper.

BIG DATA SYSTEMS ENTERPRISE DEPLOYMENT, INTEGRATION, SCALABILITY
AND SECURITY ISSUES 7
The first role would involve, staging the data as processed by the EDW so as to develop
root data for particular functions including reporting, analytics and debasing into databases. The
latter roles will be performed by native ETL techniques. Hadoop is expected to enable the
company to deploy a highly scalable and frugal ETL framework. For instance, a reckoned ETL
use case involves offloading hefty translation into Hadoop from the data warehouse, hence "T" in
ETL (Smith 2018). The solution behind it was derived because of the organisation's entirety in
struggling to scale its conventional ETL architecture. Particularly, numerous data consolidation
platforms campaigned for the transition towards the data warehouse. The latter is attributable to
why contemporary platforms of data integration in EDW hike to around 80% of database volume
as well as resources. The result is untenable expenditures, unending bids for maintenance, and
inadequate response to users' query. Therefore, the move to shift T to Hadoop, SliceDice is able
to tremendously cut costs as well as leverage database volume and resources, hence, a quick
execution to users' query.
Archiving data had conventionally bored only three alternatives with regards to filling
away data. The methods included either leaving it amongst a comparative database, moving to an
offline program library for storage, or casting. Implementation of Hadoop's measurability, as
well as minimum costs, modify the organizations' ability to store data entirely in a promptly
approachable online surrounding. Next step involves an outline of flexibility since relational
database management systems implemented data warehouse implementations. The latter is thus
amply fitted to lay in extremely structured data, derived from databases, Enterprise Resource
Systems (ERP), and Customer Relationship Management systems (CRM). However, Hadoop
complementarily offers faster and easy ingestion of any data format letting in developing
schemas that enable website variable tests as well as no schemas including multimedia files. The

BIG DATA SYSTEMS ENTERPRISE DEPLOYMENT, INTEGRATION, SCALABILITY
AND SECURITY ISSUES 8
implemented Hadoop architecture will enhance the organization's flexibility to process using the
Hadoop NoSQL approach.
The innate system will be used to manipulated contemporary data types as well as enable
sequential litigation. The latter is expected as a potent method in conducting use cases of time-
series analysis as well as identification of gaps. As mentioned earlier, the architecture backs up a
number of programming languages hence source to more capacity than SQL. The method used to
augment and bolster the organisation's EDW through Hadoop clustering involves the following
steps. Extending storage of succinct structured data from OLTP as well as behind office
frameworks to the EDW. Secondly, storage of unstructured data in Hadoop/NoSQL since they
barely fit into the tables in a nice way. Therefore, major communications with the client from
call logs, client feedback, GPS placement, pictures, messages, tweets and emails will be laid in
Hadoop. Lastly, correlating data in EDW to the Hadoop clustered data will seek to derive more
adept perceptivity concerning clients, merchandise, and equipment. SliceDice will thus be able to
execute ad-hoc analytics as well as bunching up, directing exemplars versus coordinated data in
Hadoop; which is extremely intensive in computation.
Security Suggestions
Apache Hadoop was originally designed without a security model, however, its
popularity has led to scrutiny among professionals. A number of threat were reported on Hadoop
cluster where an attacker could devise code as a pretext of a Hadoop user. However, the recent
growth of Hadoop security in the marketplace will see the SliceDice integration of vendor
released enhancement using Intel's secure Hadoop distribution (Haber 2018). Similarly, Apache
Accumulo will be used to provide techniques for appending security in Hadoop.

BIG DATA SYSTEMS ENTERPRISE DEPLOYMENT, INTEGRATION, SCALABILITY
AND SECURITY ISSUES 9
Challenges in Security
As in the .20.20x distribution security concerns for customer data will be accomplished
through the following:
 Bolstering HDFS file licenses as an access control to files through the (ACL) of our
clients and groups.
 Commission items for accompanying hallmarking checks will be used amid clients and
services proceeding their first authentication, hence minimizing all potential overheads.
 Job items to bolster authorization of activities will be created using a JobTracker and
drawn to TaskTrackers in the bid to ascertain that activities only perform on their
assignments.
 Encoding network connections that use SASL with Quality of Protection (QoP) to ensure
confidentiality at the network level. MapReduce, as well as web consoles, will be
configured to use SSL.
 HDFS file transfer is configured for encryption.
Nevertheless, a number of challenges pose as serious security concerns for Hadoop as
outlined by Boris and Alexey. First includes how to bolster authentication for clients using web
consoles. Secondly is determining how to inhibit all rascal services in a pretext of a real Hadoop
service. Thirdly, bolstering the access control to stored data in line with existent policies as well
as client certificates. A fourth challenge faced by SliceDice involves the implementation of
Attribute-Based Access Control (ABAC) otherwise known as Role-Based Access Control
(RBAC) (Moura and Serrão 2018).

BIG DATA SYSTEMS ENTERPRISE DEPLOYMENT, INTEGRATION, SCALABILITY
AND SECURITY ISSUES 10
Data Visualization
A HortonWorks developed approach will be used for data management policy in Apache
Hadoop. The governance of data and functioning to make protection rapport meeting needs for
the enterprise include assortment-based policy, placement-based policy, data expiration-based
policy and inhibition-based policy (Hortonworks 2018). The company will use DataMeer for
Apache Hadoop for visualizing trends of data according to various analysis. The software was
chosen since it was designed for actively visualizing data, integrate as well as prepare numerous
data types. Through the end to end system Hadoop is made simpler for the company's audiences.
DataMeer contains a handy feature that makes it compatible with Hadoop as well as combining a
self-service data incorporation tool.
References
Demchenko, Y., De Laat, C. and Membrey, P., 2014, May. Defining architecture components of
the Big Data Ecosystem. In Collaboration Technologies and Systems (CTS), 2014
International Conference on (pp. 104-112). IEEE.

BIG DATA SYSTEMS ENTERPRISE DEPLOYMENT, INTEGRATION, SCALABILITY
AND SECURITY ISSUES 11
Hortonworks. (2018). Apache Hadoop Data Governance, Security and Operations |
Hortonworks. [online] Available at: https://hortonworks.com/solutions/security-and-
governance/ [Accessed 8 Oct. 2018].
Haber, M. (2018). The data governance story: How to develop policies & rules. [online]
IBM Big Data & Analytics Hub. Available at:
https://www.ibmbigdatahub.com/blog/data-governance-story-how-develop-policies-rules
[Accessed 8 Oct. 2018].
Moura, J. and Serrão, C. (2018). Security and Privacy Issues of Big Data. [online] Arxiv.org.
Available at: https://arxiv.org/ftp/arxiv/papers/1601/1601.06206.pdf [Accessed 8 Oct.
2018].
Smith, K. (2018). Big Data Security: The Evolution of Hadoop’s Security Model. [online] InfoQ.
Available at: https://www.infoq.com/articles/HadoopSecurityModel [Accessed 8 Oct.
2018].
Sharif, A., Cooney, S., Gong, S. and Vitek, D., 2015, October. Current security threats and
prevention measures relating to cloud services, Hadoop concurrent processing, and big
data. In Big Data (Big Data), 2015 IEEE International Conference on(pp. 1865-1870).
IEEE.

BIG DATA SYSTEMS ENTERPRISE DEPLOYMENT, INTEGRATION, SCALABILITY
AND SECURITY ISSUES 12