Big Data: Applications, Challenges, and Governance in Finance

This report delves into the application of big data within the finance sector, examining its features, including data capture, storage, analysis, and retrieval. It highlights the finance industry's reliance on big data tools to address challenges such as financial crimes, employee management, and customer intelligence. The report discusses the five V's of big data, predictive modeling, and automated trading. It also explores the cons of big data, like logistical issues and privacy concerns. Various technologies used for data analysis, storage, and governance are discussed, along with the benefits such as cost reduction and improved efficiency. Furthermore, the report addresses risks like data security breaches and the importance of data governance, using examples like LinkedIn to illustrate its impact. Finally, the report recommends effective data governance tools to ensure compliance and maximize the benefits of big data in finance.

BIG DATA

Table of Contents
INTRODUCTION...........................................................................................................................1
MAIN BODY...................................................................................................................................1
CONCLUSIONS..............................................................................................................................7
REFERENCES................................................................................................................................8

PART A
INTRODUCTION
Big data are those information sets which are very tortuous and complex, such that they
can not be dealt by the application of traditional data handling tools/software. There are various
features of big data such as capturing information, data storage, data analysis, retrieval, sharing,
searching, transferring, etc. (Chen and et. al., 2014). Majorly, this application is usable for the
government, education, healthcare, international development, information technology and
finance management. In the present world globalisation, modernisation, businesses, banking etc.
are tremendously growing and spreading their roots. All these factors demands for effective
management of the finances. The financial sector has to deal with 0.3 quintillion of data on a
daily basis. These services are widely adopted to manage big data to inform better investment
decisions with constant returns. In the present undertaking report, aspects of big data in context
to the finance sector will be studied.
MAIN BODY
Finance sector involves antithetic services such as banking, insurance, stock market,
treasury or debt instruments, wealth management, mutual funds, tax consultants, etc. All these
segments are widely spread and demands for the application of big data tools so that information
can be managed effectively (Trnka, 2014). There are various problems are being faced by the
finance industry. The issues leaving the severe influence on the whole sector is the increasing
financial crimes and frauds in absence of the effective management of the internal information.
The previous data leaked or not managed in appropriate manners also results in this type of
situation. In addition to this, there are large number of people who are engaged in different
segments of the factor such as in banks, finance institutions, share & stock markets, etc. It is
quite difficult and challenging for the industry to manage the people working in organisations. In
this way, it is necessary to take assistance from the big data management tools. The significance
of bid data can be expressed with its five V characteristics, which are volume, variety, velocity,
veracity and variability (Lazer and et. al., 2014).
The use of mentioned application assists in effective monitoring and surveillance of
employees engaged in the organisations belonging to finance industry. Big data is a reliable tool
in the information technology that provide predictive models that can be underpinned by
1

insurance underwriters for premium settings, loan rendering professionals in making lending
decisions, etc. It further provides assistance in the pricing of solid assets, for instance: real estate.
Further, time is regarded as money in financial services, the application of big data promoted
automated trading with the real time analysis. It saves time on taking decisions on the
investments. This advantage is highly appreciated as it has streamlined the complete trading
process. Customer intelligence is the most effective tool which is widely used by the various
companies belonging to financial sectors (Groves and et. al., 2016). For instance: Nowadays,
the banks are getting changed form product centred to customer centred. This customer
intelligence applications helps in achieving the desired growth.
In contrary to this, Sandryhaila and Moura, (2014 ) stated that there are different cons of
using big data tools in finance sector. Firstly, it develops logistical issues that is entities which
are having big data have to modify their compete business procedures. This will laid many
negative impacts on the effective functioning of their organisations. In addition to this, real-time
big data required to conduct sophisticated and complex analysis which can result in formulation
of entirely incorrect business strategies. Further, the biggest concern of applying big data tools is
associated with the privacy of data and information.
There are four different types of big data that are used in finance sector, they are
prescriptive, predictive, diagnostic and descriptive. Prospective aids in understanding the actions
that should be taken. Predictive data includes the scenarios that might happen. In addition to this,
diagnostic form of data looks at the past performance to analyse the situation in relatable way.
Further, descriptive data is used to analyse the present scenario on the basis of incoming data.
There exits various technologies with the help of which the big data is treated. Firstly, in context
of data analysation, the tools used are A/B testing, machine learning and natural language
processors. The most effective technologies of big data further involves business intelligence,
cloud computing and databases (Einav and Levin, 2014).
There exits a wide range of technologies that can be used to store and analyse the big
data. Apache Hadoop is regarded as the most effective of all. It is a Java based software system
which is specially designed to store a large sized data in form of cluster. The mentioned
framework operates in parallel with the cluster and renders user with the ability to access the data
at all the nodes. The big data is split and distributed across different nodes of cluster with the
help of Hadoop distributed file system (HDFS). There exist more tool, known as Microsoft
2

HDInsight which is a big data solution rendered by Microsoft. It is available as an in-built cloud
service. Further, NoSQL is the modified SQL which does not only handle the large amount of
data but unstructured information is also handled. Some other technologies which are widely
used by the people includes Big data in Excel, Presto, PolyBase, Sqoop, Hive, etc. All these tools
are highly effective in their purpose of saving, analysing and managing big data (Del Río and et.
al., 2014).
There are numerous benefits of using these technologies in the modern day-to-day life. In
context to the financial sector, the services offered with the help of big data technologies are
recommendable. Firstly, there is a myth that big data technologies increases the cost associated
with the business operations. It actually reduce the operation cost in managing data in different
noes by making use of varied tools. The maintenance charge of the organisation can be decrease
with the big data applications. It further helps in increasing the efficiency of businesses by saving
a lot of time and cost. In context to finances, business intelligence tools assist in evaluating the
data with effectiveness (Turner, Schriech and Shockley 2017). This, in turn, provides a clearer
picture of the business or an undertaken project. In addition to this, the reductions in the cost
with the help of big data tools can allow companies to reduce their pricing structure. This will
further achieve a good consumer base base benefiting the business value, brand image, etc. With
the help if same technologies, establishment can be turned more sophisticated which will assist
in sustaining in the competitive environment by competing effectively with the key rivals. Big
Data further allows to zoom into the local customers with raise the focus on regional
marketplaces fostering the overall business value. In this way, companies can increases their
sales and loyalty. Employees are regarded as the key assets of an organisation. With the help of
these big data tools, recruitment procedures of a firm can be improved. This is because of the
fact that these tools possess calibre to scan the keywords. Hence, companies can scan for the
desired qualities which these technologies will search from their submitted resumes (López, and
et. al., 2015).
On the other hand, it should be recognised that there are also some risks associated with
the use of technologies based on big data. There are different issues that can harm the
effectiveness of the businesses which are data security, data privacy, cost, bad analytics and bad
data. Theft of information is the most growing crime and impacting on major factors. This crime
3

rate is getting fostered by the application of different big data tools (Russell Neuman, and et. al.,
2014).For better understanding, the more will be the quantity or volume of information, the
criminal will easily steal it. The increased number of cyberattacks are examples of this. The
number of bad hackers has been increased which can harm the security of data with intentions to
attempt any illegal task. The other risk is associated with the privacy of data.
There are various strict laws framed for the protection of confidential information of
employees, customers such as account number, contact details, etc. However, the storage and
management of large size data makes it quite challenging to pay attention on this factor. In
addition to this, all the operations which are performed to manage the big data such as collection,
aggregation, storage and analysis requires a sum of money. This will increase the operating cost
of the companies working in the financial industry. The other issue is associated with the
accuracy of data. Financial industry is highly concerned with the reliable information to carry
business operations effectively. however, management of big size data somewhere decreases the
quality of information which can put negative influence on the companies. This is the case of bad
analytics or misinterpreting (Russell Neuman and et. al., 2014.)
In order to avoid the mentioned risks to affect the entities, it is suggested to use effective
data governance tools to make compliance with the GDPR. Data governance is the system
through which right decision can be taken which is used to promote the desirable behaviour in
context to the valuation,m creation, storage, use , retrieval and deletion of data and information.
It helps organisation in defining policies, standards, and strategies so that effectivve functioning
can be assured (Del Río and et. al., 2014). Some best practices involved in the data governance
are obtaining executive sponsorship, defining data position during the early stages, establishing
quantifiable benefits, etc. Major data governance software that are widely used includes
Blueprint OneWorld, erwin, ONeSoft, etc. This governance has become crucial after the time
when General Data protection regulation that is GDPR came into action. There are twelve steps
in which GDPR can be implemented, these are awareness, present information, communicating
privacy information, individual rights and subject access requests, legal basis of data processing,
consent & children, data breaches, design for data protection, assessment on impact of data
protection, designating data protection officer, further, if business is operating internationally
identification of suitable data protection officer is needed.
4

The effectiveness of the big data can also be grabbed if suitable data governance is there.
In case when companies are not complied there major issues may have to face in context to the
availability, usability, integrity and security. The lack of thorough management of the data assets
may force to anti-trust situation while decision-making. This can further harm the effectiveness
of overall business strategy. The best example of understanding the significance of the data
governance in success or failure of an entity can be analysed with the help of LinkedIn. Initially,
when the establishment was small there was not any data governance tool as there was not bigger
data that creates log or affected systems. However, as the data starts to grew a large number of
teams were prepared to emit and consumer data. This has increased cost, quality, work pressure,
etc. in the company. However, a Hadoop environment has been created following this. Now
people can observed the better governing of data by LinkedIn. There is also an internally
developed systems which assist in development of metrics data for the reporting uses. This
systems is termed as the United Metrics Platform. Some companies which does not have any
data governance policy includes Disney, the City of Los Angeles, Motorola, Shell, etc. All the
cited firms are thus facing several issues regarding data archiving, backup, e-discovery, etc.
There are various companies which provides a complete data governance system such as IBM,
Hortonworks, Qumran, etc. The organisations are suggested to have a data governance policy so
that big data can be managed effectively (Del Río and et. al., 2014).
Some examples of the management of finances by taking assistance from big data
includes handling of 1 million customers of Walmart every hour. The data imported in this
period is 167 times of all the books in the US Library of Congress. In addition to this, a real
estate company Windermeser uses location information from nearly 100 million drivers so that it
can help new home buyers. The biggest example of global level assistance of biog data tool can
be judged by the FICO card detection system (Chen and et. al., 2014). It is developed to protect
accounts worldwide. Amazon.com is a company that handles millions of back-end organisational
functions each day. Also, everyday more than 1/2 million customers gives queries which is
treated by the establishment. From these examples it can be observed that the role of big data in
financial sector is recommendable. There exits various major functions which are played by the
relevant technologies.
The rationale behind selecting finance industry for understanding the significance of big
data concepts is that every business is somewhere related to the finance. This industry covers
5

major bodies such as internation government, government, MNCs to smaller enterprises running
in regions. In addition to this education, healthcare, etc. are there that are reliant on managing
finance at some extent. Everyday banks, insurance companies stock markets, etc. performs action
which requires a large volume of data and in order to manage this, big data tools are necessary.
Banks required to have a public cloud so that they can get a clearer picture of manageable pat for
the large public cloud adoption (Chen and et. al., 2014). In addition to this, in order to control
the financial frauds, this data is required to be managed with the help of big data technologies.
However, the biggest reason of sleeting finance sector as the subject of research is that he service
of this industry are turning to IoT and streaming. The most common examples are the ATMs,
mobiles,, etc. Further, customer intelligence has become a very important factor to consider
while rendering financial services. It should be moreover recognised that the countries are much
reliant on these tools in context of computing GFP, different rates, analysing the economical
condition of the nation, etc. All these operations required assessment of really larger data that is
very unconventional to perform otherwise. The role of big data is major in finance and trading
sector. In the mentioned sector, the bid data is ruling because of three major functions which are
technical analysis, real-time analysis and machine analysis.
There exits various big data technology vendors available in market such as IBM,
Oracle, Cloudera, Teradata, etc. These entities aims on rendering a supporting software that can
perform the following operations: performance optimisation, improvement, understanding and
serving clients, monitoring, real time adjustments, etc. These organisations assists the companies
belonging to different industries that deals with the big data. One of the important sector where
big data application can be applied is communication, media and entertainment. In the present
world, communication through SMS, emails, calls, etc. has been raised which has increased the
demand of managing big data (Trnka, 2014). In addition to this, the need of big data tools is also
in healthcare sector. Healthcare providers are required to keep the complete medical history, past
records, etc. of patients. All these procedures need a space to save the resulted data. With the
help of developed big data technologies, the companies working in this sector can be benefited.
Further, education sector is regarded as the most important industry of any country. It is the basic
of development of any other sector and hence, a large emphasis is needed on the data
management of educational institutions. Some other areas in which these technologies are widely
used involves manufacturing and natural resources, government, insurance, crop-data
6

management, retail and whole sale trade, transportation, etc. In addition to this, the biggest sector
covered is the government itself (Lazer and et. al., 2014). There exist a large quantity of data
and information from different sources which government is entailed to managed. In this order,
big data technologies plays a key role.
CONCLUSIONS
From the present report, it can be concluded that big data can be regarded as the key part
of finance. There are different advantages which are brought by this in the operations associated
with commerce. In this report, different big data technologies that can be used by the
organisations are mentioned. These tactics will assist them in dealing with the various risks
associated with the data saving, archiving and other operations. In addition to this, the present
research has shed lights on the significance of data governance in the finance operations. Further,
different risks that can raised in absenteeism of GPDS are analysed. Further, different sectors in
which big data technologies are required are also analysed in the present undertaking.
PART B
Peer evaluation form for group work
On a scale of 1-4 all the team members will be judged. 4-highly satisfied 1- least satisfied
Evaluation
criteria
Group member Group member Group member Group member
Attends group
meetings every
time and came on
time
4 3 3 4
Meaningful
contribution in
group work
4 4 4 4
Accomplished
assigned task on
time
3 4 3 4
7

Contributed
importantly in the
project success.
4 4 4 4
TOTAL 15 15 14 16
Discussion on the effectiveness of group work:
In order to accomplish the present report, a reliable tool that is group dynamics was taken
assistance from. This tool has assisted in performing decision-making system. As per this,
competencies of each and every member of the team were analysed. Depending upon those
skills, different tasks were assigned to them. Being the leader of the group, I have divided the
work into smaller segments within team members. Later, these work segments were assigned to
group members on the basis of their competencies. For instance: a team member has keen
interest in research work so data related to the use of big data in finance was gathered by him. In
addition to this, a healthy working environment was maintained so that conflicts does not take
place.
Behaviour of team members:
A high level of communication was also managed in the group so that each and every team
member can present their views and opinions effectively. This has helped in avoiding any
conflicts between the team members by maintaining transparency of each and every operation.
No issues were detected to be damaging the team work due to behaviour of any member.
Learning from present group work:
There were different things learned from the present work. The major is the working on any
project should be carried after developing an effective plan carrying all the tasks. The work was
organised effectively by making a suitable plan. This plan was carrying all the tasks that are to be
performed for preparing the report. These activities are project planning, research, data
collection, developing design, etc. A time limit was set with every task so that they can be
achieved in a stipulated time frame.
Limitations in working
There were some limitation which had created some challenges in accomplishing the work.
Firstly, it was a bit difficult to gather all the data in very less time-period. Second, as there were
8

more people engaged in the report thus, decisions were made on the basis of each one's
perception. It has somewhere affected the quality of the decisions.
Evaluation on each team members work:
1.
He has a great sense of researching on any topic. Further, it has been observed that he consumes
less time in searching as compared to others and also, the data are more realistic. Hence,. he was
assigned with the task for searching the data for the sue of big data in different sectors.
2.
She has a good experience in the sector chosen for the research. It is due to her last internship.
Also, she has made use of everything so perfectly that the actual report can be portrayed
effectively.
3.
He is the most motivated person of our group who is also committed to punctuality. He has
helped a lot in inspiring other for accomplishing their tasks on time. With his assistance, the
project was completed on time.
4.
He is a motivated team player having positive attitude. This has helped us in maintaining an
effective and healthy working environment throughout the work. Further, all the meetings
regarding the research was attended by him with all the idea he can render for making the work
better. Also, his motivation has helped in submitting the necessary work on time.
9

REFERENCES
Books and Journals
Chen, M., and et. al., 2014. Big data analysis. In Big Data (pp. 51-58). Springer International
Publishing.
Del Río, S., and et. al., 2014. On the use of MapReduce for imbalanced big data using Random
Forest. Information Sciences, 285, pp.112-137.
Einav, L. and Levin, J., 2014. Economics in the age of big data. Science, 346(6210), p.1243089.
Groves, P.,and et. al., 2016. The'big data'revolution in healthcare: Accelerating value and
innovation.
Lazer, D., and et. al., 2014. The parable of Google Flu: traps in big data
analysis. Science, 343(6176), pp.1203-1205.
López, V., and et. al., 2015. Cost-sensitive linguistic fuzzy rule based classification systems
under the MapReduce framework for imbalanced big data. Fuzzy Sets and Systems, 258,
pp.5-38.
Russell Neuman, W., Guggenheim, L., Mo Jang, S. and Bae, S.Y., 2014. The dynamics of public
attention: Agenda‐setting theory meets big data. Journal of Communication, 64(2),
pp.193-214.
Sandryhaila, A. and Moura, J.M., 2014. Big data analysis with signal processing on graphs:
Representation and processing of massive data sets with irregular structure. IEEE Signal
Processing Magazine, 31(5), pp.80-90.
Trnka, A., 2014. Big data analysis. European Journal of Science and Theology, 10(1), pp.143-
148.
Online
Turner D., Schriech M. and Shockley R., 2017. Big Data. [PDF]. Available through:
<https://www-935.ibm.com/services/multimedia/Analytics_The_real_world_use_of_big_
data_in_Financial_services_Mai_2013.pdf>
10