Data Mining Applications for Business Intelligence and Analytics

This paper explores the applications of data mining in business intelligence and analytics, with a case study of South West Airlines. It discusses the benefits of using data mining techniques for decision-making and increasing profit margins.

Running Head: DATA MINING APPLICATIONS FOR BUSINESS INTELLIGENCE AND
ANALYTICS 1
BIS551 Data Mining Applications for Business Intelligence and Analytics: A Case Study of
South West Airlines
By (Name/Group of Student)
(Institutional Affiliation)
(Date of Submission)

DATA MINING APPLICATIONS FOR BUSINESS INTELLIGENCE AND ANALYTICS 2
Table of Contents
Abstract............................................................................................................................................2
Introduction......................................................................................................................................3
Background and Literature Review.................................................................................................4
Methodology....................................................................................................................................4
Data Analysis and Results...............................................................................................................6
Conclusion.....................................................................................................................................16
References......................................................................................................................................17

DATA MINING APPLICATIONS FOR BUSINESS INTELLIGENCE AND ANALYTICS 3
Abstract
Business Intelligence is appreciated as a result of intense research from statistical through data
mining application tools such as MIS (Management Information System) and OLAP (Online
Analytical Processing). Old style techniques of decision making through expertise, sometimes
guess work and knowledge were very ineffective in the area of Business Intelligence. This paper
aims to apply data mining techniques and the analytical approaches at South West Airlines in
order to determine the application of data mining and business intelligence within the company.
The application of data mining in business intelligence has been known to add values through the
implementation of data mining technique which increases profit margin by larger percentages.
Companies such Amazon, Shell Services International, Google, etc., have gained functional
efficiency via Business Intelligence and Analytics applications through implementation. In this
paper, we aim to analyze the applications of the data mining in business intelligence within
Southwest Airlines which is known to be applying data mining techniques in most of its business
operations. This paper also presents some the analytics of the achievements and weakness that
have been encountered by Southwest Airline through the application of data mining in business
intelligence.

DATA MINING APPLICATIONS FOR BUSINESS INTELLIGENCE AND ANALYTICS 4
Introduction
Data mining can be defined as the practice of data collection from diverse sources with
many methods such the use of artificial intelligence techniques and get the results which are
important for discovering the rules that are hidden in the data and help in improving the business
itself. Data mining applies techniques such as neutral networks, advanced statistical tools and
artificial intelligence to determine the staff skills, prices and also analyzing their competitors
According to Brijs, Business Intelligence refers to a technology driven process for collecting,
analyzing data and presenting processed information to assist organization management team,
end users and business managers make well-informed business decisions (Brijs, 2016).
Data mining and business intelligence comprises of various tools methodologies and
applications that help institutions to collect data from external sources and internal systems,
prepare it for analysis, create and execute queries against the data, and develop reports and data
visualizations to make the analytical results available to organization decision makers and
functional employees. It provides important information to organization management
concerning customers, employees, business associates, and suppliers, which is used in efficient
decision making.
Notwithstanding, the expansion in consumer items defaults and rivalry in the business
market, the majority of the organizations are hesitant to utilize Business Intelligence (BI)
innovations in their basic leadership schedules (Kin, 2010); Yan and Xiangjun, 2010). For the
most part, an organization's workers depend on data mining techniques to direct them in
assessing the value of strategic plan within the firm. Furthermore, strategic agreement
assessments depend on a representative abstract appraisal. Such judgment is wasteful,
conflicting, and non-uniform; this will prompt increment the expense of activities, hazard, and

DATA MINING APPLICATIONS FOR BUSINESS INTELLIGENCE AND ANALYTICS 5
potential misfortune to the organizations. A lot of business decision-making system has been in
existence since the 1980s. In 1989, Howard Dresner introduced the term Business Intelligence,
which was the umbrella for such systems (Sherman, 2015).
Background and Literature Review
Data mining as a strategic business tool has long been proposed in an effort to increase a
company's competitiveness (Porter, 2010). It is an important aspect of strategic management
because it serves as the first link in the chain of perceptions and actions that permit an
organization to adapt to its environment. Being a relatively new management tool in the business
world, Business intelligence plays an important role to support managers today for better
decision making and strategic planning.
Hannula and Pirttimaki (2013) argue that a competitive edge is gained through the ability
to anticipate information, turn it into knowledge, craft it into intelligence relevant to the business
environment, and actually use the knowledge gained from it (Calof and Wright, 2018). The intent
of any business intelligence System is simply to provide a system for developing or improving
processes through a structured approach, effective deployment and better control.
An organisation which does not rigorously monitor and analyze key competitors is poorly
equipped to compose and deploy effective competitive strategy and this approach leaves the firm
and its markets vulnerable to attack and its performance decline (Prescott and Bharwajh, 2015).
Failure to collect, analyze and act upon competitive information in an organized fashion can lead
to the failure of the firm itself (Marceaus and Sawka, 2011).
Business Intelligence and Data Mining
Application of data mining for Business Intelligence (BI) enables the business to make
intelligent, fact-based decisions. The most cogent argument for establishing a new roadmap to

DATA MINING APPLICATIONS FOR BUSINESS INTELLIGENCE AND ANALYTICS 6
business Intelligence (BI) excellence is to rid the organization of the technology scramble and
cobbled together solutions that Information Technology (IT) has had to deal with as it struggled
to meet business requirements.
According to Ranjan (2009) a Business Intelligence (BI) organization fully exploits data
at every phase of the Business Intelligence (BI) architecture as it progresses through various
levels of informational metamorphosis. Data is first collected including metadata, such as the
creator or creating system, the time of creation, the channel on which it was delivered, sentiment
contained in plain text, and so on. According to Olszak and Ziemba (2016) metadata facilitate
the process of extracting, transforming and loading data through presenting sources of data in the
layout of data warehouses. Metadata are also used to automate summary data creation and
queries management.
Challenges of using Data Mining Applications for Business Intelligence
According to (Chuah and Wong, 2013) data mining applications have appeared the top
spending priority for many Chief Information Officers (CIO) and it remain the most important
technologies to be purchased for past five years (Gartner Research 2017; 2018; 2009). Although
there has been a growing interest in Business Intelligence (BI) area, success for implementing
Business Intelligence (BI) is still questionable (Ang and Teo 2010; Lupu et.al., 2017;
Computerworld, 2013). Lupuet.al. (2017) reported that about sixty percent of business
intelligence applications fail due to the technology, organizational, cultural and infrastructure
issues. Furthermore, EMC Corporation argued that many Business Intelligence (BI) initiatives
have failed because tools were not accessible through to end users and the result of not meeting
the end users‟ need effectively.

DATA MINING APPLICATIONS FOR BUSINESS INTELLIGENCE AND ANALYTICS 7
According to Chuah and Wong (2013) the first challenge facing data mining and
Business Intelligence (BI) system is the cost of technology, upkeep and implementation. The
second challenge is the number of users, the number of business users now tapping into Business
Intelligence (BI) is increasing dramatically, especially as we begin to move into operational
intelligence. The third challenge is in the area of operational data mining and Business
Intelligence and the new sources of data available. We are seeing a tremendous increase in the
volumes of data (big data) being analyzed and stored in data warehouses and experimental areas.
This data is used for complex advanced, embedded and streaming analytics (Wong and Chuah,
2013). There are now very interesting sets of data in Business Intelligence (BI), which is
certainly different from the traditional, more strategic or tactical forms of Business Intelligence.
These big challenges lead to the fourth, which is the performance and scalability of the
environment. Obviously, if we are starting to bring in operational people, operational Business
Intelligence(BI), streaming analytics, big data applications, etc., it means that the performance
has to be a major focus of the Business Intelligence (BI) implementers – sub-second response
time for many operational intelligence queries while simultaneously supporting the more
strategic or long running queries as well. It’s a mixed workload environment, and that can cause
a performance issue as per Lupu et.al (2017).
Financial institutions as well as other non-financial organizations are challenged by big
data and require them to be proactive in managing and utilizing corporate it if they want to keep
up with or stay ahead of the competition. Data mining application in business Intelligence gives
enterprises the capability to analyze the vast amounts of information they already have to make
the best business decisions. Financial institutions are able to tap into their huge databases and
deliver easy-to-comprehend insight to improve business performance and maintain regulatory

DATA MINING APPLICATIONS FOR BUSINESS INTELLIGENCE AND ANALYTICS 8
compliance (Nemati, 2005). The applications of business intelligence in the financial sector are
therefore far-reaching. While the Business Intelligence (BI) solution typically contains the
necessary data that are required for identifying opportunities for improvement, significant effort
is often required to get to these insights. Often, the level of effort required to find valuable data
points exceed the cost of finding it. Moldovan (2011) studied the 23 financial industries and
found that mining financial data presents some challenges, difficulties and sources of confusion,
especially when determining short term trends and validating them.
For data to be used, it is important to ensure it is clean. Venter and Tustin (2009) depicts
that the purpose of a data warehouse is to provide rich, timely, clean and well-structured
information to Business Intelligence (BI) analysis tools. Once that is done, the organization can
take advantage of the vast amounts of information; give it to users in a way they can understand.
Deliver predictive scores to the customer service representatives, so they know which offers are
most likely to result in a positive outcome. Provide sophisticated visualization tools to analysts
who can see patterns in millions of data points. Deliver a dashboard to the Vice President (VP) of
marketing with social media sentiment scores about that new product.
According to Olszak and Ziemba (2016) beneficiaries of Business Intelligence (BI) systems
include a wide group of user such as insurance companies, oil and mining industry, security
systems, banks and supermarkets. Banks are amongst the most common sectors that use Business
Intelligence (BI) systems; Business Intelligence (BI) systems also assist in determining the
profitability of individual customers who are current and long term. This provide the basis for
high profit sales and relationship banking, thus maximizing sales to high value customers,
reducing costs to low value customers. This provides a means to maximize profitability of new
innovative products and services therefore promoting value creation in insurance sector.

DATA MINING APPLICATIONS FOR BUSINESS INTELLIGENCE AND ANALYTICS 9
Data Analysis
South West Airline applied the technique of data mining and business intelligence in predicting
airfare prices through the utilization of the previous records. Data mining tools predict behavior
and help business make knowledge based decisions. In this project we are using XLMiner, add-
in available for MS Excel to perform data mining techniques on dataset. Southwest Airlines
strategy to cover major cities and provide services as secondary airport has been different from
the model followed by all airlines in the United States. The presence of discount airlines is
therefore believed to reduce the airfares. The questions and goals that we plan to investigate in
our dataset are,
 By correlation tables we can explores categorical predictor’s vs. Response (FARE).
 Conduct data analysis using pivot tables and scatterplots. Using Pivot tables, data can be
easily summarized in a tabular format and examined. Scatter plot is a visualization
technique that helps with data quality issues and outliners.
 Reduce the use of dummy variables if any.
 Compare the models for reducing predictors using exhaustive search and using the
variables suitable for the model.
 Prepare lift charts, decile-lift charts for training data to predictive accuracy for the
models.
 Using multiple linear regression, predict the regression model, coefficients, Average
Errors to predict best suited model.
Data Description
1. Below are variables from the dataset and their description.

DATA MINING APPLICATIONS FOR BUSINESS INTELLIGENCE AND ANALYTICS 10
2. There are 18 attributes or variables in the dataset for each record. File contains 638
records of information regarding air routes in United States.
3. Sample data: Please find below first 20 records from the dataset
4. The reason for choosing airfare data is the flexibility of booking flight tickets using
internet. By using this data and analyzing the predictions for airfares, consumers can save
some money. After predicting airfares, this analysis can be extended to care rentals,
hotels and so on. Hence we see more benefit to understanding this dataset outside this
course.

DATA MINING APPLICATIONS FOR BUSINESS INTELLIGENCE AND ANALYTICS 11
Results
1. Best Single Predictor of FARE: Using scatterplot we can obtain the correlation between
two characteristics and also find the best numerical predictors. Sometimes when there is a
good correlation between characteristics, we can predict the other variable which is tough
to measure. Scatterplots are known for providing 2-D visualization for both discrete-
values variables and continuous valued variables.

DATA MINING APPLICATIONS FOR BUSINESS INTELLIGENCE AND ANALYTICS 12

DATA MINING APPLICATIONS FOR BUSINESS INTELLIGENCE AND ANALYTICS 13

DATA MINING APPLICATIONS FOR BUSINESS INTELLIGENCE AND ANALYTICS 14

DATA MINING APPLICATIONS FOR BUSINESS INTELLIGENCE AND ANALYTICS 15
After carefully observing the scatterplots between all the numerical predictors with
Response (FARE), we can say that “Distance” has best correlation with Response
(FARE).
2. Find the categorical predictor: To find categorical predictor we are using Pivot
table. Pivot Table is widely used for reporting and also used for obtaining a
summary of all the variables. By looking at the table below, we can say that the
variable with highest Count of Total between the categories is the best categorical
variable. Here Slot is the best categorical variable.
Vacation Count of FARE
No 468
No 329
Controlled
Constrained 18
Free 114
Free
Constrained 78

DATA MINING APPLICATIONS FOR BUSINESS INTELLIGENCE AND ANALYTICS 16
Free 119
Yes 139
Controlled
Constrained 1
Free 17
Free
Constrained 7
Free 114
Yes 170
No 115
Controlled
Free 28
Free
Constrained 20
Free 67
Yes 55
Controlled
Free 4
Free
Free 51
Grand Total 638

DATA MINING APPLICATIONS FOR BUSINESS INTELLIGENCE AND ANALYTICS 17
3. Find best suited model: Multiple linear regression analysis is used to find linear
relationship between quantitative dependent variables and predictors. Mainly used for
performance assessment for reaching the predicted goal.
Please refer to the excel “Final_Project_XLMiner_Work” for MLR ouput
Multiple Linear Regressions Excel

DATA MINING APPLICATIONS FOR BUSINESS INTELLIGENCE AND ANALYTICS 18
Exhaustive Search Output:

DATA MINING APPLICATIONS FOR BUSINESS INTELLIGENCE AND ANALYTICS 19
Provided a comparison table showing Lift charts for Logistic Regression and Multiple Linear
Regression
Logistic Regression Multiple Linear Regression

DATA MINING APPLICATIONS FOR BUSINESS INTELLIGENCE AND ANALYTICS 20
Comparing the RMSE, Average Errors, lift charts and Decile-wise lift charts for training dataset,
we can say that model with minimum number of variables yield better results. The values
obtained in the model should be less when compared and also lift charts should yield a linear
structure. In our case, Logistic Regression is the best model for predicting airfare for new routes.
In this project, we predicted the outcomes by using multiple linear regression and logistic
regression and choosing the best suited model for solving a business problem for Southwest
Airlines. However from our outputs obtained in the XLMiner, it is determined that Logistic
Regression with Exhaustive Search is the suited model for predicting the airfares.
Discussion and Conclusion
Conclusively, in a business enterprise, management teams have a primary role in the success of
operations. These teams have to make decisions that directly affect the institutions. An
inappropriate decision formed by poorly informed data source results to the calamitous ending.
The deployment of BIA improves the decision taken by the management team. The BIA systems
endow workers with enough message and make them capable of decision making. A lot of data

DATA MINING APPLICATIONS FOR BUSINESS INTELLIGENCE AND ANALYTICS 21
is present in organizations. These data can be that concerning invoices, suppliers, customers,
employee information, purchase orders, client data, financial data, training data, etc. BIA
systems are helpful in the management of such huge data. Management of such large data
manually would have been cumbersome.
Despite the fact that the idea of data mining and business intelligence just risen quite a few years
prior, it presently is turning into a noteworthy worry for endeavors paying little attention to its
size to mull over it whether they ought to put resources into this framework or not so as to fulfill
the client needs and demands. These days, BI sets up a genuine business estimation of
information resource and gives astounding improvement in perceiving and exploiting business
openings. Numerous global partnerships have received business intelligence framework, yet
some of them bombed in adjusting this framework. Operational and authoritative factors, for
example, technique, human capital, initiative, culture, quality administration and vital
introduction of a firm fundamentally influence the data mining and business intelligence
framework's execution and joining. Understanding capacities of both innovative and the decision
makers viewpoint is a key achievement in receiving data mining and business intelligence
framework in any organisation.

DATA MINING APPLICATIONS FOR BUSINESS INTELLIGENCE AND ANALYTICS 22
References
Bergh, D. D., & Ketchen, D. J. 2009. Research Methodology in Strategy and Management.
Journal of Business Intelligence, 210(4), p. 121..
Brijs, B. 2016). Business Analysis for Business Intelligence. Retrieved March 27, 2019,
http://www.businessdictonary.com/definition/data-
mining.html (n.d.). Retrieved from http://docs.orange.biolab.si/3/visua
Hsu, & Wynne. 2007. Temporal and Spatio-Temporal Data Mining. Hershey: IGI Global.
Klopotek, M. A., Wierzchon, . T., & Trojanowski, . (2015). Intelligent Information Processing
and Web Mining. Berlin: Springer Science & Business Media.
Miller, S., & Hutchinson, W. (2013). Oracle Business Intelligence Applications. New York:
McGraw-Hill Professional.
Miner, IV, J. E., Fast, A., Hill, T., Nisbet, R., & Delen, D. 2012. Practical Text Mining and
Statistical Analysis for Non-structured Text Data Applications. Massachusetts: Academic Press.
Sherman, R. 2015, June. Understanding BI analytics tools and their benefits. Athena IT
Solutions.
Shmueli, G. 2010. Data mining for business intelligence. Concept, techniques, and application in
Microsoft office excel with XLMiner. (2nd Ed.). Hoboken, NY: John Wiley & Sons, Inc.
Surma, J. 2011. Business Intelligence: Making Decisions Through Data Analytics. New York:
Business Expert Press.
Thuraisingham, B. 2013. Web Data Mining and Applications in Business Intelligence and
Counter-Terrorism. Florida: CRC Press.

DATA MINING APPLICATIONS FOR BUSINESS INTELLIGENCE AND ANALYTICS 23
Wong, T. & Chuah T., 2013. Data Mining and Business Intelligence. Journal of Business
Intelliegenc in action, 272 (5), 42-211.