COIT20253: Big Data Report for Woolworths - Technologies and Insights

This report provides a comprehensive analysis of Big Data strategies for Woolworths, a major Australian retail company. It begins by outlining key business challenges and introducing relevant Big Data use cases, such as customer behavior analysis, pinpoint marketing, security enhancements, customer journey analysis, and supply chain optimization. The report then delves into a critical analysis of various Big Data technologies, data modeling techniques (including those for social networks, cloud environments, and ontology-based models), and big data architecture solutions. The report also discusses the importance of data modeling in handling structured, semi-structured, and unstructured data. The report highlights the benefits of big data in retail, including improved customer loyalty, targeted marketing campaigns, and enhanced operational efficiency. Finally, the report illustrates the big data technology stack and processing architecture required to support the specified use cases. The target audience is executive management, and the report aims to provide actionable insights for data-driven decision-making.

Student Name:
Student Id:
Unit Id:
Campus Lecturer:
Tutor Name:

Executive Summary
WoolWorths Pvt ltd is a largest Australian retail company and wants to expand their business
by using the current big data trends. Here we are introducing the use cases for the big data
and critical analysis of this big data technologies and solution. To stand with the capacity to
analyse, access and manage the huge volume of data when the information architecture is
rapidly evolving is really critical to any retailer like Woolworths to improve their business
performance and efficiency. (Chaudhuri, 2014)
But Favourable Customer experience, operational efficiency, loyalty and retention of
customers is their key of success, but expected demand is really important for more efficient
cash management, inventory management and gross profitability. As Woolworths is a larger
and diverse supermarket so here the data is also become large and diverse so this type of data
management becomes more complex. So we need to analyse the data to better understand
about what products gives the highest profit per square foot. So the retailers have collect the
data of customers from their respective loyalty cards from which we analyse about the
previously purchasing of customers, from this analysis we can monitor their buying patterns,
but still we cannot predict of future demand exactly. But by gathering and utilizing data from
other sources retailers can better understand about the customer future demand which only
possible by big data. By applying big data technologies retailer can gain a better view to
understand the customer and their families’ network and buying patterns. (Chaudhuri, 2014)
The Business analyst desired to have huge data to investigate at higher rates , also stored for
longer and like to analyse it faster, then the Big data is the solution to meet all these
requirements. Here I am representing the overview of Big Data used by the Woolworths
supermarket and analytic capabilities as part of the architecture which needs to meet all the
requirements of dynamic Woolworth’s supermarket. (Chaudhuri, 2014)

Table of Contents
Contents
Introduction...........................................................................................................................................3
Key Business Challenges.......................................................................................................................4
Big Data Use Cases...............................................................................................................................5
Critical Analysis of Big Data Technologies...........................................................................................6
Data Modelling Techniques...............................................................................................................7
Big Data Architecture Solution............................................................................................................10
Conclusion...........................................................................................................................................13
References...........................................................................................................................................14

Introduction (Katal, 2013)
Big data is introduced to handle and scale the unstructured data. It would give the flexible
database so no need to the referential integrity concept. It can deal with de-normalized data
and do not need any joins. So no need to apply ACID properties as it supports CAP
Properties that is Consistency, Availability, and partitioning. Out of these 3 only any two will
applicable. Instead of ACID, it supports by BASE that is Basically Available Soft state,
Eventual consistency(BASE) . An example of Big Data tools is BigTable, HBASE,
Cassandra etc.
As mentioned in the Australian Food and Grocery Supply Chain Report 2016 there “ hees
been huge focus on customers loyalty , as customers are increasingly purchasing from
multiple stores and the retailer have to work hard to attract the customers”. Woolworths are
well known Australian supermarket among various retailers and big data gives them the
better opportunity to understand customer needs and improve consumer faith. Since many
years they invested in big data to capture the customers by ensuring they are able to make
fact based decisions.
Today now the big data is getting more popular because of its application prospects and
broad research. As the use of big data is increasing so the effort is also increasing to analyse
and store this data. Earlier DBMS works only the structure data but today’s data is having
some unstructured components and analysis of these unstructured components is play very
important role for decision making. Today maximum data is unstructured and to store and
manage this type of data is still a challenge, so before analysis on this data it should be
modelled, this modelling is also important in big data as it contains structured, semi-
structured and unstructured data in the data base. For mapping of all these types of data we
modelling also plays important role for data analytics.

Key Business Challenges
Today the retailers used the data warehouses and business intelligence tools for reorting and
analysing customer operations and behaviour. By implementing Big data Management
systems that contains data reservoirs(like NoSQL and/or Hadoop) gives the benefits in these
areas can be find like:
» By analysing the existing costumers and their family purchasing pattern increase the up –
selling or cross-selling
»Organise the effective and targeted campaigns of marketing and obtain higher value from
the spent money.
»To gain higher customer retention , streamlined and effective operation have performed to
meet customer demands.
»Big data provides the predictive analysis to predict customer future demand and preferences
to enhance retail operations.
Use Cases in Big Data

To learn the importance of big data and analysis in the retail market, we have to look for the
following 5 use cases which best suited on retail companies:
1. Behaviour Analysis of customer in Retail Market (Hitchcook, 2018)
It is critical to improve customer conversion rate, because the customer is interacting
from different companies for purchasing by the Different sources like social media,
mobile, ecommerce websites and stores, so it increase the complexity to gather and
analyse the varied data.
While this data is aggregated and analysed you will see the customer’s interest on
purchasing as which they purchase more, and also try to predict what can purchase
next, by doing all this you can acquire more customer loyalty.
Big data gives the technologies to gather and analyse and make decisions to get more
customers.
2. Pin-Point Marketing Strategies (DataCentre, 2017)
This is another use case of retail market. By using big data analytics company analyse
where to spend their money in marketing for getting maximum return, whether it is in
Social media campaigns, advertising, in-store promotions, direct marketing or by
other channels.
Maximum supermarket retailers using the NBO(Next Best Offer) technology. This
technology uses real-time analytics to send offers to customers via phone. Today In-
store mapping is gaining much popularity and when it is combine with mobile offers
the selling rate can goes higher.
3. Better Security (DataCentre, 2017)
This is also a use case for retailing market we need to identify the fraudulent
behaviour by big data analytics. We are gathering data from sales projections, point-
of-sale, return rate, warehouse movements and from many other sources where
anomalies is identified which could point towards fraud. And by using big data
analytics retail supermarket is prevented from data breach.
4. Analysis Of Customer Journey (Hitchcook, 2018)
Now a days in world Customers are connected to the technologies very much and
empowered their strength by accessing mobile, e-commerce, social media customers
can gather any kind of information within a second, these information helps them to
make decisions what they should buy, at what price and from where.
By Big Data technologies bring all the structured and unstructured data into the
Hadoop big data tool and analyse that data to get the pattern and insights that was not
possible with traditional analytics.
By this analysis we can get answers to complex retail concerns like:
 Monitoring of customer journey
 Know about the high-value customer and their behaviour
 Best way to reach among the customers
5. Supply Chain Analysis and Operational Analytics (Hitchcook, 2018)

Today in the rapid changing product life cycles and increasing complex operations it
is essential to use big data analytics to learn about the supply chain and product
distribution to reduce cost. Many and specially Woolworth supermarket knows very
well the severe pressure to better asset utilization , performance, service quality and
budgets. It is necessary to stand in the competition and drive better performance.
Big data technologies allows to combine the structured data like mainframe, CRM,
geo location, ERP and public data with unstructured data. Then Use the right analytic
tool to analyse pattern and behaviour and change according to customer needs and to
capture the customer
Critical Analysis of Big Data Technologies
Today now the big data is getting more popular because of its application prospects and
broad research. As the use of big data is increasing so the effort is also increasing to analyse
and store this data.
Hence, before applying the analytics on big data , it should be modelled. Modelling the big
data is important because it contains structured, semi structured, unstructured data. And 85%
of data is unstructured and semi-structured. To map all these varieties of data, modelling
plays an important role in big data analytics.

Data Modelling Techniques
1. Data Modelling in Social Networks: This proposed model is based on big table. In
figure we have shows the data model which utilizes the Google’s big table to store
Social networking data like contents and comments, And this table can be viewed as
key/value based data model. this table contains n rows and Unique identifier is
defined for every row in the row key field. There are several columns in the table for
each row and in these columns, Column-key and value is stored. There is key value
pair in the column and column key which is utilized to identify each data uniquely.
(Xiaoyue Han, 2012)
2. Data Modeling In Cloud Environment: In this proposed model data is modeled in
cloud environment. In this model we have to build schema for big data first. But for
creation of this schema this method first need to know about the type of data coming
from multiple sources.If identified data is unstructured then its key information is
obtained from developing metadata. After development of meta data it extracts the
entities with information about publisher, names etc. and also extracts the facts with
information about issues, type of content etc. This metadata is developed besd on the
Dublin Metadata Development. Using this 15 elements were used to develop metadata
for each unstructured data. Figure Shows those 15 elements. (Imran Khan, 2015)
Figure 3: 15 elements in DCME
When the desired information is pull out then it will classified according to the type of table
created and data and then mapped with the structured data schema. When mapping of both
schemas done then an unified schema is resulted, which contains data information about all
that data stored in Big Data Dictionary.

When this model is applied on cloud storage level , we use the commodity hardware using
Hadoop’s HDFS. In this clusters are modelled bases on the type of data , here we make two
cluster for storage of structure and unstructured data storage (Abdullah, 2013)
3. Ontology Based Big Data Model: This model is based on Ontology technique. This
introduces the explicit specification of the conceptualization, which is the abstraction
of real world things like identity, constraints and concepts. Because big data is united
with heterogeneous data sources, that is why data could not be understand and shared
by respectively. This model uses ontology technology with Map Reduce Framework
to solve the problem with unstructured data. (Li Kang, 2014)
4. Ontology Based Key/Value Storage Data Model: If we using big data for the
database then storage, query and analysis of this data should have the characteristics
like high Performance(when reading or writing the data gives the real time and high
performance in query processing), High Scalability(to reach on the growing needs of
huge data and resources), Fault Tolerance(It should be sure that distributed system
will be available when increasing data). These all features can find from this model ,
the data is stored in Key/value model and handled directly by HBase NoSQL
databases which is easy for updating of data dynamically, it uses the concept of
Google’s big table based on HAdoop HDFS of HBase which provides a platform for
support for addressing of big data storage, query and analysis. By this modelling all
desired requirements have meet for high concurrent data processing in a big data
environment.(Li Kang, 2014)
In the following figure we have shows the Hbase instance tables:

5. NLP And Ontology Modelling For Unstructured Data. In this proposed modelling
we have modelled the unstructured data using NLP with ontology that data is coming
from multiple sources. The term Unstructured data is known as the data did not have
specific structure but have grammar, this type of data include advertisements,emails,
user feedbacks on commercial websites text, comments etc. Based on this grammar
NLP will extracts the required information in unstructured data and store that in
organization particular format. There are various NLP techniques are there such as
future pacing, Reframing, Swish,Ecology, Anchoring , all these are used to find the
meaning from unstructured data and stored it in structured form. In NLP text is
handled in various layers that are shown in figure below. (Kumud, 2014)
Figure 4: NLP Layers
The Main challenges in BIG data technologies have grouped in to three catagories on the
basis of data life cycle that are explained below:
1) Data Challenges: It is related to the features of data itself
2) Process challenge: It is related to the right methods used to find, transform ,integrate,
analyse the data and produce the result .
3) Management challenge is related to cover aspect like security, Ethical aspects,
governance and privacy.

Big Data Architecture Solution (M. L. Kersten, 2011)
Figure shows the main components in representation information architecture. we can see by
the approach shown in figure the data is extract and managed in better way and then analyse
to make information meaningful which help to make business decisions. As we discussed
above security, governance and management are critical issues and always need to keep in
the mind for all retail companies as Woolworth is also has this concern.
From where we have to determine these components are always part of the architecture
should meets all the particular requirements of any company. The first step for defining the
architecture of state is representation of the ongoing state, their capabilities with functinal
gap. Basically the Curent state architecture of data warehouse should look like in below
figure.
The first gap is fullfilled by providing agile analysis and reporting environment
where ad-hoc reports and new data are growing. Many data and information
discovery engines provides this type of potentiality. When In the above
architecture, an Information discovery is included, then the architecture will look
like below figure.

Now we are analyze the data in better way our next step is to explore the new data and
information its types, these information can be 3rd party,internal, unstructured, structured or
from unknown structure.But when we trying to store the unknown stuctured data, best
approach is to use Hadoop based data resevoir, The woolworth will also use this type of big
data architecture solution. Figure illustrate the overall arcitcture
The profiling of data like how it is acquired, the frequency of updation, how it
should be formatted and the quality of data all these information help us to apply
accurate technology which is best suited for the specific situation. Also we need
to learn about the periodicity of processing required on the basis of availability
of data. Figure shows how all thing are integrated and find the desired result.