Data Integration and System Design: ICT705 Office Locator Application

Past papers and solved assignments for students. This project demonstrates data integration and RESTful API development.

ICT705
Data and System Integration
Task 2

Executive Summary
The main goal to create or build this application is to locate the offices on the bases of
services the office serves or postcode of the office. Mainly python script is used to build this
application and RESTful services are used to build the HTML page and Bottle framework is
written to run it on the server. This application is based on the guidelines are given such as
in 1at part cleaning of data was performed and in 2nd part combining of data was performed
and next part was to run it on the server and finally, Mashup is used to represent the
information.
2

Table of Contents
Executive Summary...............................................................................................................................2
Introduction...........................................................................................................................................4
Data Integration....................................................................................................................................5
Data Cleansing............................................................................................................................5
Data Merging................................................................................................................................6
RESTful Web Server...............................................................................................................................7
Mashup..................................................................................................................................................9
Running Instructions..............................................................................................................................9
Conclusion...........................................................................................................................................11
References...........................................................................................................................................12
Appendix.............................................................................................................................................13
Table of Figures
Figure 1: Unstructured data..................................................................................................................5
Figure 2 Code for cleaning the data.......................................................................................................5
Figure 3 after cleansing.........................................................................................................................6
Figure 4: Python code for combining files.............................................................................................6
Figure 5 Data integration.......................................................................................................................7
Figure 6 Python code for RESTful services (1)........................................................................................8
Figure 7 Python code for RESTful services (2)........................................................................................8
Figure 8: Command to run office_locator.py code...............................................................................9
Figure 9: Bottle framework implementation.........................................................................................9
Figure 10: Final Output (1)...................................................................................................................10
Figure 11 Final Output (2)....................................................................................................................10
3

Introduction
In this assignment, there are different parts which are to be performed in order to achieve
the completion of the assignment. This assignment is built to locate the offices through
postcodes and services. There is a python script written to clean the data and insert the
missing values of Australian code and regional code. In the next part, different files are
merged into a single file to obtain useful results and RESTful architecture and Bottle
framework are used to run the application on the server and Mashup language is used to
display the data.
4

Data Integration
Data Integrating is the process of integrating or combining the datasets, tables, information
from various sources and stores this meaningful information/data in data warehouses. All
the data which has gone through the data Integration process has been passed through the
cleaning process and shows the data in a clean format.
Data Cleansing
This process refers to data cleaning in which all the error or inappropriate information is
cleaned or deleted from the tables or datasets. The use of this part in this program is
important because the phone numbers which are entered might not be the incorrect format
and so the results wouldn’t be appropriate. The coding is done in a way that detects the
missing Australian code continued with regional code and the number. The code
automatically detects the problem and accordingly makes the changes.
Figure 1: Unstructured data
Figure 2 Code for cleaning the data
5

Figure 3 after cleansing
Data Merging
Data Merging is the process in which data present in different files are combined into a
single file so that the same type of data is present in the same file. In this, this file is saved as
Office_Service_Locations.csv
Figure 4: Python code for combining files
6

Figure 5 Data integration
RESTful Web Server
RESTful Services stands for Representational State Transfer and this architecture is used to
build lightweight websites which are easily maintainable. HTTP is the basic protocol for
REST.
7

Figure 6 Python code for RESTful services (1)
Figure 7 Python code for RESTful services (2)
8

Figure 8: Command to run office_locator.py code.
Figure 9: Bottle framework implementation
Mashup
This application is used to create content from many resources. Important characteristics
are visualization, aggregation, and combination. From Mashup, we can create existing data
in a useful manner. To permanently access other services data, mashup application is
hosted online or used as the client application. There are many types of Mashup like
consumer mashup, business mashup, and data mashup.
Running Instructions
First, the files are to be merged into a single file and to perform this step,
“data_integration.py” is to be run. Then “Office_locations.py” file is to be run so that the
local server is to be created and the application can on that server.
9

Figure 10: Final Output (1)
Figure 11 Final Output (2)
10

Conclusion
This assignment successfully shows the data and clean the data provided by adding the
Australian code and regional code to the file and then different files are successfully
combined together to obtain useful information. This application successfully shows the
nearby offices on the bases of postcode and services served by the office. All these steps are
performed online with the use of RESTful architecture and Mashup language. Mash
language is used to display the information.
11

References
 Azeroual, O., Saake, G. and Abuosba, M., 2019. Data quality measures and data
cleansing for research information systems. arXiv preprint arXiv:1901.06208.
 Lagani, V., Karozou, A.D., Gomez-Cabrero, D., Silberberg, G. and Tsamardinos, I.,
2016. A comparative evaluation of data-merging and meta-analysis methods for
reconstructing gene-gene interactions. BMC bioinformatics, 17(5), p.S194.
 Kini, L.G., Davis, K.A. and Wagenaar, J.B., 2016. Data integration: combined imaging
and electrophysiology data in the cloud. NeuroImage, 124, pp.1175-1181.
 Richardson, L. and Ruby, S., 2008. RESTful web services. " O'Reilly Media, Inc.".
12