ICT705 Data and System Integration

ICT705
Assignment 2
[DATE]
[Company name]
[Company address]

ICT705 Data and System Integration
Contents
Executive Summary...................................................................................................................................2
Introduction.................................................................................................................................................2
Concepts.....................................................................................................................................................2
Data cleaning and merging...................................................................................................................2
RESTful web services...........................................................................................................................5
AJAX........................................................................................................................................................8
Running Instruction..................................................................................................................................13
Bibliography..............................................................................................................................................13
Conclusion................................................................................................................................................13
Appendix...................................................................................................................................................13
1

ICT705 Data and System Integration
Executive Summary
The assignment is done as per my course work for subject ICT705. The requirements
accomplished in the assignment are:
 Part 1: Data loading, data transformation and data merging was done using PETL.
 Part 2: A web service was created using Bottle, querying the database using PETL to
provide data on HTTP GET requests.
 Part 3: Mashup was done between HTML, AJAX and web service API to plot the offices
and services data on google maps.
Introduction
The mashup application program is built to demonstrate the ability to use the python and
supportive library PETL and Bottle framework. The application performs the task of cleaning the
data from the raw csv file and transforms it into specific file which is optimally for querying and
servicing user’s requests. It also provides API to access data on HTTP request and finally an
implement to plot data on google maps. The application using python, PETL and Bottle is built to
provide office location to the users based on the services they are looking for.
Concepts
The three concepts of building this application are data cleaning and merging, web API and final
using AJAX to send requests to the server.
Data cleaning and merging
Data cleaning is defined as the activity of cleaning the data with respect to a specified format.
The cleaned result in faster query retrieval and better system performance. The data cleaning
process consist of removing unwanted data like null values, garbage value and incorrect values.
The data merging process is defined as the activity of combining data from multiple file of
table, this is usually done with the help of keys, which help in matching the columns from the
tables that are to be merged.
data_merger.py
import petl as etl
office_data=etl.fromcsv("offices.csv")
#to check the file data
2

ICT705 Data and System Integration
print(etl.head(office_data,5))
print(etl.tail(office_data,5))
phone_data=[["phone","officeid"]]
#cleaning phone numbers using given state code
for rid in range(1,len(office_data)):
state=str(office_data[rid][3])
phone=str(office_data[rid][5])
phone=phone.strip()
regcode=0
if(state.upper()=="NSW"):
regcode=2
elif(state.upper()=="VIC"):
regcode=3
elif(state.upper()=="QLD"):
regcode=7
elif(state.upper()=="SA"):
regcode=9
phone_formatted="+61 "+str(regcode)
phone=phone.replace(" ","")
phone=phone[len(phone)-8:len(phone)]
phone_formatted=phone_formatted+" "+phone[0:4]+"
"+phone[4:8]
print(phone_formatted)
phone_data.append((phone_formatted,office_data[rid][0]))
print(phone_data)
#combining phone numbers cleaned data with other field
3