ICT_705 Project: Data Integration and Mashup for Clinic Locations

This project report details the development of a mashup application for integrating clinic location data. The process involves cleaning and merging data from CSV and XML files using Python, implementing a RESTful web service with the Bottle framework to serve JSON data, and creating a mashup using HTML, CSS, and JavaScript to display clinic locations on a Google Map based on user-entered postcodes and service IDs. The application allows users to locate clinics and initiate email inquiries, demonstrating a practical approach to data integration and web service utilization. The project leverages technologies like Petl for data manipulation and Google Maps API for location visualization.

ICT_705

EXECUTIVE SUMMARY
I have made this report, which includes all the methods of different kind of programming model
which is mainly used for achieving the data integration into any single data. I have developed a
python program, which is going to clean all the input data & then it will make an input data filter
from the unwanted data. Now the data we have cleaned will go & it will be merged with another
data. The merged data will be fetched by Restful web server data by using bottle framework. By
fetching data, we got JSON data. This will create a flag on the screen map & this will display
proper locality of the clinic so that patients can easily reach them.
i

Table of Contents
EXECUTIVE SUMMARY..............................................................................................................1
INTRODUCTION..........................................................................................................................3
KEY SYSTEM CONCEPTS.........................................................................................................4
Data Cleaning and Merging......................................................................................................4
RESTful Web Service Implementation....................................................................................9
Mashups....................................................................................................................................11
Demo Running Instructions........................................................................................................13
Perquisites.................................................................................................................................13
Data Cleaning Process..............................................................................................................13
RESTful web Service initiation...............................................................................................13
Implementation of Mashup web service.................................................................................15
CONCLUSION.............................................................................................................................16
List of Figures
Figure 1: Clinics (.csv file)...............................................................................................................4
Figure 2: output after cleaning the clinic.csv file.............................................................................5
Figure 3: cliniclocations(.xml file)...................................................................................................6
Figure 4: code to convert XML to csv..............................................................................................7
Figure 5: Email cleaning code from clinic.csv file...........................................................................7
Figure 6 RESTful Web Service Implementation..............................................................................9
Figure 8: initiation of RESTful Web Service.................................................................................11
Figure 9 localhost 8080..................................................................................................................11
Figure 10: JSON data for service ID request= 4............................................................................12
Figure 11: Double click on clinic_map.html to open clinic_map..................................................13
Figure 12:web page........................................................................................................................13
ii

INTRODUCTION
The main motto of preparing this document is to understand that how we make software. I made
mashup software, which is going to focus on the data integration. Firstly .csv are cleaned & then
these are sent to another place for merging to make a single data. We have the file clinic.csv
which is consisting the entire information about clinics like containing an electronic mail in an
incomplete format which will make next process easier. This mail will be cleared by
automatically. If data cleaning is being in the process then it will also make the process at ease.
There is cliniclocations.xml file which is having all the data of clinic locations. This file is in
XML extension & we will have to convert it into CSV extension.
We need two files for merging to make a single data file. These files are service.csv that contains
all the data related to the services that are provided by the clinics & another file named
clinicservices.csv that contains data for all the clinic's services. Both files are merged & the new
file is made named as clinicservicelocations.csv.
Next process is completely based on the Restful web service server. This is going to make a web
server, which runs only on the bottle framework. This server will take the input from the get
clinic serviceID=x where x refers to the clinic ID, which will enter by the user or patient. For the
response to those request, web service server will go & respond with JSON data, which is going
to use later on the mashup demo in task third.
In the next task, mashup service will be used for the JSON data from the previous part or task &
then JSON data will be fetched from the web server. If the user or patient will enter their service
id & postcode than the map will fetch and display all the data from the previous task & then it
will show a flag on the map with complete demographic location. If the user will click on this
flag then a drafted mail will be automatically directed to the patient with the demographic
location with a subject “clinic inquiry”. This project will use petrol module in order to fetch the
information from the .csv file & then I will convert XML file to CSV file & then it will be
automatically merged.
The used technologies are hypertext markup, Cascading style sheets, Javascript, and Python.
1

KEY SYSTEM CONCEPTS
This document mainly focuses on making mashup software, which will help us in making us
complete proposed project which developed using many small parts.
Data Cleaning and Merging
Whenever I am looking for entering the new information, it arises could be of various
organizations like comma delimited qualities (.CSV), tab delimited context (.TXT) design. The
information is ready to coordinate to the comparable sort of information about service location.
This date could be from comparative source, comparative source or may be comparable structure.
Figure 1: Clinics (.csv file)
2

Cleaning of data
Sometimes arose information in the crude configuration have emitted qualities. Now I will have
to look for a cleaning procedure to get those missing information, in which all the omitted
qualities are taken care of information is made significant. Data cleaning will be done in this area
only. I will center on the scrubbing part & after that, I will combine files. This process will be
finished by using python.
Figure 2: output after cleaning the clinic.csv file
So these two documents are as follows:
3

 Clinics.csv: This file is consisting entire information or details like email as metadata,
suburb, name, postcode etc. This file is having entire details which are important to goal
to accomplish the entire outcome which is going to combine information of clinic.csv
with the information to the locations.xml.
 cliniclocations.xml: This file contains all the details about the proper location of the
clinic. This file is containing latitude and longitude points of the clinics with the clinic id.
I will have to convert XML file into the CSV file.
Figure 3: cliniclocations(.xml file)
4

xml to csv conversion code:
Figure 4: code to convert XML to csv
Email cleaning code:
Figure 5: Email cleaning code from clinic.csv file
5

RESTful Web Service Implementation
It’s a type of web design that completely depends on Virtual Hierarchical organizer structure.
This Web service Server acknowledges the variable from the client and demonstrates the
outcome in the like mannered way.
http://localhost:8080/ get clinics? serviceid=x
•/get clinics = this is the capacity in python code which is called each time the customer
composes this in the URL.
•/serviced=4 = this is a service ID request through which JSON data is fetched from the
clinicservicelocation.csv. Service id of psychiatry & properties or qualities of these is shown in
the JSON reaction. Moreover, for this web service server, I have made a python document named
as clinic_locator.py, which will conjure the RESTful web server whenever the user or the patient
enters the connection composed previously.
Composed web information will continue to keep running on any kind of coveted web service
program. In the beginning, RESTful web service server will start and “clinic_locator.py” is keep
running on the command prompt on all the necessities (Hamad, H., Saad, M. and Abed, R.2010).
6

Figure 6 RESTful Web Service Implementation
7

Mashups
Mashup can be defined as a crossbreed application, which is produced from at least two sources,
which are like network application or site pages. The mashups take the use of those applications
to develop the solitary graphical interface or the solitary application. So, in this task of the
project, I have to do this. & these tasks or points are as follows:
•The RESTful Web Service is created which takes the contribution from the yield of the code of
python composed on the framework of handling information.
•The output from the restful web service server which is sustained just as a kind of contribution
to the applications of mashup as JSON details & then further handling is also finished (Wong, J.
and Hong, J.I.2007).
In this, hypertext mark-up language is made with all the assistance of JS (JavaScript) & cascaded
style sheets. I have made a record in this procedure, which is named as clinic_map.html.
Therefore, the main motto of the above mashup is in some points & they are written below:
•I already created hypertext mark-up language site pages will take all the information including
user’s or patient’s postcode, so then he can catch location also.
•HTML page will extract all the information of user or patients from the web service server in the
frame of JSON data & then it will be shown the exact are & correct Google location.
For demonstrating, I have used the Google Maps that uses Google API are taken from the
internet. We could demonstrate the exact area & of the center whose postcode has been entered in
the option of search bar (Batty, Hudson-Smith, Milton & Crooks, 2010).
Mashup Sources:
There are two sources of mashups are as follows:
 The main motive of the developing RESTful Web Service is to initiate Web Service
server.
 File cliniclocationservice.csv document that is prepared as a data procedure’s first step on
the cliniclocations.xml and Clinics.csv records i.e. Data cleaning and merging step.
8

9

Demo Running Instructions
Three tasks are implemented to complete these tasks that are Data Cleaning and Merging,
Making RESTful Web Service, Making a Mashup from the previous steps.
Perquisites
Two modules are used for this program i.e. Bottle framework and Petl.
Data Cleaning Process
Open Command Prompt; go to the file data_merger.py’s file location using cd command
RESTful web Service initiation
To run the clinic_locator.py open it with Python IDLE and click on run then the following output
will be shown:
Figure 7: initiation of RESTful Web Service
Open web browser and put the below link:
 http://localhost:8080/.
Then a page will be opened as shown below in the screenshot:
10

Figure 8 localhost 8080
Figure 9: JSON data for service ID request= 4
11

Implementation of Mashup web service
Double clicks on open clinic_map.html file. This page will be opened in the default browser.
Figure 10: Double click on clinic_map.html to open clinic_map
When user double clicks on the clinic_map.html file then the index page of the application will
be opened into the default browser.
Figure 11: web page
When user enter postcode, select any services from the drop down list and click on the locate
clinic then the flag for location of the clinic will be displayed in the map. When we put the mouse
on the flag then the location will be shown with mail id to contact through mail.
12

CONCLUSION
I have made this report. This assessment report focuses on creating a Mashup application using
which patients can get the preferred clinic’s location by entering the postcode of that specific
clinic. In the csv, postcode will search and match up with the location if the location is displayed
it will be shown if it is not there the default location will be displayed. To achieve this, several
things are done. At first, the data that is specified is prepared for absent values and after that
merged with the diverse file types. The output is then stored in a file that can done through
clinic_map.html and clinic_locator.py. This step is considered as an important step as it will alter
the entire result. In the next step, the net server is designed using Bottle framework. The
cliniclocationservice.csv file is taken as input and returns JSON data which is later used by
clinic_map.html to search and display locations. In the final step, I have developed the mash up
application using above two processes. This Mashup application displays clinic.py’s location
retrieving postcode by using the above developed files. Also, an Enquiry button is given which
will help the user to send a mail with an already filled subject line "Clinic Enquiry” in it. So that
patients can easily inquiry.
13

References
 Batty, M., Hudson-Smith, A., Milton, R. and Crooks, A., (2010). Map mashups, Web 2.0
and the GIS revolution. Annals of GIS, 16(1), pp.1-13.
 Wong, J. and Hong, J.I. (2007), April. Making mashups with Marmite: towards end-user
programming for the web. In Proceedings of the SIGCHI conference on Human factors in
computing systems (pp. 1435-1444). ACM.
 Hamad, H., Saad, M. and Abed, R., (2010) Performance Evaluation of RESTful Web
Services for Mobile Devices. Int. Arab J. e-Technol., 1(3), pp.72-78.
14