Data and System Integration Project: Clinic Location Finder System

This project focuses on data and system integration, specifically addressing the challenge of locating clinics using integrated data. The project begins with an abstract outlining the use of information services and computing architecture, with Python code employed for the data merging and cleansing processes. Key components include data merging and cleansing using Python packages like PETL, RESTful Web Services for accessing and managing resources, and Mashups for combining data from various sources. The project demonstrates how to combine CSV and XML files, clean the data, and use RESTful web services to search for clinic locations. The code includes explanations for data merging, web service implementation, and location search functionality. The final outcome allows users to easily find the geolocation of clinics and display the location on Google Maps. The project concludes with a discussion of the integration process, scalability, and the technologies utilized, including references to relevant research papers.

DATA AND SYSTEM
INTEGRATION
2018

Abstract:
The Integration system discuss about the information services and computing architecture.
While python code is used for this second half of this section. The computing system is the
functional one. The xml and csv are combine to merge the two file is importing thing. The
merged file is called as saved as data_merge.py. The main part of this project is to clean the
data using the data cleansing method. If the data is merging used to the cleaned data. The user
easily find out the exact location of the given clinics data easily. To know the geolocation of the
clinics address. Finally, user easily find out the current location.
1

Table of Contents
Abstract:........................................................................................................................................1
1.0 Introduction:..........................................................................................................................4
2.0 Key System Components:.................................................................................................4
2.1 Data Merging and cleansing:............................................................................................4
2.2 RESTful Web Services:.......................................................................................................5
2.3 Mashups:................................................................................................................................6
3.0 Demo Running Instruction:...............................................................................................7
Combine the two files................................................................................................................7
Code explanation:.......................................................................................................................7
Restful web services:................................................................................................................8
Code Explain:...............................................................................................................................8
To search the Location:............................................................................................................9
Code Explain:.............................................................................................................................10
4.0 Conclusion:..........................................................................................................................10
Reference:...................................................................................................................................10
2

Table of Figures
Figure 1 combine two files...........................................................................................................8
Figure 2 Restful web services.....................................................................................................9
Figure 3 location search by user..............................................................................................10
Figure 4 Display the location in Google Map..........................................................................10
3

1.0 Introduction:
Data Integration is widely used to merge the data easily .If the data takes from
different sources. The main purpose to retrieve the data easily and simply. It merge the data
from different database. The integration is used for the merging process (Huang and Zhu, 2013).
The different techniques are used for this data integration namely enterprise application and
process management. It focus to increasing the number of the customer. The system
integration is existing. The important role of the integration system is connected the internet.
The system integration easily to satisfy the user requirements. Can’t access all the technology, it
only access the trending new technology. The system integration is mainly used for the cloud
services.
2.0 Key System Components:
2.1 Data Merging and cleansing:
Data Merging is the process, the aim of the process id to combine the different
type of the data folder. It combine the more than one file. The database should contain the
table stored the data. The cleansing is the one type of detecting the data and correct it. After
completing the work, it should be compare the other system.
PETL python package
Petl is a python package index. The following command is used to describe the pip
$ pip install petl
And to download manually, extract and run by following command
python setup.py install
To verify the installation following command is used
$ pip install nose
$ nosetests –v pet1
We are using the python version 2.7 and 3.4. The UNIX and WINDOWS operating system
is used to perform python.
ETL pipelines
4

Using this package we can easily avoid the lazy evolution and iterations. The pipeline will
not execute accurately, until the data is required.
For instance
>>> example_data = """foo,bar,baz
... a,1,3.4
... b,2,7.4
... c,6,2.2
... d,9,8.1
... """
>>> with open('example.csv', 'w') as f:
... f.write(example_data)
...
petl.util.vis.look() is a calling function. Using this function easily write the data and files
or database.
Following codes are some examples
petl. Io csv. tocsv()
petl.io.db.todb()
Table containers are used to perform the data extraction. Each table contains table
containers and table iterations. First we need to accept the requested data otherwise the actual
transformation is not done. All the transformations are run using pipeline.
2.2 RESTful Web Services:
It is one type of architecture. It based on the Web Services. RESTful Web Services
is contain the resources of the system and organization. It is the type of protocols. It is the open
5

source protocols. And RESTful web services give the access to the web and for the client access
and the resources is the kind of object-oriented contents. And the resources are represented in
the way of understandability, completeness and link ability (Wang, Shen and Sun, 2013). The
client and server should understand the resource and analyze what kind of resource. And the
resources should be complete and can be access by others and in the link ability it should have
link with others and in any way finally it deliver the xml format. RESTful web services uses the
protocol like HTTP for the data communication. And the request and response are delivered in
the form of HTTP response. In the request message verb, uniform resource locator and for that
request body and header must be needed. And for the response the version of HTTP and the
status and for this also the body and header also needed. And the RESTful web services used for
the web applications built in different programming languages and it is used to for the
applications that can be reside in the platforms like windows and Linux and it provides the
flexibility to the web services can easily communicate with others (Maritime over the Horizon
Sensor Integration: High Frequency Surface-Wave-Radar and Automatic Identification System Data
Integration Algorithm, 2018). And in the RESTful web services addresses should be like use noun as
plural and keep the backward compatibility and the usage of lower case letters and ignore the
spaces. HTTP protocol performs the REST architecture style. In RESTful design is based on the
following concepts.
 Uniform resource identifiers are used. Basically it’s a hierarchal structure based. Each
and every resource must have at least one URI.
 To read and manipulate the resources using uniform interfaces
 There are four basic HTTP operations are applicable. They are GET, POST, PUT, and
DELETE.
 And also support other operations such as HEAD, OPTIONS
 Using these operations the meat data is easily performed.
 The accepted messages are represented in different formats like HTML, JSON, or XML.
So self-descriptive messages are applicable
 It’s also known as stateless interaction. The session date is does not maintain by the
stateless interaction.
6

 That means all the information’s are meaningful including the HTTP message
 For instance resource name and message
2.3 Mashups:
It is the one type of web application. The mashup is the Greek word comes from
British. The overall mashup contains the Web application and Web pages. The main purpose of
the mashup are given namely, they are Combination, Aggregation and visualization. The
mashup technologies include the gadgets and widgets and JavaScript and supporting languages
like css and html and interfaces like active x controls and the java applets and these are all in
the presentation kind of mashups (Estridge, Eveleigh and Tanju, 2016). And in data related
mashups it includes the data combination and the sites of webpages and the languages like xml
and json and this data related or data oriented services also called as data as a service and it is
used to provide the application programming interface and the components and in this data
mashups the process is vary in the form in and out and in process data mashups it uses the
technology like JavaScript and Ajax and it includes the configuration of data retrieving from the
servers and about the user interface and in the out process mashups it is used the technologies
like java and python and used to create new data models. And in the process related mashups it
includes the functions and the programming languages like python and java and in that the
function is used to the inter process communication (Arputhamary and Arockiam, 2015). And the
data is exchanged in between the processes and the results are different in each data oriented
process. The mashup mechanism is known as mashup logic. The mashup logic is used to tell
about how the mashup operators are created and how the mashup components are created.
They are used to specify some of operators between the different components. Control flow,
data flow, and data mediation, are types of operations performed by mashup tools.
3.0 Demo Running Instruction:
Combine the two files.
7

Code explanation:
The python program that is “dataMerger.py” is used to merge given content and the files are
imported by using the keyword “import” and every attributes in the codes are used to form the
tree and the web service side the python program that is “clinics_locator.py” used to search
and locate the address in the nearest tab.
Figure 1combine two files
Restful web services:
Code Explain:
Which was carried out to the execution of python files and save the location on .csv files.
For show the results of the operation “import csv” was used. For opening the information
“clinicopen()” was used. To read the file we need to use “clinicFileReader()” was used. If length
of the purpose we need to use the “If(len !=row)”. To increase the no of rows we need to use
“ScoreList[]=ScoreList[]+row”. For exiting from the file we need to use “ScoreFile.close”.
8

Figure 2Restful web services
9

To search the Location:
Figure 3location search by user
Displayed the Location (Google Map):
Figure 4Display the location in Google Map
10

Code Explain:
The clinics_html file used to show the geolocation and direction of the clinic wants to know.
We can able to see the MAP which contains the direction for the clinic location. It very useful
show the location of the clinics services location easily.
4.0 Conclusion:
Finally , Integrating the information and demonstrations are also performed. The
position of the stores in the MAP was identified. The IT structure are mainly used to access the
data centers and based on the Functionality of the dependent on the type of the Infrastructure.
Growth of the process is slowly increasing and non – dynamic. The techniques are used to
compute the responsibilities of the system. The system integration of various data the final
required data was recovered. Scalability of the system was ensured by the virtualizing
techniques.
Reference:
Arputhamary, B. and Arockiam, L. (2015). Data Integration in Big Data Environment. Bonfring
International Journal of Data Mining, 5(1), pp.01-05.
Estridge, C., Eveleigh, T. and Tanju, B. (2016). Integration of System Modeling and Design
Processes, Data and Technology for Streamlining Enterprise Integration. Procedia
Computer Science, 95, pp.311-318.
Huang, X. and Zhu, W. (2013). An Enterprise Data Integration ERP System Conversion System
Design and Implementation. Applied Mechanics and Materials, 433-435, pp.1765-1769.
Maritime over the Horizon Sensor Integration: High Frequency Surface-Wave-Radar and
Automatic Identification System Data Integration Algorithm. (2018). Sensors, 18(4), p.1147.
11