Mobile Visual Search Application with Landmark Recognition

The proposed project aims to develop a real-time landmark recognition Android app that serves as a digital tourist assistant. The app will enable users to take a picture of a landmark or point of interest, providing relevant information about the image object, such as historical facts and geographical location via GPS. Additional features include user interaction with reviewed local areas of interest, image saving, and social media sharing. Furthermore, the app will incorporate augmented reality technology for guided tours and incorporate APIs from Yelp and Google Maps. The project requires learning resources, Android Studio IDE, Java framework, a substantial dataset/database, Google Maps and Yelp APIs, software for data analysis, an augmented reality platform, and user feedback forms. The expected outcomes include efficient landmark recognition capabilities, large-scale and secure image search facility, and relevant detailed information about landmarks.

Proposed Project Title: Artificial Intelligence & Image Recognition:
Requirements of a High-Performance
Mobile Visual Search System

Brief description of the background to the proposed project:
(i.e. description of problem area, nature of challenge)
Image recognition (or computer vision) is an area of artificial intelligence (AI), or more
specifically machine learning, that looks at a computer's ability to gain a high-level of
understanding from images or video using neural networks.
The aim here is to develop an artificially intelligent image recognition app that is able to
perceive and process visual information, and to then deliver required relevant
information accordingly.
A number of key issues present themselves when considering mobile visual searches
which require access to memory efficient image databases, including the following
points:
1. The ability to develop a secure database representation that can be searched
quickly across many images by an sql query. This will depend on how the database is
set-up and where it is stored (either locally on a device or on a server).
2.The important consideration of possible visual distortions between two given images
(such as geometric or photometric distortions and/or surrounding clutter) and how this
could affect the accuracy of returned results.
These identified factors present a chance to carry out further research through
investigating the best methods/optimisation techniques to use for solving them.
This will also enable the effective incorporation of other related technologies such
navigational image augmentation features for improved functionality.

Brief description of proposed project objectives:
(including scope of proposal, planned success criteria)
Primary Objectives
The proposed project is a real-time landmark recognition Android app that will act
as a digital tourist assistant. It will enable a user/tourist to take a picture of a
landmark/point of interest, and to then receive relevant information about that
given image object (such as it's historical information and/or its opening hours
etc.).
The app will also provide user interaction, with the exact geographical location of
the landmark being displayed via GPS, plus reviewed local areas of interest such
as restaurants or other local attractions.
An option to save the given image will also be provided.
This will also allow tourists to share their real-time experiences on social networks
as well.
Extension Objectives:
Since this app will be capable of recognising both landmarks and the GPS
location, an interesting feature that can be added is the capability to incorporate
overlay pointers onto the image (showing directions for a guided tour etc.).
T
his could be implemented using augmented reality technology, a related branch of
AI and image/object recognition. Augmented reality platforms such as Vuforia
would enable this through the integration of software development kits (SDKs).
Research: Thus far, reading has been carried out about some of the techniques
that can be used for both storing and recognising images for efficient and secure
mobile visual search.
Augmentation techniques for developing this app further have also been
considered and will be continually researched, particularly its involvement (if any)
in landmark recognition.

Brief description of the likely resources needed by the proposal:
(i.e. hardware, software, access to information/expertise, user
involvement, access to rooms.)
Some required resources include:
 Learning resources
 Android Studio IDE
 An object orientated Java neural framework for creating and training
neural network algorithms for image recognition
 A substantial and secure dataset/database of local area landmarks for a
chosen area
 Google Maps and Yelp APIs
 Appropriate software for data analysis and representations
 An augmented reality platform
 A smart phone to test the app created
 User feedback forms
 Survey participants (minimum 6)

Although an Android app is being created for this project, cross-platform
development solutions are factors which will be considered later on in the project if
the schedule allows.
Brief description of the likely outcomes and deliverables from the
proposed project:
(e.g. plans, documentation, use and/or production of software etc.).
On completing this project, it is hoped that a sufficient understanding of neural
network architectures, image recognition and mobile visual search optimisation
techniques/will be gained. Digital augmentation and its relation to image
recognition will also be research thoroughly and incorporated accordingly.
The application created will have efficient landmark recognition capabilities, a
large-scale and secure image search facility and the ability to provide relevant
detailed information about a given landmark (such as historical facts and
geographical information via GPS).
Both the dataset and GPS will not need an active network connection in order
to function.
Additional sources of information will also be available through the integration of
local area information and reviews though the use of APIs such as Yelp or
Google reviews when a network is available.
Augmented reality image navigational pointers will also be added to this device.
The app will include a user guide containing details about the range of different
functions available and how to use these effectively.
Java is a high-level object orientated programming language that has been
mastered to a proficient standard.
For this reason, the aim is to use an object orientated Java neural network to
create optimised image recognition algorithms, with the object orientated
framework having the potential of providing improved software extensibility and
maintainability.
Android Studio will be used as it is also a well known development and testing
environment. Because it uses Java for front-end design, the app’s front-end will
also be coded in Java.

When considering the design of this app, both accessibility and functionality will be
at the fore-front. This will include the use of high-contrasting colours, bold readable
text and clearly defined buttons for easy operation and navigation.
Databases and networks used will be secure and efficient.
Legal requirements from the collection and storage of data from organisations such
as Yelp and Google Maps will also be considered and their terms of use adhered to.
Deliverables will also include an extensive project report (containing project
management strategies and plans) along with a presentation describing the
development process and functionality of the app created.
Bar graphs will be used to monitor location recognition accuracy percentages
against different testing conditions.
These will then be displayed using pie charts for displaying analysis results.
User surveys/feedback may also be used to gain a better understanding of any
UX/UI improvements that may be required.
A Gantt chart has been produced in order to manage the progress of this project
and to monitor tasks that have or have not been completed against set time scales.
The following documents will also be created:
 User Stories: to establish possible user requirements throughout the entire
project
 Data Flow Diagram: for outlining the system architecture and the flow of
data
 Activity Network Diagram: to illustrate the sequential relationship of
activities using arrows and/or nodes and for establishing a critical path
This project will take approximately six months to complete, allowing enough time
for testing, reflection and evaluation.