Bioinspired AI Project: Natural Language Processing Methods Explored

This project provides an in-depth exploration of bioinspired AI, specifically focusing on natural language processing (NLP) techniques. The project begins with an introduction to NLP, defining its role in computer science, artificial intelligence, and computational linguistics, and highlighting its evoluti...Read More

Assignment 2- Bio inspired AI
Assessment Feedback
Marking
Scheme
A+ A -to A B C D F/G
Task
Feedforward
(10%)
As B
Must also include an
example you have taken
from another source
external to the university.
Included more
than two
feedback
examples and
stated how,
with more than
basic details,
they will be
used within the
assignment
At least one
example can
from this
module but
also at least
one different
example comes
another
module.
Included
more than
one feedback
example and
stated how
the feedback
will be used
within this
assignment
but with
basic details
provided .
All examples
can from this
module.
Just stated one
feedback
example.
No
attempt
Program
Success
As A-
to A
As B but the
model is based
As C
Successful
Re-used
code or
Re-used code or
software that is
No
attempt

(25%) but
written
by the
student
based
on
their
own
ideas.
around a neural
network /
genetic
algorithm/ACO.
Testing must
demonstrate the
success of the
software used
in the context of
the problem set
implementation
of a model
published in a
conference or
journal paper.
Testing must
demonstrate
the success of
the software
used in the
context of the
problem set
software that
is available
(not
necessary to
write your
own)
Testing must
demonstrate
the success
of the
software
used in the
context of
the problem
set
available (not
necessary to
write your own)
Limited Testing
or the testing
unsatisfactorily
demonstrate the
success of the
software used in
the context of the
problem set
(G)
Very
limited
attempt(F)
Conclusions
(30%)
As B but there is novelty
to the conclusions.
Conclusions
drawn are
based on both
literature
review and
satisfactory
evidence of
drawn from the
software used.
Conclusion
show clear
evidence of
insight arrived
at through
these sources
Conclusions
drawn are
based on
literature
review and
limited
evidence of
being drawn
from the
software
used.
Limited evidence
that conclusions
are drawn are
based on
literature review,
and limited
evidence of being
drawn from the
software used.
No
attempt
(G)
Very
limited
attempt(F)
Research As B over 20 references As C but must An As C but limited No

(25%) used but
- At least 10 are
journal articles
from within the
last five years.
- Of the remaining
references at least
five are from
books or older
journal
- No blogs or wikis
used.
include
references
from three
IEEE journals.
Literature
review draws
together all the
material to
draw a
conclusion.
appropriate
level of
discussion of
one or more
of the
techniques
applied to
Natural
Language
Processing.
Needs to be
appropriate
level of
evidence of
reading
around the
topic.
discussion of the
techniques OR
limited
discussion or
misunderstanding
on Natural
Language
Processing.
attempt
(G)
Very
limited
attempt(F)
Report
writing
(10%)
As B with excellent
written presentation.
The work is also meets
the requirements for the
following in:
- Program success.
(A+)
- Research (A
range)
- Conclusions (A
range)
Structure
follows
guidance in the
deliverables
section.
Clarity of the
points made is
of a high
standard.
Appropriate
Harvard
referencing
with no faults.
Structure
follows
guidance in
the
deliverables
section.
Satisfactorily
clearly
written.
Appropriate
Harvard
referencing
with minor
Structure does
not follows
guidance in the
deliverables
section.
Appropriate
Harvard
referencing
Very
Limited
or no
attempt

faults
G
Specific aspects of the assignment that
the marker likes:
Specific aspects of the assignment that need
more work:
Tutor’s Signature: sjt Date: Grade:
NB. By entering your name(s) and student ID(s) you are asserting that this submission is
entirely your own individual (or group)

Do Not handed-in with your submission, either Self Feedback Form A
Self-feedback form A- issues to consider.
To consider. Yes/No
Is your whole document below 20MB? Yes
Is the front sheet included? Yes
Task 1
More than two examples? Yes
Several sources used? Yes
Task 2
Is the report structure clear? Yes
Have you put references to sources at appropriate places? Yes
Have you included an introduction? Yes
How you include background theory? Yes
Is the theory relevant to the task? Yes
Have you included details of how the values in the sequences
were arrived at?
Yes
Do the extracts of code (if any) in the report adequately
highlight the work you have produced?
Yes
Is there adequate discussion of the results? Yes
Have you adequately discussed the strengths and weaknesses of
the approach?
Yes
Have you included discussion of different approaches that could
have been considered if the work was to be repeated?
Yes
Have you consider within the report if there are potential uses
for a solution like yours?
Yes
Can you link your solution with ideas of others? If so is there
appropriate referencing to the sources?
Yes
Have you used Harvard referencing throughout? Yes

Task 1
Natural Language processing is one the discipline of computer science, artificial intelligence,
and computational linguistics that concentrate on augmenting and concentrating systems that
majorly allow computers to communicate with human beings people using everyday language.
With time there has been advancement in how natural language processing has evolved .In my
report ,I have discussed the various techniques applied in natural language processing and their
respective application .I have also briefly explained about python language and how it has been
applied in natural language processing
Task 2
Natural Language processing
Introduction.
Natural Language processing is one the discipline of computer science, artificial intelligence,
and computational linguistics that concentrate on augmenting and concentrating systems that
majorly allow computers to communicate with human beings people using everyday language.
Example of such languages include French, Portuguese, and English etc. (Christopher D.
Manning, 1989) Natural languages has advanced from generation to generation and it is difficult
to explain with straightforward rules. Below is a simple illustration on natural language
processing.
Language
Computer
Language

From the above illustration, we are able to conclude that NLP is all about Natural language
understanding and generation.
Advancement that are establish on natural language processing are becoming more widespread
daily with the new technologies coming up due to the changes in our modern digital world . A
very good example of such include tablets, phones and mini computers that are able to greatly
provide predictive text and handwriting recognition .Web search initiators provide access to
information that are interlocked in unstructured text.
Basically machine translation gives insight on how to retrieve texts written in Portuguese and read
them in English. By yielding more advanced technological natural human-machine interfaces, and
more superior access to stored information, language processing is generally now at the Centre
our multilingual world.
Background and Theory
Natural language processing systems involve having inputs as words or rather sentences and then
evaluating the same to produce outputs which are a well-structured representations of this string
of words. A natural language understanding system used as an interface to a database usually
accept questions in English that correlates with type and kind of data stored in the database.
The first use of computers to manipulate natural languages is dated back in the 1950s with various
attempts to develop an automatic system to translate Russian to English and vice versa. (Nitin
Indurkhya, 2010) These systems failed since they did require a human Russian-English translators
to be able to pre-edit the Russian and post-edit the English. Based on World War II code breaking
techniques, individual words was taken in isolation and scrutinize their definition in a dictionary
but They were of little importance practically .
Generally most stories arising from these systems cite many incorrectly translated words and
phrases translations .Examples include the phrase "hydraulic ram" translated as "water goat".
(Christopher D. Manning, 1989)
During the 1960s natural language processing systems started to review sentence structure but this
led to a weird manner in which they were presented. The basic of this systems were based on
pattern matching and less derived meaning representations.

various sophisticated developments in natural language processing was more evident in the early
& mid 1970s as systems begin to use more unspecific approaches and steps to formally describe
the rules of the language they worked with. LUNAR was able to provide an English interface to a
database that put together details of moon rock samples. (Church, n.d.) SHRDLU interfaced with
a virtual robot in a world of blocks, had a capacity of accepting English commands to move the
blocks around and provide answer questions about the state of the world. Since that period,
several parallel development of advanced ideas and technologies that formed the basis for modern
natural language processing systems. (ayes & Carbonell, 1983)
Various research undertaken in computer linguistics has provided greater understanding and
knowledge of grammar construction and basically artificial Intelligence researchers have led to
more effective and convenient mechanisms for analysis of natural languages and for representing
meanings. Thus natural language processing systems has now form a solid ground of linguistic
study and use highly sophisticated semantic representations.
In the 1990s natural language systems have majorly focused on specific subject with the aim of
providing a general purpose language understanding ability but there was much success.
Essentially major objective in contemporary language processing research is to come up with
systems which work with complete threads of discourse
Discussion
Techniques applied in general to Natural Language Processing.
Natural Language Processing is a field of computing looking at the interaction between human
languages and computers. Many challenges in Natural language processing involve natural
language understanding, that is, enabling computers to derive meaning from human or natural
language input, and others involve natural language generation. Natural language communication
with computers has long been a major goal of Artificial Intelligence both for what it can tell us
about intelligence in general and for its practical utility, data bases, software packages. (ayes &

Carbonell, 1983).There are three techniques applied in general to Natural Language Processing.
These techniques are as discussed below.
Genetic Algorithms is one technique of natural language processing. Chromosomes represent
context-free grammars with variable-length strings of non-terminals and pre-terminals. Genetic
algorithm is a method for solving both constrained and unconstrained optimization problems
based on a natural selection process that mimics biological evolution. The algorithm repeatedly
modifies a population of individual solutions. They are search methods that have been
successfully applied in many real applications.
A Genetic Algorithm is an iterative technique that applies stochastic operators on a pool of
tentative solutions. A fitness function allocates a real value to every individual indicating its
suitability to the problem. Traditionally, Genetic Algorithms are associated to the use of a binary
representation, but nowadays you can find genetic algorithms that use other types of
representations. A Genetic algorithm usually applies a recombination operator on two solutions,
plus a mutation one that randomly modifies the individual contents to promote diversity and thus
reaching new portions of the search space not implicitly present in the previous generations.
One of the specific applications of genetic algorithm is that in statistical models applied to deal
with Natural Language Processing tasks require designing specific algorithms to be trained and
applied to process new texts. Genetic Algorithm is attractive since they offer a general design, yet
providing a high performance in particular conditions of application. Many works which apply
genetic algorithm to different Natural Language Processing problems, includes syntactic and
semantic analysis, grammar induction, summaries and text generation, document clustering and
machine translation. (Araujo, 2007)
Neural Networks are computational models inspired by biological neural networks, and are used
to approximate functions that are generally unknown. Particularly, they are inspired by the
behavior of neurons and the electrical signals they convey between input such as from the nerve
endings in the hand, processing, and output from the brain such as reacting to heat.
Application of neural networks is in pattern matching. Pattern matching involves taking and
recognizing input utterances as a whole by matching them against patterns of words. Each pattern
is given an interpretation and therefore for every matched pattern, an interpretation is returned as
output.

An example of a system which uses pattern matching parsing is the ELIZA system. In this
system, the internal interpretation is omitted but a response is returned for every input sentence
instead.
(ayes &
Carbonell, 1983)
For example for a sentence "They ran after me." A possible reply would be "Why do you think
they ran after you?" To reduce the number of patterns used, hierarchical pattern matching is used.
Here, some patterns might match only parts of a complex input sentence. These matched parts
will be replaced by some canonical results for the internal representation.
Other higher level patterns can then matched on these results in a similar way till the whole
sentence is finally matched into a single internal representation. As well as matching patterns of
words, it is also possible to analyze natural language input by matching patterns of semantic
elements, with potentially very powerful results.
Ant colony optimization (ACO) is an optimization technique that was inspired by the foraging
behavior of real ant colonies. Originally, the method was introduced for the application to discrete
optimization problems. Recent research efforts led to the development of algorithms for the
application to continuous optimization problems. In this work we extend and apply one of the
most successful variants for the training of feed-forward neural networks. For evaluating our
algorithm we apply it to patter classification problems from the medical field. The results show
that our algorithm is comparable to specialized algorithms for neural network training, and that it
has advantages over other general purpose optimizers. (Blum & IRIDIA, n.d.).
One specific application of Ant Colony Optimization solution is in syntactically driven parsing.
Syntax is the ways that words can fit together to form higher level units for examples clauses and
sentences. Syntactically driven parsing is therefore interpretation of larger groups of words that
are built up out of the interpretation of their syntactic constituent words.

r In a way this is the opposite of pattern matching as here the interpretation of the input is done as
a whole. Syntactically driven parsing is done by first constructing a complete syntactic analysis
of the input utterance and then create an internal representation, which is easily understood by the
computer, from it. A simple form of syntactic analysis is to create a parse tree using context-free
grammar. An example of parse tree is as below
(ayes & Carbonell, 1983)
S=sentence, NP=noun phrase, VP=verb phrase, PP=prepositional phrase, N=noun, V=verb,
AT=article, P=preposition
Syntactic analyses are obtained by application of a grammar that determines what sentences are
legal in the language that is to be parsed. (MichaelCollins & MITCSAIL, 2005)
Transformational grammar is another language processing techniques. Transformational
grammar is a generative grammar, especially of a natural language, that involves the use of
defined operations called transformations to produce new sentences from existing ones. Natural
language analysis based on semantic grammar is almost similar to syntactically driven parsing
except that in semantic grammar the categories used are defined semantically and syntactically
(Leininger, 1984)
Software used in natural language processing
Natural Language Processing (NLP) research at Google focuses on algorithms that apply at scale,
across languages, and across domains. Our systems are used in numerous ways across Google,
impacting user experience in search, mobile, apps, ads, translate and more.
Our work spans the range of traditional NLP tasks, with general-purpose syntax and semantic
algorithms underpinning more specialized systems. We are particularly interested in algorithms
that scale well and can be run efficiently in a highly distributed environment.

Our syntactic systems predict part-of-speech tags for each word in a given sentence, as well as
morphological features such as gender and number. They also label relationships between words,
such as subject, object, modification, and others. We focus on efficient algorithms that leverage
large amounts of unlabeled data, and recently have incorporated neural net technology.
On the semantic side, we identify entities in free text, label them with types (such as person,
location, or organization), cluster mentions of those entities within and across documents
(reference resolution), and resolve the entities to the Knowledge Graph.
Recent work has focused on incorporating multiple sources of knowledge and information to aid
with analysis of text, as well as applying frame semantics at the noun phrase, sentence, and
document level.
Software used in language processing for example Python. Python is a widely used high-level,
general-purpose, interpreted and dynamic programming language. Its design emphasizes code
readability, and its syntax allows programmers to express concepts in fewer lines of code than
what is possible in languages such as C++ or Java. Natural Language Toolkit is a leading
platform for building Python programs to work with human language data. Python allows you to
type directly into the interactive interpreter the program that will be running your Python
programs.
The Python interpreter changes the prompt from >>> to ... after encountering the colon at the end
of the first line. The ... prompt indicates that Python expects an indented code block to appear
next. It is up to you to do the indentation, by typing four spaces or hitting the tab key. To finish
the indented block just enter a blank line. (steven bird, n.d.)

Program code
(steven bird,
n.d.)
From the above program code, it is evident that python language can be modified to be used in
NLP.
Advantages of Natural language processing
Natural language processing has some of positive implications in the society. The natural
language processing does not require training and it relieves burden of learning syntax.
Challenges of Natural language processing
Because of some fundamental limitation that cannot be overcome natural language processing
technology has not achieved a major impact on society. Some of the challenges met are:
1. Scalability in domains, most reliable solutions work for specific domains. Generic solutions are
handicapped in one way or another.
2. Scalability in languages. English gets the lion's share of attention for obvious reasons. Even
languages with tens of millions of speakers are under-served.
3. Scalability in medium. Processing speech makes text-based challenges look like a child's game.
Even in text, colloquial and misspelled content (e.g. social media) is a lot more difficult to work.
4. Very long sentences and cross-sentence links.

CONCLUSION
Natural language processing deals with the application of computational models to text or speech
data. It is therefore a very important field with application in areas such as automatic (machine)
translation between languages; dialogue systems, which allow a human to interact with a machine
using natural language; and information extraction. In this report I have a clear explanation of
what natural language processing is and how it can be applied. I have also explained about the
software that is used in language processing. Thus I have explained python programming that
uses natural language processing toolkit for program editing. I have demonstrated use of the
software with a python code.
In the report there is a good explanation of three techniques used in the natural language
processing. These are neural networks, genetic algorithm and Ant colony Optimization. There is
also discussion on the application of the three techniques. The laboratory experiment and report is
therefore successful in achieving objectives intended.
References
Araujo, L., 2007. Artificial Intelligence. How evolutionary algorithms are applied to statistical
natural language processing.
ayes, P. J. H. & Carbonell, J. G. J. G. )., 1983. A tutorial on techniques and applications for
natural language processing.
Blum, C. & IRIDIA, S. K. S., n.d. Training feeed-forward neural networks with ant colony
optimization.
Christopher D. Manning, . S., 1989. Foundations of Statistical Natural Language Processing.
s.l.:s.n.
Church, K. a. E. H., n.d. overview of natural language processing.
Nitin Indurkhya, . J. D., 2010. Handbook of Natural Language Processing,. Second Edition ed.
s.l.:s.n.
steven bird, E. k. ,. k., n.d. Natural language processing with python. s.l.:s.n.