Report on Bio-inspired AI and Natural Language Processing Techniques

This report provides an overview of Natural Language Processing (NLP) and its evolution, highlighting the shift from rule-based systems to more sophisticated approaches. It discusses the core concepts of NLP, including natural language understanding and generation, and its applications ...Read More

Assignment 2- Bio inspired AI
Assessment Feedback
Specific aspects of the assignment that
the marker likes:
Specific aspects of the assignment that need
more work:
Tutor’s Signature: sjt Date: Grade:
NB. By entering your name(s) and student ID(s) you are asserting that this submission is
entirely your own individual (or group)

Natural Language processing
Introduction
Natural Language processing is one the discipline of computer science, artificial intelligence,
and computational linguistics that concentrate on augmenting and concentrating systems that
majorly allow computers to communicate with human beings people using everyday language.
Example of such languages include French, Portuguese, and English etc. (Christopher D.
Manning, 1989) Natural languages has advanced from generation to generation and it is difficult
to explain with straightforward rules. Below is a simple illustration on natural language
processing.
Language
Computer
Language
From the above illustration, we are able to conclude that NLP is all about Natural language
understanding and generation.
Advancement that are establish on natural language processing are becoming more widespread
daily with the new technologies coming up due to the changes in our modern digital world . A
very good example of such include tablets, phones and mini computers that are able to greatly
provide predictive text and handwriting recognition .Web search initiators provide access to
information that are interlocked in unstructured text.
Basically machine translation gives insight on how to retrieve texts written in Portuguese and
read them in English. By yielding more advanced technological natural human-machine
interfaces, and more superior access to stored information, language processing is generally now
at the Centre our multilingual world.

Background and Theory
Natural language processing systems involve having inputs as words or rather sentences and
then evaluating the same to produce outputs which are a well-structured representations of this
string of words. A natural language understanding system used as an interface to a database
usually accept questions in English that correlates with type and kind of data stored in the
database.
The first use of computers to manipulate natural languages is dated back in the 1950s with
various attempts to develop an automatic system to translate Russian to English and vice versa.
(Nitin Indurkhya, 2010) These systems failed since they did require a human Russian-English
translator to be able to pre-edit the Russian and post-edit the English. Based on World War II
code breaking techniques, individual words was taken in isolation and scrutinize their
definition in a dictionary but They were of little importance practically .
Generally most stories arising from these systems cite many incorrectly translated words and
phrases translations .Examples include the phrase "hydraulic ram" translated as "water goat".
(Christopher D. Manning, 1989)
During the 1960s natural language processing systems started to review sentence structure but
this led to a weird manner in which they were presented. The basic of this systems were based on
pattern matching and less derived meaning representations.
various sophisticated developments in natural language processing was more evident in the early
& mid 1970s as systems begin to use more unspecific approaches and steps to formally
describe the rules of the language they worked with. LUNAR was able to provide an English
interface to a database that put together details of moon rock samples. (Church, n.d.) SHRDLU
interfaced with a virtual robot in a world of blocks, had a capacity of accepting English
commands to move the blocks around and provide answer questions about the state of the world.
Since that period, several parallel development of advanced ideas and technologies that formed
the basis for modern natural language processing systems. (ayes & Carbonell, 1983)

Various research undertaken in computer linguistics has provided greater understanding and
knowledge of grammar construction and basically artificial Intelligence researchers have led to
more effective and convenient mechanisms for analysis of natural languages and for representing
meanings. Thus natural language processing systems has now form a solid ground of linguistic
study and use highly sophisticated semantic representations.
In the 1990s natural language systems have majorly focused on specific subject with the aim of
providing a general purpose language understanding ability but there was much success.
Essentially major objective in contemporary language processing research is to come up with
systems which work with complete threads of discourse
Discussion
Techniques applied in general to Natural Language Processing.
Natural Language Processing is basically a field of diverse computing that focused on the
interaction between human languages and computers. Several challenges are associated with
Natural language processing and among them are natural language understanding, that is,
equipping computers to extract meaning from human or natural language input, and others
involve natural language generation.
Natural language interaction with computers has long been a vital goal of Artificial Intelligence
both for what it can tell us about intelligence in general and for its practical utility, data bases,
software packages. (ayes & Carbonell, 1983)

Essentially there are three techniques applied in general to Natural Language Processing. These
techniques are briefly discussed below.
Genetic Algorithms is among the technique of natural language processing. In this technique
Chromosomes generally represent context-free grammars composed of variable-length strings
of non-terminals and pre-terminals.
Genetic algorithm techniques is used for solving both constrained and unconstrained
optimization problems based on a natural selection process that imitates biological evolution. In
this regard the algorithm repeatedly modifies a population of individual solutions. Search
methods have been incorporated in this technique and they are real time applications.
A Genetic Algorithm is a repetitive technique that applies stochastic operators on a wide
tentative solution. Basically a fitness function allocates a real value to every individual
indicating its suitability to the problem. Traditionally, Genetic Algorithms are associated to the
use of a binary representation, but nowadays you can find genetic algorithms that use other types
of representations.
In genetic algorithm recombination of operator on two solutions is applied and addition of one
mutation that randomly modifies the individual contents to promote diversity and thus reaching
new portions of the search space not necessarily present in the previous generations.
Genetic algorithm has been largely applied in statistical models to deal with Natural Language
Processing tasks require designing specific algorithms to be trained and applied to process new
texts. Genetic Algorithm has an advantage of being attractive since they offer a general design,
yet providing a high performance in particular conditions of application. Many works which
apply genetic algorithm to different Natural Language Processing problems, includes syntactic
and semantic analysis, grammar induction, summaries and text generation, document clustering
and machine translation. (Araujo, 2007)
Neural Networks technique are fundamentally a computational models based on biological
neural networks, and are used to approximate functions that are generally unknown. Specifically,
they are inspired by the behavior of neurons and the electrical signals they convey between input

such as from the nerve endings in the leg, processing, and output from the brain such as reacting
to change in weather.
neural networks has been largely applied in pattern matching which involves taking and
recognizing input utterances as a whole by matching them against patterns of words. Each
pattern is given an interpretation and therefore for every matched pattern, an interpretation is
returned as output.
ELIZA system is a very good example of a system which basically applies pattern matching
parsing is the ELIZA system and it is illustrated below. In this system, the internal interpretation
is removed but a response is returned for every input sentence instead.
(ayes &
Carbonell, 1983)
For example for a sentence "They ran after me." A possible reply would be "Why do you think
they ran after you?" To limit the number of patterns used, hierarchical pattern matching is used.
What happens is that patterns might match only parts of a complex input sentence. These
matched parts will be eventually replaced by some different results for the internal
representation.
In this method, different higher level patterns can then matched on these results in a similar way
till the whole sentence is finally matched into a single internal representation. In addition to
matching patterns of words, it is also possible to analyze natural language input by matching
patterns of semantic elements, with potentially very good results.
Another technique is Ant colony optimization (ACO) which is basically an optimization
technique that was motivated by the focus on foraging behavior of real ant colonies. When this
method comes into place, its focused was generally on the application to discrete optimization

challenges. Recent research efforts led to the development of algorithms which eventually led to
the application to continuous optimization problems.
Ant colony optimization Ant colony was extended and apply on one of the most successful
variants for the training of feed-forward neural networks. For evaluating our algorithm we apply
it to patter classification problems from the medical field. The results show that our algorithm is
comparable to specialized algorithms for neural network training, and that it has advantages over
other general purpose optimizers. (Blum & IRIDIA, n.d.).
Ant Colony Optimization has been specifically applied in syntactically driven parsing. Syntax is
the ways that words can fit together to form higher level units for examples clauses and
sentences. Syntactically driven parsing is therefore interpretation of larger groups of words that
are built up out of the interpretation of their syntactic constituent words.
Generally this is the other side of pattern matching simply because the interpretation of the input
is done as a whole. Syntactically driven parsing is normally done by first constructing a
complete syntactic analysis of the input utterance and then developed an internal representation,
which is easily understood by the computer, from it.
A simple form of syntactic analysis is to create a parse tree using context-free grammar. An
example of parse tree is as below
(ayes & Carbonell, 1983)
S=sentence, NP=noun phrase, VP=verb phrase, PP=prepositional phrase, N=noun, V=verb,
AT=article, P=preposition
Syntactic analyses are obtained by application of a grammar that determines what sentences are
legal in the language that is to be parsed. (MichaelCollins & MITCSAIL, 2005)

Finally is a technique referred to as transformational grammar which basically a
transformational grammar that is a generative grammar and it majorly involves the use of defined
operations referred to as transformations to generate new sentences from existing ones.
Natural language analysis based on semantic grammar is almost similar to syntactically driven
parsing except that in semantic grammar the categories used are defined semantically and
syntactically (Leininger, 1984)
Software used in natural language processing
Several software have been developed to aid Natural language processing. Recent work has
focused on incorporating multiple sources of knowledge and information to assist with analysis
of text, as well as applying frame semantics at the noun phrase, sentence, and document level.
Below is a brief introduction at Google about Natural language processing
Google research on Natural Language Processing (NLP) provides more attention on algorithms
that basically apply at scale, across languages, and across domains. Systems also available at
Google are used in numerous ways across Google, creating user experience in search, mobile,
apps, ads, translate and more.
The work extension at Google offer a range of traditional NLP tasks, with general-purpose
syntax and semantic algorithms which are more specialized systems. Moreover, they are
specifically interested in algorithms that scale well and can be run efficiently in a highly
distributed environment.
Syntactic systems also present at Google predict part-of-speech tags for each word in a given
sentence, as well as define features such as gender and number. They also label link between
words, such as subject, object, modification, and others. The focus is generally on efficient
algorithms that leverage large amounts of unlabeled data, and recently have incorporated neural
net technology.

One of the software used in language processing is Python. Python is a widely used high-level,
general-purpose, interpreted and dynamic programming language. Its design focuses on code
readability where its syntax allows programmers to express concepts in fewer lines of code than
what is possible in languages such as C++ or Java.
Natural Language Toolkit is a leading platform for building Python programs to work with
human language data. Python allows you to type directly into the interactive interpreter the
program that will be running your Python programs.
The Python interpreter basically changes the prompt from >>> to ... after interacting with the
colon just at the end of the first line. The ... prompt function indicates that python should be
ready to accept an indented code block to appear next. It is up to a user to do the indentation, by
typing four spaces or pressing the tab key. To finish the indented block just enter a blank line.
(steven bird, n.d.)
Program code
(steven
bird, n.d.)

Advantages of Natural language processing
Natural language processing has some of positive implications in the society. The natural
language processing does not require training and it relieves burden of learning syntax.
Challenges of Natural language processing
Because of some fundamental limitation that cannot be overcome natural language processing
technology has not achieved a major impact on society. Some of the challenges met are:
1. Scalability in domains, most reliable solutions work for specific domains. Generic solutions
are handicapped in one way or another.
2. Scalability in languages. English gets the lion's share of attention for obvious reasons. Even
languages with tens of millions of speakers are under-served.
3. Scalability in medium. Processing speech makes text-based challenges look like a child's
game. Even in text, colloquial and misspelled content (e.g. social media) is a lot more difficult to
work.
4. Very long sentences and cross-sentence links.
CONCLUSION
Natural language processing basically deals with the incorporation of computational models that
are able to create text or speech data. It is thus a very important field in our modern world since
it finds application in areas such as automatic (machine) translation between languages which is
real time; dialogue systems, which enables a human to interact with a machine using natural
language; and finally in information extraction.
In this report I have a clear explanation of what natural language processing is and how it can be
applied. I have also explained about the software that is used in language processing. Thus I have

explained python programming that uses natural language processing toolkit for program editing.
I have demonstrated use of the software with a python code.
In the report there is a good explanation of three techniques used in the natural language
processing. These are neural networks, genetic algorithm and Ant colony Optimization. There is
also discussion on the application of the three techniques. The laboratory experiment and report
is therefore successful in achieving objectives intended.

References
Araujo, L., 2007. Artificial Intelligence. How evolutionary algorithms are applied to statistical
natural language processing.
ayes, P. J. H. & Carbonell, J. G. J. G. )., 1983. A tutorial on techniques and applications for
natural language processing.
Blum, C. & IRIDIA, S. K. S., n.d. Training feeed-forward neural networks with ant colony
optimization.
Christopher D. Manning, . S., 1989. Foundations of Statistical Natural Language Processing.
s.l.:s.n.
Church, K. a. E. H., n.d. overview of natural language processing.
Nitin Indurkhya, . J. D., 2010. Handbook of Natural Language Processing,. Second Edition ed.
s.l.:s.n.
steven bird, E. k. ,. k., n.d. Natural language processing with python. s.l.:s.n.