CSI6210: Review of Natural Language Processing Techniques

This report provides a comprehensive review of Natural Language Processing (NLP), a crucial subfield of artificial intelligence (AI). It begins with an introduction to NLP and its significance in enabling human-computer interaction through natural language. The report outlines various AI-based techniques that support NLP, including word embeddings, convolutional neural networks (CNN), recurrent neural networks (RNN), reinforcement learning, and deep generative models. It then offers a comparative analysis of NLP with related techniques like text mining. The report also reviews three scholarly journals and conference papers, highlighting the practical applications of NLP in areas such as machine translation, plagiarism detection, and big data analysis. Finally, the report concludes by emphasizing the importance of NLP in making machines more intelligent and facilitating effective communication with humans. The report is well-structured and includes references to relevant literature.

Running head: NATURAL LANGUAGE PROCESSING
NATURAL LANGUAGE PROCESSING
Name of student
Name of university
Author’s note:

1
NATURAL LANGUAGE PROCESSING
1. Introduction and Background
Natural Language Processing, referred to as NLP is an important subfield of artificial
intelligence or AI. It provides context for human computer interaction through natural
language that is considered for human conversation. Interaction between human and
computers are limited to application of programming language. However, one of the main
objectives of artificial intelligence is to make machines more intelligence which are capable
of processing human queries in most natural way possible and for this, identification and
processing of natural language is of significant importance (Goldberg, 2017). Natural
language processing is therefore becoming an interesting topics among researchers for its
capability in making machines more intelligent and make conversation with humans in
natural language. It will not only make conversation more effective but it will enhance value
of conversation as well. NLP is applied for reading and decoding human language so that it
make sense to machines and provide output that is meaningful and valuable as well.
2. Outline of the methods, techniques that support this technology
Natural language is having a significant potential in transforming applications of artificial
intelligence. However, processing natural language by machines is not an easy thing to
accomplish and therefore, for this various techniques are being created and these techniques
are based on applications of artificial intelligence or AI (Young et al., 2018). In this report
some important AI based techniques for natural language processing are analysed.
Word embeddings:
Distributional vectors also known as word embeddings is an important technique for natural
language processing. This method is based on shallow neural networks, an important aspect
of artificial intelligence (Yin et al., 2017). This technique follows distributional hypothesis
which say that words that appear within same context are having similar meaning. Word

2
NATURAL LANGUAGE PROCESSING
embeddings while applied on a task require some modifications. In order to apply this
technique, the task is first pre-trained and its object is to make prediction of a word according
to its context. To implement this, it requires application of shallow neural networks (Sun, Luo
& Chen, 2017). In shallow neural networks, there is only one hidden layer compared to deep
neural network that where several hidden layers are present. Therefore, this layer is relatively
easy to apply but it is not applicable for process where sophisticated natural language
processing is required. However, word vectors are capable of embedding syntactical and also
semantic information which helps in some important NLP related tasks for example analysis
of sentiment in sentence and composing sentences as well.
A neural language model for word embedding is provided in this context.
Convolutional Neural Network
Convolutional Neural Network or CNN is a technique that is based on neural networking
approach. It provides a feature function which constitute words or n-grams for extracting
higher level features from a language (Goldberg, 2017). This technique is therefore
considered for applications like analysing sentiment in a sentence, machine translation and

3
NATURAL LANGUAGE PROCESSING
question answer which requires advanced language processing. In this technique, words are
transformed into vector and it is then represented with a look-up table. It provides an
approach for word embedding where neural network is trained and it learns weights. This
process is described through a model provided in this context.

4
NATURAL LANGUAGE PROCESSING
Recurrent Neural Network
This technique is mainly considered for processing sequential information. Computation is
applied for every instances of a sequence in a sentence and it is conditioned according to
previous result computed from a sequence. One of the most important feature of this method
is that it is capable of memorizing result that is obtained from previous computation and
therefore, it is applicable for modelling context while considering dependencies in inputs
which may be of any length. It helps in proper composition of input (Li, 2017). If this model
is compared with CNN technique, then it might be similarly effective or in some cases it
might perform better for some specific tasks which require natural language processing, but it
should not be considered superior to CNN model.
This process is described in this context.
Reinforcement Learning
Reinforcement Learning is based on methods of machine learning which an important
application of artificial intelligence is. In this methods, agents are trained for performing
various tasks related to natural language generation for example summarizing text for which
reinforcement learning is required (Do et al., 2019).

5
NATURAL LANGUAGE PROCESSING
Deep Generative Models
In order to ensure effective natural language processing, it is important to discover rich
structure that is associated with a natural language. In order to discover that, a process is
considered where realistic sentences are generated from latent code space (Young et al.,
2018).
Some popular deep generative models are variational autoenconders (VAEs) and generative
adversarial networks (GANs). A figure is provided in this context that provides details
regarding a RNN-based VAE for generating a sentence.
3. A comparative analysis of the technology with other related techniques
In this report, a comparison has been made between natural language processing and other
related techniques.
Natural language processing Text mining
Input text is processed and turned into
structured data. It also consists of components
for example Natural language understanding
It extract information from analysis of text
files

6
NATURAL LANGUAGE PROCESSING
and Natural language generation.
Natural language understanding retrieves
intelligence from textual and also statistical
data while Natural language generation is
considering for transforming structured data
into text and information is written into a
language that is interpreted by human
intelligence (Goldberg, 2017).
NLP deals with text and speech recognition Text mining deals with evaluation of quality
of a text
For natural language processing, knowledge
regarding neural networks and deep learning
is required
For text mining, knowledge regarding feature
hashing, programming language for example
Perl or Python, and statistical models is
required in this context.
4. A review of 3 scholarly journals and conference papers
In this review, some significant applications of natural language processing are also provided
to provide an overview of how natural language processing is transforming industry through
innovation.
As popularity of digitalization is increasing, most of the features are now accessible online.
However, to ensure that digitalization is effective, it is important to ensure that data is not
only available but also accessible and this is a challenging task to accomplish (Khurana et al.,
2017). Language barrier an important issue that contribute to this challenge. Machine
translation which works on application on natural language processing helps in translating

7
NATURAL LANGUAGE PROCESSING
sentence from one language to another language. Google translator, which is statistical engine
helps in translating any language to any preferred language. Although, it now seems to be an
easy process, but without application of natural language processing, it was not possible to
design search application.
When a research paper is submitted for publishing as a conference paper or submitted for
publishing in a journal, it is important to ensure that these papers are free from plagiarism.
Although there are some commercial tools for detecting plagiarism in a paper, there is a
major drawback in these tools and that is these tools are not capable of detecting plagiarism if
grammar of a sentence is changed or if words are replaced by their synonyms. Therefore, to
enhance efficiency of these tools, concepts of natural language processing are applied to
design a plagiarism detection tool (Desai et al., 2016). It is not only capable of analysing
sentences in a paper, but it is also capable of analysing structure and semantics of a sentence,
therefore, ensure that analysis of plagiarism is effective and efficient as well.
In modern researches, application of big data is playing an important role. However,
processing of big data requires advanced data analytics techniques (Taskin & Al, 2019). In
this context, natural language processing is becoming popular for its capability to analyse
large set of data and retrieve information from these data. natural language processing is
applicable for data analysis that is not organized and therefore, it is an important technique
for big data analysis as in big data is often not organized in nature which makes analysis of
big data challenging as well.
5. Conclusions
Natural language processing has emerged as an interesting topics among researchers. It is
capable making machines more intelligent. However, the most significant feature of this
technology is its capability to communicate with humans in natural language. It ensures that

8
NATURAL LANGUAGE PROCESSING
conversation with computers is more effective and also enhance value of conversation as
well. NLP helps to read and decode human language and it helps machines to process this
language properly and provide meaningful and valuable output through intelligent language
processing as well. However, it is not easy for machines to process natural language and for
machines to accomplish this intelligent and advanced techniques are required. Therefore, to
ensure this, various techniques are being applied and these techniques are mainly based on
artificial intelligence or AI. Some of the popular and effective techniques for natural language
processing are word embeddings, Convolutional Neural Network, Recurrent Neural Network,
Reinforcement Learning and Deep Generative Models. These techniques are making
applications of machine learning effective and efficient as well.

9
NATURAL LANGUAGE PROCESSING
6. References
Desai, T., Deshmukh, U., Gandhi, M., & Kurup, L. (2016, March). A hybrid approach for
detection of plagiarism using natural language processing. In Proceedings of the
Second International Conference on Information and Communication Technology for
Competitive Strategies (pp. 1-6).
Do, H. H., Prasad, P. W. C., Maag, A., & Alsadoon, A. (2019). Deep learning for aspect-
based sentiment analysis: a comparative review. Expert Systems with
Applications, 118, 272-299.
Goldberg, Y. (2017). Neural network methods for natural language processing. Synthesis
Lectures on Human Language Technologies, 10(1), 1-309.
Khurana, D., Koli, A., Khatter, K., & Singh, S. (2017). Natural language processing: State of
the art, current trends and challenges. arXiv preprint arXiv:1708.05148.
Li, Y. (2017). Deep reinforcement learning: An overview. arXiv preprint arXiv:1701.07274.
Poria, S., Cambria, E., Hazarika, D., Mazumder, N., Zadeh, A., & Morency, L. P. (2017,
November). Multi-level multiple attentions for contextual multimodal sentiment
analysis. In 2017 IEEE International Conference on Data Mining (ICDM) (pp. 1033-
1038). IEEE.
Sun, S., Luo, C., & Chen, J. (2017). A review of natural language processing techniques for
opinion mining systems. Information fusion, 36, 10-25.
Taskin, Z., & Al, U. (2019). Natural language processing applications in library and
information science. Online Information Review.
Yin, W., Kann, K., Yu, M., & Schütze, H. (2017). Comparative study of cnn and rnn for
natural language processing. arXiv preprint arXiv:1702.01923.

10
NATURAL LANGUAGE PROCESSING
Young, T., Hazarika, D., Poria, S., & Cambria, E. (2018). Recent trends in deep learning
based natural language processing. ieee Computational intelligenCe magazine, 13(3),
55-75.