CO636 Cognitive Neural Networks - Retrieval Coursework: XOR Problem

This document presents a solution for a coursework assignment in the CO636 Cognitive Neural Networks module. The assignment requires the implementation of the backpropagation algorithm to solve the XOR (exclusive or) problem using a programming language like Java, Python, or Matlab. The solution includes a detailed explanation of the backpropagation algorithm, focusing on its gradient descent approach to minimize error, and how it is used to train artificial neural networks (ANNs). The implementation involves setting up a neural network with either one or two hidden units and then testing the network with the XOR problem to produce the correct output for given inputs. The document also provides steps for the algorithm, including random weight initialization, data iteration, computation of expected results using the sigmoid function, loss computation with square error, and weight and bias updates. The goal is to find the minimum error through iterative adjustments. The document also contains the program output. This solution can be useful for students studying AI and neural networks to understand the practical application of the backpropagation algorithm.

[Type the company name]

Table of Contents
Backpropagation algorithm.........................................................................................................................3
Program.......................................................................................................................................................5
Output:........................................................................................................................................................9
Page | 2

Backpropagation algorithm
The backpropagation algorithm discovers the minimum error's value in the weight section known as the
gradient descent or base delta technique. Here the solution of the learning problem is considered that
of the weights which minimize the error function. The backpropagation algorithms are nothing but the
set of methods used effectively training of the ANN (artificial neural networks), using a stepwise
regression method by following the chain rule principle. The main functional backbone of this algorithm
is its capability to use its iterative, recursive and efficient weight update calculation function that
improves network performance. This relationship depends on the Gauss-Newton algorithm. Here, the
Backpropagation algorithm requires knowledge of the derivative of the operating function at the time of
network design. Here, the Automatic differentiation is nothing but the technique that provides the
derivative of the training algorithm automatically and analytically. In the learning context,
backpropagation algorithms are usually used to adjust the weight of the neuron through calculating the
loss function that is gradient; The gradient backpropagation calculated, while the gradient transition
(random) is used for model training (by improvement).
Here, the Backpropagation is nothing but the backward propagation and it is a mathematical tool.
Backpropagation tool is used for increasing the correctness of automated learning expectations and data
mining. In principle, algorithms are used along for quickly calculate the derivatives. The desired outputs
will be compared to the other system output. The system will adjust the connection weight are the
difference between the adjusting connection system. The algorithm gets its name because the weight
returns from the output to the input. It is difficult to understand the biases and changing weight in the
artificial neural networks performance, these are factors that make it difficult to use artificial neural
networks. In the 2000s, here, the computers offer to create the necessary insights. Until the early 1990s,
Page | 3

the multi-purpose algorithm is now used in artificial intelligence (AI), including character recognition in
the NLP (Natural language processing) and image processing.
Due to the performance of the loss level in calculating the backup process, the known and expected
results for each input value need to be used, as a rule, they are categorized as learning methods. In
addition to measures like decision trees and Naïve Bayesian, backpropagation algorithms have become
an important part of applications that have automatic learning forecasting. Here, the Gradient descent is
done by using the backward propagation is used to find the advanced error of global minimum value is
detected, at least by inversion but by using the local minimum value. Moreover, it causes problems
when crossed the error functions.
Between the interconnections of the neurons, the neural network information is stored which is nothing
but the weight. By taking the help of a learning algorithm we can learn neural network by updating its
weights. This algorithm also helps us to converge to the appropriate result. Depending on the loss
function we can change the weight and biases in this algorithm
1) We need to weight initialization and random biases.
2) data iteration needs to be done.
A) By making use of sigmoid function we can compute the expected result.
B) By using square error we can compute loss.
C) W(new) = W(old) — α ∆W
D) B(new) = B(old) — α ∆B
3) Repeat the steps until we get minimum errors.
This algorithm is very simple. We only need to us4 arithmetic operations to update weights and biases.
We can devise the algorithm in two parts
Forward Pass
Backward pass
Page | 4

Hence known as the backpropagation algorithm.
Program
Page | 5

Page | 6

Page | 7

Page | 8

Output:
Page | 9