Training Self-Driving Cars using Deep Reinforcement Learning

This proposal aims to develop an autonomous system for self-driving cars using deep reinforcement learning. It discusses the problem, literature review, objectives, and methodology for the project.

Running head: PROJECT PROPOSAL
Training Self-Driving Cars using Deep Reinforcement Learning
Name of the student
Name of the University
Author note

2PROJECT PROPOSAL
Abstract
The aim of this proposal is to propose a project that can put forward the development of an
autonomous system with capabilities to navigate in the absence of maps and explicit rules,
relying - similarly to humans - on a comprehensive understanding of the environment, while
following simple higher-level directions. The problem, literature review and objectives have
been discussed in this proposal.

3PROJECT PROPOSAL
Table of Content
Introduction:...............................................................................................................................4
Background and brief literature review:....................................................................................4
Problem Statement:....................................................................................................................4
Research questions:....................................................................................................................5
Aims and Objectives:.................................................................................................................5
Methodology:.............................................................................................................................6
Significance:...............................................................................................................................6
Reference:..................................................................................................................................7

4PROJECT PROPOSAL
Introduction:
The process of teaching an autonomous vehicle to drive is a very advanced as well as
ubiquitous task. The operational efficiency, functional safety and many other obligations are
there while at the same time the ability to effectively respond to unexpected events is equally
essential (Zhang et al., 2016). The purpose of this proposal is to propose a project that can put
forward the development of an autonomous system with capabilities to navigate in the
absence of maps and explicit rules, relying - similarly to humans - on a comprehensive
understanding of the environment, while following simple higher-level directions.
Background and brief literature review:
According to Sallab (2017), Controlling an agent through high dimensional sensory
inputs is the core of any reinforcement learning procedure. Another popular approach to
solve the problem of high dimensional data as input is a field in Machine Learning (ML)
called Deep Learning (DL). Some of the major implementation procedures of reinforcement
learning are neural networking, genetic algorithm and linear parametrization. Every possible
implementation has some advantages as well as disadvantages. As opined by Cui, Yang and
Sharma (2017), The basic concept of reinforcement learning has come from the deep learning
methods. In recent years the deep learning methods have been utilized as function
approximators that formulate the Reinforcement Learning (RL) models (Wang, Jia & Weng,
2018). The research conducted by Vitelli and Nayebi (2016), brought the approach of
combining deep learning with reinforcement learning to center-stage by demonstrating a
convolutional neural network (CNN), trained with a variant of Q-learning, that can learn
successful control policies from raw video data in order to play Atari.

5PROJECT PROPOSAL
Problem Statement:
There are many researches on theoretical approach to implement deep reinforced
learning for autonomous cars. However, few studies have been done through practical
simulation involving real model cars with proper assembled hardware. The intervention of Q-
learning based RL algorithm can be assessed only through real life simulation. It will also
allow measuring the time and space complexity of the chosen RL algorithm. Another,
problem is that currently there is no study conducted to measure the accuracy and efficiency
of decision making in an unexpected event.
Research questions:
Considering the scope of this study and the identified problems in this field the following
research questions have been chosen for this project:
 What is the most efficient RL Q-learning approach in practical implementation in
autonomous vehicle?
 What is the most efficient reinforce learning process for decision making in an
unexpected event?
 What is the most simplified way to accumulate the most efficient available
approaches for enhanced decision making ability?
 How to implement the concept of collective Q-learning approaches in a physical
system with maximum possible efficiency?
Aims and Objectives:
The aim of this project is to find the most efficient practically implementable solution
for Q-learning approach in Reinforce Learning for Autonomous Car driving with capabilities
to navigate in the absence of maps and explicit rules, relying - similarly to humans - on a

6PROJECT PROPOSAL
comprehensive understanding of the environment, while following simple higher-level
directions.
As per the project aim the objectives of this project are:
 To develop a physical simulated environment to test the most efficient way of
collective Q-learning approaches
 To measure and compare the time and space complexities of different considering
delays in an unexpected event
 To find out the best reinforce learning approach in a physical system with maximum
possible efficiency
 To develop the blueprint of final autonomous vehicle system
Methodology:
For this project an inductive approach has been planned aiming at utilizing a staged
methodology. Three phase wise approach that will be used for this project are: (i) utilising a
simulated environment for training/testing to enhance the problem understanding, and to
enable the design of suitable model architectures and hyperparameter selection for lane
following task; then (ii) transferring the knowledge from the simulated task, and continuation
of training in a real environment; for finally (iii) instructing an autonomous vehicle through
reinforcement, aiming at on-line improvement, with a human safety driver taking over during
the navigation by providing instructions, rewards and penalties to endow an autonomous
vehicle. It has been also expected that local UK Company will work in this project
collaboratively, so that their autonomous pod can be used to validate the proposed approach.
Significance:
In the field of advance artificial intelligence technology this project can present a
practical implication of reinforced learning. Through this practical intervention based

7PROJECT PROPOSAL
assessment the practicality of available Q-learning approaches can be understood with
adequate evidences. Through this planning new companies can develop their own
autonomous vehicles for personal as well as commercial use.

8PROJECT PROPOSAL
Reference:
Cui, R., Yang, C., Li, Y., & Sharma, S. (2017). Adaptive neural network control of AUVs
with control input nonlinearities using reinforcement learning. IEEE Transactions on
Systems, Man, and Cybernetics: Systems, 47(6), 1019-1029.
Sallab, A. E., Abdou, M., Perot, E., & Yogamani, S. (2017). Deep reinforcement learning
framework for autonomous driving. Electronic Imaging, 2017(19), 70-76.
Vitelli, M., & Nayebi, A. (2016). Carma: A deep reinforcement learning approach to
autonomous driving. Tech. rep. Stanford University.
Wang, S., Jia, D., & Weng, X. (2018). Deep reinforcement learning for autonomous
driving. arXiv preprint arXiv:1811.11329.
Zhang, T., Kahn, G., Levine, S., & Abbeel, P. (2016, May). Learning deep control policies
for autonomous aerial vehicles with mpc-guided policy search. In 2016 IEEE
international conference on robotics and automation (ICRA) (pp. 528-535). IEEE.