Transferability of Features in Deep Learning

This pre-research proposal examines the phenomenon of feature transferability in deep neural networks. It aims to quantify the degree to which different layers exhibit general or specific feature representations. The research will investigate whether the transition from general to specific features occurs abruptly or gradually across multiple layers, and determine the location of this transition within the network architecture. A detailed timeline outlines the stages of the research process, including data collection, analysis, and writing.

PRE-RESEARCH PROPOSAL: TRANSFERABILITY OF FEATURES IN DEEP NEURAL
NETWORKS 1
Pre-Research Proposal: Transferability of Features in Deep Neural
Networks
Name
Date

PRE-RESEARCH PROPOSAL: TRANSFERABILITY OF FEATURES IN DEEP NEURAL
NETWORKS 2
Problem Definition
Deep neural networks in the modern world exhibit a curios phenomenon in that when
trained with images, they have a tendency to all learn first layer features that are similar to color
blobs or Gabor filters. These filters appear so commonly that if anything else is obtained in natural
image datasets, the result is a suspicion that the hyper parameters chosen was done poorly or there
is a bug in the software. This phenomenon is seen in different datasets as well as where the training
objectives are very different including in situations of supervised image classification, unsupervised
sparse representations learning, and unsupervised density learning. Regardless of the natural dataset
and the specific cost function, the standard features in first layer systems seem to occur and so
these features (first features) are considered general. Further, last layer trained network computed
features must greatly depend on the chosen task and dataset; the last layer features are thus termed
specific (Singh et al., 2015). Given that the first layers are general while last layers are specific,
then within the network, there must be a point of transition from general to specific (Joshi, 2017).
With this in mind, this pre-research proposal has the following objectives;
Objectives
 To quantify the degree to which a specific layer is specific or general
 To establish whether the transition from general to specific occur suddenly at a singe layer
or whether it occurs spread out out in over many layers
 To establish where the transition occurs; whether it is near the first, the middle, or the last
layer in the network
Time Table
Task Duration/ Time
Evaluating research topics and identifying
suitable research area
Three Days (Nov 25 2017 to Nov 28 2017)
Writing preproposal One day (Nov 29 2017)
Pre research data and materials collection One Week
Writing formal research proposal One Week
Getting professor feedback and making
necessary adjustments
Two Weeks
Designing research methodology Three Days

PRE-RESEARCH PROPOSAL: TRANSFERABILITY OF FEATURES IN DEEP NEURAL
NETWORKS 3
Collecting materials for the research One Week
Literature Reviews Two Weeks
Designing experimental setup One Week
Data Collection One Week
Data analysis One Week
Discussion of research findings Four Days
Making Draft Research Five days
Obtaining professor feedback Two weeks
Making adjustments and writing final
research paper with conclusions and
recommendations
Two weeks
Presenting research One day

PRE-RESEARCH PROPOSAL: TRANSFERABILITY OF FEATURES IN DEEP NEURAL
NETWORKS 4
References
Joshi, N. (2017). Combinational neural network using Gabor filters for the classification of
handwritten digits (pp. 1-4). Frankfurt: Frankfurt Institute for Advanced Studie. Retrieved
from https://arxiv.org/pdf/1709.05867.pdf
Singh, B., De, S., Zhang, Y., Goldstein, T., & Taylor, G., & 2015 (December 01, 2015). Layer-
Specific Adaptive Learning Rates for Deep Networks. IEEE 14th International Conference
on Machine Learning and Applications (ICMLA). 364-368.