Title page for ETD etd-03032016-170427

Type of Document Master's Thesis
Author Cogswell, Michael Andrew
Author's Email Address cogswell@vt.edu
URN etd-03032016-170427
Title Understanding Representations and Reducing their Redundancy in Deep Networks
Degree Master of Science
Department Computer Science
Advisory Committee
Advisor Name Title
Bert Huang Committee Chair
Dhruv Batra Committee Co-Chair
B. Aditya Prakash Committee Member
  • Object Recognition
  • Overfitting
  • Computer Vision
  • Machine Learning
Date of Defense 2016-02-18
Availability unrestricted
Neural networks in their modern deep learning incarnation have achieved state of the art performance on a wide variety of tasks and domains. A core intuition behind these methods is that they learn layers of features which interpolate between two domains in a series of related parts.

The first part of this thesis introduces the building blocks of neural networks for computer vision. It starts with linear models then proceeds to deep multilayer perceptrons and convolutional neural networks, presenting the core details of each. However, the introduction also focuses on intuition by visualizing concrete examples of the parts of a modern network.

The second part of this thesis investigates regularization of neural networks. Methods like dropout and others have been proposed to favor certain (empirically better) solutions over others. However, big deep neural networks still overfit very easily. This section proposes a new regularizer called DeCov, which leads to significantly reduced overfitting (difference between train and val performance) and greater generalization, sometimes better than dropout and other times not. The regularizer is based on the cross-covariance of hidden representations and takes advantage of the intuition that different features should try to represent different things, an intuition others have explored with similar losses. Experiments across a range of datasets and network architectures demonstrate reduced overfitting due to DeCov while almost always maintaining or increasing generalization performance and often improving performance over dropout.

  Filename       Size       Approximate Download Time (Hours:Minutes:Seconds) 
 28.8 Modem   56K Modem   ISDN (64 Kb)   ISDN (128 Kb)   Higher-speed Access 
  Cogswell_M_T_2016.pdf 2.93 Mb 00:13:32 00:06:57 00:06:05 00:03:02 00:00:15

Browse All Available ETDs by ( Author | Department )

dla home
etds imagebase journals news ereserve special collections
virgnia tech home contact dla university libraries

If you have questions or technical problems, please Contact DLA.