Type of Document Master's Thesis Author Cogswell, Michael Andrew Author's Email Address email@example.com URN etd-03032016-170427 Title Understanding Representations and Reducing their Redundancy in Deep Networks Degree Master of Science Department Computer Science Advisory Committee
Advisor Name Title Bert Huang Committee Chair Dhruv Batra Committee Co-Chair B. Aditya Prakash Committee Member Keywords
- Object Recognition
- Computer Vision
- Machine Learning
Date of Defense 2016-02-18 Availability unrestricted AbstractNeural networks in their modern deep learning incarnation have achieved state of the art performance on a wide variety of tasks and domains. A core intuition behind these methods is that they learn layers of features which interpolate between two domains in a series of related parts.
The first part of this thesis introduces the building blocks of neural networks for computer vision. It starts with linear models then proceeds to deep multilayer perceptrons and convolutional neural networks, presenting the core details of each. However, the introduction also focuses on intuition by visualizing concrete examples of the parts of a modern network.
The second part of this thesis investigates regularization of neural networks. Methods like dropout and others have been proposed to favor certain (empirically better) solutions over others. However, big deep neural networks still overfit very easily. This section proposes a new regularizer called DeCov, which leads to significantly reduced overfitting (difference between train and val performance) and greater generalization, sometimes better than dropout and other times not. The regularizer is based on the cross-covariance of hidden representations and takes advantage of the intuition that different features should try to represent different things, an intuition others have explored with similar losses. Experiments across a range of datasets and network architectures demonstrate reduced overfitting due to DeCov while almost always maintaining or increasing generalization performance and often improving performance over dropout.
Filename Size Approximate Download Time (Hours:Minutes:Seconds)
28.8 Modem 56K Modem ISDN (64 Kb) ISDN (128 Kb) Higher-speed Access Cogswell_M_T_2016.pdf 2.93 Mb 00:13:32 00:06:57 00:06:05 00:03:02 00:00:15
If you have questions or technical problems, please Contact DLA.