Skip navigation
Please use this identifier to cite or link to this item:
Title: Improved Binary Classification with Compact Neural Networks using AdaBoost, SCANN, & SMOTE Oversampling
Authors: Zuluaga, David
Advisors: Jha, Niraj K
Department: Electrical Engineering
Certificate Program: Applications of Computing Program
Class Year: 2020
Abstract: AdaBoost is a well known boosting algorithm that excels at taking an ensemble of weak learners and creating a strong classifier by taking the weighted sum of their predictions. However, it has not been used extensively with neural networks because they are already strong classifiers. Existing implementations of the two concepts create a large ensemble of strong classifiers that do not provide a significant enough improvement in accuracy to warrant their size. In this paper, we will implement a novel AdaBoost implementation of neural networks that creates a strong classifier from weak learners , while also having a more compact design relative to existing implementations. This will be done by using the SCANN [1] growing/pruning architecture along with feature compression to create weak artificially neural network (ANN) models. We will also adopt the concept of random decision forests by subsampling a different set of features for each model that is part of the ensemble. Finally, the AdaBoost algorithm is used to resample the training distribution in each iteration and bias it towards examples that were misclassified. Each model will contribute weighted predictions that are summed to determine the final ensemble decision for each input. The results show this implementation was effective in creating a strong classifier from weak learners for binary classification in the musk and epileptic datasets. However, the ensemble’s accuracy is still lower than that of a generic model that has a number of connections equal to the sum of all the connections in the ensemble’s members. This indicates that the novel implementation is a good launching points, but further work needs to be conducted to make this novel implementation practical.
Access Restrictions: Walk-in Access. This thesis can only be viewed on computer terminals at the Mudd Manuscript Library.
Type of Material: Princeton University Senior Theses
Language: en
Appears in Collections:Electrical Engineering, 1932-2020

Files in This Item:
File Description SizeFormat 
ZULUAGA-DAVID-THESIS.pdf509.59 kBAdobe PDF    Request a copy

Items in Dataspace are protected by copyright, with all rights reserved, unless otherwise indicated.