Overfitting Avoidance by Tolerant Training
Dr. Nick Street
Department of Computer Sciences
University of Wisconsin-Madison
street@cs.wisc.edu
2:30 pm Fri. Dec. 2 in 2310 Computer Sciences and Statistics Bldg.
Overfitting avoidance -- that is, avoiding the memorization of details
in a training set -- is a useful inductive learning bias for many
real-world applications. In particular, when the measured features
are known to be noisy and inexact, fitting them as tightly as possible
drives up the variance of the learning system unnecessarily.
Our simple and effective solution, known as either "tolerant
training" or "banded approximation," acknowledges the noise a priori
and fits the data within a parametric tolerance. This improves
generalization performance in a wide range of circumstances. I will
discuss both the theoretical and computational implications of this
approach.