Overfitting Avoidance by Tolerant Training

Dr. Nick Street
Department of Computer Sciences
University of Wisconsin-Madison
street@cs.wisc.edu

2:30 pm Fri. Dec. 2 in 2310 Computer Sciences and Statistics Bldg.

Overfitting avoidance -- that is, avoiding the memorization of details in a training set -- is a useful inductive learning bias for many real-world applications. In particular, when the measured features are known to be noisy and inexact, fitting them as tightly as possible drives up the variance of the learning system unnecessarily. Our simple and effective solution, known as either "tolerant training" or "banded approximation," acknowledges the noise a priori and fits the data within a parametric tolerance. This improves generalization performance in a wide range of circumstances. I will discuss both the theoretical and computational implications of this approach.