Providing Advice to Agents that Learn from Reinforcements
Jude W. Shavlik
Computer Sciences Dept.
University of Wisconsin-Madison
12:05 pm Tue. Feb. 13 in 4274 Chamberlin Hall
Learning from reinforcements is a promising approach for creating
intelligent agents. However, this style of machine learning usually
requires a large number of training episodes. We present an approach
that addresses this shortcoming by allowing a reinforcement learner to
accept advice given, at any time and in a natural manner, by an
external observer. In our approach, the (human) advice-giver watches
the learner and occasionally makes suggestions, expressed as
instructions in a simple programming language. Based on techniques
from knowledge-based neural networks, these programs are inserted
directly into the agent's "utility function." The advice need not be
perfectly correct nor complete; subsequent learning further integrates
and refines the advice. We present empirical evidence that shows our
approach leads to statistically-significant gains in performance.
Importantly, the advice improves the learner regardless of the stage of
training at which it is given.