Incorporating Advice into Agents that Learn from Reinforcements
Rich Maclin
Department of Computer Sciences
University of Wisconsin-Madison
street@cs.wisc.edu
2:30 pm Fri. Mar. 10 in 2310 Computer Sciences and Statistics Bldg.
Learning from reinforcements is a promising approach for creating intelligent
agents.
However, reinforcement learning usually requires a large number of training
episodes.
I will present an approach that addresses this shortcoming by allowing a
connectionist Q-learner to accept advice given, at any time and in a natural
manner, by an external observer.
In this approach, the advice-giver watches the learner and occasionally makes
suggestions, expressed as instructions in a simple programming language.
Based on techniques from knowledge-based neural networks, these programs are
inserted directly into the agent's utility function.
Subsequent reinforcement learning further integrates and refines the advice.
I will present empirical evidence that shows the approach leads to
statistically-significant gains in expected reward.