We present and evaluate an infrastructure with which to rapidly and easily build intelligent software agents for Web-based tasks. Our design is centered around two basic functions: ScoreThisLink and ScoreThisPage. If given highly accurate such functions, standard heuristic search would lead to efficient retrieval of useful information. Our approach allows users to tailor our system's behavior by providing approximate advice about the above functions. This advice is mapped into neural-network implementations of the two functions. Subsequent reinforcements from the Web (e.g., dead links) and any ratings of retrieved pages that the user wishes to provide are, respectively, used to refine the link- and page-scoring functions. Hence, our agent architecture provides an appealing middle ground between non-adaptive agent programming languages and systems that solely learn user preferences from the user's ratings of pages. We describe our internal representation of Web pages, the major predicates in our advice language, how advice is mapped into neural networks, and the mechanisms for refining advice based on subsequent feedback. We also present a case study where we provide some simple advice and specialize our general-purpose system into a "home-page finder." An empirical study demonstrates that our approach leads to a more effective home-page finder than that of a leading commercial Web search site.
This is joint work with Prof. Jude Shavlik of University of Wisconsin-Madison.