UCL logo

UCL Discovery

UCL home » Library Services » Electronic resources » UCL Discovery

On the role of tracking in stationary environments

Sutton, RS; Koop, A; Silver, D; (2007) On the role of tracking in stationary environments. In: (pp. pp. 871-878).

Full text not available from this repository.


It is often thought that learning algorithms that track the best solution, as opposed to converging to it, are important only on nonstationary problems. We present three results suggesting that this is not so. First we illustrate in a simple concrete example, the Black and White problem, that tracking can perform better than any converging algorithm on a stationary problem. Second, we show the same point on a larger, more realistic problem, an application of temporal difference learning to computer Go. Our third result suggests that tracking in stationary problems could be important for metalearning research (e.g., learning to learn, feature selection, transfer). We apply a metalearning algorithm for step-size adaptation, IDBD (Sutton, 1992a), to the Black and White problem, showing that meta-learning has a dramatic long-term effect on performance whereas, on an analogous converging problem, meta-learning has only a small second-order effect.

Type: Proceedings paper
Title: On the role of tracking in stationary environments
DOI: 10.1145/1273496.1273606
URI: http://discovery.ucl.ac.uk/id/eprint/1347365
Downloads since deposit
Download activity - last month
Download activity - last 12 months
Downloads by country - last 12 months

Archive Staff Only

View Item View Item