some lectures/examples on Q-learning using feature/linear approximation

Question

Most of the resources that I find in the internet deal with Q-learning or SARSA in the discrete sense, when state spaces and actions are finite. There are variety of examples and tutorials for this case.

I am looking for the case when state space is infinite. In this case, it is impossible to list all the states. Hence, an important technique would be to make a features-based representation. However, the resources are few and far in between.

If you know of resources, websites, examples, lectures, even video lectures, especially on the Q-learning/SARSA with features-based representation, it be great to have a look.

The classic Sutton and Barto book talks a lot about state space approximation. It is available freely online. — Don Reba, Dec 04 '15 at 07:36
Possible duplicate of How to fit weights into Q-values with linear function approximation — steffen, Dec 23 '15 at 09:35

score 1 · Answer 1 · answered Dec 23 '15 at 20:47

Georgia Tech has recently released a reinforcement learning course that covers this in a section called "Generalization." See also Gordon's paper "Stable function approximation in dynamic programming", and Ladoukis & Parr's paper on least-squares policy iteration.

You may also find value in the BURLAP continuous-domain tutorials. (BURLAP is a Java library "for the use and development of single or multi-agent planning and learning algorithms and domains to accompany them.") In particular, the lunar lander example uses SARSA($\lambda$).

some lectures/examples on Q-learning using feature/linear approximation

1 Answers1