World's most popular travel blog for travel bloggers.

What does it mean to have a continuous action space w.r.t. to reinforcement learning?

, , No Comments
Problem Detail: 

Last time I posted this question I got criticised for not being specific enough, hence this is my second attempt at trying to understand what it means to have a continuous action space.

Please refer to this paper, page 5 (A Survey on Policy Search for Robotics by Marc Peter Deisenroth, Gerhard Neumann and Jan Peters. Foundations and Trends in Robotics. 2013).

The authors say:

The RL algorithm has to manage (i) high-dimensional continuous state and action spaces, (ii) strong real- time requirements, and (iii) the high costs of robot interactions with its environment.

I do not understand how an action can be continuous, in the same way that a state-vector is continuous by virtue of having elements that are simply $x_k \in \mathbb{R}^D$ for a state vector $\mathbf{x} := [x_1,\ldots,x_K]^T$. But how can an action be quantified in the same way? It obviously can I simply do not understand how.

Asked By : Astrid
Answered By : D.W.

When you're driving of your car and you turn the wheel, is that a discrete or a continuous action? It's continuous, because you can control how much you turn the wheel. How much do you press the gas pedal? That's a continuous input. This leads to a continuous action space: e.g., for each positive real number $x$ in some range, "turn the wheel $x$ degrees to the right" is a possible action.

Best Answer from StackOverflow

Question Source :

3200 people like this

 Download Related Notes/Documents


Post a Comment

Let us know your responses and feedback