what is reinforcement learning? -凯发k8网页登录

  • documentation
main content

what is reinforcement learning?

reinforcement learning is a goal-directed computational approach where a computer learns to perform a task by interacting with an unknown dynamic environment. this learning approach enables a computer to make a series of decisions to maximize the cumulative reward for the task without human intervention and without being explicitly programmed to achieve the task. the following diagram shows a general representation of a reinforcement learning scenario.

diagram showing an agent that interacts with its environment. the observation signal goes from the environment to the agent, and the action signal goes from the agent to the environment. the reward signal goes from the environment to the reinforcement learning algorithm inside the agent. the reinforcement learning algorithm uses the available information to update a policy. the agent uses a policy to map an observation to an action. this is similar to a control diagram, shown below, in which a controller senses an error between a desired reference and a plant output and uses the error to acts on a plant input.

the goal of reinforcement learning is to train an agent to complete a task within an unknown environment. the agent receives observations and a reward from the environment and sends actions to the environment. the reward is a measure of how successful an action is with respect to completing the task goal.

the agent contains two components: a policy and a learning algorithm.

  • the policy is a mapping that selects actions based on the observations from the environment. typically, the policy is a function approximator with tunable parameters, such as a deep neural network.

  • the learning algorithm continuously updates the policy parameters based on the actions, observations, and reward. the goal of the learning algorithm is to find an optimal policy that maximizes the cumulative reward received during the task.

in other words, reinforcement learning involves an agent learning the optimal behavior through repeated trial-and-error interactions with the environment without human involvement.

as an example, consider the task of parking a vehicle using an automated driving system. the goal of this task is for the vehicle computer (agent) to park the vehicle in the correct position and orientation. to do so, the controller uses readings from cameras, accelerometers, gyroscopes, a gps receiver, and lidar (observations) to generate steering, braking, and acceleration commands (actions). the action commands are sent to the actuators that control the vehicle. the resulting observations depend on the actuators, sensors, vehicle dynamics, road surface, wind, and many other less-important factors. all these factors, that is, everything that is not the agent, make up the environment in reinforcement learning.

to learn how to generate the correct actions from the observations, the computer repeatedly tries to park the vehicle using a trial-and-error process. to guide the learning process, you provide a signal that is one when the car successfully reaches the desired position and orientation and zero otherwise (reward). during each trial, the computer selects actions using a mapping (policy) initialized with some default values. after each trial, the computer updates the mapping to maximize the reward (learning algorithm). this process continues until the computer learns an optimal mapping that successfully parks the car.

reinforcement learning workflow

the general workflow for training an agent using reinforcement learning includes the following steps.

figure showing the seven stages of a typical reinforcement learning workflow.

  1. formulate problem — define the task for the agent to learn, including how the agent interacts with the environment and any primary and secondary goals the agent must achieve.

  2. create environment — define the environment within which the agent operates, including the interface between agent and environment and the environment dynamic model. for more information, see and .

  3. define reward — specify the reward signal that the agent uses to measure its performance against the task goals and how to calculate this signal from the environment. for more information, see .

  4. create agent — create the agent, which includes defining a policy approximator (actor) an value function approximator (critic) and configuring the agent learning algorithm. for more information, see and reinforcement learning agents.

  5. train agent — train the agent approximators using the defined environment, reward, and agent learning algorithm. for more information, see train reinforcement learning agents.

  6. validate agent — evaluate the performance of the trained agent by simulating the agent and environment together. for more information, see train reinforcement learning agents.

  7. deploy policy — deploy the trained policy approximator using, for example, generated gpu code. for more information, see deploy trained reinforcement learning policies.

training an agent using reinforcement learning is an iterative process. decisions and results in later stages can require you to return to an earlier stage in the learning workflow. for example, if the training process does not converge to an optimal policy within a reasonable amount of time, you might have to update some of the following before retraining the agent:

  • training settings

  • learning algorithm configuration

  • policy and value function (actor and critic) approximators

  • reward signal definition

  • action and observation signals

  • environment dynamics

related examples

more about

    网站地图