main content

options for simulating a reinforcement learning agent within an environment -凯发k8网页登录

options for simulating a reinforcement learning agent within an environment

since r2019a

description

use an rlsimulationoptions object to specify simulation options for simulating a reinforcement learning agent within an environment. to perform the simulation, use sim.

for more information on agents training and simulation, see train reinforcement learning agents.

creation

syntax

simopts = rlsimulationoptions
opt = rlsimulationoptions(name=value)

description

simopts = rlsimulationoptions returns the default options for simulating a reinforcement learning environment against an agent. use simulation options to specify parameters about the simulation such as the maximum number of steps to run per simulation and the number of simulations to run. after configuring the options, use simopts as an input argument for sim.

example

opt = rlsimulationoptions(name=value) creates a simulation options set with the specified properties using one or more name-value pair arguments.

properties

number of steps to run the simulation, specified as the comma-separated pair consisting of 'maxsteps' and a positive integer. in general, you define episode termination conditions in the environment. this value is the maximum number of steps to run in the simulation if those termination conditions are not met.

example: maxsteps=1000

number of simulations to run, specified as the comma-separated pair consisting of 'numsimulations' and a positive integer. at the start of each simulation, sim resets the environment. you specify what happens on environment reset when you create the environment. for instance, resetting the environment at the start of each episode can include randomizing initial state values, if you configure your environment to do so. in that case, running multiple simulations allows you to validate performance of a trained agent over a range of initial conditions.

example: numsimulations=10

stop simulation when an error occurs, specified as "off" or "on". when this option is "off", errors are captured and returned in the simulationinfo output of sim, and simulation continues.

flag for using parallel simulation, specified as a logical. setting this option to true configures the simulation to use parallel processing to simulate the environment, thereby enabling usage of multiple cores, processors, computer clusters or cloud resources to speed up simulation. to specify options for parallel simulation, use the parallelizationoptions property.

note that if you want to speed up deep neural network calculations (such as gradient computation, parameter update and prediction) using a local gpu you do not need to set useparallel to true. instead, when creating your actor or critic representation, use an rlrepresentationoptions object in which the usedevice option is set to "gpu".

using parallel computing or the gpu requires parallel computing toolbox™ software. using computer clusters or cloud resources additionally requires matlab® parallel server™.

for more information about training using multicore processors and gpus, see train agents using parallel computing and gpus.

example: useparallel=true

parallelization options to control parallel simulation, specified as a paralleltraining object. for more information about training using parallel computing, see train reinforcement learning agents.

the paralleltraining object has the following properties, which you can modify using dot notation after creating the rltrainingoptions object.

randomizer initialization for workers, specified as one the following:

  • –1 — assign a unique random seed to each worker. the value of the seed is the worker id.

  • –2 — do not assign a random seed to the workers.

  • vector — manually specify the random seed for each work. the number of elements in the vector must match the number of workers.

send model and workspace variables to parallel workers, specified as "on" or "off". when the option is "on", the host sends variables used in models and defined in the base matlab workspace to the workers.

additional files to attach to the parallel pool, specified as a string or string array.

function to run before simulation starts, specified as a handle to a function having no input arguments. this function is run once per worker before simulation begins. write this function to perform any processing that you need prior to simulation.

function to run after simulation ends, specified as a handle to a function having no input arguments. you can write this function to clean up the workspace or perform other processing after simulation terminates.

object functions

simsimulate trained reinforcement learning agents within specified environment

examples

create an options set for simulating a reinforcement learning environment. set the number of steps to simulate to 1000, and configure the options to run three simulations.

you can set the options using name,value pairs when you create the options set. any options that you do not explicitly set have their default values.

simopts = rlsimulationoptions(...
    maxsteps=1000,...
    numsimulations=3)
simopts = 
  rlsimulationoptions with properties:
                  maxsteps: 1000
            numsimulations: 3
               stoponerror: "on"
               useparallel: 0
    parallelizationoptions: [1x1 rl.option.parallelsimulation]

alternatively, create a default options set and use dot notation to change some of the values.

simopts = rlsimulationoptions;
simopts.maxsteps = 1000;
simopts.numsimulations = 3;
simopts
simopts = 
  rlsimulationoptions with properties:
                  maxsteps: 1000
            numsimulations: 3
               stoponerror: "on"
               useparallel: 0
    parallelizationoptions: [1x1 rl.option.parallelsimulation]

version history

introduced in r2019a

see also

functions

objects

topics

网站地图