create specifications object for a finite-凯发k8网页登录

for this example, consider the rlsimplependulummodel simulink® model. the model is a simple frictionless pendulum that initially hangs in a downward position.

open the model.

mdl = "rlsimplependulummodel";
open_system(mdl)

create rlnumericspec and rlfinitesetspec objects for the observation and action information, respectively.

the observation is a vector containing three signals: the sine, cosine, and time derivative of the angle.

obsinfo = rlnumericspec([3 1]) 

obsinfo = 
  rlnumericspec with properties:
     lowerlimit: -inf
     upperlimit: inf
           name: [0x0 string]
    description: [0x0 string]
      dimension: [3 1]
       datatype: "double"

the action is a scalar expressing the torque and can be one of three possible values, -2 nm, 0 nm and 2 nm.

actinfo = rlfinitesetspec([-2 0 2])

actinfo = 
  rlfinitesetspec with properties:
       elements: [3x1 double]
           name: [0x0 string]
    description: [0x0 string]
      dimension: [1 1]
       datatype: "double"

you can use dot notation to assign property values for the rlnumericspec and rlfinitesetspec objects.

obsinfo.name = "observations";
actinfo.name = "torque";

assign the agent block path information, and create the reinforcement learning environment for the simulink model using the information extracted in the previous steps.

agentblk = mdl   "/rl agent";
env = rlsimulinkenv(mdl,agentblk,obsinfo,actinfo)

env = 
simulinkenvwithagent with properties:
           model : rlsimplependulummodel
      agentblock : rlsimplependulummodel/rl agent
        resetfcn : []
  usefastrestart : on

you can also specify a reset function using dot notation. for this example, randomly initialize theta0 in the model workspace.

env.resetfcn = @(in) setvariable(in,"theta0",randn,"workspace",mdl)

env = 
simulinkenvwithagent with properties:
           model : rlsimplependulummodel
      agentblock : rlsimplependulummodel/rl agent
        resetfcn : @(in)setvariable(in,"theta0",randn,"workspace",mdl)
  usefastrestart : on

if your environment has a an observation space consisting of multiple channels, some continuous, some discrete, use a vector of specification objects (each defining a single channel) to define the observation space.

for example, define an observation space as consisting of four channels. the first one carries a single number labeled 7, 9, 19 or -2. the second one carries a vector over a continuous three-dimensional space. the third channel carries a two by two matrix that can be either zero or the identity. finally, the fourth channel carries a continuous matrix with four rows and three columns.

obsinfo = [  rlfinitesetspec([7 9 19 -2])
             rlnumericspec([3 1])
             rlfinitesetspec({zeros(2), eye(2)})
             rlnumericspec([4 3]) ]

obsinfo=4×1 object
  4x1 heterogeneous rldataspec (rlfinitesetspec, rlnumericspec) array with properties:
    name
    description
    dimension
    datatype

you can access each channel specification using dot notation.

obsinfo(1)

ans = 
  rlfinitesetspec with properties:
       elements: [4x1 double]
           name: [0x0 string]
    description: [0x0 string]
      dimension: [1 1]
       datatype: "double"

obsinfo(2).name = "velocity";
obsinfo(2).description = "velocity vector in m/s in body reference frame";
obsinfo(2)

ans = 
  rlnumericspec with properties:
     lowerlimit: -inf
     upperlimit: inf
           name: "velocity"
    description: "velocity vector in m/s in body reference frame"
      dimension: [3 1]
       datatype: "double"

within reinforcement learning toolbox™ software, agents can only have one action channel. however, if your desired design involves multiple discrete action channels, you can convert them into a single action channel with a number of elements equal to the product of the number of elements of each action channel of your original design. specifically, each element of the single action channel corresponds to a particular combination of elements in the action channels of your original design.

for example, suppose that the valid values for a two-output system are [1 2] for the first output and [10 20 30] for the second output. create a discrete action space specification for all possible output combinations.

actionspec = rlfinitesetspec({[1 10],[1 20],[1 30],...
                              [2 10],[2 20],[2 30]})

actionspec = 
  rlfinitesetspec with properties:
       elements: {6x1 cell}
           name: [0x0 string]
    description: [0x0 string]
      dimension: [1 2]
       datatype: "double"