open reinforcement learning data viewer tool -凯发k8网页登录

this example shows how to visualize data logged during custom training loop using the reinforcement learning data viewer.

create a filelogger object using rldatalogger.

flgr = rldatalogger();

open a reinforcement learning data viewer window using rldataviewer.

rldataviewer(flgr);

initialize the logger object.

setup(flgr);

within a custom training loop, store data to the logger object memory and write data to mat-files. for this example, store a random scalar, a random 2-by-1 vector, and a random 5-by-5 matrix as variables named randomscalardata, randomvectordata, and randommatrixdata, respectively.

for i = 1:100
    % store a random scalar, vector, and matrix in memory
    store(flgr, "randomscalardata", rand, i);
    store(flgr, "randomvectordata", rand(2,1), i);
    store(flgr, "randommatrixdata", rand(5,5), i);
    % write data from memory to a mat-file
    write(flgr);
end

clean up the logger object. this operation performs cleanup tasks like for example writing to file any data still in memory.

cleanup(flgr);

the data pane of the reinforcement learning data viewer shows the three variables under the run [logs] folder.

to select the first variable, click on randomscalardata. the preview pane then shows information such as the name, data type, and dimensions for this variable. to visualize a trend plot of the selected variable, on the data viewer tab, click trend.

this action displays a moving average plot of the variable (using a 20-sample window), along with the data points.

to plot both elements of the second variable, select randomvectordata in the data pane and, in the data viewer tab, click line subplot.

you can select multiple data histories (by holding the ctrl button while clicking their respective squares in the data pane). when these data histories belong to the same run, the multiple lines button appears in the data viewer tab. this button allows you to visualize all the selected data in the same plot.

to show the value of randommatrixdata in the last iteration as a heat map, select randommatrixdata in the data pane and, in the data viewer tab, click image.

this image is a heatmap of the matrix, with black corresponding to the maximum value and white the minimum.

this example shows how to use the reinforcement learning data viewer to visualize data logged on disk.

to save data on the disk, use a filelogger object and use it in a custom loop for three times.

flgr = rldatalogger();
for run=1:3
    % set up logger object
    setup(flgr);
    % within a custom loop, store a 3-by-1 vector in the
    % variable rv and write it to disk at each iteration. 
    for i = 1:20
        store(flgr, "rv", rand(2,1), i);
        write(flgr);
    end
    % clean up logger object.
    cleanup(flgr);
end

when setup is used on an existing filelogger object, a _1 suffix is added to the name of the destination folder. therefore, the three loops result in data being saved in the folders logs, logs_1, and logs_1_1.

to visualize the saved data, open a reinforcement learning data viewer window using rldataviewer.

rldataviewer(["logs" "logs_1" "logs_1_1"]);

to plot the time history of rv in the second run, select the red rv square under the run [logs_1] folder in the data pane and, in the data viewer tab, click line subplot.

you can select multiple data histories (by holding the ctrl button while clicking their respective squares in the data pane). when these data histories belong to the same run, the multiple lines button appears in the data viewer tab. this button allows you to visualize all the selected data in the same plot.