TensorForce: Details for "summary_spec" agent parameters¶
summary_spec¶
TensorForce has the ability to record summary data for use with TensorBoard as well STDIO and file export. This is accomplished through dictionary parameter called "summary_spec" passed to the agent on initialization.
"summary_spec" supports the following optional dictionary entries:
Key | Value |
---|---|
directory | (str) Path to storage for TensorBoard summary data |
steps | (int) Frequency in steps between storage of summary data |
seconds | (int) Frequency in seconds to store summary data |
labels | (list) Requested Export, See "LABELS" section |
meta_dict | (dict) For used with label "configuration" |
LABELS¶
Entry | Data produced |
---|---|
losses | Training total-loss and "loss-without-regularization" |
total-loss | Final calculated loss value |
variables | Network variables |
inputs | Equivalent to: ['states', 'actions', 'rewards'] |
states | Histogram of input state space |
actions | Histogram of input action space |
rewards | Histogram of input reward space |
gradients | Histogram and scalar gradients |
gradients_histogram | Variable gradients as histograms |
gradients_scalar | Variable Mean/Variance of gradients as scalar |
regularization | Regularization values |
configuration | See Configuration Export for more detail |
configuration | Export configuration to "TEXT" tab in TensorBoard |
print_configuration | Prints configuration to STDOUT |
from tensorforce.agents import PPOAgent
# Create a Proximal Policy Optimization agent
agent = PPOAgent(
states_spec=...,
actions_spec=...,
network_spec=...,
summary_spec=dict(directory="./board/",
steps=50,
labels=['configuration',
'gradients_scalar',
'regularization',
'inputs',
'losses',
'variables']
),
...
)
Configuration Export¶
Adding the "configuration" label will create a "TEXT" tab in TensorBoard that contains all the parameters passed to the Agent. By using the additional "summary_spec" dictionary key "meta_dict", custom keys and values can be added to the data export. The user may want to pass "Description", "Experiement #", "InputDataSet", etc.
If a key is already in use within TensorForce an error will be raised to notify you to change the key value. To use the custom feature, create a dictionary with keys to export:
from tensorforce.agents import PPOAgent
metaparams['MyDescription'] = "This experiment covers the first test ...."
metaparams['My2D'] = np.ones((9,9)) # 9x9 matrix of 1.0's
metaparams['My1D'] = np.ones((9)) # Column of 9 1.0's
# Create a Proximal Policy Optimization agent
agent = PPOAgent(
states_spec=...,
actions_spec=...,
network_spec=...,
summary_spec=dict(directory="./board/",
steps=50,
meta_dict=metaparams, #Add custom keys to export
labels=['configuration',
'gradients_scalar',
'regularization',
'inputs',
'losses',
'variables']
),
...
)
Use the "print_configuration" label to export the configuration data to the command line's STDOUT.