Networks¶
Default network: LayeredNetwork
with default argument layers
, so a list
is a short-form specification of a sequential layer-stack network architecture:
Agent.create(
...
policy=dict(network=[
dict(type='dense', size=64, activation='tanh'),
dict(type='dense', size=64, activation='tanh')
]),
...
)
Multi-input and other non-sequential networks are specified as nested list of lists of layers, where each of the inner lists forms a sequential component of the overall network architecture. The following example illustrates how to specify such a more complex network, by using the special layers Register
and Retrieve
to combine the sequential network components:
Agent.create(
states=dict(
observation=dict(type='float', shape=(16, 16, 3)),
attributes=dict(type='int', shape=(4, 2), num_values=5)
),
...
policy=[
[
dict(type='retrieve', tensors=['observation']),
dict(type='conv2d', size=32),
dict(type='flatten'),
dict(type='register', tensor='obs-embedding')
],
[
dict(type='retrieve', tensors=['attributes']),
dict(type='embedding', size=32),
dict(type='flatten'),
dict(type='register', tensor='attr-embedding')
],
[
dict(
type='retrieve', aggregation='concat',
tensors=['obs-embedding', 'attr-embedding']
),
dict(type='dense', size=64)
]
],
...
)
Note that the final action/value layer of the policy/baseline network is implicitly added, so the network output can be of arbitrary size and use any activation function, and is only required to be a rank-one embedding vector, or optionally have the same shape as the action in the case of a higher-rank action shape.
-
class
tensorforce.core.networks.
AutoNetwork
(*, size=64, depth=2, final_size=None, final_depth=1, rnn=False, device=None, l2_regularization=None, name=None, inputs_spec=None, internal_rnn=None)¶ Network which is automatically configured based on its input tensors, offering high-level customization (specification key:
auto
).Parameters: - size (int > 0) – Layer size, before concatenation if multiple states (default: 64).
- depth (int > 0) – Number of layers per state, before concatenation if multiple states (default: 2).
- final_size (int > 0) – Layer size after concatenation if multiple states (default: layer size).
- final_depth (int > 0) – Number of layers after concatenation if multiple states (default: 1).
- rnn (false | parameter, int >= 0) – Whether to add an LSTM cell with internal state as last layer, and if so, horizon of the LSTM for truncated backpropagation through time (default: false).
- device (string) – Device name (default: inherit value of parent module).
- l2_regularization (float >= 0.0) – Scalar controlling L2 regularization (default: inherit value of parent module).
- name (string) – internal use.
- inputs_spec (specification) – internal use.
-
class
tensorforce.core.networks.
LayeredNetwork
(layers, *, device=None, l2_regularization=None, name=None, inputs_spec=None)¶ Network consisting of Tensorforce layers (specification key:
custom
orlayered
), which can be specified as either a list of layer specifications in the case of a standard sequential layer-stack architecture, or as a list of list of layer specifications in the case of a more complex architecture consisting of multiple sequential layer-stacks. Note that the final action/value layer of the policy/baseline network is implicitly added, so the network output can be of arbitrary size and use any activation function, and is only required to be a rank-one embedding vector, or optionally have the same shape as the action in the case of a higher-rank action shape.Parameters: - layers (iter[specification] | iter[iter[specification]]) – Layers configuration, see the layers documentation (required).
- device (string) – Device name (default: inherit value of parent module).
- l2_regularization (float >= 0.0) – Scalar controlling L2 regularization (default: inherit value of parent module).
- name (string) – internal use.
- inputs_spec (specification) – internal use.