tensorforce.core.networks package

Submodules

tensorforce.core.networks.complex_network module

class tensorforce.core.networks.complex_network.ComplexLayeredNetwork(complex_layers_spec, scope='layered-network', summary_labels=())

Bases: tensorforce.core.networks.network.LayerBasedNetwork

Complex Network consisting of a sequence of layers, which can be created from a specification dict.

__init__(complex_layers_spec, scope='layered-network', summary_labels=())

Complex Layered network.

Parameters:complex_layers_spec – List of layer specification dicts
add_layer(layer)
static from_json(filename)

Creates a complex_layered_network_builder from a JSON.

Parameters:filename – Path to configuration

Returns: A ComplexLayeredNetwork class with layers generated from the JSON

from_spec(spec, kwargs=None)

Creates a network from a specification dict.

get_list_of_named_tensor()

Returns a list of the names of tensors available.

Returns:List of the names of tensors available.
get_named_tensor(name)

Returns a named tensor if available.

Returns:True if named tensor found, False otherwise tensor: If valid, will be a tensor, otherwise None
Return type:valid
get_summaries()
get_variables(include_nontrainable=False)
internals_spec()
set_named_tensor(name, tensor)

Returns the TensorFlow summaries reported by the network.

Returns:None
tf_apply(x, internals, update, return_internals=False)
tf_regularization_loss()
class tensorforce.core.networks.complex_network.Input(inputs, axis=1, scope='merge_inputs', summary_labels=())

Bases: tensorforce.core.networks.layer.Layer

Input layer. Used for ComplexLayerNetwork’s to collect data together as a form of output to the next layer. Allows for multiple inputs to merge into a single import for next layer.

__init__(inputs, axis=1, scope='merge_inputs', summary_labels=())

Input layer.

Parameters:
  • inputs – A list of strings that name the inputs to merge
  • axis – Axis to merge the inputs
from_spec(spec, kwargs=None)

Creates a layer from a specification dict.

get_summaries()

Returns the TensorFlow summaries reported by the layer.

Returns:List of summaries.
get_variables(include_nontrainable=False)

Returns the TensorFlow variables used by the layer.

Returns:List of variables.
internals_spec()

Returns the internal states specification.

Returns:Internal states specification
tf_apply(x, update)
tf_regularization_loss()

Creates the TensorFlow operations for the layer regularization loss.

Returns:Regularization loss tensor.
tf_tensors(named_tensors)

Attaches the named_tensors dictionary to the layer for examination and update.

Parameters:named_tensors – Dictionary of named tensors to be used as Input’s or recorded from outputs
Returns:NA
class tensorforce.core.networks.complex_network.Output(output, scope='output', summary_labels=())

Bases: tensorforce.core.networks.layer.Layer

Output layer. Used for ComplexLayerNetwork’s to capture the tensor under and name for use with Input layers. Acts as a input to output passthrough.

__init__(output, scope='output', summary_labels=())

Output layer.

Parameters:output – A string that names the tensor, will be added to available inputs
from_spec(spec, kwargs=None)

Creates a layer from a specification dict.

get_summaries()

Returns the TensorFlow summaries reported by the layer.

Returns:List of summaries.
get_variables(include_nontrainable=False)

Returns the TensorFlow variables used by the layer.

Returns:List of variables.
internals_spec()

Returns the internal states specification.

Returns:Internal states specification
tf_apply(x, update)
tf_regularization_loss()

Creates the TensorFlow operations for the layer regularization loss.

Returns:Regularization loss tensor.
tf_tensors(named_tensors)

Attaches the named_tensors dictionary to the layer for examination and update.

Parameters:named_tensors – Dictionary of named tensors to be used as Input’s or recorded from outputs
Returns:NA

tensorforce.core.networks.layer module

Collection of custom layer implementations.

class tensorforce.core.networks.layer.Conv1d(size, window=3, stride=1, padding='SAME', bias=True, activation='relu', l2_regularization=0.0, l1_regularization=0.0, scope='conv1d', summary_labels=())

Bases: tensorforce.core.networks.layer.Layer

1-dimensional convolutional layer.

__init__(size, window=3, stride=1, padding='SAME', bias=True, activation='relu', l2_regularization=0.0, l1_regularization=0.0, scope='conv1d', summary_labels=())

1D convolutional layer.

Parameters:
  • size – Number of filters
  • window – Convolution window size
  • stride – Convolution stride
  • padding – Convolution padding, one of ‘VALID’ or ‘SAME’
  • bias – If true, a bias is added
  • activation – Type of nonlinearity, or dict with name & arguments
  • l2_regularization – L2 regularization weight
  • l1_regularization – L1 regularization weight
from_spec(spec, kwargs=None)

Creates a layer from a specification dict.

get_summaries()
get_variables(include_nontrainable=False)
internals_spec()

Returns the internal states specification.

Returns:Internal states specification
tf_apply(x, update)
tf_regularization_loss()
tf_tensors(named_tensors)

Attaches the named_tensors dictionary to the layer for examination and update.

Parameters:named_tensors – Dictionary of named tensors to be used as Input’s or recorded from outputs
Returns:NA
class tensorforce.core.networks.layer.Conv2d(size, window=3, stride=1, padding='SAME', bias=True, activation='relu', l2_regularization=0.0, l1_regularization=0.0, scope='conv2d', summary_labels=())

Bases: tensorforce.core.networks.layer.Layer

2-dimensional convolutional layer.

__init__(size, window=3, stride=1, padding='SAME', bias=True, activation='relu', l2_regularization=0.0, l1_regularization=0.0, scope='conv2d', summary_labels=())

2D convolutional layer.

Parameters:
  • size – Number of filters
  • window – Convolution window size, either an integer or pair of integers.
  • stride – Convolution stride, either an integer or pair of integers.
  • padding – Convolution padding, one of ‘VALID’ or ‘SAME’
  • bias – If true, a bias is added
  • activation – Type of nonlinearity, or dict with name & arguments
  • l2_regularization – L2 regularization weight
  • l1_regularization – L1 regularization weight
from_spec(spec, kwargs=None)

Creates a layer from a specification dict.

get_summaries()
get_variables(include_nontrainable=False)
internals_spec()

Returns the internal states specification.

Returns:Internal states specification
tf_apply(x, update)
tf_regularization_loss()
tf_tensors(named_tensors)

Attaches the named_tensors dictionary to the layer for examination and update.

Parameters:named_tensors – Dictionary of named tensors to be used as Input’s or recorded from outputs
Returns:NA
class tensorforce.core.networks.layer.Dense(size=None, weights=None, bias=True, activation='relu', l2_regularization=0.0, l1_regularization=0.0, skip=False, scope='dense', summary_labels=())

Bases: tensorforce.core.networks.layer.Layer

Dense layer, i.e. linear fully connected layer with subsequent non-linearity.

__init__(size=None, weights=None, bias=True, activation='relu', l2_regularization=0.0, l1_regularization=0.0, skip=False, scope='dense', summary_labels=())

Dense layer.

Parameters:
  • size – Layer size, if None than input size matches the output size of the layer
  • weights – Weight initialization, random if None.
  • bias – If true, bias is added.
  • activation – Type of nonlinearity, or dict with name & arguments
  • l2_regularization – L2 regularization weight.
  • l1_regularization – L1 regularization weight.
  • skip – Add skip connection like ResNet (https://arxiv.org/pdf/1512.03385.pdf), doubles layers and ShortCut from Input to output
from_spec(spec, kwargs=None)

Creates a layer from a specification dict.

get_summaries()
get_variables(include_nontrainable=False)
internals_spec()

Returns the internal states specification.

Returns:Internal states specification
tf_apply(x, update)
tf_regularization_loss()
tf_tensors(named_tensors)

Attaches the named_tensors dictionary to the layer for examination and update.

Parameters:named_tensors – Dictionary of named tensors to be used as Input’s or recorded from outputs
Returns:NA
class tensorforce.core.networks.layer.Dropout(rate=0.0, scope='dropout', summary_labels=())

Bases: tensorforce.core.networks.layer.Layer

Dropout layer. If using dropout, add this layer after inputs and after dense layers. For LSTM, dropout is handled independently as an argument. Not available for Conv2d yet.

__init__(rate=0.0, scope='dropout', summary_labels=())
from_spec(spec, kwargs=None)

Creates a layer from a specification dict.

get_summaries()

Returns the TensorFlow summaries reported by the layer.

Returns:List of summaries.
get_variables(include_nontrainable=False)

Returns the TensorFlow variables used by the layer.

Returns:List of variables.
internals_spec()

Returns the internal states specification.

Returns:Internal states specification
tf_apply(x, update)
tf_regularization_loss()

Creates the TensorFlow operations for the layer regularization loss.

Returns:Regularization loss tensor.
tf_tensors(named_tensors)

Attaches the named_tensors dictionary to the layer for examination and update.

Parameters:named_tensors – Dictionary of named tensors to be used as Input’s or recorded from outputs
Returns:NA
class tensorforce.core.networks.layer.Dueling(size, bias=False, activation='none', l2_regularization=0.0, l1_regularization=0.0, output=None, scope='dueling', summary_labels=())

Bases: tensorforce.core.networks.layer.Layer

Dueling layer, i.e. Duel pipelines for Exp & Adv to help with stability

__init__(size, bias=False, activation='none', l2_regularization=0.0, l1_regularization=0.0, output=None, scope='dueling', summary_labels=())

Dueling layer.

[Dueling Networks] (https://arxiv.org/pdf/1511.06581.pdf) Implement Y = Expectation[x] + (Advantage[x] - Mean(Advantage[x]))

Parameters:
  • size – Layer size.
  • bias – If true, bias is added.
  • activation – Type of nonlinearity, or dict with name & arguments
  • l2_regularization – L2 regularization weight.
  • l1_regularization – L1 regularization weight.
  • output – None or tuple of output names for (‘expectation’,’advantage’,’mean_advantage’)
from_spec(spec, kwargs=None)

Creates a layer from a specification dict.

get_summaries()
get_variables(include_nontrainable=False)
internals_spec()

Returns the internal states specification.

Returns:Internal states specification
tf_apply(x, update)
tf_regularization_loss()
tf_tensors(named_tensors)

Attaches the named_tensors dictionary to the layer for examination and update.

Parameters:named_tensors – Dictionary of named tensors to be used as Input’s or recorded from outputs
Returns:NA
class tensorforce.core.networks.layer.Embedding(indices, size, l2_regularization=0.0, l1_regularization=0.0, scope='embedding', summary_labels=())

Bases: tensorforce.core.networks.layer.Layer

Embedding layer.

__init__(indices, size, l2_regularization=0.0, l1_regularization=0.0, scope='embedding', summary_labels=())

Embedding layer.

Parameters:
  • indices – Number of embedding indices.
  • size – Embedding size.
  • l2_regularization – L2 regularization weight.
  • l1_regularization – L1 regularization weight.
from_spec(spec, kwargs=None)

Creates a layer from a specification dict.

get_summaries()

Returns the TensorFlow summaries reported by the layer.

Returns:List of summaries.
get_variables(include_nontrainable=False)

Returns the TensorFlow variables used by the layer.

Returns:List of variables.
internals_spec()

Returns the internal states specification.

Returns:Internal states specification
tf_apply(x, update)
tf_regularization_loss()
tf_tensors(named_tensors)

Attaches the named_tensors dictionary to the layer for examination and update.

Parameters:named_tensors – Dictionary of named tensors to be used as Input’s or recorded from outputs
Returns:NA
class tensorforce.core.networks.layer.Flatten(scope='flatten', summary_labels=())

Bases: tensorforce.core.networks.layer.Layer

Flatten layer reshaping the input.

__init__(scope='flatten', summary_labels=())
from_spec(spec, kwargs=None)

Creates a layer from a specification dict.

get_summaries()

Returns the TensorFlow summaries reported by the layer.

Returns:List of summaries.
get_variables(include_nontrainable=False)

Returns the TensorFlow variables used by the layer.

Returns:List of variables.
internals_spec()

Returns the internal states specification.

Returns:Internal states specification
tf_apply(x, update)
tf_regularization_loss()

Creates the TensorFlow operations for the layer regularization loss.

Returns:Regularization loss tensor.
tf_tensors(named_tensors)

Attaches the named_tensors dictionary to the layer for examination and update.

Parameters:named_tensors – Dictionary of named tensors to be used as Input’s or recorded from outputs
Returns:NA
class tensorforce.core.networks.layer.InternalLstm(size, dropout=None, lstmcell_args={}, scope='internal_lstm', summary_labels=())

Bases: tensorforce.core.networks.layer.Layer

Long short-term memory layer for internal state management.

__init__(size, dropout=None, lstmcell_args={}, scope='internal_lstm', summary_labels=())

LSTM layer.

Parameters:
  • size – LSTM size.
  • dropout – Dropout rate.
from_spec(spec, kwargs=None)

Creates a layer from a specification dict.

get_summaries()

Returns the TensorFlow summaries reported by the layer.

Returns:List of summaries.
get_variables(include_nontrainable=False)

Returns the TensorFlow variables used by the layer.

Returns:List of variables.
internals_spec()
tf_apply(x, update, state)
tf_regularization_loss()

Creates the TensorFlow operations for the layer regularization loss.

Returns:Regularization loss tensor.
tf_tensors(named_tensors)

Attaches the named_tensors dictionary to the layer for examination and update.

Parameters:named_tensors – Dictionary of named tensors to be used as Input’s or recorded from outputs
Returns:NA
class tensorforce.core.networks.layer.Layer(scope='layer', summary_labels=None)

Bases: object

Base class for network layers.

__init__(scope='layer', summary_labels=None)

Layer.

static from_spec(spec, kwargs=None)

Creates a layer from a specification dict.

get_summaries()

Returns the TensorFlow summaries reported by the layer.

Returns:List of summaries.
get_variables(include_nontrainable=False)

Returns the TensorFlow variables used by the layer.

Returns:List of variables.
internals_spec()

Returns the internal states specification.

Returns:Internal states specification
tf_apply(x, update)

Creates the TensorFlow operations for applying the layer to the given input.

Parameters:
  • x – Layer input tensor.
  • update – Boolean tensor indicating whether this call happens during an update.
Returns:

Layer output tensor.
tf_regularization_loss()

Creates the TensorFlow operations for the layer regularization loss.

Returns:Regularization loss tensor.
tf_tensors(named_tensors)

Attaches the named_tensors dictionary to the layer for examination and update.

Parameters:named_tensors – Dictionary of named tensors to be used as Input’s or recorded from outputs
Returns:NA
class tensorforce.core.networks.layer.Linear(size, weights=None, bias=True, l2_regularization=0.0, l1_regularization=0.0, scope='linear', summary_labels=())

Bases: tensorforce.core.networks.layer.Layer

Linear fully-connected layer.

__init__(size, weights=None, bias=True, l2_regularization=0.0, l1_regularization=0.0, scope='linear', summary_labels=())

Linear layer.

Parameters:
  • size – Layer size.
  • weights – Weight initialization, random if None.
  • bias – Bias initialization, random if True, no bias added if False.
  • l2_regularization – L2 regularization weight.
  • l1_regularization – L1 regularization weight.
from_spec(spec, kwargs=None)

Creates a layer from a specification dict.

get_summaries()

Returns the TensorFlow summaries reported by the layer.

Returns:List of summaries.
get_variables(include_nontrainable=False)

Returns the TensorFlow variables used by the layer.

Returns:List of variables.
internals_spec()

Returns the internal states specification.

Returns:Internal states specification
tf_apply(x, update=False)
tf_regularization_loss()
tf_tensors(named_tensors)

Attaches the named_tensors dictionary to the layer for examination and update.

Parameters:named_tensors – Dictionary of named tensors to be used as Input’s or recorded from outputs
Returns:NA
class tensorforce.core.networks.layer.Lstm(size, dropout=None, scope='lstm', summary_labels=(), return_final_state=True)

Bases: tensorforce.core.networks.layer.Layer

__init__(size, dropout=None, scope='lstm', summary_labels=(), return_final_state=True)

LSTM layer.

Parameters:
  • size – LSTM size.
  • dropout – Dropout rate.
from_spec(spec, kwargs=None)

Creates a layer from a specification dict.

get_summaries()

Returns the TensorFlow summaries reported by the layer.

Returns:List of summaries.
get_variables(include_nontrainable=False)

Returns the TensorFlow variables used by the layer.

Returns:List of variables.
internals_spec()

Returns the internal states specification.

Returns:Internal states specification
tf_apply(x, update, sequence_length=None)
tf_regularization_loss()

Creates the TensorFlow operations for the layer regularization loss.

Returns:Regularization loss tensor.
tf_tensors(named_tensors)

Attaches the named_tensors dictionary to the layer for examination and update.

Parameters:named_tensors – Dictionary of named tensors to be used as Input’s or recorded from outputs
Returns:NA
class tensorforce.core.networks.layer.Nonlinearity(name='relu', alpha=None, beta=1.0, max=None, min=None, scope='nonlinearity', summary_labels=())

Bases: tensorforce.core.networks.layer.Layer

Non-linearity layer applying a non-linear transformation.

__init__(name='relu', alpha=None, beta=1.0, max=None, min=None, scope='nonlinearity', summary_labels=())

Non-linearity activation layer.

Parameters:
  • name – Non-linearity name, one of ‘elu’, ‘relu’, ‘selu’, ‘sigmoid’, ‘swish’, ‘softmax’, ‘leaky_relu’ (or ‘lrelu’), ‘crelu’, ‘softmax’, ‘softplus’, ‘softsign’, ‘tanh’ or ‘none’.
  • alpha – (float|int) Alpha value for leaky Relu
  • beta – (float|int|’learn’) Beta value or ‘learn’ to train value (default 1.0)
  • max – (float|int) maximum (beta * input) value passed to non-linearity function
  • min – (float|int) minimum (beta * input) value passed to non-linearity function
  • summary_labels – Requested summary labels for tensorboard export, add ‘beta’ to watch beta learning
from_spec(spec, kwargs=None)

Creates a layer from a specification dict.

get_summaries()

Returns the TensorFlow summaries reported by the layer.

Returns:List of summaries.
get_variables(include_nontrainable=False)

Returns the TensorFlow variables used by the layer.

Returns:List of variables.
internals_spec()

Returns the internal states specification.

Returns:Internal states specification
tf_apply(x, update)
tf_regularization_loss()

Creates the TensorFlow operations for the layer regularization loss.

Returns:Regularization loss tensor.
tf_tensors(named_tensors)

Attaches the named_tensors dictionary to the layer for examination and update.

Parameters:named_tensors – Dictionary of named tensors to be used as Input’s or recorded from outputs
Returns:NA
class tensorforce.core.networks.layer.Pool2d(pooling_type='max', window=2, stride=2, padding='SAME', scope='pool2d', summary_labels=())

Bases: tensorforce.core.networks.layer.Layer

2-dimensional pooling layer.

__init__(pooling_type='max', window=2, stride=2, padding='SAME', scope='pool2d', summary_labels=())

2-dimensional pooling layer.

Parameters:
  • pooling_type – Either ‘max’ or ‘average’.
  • window – Pooling window size, either an integer or pair of integers.
  • stride – Pooling stride, either an integer or pair of integers.
  • padding – Pooling padding, one of ‘VALID’ or ‘SAME’.
from_spec(spec, kwargs=None)

Creates a layer from a specification dict.

get_summaries()

Returns the TensorFlow summaries reported by the layer.

Returns:List of summaries.
get_variables(include_nontrainable=False)

Returns the TensorFlow variables used by the layer.

Returns:List of variables.
internals_spec()

Returns the internal states specification.

Returns:Internal states specification
tf_apply(x, update)
tf_regularization_loss()

Creates the TensorFlow operations for the layer regularization loss.

Returns:Regularization loss tensor.
tf_tensors(named_tensors)

Attaches the named_tensors dictionary to the layer for examination and update.

Parameters:named_tensors – Dictionary of named tensors to be used as Input’s or recorded from outputs
Returns:NA
class tensorforce.core.networks.layer.TFLayer(layer, scope='tf-layer', summary_labels=(), **kwargs)

Bases: tensorforce.core.networks.layer.Layer

Wrapper class for TensorFlow layers.

__init__(layer, scope='tf-layer', summary_labels=(), **kwargs)

Creates a new layer instance of a TensorFlow layer.

Parameters:
  • name – The name of the layer, one of ‘dense’.
  • **kwargs –

    Additional arguments passed on to the TensorFlow layer constructor.
from_spec(spec, kwargs=None)

Creates a layer from a specification dict.

get_summaries()

Returns the TensorFlow summaries reported by the layer.

Returns:List of summaries.
get_variables(include_nontrainable=False)

Returns the TensorFlow variables used by the layer.

Returns:List of variables.
internals_spec()

Returns the internal states specification.

Returns:Internal states specification
tf_apply(x, update)
tf_layers = {'conv1d': <sphinx.ext.autodoc._MockObject object>, 'dropout': <sphinx.ext.autodoc._MockObject object>, 'max_pooling1d': <sphinx.ext.autodoc._MockObject object>, 'flatten': <sphinx.ext.autodoc._MockObject object>, 'average_pooling1d': <sphinx.ext.autodoc._MockObject object>, 'separable_conv2d': <sphinx.ext.autodoc._MockObject object>, 'max_pooling3d': <sphinx.ext.autodoc._MockObject object>, 'max_pooling2d': <sphinx.ext.autodoc._MockObject object>, 'conv2d_transpose': <sphinx.ext.autodoc._MockObject object>, 'conv3d': <sphinx.ext.autodoc._MockObject object>, 'dense': <sphinx.ext.autodoc._MockObject object>, 'batch_normalization': <sphinx.ext.autodoc._MockObject object>, 'average_pooling3d': <sphinx.ext.autodoc._MockObject object>, 'conv3d_transpose': <sphinx.ext.autodoc._MockObject object>, 'average_pooling2d': <sphinx.ext.autodoc._MockObject object>, 'conv2d': <sphinx.ext.autodoc._MockObject object>}
tf_regularization_loss()
tf_tensors(named_tensors)

Attaches the named_tensors dictionary to the layer for examination and update.

Parameters:named_tensors – Dictionary of named tensors to be used as Input’s or recorded from outputs
Returns:NA

tensorforce.core.networks.network module

class tensorforce.core.networks.network.LayerBasedNetwork(scope='layerbased-network', summary_labels=())

Bases: tensorforce.core.networks.network.Network

Base class for networks using TensorForce layers.

__init__(scope='layerbased-network', summary_labels=())

Layer-based network.

add_layer(layer)
from_spec(spec, kwargs=None)

Creates a network from a specification dict.

get_list_of_named_tensor()

Returns a list of the names of tensors available.

Returns:List of the names of tensors available.
get_named_tensor(name)

Returns a named tensor if available.

Returns:True if named tensor found, False otherwise tensor: If valid, will be a tensor, otherwise None
Return type:valid
get_summaries()
get_variables(include_nontrainable=False)
internals_spec()
set_named_tensor(name, tensor)

Returns the TensorFlow summaries reported by the network.

Returns:None
tf_apply(x, internals, update, return_internals=False)

Creates the TensorFlow operations for applying the network to the given input.

Parameters:
  • x – Network input tensor or dict of input tensors.
  • internals – List of prior internal state tensors
  • update – Boolean tensor indicating whether this call happens during an update.
  • return_internals – If true, also returns posterior internal state tensors
Returns:

Network output tensor, plus optionally list of posterior internal state tensors
tf_regularization_loss()
class tensorforce.core.networks.network.LayeredNetwork(layers, scope='layered-network', summary_labels=())

Bases: tensorforce.core.networks.network.LayerBasedNetwork

Network consisting of a sequence of layers, which can be created from a specification dict.

__init__(layers, scope='layered-network', summary_labels=())

Single-stack layered network.

Parameters:layers – List of layer specification dicts.
add_layer(layer)
static from_json(filename)

Creates a layer_networkd_builder from a JSON.

Parameters:filename – Path to configuration

Returns: A layered_network_builder function with layers generated from the JSON

from_spec(spec, kwargs=None)

Creates a network from a specification dict.

get_list_of_named_tensor()

Returns a list of the names of tensors available.

Returns:List of the names of tensors available.
get_named_tensor(name)

Returns a named tensor if available.

Returns:True if named tensor found, False otherwise tensor: If valid, will be a tensor, otherwise None
Return type:valid
get_summaries()
get_variables(include_nontrainable=False)
internals_spec()
set_named_tensor(name, tensor)

Returns the TensorFlow summaries reported by the network.

Returns:None
tf_apply(x, internals, update, return_internals=False)
tf_regularization_loss()
class tensorforce.core.networks.network.Network(scope='network', summary_labels=None)

Bases: object

Base class for neural networks.

__init__(scope='network', summary_labels=None)

Neural network.

static from_spec(spec, kwargs=None)

Creates a network from a specification dict.

get_list_of_named_tensor()

Returns a list of the names of tensors available.

Returns:List of the names of tensors available.
get_named_tensor(name)

Returns a named tensor if available.

Returns:True if named tensor found, False otherwise tensor: If valid, will be a tensor, otherwise None
Return type:valid
get_summaries()

Returns the TensorFlow summaries reported by the network.

Returns:List of summaries
get_variables(include_nontrainable=False)

Returns the TensorFlow variables used by the network.

Returns:List of variables
internals_spec()

Returns the internal states specification.

Returns:Internal states specification
set_named_tensor(name, tensor)

Returns the TensorFlow summaries reported by the network.

Returns:None
tf_apply(x, internals, update, return_internals=False)

Creates the TensorFlow operations for applying the network to the given input.

Parameters:
  • x – Network input tensor or dict of input tensors.
  • internals – List of prior internal state tensors
  • update – Boolean tensor indicating whether this call happens during an update.
  • return_internals – If true, also returns posterior internal state tensors
Returns:

Network output tensor, plus optionally list of posterior internal state tensors
tf_regularization_loss()

Creates the TensorFlow operations for the network regularization loss.

Returns:Regularization loss tensor

Module contents

class tensorforce.core.networks.Layer(scope='layer', summary_labels=None)

Bases: object

Base class for network layers.

__init__(scope='layer', summary_labels=None)

Layer.

static from_spec(spec, kwargs=None)

Creates a layer from a specification dict.

get_summaries()

Returns the TensorFlow summaries reported by the layer.

Returns:List of summaries.
get_variables(include_nontrainable=False)

Returns the TensorFlow variables used by the layer.

Returns:List of variables.
internals_spec()

Returns the internal states specification.

Returns:Internal states specification
tf_apply(x, update)

Creates the TensorFlow operations for applying the layer to the given input.

Parameters:
  • x – Layer input tensor.
  • update – Boolean tensor indicating whether this call happens during an update.
Returns:

Layer output tensor.
tf_regularization_loss()

Creates the TensorFlow operations for the layer regularization loss.

Returns:Regularization loss tensor.
tf_tensors(named_tensors)

Attaches the named_tensors dictionary to the layer for examination and update.

Parameters:named_tensors – Dictionary of named tensors to be used as Input’s or recorded from outputs
Returns:NA
class tensorforce.core.networks.TFLayer(layer, scope='tf-layer', summary_labels=(), **kwargs)

Bases: tensorforce.core.networks.layer.Layer

Wrapper class for TensorFlow layers.

__init__(layer, scope='tf-layer', summary_labels=(), **kwargs)

Creates a new layer instance of a TensorFlow layer.

Parameters:
  • name – The name of the layer, one of ‘dense’.
  • **kwargs –

    Additional arguments passed on to the TensorFlow layer constructor.
from_spec(spec, kwargs=None)

Creates a layer from a specification dict.

get_summaries()

Returns the TensorFlow summaries reported by the layer.

Returns:List of summaries.
get_variables(include_nontrainable=False)

Returns the TensorFlow variables used by the layer.

Returns:List of variables.
internals_spec()

Returns the internal states specification.

Returns:Internal states specification
tf_apply(x, update)
tf_layers = {'conv1d': <sphinx.ext.autodoc._MockObject object>, 'dropout': <sphinx.ext.autodoc._MockObject object>, 'max_pooling1d': <sphinx.ext.autodoc._MockObject object>, 'flatten': <sphinx.ext.autodoc._MockObject object>, 'average_pooling1d': <sphinx.ext.autodoc._MockObject object>, 'separable_conv2d': <sphinx.ext.autodoc._MockObject object>, 'max_pooling3d': <sphinx.ext.autodoc._MockObject object>, 'max_pooling2d': <sphinx.ext.autodoc._MockObject object>, 'conv2d_transpose': <sphinx.ext.autodoc._MockObject object>, 'conv3d': <sphinx.ext.autodoc._MockObject object>, 'dense': <sphinx.ext.autodoc._MockObject object>, 'batch_normalization': <sphinx.ext.autodoc._MockObject object>, 'average_pooling3d': <sphinx.ext.autodoc._MockObject object>, 'conv3d_transpose': <sphinx.ext.autodoc._MockObject object>, 'average_pooling2d': <sphinx.ext.autodoc._MockObject object>, 'conv2d': <sphinx.ext.autodoc._MockObject object>}
tf_regularization_loss()
tf_tensors(named_tensors)

Attaches the named_tensors dictionary to the layer for examination and update.

Parameters:named_tensors – Dictionary of named tensors to be used as Input’s or recorded from outputs
Returns:NA
class tensorforce.core.networks.Nonlinearity(name='relu', alpha=None, beta=1.0, max=None, min=None, scope='nonlinearity', summary_labels=())

Bases: tensorforce.core.networks.layer.Layer

Non-linearity layer applying a non-linear transformation.

__init__(name='relu', alpha=None, beta=1.0, max=None, min=None, scope='nonlinearity', summary_labels=())

Non-linearity activation layer.

Parameters:
  • name – Non-linearity name, one of ‘elu’, ‘relu’, ‘selu’, ‘sigmoid’, ‘swish’, ‘softmax’, ‘leaky_relu’ (or ‘lrelu’), ‘crelu’, ‘softmax’, ‘softplus’, ‘softsign’, ‘tanh’ or ‘none’.
  • alpha – (float|int) Alpha value for leaky Relu
  • beta – (float|int|’learn’) Beta value or ‘learn’ to train value (default 1.0)
  • max – (float|int) maximum (beta * input) value passed to non-linearity function
  • min – (float|int) minimum (beta * input) value passed to non-linearity function
  • summary_labels – Requested summary labels for tensorboard export, add ‘beta’ to watch beta learning
from_spec(spec, kwargs=None)

Creates a layer from a specification dict.

get_summaries()

Returns the TensorFlow summaries reported by the layer.

Returns:List of summaries.
get_variables(include_nontrainable=False)

Returns the TensorFlow variables used by the layer.

Returns:List of variables.
internals_spec()

Returns the internal states specification.

Returns:Internal states specification
tf_apply(x, update)
tf_regularization_loss()

Creates the TensorFlow operations for the layer regularization loss.

Returns:Regularization loss tensor.
tf_tensors(named_tensors)

Attaches the named_tensors dictionary to the layer for examination and update.

Parameters:named_tensors – Dictionary of named tensors to be used as Input’s or recorded from outputs
Returns:NA
class tensorforce.core.networks.Dropout(rate=0.0, scope='dropout', summary_labels=())

Bases: tensorforce.core.networks.layer.Layer

Dropout layer. If using dropout, add this layer after inputs and after dense layers. For LSTM, dropout is handled independently as an argument. Not available for Conv2d yet.

__init__(rate=0.0, scope='dropout', summary_labels=())
from_spec(spec, kwargs=None)

Creates a layer from a specification dict.

get_summaries()

Returns the TensorFlow summaries reported by the layer.

Returns:List of summaries.
get_variables(include_nontrainable=False)

Returns the TensorFlow variables used by the layer.

Returns:List of variables.
internals_spec()

Returns the internal states specification.

Returns:Internal states specification
tf_apply(x, update)
tf_regularization_loss()

Creates the TensorFlow operations for the layer regularization loss.

Returns:Regularization loss tensor.
tf_tensors(named_tensors)

Attaches the named_tensors dictionary to the layer for examination and update.

Parameters:named_tensors – Dictionary of named tensors to be used as Input’s or recorded from outputs
Returns:NA
class tensorforce.core.networks.Flatten(scope='flatten', summary_labels=())

Bases: tensorforce.core.networks.layer.Layer

Flatten layer reshaping the input.

__init__(scope='flatten', summary_labels=())
from_spec(spec, kwargs=None)

Creates a layer from a specification dict.

get_summaries()

Returns the TensorFlow summaries reported by the layer.

Returns:List of summaries.
get_variables(include_nontrainable=False)

Returns the TensorFlow variables used by the layer.

Returns:List of variables.
internals_spec()

Returns the internal states specification.

Returns:Internal states specification
tf_apply(x, update)
tf_regularization_loss()

Creates the TensorFlow operations for the layer regularization loss.

Returns:Regularization loss tensor.
tf_tensors(named_tensors)

Attaches the named_tensors dictionary to the layer for examination and update.

Parameters:named_tensors – Dictionary of named tensors to be used as Input’s or recorded from outputs
Returns:NA
class tensorforce.core.networks.Pool2d(pooling_type='max', window=2, stride=2, padding='SAME', scope='pool2d', summary_labels=())

Bases: tensorforce.core.networks.layer.Layer

2-dimensional pooling layer.

__init__(pooling_type='max', window=2, stride=2, padding='SAME', scope='pool2d', summary_labels=())

2-dimensional pooling layer.

Parameters:
  • pooling_type – Either ‘max’ or ‘average’.
  • window – Pooling window size, either an integer or pair of integers.
  • stride – Pooling stride, either an integer or pair of integers.
  • padding – Pooling padding, one of ‘VALID’ or ‘SAME’.
from_spec(spec, kwargs=None)

Creates a layer from a specification dict.

get_summaries()

Returns the TensorFlow summaries reported by the layer.

Returns:List of summaries.
get_variables(include_nontrainable=False)

Returns the TensorFlow variables used by the layer.

Returns:List of variables.
internals_spec()

Returns the internal states specification.

Returns:Internal states specification
tf_apply(x, update)
tf_regularization_loss()

Creates the TensorFlow operations for the layer regularization loss.

Returns:Regularization loss tensor.
tf_tensors(named_tensors)

Attaches the named_tensors dictionary to the layer for examination and update.

Parameters:named_tensors – Dictionary of named tensors to be used as Input’s or recorded from outputs
Returns:NA
class tensorforce.core.networks.Embedding(indices, size, l2_regularization=0.0, l1_regularization=0.0, scope='embedding', summary_labels=())

Bases: tensorforce.core.networks.layer.Layer

Embedding layer.

__init__(indices, size, l2_regularization=0.0, l1_regularization=0.0, scope='embedding', summary_labels=())

Embedding layer.

Parameters:
  • indices – Number of embedding indices.
  • size – Embedding size.
  • l2_regularization – L2 regularization weight.
  • l1_regularization – L1 regularization weight.
from_spec(spec, kwargs=None)

Creates a layer from a specification dict.

get_summaries()

Returns the TensorFlow summaries reported by the layer.

Returns:List of summaries.
get_variables(include_nontrainable=False)

Returns the TensorFlow variables used by the layer.

Returns:List of variables.
internals_spec()

Returns the internal states specification.

Returns:Internal states specification
tf_apply(x, update)
tf_regularization_loss()
tf_tensors(named_tensors)

Attaches the named_tensors dictionary to the layer for examination and update.

Parameters:named_tensors – Dictionary of named tensors to be used as Input’s or recorded from outputs
Returns:NA
class tensorforce.core.networks.Linear(size, weights=None, bias=True, l2_regularization=0.0, l1_regularization=0.0, scope='linear', summary_labels=())

Bases: tensorforce.core.networks.layer.Layer

Linear fully-connected layer.

__init__(size, weights=None, bias=True, l2_regularization=0.0, l1_regularization=0.0, scope='linear', summary_labels=())

Linear layer.

Parameters:
  • size – Layer size.
  • weights – Weight initialization, random if None.
  • bias – Bias initialization, random if True, no bias added if False.
  • l2_regularization – L2 regularization weight.
  • l1_regularization – L1 regularization weight.
from_spec(spec, kwargs=None)

Creates a layer from a specification dict.

get_summaries()

Returns the TensorFlow summaries reported by the layer.

Returns:List of summaries.
get_variables(include_nontrainable=False)

Returns the TensorFlow variables used by the layer.

Returns:List of variables.
internals_spec()

Returns the internal states specification.

Returns:Internal states specification
tf_apply(x, update=False)
tf_regularization_loss()
tf_tensors(named_tensors)

Attaches the named_tensors dictionary to the layer for examination and update.

Parameters:named_tensors – Dictionary of named tensors to be used as Input’s or recorded from outputs
Returns:NA
class tensorforce.core.networks.Dense(size=None, weights=None, bias=True, activation='relu', l2_regularization=0.0, l1_regularization=0.0, skip=False, scope='dense', summary_labels=())

Bases: tensorforce.core.networks.layer.Layer

Dense layer, i.e. linear fully connected layer with subsequent non-linearity.

__init__(size=None, weights=None, bias=True, activation='relu', l2_regularization=0.0, l1_regularization=0.0, skip=False, scope='dense', summary_labels=())

Dense layer.

Parameters:
  • size – Layer size, if None than input size matches the output size of the layer
  • weights – Weight initialization, random if None.
  • bias – If true, bias is added.
  • activation – Type of nonlinearity, or dict with name & arguments
  • l2_regularization – L2 regularization weight.
  • l1_regularization – L1 regularization weight.
  • skip – Add skip connection like ResNet (https://arxiv.org/pdf/1512.03385.pdf), doubles layers and ShortCut from Input to output
from_spec(spec, kwargs=None)

Creates a layer from a specification dict.

get_summaries()
get_variables(include_nontrainable=False)
internals_spec()

Returns the internal states specification.

Returns:Internal states specification
tf_apply(x, update)
tf_regularization_loss()
tf_tensors(named_tensors)

Attaches the named_tensors dictionary to the layer for examination and update.

Parameters:named_tensors – Dictionary of named tensors to be used as Input’s or recorded from outputs
Returns:NA
class tensorforce.core.networks.Dueling(size, bias=False, activation='none', l2_regularization=0.0, l1_regularization=0.0, output=None, scope='dueling', summary_labels=())

Bases: tensorforce.core.networks.layer.Layer

Dueling layer, i.e. Duel pipelines for Exp & Adv to help with stability

__init__(size, bias=False, activation='none', l2_regularization=0.0, l1_regularization=0.0, output=None, scope='dueling', summary_labels=())

Dueling layer.

[Dueling Networks] (https://arxiv.org/pdf/1511.06581.pdf) Implement Y = Expectation[x] + (Advantage[x] - Mean(Advantage[x]))

Parameters:
  • size – Layer size.
  • bias – If true, bias is added.
  • activation – Type of nonlinearity, or dict with name & arguments
  • l2_regularization – L2 regularization weight.
  • l1_regularization – L1 regularization weight.
  • output – None or tuple of output names for (‘expectation’,’advantage’,’mean_advantage’)
from_spec(spec, kwargs=None)

Creates a layer from a specification dict.

get_summaries()
get_variables(include_nontrainable=False)
internals_spec()

Returns the internal states specification.

Returns:Internal states specification
tf_apply(x, update)
tf_regularization_loss()
tf_tensors(named_tensors)

Attaches the named_tensors dictionary to the layer for examination and update.

Parameters:named_tensors – Dictionary of named tensors to be used as Input’s or recorded from outputs
Returns:NA
class tensorforce.core.networks.Conv1d(size, window=3, stride=1, padding='SAME', bias=True, activation='relu', l2_regularization=0.0, l1_regularization=0.0, scope='conv1d', summary_labels=())

Bases: tensorforce.core.networks.layer.Layer

1-dimensional convolutional layer.

__init__(size, window=3, stride=1, padding='SAME', bias=True, activation='relu', l2_regularization=0.0, l1_regularization=0.0, scope='conv1d', summary_labels=())

1D convolutional layer.

Parameters:
  • size – Number of filters
  • window – Convolution window size
  • stride – Convolution stride
  • padding – Convolution padding, one of ‘VALID’ or ‘SAME’
  • bias – If true, a bias is added
  • activation – Type of nonlinearity, or dict with name & arguments
  • l2_regularization – L2 regularization weight
  • l1_regularization – L1 regularization weight
from_spec(spec, kwargs=None)

Creates a layer from a specification dict.

get_summaries()
get_variables(include_nontrainable=False)
internals_spec()

Returns the internal states specification.

Returns:Internal states specification
tf_apply(x, update)
tf_regularization_loss()
tf_tensors(named_tensors)

Attaches the named_tensors dictionary to the layer for examination and update.

Parameters:named_tensors – Dictionary of named tensors to be used as Input’s or recorded from outputs
Returns:NA
class tensorforce.core.networks.Conv2d(size, window=3, stride=1, padding='SAME', bias=True, activation='relu', l2_regularization=0.0, l1_regularization=0.0, scope='conv2d', summary_labels=())

Bases: tensorforce.core.networks.layer.Layer

2-dimensional convolutional layer.

__init__(size, window=3, stride=1, padding='SAME', bias=True, activation='relu', l2_regularization=0.0, l1_regularization=0.0, scope='conv2d', summary_labels=())

2D convolutional layer.

Parameters:
  • size – Number of filters
  • window – Convolution window size, either an integer or pair of integers.
  • stride – Convolution stride, either an integer or pair of integers.
  • padding – Convolution padding, one of ‘VALID’ or ‘SAME’
  • bias – If true, a bias is added
  • activation – Type of nonlinearity, or dict with name & arguments
  • l2_regularization – L2 regularization weight
  • l1_regularization – L1 regularization weight
from_spec(spec, kwargs=None)

Creates a layer from a specification dict.

get_summaries()
get_variables(include_nontrainable=False)
internals_spec()

Returns the internal states specification.

Returns:Internal states specification
tf_apply(x, update)
tf_regularization_loss()
tf_tensors(named_tensors)

Attaches the named_tensors dictionary to the layer for examination and update.

Parameters:named_tensors – Dictionary of named tensors to be used as Input’s or recorded from outputs
Returns:NA
class tensorforce.core.networks.InternalLstm(size, dropout=None, lstmcell_args={}, scope='internal_lstm', summary_labels=())

Bases: tensorforce.core.networks.layer.Layer

Long short-term memory layer for internal state management.

__init__(size, dropout=None, lstmcell_args={}, scope='internal_lstm', summary_labels=())

LSTM layer.

Parameters:
  • size – LSTM size.
  • dropout – Dropout rate.
from_spec(spec, kwargs=None)

Creates a layer from a specification dict.

get_summaries()

Returns the TensorFlow summaries reported by the layer.

Returns:List of summaries.
get_variables(include_nontrainable=False)

Returns the TensorFlow variables used by the layer.

Returns:List of variables.
internals_spec()
tf_apply(x, update, state)
tf_regularization_loss()

Creates the TensorFlow operations for the layer regularization loss.

Returns:Regularization loss tensor.
tf_tensors(named_tensors)

Attaches the named_tensors dictionary to the layer for examination and update.

Parameters:named_tensors – Dictionary of named tensors to be used as Input’s or recorded from outputs
Returns:NA
class tensorforce.core.networks.Lstm(size, dropout=None, scope='lstm', summary_labels=(), return_final_state=True)

Bases: tensorforce.core.networks.layer.Layer

__init__(size, dropout=None, scope='lstm', summary_labels=(), return_final_state=True)

LSTM layer.

Parameters:
  • size – LSTM size.
  • dropout – Dropout rate.
from_spec(spec, kwargs=None)

Creates a layer from a specification dict.

get_summaries()

Returns the TensorFlow summaries reported by the layer.

Returns:List of summaries.
get_variables(include_nontrainable=False)

Returns the TensorFlow variables used by the layer.

Returns:List of variables.
internals_spec()

Returns the internal states specification.

Returns:Internal states specification
tf_apply(x, update, sequence_length=None)
tf_regularization_loss()

Creates the TensorFlow operations for the layer regularization loss.

Returns:Regularization loss tensor.
tf_tensors(named_tensors)

Attaches the named_tensors dictionary to the layer for examination and update.

Parameters:named_tensors – Dictionary of named tensors to be used as Input’s or recorded from outputs
Returns:NA
class tensorforce.core.networks.Network(scope='network', summary_labels=None)

Bases: object

Base class for neural networks.

__init__(scope='network', summary_labels=None)

Neural network.

static from_spec(spec, kwargs=None)

Creates a network from a specification dict.

get_list_of_named_tensor()

Returns a list of the names of tensors available.

Returns:List of the names of tensors available.
get_named_tensor(name)

Returns a named tensor if available.

Returns:True if named tensor found, False otherwise tensor: If valid, will be a tensor, otherwise None
Return type:valid
get_summaries()

Returns the TensorFlow summaries reported by the network.

Returns:List of summaries
get_variables(include_nontrainable=False)

Returns the TensorFlow variables used by the network.

Returns:List of variables
internals_spec()

Returns the internal states specification.

Returns:Internal states specification
set_named_tensor(name, tensor)

Returns the TensorFlow summaries reported by the network.

Returns:None
tf_apply(x, internals, update, return_internals=False)

Creates the TensorFlow operations for applying the network to the given input.

Parameters:
  • x – Network input tensor or dict of input tensors.
  • internals – List of prior internal state tensors
  • update – Boolean tensor indicating whether this call happens during an update.
  • return_internals – If true, also returns posterior internal state tensors
Returns:

Network output tensor, plus optionally list of posterior internal state tensors
tf_regularization_loss()

Creates the TensorFlow operations for the network regularization loss.

Returns:Regularization loss tensor
class tensorforce.core.networks.LayerBasedNetwork(scope='layerbased-network', summary_labels=())

Bases: tensorforce.core.networks.network.Network

Base class for networks using TensorForce layers.

__init__(scope='layerbased-network', summary_labels=())

Layer-based network.

add_layer(layer)
from_spec(spec, kwargs=None)

Creates a network from a specification dict.

get_list_of_named_tensor()

Returns a list of the names of tensors available.

Returns:List of the names of tensors available.
get_named_tensor(name)

Returns a named tensor if available.

Returns:True if named tensor found, False otherwise tensor: If valid, will be a tensor, otherwise None
Return type:valid
get_summaries()
get_variables(include_nontrainable=False)
internals_spec()
set_named_tensor(name, tensor)

Returns the TensorFlow summaries reported by the network.

Returns:None
tf_apply(x, internals, update, return_internals=False)

Creates the TensorFlow operations for applying the network to the given input.

Parameters:
  • x – Network input tensor or dict of input tensors.
  • internals – List of prior internal state tensors
  • update – Boolean tensor indicating whether this call happens during an update.
  • return_internals – If true, also returns posterior internal state tensors
Returns:

Network output tensor, plus optionally list of posterior internal state tensors
tf_regularization_loss()
class tensorforce.core.networks.LayeredNetwork(layers, scope='layered-network', summary_labels=())

Bases: tensorforce.core.networks.network.LayerBasedNetwork

Network consisting of a sequence of layers, which can be created from a specification dict.

__init__(layers, scope='layered-network', summary_labels=())

Single-stack layered network.

Parameters:layers – List of layer specification dicts.
add_layer(layer)
static from_json(filename)

Creates a layer_networkd_builder from a JSON.

Parameters:filename – Path to configuration

Returns: A layered_network_builder function with layers generated from the JSON

from_spec(spec, kwargs=None)

Creates a network from a specification dict.

get_list_of_named_tensor()

Returns a list of the names of tensors available.

Returns:List of the names of tensors available.
get_named_tensor(name)

Returns a named tensor if available.

Returns:True if named tensor found, False otherwise tensor: If valid, will be a tensor, otherwise None
Return type:valid
get_summaries()
get_variables(include_nontrainable=False)
internals_spec()
set_named_tensor(name, tensor)

Returns the TensorFlow summaries reported by the network.

Returns:None
tf_apply(x, internals, update, return_internals=False)
tf_regularization_loss()