tensorforce.core.networks package

Submodules

tensorforce.core.networks.layer module

Collection of custom layer implementations. We prefer not to use contrib-layers to retain full control over shapes and internal states.

class tensorforce.core.networks.layer.Conv1d(size, window=3, stride=1, padding='SAME', bias=True, activation='relu', l2_regularization=0.0, l1_regularization=0.0, scope='conv1d', summary_labels=())

Bases: tensorforce.core.networks.layer.Layer

1-dimensional convolutional layer.

get_summaries()
get_variables(include_non_trainable=False)
tf_apply(x, update)
tf_regularization_loss()
class tensorforce.core.networks.layer.Conv2d(size, window=3, stride=1, padding='SAME', bias=True, activation='relu', l2_regularization=0.0, l1_regularization=0.0, scope='conv2d', summary_labels=())

Bases: tensorforce.core.networks.layer.Layer

2-dimensional convolutional layer.

get_summaries()
get_variables(include_non_trainable=False)
tf_apply(x, update)
tf_regularization_loss()
class tensorforce.core.networks.layer.Dense(size=None, bias=True, activation='tanh', l2_regularization=0.0, l1_regularization=0.0, skip=False, scope='dense', summary_labels=())

Bases: tensorforce.core.networks.layer.Layer

Dense layer, i.e. linear fully connected layer with subsequent non-linearity.

get_summaries()
get_variables(include_non_trainable=False)
tf_apply(x, update)
tf_regularization_loss()
class tensorforce.core.networks.layer.Dropout(rate=0.0, scope='dropout', summary_labels=())

Bases: tensorforce.core.networks.layer.Layer

Dropout layer. If using dropout, add this layer after inputs and after dense layers. For LSTM, dropout is handled independently as an argument. Not available for Conv2d yet.

tf_apply(x, update)
class tensorforce.core.networks.layer.Dueling(size, bias=False, activation='none', l2_regularization=0.0, l1_regularization=0.0, output=None, scope='dueling', summary_labels=())

Bases: tensorforce.core.networks.layer.Layer

Dueling layer, i.e. Duel pipelines for Exp & Adv to help with stability

get_summaries()
get_variables(include_non_trainable=False)
tf_apply(x, update)
tf_regularization_loss()
class tensorforce.core.networks.layer.Embedding(indices, size, l2_regularization=0.0, l1_regularization=0.0, scope='embedding', summary_labels=())

Bases: tensorforce.core.networks.layer.Layer

Embedding layer.

tf_apply(x, update)
tf_regularization_loss()
class tensorforce.core.networks.layer.Flatten(scope='flatten', summary_labels=())

Bases: tensorforce.core.networks.layer.Layer

Flatten layer reshaping the input.

tf_apply(x, update)
class tensorforce.core.networks.layer.InternalLstm(size, dropout=None, scope='internal_lstm', summary_labels=())

Bases: tensorforce.core.networks.layer.Layer

Long short-term memory layer for internal state management.

internals_init()
internals_input()
tf_apply(x, update, state)
class tensorforce.core.networks.layer.Layer(num_internals=0, scope='layer', summary_labels=None)

Bases: object

Base class for network layers.

static from_spec(spec, kwargs=None)

Creates a layer from a specification dict.

get_summaries()

Returns the TensorFlow summaries reported by the layer.

Returns:List of summaries.
get_variables(include_non_trainable=False)

Returns the TensorFlow variables used by the layer.

Returns:List of variables.
internals_init()

Returns the TensorFlow tensors for internal state initializations.

Returns:List of internal state initialization tensors.
internals_input()

Returns the TensorFlow placeholders for internal state inputs.

Returns:List of internal state input placeholders.
tf_apply(x, update)

Creates the TensorFlow operations for applying the layer to the given input.

Parameters:
  • x – Layer input tensor.
  • update – Boolean tensor indicating whether this call happens during an update.
Returns:

Layer output tensor.
tf_regularization_loss()

Creates the TensorFlow operations for the layer regularization loss.

Returns:Regularization loss tensor.
tf_tensors(named_tensors)

Attaches the named_tensors dictionary to the layer for examination and update.

Parameters:named_tensors – Dictionary of named tensors to be used as Input’s or recorded from outputs
Returns:NA
class tensorforce.core.networks.layer.Linear(size, weights=None, bias=True, l2_regularization=0.0, l1_regularization=0.0, scope='linear', summary_labels=())

Bases: tensorforce.core.networks.layer.Layer

Linear fully-connected layer.

tf_apply(x, update=False)
tf_regularization_loss()
class tensorforce.core.networks.layer.Lstm(size, dropout=None, scope='lstm', summary_labels=(), return_final_state=True)

Bases: tensorforce.core.networks.layer.Layer

tf_apply(x, update, sequence_length=None)
class tensorforce.core.networks.layer.Nonlinearity(name='relu', scope='nonlinearity', summary_labels=())

Bases: tensorforce.core.networks.layer.Layer

Non-linearity layer applying a non-linear transformation.

tf_apply(x, update)
class tensorforce.core.networks.layer.Pool2d(pooling_type='max', window=2, stride=2, padding='SAME', scope='pool2d', summary_labels=())

Bases: tensorforce.core.networks.layer.Layer

2-dimensional pooling layer.

tf_apply(x, update)

tensorforce.core.networks.network module

class tensorforce.core.networks.network.LayerBasedNetwork(scope='layerbased-network', summary_labels=())

Bases: tensorforce.core.networks.network.Network

Base class for networks using TensorForce layers.

add_layer(layer)
get_summaries()
get_variables(include_non_trainable=False)
internals_init()
internals_input()
tf_regularization_loss()
class tensorforce.core.networks.network.LayeredNetwork(layers_spec, scope='layered-network', summary_labels=())

Bases: tensorforce.core.networks.network.LayerBasedNetwork

Network consisting of a sequence of layers, which can be created from a specification dict.

static from_json(filename)

Creates a layer_networkd_builder from a JSON.

Parameters:filename – Path to configuration

Returns: A layered_network_builder function with layers generated from the JSON

tf_apply(x, internals, update, return_internals=False)
class tensorforce.core.networks.network.Network(scope='network', summary_labels=None)

Bases: object

Base class for neural networks.

static from_spec(spec, kwargs=None)

Creates a network from a specification dict.

get_summaries()

Returns the TensorFlow summaries reported by the network.

Returns:List of summaries
get_variables(include_non_trainable=False)

Returns the TensorFlow variables used by the network.

Returns:List of variables
internals_init()

Returns the TensorFlow tensors for internal state initializations.

Returns:List of internal state initialization tensors
internals_input()

Returns the TensorFlow placeholders for internal state inputs.

Returns:List of internal state input placeholders
tf_apply(x, internals, update, return_internals=False)

Creates the TensorFlow operations for applying the network to the given input.

Parameters:
  • x – Network input tensor or dict of input tensors.
  • internals – List of prior internal state tensors
  • update – Boolean tensor indicating whether this call happens during an update.
  • return_internals – If true, also returns posterior internal state tensors
Returns:

Network output tensor, plus optionally list of posterior internal state tensors
tf_regularization_loss()

Creates the TensorFlow operations for the network regularization loss.

Returns:Regularization loss tensor

Module contents

class tensorforce.core.networks.Layer(num_internals=0, scope='layer', summary_labels=None)

Bases: object

Base class for network layers.

static from_spec(spec, kwargs=None)

Creates a layer from a specification dict.

get_summaries()

Returns the TensorFlow summaries reported by the layer.

Returns:List of summaries.
get_variables(include_non_trainable=False)

Returns the TensorFlow variables used by the layer.

Returns:List of variables.
internals_init()

Returns the TensorFlow tensors for internal state initializations.

Returns:List of internal state initialization tensors.
internals_input()

Returns the TensorFlow placeholders for internal state inputs.

Returns:List of internal state input placeholders.
tf_apply(x, update)

Creates the TensorFlow operations for applying the layer to the given input.

Parameters:
  • x – Layer input tensor.
  • update – Boolean tensor indicating whether this call happens during an update.
Returns:

Layer output tensor.
tf_regularization_loss()

Creates the TensorFlow operations for the layer regularization loss.

Returns:Regularization loss tensor.
tf_tensors(named_tensors)

Attaches the named_tensors dictionary to the layer for examination and update.

Parameters:named_tensors – Dictionary of named tensors to be used as Input’s or recorded from outputs
Returns:NA
class tensorforce.core.networks.Nonlinearity(name='relu', scope='nonlinearity', summary_labels=())

Bases: tensorforce.core.networks.layer.Layer

Non-linearity layer applying a non-linear transformation.

tf_apply(x, update)
class tensorforce.core.networks.Dropout(rate=0.0, scope='dropout', summary_labels=())

Bases: tensorforce.core.networks.layer.Layer

Dropout layer. If using dropout, add this layer after inputs and after dense layers. For LSTM, dropout is handled independently as an argument. Not available for Conv2d yet.

tf_apply(x, update)
class tensorforce.core.networks.Flatten(scope='flatten', summary_labels=())

Bases: tensorforce.core.networks.layer.Layer

Flatten layer reshaping the input.

tf_apply(x, update)
class tensorforce.core.networks.Pool2d(pooling_type='max', window=2, stride=2, padding='SAME', scope='pool2d', summary_labels=())

Bases: tensorforce.core.networks.layer.Layer

2-dimensional pooling layer.

tf_apply(x, update)
class tensorforce.core.networks.Embedding(indices, size, l2_regularization=0.0, l1_regularization=0.0, scope='embedding', summary_labels=())

Bases: tensorforce.core.networks.layer.Layer

Embedding layer.

tf_apply(x, update)
tf_regularization_loss()
class tensorforce.core.networks.Linear(size, weights=None, bias=True, l2_regularization=0.0, l1_regularization=0.0, scope='linear', summary_labels=())

Bases: tensorforce.core.networks.layer.Layer

Linear fully-connected layer.

tf_apply(x, update=False)
tf_regularization_loss()
class tensorforce.core.networks.Dense(size=None, bias=True, activation='tanh', l2_regularization=0.0, l1_regularization=0.0, skip=False, scope='dense', summary_labels=())

Bases: tensorforce.core.networks.layer.Layer

Dense layer, i.e. linear fully connected layer with subsequent non-linearity.

get_summaries()
get_variables(include_non_trainable=False)
tf_apply(x, update)
tf_regularization_loss()
class tensorforce.core.networks.Dueling(size, bias=False, activation='none', l2_regularization=0.0, l1_regularization=0.0, output=None, scope='dueling', summary_labels=())

Bases: tensorforce.core.networks.layer.Layer

Dueling layer, i.e. Duel pipelines for Exp & Adv to help with stability

get_summaries()
get_variables(include_non_trainable=False)
tf_apply(x, update)
tf_regularization_loss()
class tensorforce.core.networks.Conv1d(size, window=3, stride=1, padding='SAME', bias=True, activation='relu', l2_regularization=0.0, l1_regularization=0.0, scope='conv1d', summary_labels=())

Bases: tensorforce.core.networks.layer.Layer

1-dimensional convolutional layer.

get_summaries()
get_variables(include_non_trainable=False)
tf_apply(x, update)
tf_regularization_loss()
class tensorforce.core.networks.Conv2d(size, window=3, stride=1, padding='SAME', bias=True, activation='relu', l2_regularization=0.0, l1_regularization=0.0, scope='conv2d', summary_labels=())

Bases: tensorforce.core.networks.layer.Layer

2-dimensional convolutional layer.

get_summaries()
get_variables(include_non_trainable=False)
tf_apply(x, update)
tf_regularization_loss()
class tensorforce.core.networks.InternalLstm(size, dropout=None, scope='internal_lstm', summary_labels=())

Bases: tensorforce.core.networks.layer.Layer

Long short-term memory layer for internal state management.

internals_init()
internals_input()
tf_apply(x, update, state)
class tensorforce.core.networks.Lstm(size, dropout=None, scope='lstm', summary_labels=(), return_final_state=True)

Bases: tensorforce.core.networks.layer.Layer

tf_apply(x, update, sequence_length=None)
class tensorforce.core.networks.Network(scope='network', summary_labels=None)

Bases: object

Base class for neural networks.

static from_spec(spec, kwargs=None)

Creates a network from a specification dict.

get_summaries()

Returns the TensorFlow summaries reported by the network.

Returns:List of summaries
get_variables(include_non_trainable=False)

Returns the TensorFlow variables used by the network.

Returns:List of variables
internals_init()

Returns the TensorFlow tensors for internal state initializations.

Returns:List of internal state initialization tensors
internals_input()

Returns the TensorFlow placeholders for internal state inputs.

Returns:List of internal state input placeholders
tf_apply(x, internals, update, return_internals=False)

Creates the TensorFlow operations for applying the network to the given input.

Parameters:
  • x – Network input tensor or dict of input tensors.
  • internals – List of prior internal state tensors
  • update – Boolean tensor indicating whether this call happens during an update.
  • return_internals – If true, also returns posterior internal state tensors
Returns:

Network output tensor, plus optionally list of posterior internal state tensors
tf_regularization_loss()

Creates the TensorFlow operations for the network regularization loss.

Returns:Regularization loss tensor
class tensorforce.core.networks.LayerBasedNetwork(scope='layerbased-network', summary_labels=())

Bases: tensorforce.core.networks.network.Network

Base class for networks using TensorForce layers.

add_layer(layer)
get_summaries()
get_variables(include_non_trainable=False)
internals_init()
internals_input()
tf_regularization_loss()
class tensorforce.core.networks.LayeredNetwork(layers_spec, scope='layered-network', summary_labels=())

Bases: tensorforce.core.networks.network.LayerBasedNetwork

Network consisting of a sequence of layers, which can be created from a specification dict.

static from_json(filename)

Creates a layer_networkd_builder from a JSON.

Parameters:filename – Path to configuration

Returns: A layered_network_builder function with layers generated from the JSON

tf_apply(x, internals, update, return_internals=False)