tensorforce.core.networks package¶
Submodules¶
tensorforce.core.networks.layer module¶
Collection of custom layer implementations. We prefer not to use contrib-layers to retain full control over shapes and internal states.
-
class
tensorforce.core.networks.layer.
Conv1d
(size, window=3, stride=1, padding='SAME', bias=True, activation='relu', l2_regularization=0.0, l1_regularization=0.0, scope='conv1d', summary_labels=())¶ Bases:
tensorforce.core.networks.layer.Layer
1-dimensional convolutional layer.
-
get_summaries
()¶
-
get_variables
(include_non_trainable=False)¶
-
tf_apply
(x, update)¶
-
tf_regularization_loss
()¶
-
-
class
tensorforce.core.networks.layer.
Conv2d
(size, window=3, stride=1, padding='SAME', bias=True, activation='relu', l2_regularization=0.0, l1_regularization=0.0, scope='conv2d', summary_labels=())¶ Bases:
tensorforce.core.networks.layer.Layer
2-dimensional convolutional layer.
-
get_summaries
()¶
-
get_variables
(include_non_trainable=False)¶
-
tf_apply
(x, update)¶
-
tf_regularization_loss
()¶
-
-
class
tensorforce.core.networks.layer.
Dense
(size=None, bias=True, activation='tanh', l2_regularization=0.0, l1_regularization=0.0, skip=False, scope='dense', summary_labels=())¶ Bases:
tensorforce.core.networks.layer.Layer
Dense layer, i.e. linear fully connected layer with subsequent non-linearity.
-
get_summaries
()¶
-
get_variables
(include_non_trainable=False)¶
-
tf_apply
(x, update)¶
-
tf_regularization_loss
()¶
-
-
class
tensorforce.core.networks.layer.
Dropout
(rate=0.0, scope='dropout', summary_labels=())¶ Bases:
tensorforce.core.networks.layer.Layer
Dropout layer. If using dropout, add this layer after inputs and after dense layers. For LSTM, dropout is handled independently as an argument. Not available for Conv2d yet.
-
tf_apply
(x, update)¶
-
-
class
tensorforce.core.networks.layer.
Dueling
(size, bias=False, activation='none', l2_regularization=0.0, l1_regularization=0.0, output=None, scope='dueling', summary_labels=())¶ Bases:
tensorforce.core.networks.layer.Layer
Dueling layer, i.e. Duel pipelines for Exp & Adv to help with stability
-
get_summaries
()¶
-
get_variables
(include_non_trainable=False)¶
-
tf_apply
(x, update)¶
-
tf_regularization_loss
()¶
-
-
class
tensorforce.core.networks.layer.
Embedding
(indices, size, l2_regularization=0.0, l1_regularization=0.0, scope='embedding', summary_labels=())¶ Bases:
tensorforce.core.networks.layer.Layer
Embedding layer.
-
tf_apply
(x, update)¶
-
tf_regularization_loss
()¶
-
-
class
tensorforce.core.networks.layer.
Flatten
(scope='flatten', summary_labels=())¶ Bases:
tensorforce.core.networks.layer.Layer
Flatten layer reshaping the input.
-
tf_apply
(x, update)¶
-
-
class
tensorforce.core.networks.layer.
InternalLstm
(size, dropout=None, scope='internal_lstm', summary_labels=())¶ Bases:
tensorforce.core.networks.layer.Layer
Long short-term memory layer for internal state management.
-
internals_init
()¶
-
internals_input
()¶
-
tf_apply
(x, update, state)¶
-
-
class
tensorforce.core.networks.layer.
Layer
(num_internals=0, scope='layer', summary_labels=None)¶ Bases:
object
Base class for network layers.
-
static
from_spec
(spec, kwargs=None)¶ Creates a layer from a specification dict.
-
get_summaries
()¶ Returns the TensorFlow summaries reported by the layer.
Returns: List of summaries.
-
get_variables
(include_non_trainable=False)¶ Returns the TensorFlow variables used by the layer.
Returns: List of variables.
-
internals_init
()¶ Returns the TensorFlow tensors for internal state initializations.
Returns: List of internal state initialization tensors.
-
internals_input
()¶ Returns the TensorFlow placeholders for internal state inputs.
Returns: List of internal state input placeholders.
-
tf_apply
(x, update)¶ Creates the TensorFlow operations for applying the layer to the given input.
Parameters: - x – Layer input tensor.
- update – Boolean tensor indicating whether this call happens during an update.
Returns: Layer output tensor.
-
tf_regularization_loss
()¶ Creates the TensorFlow operations for the layer regularization loss.
Returns: Regularization loss tensor.
-
tf_tensors
(named_tensors)¶ Attaches the named_tensors dictionary to the layer for examination and update.
Parameters: named_tensors – Dictionary of named tensors to be used as Input’s or recorded from outputs Returns: NA
-
static
-
class
tensorforce.core.networks.layer.
Linear
(size, weights=None, bias=True, l2_regularization=0.0, l1_regularization=0.0, scope='linear', summary_labels=())¶ Bases:
tensorforce.core.networks.layer.Layer
Linear fully-connected layer.
-
tf_apply
(x, update=False)¶
-
tf_regularization_loss
()¶
-
-
class
tensorforce.core.networks.layer.
Lstm
(size, dropout=None, scope='lstm', summary_labels=(), return_final_state=True)¶ Bases:
tensorforce.core.networks.layer.Layer
-
tf_apply
(x, update, sequence_length=None)¶
-
-
class
tensorforce.core.networks.layer.
Nonlinearity
(name='relu', scope='nonlinearity', summary_labels=())¶ Bases:
tensorforce.core.networks.layer.Layer
Non-linearity layer applying a non-linear transformation.
-
tf_apply
(x, update)¶
-
-
class
tensorforce.core.networks.layer.
Pool2d
(pooling_type='max', window=2, stride=2, padding='SAME', scope='pool2d', summary_labels=())¶ Bases:
tensorforce.core.networks.layer.Layer
2-dimensional pooling layer.
-
tf_apply
(x, update)¶
-
tensorforce.core.networks.network module¶
-
class
tensorforce.core.networks.network.
LayerBasedNetwork
(scope='layerbased-network', summary_labels=())¶ Bases:
tensorforce.core.networks.network.Network
Base class for networks using TensorForce layers.
-
add_layer
(layer)¶
-
get_summaries
()¶
-
get_variables
(include_non_trainable=False)¶
-
internals_init
()¶
-
internals_input
()¶
-
tf_regularization_loss
()¶
-
-
class
tensorforce.core.networks.network.
LayeredNetwork
(layers_spec, scope='layered-network', summary_labels=())¶ Bases:
tensorforce.core.networks.network.LayerBasedNetwork
Network consisting of a sequence of layers, which can be created from a specification dict.
-
static
from_json
(filename)¶ Creates a layer_networkd_builder from a JSON.
Parameters: filename – Path to configuration Returns: A layered_network_builder function with layers generated from the JSON
-
tf_apply
(x, internals, update, return_internals=False)¶
-
static
-
class
tensorforce.core.networks.network.
Network
(scope='network', summary_labels=None)¶ Bases:
object
Base class for neural networks.
-
static
from_spec
(spec, kwargs=None)¶ Creates a network from a specification dict.
-
get_summaries
()¶ Returns the TensorFlow summaries reported by the network.
Returns: List of summaries
-
get_variables
(include_non_trainable=False)¶ Returns the TensorFlow variables used by the network.
Returns: List of variables
-
internals_init
()¶ Returns the TensorFlow tensors for internal state initializations.
Returns: List of internal state initialization tensors
-
internals_input
()¶ Returns the TensorFlow placeholders for internal state inputs.
Returns: List of internal state input placeholders
-
tf_apply
(x, internals, update, return_internals=False)¶ Creates the TensorFlow operations for applying the network to the given input.
Parameters: - x – Network input tensor or dict of input tensors.
- internals – List of prior internal state tensors
- update – Boolean tensor indicating whether this call happens during an update.
- return_internals – If true, also returns posterior internal state tensors
Returns: Network output tensor, plus optionally list of posterior internal state tensors
-
tf_regularization_loss
()¶ Creates the TensorFlow operations for the network regularization loss.
Returns: Regularization loss tensor
-
static
Module contents¶
-
class
tensorforce.core.networks.
Layer
(num_internals=0, scope='layer', summary_labels=None)¶ Bases:
object
Base class for network layers.
-
static
from_spec
(spec, kwargs=None)¶ Creates a layer from a specification dict.
-
get_summaries
()¶ Returns the TensorFlow summaries reported by the layer.
Returns: List of summaries.
-
get_variables
(include_non_trainable=False)¶ Returns the TensorFlow variables used by the layer.
Returns: List of variables.
-
internals_init
()¶ Returns the TensorFlow tensors for internal state initializations.
Returns: List of internal state initialization tensors.
-
internals_input
()¶ Returns the TensorFlow placeholders for internal state inputs.
Returns: List of internal state input placeholders.
-
tf_apply
(x, update)¶ Creates the TensorFlow operations for applying the layer to the given input.
Parameters: - x – Layer input tensor.
- update – Boolean tensor indicating whether this call happens during an update.
Returns: Layer output tensor.
-
tf_regularization_loss
()¶ Creates the TensorFlow operations for the layer regularization loss.
Returns: Regularization loss tensor.
-
tf_tensors
(named_tensors)¶ Attaches the named_tensors dictionary to the layer for examination and update.
Parameters: named_tensors – Dictionary of named tensors to be used as Input’s or recorded from outputs Returns: NA
-
static
-
class
tensorforce.core.networks.
Nonlinearity
(name='relu', scope='nonlinearity', summary_labels=())¶ Bases:
tensorforce.core.networks.layer.Layer
Non-linearity layer applying a non-linear transformation.
-
tf_apply
(x, update)¶
-
-
class
tensorforce.core.networks.
Dropout
(rate=0.0, scope='dropout', summary_labels=())¶ Bases:
tensorforce.core.networks.layer.Layer
Dropout layer. If using dropout, add this layer after inputs and after dense layers. For LSTM, dropout is handled independently as an argument. Not available for Conv2d yet.
-
tf_apply
(x, update)¶
-
-
class
tensorforce.core.networks.
Flatten
(scope='flatten', summary_labels=())¶ Bases:
tensorforce.core.networks.layer.Layer
Flatten layer reshaping the input.
-
tf_apply
(x, update)¶
-
-
class
tensorforce.core.networks.
Pool2d
(pooling_type='max', window=2, stride=2, padding='SAME', scope='pool2d', summary_labels=())¶ Bases:
tensorforce.core.networks.layer.Layer
2-dimensional pooling layer.
-
tf_apply
(x, update)¶
-
-
class
tensorforce.core.networks.
Embedding
(indices, size, l2_regularization=0.0, l1_regularization=0.0, scope='embedding', summary_labels=())¶ Bases:
tensorforce.core.networks.layer.Layer
Embedding layer.
-
tf_apply
(x, update)¶
-
tf_regularization_loss
()¶
-
-
class
tensorforce.core.networks.
Linear
(size, weights=None, bias=True, l2_regularization=0.0, l1_regularization=0.0, scope='linear', summary_labels=())¶ Bases:
tensorforce.core.networks.layer.Layer
Linear fully-connected layer.
-
tf_apply
(x, update=False)¶
-
tf_regularization_loss
()¶
-
-
class
tensorforce.core.networks.
Dense
(size=None, bias=True, activation='tanh', l2_regularization=0.0, l1_regularization=0.0, skip=False, scope='dense', summary_labels=())¶ Bases:
tensorforce.core.networks.layer.Layer
Dense layer, i.e. linear fully connected layer with subsequent non-linearity.
-
get_summaries
()¶
-
get_variables
(include_non_trainable=False)¶
-
tf_apply
(x, update)¶
-
tf_regularization_loss
()¶
-
-
class
tensorforce.core.networks.
Dueling
(size, bias=False, activation='none', l2_regularization=0.0, l1_regularization=0.0, output=None, scope='dueling', summary_labels=())¶ Bases:
tensorforce.core.networks.layer.Layer
Dueling layer, i.e. Duel pipelines for Exp & Adv to help with stability
-
get_summaries
()¶
-
get_variables
(include_non_trainable=False)¶
-
tf_apply
(x, update)¶
-
tf_regularization_loss
()¶
-
-
class
tensorforce.core.networks.
Conv1d
(size, window=3, stride=1, padding='SAME', bias=True, activation='relu', l2_regularization=0.0, l1_regularization=0.0, scope='conv1d', summary_labels=())¶ Bases:
tensorforce.core.networks.layer.Layer
1-dimensional convolutional layer.
-
get_summaries
()¶
-
get_variables
(include_non_trainable=False)¶
-
tf_apply
(x, update)¶
-
tf_regularization_loss
()¶
-
-
class
tensorforce.core.networks.
Conv2d
(size, window=3, stride=1, padding='SAME', bias=True, activation='relu', l2_regularization=0.0, l1_regularization=0.0, scope='conv2d', summary_labels=())¶ Bases:
tensorforce.core.networks.layer.Layer
2-dimensional convolutional layer.
-
get_summaries
()¶
-
get_variables
(include_non_trainable=False)¶
-
tf_apply
(x, update)¶
-
tf_regularization_loss
()¶
-
-
class
tensorforce.core.networks.
InternalLstm
(size, dropout=None, scope='internal_lstm', summary_labels=())¶ Bases:
tensorforce.core.networks.layer.Layer
Long short-term memory layer for internal state management.
-
internals_init
()¶
-
internals_input
()¶
-
tf_apply
(x, update, state)¶
-
-
class
tensorforce.core.networks.
Lstm
(size, dropout=None, scope='lstm', summary_labels=(), return_final_state=True)¶ Bases:
tensorforce.core.networks.layer.Layer
-
tf_apply
(x, update, sequence_length=None)¶
-
-
class
tensorforce.core.networks.
Network
(scope='network', summary_labels=None)¶ Bases:
object
Base class for neural networks.
-
static
from_spec
(spec, kwargs=None)¶ Creates a network from a specification dict.
-
get_summaries
()¶ Returns the TensorFlow summaries reported by the network.
Returns: List of summaries
-
get_variables
(include_non_trainable=False)¶ Returns the TensorFlow variables used by the network.
Returns: List of variables
-
internals_init
()¶ Returns the TensorFlow tensors for internal state initializations.
Returns: List of internal state initialization tensors
-
internals_input
()¶ Returns the TensorFlow placeholders for internal state inputs.
Returns: List of internal state input placeholders
-
tf_apply
(x, internals, update, return_internals=False)¶ Creates the TensorFlow operations for applying the network to the given input.
Parameters: - x – Network input tensor or dict of input tensors.
- internals – List of prior internal state tensors
- update – Boolean tensor indicating whether this call happens during an update.
- return_internals – If true, also returns posterior internal state tensors
Returns: Network output tensor, plus optionally list of posterior internal state tensors
-
tf_regularization_loss
()¶ Creates the TensorFlow operations for the network regularization loss.
Returns: Regularization loss tensor
-
static
-
class
tensorforce.core.networks.
LayerBasedNetwork
(scope='layerbased-network', summary_labels=())¶ Bases:
tensorforce.core.networks.network.Network
Base class for networks using TensorForce layers.
-
add_layer
(layer)¶
-
get_summaries
()¶
-
get_variables
(include_non_trainable=False)¶
-
internals_init
()¶
-
internals_input
()¶
-
tf_regularization_loss
()¶
-
-
class
tensorforce.core.networks.
LayeredNetwork
(layers_spec, scope='layered-network', summary_labels=())¶ Bases:
tensorforce.core.networks.network.LayerBasedNetwork
Network consisting of a sequence of layers, which can be created from a specification dict.
-
static
from_json
(filename)¶ Creates a layer_networkd_builder from a JSON.
Parameters: filename – Path to configuration Returns: A layered_network_builder function with layers generated from the JSON
-
tf_apply
(x, internals, update, return_internals=False)¶
-
static