tensorforce.core.optimizers.solvers package¶
Submodules¶
tensorforce.core.optimizers.solvers.conjugate_gradient module¶
-
class
tensorforce.core.optimizers.solvers.conjugate_gradient.
ConjugateGradient
(max_iterations, damping, unroll_loop=False)¶ Bases:
tensorforce.core.optimizers.solvers.iterative.Iterative
Conjugate gradient algorithm which iteratively finds a solution $x$ for a system of linear equations of the form $A x = b$, where $A x$ could be, for instance, a locally linear approximation of a high-dimensional function.
See below pseudo-code taken from Wikipedia:
def conjgrad(A, b, x_0): r_0 := b - A * x_0 c_0 := r_0 r_0^2 := r^T * r for t in 0, ..., max_iterations - 1: Ac := A * c_t cAc := c_t^T * Ac lpha := r_t^2 / cAc x_{t+1} := x_t + lpha * c_t r_{t+1} := r_t - lpha * Ac r_{t+1}^2 := r_{t+1}^T * r_{t+1} if r_{t+1} < \epsilon: break eta = r_{t+1}^2 / r_t^2 c_{t+1} := r_{t+1} + eta * c_t return x_{t+1}
-
tf_initialize
(x_init, b)¶ Initialization step preparing the arguments for the first iteration of the loop body: $x_0, 0, p_0, r_0, r_0^2$.
Parameters: - x_init – Initial solution guess $x_0$, zero vector if None.
- b – The right-hand side $b$ of the system of linear equations.
Returns: Initial arguments for tf_step.
-
tf_next_step
(x, iteration, conjugate, residual, squared_residual)¶ Termination condition: max number of iterations, or residual sufficiently small.
Parameters: - x – Current solution estimate $x_t$.
- iteration – Current iteration counter $t$.
- conjugate – Current conjugate $c_t$.
- residual – Current residual $r_t$.
- squared_residual – Current squared residual $r_t^2$.
Returns: True if another iteration should be performed.
-
tf_solve
(fn_x, x_init, b)¶ Iteratively solves the system of linear equations $A x = b$.
Parameters: - fn_x – A callable returning the left-hand side $A x$ of the system of linear equations.
- x_init – Initial solution guess $x_0$, zero vector if None.
- b – The right-hand side $b$ of the system of linear equations.
Returns: A solution $x$ to the problem as given by the solver.
-
tf_step
(x, iteration, conjugate, residual, squared_residual)¶ Iteration loop body of the conjugate gradient algorithm.
Parameters: - x – Current solution estimate $x_t$.
- iteration – Current iteration counter $t$.
- conjugate – Current conjugate $c_t$.
- residual – Current residual $r_t$.
- squared_residual – Current squared residual $r_t^2$.
Returns: Updated arguments for next iteration.
-
tensorforce.core.optimizers.solvers.iterative module¶
-
class
tensorforce.core.optimizers.solvers.iterative.
Iterative
(max_iterations, unroll_loop=False)¶ Bases:
tensorforce.core.optimizers.solvers.solver.Solver
Generic solver which iteratively solves an equation/optimization problem. Involves an initialization step, the iteration loop body and the termination condition.
-
tf_initialize
(x_init, *args)¶ Initialization step preparing the arguments for the first iteration of the loop body (default: initial solution guess and iteration counter).
Parameters: - x_init – Initial solution guess $x_0$.
- *args –
Additional solver-specific arguments.
Returns: Initial arguments for tf_step.
-
tf_next_step
(x, iteration, *args)¶ Termination condition (default: max number of iterations).
Parameters: - x – Current solution estimate.
- iteration – Current iteration counter.
- *args –
Additional solver-specific arguments.
Returns: True if another iteration should be performed.
-
tf_solve
(fn_x, x_init, *args)¶ Iteratively solves an equation/optimization for $x$ involving an expression $f(x)$.
Parameters: - fn_x – A callable returning an expression $f(x)$ given $x$.
- x_init – Initial solution guess $x_0$.
- *args –
Additional solver-specific arguments.
Returns: A solution $x$ to the problem as given by the solver.
-
tf_step
(x, iteration, *args)¶ Iteration loop body of the iterative solver (default: increment iteration step). The first two loop arguments have to be the current solution estimate and the iteration step.
Parameters: - x – Current solution estimate.
- iteration – Current iteration counter.
- *args –
Additional solver-specific arguments.
Returns: Updated arguments for next iteration.
-
tensorforce.core.optimizers.solvers.line_search module¶
-
class
tensorforce.core.optimizers.solvers.line_search.
LineSearch
(max_iterations, accept_ratio, mode, parameter, unroll_loop=False)¶ Bases:
tensorforce.core.optimizers.solvers.iterative.Iterative
Line search algorithm which iteratively optimizes the value $f(x)$ for $x$ on the line between $x’$ and $x_0$ by optimistically taking the first acceptable $x$ starting from $x_0$ and moving towards $x’$.
-
tf_initialize
(x_init, base_value, target_value, estimated_improvement)¶ Initialization step preparing the arguments for the first iteration of the loop body.
Parameters: - x_init – Initial solution guess $x_0$.
- base_value – Value $f(x’)$ at $x = x’$.
- target_value – Value $f(x_0)$ at $x = x_0$.
- estimated_improvement – Estimated value at $x = x_0$, $f(x’)$ if None.
Returns: Initial arguments for tf_step.
-
tf_next_step
(x, iteration, deltas, improvement, last_improvement, estimated_improvement)¶ Termination condition: max number of iterations, or no improvement for last step, or improvement less than acceptable ratio, or estimated value not positive.
Parameters: - x – Current solution estimate $x_t$.
- iteration – Current iteration counter $t$.
- deltas – Current difference $x_t - x’$.
- improvement – Current improvement $(f(x_t) - f(x’)) / v’$.
- last*improvement –
Last improvement $(f(x*{t-1}) - f(x’)) / v’$.
- estimated_improvement – Current estimated value $v’$.
Returns: True if another iteration should be performed.
-
tf_solve
(fn_x, x_init, base_value, target_value, estimated_improvement=None)¶ Iteratively optimizes $f(x)$ for $x$ on the line between $x’$ and $x_0$.
Parameters: - fn_x – A callable returning the value $f(x)$ at $x$.
- x_init – Initial solution guess $x_0$.
- base_value – Value $f(x’)$ at $x = x’$.
- target_value – Value $f(x_0)$ at $x = x_0$.
- estimated_improvement – Estimated improvement for $x = x_0$, $f(x’)$ if None.
Returns: A solution $x$ to the problem as given by the solver.
-
tf_step
(x, iteration, deltas, improvement, last_improvement, estimated_improvement)¶ Iteration loop body of the line search algorithm.
Parameters: - x – Current solution estimate $x_t$.
- iteration – Current iteration counter $t$.
- deltas – Current difference $x_t - x’$.
- improvement – Current improvement $(f(x_t) - f(x’)) / v’$.
- last*improvement –
Last improvement $(f(x*{t-1}) - f(x’)) / v’$.
- estimated_improvement – Current estimated value $v’$.
Returns: Updated arguments for next iteration.
-
tensorforce.core.optimizers.solvers.solver module¶
-
class
tensorforce.core.optimizers.solvers.solver.
Solver
¶ Bases:
object
Generic TensorFlow-based solver which solves a not yet further specified equation/optimization problem.
-
static
from_config
(config, kwargs=None)¶ Creates a solver from a specification dict.
-
tf_solve
(fn_x, *args)¶ Solves an equation/optimization for $x$ involving an expression $f(x)$.
Parameters: - fn_x – A callable returning an expression $f(x)$ given $x$.
- *args –
Additional solver-specific arguments.
Returns: A solution $x$ to the problem as given by the solver.
-
static
Module contents¶
-
class
tensorforce.core.optimizers.solvers.
Solver
¶ Bases:
object
Generic TensorFlow-based solver which solves a not yet further specified equation/optimization problem.
-
static
from_config
(config, kwargs=None)¶ Creates a solver from a specification dict.
-
tf_solve
(fn_x, *args)¶ Solves an equation/optimization for $x$ involving an expression $f(x)$.
Parameters: - fn_x – A callable returning an expression $f(x)$ given $x$.
- *args –
Additional solver-specific arguments.
Returns: A solution $x$ to the problem as given by the solver.
-
static
-
class
tensorforce.core.optimizers.solvers.
Iterative
(max_iterations, unroll_loop=False)¶ Bases:
tensorforce.core.optimizers.solvers.solver.Solver
Generic solver which iteratively solves an equation/optimization problem. Involves an initialization step, the iteration loop body and the termination condition.
-
tf_initialize
(x_init, *args)¶ Initialization step preparing the arguments for the first iteration of the loop body (default: initial solution guess and iteration counter).
Parameters: - x_init – Initial solution guess $x_0$.
- *args –
Additional solver-specific arguments.
Returns: Initial arguments for tf_step.
-
tf_next_step
(x, iteration, *args)¶ Termination condition (default: max number of iterations).
Parameters: - x – Current solution estimate.
- iteration – Current iteration counter.
- *args –
Additional solver-specific arguments.
Returns: True if another iteration should be performed.
-
tf_solve
(fn_x, x_init, *args)¶ Iteratively solves an equation/optimization for $x$ involving an expression $f(x)$.
Parameters: - fn_x – A callable returning an expression $f(x)$ given $x$.
- x_init – Initial solution guess $x_0$.
- *args –
Additional solver-specific arguments.
Returns: A solution $x$ to the problem as given by the solver.
-
tf_step
(x, iteration, *args)¶ Iteration loop body of the iterative solver (default: increment iteration step). The first two loop arguments have to be the current solution estimate and the iteration step.
Parameters: - x – Current solution estimate.
- iteration – Current iteration counter.
- *args –
Additional solver-specific arguments.
Returns: Updated arguments for next iteration.
-
-
class
tensorforce.core.optimizers.solvers.
ConjugateGradient
(max_iterations, damping, unroll_loop=False)¶ Bases:
tensorforce.core.optimizers.solvers.iterative.Iterative
Conjugate gradient algorithm which iteratively finds a solution $x$ for a system of linear equations of the form $A x = b$, where $A x$ could be, for instance, a locally linear approximation of a high-dimensional function.
See below pseudo-code taken from Wikipedia:
def conjgrad(A, b, x_0): r_0 := b - A * x_0 c_0 := r_0 r_0^2 := r^T * r for t in 0, ..., max_iterations - 1: Ac := A * c_t cAc := c_t^T * Ac lpha := r_t^2 / cAc x_{t+1} := x_t + lpha * c_t r_{t+1} := r_t - lpha * Ac r_{t+1}^2 := r_{t+1}^T * r_{t+1} if r_{t+1} < \epsilon: break eta = r_{t+1}^2 / r_t^2 c_{t+1} := r_{t+1} + eta * c_t return x_{t+1}
-
tf_initialize
(x_init, b)¶ Initialization step preparing the arguments for the first iteration of the loop body: $x_0, 0, p_0, r_0, r_0^2$.
Parameters: - x_init – Initial solution guess $x_0$, zero vector if None.
- b – The right-hand side $b$ of the system of linear equations.
Returns: Initial arguments for tf_step.
-
tf_next_step
(x, iteration, conjugate, residual, squared_residual)¶ Termination condition: max number of iterations, or residual sufficiently small.
Parameters: - x – Current solution estimate $x_t$.
- iteration – Current iteration counter $t$.
- conjugate – Current conjugate $c_t$.
- residual – Current residual $r_t$.
- squared_residual – Current squared residual $r_t^2$.
Returns: True if another iteration should be performed.
-
tf_solve
(fn_x, x_init, b)¶ Iteratively solves the system of linear equations $A x = b$.
Parameters: - fn_x – A callable returning the left-hand side $A x$ of the system of linear equations.
- x_init – Initial solution guess $x_0$, zero vector if None.
- b – The right-hand side $b$ of the system of linear equations.
Returns: A solution $x$ to the problem as given by the solver.
-
tf_step
(x, iteration, conjugate, residual, squared_residual)¶ Iteration loop body of the conjugate gradient algorithm.
Parameters: - x – Current solution estimate $x_t$.
- iteration – Current iteration counter $t$.
- conjugate – Current conjugate $c_t$.
- residual – Current residual $r_t$.
- squared_residual – Current squared residual $r_t^2$.
Returns: Updated arguments for next iteration.
-
-
class
tensorforce.core.optimizers.solvers.
LineSearch
(max_iterations, accept_ratio, mode, parameter, unroll_loop=False)¶ Bases:
tensorforce.core.optimizers.solvers.iterative.Iterative
Line search algorithm which iteratively optimizes the value $f(x)$ for $x$ on the line between $x’$ and $x_0$ by optimistically taking the first acceptable $x$ starting from $x_0$ and moving towards $x’$.
-
tf_initialize
(x_init, base_value, target_value, estimated_improvement)¶ Initialization step preparing the arguments for the first iteration of the loop body.
Parameters: - x_init – Initial solution guess $x_0$.
- base_value – Value $f(x’)$ at $x = x’$.
- target_value – Value $f(x_0)$ at $x = x_0$.
- estimated_improvement – Estimated value at $x = x_0$, $f(x’)$ if None.
Returns: Initial arguments for tf_step.
-
tf_next_step
(x, iteration, deltas, improvement, last_improvement, estimated_improvement)¶ Termination condition: max number of iterations, or no improvement for last step, or improvement less than acceptable ratio, or estimated value not positive.
Parameters: - x – Current solution estimate $x_t$.
- iteration – Current iteration counter $t$.
- deltas – Current difference $x_t - x’$.
- improvement – Current improvement $(f(x_t) - f(x’)) / v’$.
- last*improvement –
Last improvement $(f(x*{t-1}) - f(x’)) / v’$.
- estimated_improvement – Current estimated value $v’$.
Returns: True if another iteration should be performed.
-
tf_solve
(fn_x, x_init, base_value, target_value, estimated_improvement=None)¶ Iteratively optimizes $f(x)$ for $x$ on the line between $x’$ and $x_0$.
Parameters: - fn_x – A callable returning the value $f(x)$ at $x$.
- x_init – Initial solution guess $x_0$.
- base_value – Value $f(x’)$ at $x = x’$.
- target_value – Value $f(x_0)$ at $x = x_0$.
- estimated_improvement – Estimated improvement for $x = x_0$, $f(x’)$ if None.
Returns: A solution $x$ to the problem as given by the solver.
-
tf_step
(x, iteration, deltas, improvement, last_improvement, estimated_improvement)¶ Iteration loop body of the line search algorithm.
Parameters: - x – Current solution estimate $x_t$.
- iteration – Current iteration counter $t$.
- deltas – Current difference $x_t - x’$.
- improvement – Current improvement $(f(x_t) - f(x’)) / v’$.
- last*improvement –
Last improvement $(f(x*{t-1}) - f(x’)) / v’$.
- estimated_improvement – Current estimated value $v’$.
Returns: Updated arguments for next iteration.
-