ribs.emitters.GradientArborescenceEmitter

class ribs.emitters.GradientArborescenceEmitter(archive, *, x0, sigma0, lr, ranker='2imp', selection_rule='filter', restart_rule='no_improvement', grad_opt='adam', grad_opt_kwargs=None, es='cma_es', es_kwargs=None, normalize_grad=True, bounds=None, batch_size=None, epsilon=1e-08, seed=None)

Generates solutions with a gradient arborescence, with coefficients parameterized by an evolution strategy.

This emitter originates in Fontaine 2021. It leverages the gradient information of the objective and measure functions, generating new solutions around a solution point \(\boldsymbol{\theta}\) using gradient arborescence, with coefficients drawn from a Gaussian distribution. Essentially, this means that the emitter samples coefficients \(\boldsymbol{c_i} \sim \mathcal{N}(\boldsymbol{\mu}, \boldsymbol{\Sigma})\) and creates new solutions \(\boldsymbol{\theta'_i}\) according to

\[\boldsymbol{\theta'_i} \gets \boldsymbol{\theta} + c_{i,0} \boldsymbol{\nabla} f(\boldsymbol{\theta}) + \sum_{j=1}^k c_{i,j}\boldsymbol{\nabla}m_j(\boldsymbol{\theta})\]

Where \(k\) is the number of measures, and \(\boldsymbol{\nabla} f(\boldsymbol{\theta})\) and \(\boldsymbol{\nabla} m_j(\boldsymbol{\theta})\) are the objective and measure gradients of the solution point \(\boldsymbol{\theta}\), respectively.

Based on how the solutions are ranked after being inserted into the archive (see ranker), the solution point \(\boldsymbol{\theta}\) is updated with gradient ascent, and the coefficient distribution parameters \(\boldsymbol{\mu}\) and \(\boldsymbol{\Sigma}\) are updated with an ES (the default ES is CMA-ES).

Note

Unlike non-gradient emitters, GradientArborescenceEmitter requires calling ask_dqd() and tell_dqd() (in this order) before calling ask() and tell() to communicate the gradient information to the emitter.

See also

Our DQD tutorial goes into detail on how to use this emitter: Generating Tom Cruise Images with DQD Algorithms

Parameters

• archive (ribs.archives.ArchiveBase) – An archive to use when creating and inserting solutions. For instance, this can be ribs.archives.GridArchive.

• x0 (np.ndarray) – Initial solution.

• sigma0 (float) – Initial step size / standard deviation of the distribution of gradient coefficients.

• lr (float) – Learning rate for the gradient optimizer.

• ranker (Callable or str) – The ranker is a RankerBase object that orders the solutions after they have been evaluated in the environment. This parameter may be a callable (e.g. a class or a lambda function) that takes in no parameters and returns an instance of RankerBase, or it may be a full or abbreviated ranker name as described in ribs.emitters.rankers.

• selection_rule ("mu" or "filter") – Method for selecting parents in CMA-ES. With “mu” selection, the first half of the solutions will be selected as parents, while in “filter”, any solutions that were added to the archive will be selected.

• restart_rule (int, "no_improvement", and "basic") – Method to use when checking for restarts. If given an integer, then the emitter will restart after this many iterations, where each iteration is a call to tell(). With “basic”, only the default CMA-ES convergence rules will be used, while with “no_improvement”, the emitter will restart when none of the proposed solutions were added to the archive.

• grad_opt (Callable or str) – Gradient optimizer to use for the gradient ascent step of the algorithm. The optimizer is a GradientOptBase object. This parameter may be a callable (e.g. a class or a lambda function) which takes in the theta0 and lr arguments, or it may be a full or abbreviated name as described in ribs.emitters.opt.

• grad_opt_kwargs (dict) – Additional arguments to pass to the gradient optimizer. See the gradient-based optimizers in ribs.emitters.opt for the arguments allowed by each optimizer. Note that we already pass in theta0 and lr.

• es (Callable or str) – The evolution strategy is an EvolutionStrategyBase object that is used to adapt the distribution from which new solutions are sampled. This parameter may be a callable (e.g. a class or a lambda function) that takes in the parameters of EvolutionStrategyBase along with kwargs provided by the es_kwargs argument, or it may be a full or abbreviated optimizer name as described in ribs.emitters.opt.

• es_kwargs (dict) – Additional arguments to pass to the evolution strategy optimizer. See the evolution-strategy-based optimizers in ribs.emitters.opt for the arguments allowed by each optimizer.

• normalize_grad (bool) – If true (default), then gradient infomation will be normalized. Otherwise, it will not be normalized.

• bounds – This argument may be used for providing solution space bounds in the future. This emitter does not currently support solution space bounds, as bounding solutions for DQD algorithms such as CMA-MEGA is an open problem. Hence, this argument must be set to None.

• batch_size (int) – Number of solutions to return in ask(). If not passed in, a batch size will be automatically calculated using the default CMA-ES rules. This does not account for the one solution returned by ask_dqd(), which is the solution point maintained by the gradient optimizer.

• epsilon (float) – For numerical stability, we add a small epsilon when normalizing gradients in tell_dqd() – refer to the implementation here. Pass this parameter to configure that epsilon.

• seed (int) – Value to seed the random number generator. Set to None to avoid a fixed seed.

Raises

• ValueError – There is an error in x0 or initial_solutions.

• ValueError – bounds is set even though it is not currently supported.

• ValueError – If restart_rule, selection_rule, or ranker is invalid.

Methods

ask()	Samples new solutions from a gradient arborescence parameterized by a multivariate Gaussian distribution.
ask_dqd()	Samples a new solution from the gradient optimizer.
tell(solution, objective, measures, ...)	Gives the emitter results from evaluating solutions.
tell_dqd(solution, objective, measures, ...)	Gives the emitter results from evaluating the gradient of the solutions.

Attributes

archive	The archive which stores solutions generated by this emitter.
batch_size	Number of solutions to return in ask().
batch_size_dqd	Number of solutions to return in ask_dqd().
epsilon	The epsilon added for numerical stability when normalizing gradients in tell_dqd().
itrs	The number of iterations for this emitter.
lower_bounds	(solution_dim,) array with lower bounds of solution space.
restarts	The number of restarts for this emitter.
solution_dim	The dimension of solutions produced by this emitter.
upper_bounds	(solution_dim,) array with upper bounds of solution space.
x0	Initial solution for the optimizer.

ask()

Samples new solutions from a gradient arborescence parameterized by a multivariate Gaussian distribution.

The multivariate Gaussian is parameterized by the evolution strategy optimizer self._opt.

This method returns batch_size solutions, even though one solution is returned via ask_dqd.

Returns

(batch_size, solution_dim) array – a batch of new solutions to evaluate.

Raises

RuntimeError – This method was called without first passing gradients with calls to ask_dqd() and tell_dqd().

ask_dqd()

Samples a new solution from the gradient optimizer.

Call :meth:`ask_dqd` and :meth:`tell_dqd` (in this order) before calling :meth:`ask` and :meth:`tell`.

Returns

a new solution to evaluate.

tell(solution, objective, measures, add_info, **fields)

Gives the emitter results from evaluating solutions.

The solutions are ranked based on the rank() function defined by self._ranker.

Parameters

• solution (array-like) – (batch_size, solution_dim) array of solutions generated by this emitter’s ask() method.

• objective (array-like) – 1D array containing the objective function value of each solution.

• measures (array-like) – (batch_size, measure space dimension) array with the measure space coordinates of each solution.

• add_info (dict) – Data returned from the archive add() method.

• fields (keyword arguments) – Additional data for each solution. Each argument should be an array with batch_size as the first dimension.

Raises

RuntimeError – This method was called without first passing gradients with calls to ask_dqd() and tell_dqd().

tell_dqd(solution, objective, measures, jacobian, add_info, **fields)

Gives the emitter results from evaluating the gradient of the solutions.

Parameters

• solution (array-like) – (batch_size, solution_dim) array of solutions generated by this emitter’s ask() method.

• objective (array-like) – 1D array containing the objective function value of each solution.

• measures (array-like) – (batch_size, measure space dimension) array with the measure space coordinates of each solution.

• jacobian (array-like) – (batch_size, 1 + measure_dim, solution_dim) array consisting of Jacobian matrices of the solutions obtained from ask_dqd(). Each matrix should consist of the objective gradient of the solution followed by the measure gradients.

• add_info (dict) – Data returned from the archive add() method.

• fields (keyword arguments) – Additional data for each solution. Each argument should be an array with batch_size as the first dimension.

property archive

The archive which stores solutions generated by this emitter.

Type

ribs.archives.ArchiveBase

property batch_size

Number of solutions to return in ask().

Type

int

property batch_size_dqd

Number of solutions to return in ask_dqd().

This is always 1, as we only return the solution point in ask_dqd().

Type

int

property epsilon

The epsilon added for numerical stability when normalizing gradients in tell_dqd().

Type

int

property itrs

The number of iterations for this emitter.

Type

int

property lower_bounds

(solution_dim,) array with lower bounds of solution space.

For instance, [-1, -1, -1] indicates that every dimension of the solution space has a lower bound of -1.

Type

numpy.ndarray

property restarts

The number of restarts for this emitter.

Type

int

property solution_dim

The dimension of solutions produced by this emitter.

Type

int

property upper_bounds

(solution_dim,) array with upper bounds of solution space.

For instance, [1, 1, 1] indicates that every dimension of the solution space has an upper bound of 1.

Type

numpy.ndarray

property x0

Initial solution for the optimizer.

Type

numpy.ndarray