ribs.emitters.GradientArborescenceEmitter¶

class ribs.emitters.GradientArborescenceEmitter(archive: ArchiveBase, *, x0: ArrayLike, sigma0: Float, lr: Float, ranker: collections.abc.Callable[[Int | None], RankerBase] | str = '2imp', selection_rule: 'mu' | 'filter' = 'filter', restart_rule: 'no_improvement' | 'basic' | int = 'no_improvement', grad_opt: collections.abc.Callable[..., GradientOptBase] | str = 'adam', grad_opt_kwargs: dict | None = None, es: collections.abc.Callable[..., EvolutionStrategyBase] | str = 'cma_es', es_kwargs: dict | None = None, normalize_grad: bool = True, bounds: Collection[tuple[None | Float, None | Float]] | None = None, lower_bounds: ArrayLike | None = None, upper_bounds: ArrayLike | None = None, batch_size: Int | None = None, epsilon: Float = 1e-08, seed: Int | None = None)[source]¶

Generates solutions with a gradient arborescence, with coefficients parameterized by an evolution strategy.

This emitter originates in Fontaine 2021. It leverages the gradient information of the objective and measure functions, generating new solutions around a solution point \(\boldsymbol{\theta}\) using gradient arborescence, with coefficients drawn from a Gaussian distribution. Essentially, this means that the emitter samples coefficients \(\boldsymbol{c_i} \sim \mathcal{N}(\boldsymbol{\mu}, \boldsymbol{\Sigma})\) and creates new solutions \(\boldsymbol{\theta'_i}\) according to

\[\boldsymbol{\theta'_i} \gets \boldsymbol{\theta} + c_{i,0} \boldsymbol{\nabla} f(\boldsymbol{\theta}) + \sum_{j=1}^k c_{i,j}\boldsymbol{\nabla}m_j(\boldsymbol{\theta})\]

Where \(k\) is the number of measures, and \(\boldsymbol{\nabla} f(\boldsymbol{\theta})\) and \(\boldsymbol{\nabla} m_j(\boldsymbol{\theta})\) are the objective and measure gradients of the solution point \(\boldsymbol{\theta}\), respectively.

Based on how the solutions are ranked after being inserted into the archive (see ranker), the solution point \(\boldsymbol{\theta}\) is updated with gradient ascent, and the coefficient distribution parameters \(\boldsymbol{\mu}\) and \(\boldsymbol{\Sigma}\) are updated with an ES (the default ES is CMA-ES).

Note

Unlike non-gradient emitters, GradientArborescenceEmitter requires calling ask_dqd() and tell_dqd() (in this order) before calling ask() and tell() to communicate the gradient information to the emitter.

See also

Our DQD tutorial goes into detail on how to use this emitter: Generating Tom Cruise Images with DQD Algorithms

Parameters:¶

archive: ArchiveBase¶: Archive of solutions, e.g., ribs.archives.GridArchive.
x0: ArrayLike¶: Initial solution.
sigma0: Float¶: Initial step size / standard deviation of the distribution of gradient coefficients.
lr: Float¶: Learning rate for the gradient optimizer.
ranker: collections.abc.Callable[[Int | None], RankerBase] | str = '2imp'¶: The ranker is a RankerBase object that orders the solutions after they have been evaluated in the environment. This parameter may be a callable (e.g. a class or a lambda function) that takes in no parameters and returns an instance of RankerBase, or it may be a full or abbreviated ranker name as described in ribs.emitters.rankers.
selection_rule: 'mu' | 'filter' = 'filter'¶: Method for selecting parents in CMA-ES. With “mu” selection, the first half of the solutions will be selected as parents, while in “filter”, any solutions that were added to the archive will be selected.
restart_rule: 'no_improvement' | 'basic' | int = 'no_improvement'¶: Method to use when checking for restarts. If given an integer, then the emitter will restart after this many iterations, where each iteration is a call to tell(). With “basic”, only the default CMA-ES convergence rules will be used, while with “no_improvement”, the emitter will restart when none of the proposed solutions were added to the archive.
grad_opt: collections.abc.Callable[..., GradientOptBase] | str = 'adam'¶: Gradient optimizer to use for the gradient ascent step of the algorithm. The optimizer is a GradientOptBase object. This parameter may be a callable (e.g. a class or a lambda function) which takes in the theta0 and lr arguments, or it may be a full or abbreviated name as described in ribs.emitters.opt.
grad_opt_kwargs: dict | None = None¶: Additional arguments to pass to the gradient optimizer. See the gradient-based optimizers in ribs.emitters.opt for the arguments allowed by each optimizer. Note that we already pass in theta0 and lr.
es: collections.abc.Callable[..., EvolutionStrategyBase] | str = 'cma_es'¶: The evolution strategy is an EvolutionStrategyBase object that is used to adapt the distribution from which new solutions are sampled. This parameter may be a callable (e.g. a class or a lambda function) that takes in the parameters of EvolutionStrategyBase along with kwargs provided by the es_kwargs argument, or it may be a full or abbreviated optimizer name as described in ribs.emitters.opt.
es_kwargs: dict | None = None¶: Additional arguments to pass to the evolution strategy optimizer. See the evolution-strategy-based optimizers in ribs.emitters.opt for the arguments allowed by each optimizer.
normalize_grad: bool = True¶: If true (default), then gradient infomation will be normalized. Otherwise, it will not be normalized.
bounds: Collection[tuple[None | Float, None | Float]] | None = None¶: This argument may be used for providing solution space bounds in the future. This emitter does not currently support solution space bounds, as bounding solutions for DQD algorithms such as CMA-MEGA is an open problem. Hence, this argument must be set to None.
lower_bounds: ArrayLike | None = None¶: This argument may be used for providing solution space bounds in the future. This emitter does not currently support solution space bounds, as bounding solutions for DQD algorithms such as CMA-MEGA is an open problem. Hence, this argument must be set to None.
upper_bounds: ArrayLike | None = None¶: This argument may be used for providing solution space bounds in the future. This emitter does not currently support solution space bounds, as bounding solutions for DQD algorithms such as CMA-MEGA is an open problem. Hence, this argument must be set to None.
batch_size: Int | None = None¶: Number of solutions to return in ask(). If not passed in, a batch size will be automatically calculated using the default CMA-ES rules. This does not account for the one solution returned by ask_dqd(), which is the solution point maintained by the gradient optimizer.
epsilon: Float = 1e-08¶: For numerical stability, we add a small epsilon when normalizing gradients in tell_dqd() – refer to the implementation here. Pass this parameter to configure that epsilon.
seed: Int | None = None¶: Value to seed the random number generator. Set to None to avoid a fixed seed.

Raises:¶

ValueError – There is an error in x0 or initial_solutions.
ValueError – bounds is set even though it is not currently supported.
ValueError – If restart_rule, selection_rule, or ranker is invalid.

Methods

`ask`()	Samples new solutions from a gradient arborescence.
`ask_dqd`()	Returns a new solution from the gradient optimizer.
`tell`(solution, objective, measures, ...)	Gives the emitter results from evaluating solutions.
`tell_dqd`(solution, objective, measures, ...)	Gives the emitter results from evaluating the gradient of the solutions.

Attributes

`archive`	Stores solutions generated by this emitter.
`batch_size`	Number of solutions to return in `ask()`.
`batch_size_dqd`	Number of solutions to return in `ask_dqd()`.
`epsilon`	Added for numerical stability when normalizing gradients in `tell_dqd()`.
`itrs`	The number of iterations for this emitter.
`lower_bounds`	`(solution_dim,)` array with lower bounds of solution space.
`restarts`	The number of restarts for this emitter.
`solution_dim`	Dimensionality of solutions produced by this emitter.
`upper_bounds`	`(solution_dim,)` array with upper bounds of solution space.
`x0`	Initial solution for the optimizer.

ask() → ndarray[source]¶

Samples new solutions from a gradient arborescence.

The coefficients come from a multivariate Gaussian distribution that is managed by the evolution strategy optimizer self._opt.

This method returns batch_size solutions, even though one solution is returned via ask_dqd.

Returns:¶: (batch_size, solution_dim) array – a batch of new solutions to evaluate.
Raises:¶: RuntimeError – This method was called without first passing gradients with calls to ask_dqd() and tell_dqd().

ask_dqd() → ndarray[source]¶

Returns a new solution from the gradient optimizer.

Call :meth:`ask_dqd` and :meth:`tell_dqd` (in this order) before calling :meth:`ask` and :meth:`tell`.

Returns:¶: A new solution to evaluate.

tell(solution: numpy.typing.ArrayLike, objective: numpy.typing.ArrayLike, measures: numpy.typing.ArrayLike, add_info: dict[str, ndarray], **fields: numpy.typing.ArrayLike) → None[source]¶

Gives the emitter results from evaluating solutions.

The solutions are ranked based on the rank() function defined by self._ranker.

Parameters:¶

solution: numpy.typing.ArrayLike¶: (batch_size, solution_dim) array of solutions generated by this emitter’s ask() method.
objective: numpy.typing.ArrayLike¶: 1D array containing the objective function value of each solution.
measures: numpy.typing.ArrayLike¶: (batch_size, measure space dimension) array with the measure space coordinates of each solution.
add_info: dict[str, ndarray]¶: Data returned from the archive add() method.
**fields: numpy.typing.ArrayLike¶: Additional data for each solution. Each argument should be an array with batch_size as the first dimension.

Raises:¶

RuntimeError – This method was called without first passing gradients with calls to ask_dqd() and tell_dqd().

tell_dqd(solution: numpy.typing.ArrayLike, objective: numpy.typing.ArrayLike, measures: numpy.typing.ArrayLike, jacobian: numpy.typing.ArrayLike, add_info: dict[str, ndarray], **fields: numpy.typing.ArrayLike) → None[source]¶

Gives the emitter results from evaluating the gradient of the solutions.

Parameters:¶

solution: numpy.typing.ArrayLike¶: (batch_size, solution_dim) array of solutions generated by this emitter’s ask() method.
objective: numpy.typing.ArrayLike¶: 1D array containing the objective function value of each solution.
measures: numpy.typing.ArrayLike¶: (batch_size, measure space dimension) array with the measure space coordinates of each solution.
jacobian: numpy.typing.ArrayLike¶: (batch_size, 1 + measure_dim, solution_dim) array consisting of Jacobian matrices of the solutions obtained from ask_dqd(). Each matrix should consist of the objective gradient of the solution followed by the measure gradients.
add_info: dict[str, ndarray]¶: Data returned from the archive add() method.
**fields: numpy.typing.ArrayLike¶: Additional data for each solution. Each argument should be an array with batch_size as the first dimension.

property archive : ArchiveBase¶: Stores solutions generated by this emitter.

property batch_size : int | integer¶: Number of solutions to return in ask().

property batch_size_dqd : int | integer¶

Number of solutions to return in ask_dqd().

This is always 1, as we only return the solution point in ask_dqd().

property epsilon : float | floating¶: Added for numerical stability when normalizing gradients in tell_dqd().

property itrs : int¶: The number of iterations for this emitter.

property lower_bounds : ndarray¶

(solution_dim,) array with lower bounds of solution space.

For instance, [-1, -1, -1] indicates that every dimension of the solution space has a lower bound of -1.

property restarts : int¶: The number of restarts for this emitter.

property solution_dim : int | integer | tuple[int | integer, ...]¶: Dimensionality of solutions produced by this emitter.

property upper_bounds : ndarray¶

(solution_dim,) array with upper bounds of solution space.

For instance, [1, 1, 1] indicates that every dimension of the solution space has an upper bound of 1.

property x0 : ndarray¶: Initial solution for the optimizer.