ribs.emitters.opt.OpenAIEvolutionStrategy

class ribs.emitters.opt.OpenAIEvolutionStrategy(sigma0, solution_dim, batch_size=None, seed=None, dtype=<class 'numpy.float64'>, lower_bounds=-inf, upper_bounds=inf, mirror_sampling=True, **adam_kwargs)

OpenAI-ES optimizer for use with emitters.

Refer to EvolutionStrategyBase for usage instruction.

Parameters

• sigma0 (float) – Initial step size.

• batch_size (int) – Number of solutions to evaluate at a time. If None, we calculate a default batch size based on solution_dim.

• solution_dim (int) – Size of the solution space.

• seed (int) – Seed for the random number generator.

• dtype (str or data-type) – Data type of solutions.

• lower_bounds (float or np.ndarray) – scalar or (solution_dim,) array indicating lower bounds of the solution space. Scalars specify the same bound for the entire space, while arrays specify a bound for each dimension. Pass -np.inf in the array or scalar to indicated unbounded space.

• upper_bounds (float or np.ndarray) – Same as above, but for upper bounds (and pass np.inf instead of -np.inf).

• mirror_sampling (bool) – Whether to use mirror sampling when gathering solutions. Defaults to True.

• adam_kwargs (dict) – Keyword arguments passed to AdamOpt.

Methods

ask([batch_size])	Samples new solutions.
check_stop(ranking_values)	Checks if the ES should stop and be reset.
reset(x0)	Resets the ES to start at x0.
tell(ranking_indices, ranking_values, ...)	Passes the solutions back to the ES.

ask(batch_size=None)

Samples new solutions.

Parameters

batch_size (int) – batch size of the sample. Defaults to self.batch_size.

check_stop(ranking_values)

Checks if the ES should stop and be reset.

Parameters

ranking_values (numpy.ndarray) – Array of values that were used to rank the solutions. Shape can be either (batch_size,) or (batch_size, n_values)``, where batch_size is the number of solutions and n_values is the number of values that the ranker used. Note that unlike in tell(), these values must be sorted according to the ranking_indices passed to tell().

Returns

True if any of the stopping conditions are satisfied.

reset(x0)

Resets the ES to start at x0.

Parameters

x0 (numpy.ndarray) – Initial mean.

tell(ranking_indices, ranking_values, num_parents)

Passes the solutions back to the ES.

Parameters

• ranking_indices (numpy.ndarray) – Integer indices that are used to rank the solutions returned in ask(). Note that these are NOT the ranks of the solutions. Rather, they are indices such that solutions[ranking_indices] will correctly rank the solutions (think of an argsort).

• ranking_values (numpy.ndarray) – Array of values that were used to rank the solutions. Shape can be either (batch_size,) or (batch_size, n_values)``, where batch_size is the number of solutions and n_values is the number of values that the ranker used. Note that we assume a descending sort, i.e., higher values should come first.

• num_parents (int) – Number of top solutions to select from the ranked solutions.