ribs.schedulers.BanditScheduler

class ribs.schedulers.BanditScheduler(archive, emitter_pool, num_active, *, reselect='terminated', zeta=0.05, result_archive=None, add_mode='batch')

Schedules emitters with a bandit algorithm.

This implementation is based on Cully 2021.

Note

This class follows the similar ask-tell framework as Scheduler, and enforces similar constraints in the arguments and methods. Please refer to the documentation of Scheduler for more details.

Note

The main difference between BanditScheduler and Scheduler is that, unlike Scheduler, DQD emitters are not supported by BanditScheduler.

To initialize this class, first create an archive and a list of emitters for the QD algorithm. The BanditScheduler will schedule the emitters using the Upper Confidence Bound - 1 algorithm (UCB1). Everytime ask() is called, the emitters are sorted based on the potential reward function from UCB1. Then, the top num_active emitters are used for ask-tell.

Parameters

• archive (ribs.archives.ArchiveBase) – An archive object, e.g. one selected from ribs.archives.

• emitter_pool (list of ribs.archives.EmitterBase) – A pool of emitters to select from, e.g. ribs.emitters.GaussianEmitter. On the first iteration, the first num_active emitters from the emitter_pool will be activated.

• num_active (int) – The number of active emitters at a time. Active emitters are used when calling ask-tell.

• zeta (float) – Hyperparamter of UCB1 that balances the trade-off between the accuracy and the uncertainty of the emitters. Increasing this parameter will emphasize the uncertainty of the emitters. Refer to the original paper for more information.

• reselect (str) – Indicates how emitters are reselected from the pool. The default is “terminated”, where only terminated/restarted emitters are deactivated and reselected (but they might be selected again). Alternatively, use “all” to reselect all active emitters every iteration.

• add_mode (str) – Indicates how solutions should be added to the archive. The default is “batch”, which adds all solutions with one call to add(). Alternatively, use “single” to add the solutions one at a time with add_single(). “single” mode is included for legacy reasons, as it was the only mode of operation in pyribs 0.4.0 and before. We highly recommend using “batch” mode since it is significantly faster.

• result_archive (ribs.archives.ArchiveBase) – In some algorithms, such as CMA-MAE, the archive does not store all the best-performing solutions. The result_archive is a secondary archive where we can store all the best-performing solutions.

Raises

• TypeError – The emitter_pool argument was not a list of emitters.

• ValueError – Number of active emitters is less than one.

• ValueError – Less emitters in the pool than the number of active emitters.

• ValueError – The emitters passed in do not have the same solution dimensions.

• ValueError – The same emitter instance was passed in multiple times. Each emitter should be a unique instance (see the warning above).

• ValueError – Invalid value for add_mode.

• ValueError – The result_archive and archive are the same object (result_archive should not be passed in in this case).

• ValueError – The result_archive and archive have different fields.

Methods

ask()	Generates a batch of solutions by calling ask() on all active emitters.
ask_dqd()	Generates a batch of solutions by calling ask_dqd() on all DQD emitters.
tell(objective, measures, **fields)	Returns info for solutions from ask().
tell_dqd(objective, measures, jacobian)	Returns info for solutions from ask_dqd().

Attributes

active	Boolean array indicating which emitters in the emitter_pool are currently active.
archive	Archive for storing solutions found in this scheduler.
emitter_pool	The pool of emitters available in the scheduler.
result_archive	Another archive for storing solutions found in this optimizer.

ask()

Generates a batch of solutions by calling ask() on all active emitters.

The emitters used by ask are determined by the UCB1 algorithm. Briefly, emitters that have never been selected before are prioritized, then emitters are sorted in descending order based on the accurary of their past prediction.

Note

The order of the solutions returned from this method is important, so do not rearrange them.

Returns

An array of n solutions to evaluate. Each row contains a single solution.

Return type

(batch_size, dim) array

Raises

RuntimeError – This method was called immediately after calling an ask method.

ask_dqd()

Generates a batch of solutions by calling ask_dqd() on all DQD emitters.

This method is not supported for this scheduler and throws an error if called.

Raises

NotImplementedError – This method is not supported by this scheduler.

tell(objective, measures, **fields)

Returns info for solutions from ask().

The emitters are the same with those used in the last call to ask().

Note

The objective and measures arrays must be in the same order as the solutions created by ask_dqd(); i.e. objective[i] and measures[i] should be the objective and measures for solution[i].

Parameters

• objective ((batch_size,) array or None) – Each entry of this array contains the objective function evaluation of a solution. This can also be None to indicate there is no objective – this would be the case in diversity optimization problems.

• measures ((batch_size, measures_dm) array) – Each row of this array contains a solution’s coordinates in measure space.

• fields (keyword arguments) – Additional data for each solution. Each argument should be an array with batch_size as the first dimension.

Raises

• RuntimeError – This method was called without first calling ask().

• ValueError – One of the inputs has the wrong shape.

tell_dqd(objective, measures, jacobian)

Returns info for solutions from ask_dqd().

This method is not supported for this scheduler and throws an error if called.

Raises

NotImplementedError – This method is not supported by this scheduler.

property active

Boolean array indicating which emitters in the emitter_pool are currently active.

Type

numpy.ndarray

property archive

Archive for storing solutions found in this scheduler.

Type

ribs.archives.ArchiveBase

property emitter_pool

The pool of emitters available in the scheduler.

Type

list of ribs.archives.EmitterBase

property result_archive

Another archive for storing solutions found in this optimizer. If result_archive was not passed to the constructor, this property is the same as archive.

Type

ribs.archives.ArchiveBase