ribs.schedulers.BanditScheduler

class ribs.schedulers.BanditScheduler(archive, emitter_pool, num_active, *, reselect='terminated', zeta=0.05, result_archive=None, add_mode='batch')

Schedules emitters with a bandit algorithm.

This implementation is based on Cully 2021.

Note

This class follows the similar ask-tell framework as Scheduler, and enforces similar constraints in the arguments and methods. Please refer to the documentation of Scheduler for more details.

Note

The main difference between BanditScheduler and Scheduler is that, unlike Scheduler, DQD emitters are not supported by BanditScheduler.

To initialize this class, first create an archive and a list of emitters for the QD algorithm. The BanditScheduler will schedule the emitters using the Upper Confidence Bound - 1 algorithm (UCB1). Everytime ask() is called, the emitters are sorted based on the potential reward function from UCB1. Then, the top num_active emitters are used for ask-tell.

Parameters

• archive (ribs.archives.ArchiveBase) – An archive object, e.g. one selected from ribs.archives.

• emitter_pool (list of ribs.archives.EmitterBase) – A pool of emitters to select from, e.g. ribs.emitters.GaussianEmitter. On the first iteration, the first num_active emitters from the emitter_pool will be activated.

• num_active (int) – The number of active emitters at a time. Active emitters are used when calling ask-tell.

• zeta (float) – Hyperparamter of UCB1 that balances the trade-off between the accuracy and the uncertainty of the emitters. Increasing this parameter will emphasize the uncertainty of the emitters. Refer to the original paper for more information.

• reselect (str) – Indicates how emitters are reselected from the pool. The default is “terminated”, where only terminated/restarted emitters are deactivated and reselected (but they might be selected again). Alternatively, use “all” to reselect all active emitters every iteration.

• add_mode (str) – Indicates how solutions should be added to the archive. The default is “batch”, which adds all solutions with one call to add(). Alternatively, use “single” to add the solutions one at a time with add_single(). “single” mode is included for legacy reasons, as it was the only mode of operation in pyribs 0.4.0 and before. We highly recommend using “batch” mode since it is significantly faster.

• result_archive (ribs.archives.ArchiveBase) – In some algorithms, such as CMA-MAE, the archive does not store all the best-performing solutions. The result_archive is a secondary archive where we can store all the best-performing solutions.

Raises

• ValueError – Number of active emitter is less than one.

• ValueError – Less emitters in the pool than the number of active emitters.

• ValueError – The emitters passed in do not have the same solution dimensions.

• ValueError – The same emitter instance was passed in multiple times. Each emitter should be a unique instance (see the warning above).

• ValueError – Invalid value for add_mode.

Methods

ask()	Generates a batch of solutions by calling ask() on all active emitters.
ask_dqd()	Generates a batch of solutions by calling ask_dqd() on all DQD emitters.
tell(objective_batch, measures_batch[, ...])	Returns info for solutions from ask().
tell_dqd(objective_batch, measures_batch, ...)	Returns info for solutions from ask_dqd().

Attributes

archive	Archive for storing solutions found in this scheduler.
emitters	Emitters for generating solutions in this scheduler.
result_archive	Another archive for storing solutions found in this optimizer.

ask()

Generates a batch of solutions by calling ask() on all active emitters.

The emitters used by ask are determined by the UCB1 algorithm. Briefly, emitters that have never been selected before are prioritized, then emitters are sorted in descending order based on the accurary of their past prediction.

Note

The order of the solutions returned from this method is important, so do not rearrange them.

Returns

An array of n solutions to evaluate. Each row contains a single solution.

Return type

(batch_size, dim) array

Raises

RuntimeError – This method was called without first calling tell().

ask_dqd()

Generates a batch of solutions by calling ask_dqd() on all DQD emitters.

This method is not supported for this scheduler and throws an error if called.

Raises

NotImplementedError – This method is not supported by this scheduler.

tell(objective_batch, measures_batch, metadata_batch=None)

Returns info for solutions from ask().

The emitters are the same with those used in the last call to ask().

Note

The objective batch, measures batch, and metadata batch must be in the same order as the solutions created by ask(); i.e. objective_batch[i], measures_batch[i], and metadata_batch[i] should be the objective, measures, and metadata for solution_batch[i].

Parameters

• objective_batch ((batch_size,) array) – Each entry of this array contains the objective function evaluation of a solution.

• measures_batch ((batch_size, measures_dm) array) – Each row of this array contains a solution’s coordinates in measure space.

• metadata_batch ((batch_size,) array) – Each entry of this array contains an object holding metadata for a solution.

Raises

• RuntimeError – This method is called without first calling ask().

• ValueError – objective_batch, measures_batch, or metadata_batch has the wrong shape.

tell_dqd(objective_batch, measures_batch, jacobian_batch, metadata_batch=None)

Returns info for solutions from ask_dqd().

This method is not supported for this scheduler and throws an error if called.

Raises

NotImplementedError – This method is not supported by this scheduler.

property archive

Archive for storing solutions found in this scheduler.

Type

ribs.archives.ArchiveBase

property emitters

Emitters for generating solutions in this scheduler.

Type

list of ribs.archives.EmitterBase

property result_archive

Another archive for storing solutions found in this optimizer. If result_archive was not passed to the constructor, this property is the same as archive.

Type

ribs.archives.ArchiveBase