ribs.schedulers.BanditScheduler¶
- class ribs.schedulers.BanditScheduler(archive, emitter_pool, num_active, *, reselect='terminated', zeta=0.05, result_archive=None, add_mode='batch')[source]¶
Schedules emitters with a bandit algorithm.
This implementation is based on Cully 2021.
Note
This class follows the similar ask-tell framework as
Scheduler
, and enforces similar constraints in the arguments and methods. Please refer to the documentation ofScheduler
for more details.Note
The main difference between
BanditScheduler
andScheduler
is that, unlikeScheduler
, DQD emitters are not supported byBanditScheduler
.To initialize this class, first create an archive and a list of emitters for the QD algorithm. The BanditScheduler will schedule the emitters using the Upper Confidence Bound - 1 algorithm (UCB1). Everytime
ask()
is called, the emitters are sorted based on the potential reward function from UCB1. Then, the top num_active emitters are used for ask-tell.- Parameters
archive (ribs.archives.ArchiveBase) – An archive object, e.g. one selected from
ribs.archives
.emitter_pool (list of ribs.archives.EmitterBase) – A pool of emitters to select from, e.g.
ribs.emitters.GaussianEmitter
. On the first iteration, the first num_active emitters from the emitter_pool will be activated.num_active (int) – The number of active emitters at a time. Active emitters are used when calling ask-tell.
zeta (float) – Hyperparamter of UCB1 that balances the trade-off between the accuracy and the uncertainty of the emitters. Increasing this parameter will emphasize the uncertainty of the emitters. Refer to the original paper for more information.
reselect (str) – Indicates how emitters are reselected from the pool. The default is “terminated”, where only terminated/restarted emitters are deactivated and reselected (but they might be selected again). Alternatively, use “all” to reselect all active emitters every iteration.
add_mode (str) – Indicates how solutions should be added to the archive. The default is “batch”, which adds all solutions with one call to
add()
. Alternatively, use “single” to add the solutions one at a time withadd_single()
. “single” mode is included for legacy reasons, as it was the only mode of operation in pyribs 0.4.0 and before. We highly recommend using “batch” mode since it is significantly faster.result_archive (ribs.archives.ArchiveBase) – In some algorithms, such as CMA-MAE, the archive does not store all the best-performing solutions. The
result_archive
is a secondary archive where we can store all the best-performing solutions.
- Raises
TypeError – The
emitter_pool
argument was not a list of emitters.ValueError – Number of active emitters is less than one.
ValueError – Less emitters in the pool than the number of active emitters.
ValueError – The emitters passed in do not have the same solution dimensions.
ValueError – The same emitter instance was passed in multiple times. Each emitter should be a unique instance (see the warning above).
ValueError – Invalid value for
add_mode
.ValueError – The
result_archive
andarchive
are the same object (result_archive
should not be passed in in this case).ValueError – The
result_archive
andarchive
have different fields.
Methods
ask
()Generates a batch of solutions by calling ask() on all active emitters.
ask_dqd
()Generates a batch of solutions by calling ask_dqd() on all DQD emitters.
tell
(objective, measures, **fields)Returns info for solutions from
ask()
.tell_dqd
(objective, measures, jacobian)Returns info for solutions from
ask_dqd()
.Attributes
Boolean array indicating which emitters in the
emitter_pool
are currently active.Archive for storing solutions found in this scheduler.
The pool of emitters available in the scheduler.
Another archive for storing solutions found in this optimizer.
- ask()[source]¶
Generates a batch of solutions by calling ask() on all active emitters.
The emitters used by ask are determined by the UCB1 algorithm. Briefly, emitters that have never been selected before are prioritized, then emitters are sorted in descending order based on the accurary of their past prediction.
Note
The order of the solutions returned from this method is important, so do not rearrange them.
- Returns
An array of n solutions to evaluate. Each row contains a single solution.
- Return type
(batch_size, dim) array
- Raises
RuntimeError – This method was called immediately after calling an ask method.
- ask_dqd()[source]¶
Generates a batch of solutions by calling ask_dqd() on all DQD emitters.
This method is not supported for this scheduler and throws an error if called.
- Raises
NotImplementedError – This method is not supported by this scheduler.
- tell(objective, measures, **fields)[source]¶
Returns info for solutions from
ask()
.The emitters are the same with those used in the last call to
ask()
.Note
The objective and measures arrays must be in the same order as the solutions created by
ask_dqd()
; i.e.objective[i]
andmeasures[i]
should be the objective and measures forsolution[i]
.- Parameters
objective ((batch_size,) array or None) – Each entry of this array contains the objective function evaluation of a solution. This can also be None to indicate there is no objective – this would be the case in diversity optimization problems.
measures ((batch_size, measures_dm) array) – Each row of this array contains a solution’s coordinates in measure space.
fields (keyword arguments) – Additional data for each solution. Each argument should be an array with batch_size as the first dimension.
- Raises
RuntimeError – This method was called without first calling
ask()
.ValueError – One of the inputs has the wrong shape.
- tell_dqd(objective, measures, jacobian)[source]¶
Returns info for solutions from
ask_dqd()
.This method is not supported for this scheduler and throws an error if called.
- Raises
NotImplementedError – This method is not supported by this scheduler.
- property active¶
Boolean array indicating which emitters in the
emitter_pool
are currently active.- Type
- property archive¶
Archive for storing solutions found in this scheduler.
- property emitter_pool¶
The pool of emitters available in the scheduler.
- Type
list of ribs.archives.EmitterBase