What’s New in v0.5.0¶

The new version of pyribs is here! Much has changed since our last version, v0.4.0. Most of these changes can be categorized under two goals: (1) integrate new algorithms via new emitters and schedulers, and (2) improve archive performance via batched operations. We will review the changes related to these goals and several other changes made to pyribs. For the full list of changes, refer to our History page.

General¶

We’ll start with some important general changes.

Tutorials¶

We’ve added new tutorials and expanded our old ones! We recommend going over the updated version of our introductory tutorial, Using CMA-ME to Land a Lunar Lander Like a Space Shuttle, to get a feel for the new API. Then, learn how we have integrated the CMA-MAE algorithm in Upgrading CMA-ME to CMA-MAE on the Sphere Benchmark, and implement Differentiable Quality Diversity (DQD) algorithms in the tutorial Generating Tom Cruise Images with DQD Algorithms.

Terminology¶

• measures vs behaviors: We have adopted the measures terminology of recent literature over the behaviors terminology in pyribs 0.4.0. Names such as behavior_values are now referred to as measures. While QD originated in neuroevolution with the purpose of producing diverse collections of agents, QD optimization has grown into a general-purpose optimization paradigm. For example, in an application where QD generates images of a face with varying age and hair length, it seems odd to refer to age and hair length as behaviors. Our new terminology instead refers to the age and hair length as measures of the solutions, where QD optimization must vary the outputs of those measures.

• Batch arguments: Many of our methods now operate in batch. The batch arguments are referred to as solution_batch, objective_batch, measures_batch, and metadata_batch, while individual arguments are referred to as solution, objective, measures, and metadata.

• cells vs bins: For consistency with the literature, we have replaced the term bins with cells when discussing archives.

• optimizers are now schedulers: To better reflect their role, the “optimizers” from pyribs v0.4.0 are now referred to as “schedulers”. All schedulers are under the ribs.schedulers module, including Scheduler and BanditScheduler.

Installation¶

There are now three distributions of pyribs, as shown in the table below. If you do not use ribs.visualize, we recommend installing the default distribution with with pip install ribs or conda install pyribs. If you do use ribs.visualize, you can install pyribs with pip install ribs[visualize] or conda install pyribs-visualize.

Goal 1: Implement New Algorithms¶

Our first major goal in pyribs v0.5.0 has been to implement an array of new algorithms via new emitters and schedulers.

Flexible CMA-ME and CMA-MAE with EvolutionStrategyEmitter¶

We introduce the EvolutionStrategyEmitter which replaces the earlier ImprovementEmitter, RandomDirectionEmitter, and OptimizingEmitter. This emitter may be used in both CMA-ME and CMA-MAE. The behaviors of these earlier emitters may be replicated by selecting an appropriate ranker from the ribs.emitters.rankers module. For example:

EvolutionStrategyEmitter(archive, ..., ranker="2imp")  # Equivalent to ImprovementEmitter.
EvolutionStrategyEmitter(archive, ..., ranker="imp")  # Single-stage improvement ranking as is used in CMA-MAE.
EvolutionStrategyEmitter(archive, ..., ranker="2rd")  # Two-stage random direction ranking.
EvolutionStrategyEmitter(archive, ..., ranker="obj")  # Objective ranking as was done in OptimizingEmitter.

EvolutionStrategyEmitter also supports evolution strategies other than CMA-ES, thus enabling it to implement scalable variants of CMA-ME and CMA-MAE described in Tjanaka 2023. These evolution strategies are available in the ribs.emitters.opt module. For example:

EvolutionStrategyEmitter(archive, ..., es="sep_cma_es")  # sep-CMA-ES instead of CMA-ES.

DQD Algorithms with GradientArborescenceEmitter¶

We have added a GradientArborescenceEmitter which supports DQD algorithms. For usage examples, see the tutorial Generating Tom Cruise Images with DQD Algorithms.

Custom Initial Solutions in Emitters¶

By default, on the first iteration (the first iteration is detected by checking that the archive is empty), both GaussianEmitter and IsoLineEmitter sample solutions from a Gaussian distribution. However, many implementations of MAP-Elites sample solutions from a uniform distribution on the first iteration. More generally, users may seek to provide any custom population of solutions for the first iteration.

Before, it was possible to provide custom initial solutions by evaluating the initial solutions and directly add()’ing them to the archive, like so:

archive = ...
emitters = [GaussianEmitter(archive, ...)]
scheduler = Scheduler(archive, emitters)

initial_solutions = ...
objectives, measures = evaluate(initial_solutions)
archive.add(initial_solutions, objectives, measures)

for itr in range(1000):
    ...

However, it can be inconvenient to have to add this special case before the main pyribs loop, e.g., if the evaluation function is more complex than a single line. Thus, we now make it possible to pass the initial solutions to the emitters so that on the first iteration of the QD algorithm (more specifically, when the archive is empty), these initial solutions are returned.

archive = ...
emitters = [GaussianEmitter(archive, ..., initial_solutions=[[0.0, 1.0], [1.3, 2.0]])]
scheduler = Scheduler(archive, emitters)

for itr in range(1000):
    # On the first iteration, `solutions` will be the `initial_solutions` that
    # were passed into GaussianEmitter.
    solutions = scheduler.ask()

ME-MAP-Elites with BanditScheduler¶

We have added a new BanditScheduler which maintains a pool of emitters and only asks a subset of emitters for solutions on each iteration. The emitters to ask are selected with a multi-armed bandit algorithm. This scheduler is our implementation of the ME-MAP-Elites algorithm.

Goal 2: Improve Archive Performance via Batching¶

Before, pyribs archives operated on solutions one at a time. Now, most archives (with the exception of SlidingBoundariesArchive) improve performance by relying on batch operations.

New Archive Methods¶

The following table shows the old methods and the names of the new methods which operate on batched inputs. As a convenience, we have also included “single” methods which can operate on single inputs.

To elaborate on these changes:

• All archive indices must now be integers (before, indices could be tuples of integers). Thus, the get_index() method has been replaced with index_of(), which takes in a batch of measures (i.e. a (batch_size, measure_dim) array) and returns a batch of integer indices. Furthermore, whereas ArchiveBase used to take in a storage_dims argument, it now takes in a single storage_dim argument because indices are all integers.

  • Since grid indices have meaning in GridArchive and SlidingBoundariesArchive, we have also added a grid_to_int_index() and int_to_grid_index() method to convert to and from grid indices in these archives.

• add() formerly operated on single solutions. Now, it inserts solutions into the archive in batch. There is also an add_single() which inserts solutions one at a time. The source code for add_single() is more amenable to modifications than that of the batched add() method.

• get_random_elite(), which sampled one elite from the archive, has been replaced with a batched sample_elites() method which samples multiple elites at once.

• The elite_with_behavior() method which retrieved one elite with given measures has been replaced with retrieve(), which retrieves a batch of such elites.

Some of these methods also include “single” versions which operate on single solutions, e.g., index_of_single() returns the integer index of a single measures array.

EliteBatch¶

We have added an EliteBatch class to represent batches of elites.

ArchiveDataFrame Column Renaming¶

To reflect the terminology changes and the switch to integer indices in archives, columns in ArchiveDataFrame (returned by as_pandas()) are as follows:

Before, they were:

We have also renamed the ArchiveDataFrame methods as follows:

Miscellaneous¶

Finally, here are several miscellaneous improvements we have made to pyribs.

Removal of initialize() Method for Archives¶

Previously, archives required calling initialize(solution_dim) after being constructed. This method was typically called by the Optimizer/Scheduler. Now, archives are directly constructed with the solution_dim argument for simplicity.

# v0.4.0 (OLD)
archive = GridArchive(...)
archive.initialize(solution_dim)  # Would be called by Optimizer / Scheduler.

# v0.5.0 (NEW)
archive = GridArchive(solution_dim, ...)

More Flexible Visualization Tools¶

In all heatmap visualization tools, we have made the colorbar more flexible by adding a cbar option to control which axes the colorbar appears on and a cbar_kwargs option to pass arguments directly to Matplotlib’s colorbar().

from ribs.visualize import grid_archive_heatmap  # cvt_archive_heatmap and sliding_boundaries_archive_heatmap also work

archive = ...

grid_archive_heatmap(archive, cbar="auto")  # Display the colorbar as part of the current axes (default).
grid_archive_heatmap(archive, cbar=None)  # Don't display the colorbar at all.
grid_archive_heatmap(archive, cbar=cbar_ax)  # Display colorbar on a custom Axes.
grid_archive_heatmap(archive, ..., cbar_kwargs={...})  # Pass arguments to the colorbar.

In addition:

• We have added an aspect argument which can set the aspect ratio of the heatmap, i.e., the ratio height / width.

• We now support heatmaps for 1D grid archives.

Continuous Quality Diversity (CQD) Score¶

We support computing the Continuous Quality Diversity (CQD) Score in all archives with the cqd_score() method.

ArchiveStats¶

The ArchiveStats object now includes the normalized QD score (i.e., the QD score divided by the number of cells in the archive). Furthermore, ArchiveStats is now a dataclass rather than a namedtuple.

archive.stats.norm_qd_score  # Normalized QD score.

Deprecation of Positional Arguments¶

Following scikit-learn, almost all constructor arguments must now be passed in as keyword arguments. Given the many parameters that these objects have, this makes it easier to see what each parameter means.

To illustrate, the signature for EvolutionStrategyEmitter is:

class EvolutionStrategyEmitter(...):

    def __init__(archive, *, x0, sigma0, ranker='2imp', es='cma_es', es_kwargs=None, selection_rule='filter', restart_rule='no_improvement', bounds=None, batch_size=None, seed=None):
        ...

All parameters after the * are keyword-only. The following will result in an error:

EvolutionStrategyEmitter(archive, np.zeros(10), 0.1)

While the following will be accepted:

EvolutionStrategyEmitter(archive, x0=np.zeros(10), sigma=0.1)

Input Validation¶

Many of our methods, e.g., the archives’ add() method and schedulers’ ask() and tell() methods, now have input validation to help catch common argument errors. For example, we check that arguments are the correct shape, and we check that they do not have NaN or inf values.

Deprecation of ribs.factory¶

We have removed ribs.factory as it is out of scope for the current library.