ribs.archives.SlidingBoundariesArchive¶

class ribs.archives.SlidingBoundariesArchive(*, solution_dim, dims, ranges, epsilon=1e-06, qd_score_offset=0.0, seed=None, dtype=<class 'numpy.float64'>, remap_frequency=100, buffer_capacity=1000)[source]¶

An archive with a fixed number of sliding boundaries on each dimension.

This archive is the container described in Fontaine 2019. Just like the GridArchive, it can be visualized as an n-dimensional grid in the measure space that is divided into a certain number of cells in each dimension. Internally, this archive stores a buffer with the buffer_capacity most recent solutions and uses them to determine the boundaries of each dimension of the measure space. After every remap_frequency solutions are inserted, the archive remaps the boundaries based on the solutions in the buffer.

Initially, the archive has no solutions, so it cannot automatically calculate the boundaries. Thus, until the first remap, this archive divides the measure space defined by ranges into equally sized cells.

Overall, this archive attempts to make the distribution of the space illuminated by the archive more accurately match the true distribution of the measures when they are not uniformly distributed.

Note

Unlike other archives, this archive does not currently support thresholds or batched addition (see add()).

Parameters

solution_dim (int) – Dimension of the solution space.
dims (array-like) – Number of cells in each dimension of the measure space, e.g. [20, 30, 40] indicates there should be 3 dimensions with 20, 30, and 40 cells. (The number of dimensions is implicitly defined in the length of this argument).
ranges (array-like of (float, float)) – Initial upper and lower bound of each dimension of the measure space, e.g. [(-1, 1), (-2, 2)] indicates the first dimension should have bounds \([-1,1]\) (inclusive), and the second dimension should have bounds \([-2,2]\) (inclusive). ranges should be the same length as dims.
epsilon (float) – Due to floating point precision errors, we add a small epsilon when computing the archive indices in the index_of() method – refer to the implementation here. Pass this parameter to configure that epsilon.
qd_score_offset (float) – Archives often contain negative objective values, and if the QD score were to be computed with these negative objectives, the algorithm would be penalized for adding new cells with negative objectives. Thus, a standard practice is to normalize all the objectives so that they are non-negative by introducing an offset. This QD score offset will be subtracted from all objectives in the archive, e.g., if your objectives go as low as -300, pass in -300 so that each objective will be transformed as objective - (-300).
seed (int) – Value to seed the random number generator. Set to None to avoid a fixed seed.
dtype (str or data-type) – Data type of the solutions, objectives, and measures. We only support "f" / np.float32 and "d" / np.float64.
remap_frequency (int) – Frequency of remapping. Archive will remap once after remap_frequency number of solutions has been found.
buffer_capacity (int) – Number of solutions to keep in the buffer. Solutions in the buffer will be reinserted into the archive after remapping.

Methods

`__iter__`()	Creates an iterator over the `Elite`'s in the archive.
`__len__`()	Number of elites in the archive.
`add`(solution_batch, objective_batch, ...[, ...])	Inserts a batch of solutions into the archive.
`add_single`(solution, objective, measures[, ...])	Inserts a single solution into the archive.
`as_pandas`([include_solutions, include_metadata])	Converts the archive into an `ArchiveDataFrame` (a child class of `pandas.DataFrame`).
`clear`()	Removes all elites from the archive.
`cqd_score`(iterations, target_points, ...[, ...])	Computes the CQD score of the archive.
`grid_to_int_index`(grid_index_batch)	Converts a batch of grid indices into a batch of integer indices.
`index_of`(measures_batch)	Returns archive indices for the given batch of measures.
`index_of_single`(measures)	Returns the index of the measures for one solution.
`int_to_grid_index`(int_index_batch)	Converts a batch of indices into indices in the archive's grid.
`retrieve`(measures_batch)	Retrieves the elites with measures in the same cells as the measures specified.
`retrieve_single`(measures)	Retrieves the elite with measures in the same cell as the measures specified.
`sample_elites`(n)	Randomly samples elites from the archive.

Attributes

`best_elite`	The elite with the highest objective in the archive.
`boundaries`	The dynamic boundaries of the cells in each dimension.
`buffer_capacity`	Maximum capacity of the buffer.
`cells`	Total number of cells in the archive.
`dims`	Number of cells in each dimension.
`dtype`	The dtype of the solutions, objective, and measures.
`empty`	Whether the archive is empty.
`epsilon`	Epsilon for computing archive indices.
`interval_size`	The size of each dim (upper_bounds - lower_bounds).
`learning_rate`	The learning rate for threshold updates.
`lower_bounds`	Lower bound of each dimension.
`measure_dim`	Dimensionality of the measure space.
`qd_score_offset`	The offset which is subtracted from objective values when computing the QD score.
`remap_frequency`	Frequency of remapping.
`solution_dim`	Dimensionality of the solutions in the archive.
`stats`	Statistics about the archive.
`threshold_min`	The initial threshold value for all the cells.
`upper_bounds`	Upper bound of each dimension.

__iter__()¶

Creates an iterator over the Elite’s in the archive.

Example

for elite in archive:
    elite.sol
    elite.obj
    ...

__len__()¶: Number of elites in the archive.

add(solution_batch, objective_batch, measures_batch, metadata_batch=None)[source]¶

Inserts a batch of solutions into the archive.

Note

Unlike in other archives, this method (currently) is not truly batched; rather, it is implemented by calling add_single() on the solutions in the batch, in the order that they are passed in. As such, this method is not invariant to the ordering of the solutions in the batch.

See ArchiveBase.add() for arguments and return values.

add_single(solution, objective, measures, metadata=None)[source]¶

Inserts a single solution into the archive.

This method remaps the archive after every remap_frequency solutions are added. Remapping involves changing the boundaries of the archive to the percentage marks of the measures stored in the buffer and re-adding all of the solutions stored in the buffer and the current archive.

See ArchiveBase.add_single() for arguments and return values.

as_pandas(include_solutions=True, include_metadata=False)¶

Converts the archive into an ArchiveDataFrame (a child class of pandas.DataFrame).

The implementation of this method in ArchiveBase creates a dataframe consisting of:

1 column of integers (np.int32) for the index, named index. See index_of() for more info.
measure_dim columns for the measures, named measure_0, measure_1, ...
1 column for the objectives, named objective
solution_dim columns for the solution parameters, named solution_0, solution_1, ...
1 column for the metadata objects, named metadata

In short, the dataframe looks like this:

index	measure_0	…	objective	solution_0	…	metadata
		…			…

Compared to pandas.DataFrame, the ArchiveDataFrame adds methods and attributes which make it easier to manipulate archive data. For more information, refer to the ArchiveDataFrame documentation.

Parameters

include_solutions (bool) – Whether to include solution columns.
include_metadata (bool) – Whether to include the metadata column. Note that methods like to_csv() may not properly save the dataframe since the metadata objects may not be representable in a CSV.

Returns

See above.

Return type

ArchiveDataFrame

clear()¶

Removes all elites from the archive.

After this method is called, the archive will be empty.

cqd_score(iterations, target_points, penalties, obj_min, obj_max, dist_max=None, dist_ord=None)¶

Computes the CQD score of the archive.

The Continuous Quality Diversity (CQD) score was introduced in Kent 2022.

Note

This method by default assumes that the archive has an upper_bounds and lower_bounds property which delineate the bounds of the measure space, as is the case in GridArchive, CVTArchive, and SlidingBoundariesArchive. If this is not the case, dist_max must be passed in, and target_points must be an array of custom points.

Parameters

iterations (int) – Number of times to compute the CQD score. We return the mean CQD score across these iterations.
target_points (int or array-like) – Number of target points to generate, or an (iterations, n, measure_dim) array which lists n target points to list on each iteration. When an int is passed, the points are sampled uniformly within the bounds of the measure space.
penalties (int or array-like) – Number of penalty values over which to compute the score (the values are distributed evenly over the range [0,1]). Alternatively, this may be a 1D array which explicitly lists the penalty values. Known as \(\theta\) in Kent 2022.
obj_min (float) – Minimum objective value, used when normalizing the objectives.
obj_max (float) – Maximum objective value, used when normalizing the objectives.
dist_max (float) – Maximum distance between points in measure space. Defaults to the distance between the extremes of the measure space bounds (the type of distance is computed with the order specified by dist_ord). Known as \(\delta_{max}\) in Kent 2022.
dist_ord – Order of the norm to use for calculating measure space distance; this is passed to numpy.linalg.norm() as the ord argument. See numpy.linalg.norm() for possible values. The default is to use Euclidean distance (L2 norm).

Returns

The mean CQD score obtained with iterations rounds of calculations.

Raises

RuntimeError – The archive does not have the bounds properties mentioned above, and dist_max is not specified or the target points are not provided.
ValueError – target_points or penalties is an array with the wrong shape.

grid_to_int_index(grid_index_batch)¶

Converts a batch of grid indices into a batch of integer indices.

Refer to index_of() for more info.

Parameters: grid_index_batch (array-like) – (batch_size, measure_dim) array of indices in the archive grid.
Returns: (batch_size,) array of integer indices.
Return type: numpy.ndarray
Raises: ValueError – grid_index_batch is not of shape (batch_size, measure_dim)

index_of(measures_batch)[source]¶

Returns archive indices for the given batch of measures.

First, values are clipped to the bounds of the measure space. Then, the values are mapped to cells via a binary search along the boundaries in each dimension.

At this point, we have “grid indices” – indices of each measure in each dimension. Since indices returned by this method must be single integers (as opposed to a tuple of grid indices), we convert these grid indices into integer indices with numpy.ravel_multi_index() and return the result.

It may be useful to have the original grid indices. Thus, we provide the grid_to_int_index() and int_to_grid_index() methods for converting between grid and integer indices.

As an example, the grid indices can be used to access boundaries of a measure value’s cell. For example, the following retrieves the lower and upper bounds of the cell along dimension 0:

# Access only element 0 since this method operates in batch.
idx = archive.int_to_grid_index(archive.index_of(...))[0]

lower = archive.boundaries[0][idx[0]]
upper = archive.boundaries[0][idx[0] + 1]

See boundaries for more info.

Parameters: measures_batch (array-like) – (batch_size, measure_dim) array of coordinates in measure space.
Returns: (batch_size,) array of integer indices representing the flattened grid coordinates.
Return type: numpy.ndarray
Raises: ValueError – measures_batch is not of shape (batch_size, measure_dim).

index_of_single(measures)¶

Returns the index of the measures for one solution.

While index_of() takes in a batch of measures, this method takes in the measures for only one solution. If index_of() is implemented correctly, this method should work immediately (i.e. “out of the box”).

Parameters

measures (array-like) – (measure_dim,) array of measures for a single solution.

Returns

Integer index of the measures in the archive’s storage arrays.

Return type

int or numpy.integer

Raises

ValueError – measures is not of shape (measure_dim,).
ValueError – measures has non-finite values (inf or NaN).

int_to_grid_index(int_index_batch)¶

Converts a batch of indices into indices in the archive’s grid.

Refer to index_of() for more info.

Parameters: int_index_batch (array-like) – (batch_size,) array of integer indices such as those output by index_of().
Returns: (batch_size, measure_dim) array of indices in the archive grid.
Return type: numpy.ndarray
Raises: ValueError – int_index_batch is not of shape (batch_size,).

retrieve(measures_batch)¶

Retrieves the elites with measures in the same cells as the measures specified.

This method operates in batch, i.e. it takes in a batch of measures and outputs an EliteBatch. Since EliteBatch is a namedtuple, it can be unpacked:

solution_batch, objective_batch, measures_batch, \
    index_batch, metadata_batch = archive.retrieve(...)

Or the fields may be accessed by name:

elite_batch = archive.retrieve(...)
elite_batch.solution_batch
elite_batch.objective_batch
elite_batch.measures_batch
elite_batch.index_batch
elite_batch.metadata_batch

If the cell associated with measures_batch[i] has an elite in it, then elite_batch.solution_batch[i], elite_batch.objective_batch[i], elite_batch.measures_batch[i], elite_batch.index_batch[i], and elite_batch.metadata_batch[i] will be set to the properties of the elite. Note that elite_batch.measures_batch[i] may not be equal to measures_batch[i] since the measures only need to be in the same archive cell.

If the cell associated with measures_batch[i] does not have any elite in it, then the corresponding outputs are set to empty values – namely:

elite_batch.solution_batch[i] will be an array of NaN
elite_batch.objective_batch[i] will be NaN
elite_batch.measures_batch[i] will be an array of NaN
elite_batch.index_batch[i] will be -1
elite_batch.metadata_batch[i] will be None

If you need to retrieve a single elite associated with some measures, consider using retrieve_single().

Parameters

measures_batch (array-like) – (batch_size, measure_dim) array of coordinates in measure space.

Returns

See above.

Return type

EliteBatch

Raises

ValueError – measures_batch is not of shape (batch_size, measure_dim).
ValueError – measures_batch has non-finite values (inf or NaN).

retrieve_single(measures)¶

Retrieves the elite with measures in the same cell as the measures specified.

While retrieve() takes in a batch of measures, this method takes in the measures for only one solution and returns a single Elite.

Parameters

measures (array-like) – (measure_dim,) array of measures.

Returns

If there is an elite with measures in the same cell as the measures specified, then this method returns an Elite where all the fields hold the info of that elite. Otherwise, this method returns an Elite filled with the same “empty” values described in retrieve().

Raises

ValueError – measures is not of shape (measure_dim,).
ValueError – measures has non-finite values (inf or NaN).

sample_elites(n)¶

Randomly samples elites from the archive.

Currently, this sampling is done uniformly at random. Furthermore, each sample is done independently, so elites may be repeated in the sample. Additional sampling methods may be supported in the future.

Since EliteBatch is a namedtuple, the result can be unpacked (here we show how to ignore some of the fields):

solution_batch, objective_batch, measures_batch, *_ = \
    archive.sample_elites(32)

Or the fields may be accessed by name:

elite = archive.sample_elites(16)
elite.solution_batch
elite.objective_batch
...

Parameters: n (int) – Number of elites to sample.
Returns: A batch of elites randomly selected from the archive.
Return type: EliteBatch
Raises: IndexError – The archive is empty.

property best_elite¶

The elite with the highest objective in the archive.

None if there are no elites in the archive.

Note

If the archive is non-elitist (this occurs when using the archive with a learning rate which is not 1.0, as in CMA-MAE), then this best elite may no longer exist in the archive because it was replaced with an elite with a lower objective value. This can happen because in non-elitist archives, new solutions only need to exceed the threshold of the cell they are being inserted into, not the objective of the elite currently in the cell. See #314 for more info.

Type: Elite

property boundaries¶

The dynamic boundaries of the cells in each dimension.

Entry i in this list is an array that contains the boundaries of the cells in dimension i. The array contains self.dims[i] + 1 entries laid out like this:

Archive cells:  | 0 | 1 |   ...   |    self.dims[i]    |
boundaries[i]:  0   1   2   self.dims[i] - 1     self.dims[i]

Thus, boundaries[i][j] and boundaries[i][j + 1] are the lower and upper bounds of cell j in dimension i. To access the lower bounds of all the cells in dimension i, use boundaries[i][:-1], and to access all the upper bounds, use boundaries[i][1:].

Type: list of numpy.ndarray

property buffer_capacity¶

Maximum capacity of the buffer.

Type: int

property cells¶

Total number of cells in the archive.

Type: int

property dims¶

Number of cells in each dimension.

Type: (measure_dim,) numpy.ndarray

property dtype¶

The dtype of the solutions, objective, and measures.

Type: data-type

property empty¶

Whether the archive is empty.

Type: bool

property epsilon¶

Epsilon for computing archive indices. Refer to the documentation for this class.

Type: dtype

property interval_size¶

The size of each dim (upper_bounds - lower_bounds).

Type: (measure_dim,) numpy.ndarray

property learning_rate¶

The learning rate for threshold updates.

Type: float

property lower_bounds¶

Lower bound of each dimension.

Type: (measure_dim,) numpy.ndarray

property measure_dim¶

Dimensionality of the measure space.

Type: int

property qd_score_offset¶

The offset which is subtracted from objective values when computing the QD score.

Type: float

property remap_frequency¶

Frequency of remapping. Archive will remap once after remap_frequency number of solutions has been found.

Type: int

property solution_dim¶

Dimensionality of the solutions in the archive.

Type: int

property stats¶

Statistics about the archive.

See ArchiveStats for more info.

Type: ArchiveStats

property threshold_min¶

The initial threshold value for all the cells.

Type: float

property upper_bounds¶

Upper bound of each dimension.

Type: (measure_dim,) numpy.ndarray