{
"cells": [
{
"cell_type": "markdown",
"metadata": {
"id": "Ne0mog-NFKhE"
},
"source": [
"# Using CMA-ME to Land a Lunar Lander Like a Space Shuttle\n",
"\n",
"_This tutorial is part of the series of pyribs tutorials! See [here](https://docs.pyribs.org/en/latest/tutorials.html) for the list of all tutorials and the order in which they should be read._\n",
"\n",
"In the [Lunar Lander](https://gymnasium.farama.org/environments/box2d/lunar_lander/) environment, an agent controls a spaceship to touch down gently within a goal zone near the bottom of the screen. Typically, agents in Lunar Lander take a direct approach, hovering straight down:"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 262
},
"id": "Powj48a5GIjH",
"outputId": "fed99a40-6434-40df-dedc-343f98af9fcc"
},
"outputs": [
{
"data": {
"text/html": [
""
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"from IPython.display import display, HTML\n",
"display(HTML(\"\"\"\"\"\"))"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "TRSA2NXJGb8u"
},
"source": [
"Of course, this works fine, and the lander safely lands on the landing pad. However, there are many (and more theatric) ways we can safely achieve our goal. For instance, a different solution is to land like a space shuttle:"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 262
},
"id": "Q9-J8MePGY9L",
"outputId": "bbf787d9-8654-4621-8f43-c6ff882ab523"
},
"outputs": [
{
"data": {
"text/html": [
""
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"display(HTML(\"\"\"\"\"\"))"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "k1L8dvGuGkSJ"
},
"source": [
"And we can also approach from the right:"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 262
},
"id": "TFTEeCHNGkrt",
"outputId": "8d2c02d6-2245-4c36-b918-12a35add4b2a"
},
"outputs": [
{
"data": {
"text/html": [
""
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"display(HTML(\"\"\"\"\"\"))"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "q3GeMrGjGk41"
},
"source": [
"The primary difference between these trajectories is their \"point of impact,\" that is, the $x$-position of the lander when one of its legs hits the ground for the first time. In the vertical trajectory, the lander first impacts the ground at $x \\approx -0.1$, and when approaching from the left and right, it first impacts at $x \\approx -0.5$ and $x \\approx 0.6$, respectively.\n",
"\n",
"Though these trajectories look different, they all achieve good performance (200+), leading to an important insight: there are characteristics of landing a lunar lander that are not necessarily important for performance, but nonetheless determine the behavior of the lander. In quality diversity (QD) terms, we call these measures. In this tutorial, we will search for policies that yield different trajectories using the pyribs implementation of the QD algorithm [CMA-ME](https://arxiv.org/abs/1912.02400).\n",
"\n",
"**_By the way: Recent work introduced [CMA-MAE](https://arxiv.org/abs/2205.10752), an algorithm which builds on CMA-ME and achieves higher performance in a variety of domains. Once you finish this tutorial, be sure to check out the [next tutorial](https://docs.pyribs.org/en/latest/tutorials/cma_mae.html) to learn about CMA-MAE._**"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "Pry368FHFKhO"
},
"source": [
"## Setup\n",
"\n",
"First, let's install pyribs and Gymnasium. [Gymnasium](https://gymnasium.farama.org) is the successor to [OpenAI Gym](https://www.gymlibrary.dev), which was deprecated in late 2022. We use the visualize extra of pyribs (`ribs[visualize]` instead of just `ribs`) so that we obtain access to the [`ribs.visualize`](https://docs.pyribs.org/en/latest/api/ribs.visualize.html) module."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "-83S1zt1FKhQ",
"outputId": "fa3991ec-80d5-4b91-a9aa-adb038f1c890"
},
"outputs": [],
"source": [
"%pip install swig # Must be installed before box2d.\n",
"%pip install ribs[visualize] gymnasium[box2d]==0.29.1 \"moviepy>=1.0.0\" dask distributed\n",
"\n",
"# An uninstalled version of decorator is occasionally loaded. This loads the\n",
"# newly installed version of decorator so that moviepy works properly -- see\n",
"# https://github.com/Zulko/moviepy/issues/1625\n",
"import importlib\n",
"import decorator\n",
"importlib.reload(decorator)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "oNMaEGunFKhU"
},
"source": [
"Now, we import Gymnasium and several utilities."
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {
"id": "WhemB036FKhV"
},
"outputs": [],
"source": [
"import sys\n",
"import time\n",
"\n",
"import gymnasium as gym\n",
"import matplotlib.pyplot as plt\n",
"import numpy as np\n",
"from dask.distributed import Client\n",
"from tqdm import tqdm, trange"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "Jva_esiqFKhW"
},
"source": [
"## Problem Description\n",
"\n",
"We treat the Lunar Lander as a quality diversity (QD) problem. For the objective, we use the default rewards provided by the environment, which encourages landing on the landing pad and penalizes engine usage.\n",
"\n",
"For the measure functions, we are interested in several factors at the time of \"impact.\" We define impact to be the first time when either of the lunar lander's legs touches the ground. When this happens, we measure the following:\n",
"\n",
"- $x$-position: This will lead to markedly different trajectories, as seen earlier.\n",
"- $y$-velocity: Different velocities will determine how hard the lander impacts the ground.\n",
"\n",
"If the lunar lander never impacts the ground, we default the $x$-position to be the last $x$-position of the lander, and the $y$-velocity to be the maximum velocity of the lander (technically minimum since velocities are negative).\n",
"\n",
"We will search for policies that produce high-performing trajectories with these measures. For simplicity, we will use a linear policy to control the lunar lander. As the default lunar lander has discrete controls, the equation for this policy is:\n",
"\n",
"$$a = argmax(Ws)$$\n",
"\n",
"where $a$ is the action to take, $s$ is the state vector, and $W$ is our model, a matrix of weights that stays constant. Essentially, we transform the state to a vector with a \"signal\" for each possible action in the action space, and we choose the action with the highest signal. To search for a different policy, we explore the space of models $W$.\n",
"\n",
"In this tutorial, we will search for policies solving a fixed scenario with a flat terrain. To create this scenario, we use a fixed seed of 52 in the environment.\n",
"\n",
"**Note: Determinism**\n",
"\n",
"> Since our policy and environment are both deterministic, we only have to simulate the policy once to find its performance. Typically, we would run our policy multiple times to gauge average performance and have it generalize, but we ignore that to keep this example simple."
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {
"id": "iEnaEBMgFKhX"
},
"outputs": [],
"source": [
"# Create an environment so that we can obtain information about it.\n",
"reference_env = gym.make(\"LunarLander-v2\")\n",
"action_dim = reference_env.action_space.n\n",
"obs_dim = reference_env.observation_space.shape[0]\n",
"\n",
"# Seed for all environments in this notebook.\n",
"env_seed = 52"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "NI2KJcyAFKhY"
},
"source": [
"We can summarize our problem description with the following `simulate` function, which takes in the model and rolls it out in the Lunar Lander environment."
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {
"id": "UlhzwjDlFKhZ"
},
"outputs": [],
"source": [
"def simulate(model, seed=None, video_env=None):\n",
" \"\"\"Simulates the lunar lander model.\n",
"\n",
" Args:\n",
" model (np.ndarray): The array of weights for the linear policy.\n",
" seed (int): The seed for the environment.\n",
" video_env (gym.Env): If passed in, this will be used instead of creating\n",
" a new env. This is used primarily for recording video during\n",
" evaluation.\n",
" Returns:\n",
" total_reward (float): The reward accrued by the lander throughout its\n",
" trajectory.\n",
" impact_x_pos (float): The x position of the lander when it touches the\n",
" ground for the first time.\n",
" impact_y_vel (float): The y velocity of the lander when it touches the\n",
" ground for the first time.\n",
" \"\"\"\n",
" if video_env is None:\n",
" # Since we are using multiple processes, it is simpler if each worker\n",
" # just creates their own copy of the environment instead of trying to\n",
" # share the environment. This also makes the function \"pure.\" However,\n",
" # we should use the video_env if it is passed in.\n",
" env = gym.make(\"LunarLander-v2\")\n",
" else:\n",
" env = video_env\n",
"\n",
" action_dim = env.action_space.n\n",
" obs_dim = env.observation_space.shape[0]\n",
" model = model.reshape((action_dim, obs_dim))\n",
"\n",
" total_reward = 0.0\n",
" impact_x_pos = None\n",
" impact_y_vel = None\n",
" all_y_vels = []\n",
" obs, _ = env.reset(seed=seed)\n",
" done = False\n",
"\n",
" while not done:\n",
" action = np.argmax(model @ obs) # Linear policy.\n",
" obs, reward, terminated, truncated, _ = env.step(action)\n",
" done = terminated or truncated\n",
" total_reward += reward\n",
"\n",
" # Refer to the definition of state here:\n",
" # https://gymnasium.farama.org/environments/box2d/lunar_lander/\n",
" x_pos = obs[0]\n",
" y_vel = obs[3]\n",
" leg0_touch = bool(obs[6])\n",
" leg1_touch = bool(obs[7])\n",
" all_y_vels.append(y_vel)\n",
"\n",
" # Check if the lunar lander is impacting for the first time.\n",
" if impact_x_pos is None and (leg0_touch or leg1_touch):\n",
" impact_x_pos = x_pos\n",
" impact_y_vel = y_vel\n",
"\n",
" # If the lunar lander did not land, set the x-pos to the one from the final\n",
" # timestep, and set the y-vel to the max y-vel (we use min since the lander\n",
" # goes down).\n",
" if impact_x_pos is None:\n",
" impact_x_pos = x_pos\n",
" impact_y_vel = min(all_y_vels)\n",
"\n",
" # Only close the env if it was not a video env.\n",
" if video_env is None:\n",
" env.close()\n",
"\n",
" return total_reward, impact_x_pos, impact_y_vel"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "i3PYgVwuFKha"
},
"source": [
"## CMA-ME with pyribs\n",
"\n",
"To train our policy, we will use the CMA-ME algorithm (if you are not familiar with CMA-ME, please refer to the corresponding [paper](https://arxiv.org/abs/1912.02400)). This means we need to import and initialize the `GridArchive`, `EvolutionStrategyEmitter`, and `Scheduler` from pyribs."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "hd8Fa-uoNg_6"
},
"source": [
"### GridArchive\n",
"\n",
"First, the [`GridArchive`](https://docs.pyribs.org/en/latest/api/ribs.archives.GridArchive.html) stores solutions (i.e. models for our policy) in a rectangular grid. Each dimension of the `GridArchive` corresponds to a dimension in measure space that is segmented into equally sized cells. As we have two measure functions for our lunar lander, we have two dimensions in the `GridArchive`. The first dimension is the impact $x$-position, which ranges from -1 to 1, and the second is the impact $y$-velocity, which ranges from -3 (smashing into the ground) to 0 (gently touching down). We divide both dimensions into 50 cells.\n",
"\n",
"We additionally specify the dimensionality of solutions which will be stored in the archive. While each model is a 2D matrix, pyribs archives only allow 1D arrays for efficiency; hence, we create an `initial_model` below and retrieve the size of its flattened, 1D form with `initial_model.size`."
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {
"id": "EUipJ5cDFKhc"
},
"outputs": [],
"source": [
"from ribs.archives import GridArchive\n",
"initial_model = np.zeros((action_dim, obs_dim))\n",
"\n",
"archive = GridArchive(\n",
" solution_dim=initial_model.size, # Dimensionality of solutions in the archive.\n",
" dims=[50, 50], # 50 cells along each dimension.\n",
" ranges=[(-1.0, 1.0), (-3.0, 0.0)], # (-1, 1) for x-pos and (-3, 0) for y-vel.\n",
" qd_score_offset=-600, # See the note below.\n",
")"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "DlKpfRuZzbiZ"
},
"source": [
"**Note: QD Score Offset**\n",
"\n",
"> Above, we specified `qd_score_offset=-600` when initializing the archive. The QD score ([Pugh 2016](https://doi.org/10.3389/frobt.2016.00040)) is a metric for QD algorithms which sums the objective values of all elites in the archive. However, if objectives can be negative, this metric will penalize an algorithm for discovering new cells with a negative objective. To prevent this, it is common to normalize each objective to be non-negative by subtracting an offset, typically the minimum objective, before computing the QD score. While lunar lander does not have a predefined minimum objective, we know from previous experiments that almost all solutions score above -600, so we have set the offset accordingly. Thus, if a solution has, for example, an objective of -300, then its objective will be normalized to -300 - (-600) = 300.\n",
"\n",
"**Note: Keyword Arguments**\n",
"\n",
"> In pyribs, most constructor arguments must be keyword arguments. For example, the `GridArchive` constructor looks like `GridArchive(*, solution_dim, ...)`. Notice the `*` in the parameter list. This `*` syntax indicates that any parameters listed after it must be passed as _keyword arguments_, e.g., `GridArchive(solution_dim=initial_model.size, ...)` as is done above. Furthermore, passing a _positional argument_ like `GridArchive(10, ...)` would throw a `TypeError`. We adopt this practice from [scikit-learn](https://scikit-learn-enhancement-proposals.readthedocs.io/en/latest/slep009/proposal.html) in an effort to increase code readability."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "XdCmEwtOFKhd"
},
"source": [
"### EvolutionStrategyEmitter\n",
"\n",
"Next, the [`EvolutionStrategyEmitter`](https://docs.pyribs.org/en/latest/api/ribs.emitters.EvolutionStrategyEmitter.html) with two-stage improvement ranking (\"2imp\") uses CMA-ES to search for policies that add new entries to the archive or improve existing ones. Since we do not have any prior knowledge of what the model will be, we set the initial model to be the zero vector, and we set the initial step size for CMA-ES to be 1.0, so that initial solutions are sampled from a standard isotropic Gaussian. Furthermore, we use 5 emitters so that the algorithm simultaneously searches several areas of the measure space."
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {
"id": "IuyDI0HaFKhe"
},
"outputs": [],
"source": [
"from ribs.emitters import EvolutionStrategyEmitter\n",
"\n",
"emitters = [\n",
" EvolutionStrategyEmitter(\n",
" archive=archive,\n",
" x0=initial_model.flatten(),\n",
" sigma0=1.0, # Initial step size.\n",
" ranker=\"2imp\",\n",
" batch_size=30, # If we do not specify a batch size, the emitter will\n",
" # automatically use a batch size equal to the default\n",
" # population size of CMA-ES.\n",
" ) for _ in range(5) # Create 5 separate emitters.\n",
"]"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "TwS_5rnPRCbr"
},
"source": [
"**Note: Two-Stage Improvement Ranking**\n",
"\n",
"> The term \"two-stage\" refers to how this ranking mechanism ranks solutions by a tuple of `(status, value)` defined as follows:\n",
">\n",
"> 1. `status`: Whether the solution creates a new cell in the archive, improves an existing cell, or is not inserted.\n",
"> 2. `value`: Consider the objective $f$ of the solution and the objective $f'$ of the solution currently in the cell where the solution will be inserted. When the solution creates a new cell, $f'$ is undefined because the cell was previously empty, so the value is defined as $f$. Otherwise, when the solution improves an existing cell or is not inserted at all, the value is $f - f'$.\n",
">\n",
"> During ranking, this two-stage improvement ranker will first sort by `status`, prioritizing new solutions, followed by solutions which improve existing cells, followed by solutions which are not inserted. Within each group, solutions are further ranked by their corresponding `value`. See the archive [`add`](https://docs.pyribs.org/en/latest/api/ribs.archives.GridArchive.html#ribs.archives.GridArchive.add) method for more information on statuses and values.\n",
">\n",
"> Additional rankers are available in the [ribs.emitters.rankers](https://docs.pyribs.org/en/latest/api/ribs.emitters.rankers.html) module. These rankers include those corresponding to different emitters described in the CMA-ME paper."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "CYJ5zor9FKhg"
},
"source": [
"### Scheduler\n",
"\n",
"Finally, the [`Scheduler`](https://docs.pyribs.org/en/latest/api/ribs.schedulers.Scheduler.html) controls how the emitters interact with the archive. On every iteration, the scheduler calls the emitters to generate solutions. After the user evaluates these generated solutions, the scheduler inserts the solutions into the archive and passes the feedback to the emitters (this feedback consists of the `status` and `value` for each solution that we described in the note above)."
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {
"id": "ytg5qUP2FKhj"
},
"outputs": [],
"source": [
"from ribs.schedulers import Scheduler\n",
"\n",
"scheduler = Scheduler(archive, emitters)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "S5NOJy6YFKhk"
},
"source": [
"## QD Search\n",
"\n",
"With the pyribs components defined, we start searching with CMA-ME. Since we use 5 emitters each with a batch size of 30 and we run 300 iterations, we run 5 x 30 x 300 = 45,000 lunar lander simulations. We also keep track of some logging info via `archive.stats`, which is an [`ArchiveStats`](https://docs.pyribs.org/en/latest/api/ribs.archives.ArchiveStats.html) object.\n",
"\n",
"Since it takes a relatively long time to evaluate a lunar lander solution, we parallelize the evaluation of multiple solutions with [Dask](https://distributed.dask.org/en/stable/quickstart.html). On Colab, with one worker (i.e., one CPU), the loop should take **~30 minutes** to run. With two workers, it should take **~20 minutes** to run. Feel free to increase the number of workers based on the number of CPUs your system has available to speed up the loop further."
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "pd2abvstFKhm",
"outputId": "88e53ee9-393f-4625-bf4b-74ebf49c4403"
},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"INFO:distributed.http.proxy:To route to workers diagnostics web server please install jupyter-server-proxy: python -m pip install jupyter-server-proxy\n",
"INFO:distributed.scheduler:State start\n",
"INFO:distributed.scheduler: Scheduler at: tcp://127.0.0.1:33631\n",
"INFO:distributed.scheduler: dashboard at: http://127.0.0.1:8787/status\n",
"INFO:distributed.nanny: Start Nanny at: 'tcp://127.0.0.1:41455'\n",
"INFO:distributed.nanny: Start Nanny at: 'tcp://127.0.0.1:38413'\n",
"INFO:distributed.scheduler:Register worker \n",
"INFO:distributed.scheduler:Starting worker compute stream, tcp://127.0.0.1:44505\n",
"INFO:distributed.core:Starting established connection to tcp://127.0.0.1:56380\n",
"INFO:distributed.scheduler:Register worker \n",
"INFO:distributed.scheduler:Starting worker compute stream, tcp://127.0.0.1:32931\n",
"INFO:distributed.core:Starting established connection to tcp://127.0.0.1:56384\n",
"INFO:distributed.scheduler:Receive client connection: Client-bdc1a522-4ff2-11ee-80a7-0242ac1c000c\n",
"INFO:distributed.core:Starting established connection to tcp://127.0.0.1:56394\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"> 25 itrs completed after 83.08s\n",
" - Size: 1091\n",
" - Coverage: 0.4364\n",
" - QD Score: 476152.747566642\n",
" - Max Obj: 312.4166112115108\n",
" - Mean Obj: -163.5630178124271\n",
"> 50 itrs completed after 174.29s\n",
" - Size: 1605\n",
" - Coverage: 0.642\n",
" - QD Score: 726086.9622784023\n",
" - Max Obj: 312.4166112115108\n",
" - Mean Obj: -147.6093692969456\n",
"> 75 itrs completed after 255.55s\n",
" - Size: 2046\n",
" - Coverage: 0.8184\n",
" - QD Score: 869710.060649476\n",
" - Max Obj: 312.4166112115108\n",
" - Mean Obj: -174.92176898852586\n",
"> 100 itrs completed after 332.60s\n",
" - Size: 2253\n",
" - Coverage: 0.9012\n",
" - QD Score: 997512.4561767571\n",
" - Max Obj: 312.4166112115108\n",
" - Mean Obj: -157.2514619721451\n",
"> 125 itrs completed after 423.54s\n",
" - Size: 2360\n",
" - Coverage: 0.944\n",
" - QD Score: 1096255.1246338706\n",
" - Max Obj: 312.4166112115108\n",
" - Mean Obj: -135.48511668056332\n",
"> 150 itrs completed after 518.69s\n",
" - Size: 2416\n",
" - Coverage: 0.9664\n",
" - QD Score: 1171209.568852896\n",
" - Max Obj: 312.4166112115108\n",
" - Mean Obj: -115.22782746154965\n",
"> 175 itrs completed after 626.75s\n",
" - Size: 2456\n",
" - Coverage: 0.9824\n",
" - QD Score: 1236459.7385883222\n",
" - Max Obj: 312.4166112115108\n",
" - Mean Obj: -96.55548103081347\n",
"> 200 itrs completed after 763.93s\n",
" - Size: 2479\n",
" - Coverage: 0.9916\n",
" - QD Score: 1301028.6608563724\n",
" - Max Obj: 312.4166112115108\n",
" - Mean Obj: -75.18004806116478\n",
"> 225 itrs completed after 879.66s\n",
" - Size: 2483\n",
" - Coverage: 0.9932\n",
" - QD Score: 1324682.1782379313\n",
" - Max Obj: 312.4166112115108\n",
" - Mean Obj: -66.49932410876713\n",
"> 250 itrs completed after 979.00s\n",
" - Size: 2486\n",
" - Coverage: 0.9944\n",
" - QD Score: 1341868.5549527393\n",
" - Max Obj: 312.4166112115108\n",
" - Mean Obj: -60.22986526438489\n",
"> 275 itrs completed after 1072.97s\n",
" - Size: 2490\n",
" - Coverage: 0.996\n",
" - QD Score: 1365455.2747249145\n",
" - Max Obj: 312.4166112115108\n",
" - Mean Obj: -51.62438766067689\n",
"> 300 itrs completed after 1160.53s\n",
" - Size: 2493\n",
" - Coverage: 0.9972\n",
" - QD Score: 1387979.806127139\n",
" - Max Obj: 312.4166112115108\n",
" - Mean Obj: -43.24917523981597\n",
"Iterations: 100%|██████████| 300/300 [19:17<00:00, 3.86s/it]\n"
]
}
],
"source": [
"start_time = time.time()\n",
"total_itrs = 300\n",
"workers = 2 # Adjust the number of workers based on your available CPUs.\n",
"\n",
"client = Client(\n",
" n_workers=workers, # Create this many worker processes using Dask LocalCluster.\n",
" threads_per_worker=1, # Each worker process is single-threaded.\n",
")\n",
"\n",
"for itr in trange(1, total_itrs + 1, file=sys.stdout, desc='Iterations'):\n",
" # Request models from the scheduler.\n",
" sols = scheduler.ask()\n",
"\n",
" # Evaluate the models and record the objectives and measuress.\n",
" futures = client.map(lambda model: simulate(model, env_seed), sols)\n",
" results = client.gather(futures)\n",
"\n",
" objs, meas = [], []\n",
" for obj, impact_x_pos, impact_y_vel in results:\n",
" objs.append(obj)\n",
" meas.append([impact_x_pos, impact_y_vel])\n",
"\n",
" # Send the results back to the scheduler.\n",
" scheduler.tell(objs, meas)\n",
"\n",
" # Logging.\n",
" if itr % 25 == 0:\n",
" tqdm.write(f\"> {itr} itrs completed after {time.time() - start_time:.2f}s\")\n",
" tqdm.write(f\" - Size: {archive.stats.num_elites}\") # Number of elites in the archive. len(archive) also provides this info.\n",
" tqdm.write(f\" - Coverage: {archive.stats.coverage}\") # Proportion of archive cells which have an elite.\n",
" tqdm.write(f\" - QD Score: {archive.stats.qd_score}\") # QD score, i.e. sum of objective values of all elites in the archive.\n",
" # Accounts for qd_score_offset as described in the GridArchive section.\n",
" tqdm.write(f\" - Max Obj: {archive.stats.obj_max}\") # Maximum objective value in the archive.\n",
" tqdm.write(f\" - Mean Obj: {archive.stats.obj_mean}\") # Mean objective value of elites in the archive."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "pZIzzxxWFKhp"
},
"source": [
"## Visualizing the Archive\n",
"\n",
"Using [`grid_archive_heatmap`](https://docs.pyribs.org/en/latest/api/ribs.visualize.grid_archive_heatmap.html) from the [`ribs.visualize`](https://docs.pyribs.org/en/latest/api/ribs.visualize.html) module, we can view a heatmap of the archive. The heatmap shows the measures for which CMA-ME found a solution. The color of each cell shows the objective value of the solution."
]
},
{
"cell_type": "code",
"execution_count": 12,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 564
},
"id": "ufCTDxiAFKhp",
"outputId": "b3d13c53-3a34-4385-d02e-dcf5b6041ffc"
},
"outputs": [
{
"data": {
"text/plain": [
"Text(0.5, 0, 'Impact x-position')"
]
},
"execution_count": 12,
"metadata": {},
"output_type": "execute_result"
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAqgAAAISCAYAAAAA6T1PAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAAB+yklEQVR4nO3deXxTVdoH8N9N2iTd0oWuQAsUkEXKIgoWFUEYqaLCyKC4AFWEkQEVQYQqi8AwLCKKiKC+bDoy7guiooiAWwFFEIHSAQTK1rJ239LkvH9gM1ageRJuaEp/Xz/5SNOn5567pafn3vs8mlJKgYiIiIjIRxhqugNERERERH/EASoRERER+RQOUImIiIjIp3CASkREREQ+hQNUIiIiIvIpHKASERERkU/hAJWIiIiIfAoHqERERETkUzhAJSIiIiKfwgEqEREREfmUWjNAveOOO5CQkACLxYK4uDgMHDgQR48erfZnSktLMWLECNSrVw/BwcHo168fcnJyLlGPiYiIiGrWwoUL0bZtW1itVlitViQnJ+Pzzz93fl8yVsrKykLv3r0RGBiI6OhojB07FhUVFV7td60ZoHbv3h3vvPMOMjMz8f7772Pfvn3429/+Vu3PPP744/jkk0/w7rvvYsOGDTh69CjuvPPOS9RjIiIioprVsGFDzJw5E1u2bMFPP/2Em266CX369MHOnTsBuB4r2e129O7dG+Xl5fjhhx+wfPlyLFu2DJMmTfJqvzWllPLqErxk5cqV6Nu3L8rKyuDv73/O9/Py8hAVFYUVK1Y4B7K7d+9Gq1atkJ6ejmuvvfZSd5mIiIioxkVERODZZ5/F3/72N5djpc8//xy33XYbjh49ipiYGADAokWLMG7cOJw4cQImk8krffTzSqtedvr0abz55pvo0qXLeQenALBlyxbYbDb07NnT+V7Lli2RkJBQ7QC1rKwMZWVlzq8dDgdOnz6NevXqQdM0fVeEiIiIvEIphYKCAtSvXx8Gw6W/YFxaWory8nKvtK2UOmdMYjabYTabq/05u92Od999F0VFRUhOThaNldLT05GUlOQcnAJAr169MHz4cOzcuRMdOnTQd+V+V6sGqOPGjcNLL72E4uJiXHvttVi1atUFY7Ozs2EymRAWFlbl/ZiYGGRnZ1/w52bMmIEpU6bo1WUiIiKqQYcOHULDhg0v6TJLS0vRpEkDZGef9kr7wcHBKCwsrPLe5MmT8cwzz5w3/tdff0VycjJKS0sRHByMDz/8EK1bt8a2bdtcjpWys7OrDE4rv1/5PW+p0QHq+PHjMWvWrGpjMjIy0LJlSwDA2LFjMWTIEBw8eBBTpkzBoEGDsGrVKl1nNtPS0jB69Gjn13l5eUhISMDZ23V9cwb1hdZjRXGjdj2r2zI1LUAUp1SJbssEAH+/eqK4dkF9RXEPxEeI4kbtXiaKs1WcEMVJt9+ydsNFcYO3zXUZExrYQtRWfskBUVxYkKy9AGOoKO50yW+iuGeb3yuK25EnO19XFW4WxR0v/FUUFx8iu30or6L6hzwrFZbJfgEEmWNcxtjtsvOxxJYrigux1BfFhfs1EsWdKMsQxQ2L6SuKm3PgRVGcpp3/StyfORzFurZnDUgQxeUVZ4ri9DSh6ZOiuP87+b0oLjtPFqcvBcCBkJCQS77k8vJyZGefxoGD78BqDdS17fz8YjRudBcOHToEq9XqfL+62dMWLVpg27ZtyMvLw3vvvYfBgwdjw4YNuvZLbzU6QB0zZgxSU1OrjUlMTHT+OzIyEpGRkbjiiivQqlUrxMfHY+PGjUhOTj7n52JjY1FeXo7c3Nwqfxnk5OQgNjb2gsu78BS5Bl8doAYYLcJI/fov/aNAKX23mabJLtP4abJ7YgKM1V8OcXe50m0s3X6Bwv5JlqtpRllLwr4ZhO0ZNNnHjHQbS/eZWXhJT+/+GYSDE/2X63p/OMTHgL7bzijcJtJj1GyQfeZJj+Wai5Otb0387rEIf69Ij4Ga/P1Zk7fnWYMtsAbLJiTEHI6zbf/+VL6EyWRCs2bNAAAdO3bEjz/+iHnz5uHuu+92OVaKjY3F5s1V/5CvfMq/uvHUxarRAWpUVBSioqI8+lnH7zvoj/eL/lHHjh3h7++PtWvXol+/fgCAzMxMZGVlnXdAS0RERKQrh8M5oNS1zYtuwoGysjLRWCk5ORnTp0/H8ePHER0dDQBYs2YNrFYrWrdufdF9uZBacQ/qpk2b8OOPP+L6669HeHg49u3bh4kTJ6Jp06bODXjkyBH06NEDr7/+Ojp16oTQ0FAMGTIEo0ePRkREBKxWKx555BEkJyfzCX4iIiKqE9LS0nDLLbcgISEBBQUFWLFiBdavX48vvvhCNFa6+eab0bp1awwcOBCzZ89GdnY2JkyYgBEjRrh8KOti1IoBamBgID744ANMnjwZRUVFiIuLQ0pKCiZMmODcODabDZmZmSgu/t89Qs8//zwMBgP69euHsrIy9OrVCy+//HJNrQYRERHVJT4wg3r8+HEMGjQIx44dQ2hoKNq2bYsvvvgCf/nLXwC4HisZjUasWrUKw4cPR3JyMoKCgjB48GBMnTpV19X6s1oxQE1KSsLXX39dbUzjxo3x55SuFosFCxYswIIFC7zZPSIiIiKftHjx4mq/LxkrNWrUCJ999pneXatWrRigEhEREdU6Sp196d1mHVBrSp0SERERUd3AGdQaoJRNFBcR0l4U9/df/6nbcv/ZQnZPycT/ThPFSUm3SXBgc1HcxtxFwjhRGIICGoviysr1XY/dBbL0PH5+YS5j8oV5RntZHxHFbVWyvIbHCn4WxbW19hPFTThY/e0+lSwGWfqVHGF+03rCvK/Hy3aL4sorCkRxtoo8UVyxIN2P0SB7oKGsXJZ79YQwTguVpVPyM8rS8ViMshkkh0OW9zU59AFRXHqe7BkGpSpEcWcKt4vipHlVZW3J5qUm6PwZX2c5lBfuQeUMKhERERHRJccZVCIiIiJv8IGn+GsrDlCJiIiIvIEDVI/xEj8RERER+RTOoBIRERF5A2dQPcYZVCIiIiLyKZxBJSIiIvIG5YUZVMUZVCIiIiKiS44zqDVAklQdAMYkjBbFvVCWI4qTJHs26JgQGgAirVeJ4ozGYFGcUnZRnMVcXxQXZIoWxRWWyRKSW8zxorjwgCaiuB9PlIvilif9w2XM5IM7RW2tPjNXFPd5pzGiuPt2yZKlH3PIEtw3RUdRnE1Y/CEwOFwUd9KxXxTncMiWKz328uzForji0iyXMfXDuorautH8V1HcNrVZFFdizxXFSc06/JYoLi1xkihuV26ZKE5aUESaWF8a1yyij8uY387I6qQ7hOugt+YRsmPqQP4GUZyt4vTFdOeS0ZQDms4znnq356s4g0pEREREPoUzqERERETewKf4PcYBKhEREZE3ONTZl95t1gG8xE9EREREPoUzqERERETewEv8HuMMKhERERH5FM6gEhEREXkDZ1A9xhlUIiIiIvIpmlKqbjwO5qH8/HyEhoZCgwWaplUbq3cCZKMxSBQXYIoRxZXZcl3GSJMfSxPwl1UUiOKMmmwyf1rjfqK4tN9WiOIKS34TxYUHXymKKyo7IYozGkyiuGebp4riXjj6q8uY3IpDorZCjLGiuFKVJ4rz0yyiuEgkyJaLItlyYRbFmZRsX2jCv+fbWGTnY8d6ojC8fPS/orgCddxljDRhfm7xHlFcTIisaIJJCxDF3RHSSRS36NhC2XL9rKK4G8x3iuLWl8gKBJSUHRbFTbtioihu0p4ZLmOGxI0XtbX42L9EcVJKmDQ+KKCxKK6o5IDnnTmHAmBHXl4erFbZsaCXyrHDmZ2vwhoiO/7FbReUIPzKYTWyXpcSZ1CJiIiIyKfwHlQiIiIib+A9qB7jAJWIiIjIG5io32O8xE9EREREPoUzqERERETewEv8HuMMKhERERH5FM6gEhEREXmD8sIMqjC1V23HGVQiIiIi8imcQSUiIiLyAs3hgKbzDKre7fkqVpJyobIaBGAEUH0lqZDA5qI2y2xnRHF2R7koTlqVyN8v2GVMSekRUVv/vEJWtWTK/tdEcf7GQFFcUekBUVygRVaVyCSs1uVnkFUC0TSjKE5aOasBWovi7q3vuvrT8mNZorYeiJNtu4XZGaK4UIesZNK4ZuGiuKd+2yuKs0NW2S3CIaucZVSyfVuulYniHJrsl8wxx25RnKQCWKHDdbUpd7RHsijuF2wWxfUNvlEU91HhBlFcsU1W2S23SHYsSz9XrrP0F8WtyX1OFGfyj3QZY3eUiNo6+7vMNbs9XxT3ZBNZNazZ+6eJ4vRV85Wkcn+aB2uwzpWkCksQdvVjl30lKc6gEhEREXmDUmdferdZB3CASkREROQNTDPlMT4kRUREREQ+hTOoRERERN7AGVSPcQaViIiIiHwKZ1CJiIiIvMGhzr70brMO4AwqEREREfkUzqASEREReQPvQfUYB6g6GtdwgChu8r55ojiTnywBb3mFLPF/oCHKZUyxqhC1NT3rDVGcQZiQvrDkN1HcTaGPi+IyNFlicKlye5EoLsDguhgCAJg0Wdw/EqJFcZOzfnAZ08LRTtSWNAF/pF2W4P60QZYcfuw+WZGIaNVQFPf0FbLzZ8p/hYUzNNm5MapxjCjutyJZwvScEtn6rivKdBlj0UJFbQVDVlzhFyU7z+4JkyXgf3Z2gShuxQOyBPyNTJ1EcRUOWXEFg8FfFPd96buiuACzbN/2Cx3sMubt0wtFbdkqToviEsL/IoqTJuCfdoUsof+mE7J9serMbFEc1V4coBIRERF5g0N5YQa1btyDygEqERERkTewkpTH+JAUEREREfkUzqASEREReQMfkvIYZ1CJiIiIyKdwBpWIiIjIG5QXEvXzHlQiIiIiokuPM6hERERE3sB7UD3GAaqQyT8Gmlb9hPO7J7JEbaU1HiWK25MvSwz+3mlZ4n+Ju6LSRHGbK7aJ4nJtB0VxoZZ4Udwew3ZRnFHJEmoHGMJFcXbNJor7R2wHUVykSfYBc6hEdpEjTjVzGZOryZKghzhk2+SRpiGiuIkHZAn4E+yu1wEAbJCdF0dLTKK4MCVbD6kpWTtly3XIijAchesE/AAQpTVxGaMJL5rZITveK5QsqfpXebLPxruG1xfF3Rr0V1HcN+WuC1gAQJSlpSiu0C4rOmEUFihRSvY50CjEdVGHiuOyghPSBPyp9ZJFcdPzZdt4+UnZZ/dvuZ+J4ujyxwEqERERkTdwBtVjHKASEREReYPDCw9J1ZFKUnxIioiIiIh8CmdQiYiIiLxBOc6+9G6zDuAMKhERERH5FM6gEhEREXkD70H1GGdQiYiIiMincAaViIiIyBuYZspjHKAKldtOQdO0amN2Fnwsauv+2EdEce+cnC2KCw9uJYqz+rlOgr2m9BNRW8OjZYmyF5+UJY/208yiuAFhXURx685ki+KOOmRJ0OtpjURx+bbqj5FK10cViuLmZeWL4iY0i3EZ88KeIlFbDsguHy3+rVwU18bQQhRXKCyGcE98sCjuhcP/FcWFahGiuEENI0Vx7x1KFMXd0UC2HiaD630LAIsPH3MZ4yf8yM8znBTFXa11FcUFGoTLtcuOqX3C87apaiuKaxsSKoozuc6XDwB49fhborhgk6xYQ3yg6wGJ2Rwnaiun6FdR3AotUBQXF3KNKK7YIftdoGAXxdHljwNUIiIiIm/gPage4wCViIiIyBscyguX+OvGAJUPSRERERGRT+EMKhEREZE38BK/xziDSkREREQ+hTOoRERERF7hhVKnqBtppjiDSkREREQ+hTOoRERERN7Ae1A9VitmUA8cOIAhQ4agSZMmCAgIQNOmTTF58mSUl1ef1Llbt27QNK3K6+GHH75EvSYiIiKqWTNmzMA111yDkJAQREdHo2/fvsjMrFrsorS0FCNGjEC9evUQHByMfv36IScnp0pMVlYWevfujcDAQERHR2Ps2LGoqKjwWr9rxQzq7t274XA48Morr6BZs2bYsWMHhg4diqKiIsyZM6fanx06dCimTp3q/DowUFYd48/+GvEY/A3VVzv6rOhtUVtzj30tiltw5XhR3D8PfyOKezSuk8uY7BJRU4i1yO6BeSahhyjupcP7RXHv524VxVk0WZUepWRVSxpBVqXF6i/7y3ZfQZAorn+0VRS3I891zElDjusgABEOWXWbKJOs+tdNsbLqWvMPua6EBABfZzcQxbXUZBWdChyy6kWfHZHFGTXZ3/0BRtmxUuqQbb8HGro+Rpcdlh0DFiU7fxoHW0RxPxUeF8UVa7IKa8NjO4jiph6UVXS6P/wuUdznR2SfF883k7V3qlx2rCzPOuUyZkj0EFFbK858IIo7XPSjKK607KgoLjV2gijuC2ElqWO534viapwPzKBu2LABI0aMwDXXXIOKigo89dRTuPnmm7Fr1y4EBZ39XfT444/j008/xbvvvovQ0FCMHDkSd955J77//ux2ttvt6N27N2JjY/HDDz/g2LFjGDRoEPz9/fGvf/1L3/X7Xa0YoKakpCAlJcX5dWJiIjIzM7Fw4UKXA9TAwEDExsZ6u4tEREREPmf16tVVvl62bBmio6OxZcsWdO3aFXl5eVi8eDFWrFiBm266CQCwdOlStGrVChs3bsS1116LL7/8Ert27cJXX32FmJgYtG/fHtOmTcO4cePwzDPPwGQy6d7vWnGJ/3zy8vIQEeG6hvabb76JyMhItGnTBmlpaSguLq42vqysDPn5+VVeRERERG5zOLzzAs4Zq5SVlYm6lJd39pJb5Rhqy5YtsNls6NmzpzOmZcuWSEhIQHp6OgAgPT0dSUlJiImJccb06tUL+fn52Llzpy6b6s9q5QB17969mD9/Pv7+979XG3fvvffi3//+N9atW4e0tDS88cYbuP/++6v9mRkzZiA0NNT5io+P17PrREREVFdUXuLX+wUgPj6+ynhlxowZrrvjcGDUqFG47rrr0KZNGwBAdnY2TCYTwsLCqsTGxMQgOzvbGfPHwWnl9yu/5w01eol//PjxmDVrVrUxGRkZaNmypfPrI0eOICUlBf3798fQoUOr/dlhw4Y5/52UlIS4uDj06NED+/btQ9OmTc/7M2lpaRg9erTz6/z8fA5SiYiIyKccOnQIVuv/nlMwm10/FzBixAjs2LED3333nTe7posaHaCOGTMGqamp1cYkJv7vQYejR4+ie/fu6NKlC1599VW3l9e5c2cAZ2dgLzRANZvNop1MREREVC0vPiRltVqrDFBdGTlyJFatWoVvvvkGDRs2dL4fGxuL8vJy5ObmVplFzcnJcT7DExsbi82bN1dpr/Ipf28951OjA9SoqChERUWJYo8cOYLu3bujY8eOWLp0KQwG9+9O2LZtGwAgLk72RDYRERFRbaaUwiOPPIIPP/wQ69evR5MmTap8v2PHjvD398fatWvRr18/AEBmZiaysrKQnJwMAEhOTsb06dNx/PhxREefzfSyZs0aWK1WtG7d2iv9rhVP8R85cgTdunVDo0aNMGfOHJw4ccL5vcqR+5EjR9CjRw+8/vrr6NSpE/bt24cVK1bg1ltvRb169bB9+3Y8/vjj6Nq1K9q2bVtTq0JERER1xR8eatK1TTeMGDECK1aswMcff4yQkBDnPaOhoaEICAhAaGgohgwZgtGjRyMiIgJWqxWPPPIIkpOTce211wIAbr75ZrRu3RoDBw7E7NmzkZ2djQkTJmDEiBFeu+pcKwaoa9aswd69e7F3794q09LA2b8MAMBmsyEzM9P5lL7JZMJXX32FF154AUVFRYiPj0e/fv0wYYIsFxsRERFRbbdw4UIAZ4sX/dHSpUudt1k+//zzMBgM6NevH8rKytCrVy+8/PLLzlij0YhVq1Zh+PDhSE5ORlBQEAYPHlwlz7zeNFU5wqPzys/PR2hoKNqEDYRRqz7PV5btJ1GbRWWyZNkhloaugwAoYWJjs9H1vSpG+IvaMmiyuLvDrhXFvZe7RRQ3NLqjKM4kvAMk2E92+C84fFAUN61ZfVHc4RLZ9rMYZP174dABlzFt/RuJ2hrQSHY8zd8jqyBiExZDCDbI8ujZlGz24MlWsm03fZesvVCjbJbALvxIlW6X4zgjirs5wnUBg4OFsn22t9x1YngAuNYqK+qwMV+WqP+MQRbnr2T7IhEJorhIs+zYaxshK5rw80nZMdU5StZeQYXruOlZb4jamt5koCjOIiwk4afJ4h7aPl0Upy8FwI68vDy37tXUQ+XYIXfpSFgD9Z1hzC8uQ9gDL9XIel1KtTLNFBERERFdvmrFJX4iIiKiWscHSp3WVhygEhEREXkDB6ge4yV+IiIiIvIpnEElIiIi8gblhTRTwgdFazvOoBIRERGRT+EMKhEREZE38B5Uj3EGlYiIiIh8ChP1u1CZbNfkXx+aVv14PjRAlgjdbAgWxZXa80RxDiVLvh3oV89lzKR4WWL9ogrZ3zbxgTZR3MFiWeJ6ew0drdFmWVL1fJtsu4T5y+4hei9LttxHWriOezZD1BT8XRzn7sbdnyhb7qt7ZcdxqJ/sWCm1y7Zxv0ZGUdynh2UHX7Fdts8MkCVplzrhKHAZEwBZwvBirUQUVyEsEpIcEiuKuzJMWORAeAueUbiJpXHzDu8Vxd0f3VwUJy0o8sWxYpcx7cMDRW01lIUhyCjbyFMObRTF5RT+KooLsciKnZwq2CaI8oFE/S//HdYAWSEIcdsl5Qj7xytM1E9EREREdCnxHlQiIiIib+A9qB7jAJWIiIjIC5RDQek8oNS7PV/FS/xERERE5FM4g0pERETkDUqdfendZh3AGVQiIiIi8imcQSUiIiLyBj4k5THOoBIRERGRT+EMqpA1IB4GrfrNZRCO9/VOwB9gDBPFBWrhLmNizLJlrs+TJR7WNFlSdYtB9hfh8TLZNu5Sz3ViawDItcn6d7hEdqqE+MmSWx8sliWHvyFGFvdchuvE6n9LkCVp//qYbF/8vXmpKO5gsUUU1yUqQBTXJlRW/OH/9sqSyEsLDpy0ydY3xiRbD02YHP5EeZkoLlwLchlzTDspaivUESqKU5Ad701CRGEwarJj7z+HXRclAIAHGssWvPKw7Fh5MqGpKE7TZO2V2GXH3hUhrrPrx8hOM3EC/ucO7RHFSRPwuyp0U6nEdloUZzC4/jxTSkEp2e8Cr+EMqsc4g0pEREREPoUzqERERETewBlUj3GASkREROQNHKB6jJf4iYiIiMincAaViIiIyAuU8kKpUybqJyIiIiK69DiDSkREROQNvAfVY5xBJSIiIiKfwhlUIiIiIm/gDKrHOEAVshhCYHBRFckIWVUiP01W0ccAWRWhMiWrqiKJm7pHVsWjTJNV1RkfES+KO1oqW9dgP9mJOS1DVn3n4URZRSyjsOqPtJJUhFVWaeZEmewUHXmF64sh/96v74fa5J2yqmMPNZG19/OpclFcK6vswk+kWXaebc+TtTe2pWxfvP6bKAyFNtkxEO4nO0aP2Fyf3wmGaFFbBZDtizsayCpO5dpkJ1CUSXaMDhZWiJJWprq1gfCz1iFbD3/htclSu6y9poLVlX72nBRW4ztS/osoLtgSI4o7XSCrOBVgbiiKswYkuoxRyo684l2i9sj3cIBKRERE5A2cQfUYB6hERERE3qDU2ZfebdYBfEiKiIiIiHwKZ1CJiIiIvEA5zr70brMu4AwqEREREfkUzqASEREReQMfkvIYZ1CJiIiIyKdwBpWIiIjIGziD6jEOUIUqVDkMqP7OZIMmm5AeFNlGFPfD8RJR3CHtmCguUAW7jLkqWJbI+wqrKAwFFbITSXq+rTx2RhTXv0G4KM5itIni1h6VJVXv3VBWrMEuXN8gYfJtgyAh+bHyYlFb0f4BorggYcGJ5tYiUdygRNlyLQbZvvhrvL5PEmQVy9Y3wiJLvm4sk8W1ryf7XFl7zOIyxl/4GWWArG+fH5V9RrWwyvbtKWHBjq7RsvM2R1joIlR4ntmE521BhWw7R5tlxS7G7P/GZcyo2JtEbUnPCoPwAmuQn+x3RlB4D1Hc1do1orgNtk9dxiglrLBCPokDVCIiIiIv4FP8nuMAlYiIiMgblBcu8TNRPxERERHRpccZVCIiIiJvcEB+4687bdYBnEElIiIiIp/CGVQiIiIiL1AOBaXzPah6t+erOINKRERERD6FM6hERERE3sB7UD3GAaqQRQuBQas+CbtBkyWZfvvEb7JlIlAUJ0nADwAOwVH934JCUVv/LRCFYYgw+frufNm2uyNOloBfmkD87kay5OuRwuTrZoPs0kuZQ9ZepEmWyLugwvWpfH2kbF/8eEqW0P+hprKPj+czgkRxDyTKkq/7G6TFC2QXiPyF+6x+YKko7rvjsvPx2mhpEnFZ/4KMrotEOIRtaZqsb30aui4OAADrs2XHcV6F7BhIiZPtW4tw3xqEu2LrKdlyO0TIjtEiu/AY1Vz/LpAWMWliDhXFBfvHiOKMwmFEkf2UKG5l0XxRnObi9zEAqLqSMPQyxQEqERERkTcoSP/GdK/NOoADVCIiIiIv4ENSnuNDUkRERETkUziDSkREROQNfEjKY5xBJSIiIiKfwhlUIiIiIi9QjrMvvdusCziDSkREREQ+hTOoRERERN7Ae1A9xgGqkIIDysVR4a9kifX9YRLFlWqyhOnSRMmBDtcJ06WJvAc1liVfX7Zfltz8b/GyJPIFFbKM2tJLA6H+dlHcbQ30Tfjtr8naa2yVVUT48USEy5ibY2VtdQiXFS8os8tW9i9xsrhAP1kyd4eStSdNg7+/SLa+UWZZMYmb42T7VlpwQFrUwV9w8BXbZcsMNco+o74+Jjt/7k+ULdcsPIGkWXZ+Pi37JEiOlK3H1fVkcf8tkH0mx1pk26UeGrqMGdpYloB/TtYBUdy4BteL4gKMsp1xpES2L/518LAoThMU4lDKAbv9tKg98j0coBIRERF5Ae9B9RwHqERERETeoKD/Jfm6kaefD0kRERERkW/hDCoRERGRFyh19qV3m3UBZ1CJiIiIyKdwBpWIiIjIC/iQlOc4g0pEREREPoUzqERERETewET9HuMMKhERERH5FM6gCgUiDEYXFaDMyiJqyx/+smUqWbUmf+FuNAhq6/hrsmo5/z4gq3KVECBbhw05sj8JSyr0/dMx1iKrdDX+V1kFmRFNg0VxBcLqQKW5suowzUJc74/ZO2XVgR5sJntENNRfti+iLTZR3MkyWf8iTLL2bMKKU40Cy0RxJ4T9k1Ynsxhlcbk22efFnQmu99vHh2SfFaXCilO3NZTNcZgNsiph0qpeiUGyfRZulh0D/7e/UBQXYpB9xt/dSBQmPkbvjHJdSapCeGNi34imorhT5aIwvHdynyju7/WbiOIGRT4kiltx5t8uY5QP3KzJe1A9xxlUIiIiIvIptWKAeuDAAQwZMgRNmjRBQEAAmjZtismTJ6O8vPo/8UpLSzFixAjUq1cPwcHB6NevH3Jyci5Rr4mIiKguq8yDqvfLHd988w1uv/121K9fH5qm4aOPPvpTHxUmTZqEuLg4BAQEoGfPntizZ0+VmNOnT+O+++6D1WpFWFgYhgwZgsJC2ZUHT9WKAeru3bvhcDjwyiuvYOfOnXj++eexaNEiPPXUU9X+3OOPP45PPvkE7777LjZs2ICjR4/izjvvvES9JiIiojrNoXnn5YaioiK0a9cOCxYsOO/3Z8+ejRdffBGLFi3Cpk2bEBQUhF69eqG09H+3wN13333YuXMn1qxZg1WrVuGbb77BsGHDLmrTuFIr7kFNSUlBSkqK8+vExERkZmZi4cKFmDNnznl/Ji8vD4sXL8aKFStw0003AQCWLl2KVq1aYePGjbj22msvSd+JiIiIasott9yCW2655bzfU0rhhRdewIQJE9CnTx8AwOuvv46YmBh89NFHGDBgADIyMrB69Wr8+OOPuPrqqwEA8+fPx6233oo5c+agfv36Xul3rZhBPZ+8vDxERERc8PtbtmyBzWZDz549ne+1bNkSCQkJSE9Pv+DPlZWVIT8/v8qLiIiIyF2VD0np/QJwzlilrEz28OAf7d+/H9nZ2VXGSqGhoejcubNzrJSeno6wsDDn4BQAevbsCYPBgE2bNl3cBqpGrRyg7t27F/Pnz8ff//73C8ZkZ2fDZDIhLCysyvsxMTHIzs6+4M/NmDEDoaGhzld8fLxe3SYiIiLSRXx8fJXxyowZM9xuo3I8FBMTU+X9P46VsrOzER0dXeX7fn5+iIiIqHY8dbFqdIA6fvx4aJpW7Wv37t1VfubIkSNISUlB//79MXToUN37lJaWhry8POfr0KFDui+DiIiILn9KaV55AcChQ4eqjFfS0tJqeG31VaP3oI4ZMwapqanVxiQmJjr/ffToUXTv3h1dunTBq6++Wu3PxcbGory8HLm5uVVmUXNychAbG3vBnzObzTCbZbn4iIiIiGqC1WqF1Wq9qDYqx0M5OTmIi4tzvp+Tk4P27ds7Y44fP17l5yoqKnD69Olqx1MXq0YHqFFRUYiKihLFHjlyBN27d0fHjh2xdOlSGAzVT/527NgR/v7+WLt2Lfr16wcAyMzMRFZWFpKTk93uqwYDNBcTzg5Nlj1Xc8gmrv2EE9xmTbYbu0QFuIzZeKJE1JY0of+xUlki/CCjbB3+1ki2TUrssj8yns+QJUEfIcttDbsw8XaRXbYejYJk+yOn1PX69hduuzXZsn3bLVqWMN9slJ0XMRbZ/VPlDln/gv1kifA/OxYoirsmQpZsPswky3Au2WcAUGKXHVOSQhya8OHfFmGy86JYmNDfZJDFtQktEsUdLpYlzL/SKjtGW1tlBTbybNKLjrL1dQjTBUWZXQdKMw/VD5D1TVpEIB/HXQcBAGSJ+vNtwt+jot+PbuZj8gJfT9TfpEkTxMbGYu3atc4BaX5+PjZt2oThw4cDAJKTk5Gbm4stW7agY8eOAICvv/4aDocDnTt31q8zf1IrnuI/cuQIunXrhkaNGmHOnDk4ceKE83uVo/cjR46gR48eeP3119GpUyeEhoZiyJAhGD16NCIiImC1WvHII48gOTmZT/ATERFRnVBYWIi9e/c6v96/fz+2bduGiIgIJCQkYNSoUfjnP/+J5s2bo0mTJpg4cSLq16+Pvn37AgBatWqFlJQUDB06FIsWLYLNZsPIkSMxYMAArz3BD9SSAeqaNWuwd+9e7N27Fw0bVi35pn7PWGuz2ZCZmYni4v+VfHz++edhMBjQr18/lJWVoVevXnj55Zcvad+JiIioblLKCzOobk4M//TTT+jevbvz69GjRwMABg8ejGXLluHJJ59EUVERhg0bhtzcXFx//fVYvXo1LJb/Xal48803MXLkSPTo0cM5rnrxxRd1WZ8LqRUD1NTUVJf3qjZu3Ng5WK1ksViwYMGCCyanJSIiIvKWPz7UpGeb7ujWrds546M/0jQNU6dOxdSpUy8YExERgRUrVri13ItVK9NMEREREdHlq1bMoBIRERHVOg4Nys3SpJI26wLOoBIRERGRT+EMKhEREZEXKOX+Q02SNusCzqASERERkU/hDKqQH/xgdLG5bJAl6K6HcFGcUZP9/WBTsoTkG44X6rbMOxq4TvoPAF8fkyU3bxNuEsV9fEiWeDvQT7YeduGfol9ly06Vh5oViOKOChONnyiTJXP/6bTr7Rdmkq1rvCxvPb48JtsmA5vIEvArQaJ5ACgTFjnwM8jW10+YvT7CJDv2pIUEjJq+0yDpJ10vN9BPtsyDBbLPlBDheSbdt9JtsjVXduwl15N9Jm85IzvPjpfI8gWl1JfFCU81LN3nur1sdUbU1mONZcVxpEVHpjW6RhR3plzW3qpC/Z4UV76QqN8HnuKvrTiDSkREREQ+hTOoRERERF6gvPAUv+5ZAXwUB6hEREREXsCHpDzHS/xERERE5FM4g0pERETkBXxIynOcQSUiIiIin+L2APXGG2/E66+/jpKSEm/0h4iIiOiy4HBoXnnVBW4PUDt06IAnnngCsbGxGDp0KDZu3OiNfhERERFRHaUp5f7zYBUVFVi5ciWWL1+Ozz//HM2aNcODDz6IgQMHIiYmxhv9rDH5+fkIDQ1F27DBMGrVJ0M3KNl4P1DJ0jOHGmTJ8G1KlhTaYnCdyFuauN5kkK1r91jZbc7fHZetg1H4h6PFKOuftL3romTbJf2krEGrSRZ3Z8N8UVxOqevE/81DZW0t2RspipNuuxujZIn6/Q3SY0C2L/RO/C9drrRAQL5NltDfT7idywSzKr/myQpiHC6S7YtSuyzunsaygh1RFtmxcrxEVujiaKm/KC7MX1aYwCacuTII95m/8Fgps7tuMK9CdhybhctcdjBPFFeqya6m/jW6vihu8Yktorhc20GXMUrZkVu0A3l5ebBaraJ29VI5dsi4dRBC/GXnnVSBrRytPnu9RtbrUvLoHlQ/Pz/ceeed+Pjjj3H48GHce++9mDhxIuLj49G3b198/fXXeveTiIiIiOqIi3pIavPmzZg8eTKee+45REdHIy0tDZGRkbjtttvwxBNP6NVHIiIiolqn8il+vV91gdtppo4fP4433ngDS5cuxZ49e3D77bfjP//5D3r16gXt95rWqampSElJwZw5c3TvMBEREVFtwDRTnnN7gNqwYUM0bdoUDz74IFJTUxEVFXVOTNu2bXHNNdfo0kEiIiIiqlvcHqCuXbsWN9xwQ7UxVqsV69at87hTRERERLWdQ2lw6DzjqXd7vsrte1AnT56M3Nzcc97Pz8/HTTfdpEefiIiIiKgOc3sGdcOGDSgvLz/n/dLSUnz77be6dIqIiIiotlMODUrnxPp6t+erxAPU7du3AwCUUti1axeys7Od37Pb7Vi9ejUaNGigfw+JiIiIqE4RD1Dbt28PTdOgadp5L+UHBARg/vz5unaOiIiIqLZS6uxL7zbrAvEAdf/+/VBKITExEZs3b67y9L7JZEJ0dDSMRllllNrIhnK4qpkSriJkbWk2UVyerEgLLC4qXFUyCCrrGDRhpRRhlR5phajmVtmheLBQVvHFJCxzJL0Je3uurL0BjYpFceHmUlHcnvwQUdzOfNfHQIRJVn2nW7Ssmo801ckPp2TLlV60uiVOVhEr3yarIiStYBUdIKuYk1UYLIqTVhuSVrAKNLqOky5Tev60i5B95keai0RxZ8pkn2UWo+xzoL7s0EOpQ/ZJEBdw7u1t51NcIdsuFcKBxuFy19vlzaNHRW0Zhb/2B8Wfm6HnfEL9ZJWMDso+GlHkOCWKC/F3XZnKoWzIxQ7ZgsnniAeojRo1AgA4HMJRExEREVEd5oAXnuIX/zlfu4kGqCtXrsQtt9wCf39/rFy5strYO+64Q5eOEREREdVmTNTvOdEAtW/fvsjOzkZ0dDT69u17wThN02C3yy69EBERERGdj2iA+sfL+rzET0REROSa8kKi/royg+p2on4iIiIiIm9ye4D66KOP4sUXXzzn/ZdeegmjRo3So09EREREtV7lPah6v+oCtweo77//Pq677rpz3u/SpQvee+89XTpFRERERHWX26VOT506hdDQ0HPet1qtOHnypC6dIiIiIqrtHL+/9G6zLnB7gNqsWTOsXr0aI0eOrPL+559/jsTERN065mvqOSLhp5mrjSmDLImzA7LszKGGAFGcUZhcXxInTx4uC3y0pSxB98oj5/7Rcz5B/rJJ/yDhkd0kWLYvDhbJ1tdkkGWxePOArKjDTdGyhP7XRrhOIv/fgkBRW23CCkVxR4tlWdD7J8gSbxcJE+vbhZe3wkyy8zFXkAQdAA4VBYniTMLE/zbhehTZ9SuAcqpUdrwL8/TjiDD5el6wbBtnFVf/GVupWbCsaMInR2THVP8E2XmmdwL+U+WyD6pGga6Lu4xrGi1q68MsfS8Pbz4l2ybrCw+I4jpqN4jiduIXURzVXm4PUEePHo2RI0fixIkTzpKna9euxXPPPYcXXnhB7/4RERER1UrMg+o5tweoDz74IMrKyjB9+nRMmzYNANC4cWMsXLgQgwYN0r2DRERERLWRQ0H/SlLCGfrazu0BKgAMHz4cw4cPx4kTJxAQEIDgYFntaSIiIiIiVzwaoALAiRMnkJmZCQBo2bIlIiMjdesUERERUW3HS/yeczvNVFFRER588EHExcWha9eu6Nq1K+Li4jBkyBAUFwvvmCciIiIiugC3B6ijR4/Ghg0b8MknnyA3Nxe5ubn4+OOPsWHDBowZM8YbfSQiIiKqdc7eg6r/qy5w+xL/+++/j/feew/dunVzvnfrrbciICAAd911FxYuXKhn/4iIiIiojnF7gFpcXIyYmJhz3o+OjuYlfiIiIqLf8R5Uz7k9QE1OTsbkyZPx+uuvw2I5m6i7pKQEU6ZMQXJysu4d9BVmzQ9+WvWby+oikb8zzl+22aUJ/Q2QHazXRruOC/eXJRmXXmJ4bW+IKC7AT9agv/C8tAv7lyVMwH9FiGy7SJO+SxPwVwg/iLbluj72TMIbegqFCfMjzbJE+NIE/P7CBPdG4fFeUiE7z6Tb2F+THVR6JtYHgOIK2Y7LLHC9XIvwPLMJS9XM/DxOFLdy4EFR3PZc2b5oEiSLk67H+4dkRSf+2rBMFGcRHitL9ul3rdamZCvrr8mOp00nZOdtx3qyfXGkuIEo7qD9hCjOCNf9qxvDuMuX2wPUefPmoVevXmjYsCHatWsHAPjll19gsVjwxRdf6N5BIiIiotrIAQ0OnYfKerfnq9weoLZp0wZ79uzBm2++id27dwMA7rnnHtx3330ICJCV5iQiIiK63Cl19qV3m3WBR3lQAwMDMXToUL37QkREREQkG6CuXLlS3OAdd9zhcWeIiIiILhcOpXmh1Ckv8Tv17dtX1JimabDb7RfTHyIiIiKq40QDVIdD+CgkEREREQEAlBceklJ15CEptytJ/VFpqSxVDhERERGRlNsDVLvdjmnTpqFBgwYIDg7Gb7/9BgCYOHEiFi9erHsHiYiIiGqjyqf49X7VBW4/xT99+nQsX74cs2fPrvIkf5s2bfDCCy9gyJAhunawNjFqsml3u/DokrYXGSD7O+OX066Xq4R/s1xTTxSGW+pXiOI2npIlhZYm4A/y0/cSyFUR+aK4o8WyVGs/n5ElBg836VfA4NYGp0RtfXZEtnOvDpdVjnMIk5aX2GUfRzaHbN8WCxPmSxPw20RRQKOgIlHczjxZEQt/g6x/jYNc34qVXSrbdmdkNRgwqXe2KK5CyQpY+AunTKwm2d5oZpWdj8eFFwNf3y/7nJLy12S3z10X43q5a46ViNq6JlJWUObr43miuGNHZOetJvx9VmiQLTdAWV3G2CE8kMknuT2D+vrrr+PVV1/FfffdB6Pxf78A2rVr58yLSkRERFTXVT7Fr/erLnB7BvXIkSNo1qzZOe87HA7YbNI5BiIiIqLLm4Km+0NNfEjqAlq3bo1vv/32nPffe+89dOjQQZdOEREREVHd5fYM6qRJkzB48GAcOXIEDocDH3zwATIzM/H6669j1apV3ugjERERUa3jUGdferdZF7g9g9qnTx988skn+OqrrxAUFIRJkyYhIyMDn3zyCf7yl794o49EREREVIe4PYMKADfccAPWrFmjd1+IiIiILhssdeo5t2dQH3roIaxfv94LXSEiIiIi8mCAeuLECaSkpCA+Ph5jx47Ftm3bvNAtIiIiotqt8il+vV91gduX+D/++GOcOXMG7777LlasWIG5c+eiZcuWuO+++3DvvfeicePGXuhm7WAxysb7fgbZweUvjCu1i8JgFLRnFB73cRZZAuRiu2ybSJcr3CToHFEmivs5V5a0+vOjYaI4af+kCfhL7LIG65ldJ/wODZRlIw/yk/Uts0CWBF3qqogCUdzPp2UJ7uMDZcdo81BZEYbDhcGiuJOlsiIMAUZhgQBhYYJDxa7PtQizbJl5wvzmFuE6tAuTnY+bTsm2XZFN9qurY7jsmH83S/Y5cF9jWeGRggpZkYgyh+zz8bMjrj/kQ4yyIgLfHpcVkuhSL0wU9+OpQlGctH/RFXGiuGzjYZcxdnF5DfJFbs+gAkB4eDiGDRuG9evX4+DBg0hNTcUbb7xx3vyoRERERHVR5VP8er/qAo8GqJVsNht++uknbNq0CQcOHEBMTIxe/SIiIiKiOsqjAeq6deswdOhQxMTEIDU1FVarFatWrcLhw66n3ImIiIjqAt6D6jm370Ft0KABTp8+jZSUFLz66qu4/fbbYTbL7t8hIiIiqiuYqN9zbg9Qn3nmGfTv3x9hYWFe6A4RERER1XVuX+IfOnSoc3Dau3dvHDt2TO8+ndf06dPRpUsXBAYGigfHqamp0DStyislJcW7HSUiIiLC/xL16/3yxIIFC9C4cWNYLBZ07twZmzdv1nlt9XVRD0l98803KCkp0asv1SovL0f//v0xfPhwt34uJSUFx44dc77+85//eKmHRERERL7n7bffxujRozF58mT8/PPPaNeuHXr16oXjx4/XdNcu6KIGqJfSlClT8PjjjyMpKcmtnzObzYiNjXW+wsPDvdRDIiIiov9RXnq5a+7cuRg6dCgeeOABtG7dGosWLUJgYCCWLFlyMavnVRc1QG3UqBH8/WXJd2vK+vXrER0djRYtWmD48OE4depUtfFlZWXIz8+v8iIiIiLyJX8eq5SVnb8gRnl5ObZs2YKePXs63zMYDOjZsyfS09MvVXfd5vZDUr/99hsSExMBADt27NC9Q3pKSUnBnXfeiSZNmmDfvn146qmncMsttyA9PR1G4/krfcyYMQNTpkw5532zwQh/rfrqIGUO19V8ACDAT7bZpdWVpHFK8GeXtK1vT5pEcX6arMGrI2QVP7bnyv4gklaI0oR/i/oL/5STbr9yYXWgDmGyW2h25ruu6rTzeD1RW7c1lt1X/lVWrChOuq5+muz8kVacshhlVX/yy2THcmJYrijut9wwUZz0Sdwy4fa7Ktx1taZtwvPCaJB1rmuUrIrQde9dLYpreI/sl+X7hyJEcZ3rySpJSSvAvZslO1YC/WQN9q4vq7B1dyPX7e0plPVt43HZvt15Rta3CsjO29MVsn1h02S/CyzKdWU3uxKWRPMiBc/vGa2uTQCIj4+v8v7kyZPxzDPPnBN/8uRJ2O32c3LVx8TEYPfu3br2TU9uz6A2a9YM3bt3x7///W+UlsoOuAsZP378OQ8x/fl1MRtvwIABuOOOO5CUlIS+ffti1apV+PHHH7F+/foL/kxaWhry8vKcr0OHDnm8fCIiIiJvOHToUJXxSlpaWk13SVduz6D+/PPPWLp0KUaPHo2RI0fi7rvvxpAhQ9CpUye3Fz5mzBikpqZWG1M5W6uHxMREREZGYu/evejRo8d5Y8xmM/O6EhER0UVz/P7Su00AsFqtsFqtLuMjIyNhNBqRk5NT5f2cnBzExsquhNUEt2dQ27dvj3nz5uHo0aNYsmQJjh07huuvvx5t2rTB3LlzceLECXFbUVFRaNmyZbUvk0l22ULi8OHDOHXqFOLi4nRrk4iIiOh8lNK88nKHyWRCx44dsXbtWud7DocDa9euRXJyst6rrBuPH5Ly8/PDnXfeiXfffRezZs3C3r178cQTTyA+Ph6DBg3SPT9qVlYWtm3bhqysLNjtdmzbtg3btm1DYeH/7oFq2bIlPvzwQwBAYWEhxo4di40bN+LAgQNYu3Yt+vTpg2bNmqFXr1669o2IiIjIV40ePRqvvfYali9fjoyMDAwfPhxFRUV44IEHarprF+T2Jf5KP/30E5YsWYK33noLQUFBeOKJJzBkyBAcPnwYU6ZMQZ8+fXRNAjtp0iQsX77c+XWHDh0AAOvWrUO3bt0AAJmZmcjLywMAGI1GbN++HcuXL0dubi7q16+Pm2++GdOmTeMlfCIiIvI6b17id8fdd9+NEydOYNKkScjOzkb79u2xevXqcx6c8iVuD1Dnzp2LpUuXIjMzE7feeitef/113HrrrTAYzk7GNmnSBMuWLUPjxo117eiyZcuwbNmyamPUHx5TDwgIwBdffKFrH4iIiIhqo5EjR2LkyJE13Q0xtweoCxcuxIMPPojU1NQL3ssZHR2NxYsXX3TniIiIiGorh5KnlHOnzbrA7QHqnj17XMaYTCYMHjzYow4RERERUd3m8T2odY1dKRhcJHW3XCD5/59Jk7lLSZPI2wU3roSZZJ2zC/+Ck66qNAG/vzCBuDTxtkm47aTbWPrUoZ9wPXYXWERxRs11e0F+ssT1ZpMs7o6WWaK4M7mBoriNwkICHSLyRHF+RrsorsgmO/bySmT7wmSQLdegyZYb4idrb3eB64wnJuFxZ5F9lOG6jzqL4g49sEYUZ/KTZW3JK5etR4FNtiL94mVJ6UvssjP81zzZenx4WBaXX+76GHAo2Xkr5Sf8EA2Gfpl2AKDMIUuuX0+5/ryoULL96k0KmjOxvp5t1gUXVeqUiIiIiEhvnEElIiIi8gLeg+o5DlCJiIiIvICX+D3n9iX+m266Cbm5uee8n5+fj5tuukmPPhERERFRHeb2DOr69etRXn7uTcylpaX49ttvdekUERERUW3HS/yeEw9Qt2/f7vz3rl27kJ2d7fzabrdj9erVaNCggb69IyIiIqI6RzxAbd++PTRNg6Zp572UHxAQgPnz5+vaOSIiIqLaijOonhMPUPfv3w+lFBITE7F582ZERUU5v2cymRAdHQ2jMA8oEREREdGFiAeojRo1AgA4HIJs75chk2aAv6H6Z8pKKmQJtcOE2eE1TfaknjTxv79gb5cKM/BbhAuVJtaXPpMoTcAv3SZh/rL+lQsPe+Eug594PWT9a2UtdhmzO1+WML+xMBG+1JYTEaK4G+KOi+IcSrbxTP6y87GwTJZo3KHzk7PSWRDp+gYJzu9cWQ50pK2S3a712k07RHHdYmVFCU4UB4jipIUzYiyyFc6zyX4VfpUtO1bKhTu3yCb7YOkY6bp/204JC0ToPPtmF37o2YRjBz/IJrrK4HrfVghivI1P8XvO7af4Z8yYgSVLlpzz/pIlSzBr1ixdOkVEREREdZfbA9RXXnkFLVu2POf9K6+8EosWLdKlU0RERES1nVL/uw9Vr5fiPajnl52djbi4uHPej4qKwrFjx3TpFBEREVFt5/j9pXebdYHbM6jx8fH4/vvvz3n/+++/R/369XXpFBERERHVXW7PoA4dOhSjRo2CzWZzpptau3YtnnzySYwZM0b3DhIRERHVRkppUMIHHd1psy5we4A6duxYnDp1Cv/4xz+cFaUsFgvGjRuHtLQ03TtIRERERHWL2wNUTdMwa9YsTJw4ERkZGQgICEDz5s1hNpu90T8iIiKiWon3oHrO7QFqpeDgYFxzzTV69oWIiIiIyLMB6k8//YR33nkHWVlZzsv8lT744ANdOuZrLEbXifqDzLJnzvyE2ealyeaFufV1XaY0gbxdmlRdmD1a2j9pQv9Sh3RfyPrnL+6fvgUMJEn4A4yyZW48GiOK69JQlrWjWUiRKG7biXqiuBZh+aI4aaJ+f4NsPqK4QvZxaVduP3tarSOlsuVK9q+0wMZLfz0iiisXnj9XzGwmivvoXlmRiB4xZaK4r3KCRHHtwmyiuJQ4WeL3wgrZMWATfj4a4PpY3iH80PMTfqqcKpOtq9kgS6x/VT1ZEYZvT8r2bW3BUqeec/uT9K233kKXLl2QkZGBDz/8EDabDTt37sTXX3+N0NBQb/SRiIiIiOoQtweo//rXv/D888/jk08+gclkwrx587B7927cddddSEhI8EYfiYiIiGod5aVXXeD2AHXfvn3o3bs3AMBkMqGoqAiapuHxxx/Hq6++qnsHiYiIiGqjs5f4NZ1fNb1Wl4bbA9Tw8HAUFBQAABo0aIAdO3YAAHJzc1FcXKxv74iIiIioznH7IamuXbtizZo1SEpKQv/+/fHYY4/h66+/xpo1a9CjRw9v9JGIiIio1vHGJfk6MoHq/gD1pZdeQmlpKQDg6aefhr+/P3744Qf069cPEyZM0L2DRERERFS3uD1AjYiIcP7bYDBg/PjxunaIiIiI6HLANFOe8ygPqt1ux4cffoiMjAwAQOvWrdGnTx/4+Xmc95+IiIiICIAHA9SdO3fijjvuQHZ2Nlq0aAEAmDVrFqKiovDJJ5+gTZs2uneSiIiIqLZhqVPPuT1Afeihh3DllVfip59+Qnh4OADgzJkzSE1NxbBhw/DDDz/o3klfEOCnweSikpR01l2aOkEJGwwSli+yCxoM0nkSXFohSlr5yaRvkR5xhSizzpWupMuVklQRyrPJNt4tCUdFcSfzZVV69hQEi+L+0vyQKK60xF8UVyGs5hMVWiiKO3rGKoqTKhNWYZIe82fKXbcnrXSWKysihAlfNBLF7XhoqyiuQkW4DgJwrFR2DLQIkVUTC/GTxeWUyZZ7qlx47JlkQw2zn+u466Nly9xXKDsI4gItorgoWRi+y5FViDJCVpkqSDB8qRDX4iNf5PZwZNu2bVUGp8DZ1FPTp0/HNddco2vniIiIiGorpeSTTe60WRe4PR91xRVXICcn55z3jx8/jmbNZPWWiYiIiC53ChocOr9UHZkZdnuAOmPGDDz66KN47733cPjwYRw+fBjvvfceRo0ahVmzZiE/P9/5IiIiIiJyl9uX+G+77TYAwF133QVNOzuKV7/PN99+++3OrzVNg90uu6+HiIiI6HLDS/yec3uAum7dOm/0g4iIiIgIgAcD1BtvvNEb/SAiIiK6rDDNlOc8SipUWlqK7du34/jx43A4qm6qO+64Q5eOEREREVHd5PYAdfXq1Rg0aBBOnjx5zvd43ykRERHRWSx16jm3B6iPPPII+vfvj0mTJiEmJsYbffJJpRWAw0WydoufLPVDkCzXs5g0OTxEqSn0TUjvL8wTIW3PIOyffLn6nun+wvakhQnC/CtEcQUVrpNbS4smKCUt/CDbyPUDZAm6dx2OEsW1iD0liisrl3287T0hSw4fF1wkisspChTFFQkLCYz8sIEobuGdh13GFNpk+/bpLR1Fcc933iKKG/1GY1Hc1ofOiOL2FOhbUUR6jIb6yyZgpOdtVrFJFPf9Cdf7rVOk7Pw+VqzvZ5RRkyXWN7sodFOp1CFrr1i53mcVsInaIt/k9lmek5OD0aNH16nBKREREZG7FORVJt1psy5wOw/q3/72N6xfv94LXSEiIiK6fFRe4tf7VRe4PYP60ksvoX///vj222+RlJQEf/+q16sfffRR3TpHRERERHWP2wPU//znP/jyyy9hsViwfv16Z7J+4OxDUhygEhERETFR/8Vwe4D69NNPY8qUKRg/fjwMwpueiYiIiIik3B6glpeX4+677+bglIiIiKgaTNTvObdHmYMHD8bbb7/tjb4QEREREbk/g2q32zF79mx88cUXaNu27TkPSc2dO1e3zhERERHVVkzU7zm3B6i//vorOnToAADYsWNHle/98YGpy024WYPJxW0NNuFBUy4stmUWZq+XHqyS5PX+wl1okeVSFpMmzJf2z6BzAn6LMMm92SiL04SZ7MocshX2F/QvyE924G05Fi2Kk2oemi+KC/CXJdUuKZFVujhZHCCKSxD277+nw0Vxp4UFAkL8ZBfqXvrrEVGc5NyQfkRPvepnUdwz79cTxT03QFZcIbleiSgu1xYsiosyyRLmp5+SHSvSAiCnZHn/0SVSdsxb/V1/4G46KS0UI4srssmOz6PFss8Vo/Dg2450UVxbLdlljGzvk69ye4C6bt06b/SDiIiI6LLCRP2e07deHBEREREB4CX+iyEeoN55552iuA8++MDjzhARERERiQeooaGh3uwHERER0WVFQYOCvs/n6N2erxIPUJcuXerNfhARERERAeA9qEREREReoaD/PaN15BZU9xP1ExERERF5E2dQiYiIiLyAT/F7jgNUodxyBZOh+sTF9SyyCelAP31vcI4PlCVUPl7merlG4Zy6tBZwoDBxvUmYCF9KemnA1T51Vz1TuSiuoEJ26tmV7FiRJP73ExYvaBhYLIrLKZElNz9eHCiKK66QVX+45ZVYUdz+f8gS3GcKE/A7hPtCWqyhTFiwI8JfFniq3PX2E9b+EJ+3OC5LwD/mBdl5NvNRWQL+1lbZeWYWnt/NgmUbJiNfdt7eGCXL1P9VjkkUV8/sOsYqawr1hHEZebK4uEDZp+3+AtlxfKXqLIpzCD7PHHXmYvjliQNUIiIiIi9gon7PcYBKRERE5AW8xO85PiRFRERERD6FM6hEREREXqB+/0/vNusCzqASERERkU/hDCoRERGRF/AeVM9xBpWIiIiIfApnUImIiIi8gGmmPFdrZlCnT5+OLl26IDAwEGFhYaKfUUph0qRJiIuLQ0BAAHr27Ik9e/Z4t6NEREREtZBkrJWVlYXevXsjMDAQ0dHRGDt2LCoqKqrErF+/HldddRXMZjOaNWuGZcuWud2XWjODWl5ejv79+yM5ORmLFy8W/czs2bPx4osvYvny5WjSpAkmTpyIXr16YdeuXbBYLG4tv12EBouLMkuHi2RtGYTVXIzCyj/5FbIGLYJCPdIaV9KKNNJ7ZezCOIuw4pR020n3hTROWiFKKsgoq74iWVvpNjlVJjs34kMKRXHSSlIh/jZR3A+PHBLFKcgqU/kLj6njpbJ9G2GqcB0EoMyu87Hi57pqkr/4OJYFbp0uKzcUINy3tzeUzZl8cbSeKC7YT7ZvE4NKRXFlwopTe4tk5Zr8hVNEIf6uY1qEyI67XJvsvDAJP/TiZAXlEOQnW25mrqzBogrXn402pW+lQE/UtntQXY217HY7evfujdjYWPzwww84duwYBg0aBH9/f/zrX/8CAOzfvx+9e/fGww8/jDfffBNr167FQw89hLi4OPTq1Uvcl1ozQJ0yZQoAiEfhSim88MILmDBhAvr06QMAeP311xETE4OPPvoIAwYM8FZXiYiIiGodV2OtL7/8Ert27cJXX32FmJgYtG/fHtOmTcO4cePwzDPPwGQyYdGiRWjSpAmee+45AECrVq3w3Xff4fnnn3drgFprLvG7a//+/cjOzkbPnj2d74WGhqJz585IT0+/4M+VlZUhPz+/youIiIjIXUp55wXgnLFKWVmZ19cnPT0dSUlJiImJcb7Xq1cv5OfnY+fOnc6YP469KmOqG3udz2U7QM3OzgaAKhux8uvK753PjBkzEBoa6nzFx8d7tZ9ERER0eXJ46QUA8fHxVcYrM2bM8Pr6ZGdnn3dcVfm96mLy8/NRUlIiXlaNDlDHjx8PTdOqfe3evfuS9iktLQ15eXnO16FDsvvdiIiIiC6VQ4cOVRmvpKWlnTfOF8daEjV6D+qYMWOQmppabUxiYqJHbcfGxgIAcnJyEBcX53w/JycH7du3v+DPmc1mmM1mj5ZJREREVMmbD0lZrVZYrVaX8XqOtWJjY7F58+Yq7+Xk5Di/V/n/yvf+GGO1WhEQIHyqDjU8QI2KikJUVJRX2m7SpAliY2Oxdu1a54A0Pz8fmzZtwvDhw72yTCIiIiJfoudYKzk5GdOnT8fx48cRHR0NAFizZg2sVitat27tjPnss8+q/NyaNWuQnJzs1rJqzT2oWVlZ2LZtG7KysmC327Ft2zZs27YNhYX/S3XTsmVLfPjhhwAATdMwatQo/POf/8TKlSvx66+/YtCgQahfvz769u1bQ2tBREREdYY3HpDyYpopV2Otm2++Ga1bt8bAgQPxyy+/4IsvvsCECRMwYsQI59Xnhx9+GL/99huefPJJ7N69Gy+//DLeeecdPP744271pdakmZo0aRKWL1/u/LpDhw4AgHXr1qFbt24AgMzMTOTl/S8v35NPPomioiIMGzYMubm5uP7667F69Wq3c6ASERERXe5cjbWMRiNWrVqF4cOHIzk5GUFBQRg8eDCmTp3q/JkmTZrg008/xeOPP4558+ahYcOG+L//+z+3UkwBgKaU8uJYvPbLz89HaGgopl2RBoux+oHtCVmuZwTp/GdBoI7tBRmlifBl7UkPLr0T8JuF62EyyBI5S5cr5SdsT9o/u3K9Q6QJ6U0GWXGAIGHy9fxyWdLyIH9ZovEim+yAlxZXyCqS/cEa6i/bLmajbJ8dLxVkX4c8sXqg4JhvGSorrnDNf64Wxf14z0+iuE4LZPe3bfh7liiuzC67+FfukMUVVAiTyBfIjj2L8POncaDsmN+W63q50uO9VHYYI7dMtg4WP9mCSytk7RULEvADQGGF621nU2VYkzcXeXl5ons19VQ5dhjaIA0mg76TYuWOUrx2ZEaNrNelVGsu8RMRERFR3VBrLvETERER1SZ/TKyvZ5t1AQeoRERERF7wx8T6erZZF/ASPxERERH5FM6gEhEREXmBUgp6P4teV55t5wwqEREREfkUzqASEREReYE3S51e7jiDSkREREQ+hTOoQibD2Vf1MbKExWZhwnQ/4Z8P/sK4mrhtJUiYtFyaZFqebF623AhTuSiuwCZLqi5N0i5N/K8JSx3oWUig3CFLWh5jLhLFldplHzN2h+wgMAm3sVSjoBJRnE2Y9D1XeKyECBP/SyVF5LmM+fV0qKit327bJYq7Jlq27XDwqCjsxsWyhP4L/nZMFBdrkSXCP1EmO+ZjLbJjT/o5lS8sECBprVx4OBXZZH2zmmTnY0KQbLm7cmVxx22yY8pqMLuMUT4wB+eNyqR1ZALVB/YeEREREdEfcAaViIiIyAt4D6rnOEAlIiIi8gIOUD3HS/xERERE5FM4g0pERETkBWcfktI5Ub+urfkuzqASERERkU/hDCoRERGRF/AeVM9xBpWIiIiIfApnUIXMBgWLy+TL0kTjsmXahPnIA6VJ2gXdkybMlxYb0DsBf5BRlo3apmQLlibg9xcm/pf+ZWsRJpuXJupXgp3rJ1wHg3CZh/KtorggP5sorrhC9nFUL6BUFFduFyZBV/r+nR7qL00ObxLFSRLwA8DOM66T8MdYZIUpTpXL+tZ0cmNR3M9pssT6Vz0lS+jfIbxYFJdT6jqZOwDUD5DtswrhZ/LxMtmxfKJMWtzFdUygSXbeVgg/G4P8ZHF7C4Sf3cL2Dhv3i+I6Gq90GeMnLDriTUrpXySnJoru1ATOoBIRERGRT+EMKhEREZEXKCg4dH+Kv25MoXKASkREROQFvMTvOV7iJyIiIiKfwhlUIiIiIi9w/P7Su826gDOoRERERORTOINKRERE5AVKKSidbxrVuz1fxRlUIiIiIvIpnEElIiIi8gKWOvUcB6hCxXYNDhfVeqTVkKSMwipM8jjX/ZOug7AoCEzS6kXC9qQ3h5uFy5Wur2TbnY2TLddkkFXEMgiXWyGohiSthhVsklUbslTI1kHKJKwSpoSVcBJizojiMg5HieKk288u7F/TkEJR3P78EFFcoyDX1ZUOFweI2iqukF1cWzfmlCguVFaYCqsmyCo6XRFWIoqTVpIqscv2WYBRej6KwtAkSHbMnyxzvT/KpJUHhb/1y4WjoCBheyWyXYtbg9vKAgXKHLxIXJtxgEpERETkBQ4vJOrXuz1fxQEqERERkRcoeCFRv77N+SzOfxMRERGRT+EMKhEREZEX8BK/5ziDSkREREQ+hTOoRERERF6glP73jNaRPP2cQSUiIiIi38IZVCIiIiIv4D2onuMAVcjfoGBykdTdqMmSPUunra3+sszL5Q7ZckMF7dmEbckT18vipAnzA4zSVP0y0oT5UhZhsvmrn08Qxe0Yu18UZxKUMAg2l4naKq+QfSwYhYnrQ4NkSdVP5geJ4qQJ/bNPWkVx0n0WEShbjwN5oaI4gzBxeWNhQv9Dha63n1l4nvV7q74oTm3YJopDz2tEYR1kreG1v2aJ4iJM+hZ/yCmVfXrHmGXLPVpqFMWdEtTOSAiU7dv/5svWtVM92TrsK5J9XoT4i8LEvzMkBTFKhQUYyDdxgEpERETkBQ7lhRnUOnITKgeoRERERF6gfv9P7zbrAj4kRUREREQ+hTOoRERERF6gAMETAu63WRdwBpWIiIiIfApnUImIiIi8gGmmPMcZVCIiIiLyKZxBJSIiIvICpbzwFD/TTNEfKeW6/q1FlnMZfsJk2QZhjmGLUb+D1Sxsy0+T3fYdYbKJ4oqEyeGl9C4kECJcj9IK2UGwe/xeUVx4sCy5vs0mPPgETH6yDPIlNlnm7fqzOoni1JM/iuLKymXHSpGwf+EBpaK4HEEifAC4aUkTUdzOkTtFcWXCY8pfUDghRlisAXsOisK+fyNYFOd4fbcorutEiyhu2PgCURyuaikKe+OeI6K4vzaTxa3c11AUF2ESFrvwd/3LIFtYRKBNmGyZvxXJjjub8Akgf+H12lxBUYKz7bn+7C5z1I2B3OWKA1QiIiIiL+A9qJ7jAJWIiIjICzhA9RwfkiIiIiIin8IZVCIiIiIvUL/PoerdZl3AGVQiIiIi8imcQSUiIiLyAt6D6jnOoBIRERGRT+EMKhEREZEXcAbVcxygCmna2Vd1pImIpaQ5hgOEyfUliY1tDll1AH/hMruOFoXh6BvHRXG/nQwXxV3/fKwobsMo2XLtDlnyepPRLoprOSFOFPfbjMOiOAk/P9mN9cHC4gCNx7UWxeU8tUkUp7k6wX5ntcoS6wdYZMUVgkJlmcH9T8n27ZFxP4niAvxlH79HhAUCEqyuk9e3nCxLIJ8+/qQo7vpXm4riKt5JF8UBjUVR370s23bX//OEKK6gQvbh/elvsu1nF352Hy+RJcMP83fdoCQGAHJtsvMsyixrr1h2WqBc+FxPlFkWJ1Eq7Bv5Jg5QiYiIiLzA8ft/erdZF3CASkREROQFSlNQwtLg4jbryCV+PiRFRERERD6FM6hEREREXqC88JAUZ1CJiIiIiGoAZ1CJiIiIvMABBzQ+JOURzqASERERkU/hDCoRERGRF6jfU/Xr3WZdwBlUIiIiIvIpnEEVUursqzrS6iGRJtlfP9JKUhajrD1/zXWDRmGFqKToU6K4Mx/Iqv7EvZwiijs0QFalBwEWUVjXFbJqSDDKThUVJKv6c3jwZ6K4xOc6iOIcDRq4DqqQVcNCaKgoTB06JIqLmXKVbLkFRaIwR9s2orjTD74riisp8BfFNfx4gCgus9cHorjSCtkx1VhQIQoArhgf7Too+7Sord8KA0VxyUZZJSS/XrJ9tuEJ2efKyTJZuaEfJuaJ4uwqWBQXKPystQg/R2+JzxHFfXrIdWU8g/DJ7lDZ4Y4yYVXBMH/ZNnlgWLYo7tn59UVxkqX6wjyjQ3NA0zkPKu9BJSIiIiKqAbVmgHr69Gncd999sFqtCAsLw5AhQ1BYWFjtz3Tr1g2aplV5Pfzww5eox0RERFSXObz0nzccOHAAQ4YMQZMmTRAQEICmTZti8uTJKC8vrxK3fft23HDDDbBYLIiPj8fs2bPPaevdd99Fy5YtYbFYkJSUhM8+k101/KNac4n/vvvuw7Fjx7BmzRrYbDY88MADGDZsGFasWFHtzw0dOhRTp051fh0YKLt0RURERHQxalOaqd27d8PhcOCVV15Bs2bNsGPHDgwdOhRFRUWYM2cOACA/Px8333wzevbsiUWLFuHXX3/Fgw8+iLCwMAwbNgwA8MMPP+Cee+7BjBkzcNttt2HFihXo27cvfv75Z7RpI7vdB6glA9SMjAysXr0aP/74I66++moAwPz583Hrrbdizpw5qF//wvesBAYGIjbW9f07RERERHVVSkoKUlL+9zxIYmIiMjMzsXDhQucA9c0330R5eTmWLFkCk8mEK6+8Etu2bcPcuXOdA9R58+YhJSUFY8eOBQBMmzYNa9aswUsvvYRFixaJ+1MrLvGnp6cjLCzMOTgFgJ49e8JgMGDTpk3V/uybb76JyMhItGnTBmlpaSguLq42vqysDPn5+VVeRERERO6qTDOl9wvAOWOVsrIy3fufl5eHiIgI59fp6eno2rUrTCaT871evXohMzMTZ86cccb07NmzSju9evVCenq6W8uuFQPU7OxsREdXfULVz88PERERyM6+8JOB9957L/79739j3bp1SEtLwxtvvIH777+/2mXNmDEDoaGhzld8fLwu60BERESkl/j4+CrjlRkzZuja/t69ezF//nz8/e9/d76XnZ2NmJiYKnGVX1eOxy4UU9147Xxq9BL/+PHjMWvWrGpjMjIyPG6/croZAJKSkhAXF4cePXpg3759aNq06Xl/Ji0tDaNHj3Z+nZ+fz0EqERERuc0BOzTYdW8TAA4dOgSr1ep832w+f/o16VirZcuWzq+PHDmClJQU9O/fH0OHDtWh1+6r0QHqmDFjkJqaWm1MYmIiYmNjcfz48SrvV1RU4PTp027dX9q5c2cAZ/8quNAA1Ww2X3AnExEREfkCq9VaZYB6IdKxVqWjR4+ie/fu6NKlC1599dUqcbGxscjJqZq/t/LryvHYhWLcfR6oRgeoUVFRiIqKchmXnJyM3NxcbNmyBR07dgQAfP3113A4HM5Bp8S2bdsAAHFxcW731aE02JUscbErJXZZO6H++v7VddtM1wNv1bSRrLGK8w/wz+EnS+StnZYlEO/0TidRnCMgQLbckydlcafOiOIQK7trpsGi7qI4Jd1+tnKXMcoqS8APh+wJURUtSAwPQEkLBAj3GfxkH1sRr/5VFJc9eKUozvC6LAF/q3/IMoWoq4RFIqQVO3S8/+y+SbKiCWKhIaKwGycI7/kXFgj47fnqUxFWiiqSFfZoFFQiirv2ftn20xpGuA4C8Ei060ICb/09V9TWgFGy4gUwyH5PvTo7XBQnTcAfIByV2AQfU9JTx5sUlBdKnbq3YtKxFnB25rR79+7o2LEjli5dCoOh6u+05ORkPP3007DZbPD3P1v1Yc2aNWjRogXCw8OdMWvXrsWoUaOcP7dmzRokJye71e9acQ9qq1atkJKSgqFDh2Lz5s34/vvvMXLkSAwYMMD5BP+RI0fQsmVLbN68GQCwb98+TJs2DVu2bMGBAwewcuVKDBo0CF27dkXbtm1rcnWIiIiIfMqRI0fQrVs3JCQkYM6cOThx4gSys7Or3Dt67733wmQyYciQIdi5cyfefvttzJs3r8qtkY899hhWr16N5557Drt378YzzzyDn376CSNHjnSrP7UizRRw9mn8kSNHokePHjAYDOjXrx9efPFF5/dtNhsyMzOdT+mbTCZ89dVXeOGFF1BUVIT4+Hj069cPEyZMqKlVICIiojqkNpU6XbNmDfbu3Yu9e/eiYcOGVb6nfq/1Hhoaii+//BIjRoxAx44dERkZiUmTJlV55qdLly5YsWIFJkyYgKeeegrNmzfHRx995FYOVKAWDVAjIiKqTcrfuHFj5wYEzj7dtmHDhkvRNSIiIqJznH1ISt+L1Q6dH7qqlJqa6vJeVQBo27Ytvv3222pj+vfvj/79+19Uf2rFJX4iIiIiqjtqzQwqERERUe3i0P0hKXjpEr+v4QwqEREREfkUzqASEREReYFD2aH3XODZNi9/nEElIiIiIp/CGVQhB1zf9RHiJ7svJMAoS7KrIEuUbJSmsAgJch0TJIgBgFJZUnAlbE8FyxJ5Sxn2/SYMFP6NJoxT0ipkwvY0cfJ1wTFgF/7VLUyCjt+TNLskTcAvXdeCAlmcRZZ8PW7RX0RxDmFhgh+7rRLFdVpsE8VtG7ZTFNdhcj3XQdGCGAAIc12dBgBw6JgsLkh2DKhmjUVx36b+VxTXdU4TUdyXw4pFcf17nxLFaVcJ0+lIM8kL4gZMF7Z1QhaGerLP5GGjZMVOlF32e2rhi7LzzCb8/VjTlBfuQdX/nlbfxBlUIiIiIvIpnEElIiIi8gIFO5TOc4HKS3lQfQ0HqERERERe4BDdIOhJm5c/XuInIiIiIp/CGVQiIiIiL1BQXnhISvhAXC3HGVQiIiIi8imcQSUiIiLyAqXs4pSR7rRZF3AGlYiIiIh8CmdQhRxKnlNZ0pZEv/ebieK048JEyRHhroPKZcnDYRImaXfoe++NpmTtqYSGsgYr9P1LVLPJtl/Z9E9EceZxt4jiVIjrpNpaQb6srQhhMnfpvi0pkcVJCwQI1hUAHP98QxRneKyvbLnC9e38WgtRnPryR1Fch3lJojgcPe56mSHBoqa0U2dky4wUfKYAUKGyxP9anuwY7TpKeOwJP6cefjdO1l6BsKBIkfCYN5tkcRLlFbI4g3A274RsXzw3J0oUV2CTLbdc+Atyb57r9bU5an6mkU/xe44zqERERETkUziDSkREROQFZxP163wPKhP1ExEREZGnlHLon2ZKeKtbbcdL/ERERETkUziDSkREROQFfEjKc5xBJSIiIiKfwhlUIiIiIi9gon7PcQaViIiIiHwKZ1CJiIiIvEBB6f8UP3SqGuTjOEAVCvJTCDBWf1DYHLJp/BA/2cEqruZiNsvidK7qJKHZpdVNhJP50kpX0vak28RPWOVIyDzxDlGc8pdVmtFyXR8r2r6DsmV2bC+KE2+7gABZXFmZLM5P9rFlfFhWhUv74lvZcq9qKQpTVmHVpKuby9oLlVUv0nb95jom+4SoLVQIz1tp3woKZe1Jz7Omwkpxh7JlcZFhsjibztWacmXVml54wnVFrAKb7Dy7PlI2uFm6zyKKG5d0ShT3xm+yCnWz908TxdUP6+oyxqGE+4t8EgeoRERERF5wNg+q3veg1o2n+DlAJSIiIvIKuxcuyPMhKSIiIiKiS44zqERERERecPZyPC/xe4IzqERERETkUziDSkREROQFnEH1HGdQiYiIiMincAaViIiIyAsccEDTewZV58T/vooDVKFTZRosxuoPsoYBsoPmrqnCg6uwSBSmGgmTVksSpjt0ToghTdAdEiyLM7lOWA0AqBCm4dA5Ab84sb60gIEwKb2KjHIdExMrW6ZJtg4okh2fUMJjSpocXlogQBinrrlSFKfty5It96pwUZij5RWiOEO2LNm86tjaZYx2/KSoLRw6LltmfdkxpZWXy5b7m3AbR8uSviNIWCRCyiw8N6SfU8FBorC/Nd3lMibhkyWituxfLxfFRQz8URT3r+1horjWstMCqbETRHEldtfnt81Rhg/wjWzB5HM4QCUiIiLyAt6D6jkOUImIiIi8QCn9k+p7o01fxIekiIiIiMincAaViIiIyAsUFKDzQ03KC8VTfRFnUImIiIjIp3AGlYiIiMgLvPFAU115SIozqERERETkUziDSkREROQFnEH1HAeoQv6Gs6/qY4Q3LjcRJtY/lSsKEyfftoa4DFEG4aS6NIF8RIQoTrPJEnkrs0XWXoWwyIE0sb6wf+Ik8lLS/SGJkybMLymRxZnNsjibTRRmOChL0u5omySKU6GhojjtkGy5qlVTWXvZsiT3mkGWG1FJi1iUy7azSKmsLU1arEHYN/vWw6I4YwfZYtFAWJyirEwWt/OALC5EWCCgRLbc4lLXn1OOb6aJ2vrm7l9EcSEm2fH5VNtcUVybr+aJ4ma0nCqK+zy7wGVMhRLuV/JJHKASEREReYE3ypKy1CkREREReYyX+D3Hh6SIiIiIyKdwBpWIiIjICziD6jnOoBIRERGRT+EMKhEREZFXeGO2kzOoRERERESXHGdQiYiIiLyA96B6jgNUIUmi/mK7LLExsk8IF+ovi6uwy+LKXCeb18yyxPXSxNZKmrjeIUsirx2XJUGXJpHXKipEcSpAlnhbs8vak66vOFG/3gUCJIQJ+KVUgrCARV6eKEya+F9awMCR2ETWnvRYzsuXtSek5btOXI7TwmVe3VIWdzhHFhduFYUZr0qQtWcS/uo6eVoWd0D4udJCeIzmCvYFAMTKCplcMeiY6yC77PeAxU8W12Gc68IuALB1lmxd510pS8D/XU6pKK5HdLjLmFJ7KdL1Pc3oEuIAlYiIiMgLmKjfcxygEhEREXmBUgp6P9SkpGWrazk+JEVEREREPoUzqEREREReYQcgfD5FjDOoRERERESXHGdQiYiIiLzgbEoofWdQeQ8qEREREVEN4AwqERERkVfoP4NaV+5B5QBVaFnOXhi16hPnP1y/hagtQw9ZwmLHO6NEcYitJ4srKtYnBoCKlCWYNpwQFiXI+E223PatRHFaYaGsvaBAWXvFRaI4aMKLEsJKIFqRbD0kifqVUXi6S4sDmGRFHaTFFVRkpCjOsO1XWXutmovitKPZojhYLLL2du2RtWcNlrV3SNi/U4KE6Vc2lrW1fa8srv0VojD7R5tEcVqQrDiJoXs7URwOCRLcA4BBNoD4+WlZYYKrHpcVCpEUTwEAhAa5DFG/ygpTmA2yz7xFY2WfA6WOMFGc1HHkiuIqHK6Lp9jrxjjussUBKhEREZE3eOEeVGn1u9qO96ASERERkU/hDCoRERGRFygv3C/qjTZ9EQeoRERERF7Bh6Q8xUv8RERERORTat0AdcGCBWjcuDEsFgs6d+6MzZs3Vxv/7rvvomXLlrBYLEhKSsJnn312iXpKREREdZs6+1CTni8vzqDecccdSEhIgMViQVxcHAYOHIijR49Widm+fTtuuOEGWCwWxMfHY/bs2ee0o8fYq1YNUN9++22MHj0akydPxs8//4x27dqhV69eOH6BNDY//PAD7rnnHgwZMgRbt25F37590bdvX+zYseMS95yIiIjIt3Xv3h3vvPMOMjMz8f7772Pfvn3429/+5vx+fn4+br75ZjRq1AhbtmzBs88+i2eeeQavvvqqM0avsVetGqDOnTsXQ4cOxQMPPIDWrVtj0aJFCAwMxJIlS84bP2/ePKSkpGDs2LFo1aoVpk2bhquuugovvfTSJe45ERER1T1K9/+8OYP6+OOP49prr0WjRo3QpUsXjB8/Hhs3boTNZgMAvPnmmygvL8eSJUtw5ZVXYsCAAXj00Ucxd+5cZxt6jb1qzUNS5eXl2LJlC9LS0pzvGQwG9OzZE+np6ef9mfT0dIwePbrKe7169cJHH310weWUlZWhrKzM+XVeXh4AwK5sLvtYai91GXOW7ODKLy5zHQQARcLl+hllcQLKUiKK0+x2WYPCdVUFwuWWC9tz6HzzuiZsT5rHrkIYp2eifmmxAVOFrDnhPlMmWZEIg/B4VwWy9rRCWf8c+bJiDQbpeSvdH9L2SgRJ3wuFnxXSZQq3nb1UlpBeM8qOd4NwubpuOwCFFbLPs3zZoScviiFYDyVeB9nvgRLh77MynT9D7Uq2HmUO1/0rc5zdbjVfu947y8/Pz6/ytdlshtksLBIhcPr0abz55pvo0qUL/P3PFtFIT09H165dYfpDoZZevXph1qxZOHPmDMLDwz0ae51PrRmgnjx5Ena7HTExMVXej4mJwe7du8/7M9nZ2eeNz86+cGWWGTNmYMqUKee8vy/vI5d9HJfrMsQtYanz9W2QiIi87/ua7sDlb2uePPbUqVMIDQ31XmfOw2QyITY2ttrxxsUIDg5GfHx8lfcmT56MZ5555qLbHjduHF566SUUFxfj2muvxapVq5zfy87ORpMmTarEV46zsrOzER4e7tHY63xqzQD1UklLS6sy8s/NzUWjRo2QlZV1yQ/wmpSfn4/4+HgcOnQIVqu1prtzyXC9ud51Adeb610X5OXlISEhARERstLcerJYLNi/fz/Ky4XlbN2klIL2pyt2F5o9HT9+PGbNmlVtexkZGWjZsiUAYOzYsRgyZAgOHjyIKVOmYNCgQVi1atU5y/O2WjNAjYyMhNFoRE5O1VrIOTk5iI2NPe/PxMbGuhUPXHiKPDQ0tE6d2JWsVivXuw7hetctXO+6pa6ut0F6K4XOLBYLLBZLjSz7j8aMGYPU1NRqYxITE53/joyMRGRkJK644gq0atUK8fHx2LhxI5KTky84rgLgHFt5MvY6n1rzkJTJZELHjh2xdu1a53sOhwNr165FcnLyeX8mOTm5SjwArFmz5oLxRERERJeTqKgotGzZstrXH+8p/SPH7883VD6bk5ycjG+++cb50BRwdlzVokULhIeHO2P0GHvVmgEqAIwePRqvvfYali9fjoyMDAwfPhxFRUV44IEHAACDBg2q8hDVY489htWrV+O5557D7t278cwzz+Cnn37CyJEja2oViIiIiHzOpk2b8NJLL2Hbtm04ePAgvv76a9xzzz1o2rSpc3B57733wmQyYciQIdi5cyfefvttzJs3r8qtkbqNvVQtM3/+fJWQkKBMJpPq1KmT2rhxo/N7N954oxo8eHCV+HfeeUddccUVymQyqSuvvFJ9+umnbi2vtLRUTZ48WZWWlurR/VqD6831rgu43lzvuoDrXbfW21Pbt29X3bt3VxEREcpsNqvGjRurhx9+WB0+fLhK3C+//KKuv/56ZTabVYMGDdTMmTPPaetix15KKaUpVeP5F4iIiIiInGrVJX4iIiIiuvxxgEpEREREPoUDVCIiIiLyKRygEhEREZFPqfMD1OnTp6NLly4IDAxEWFiY6GeUUpg0aRLi4uIQEBCAnj17Ys+ePVViTp8+jfvuuw9WqxVhYWEYMmQICgsLvbAGnnG3fwcOHICmaed9vfvuu864833/rbfeuhSrJOLJfunWrds56/Twww9XicnKykLv3r0RGBiI6OhojB07FhUVslr1l4K763369Gk88sgjaNGiBQICApCQkIBHH30UeXlV6wv62v5esGABGjduDIvFgs6dO2Pz5s3Vxr/77rto2bIlLBYLkpKS8Nlnn1X5vuRc9wXurPdrr72GG264AeHh4QgPD0fPnj3PiU9NTT1nv6akpHh7NdzmznovW7bsnHX6cxL1y3F/n+/zS9M09O7d2xlTG/b3N998g9tvvx3169eHpmmiuu7r16/HVVddBbPZjGbNmmHZsmXnxLj7mUGXkCepCC4nkyZNUnPnzlWjR49WoaGhop+ZOXOmCg0NVR999JH65Zdf1B133KGaNGmiSkpKnDEpKSmqXbt2auPGjerbb79VzZo1U/fcc4+X1sJ97vavoqJCHTt2rMprypQpKjg4WBUUFDjjAKilS5dWifvjdqlpnuyXG2+8UQ0dOrTKOuXl5Tm/X1FRodq0aaN69uyptm7dqj777DMVGRmp0tLSvL06Yu6u96+//qruvPNOtXLlSrV37161du1a1bx5c9WvX78qcb60v9966y1lMpnUkiVL1M6dO9XQoUNVWFiYysnJOW/8999/r4xGo5o9e7batWuXmjBhgvL391e//vqrM0Zyrtc0d9f73nvvVQsWLFBbt25VGRkZKjU1VYWGhlZJJTN48GCVkpJSZb+ePn36Uq2SiLvrvXTpUmW1WqusU3Z2dpWYy3F/nzp1qso679ixQxmNRrV06VJnTG3Y35999pl6+umn1QcffKAAqA8//LDa+N9++00FBgaq0aNHq127dqn58+cro9GoVq9e7Yxxd1vSpVXnB6iVli5dKhqgOhwOFRsbq5599lnne7m5ucpsNqv//Oc/Simldu3apQCoH3/80Rnz+eefK03T1JEjR3Tvu7v06l/79u3Vgw8+WOU9yQdHTfF0vW+88Ub12GOPXfD7n332mTIYDFV+2S1cuFBZrVZVVlamS98vhl77+5133lEmk0nZbDbne760vzt16qRGjBjh/Nput6v69eurGTNmnDf+rrvuUr17967yXufOndXf//53pZTsXPcF7q73n1VUVKiQkBC1fPly53uDBw9Wffr00burunJ3vV19xteV/f3888+rkJAQVVhY6HyvNuzvP5J87jz55JPqyiuvrPLe3XffrXr16uX8+mK3JXlXnb/E7679+/cjOzsbPXv2dL4XGhqKzp07Iz09HQCQnp6OsLAwXH311c6Ynj17wmAwYNOmTZe8z3+mR/+2bNmCbdu2YciQIed8b8SIEYiMjESnTp2wZMkSKB9JtXsx6/3mm28iMjISbdq0QVpaGoqLi6u0m5SUhJiYGOd7vXr1Qn5+Pnbu3Kn/irhJr+MxLy8PVqsVfn5+Vd73hf1dXl6OLVu2VDkvDQYDevbs6Twv/yw9Pb1KPHB2v1XGS871mubJev9ZcXExbDYbIiIiqry/fv16REdHo0WLFhg+fDhOnTqla98vhqfrXVhYiEaNGiE+Ph59+vSpcn7Wlf29ePFiDBgwAEFBQVXe9+X97QlX57ce25K8y891CP1RdnY2AFQZjFR+Xfm97OxsREdHV/m+n58fIiIinDE1SY/+LV68GK1atUKXLl2qvD916lTcdNNNCAwMxJdffol//OMfKCwsxKOPPqpb/z3l6Xrfe++9aNSoEerXr4/t27dj3LhxyMzMxAcffOBs93zHQ+X3apoe+/vkyZOYNm0ahg0bVuV9X9nfJ0+ehN1uP+9+2L1793l/5kL77Y/nceV7F4qpaZ6s95+NGzcO9evXr/KLOiUlBXfeeSeaNGmCffv24amnnsItt9yC9PR0GI1GXdfBE56sd4sWLbBkyRK0bdsWeXl5mDNnDrp06YKdO3eiYcOGdWJ/b968GTt27MDixYurvO/r+9sTFzq/8/PzUVJSgjNnzlz0uUPedVkOUMePH49Zs2ZVG5ORkYGWLVteoh5dGtL1vlglJSVYsWIFJk6ceM73/vhehw4dUFRUhGeffdarAxZvr/cfB2VJSUmIi4tDjx49sG/fPjRt2tTjdi/Wpdrf+fn56N27N1q3bo1nnnmmyvdqYn+TfmbOnIm33noL69evr/LA0IABA5z/TkpKQtu2bdG0aVOsX78ePXr0qImuXrTk5GRnPXEA6NKlC1q1aoVXXnkF06ZNq8GeXTqLFy9GUlISOnXqVOX9y3F/U+13WQ5Qx4wZg9TU1GpjEhMTPWo7NjYWAJCTk4O4uDjn+zk5OWjfvr0z5vjx41V+rqKiAqdPn3b+vDdI1/ti+/fee++huLgYgwYNchnbuXNnTJs2DWVlZTCbzS7jPXGp1rtS586dAQB79+5F06ZNERsbe86Tnzk5OQBQ6/d3QUEBUlJSEBISgg8//BD+/v7Vxl+K/X0+kZGRMBqNzu1eKScn54LrGBsbW2285FyvaZ6sd6U5c+Zg5syZ+Oqrr9C2bdtqYxMTExEZGYm9e/f6xIDlYta7kr+/Pzp06IC9e/cCuPz3d1FREd566y1MnTrV5XJ8bX974kLnt9VqRUBAAIxG40UfQ+RlNX0TrK9w9yGpOXPmON/Ly8s770NSP/30kzPmiy++8LmHpDzt34033njO09wX8s9//lOFh4d73Fc96bVfvvvuOwVA/fLLL0qp/z0k9ccnP1955RVltVpVaWmpfivgIU/XOy8vT1177bXqxhtvVEVFRaJl1eT+7tSpkxo5cqTza7vdrho0aFDtQ1K33XZblfeSk5PPeUiqunPdF7i73kopNWvWLGW1WlV6erpoGYcOHVKapqmPP/74ovurF0/W+48qKipUixYt1OOPP66Uurz3t1Jnf8eZzWZ18uRJl8vwxf39RxA+JNWmTZsq791zzz3nPCR1MccQeVedH6AePHhQbd261ZkyaevWrWrr1q1VUie1aNFCffDBB86vZ86cqcLCwtTHH3+stm/frvr06XPeNFMdOnRQmzZtUt99951q3ry5z6WZqq5/hw8fVi1atFCbNm2q8nN79uxRmqapzz///Jw2V65cqV577TX166+/qj179qiXX35ZBQYGqkmTJnl9faTcXe+9e/eqqVOnqp9++knt379fffzxxyoxMVF17drV+TOVaaZuvvlmtW3bNrV69WoVFRXlc2mm3FnvvLw81blzZ5WUlKT27t1bJf1MRUWFUsr39vdbb72lzGazWrZsmdq1a5caNmyYCgsLc2ZXGDhwoBo/frwz/vvvv1d+fn5qzpw5KiMjQ02ePPm8aaZcnes1zd31njlzpjKZTOq9996rsl8rP/MKCgrUE088odLT09X+/fvVV199pa666irVvHlzn/iDq5K76z1lyhT1xRdfqH379qktW7aoAQMGKIvFonbu3OmMuRz3d6Xrr79e3X333ee8X1v2d0FBgfP3MwA1d+5ctXXrVnXw4EGllFLjx49XAwcOdMZXppkaO3asysjIUAsWLDhvmqnqtiXVrDo/QB08eLACcM5r3bp1zhj8nuuxksPhUBMnTlQxMTHKbDarHj16qMzMzCrtnjp1St1zzz0qODhYWa1W9cADD1QZ9NY0V/3bv3//OdtBKaXS0tJUfHy8stvt57T5+eefq/bt26vg4GAVFBSk2rVrpxYtWnTe2Jri7npnZWWprl27qoiICGU2m1WzZs3U2LFjq+RBVUqpAwcOqFtuuUUFBASoyMhINWbMmCrpmGqau+u9bt26854XANT+/fuVUr65v+fPn68SEhKUyWRSnTp1Uhs3bnR+78Ybb1SDBw+uEv/OO++oK664QplMJnXllVeqTz/9tMr3Jee6L3BnvRs1anTe/Tp58mSllFLFxcXq5ptvVlFRUcrf3181atRIDR061Cd/abuz3qNGjXLGxsTEqFtvvVX9/PPPVdq7HPe3Ukrt3r1bAVBffvnlOW3Vlv19oc+kynUdPHiwuvHGG8/5mfbt2yuTyaQSExOr/B6vVN22pJqlKeUjOYCIiIiIiMBSp0RERETkYzhAJSIiIiKfwgEqEREREfkUDlCJiIiIyKdwgEpEREREPoUDVCIiIiLyKRygEhEREZFP4QCViIiIiHwKB6hERDUkNTUVffv2rTZm/fr10DQNubm5l6RPRES+gJWkiOgcqampyM3NxUcffVTTXanWgQMH0KRJE2zduhXt27ev6e64LS8vD0ophIWFAQC6deuG9u3b44UXXnDGlJeX4/Tp04iJiYGmaTXTUSKiS8yvpjtARFRXhYaGuowxmUyIjY29BL0hIvIdvMRPRC5169YNjzzyCEaNGoXw8HDExMTgtddeQ1FRER544AGEhISgWbNm+Pzzz50/U3lp+tNPP0Xbtm1hsVhw7bXXYseOHc6YU6dO4Z577kGDBg0QGBiIpKQk/Oc//6mybIfDgdmzZ6NZs2Ywm81ISEjA9OnTAQBNmjQBAHTo0AGapqFbt27n7f/UqVNRv359nDp1yvle79690b17dzgcjvP+zIEDB6BpGt566y106dIFFosFbdq0wYYNG6rEbdiwAZ06dYLZbEZcXBzGjx+PiooK5/ffe+89JCUlISAgAPXq1UPPnj1RVFQEoOol/tTUVGzYsAHz5s2DpmnQNA0HDhw47yX+999/H1deeSXMZjMaN26M5557rkqfGjdujH/961948MEHERISgoSEBLz66qvnXU8iIl/EASoRiSxfvhyRkZHYvHkzHnnkEQwfPhz9+/dHly5d8PPPP+Pmm2/GwIEDUVxcXOXnxo4di+eeew4//vgjoqKicPvtt8NmswEASktL0bFjR3z66afYsWMHhg0bhoEDB2Lz5s3On09LS8PMmTMxceJE7Nq1CytWrEBMTAwAOOO++uorHDt2DB988MF5+/7000+jcePGeOihhwAACxYswA8//IDly5fDYKj+Y3Ds2LEYM2YMtm7diuTkZNx+++3Oge6RI0dw66234pprrsEvv/yChQsXYvHixfjnP/8JADh27BjuuecePPjgg8jIyMD69etx55134nx3Vs2bNw/JyckYOnQojh07hmPHjiE+Pv6cuC1btuCuu+7CgAED8Ouvv+KZZ57BxIkTsWzZsipxzz33HK6++mps3boV//jHPzB8+HBkZmZWu65ERD5DERH9yeDBg1WfPn2cX994443q+uuvd35dUVGhgoKC1MCBA53vHTt2TAFQ6enpSiml1q1bpwCot956yxlz6tQpFRAQoN5+++0LLrt3795qzJgxSiml8vPzldlsVq+99tp5Y/fv368AqK1bt7pcp3379qmQkBA1btw4FRAQoN58881q4yvbnjlzpvM9m82mGjZsqGbNmqWUUuqpp55SLVq0UA6HwxmzYMECFRwcrOx2u9qyZYsCoA4cOHDeZZxvOz/22GNVYiq345kzZ5RSSt17773qL3/5S5WYsWPHqtatWzu/btSokbr//vudXzscDhUdHa0WLlxY7ToTEfkKzqASkUjbtm2d/zYajahXrx6SkpKc71XOah4/frzKzyUnJzv/HRERgRYtWiAjIwMAYLfbMW3aNCQlJSEiIgLBwcH44osvkJWVBQDIyMhAWVkZevTocdH9T0xMxJw5czBr1izccccduPfee53fe/jhhxEcHOx8Xaj/fn5+uPrqq539z8jIQHJycpWHl6677joUFhbi8OHDaNeuHXr06IGkpCT0798fr732Gs6cOXNR65GRkYHrrruuynvXXXcd9uzZA7vd7nzvj/tL0zTExsaes2+IiHwVB6hEJOLv71/la03TqrxXOUi70D2d5/Pss89i3rx5GDduHNatW4dt27ahV69eKC8vBwAEBATo0PP/+eabb2A0GnHgwIEq94lOnToV27Ztc770YjQasWbNGnz++edo3bo15s+fjxYtWmD//v26LeNCzre/3Nk3REQ1iQNUIvKqjRs3Ov995swZ/Pe//0WrVq0AAN9//z369OmD+++/H+3atUNiYiL++9//OuObN2+OgIAArF279rxtm0wmAKgyc3ghb7/9Nj744AOsX78eWVlZmDZtmvN70dHRaNasmfN1of5XVFRgy5Ytzv63atUK6enpVe4p/f777xESEoKGDRsCODswvO666zBlyhRs3boVJpMJH3744QXXx9W6tGrVCt9//32V977//ntcccUVMBqNLrcDEVFtwDRTRORVU6dORb169RATE4Onn34akZGRzifXmzdvjvfeew8//PADwsPDMXfuXOTk5KB169YAAIvFgnHjxuHJJ5+EyWTCddddhxMnTmDnzp0YMmQIoqOjERAQgNWrV6Nhw4awWCznTd10+PBhDB8+HLNmzcL111+PpUuX4rbbbsMtt9yCa6+9ttr+L1iwAM2bN0erVq3w/PPP48yZM3jwwQcBAP/4xz/wwgsv4JFHHsHIkSORmZmJyZMnY/To0TAYDNi0aRPWrl2Lm2++GdHR0di0aRNOnDjhHOD+WePGjbFp0yYcOHAAwcHBiIiIOCdmzJgxuOaaazBt2jTcfffdSE9Px0svvYSXX37Znd1CROTTOINKRF41c+ZMPPbYY+jYsSOys7PxySefOGc+J0yYgKuuugq9evVCt27dEBsbe05lpYkTJ2LMmDGYNGkSWrVqhbvvvtt5L6Wfnx9efPFFvPLKK6hfvz769OlzzvKVUkhNTUWnTp0wcuRIAECvXr0wfPhw3H///SgsLHTZ/5kzZ6Jdu3b47rvvsHLlSkRGRgIAGjRogM8++wybN29Gu3bt8PDDD2PIkCGYMGECAMBqteKbb77BrbfeiiuuuAITJkzAc889h1tuueW8y3riiSdgNBrRunVrREVFOe/F/aOrrroK77zzDt566y20adMGkyZNwtSpU5GamlrtehAR1SasJEVEXrF+/Xp0794dZ86ccVZKqk1qe5UqIqLajDOoRERERORTOEAlIiIiIp/CS/xERERE5FM4g0pEREREPoUDVCIiIiLyKRygEhEREZFP4QCViIiIiHwKB6hERERE5FM4QCUiIiIin8IBKhERERH5FA5QiYiIiMin/D/TpNno3BAemQAAAABJRU5ErkJggg==\n",
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"from ribs.visualize import grid_archive_heatmap\n",
"\n",
"plt.figure(figsize=(8, 6))\n",
"grid_archive_heatmap(archive, vmin=-300, vmax=300)\n",
"plt.gca().invert_yaxis() # Makes more sense if larger velocities are on top.\n",
"plt.ylabel(\"Impact y-velocity\")\n",
"plt.xlabel(\"Impact x-position\")"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "8sHKT6nJFKhq"
},
"source": [
"From this heatmap, we can make a few observations:\n",
"\n",
"- CMA-ME found solutions for almost all cells in the archive (empty cells show up as white).\n",
"- Most of the high-performing solutions have lower impact $y$-velocities (see the bright area at the bottom of the map). This is reasonable, as a lander that crashes into the ground would not do well.\n",
"- The high-performing solutions are spread across a wide range of impact $x$-positions. The highest solutions seem to be at $x \\approx 0$ (the bright spot in the middle). This makes sense since an impact $x$-position of 0 corresponds to the direct vertical approach. Nevertheless, there are many high-performing solutions that had other $x$-positions, and we will visualize them in the next section."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "cG6GDOpyFKhr"
},
"source": [
"## Visualizing Individual Trajectories\n",
"\n",
"To view the trajectories for different models, we can use the [RecordVideo](https://gymnasium.farama.org/api/wrappers/misc_wrappers/#gymnasium.wrappers.RecordVideo) wrapper, and we can use IPython to display the video in the notebook. The `display_video` function below shows how to do this."
]
},
{
"cell_type": "code",
"execution_count": 13,
"metadata": {
"id": "vT-xb4myFKhr"
},
"outputs": [],
"source": [
"import base64\n",
"import glob\n",
"import io\n",
"\n",
"from IPython.display import display, HTML\n",
"\n",
"\n",
"def display_video(model):\n",
" \"\"\"Displays a video of the model in the environment.\"\"\"\n",
"\n",
" video_env = gym.wrappers.RecordVideo(\n",
" gym.make(\"LunarLander-v2\", render_mode=\"rgb_array\"),\n",
" video_folder=\"videos\",\n",
" # This will ensure all episodes are recorded as videos.\n",
" episode_trigger=lambda idx: True,\n",
" # Disables moviepy's logger to reduce clutter in the output.\n",
" disable_logger=True,\n",
" )\n",
" simulate(model, env_seed, video_env)\n",
" video_env.close() # Save video.\n",
"\n",
" # Display the video with HTML. Though we use glob, there is only 1 video.\n",
" for video_file in glob.glob(\"videos/*.mp4\"):\n",
" video = io.open(video_file, 'rb').read()\n",
" encoded = base64.b64encode(video).decode(\"ascii\")\n",
" display(\n",
" HTML(f'''\n",
" '''))"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "t2QPnuqgFKhr"
},
"source": [
"We can retrieve policies with measures that are close to a query with the [`retrieve_single`](https://docs.pyribs.org/en/latest/api/ribs.archives.GridArchive.html#ribs.archives.GridArchive.retrieve_single) method. This method will look up the cell corresponding to the queried measures. Then, the method will check if there is an elite in that cell, and return the elite if it exists (the method does not check neighboring cells for elites). The returned elite may not have the exact measures requested because the elite only has to be in the same cell as the queried measures.\n",
"\n",
"Below, we first retrieve a policy that impacted the ground on the left (approximately -0.4) with low velocity (approximately -0.10) by querying for `[-0.4, -0.10]`."
]
},
{
"cell_type": "code",
"execution_count": 14,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 296
},
"id": "FxTO1P3tFKhs",
"outputId": "24c33f76-f36d-4bb2-ce54-3d4c1b253df4"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Objective: 270.44571144188967\n",
"Measures: (x-pos: -0.3952582776546478, y-vel: -0.08481719344854355)\n"
]
},
{
"data": {
"text/html": [
"\n",
" "
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"occupied, elite = archive.retrieve_single([-0.4, -0.10])\n",
"# `occupied` indicates if there was an elite in the corresponding cell.\n",
"if occupied:\n",
" print(f\"Objective: {elite['objective']}\")\n",
" print(f\"Measures: (x-pos: {elite['measures'][0]}, y-vel: {elite['measures'][1]})\")\n",
" display_video(elite[\"solution\"])"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "JA3KJCMLFKhs"
},
"source": [
"We can also find a policy that impacted the ground on the right (0.6) with low velocity.\n",
"\n",
"**Note: Batch and Single Methods**\n",
"\n",
"> `retrieve_single` returns an elite represented as a dict, given a single `measures` array. Meanwhile, the [`retrieve`](https://docs.pyribs.org/en/latest/api/ribs.archives.GridArchive.html#ribs.archives.GridArchive.retrieve) method takes in a _batch_ of measures (named `measures_batch`) and returns a dict holding batches of data. Several archive methods in pyribs follow a similar pattern of having a batch and single version, e.g., [`add`](https://docs.pyribs.org/en/latest/api/ribs.archives.GridArchive.html#ribs.archives.GridArchive.add) and [`add_single`](https://docs.pyribs.org/en/latest/api/ribs.archives.GridArchive.html#ribs.archives.GridArchive.add_single)."
]
},
{
"cell_type": "code",
"execution_count": 15,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 351
},
"id": "0lgiOQ2sFKht",
"outputId": "0775eafb-71ca-4941-8211-038ec66518bb"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Objective: 204.3844548611375\n",
"Measures: (x-pos: 0.6270830035209656, y-vel: -0.07091592252254486)\n"
]
},
{
"data": {
"text/html": [
"\n",
" "
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"occupied, elite = archive.retrieve_single([0.6, -0.10])\n",
"if occupied:\n",
" print(f\"Objective: {elite['objective']}\")\n",
" print(f\"Measures: (x-pos: {elite['measures'][0]}, y-vel: {elite['measures'][1]})\")\n",
" display_video(elite[\"solution\"])"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "OuP_5VVDFKht"
},
"source": [
"And we can find a policy that executes a regular vertical landing, which happens when the impact $x$-position is around 0."
]
},
{
"cell_type": "code",
"execution_count": 16,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 296
},
"id": "v4nAwfmXFKht",
"outputId": "34bf9175-5014-4561-acbb-48603f159e88"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Objective: 234.53572545577555\n",
"Measures: (x-pos: 0.005561447236686945, y-vel: -0.06290675699710846)\n"
]
},
{
"data": {
"text/html": [
"\n",
" "
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"occupied, elite = archive.retrieve_single([0.0, -0.10])\n",
"if occupied:\n",
" print(f\"Objective: {elite['objective']}\")\n",
" print(f\"Measures: (x-pos: {elite['measures'][0]}, y-vel: {elite['measures'][1]})\")\n",
" display_video(elite[\"solution\"])"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "MZLDoutPFKhu"
},
"source": [
"As the archive has ~2500 solutions, we cannot view them all, but we can filter for high-performing solutions. We first retrieve the archive's elites with the [`data`](https://docs.pyribs.org/en/latest/api/ribs.archives.GridArchive.html#ribs.archives.GridArchive.data) method with `return_type=\"pandas\"`. Then, we choose solutions that scored above 200 because 200 is the [threshold for the problem to be considered solved](https://gymnasium.farama.org/environments/box2d/lunar_lander/). Note that many high-performing solutions do not land on the landing pad."
]
},
{
"cell_type": "code",
"execution_count": 17,
"metadata": {
"id": "0rHgh4fgFKhu"
},
"outputs": [],
"source": [
"df = archive.data(return_type=\"pandas\")\n",
"high_perf_sols = df.query(\"objective > 200\").sort_values(\"objective\", ascending=False)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "0_RYE1rTFKhu"
},
"source": [
"Below we visualize several of these high-performing solutions. The `iterelites` method is available because `data` returns an [`ArchiveDataFrame`](https://docs.pyribs.org/en/latest/api/ribs.archives.ArchiveDataFrame.html), a subclass of the Pandas DataFrame specialized for pyribs. `iterelites` iterates over the entries in the DataFrame and returns them as dicts."
]
},
{
"cell_type": "code",
"execution_count": 18,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 909
},
"id": "brTepWvkFKhv",
"outputId": "99083600-3fff-485c-df03-ffbe94aef2c8"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Objective: 312.4166112115108\n",
"Measures: (x-pos: -0.04941272735595703, y-vel: -0.42066681385040283)\n"
]
},
{
"data": {
"text/html": [
"\n",
" "
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Objective: 257.361619802296\n",
"Measures: (x-pos: -0.5883761644363403, y-vel: -0.11900511384010315)\n"
]
},
{
"data": {
"text/html": [
"\n",
" "
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Objective: 200.29196229839343\n",
"Measures: (x-pos: 0.4815877079963684, y-vel: -0.289840966463089)\n"
]
},
{
"data": {
"text/html": [
"\n",
" "
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"if len(high_perf_sols) > 0:\n",
" for elite in high_perf_sols.iloc[[0, len(high_perf_sols) // 2, -1]].iterelites():\n",
" print(f\"Objective: {elite['objective']}\")\n",
" print(f\"Measures: (x-pos: {elite['measures'][0]}, y-vel: {elite['measures'][1]})\")\n",
" display_video(elite['solution'])"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "P3cQJ2ctOBG5"
},
"source": [
"And finally, the [`best_elite`](https://docs.pyribs.org/en/latest/api/ribs.archives.GridArchive.html#ribs.archives.GridArchive.best_elite) property is the elite that has the highest performance in the archive."
]
},
{
"cell_type": "code",
"execution_count": 19,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 296
},
"id": "4OTjcg1XN_xz",
"outputId": "7a3eb291-2ca5-40f4-8572-a969f9babfa1"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Objective: 312.4166112115108\n",
"Measures: (x-pos: -0.04941272735595703, y-vel: -0.42066681385040283)\n"
]
},
{
"data": {
"text/html": [
"\n",
" "
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"print(f\"Objective: {archive.best_elite['objective']}\")\n",
"print(f\"Measures: (x-pos: {archive.best_elite['measures'][0]}, y-vel: {archive.best_elite['measures'][1]})\")\n",
"display_video(archive.best_elite['solution'])"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "ooGfhHCuFKhv"
},
"source": [
"## Conclusion\n",
"\n",
"As the saying goes, \"there is more than one way to land an airplane\" 😉. However, it is often difficult to shape the reward function in reinforcement learning to discover these unique and \"creative\" solutions. In such cases, a QD algorithm can help search for solutions that vary in \"interestingness.\"\n",
"\n",
"In this tutorial, we showed that this is the case for the lunar lander environment. Using CMA-ME, we searched for lunar lander trajectories with differing impact measures. Though these trajectories all take a different approach, many perform well.\n",
"\n",
"For extending this tutorial, we suggest the following:\n",
"\n",
"- Replace the impact measures with your own measures. What other properties might be interesting for the lunar lander?\n",
"- Try different terrains by changing the seed. For instance, if the environment has valleys, can the lander learn to go into this valley and glide back up?\n",
"- Try increasing the number of iterations. Will CMA-ME create a better archive if it gets to evaluate more solutions?\n",
"- Use other gym environments. What measures could you use in an environment like `BipedalWalker-v2`?\n",
"\n",
"Finally, to learn about an algorithm which performs even better on QD problems, check out the [CMA-MAE tutorial](https://docs.pyribs.org/en/latest/tutorials/cma_mae.html).\n",
"\n",
"And for a version of this tutorial that uses [Dask](https://dask.org) to parallelize evaluations and offers features like a command-line interface, refer to the [Lunar Lander example](https://docs.pyribs.org/en/latest/examples/lunar_lander.html)."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "7H8B12jOFKhw"
},
"source": [
"## Citation\n",
"\n",
"If you find this tutorial useful, please cite it as:\n",
"\n",
"```text\n",
"@article{pyribs_lunar_lander,\n",
" title = {Using CMA-ME to Land a Lunar Lander Like a Space Shuttle},\n",
" author = {Bryon Tjanaka and Sam Sommerer and Nikitas Klapsis and Matthew C. Fontaine and Stefanos Nikolaidis},\n",
" journal = {pyribs.org},\n",
" year = {2021},\n",
" url = {https://docs.pyribs.org/en/stable/tutorials/lunar_lander.html}\n",
"}\n",
"```"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "5kAdvjZiFKhw"
},
"source": [
"## Credits\n",
"\n",
"This tutorial is based on a [poster](https://web.archive.org/web/20220817214422/https://1l7puj10vwe3zflo2jsktkit-wpengine.netdna-ssl.com/wp-content/uploads/2020/08/S20-Klapsis-Poster.pdf) created by Nikitas Klapsis as part of USC's 2020 SHINE program."
]
}
],
"metadata": {
"colab": {
"provenance": []
},
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.8.17"
}
},
"nbformat": 4,
"nbformat_minor": 4
}