deep-learning-with-python-notebooks/3.7-predicting-house-prices.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "原文代码作者：François Chollet\n",
    "\n",
    "github：https://github.com/fchollet/deep-learning-with-python-notebooks\n",
    "\n",
    "中文注释制作：黄海广\n",
    "\n",
    "github：https://github.com/fengdu78\n",
    "\n",
    "代码全部测试通过。\n",
    "\n",
    "配置环境：keras 2.2.1（原文是2.0.8，运行结果一致），tensorflow 1.8，python 3.6，\n",
    "\n",
    "主机：显卡：一块1080ti；内存：32g（注：绝大部分代码不需要GPU）\n",
    "![公众号](data/gongzhong.jpg)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\ProgramData\\Anaconda3\\lib\\site-packages\\h5py\\__init__.py:36: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.\n",
      "  from ._conv import register_converters as _register_converters\n",
      "Using TensorFlow backend.\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "'2.1.5'"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import keras\n",
    "keras.__version__"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Predicting house prices: a regression example\n",
    "# 预测房价：回归问题\n",
    "\n",
    "This notebook contains the code samples found in Chapter 3, Section 6 of [Deep Learning with Python](https://www.manning.com/books/deep-learning-with-python?a_aid=keras&a_bid=76564dff). Note that the original text features far more content, in particular further explanations and figures: in this notebook, you will only find source code and related comments.\n",
    "\n",
    "----\n",
    "\n",
    "\n",
    "In our two previous examples, we were considering classification problems, where the goal was to predict a single discrete label of an \n",
    "input data point. Another common type of machine learning problem is \"regression\", which consists of predicting a continuous value instead \n",
    "of a discrete label. For instance, predicting the temperature tomorrow, given meteorological data, or predicting the time that a \n",
    "software project will take to complete, given its specifications.\n",
    "\n",
    "Do not mix up \"regression\" with the algorithm \"logistic regression\": confusingly, \"logistic regression\" is not a regression algorithm, \n",
    "it is a classification algorithm.\n",
    "\n",
    "\n",
    "前面两个例子都是分类问题，其目标是预测输入数据点所对应的单一离散的标签。另一种常见的机器学习问题是回归问题，它预测一个连续值而不是离散的标签，例如，根据气象数据预测明天的气温，或者根据软件说明书预测完成软件项目所需要的时间。\n",
    "\n",
    "注意：不要将回归问题与 logistic 回归算法混为一谈。令人困惑的是，logistic 回归不是回归算法，而是分类算法。\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## The Boston Housing Price dataset\n",
    "\n",
    "\n",
    "We will be attempting to predict the median price of homes in a given Boston suburb in the mid-1970s, given a few data points about the \n",
    "suburb at the time, such as the crime rate, the local property tax rate, etc.\n",
    "\n",
    "The dataset we will be using has another interesting difference from our two previous examples: it has very few data points, only 506 in \n",
    "total, split between 404 training samples and 102 test samples, and each \"feature\" in the input data (e.g. the crime rate is a feature) has \n",
    "a different scale. For instance some values are proportions, which take a values between 0 and 1, others take values between 1 and 12, \n",
    "others between 0 and 100...\n",
    "\n",
    "Let's take a look at the data:\n",
    "\n",
    "## 波士顿房价数据集\n",
    "\n",
    "本节将要预测 20 世纪 70 年代中期波士顿郊区房屋价格的中位数，已知当时郊区的一些数据点，比如犯罪率、当地房产税率等。本节用到的数据集与前面两个例子有一个有趣的区别。它包含的数据点相对较少，只有 506 个，分为 404 个训练样本和 102 个测试样本。输入数据的每个特征（比如犯罪率）都有不同的取值范围。例如，有些特性是比例，取值范围为 0~1；有的取值范围为 1~12；还有的取值范围为 0~100，等等。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "from keras.datasets import boston_housing\n",
    "\n",
    "(train_data, train_targets), (test_data, test_targets) =  boston_housing.load_data()#这里下载有时候会失败，多试几次，不需要翻墙"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(404, 13)"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_data.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(102, 13)"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test_data.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "As you can see, we have 404 training samples and 102 test samples. The data comprises 13 features. The 13 features in the input data are as \n",
    "follow:\n",
    "\n",
    "1. Per capita crime rate.\n",
    "2. Proportion of residential land zoned for lots over 25,000 square feet.\n",
    "3. Proportion of non-retail business acres per town.\n",
    "4. Charles River dummy variable (= 1 if tract bounds river; 0 otherwise).\n",
    "5. Nitric oxides concentration (parts per 10 million).\n",
    "6. Average number of rooms per dwelling.\n",
    "7. Proportion of owner-occupied units built prior to 1940.\n",
    "8. Weighted distances to five Boston employment centres.\n",
    "9. Index of accessibility to radial highways.\n",
    "10. Full-value property-tax rate per $10,000.\n",
    "11. Pupil-teacher ratio by town.\n",
    "12. 1000 * (Bk - 0.63) ** 2 where Bk is the proportion of Black people by town.\n",
    "13. % lower status of the population.\n",
    "\n",
    "The targets are the median values of owner-occupied homes, in thousands of dollars:\n",
    "\n",
    "如你所见，我们有 404 个训练样本和 102 个测试样本，每个样本都有 13 个数值特征，比如人均犯罪率、每个住宅的平均房间数、高速公路可达性等。\n",
    "\n",
    "目标是房屋价格的中位数，单位是千美元。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([15.2, 42.3, 50. , 21.1, 17.7, 18.5, 11.3, 15.6, 15.6, 14.4, 12.1,\n",
       "       17.9, 23.1, 19.9, 15.7,  8.8, 50. , 22.5, 24.1, 27.5, 10.9, 30.8,\n",
       "       32.9, 24. , 18.5, 13.3, 22.9, 34.7, 16.6, 17.5, 22.3, 16.1, 14.9,\n",
       "       23.1, 34.9, 25. , 13.9, 13.1, 20.4, 20. , 15.2, 24.7, 22.2, 16.7,\n",
       "       12.7, 15.6, 18.4, 21. , 30.1, 15.1, 18.7,  9.6, 31.5, 24.8, 19.1,\n",
       "       22. , 14.5, 11. , 32. , 29.4, 20.3, 24.4, 14.6, 19.5, 14.1, 14.3,\n",
       "       15.6, 10.5,  6.3, 19.3, 19.3, 13.4, 36.4, 17.8, 13.5, 16.5,  8.3,\n",
       "       14.3, 16. , 13.4, 28.6, 43.5, 20.2, 22. , 23. , 20.7, 12.5, 48.5,\n",
       "       14.6, 13.4, 23.7, 50. , 21.7, 39.8, 38.7, 22.2, 34.9, 22.5, 31.1,\n",
       "       28.7, 46. , 41.7, 21. , 26.6, 15. , 24.4, 13.3, 21.2, 11.7, 21.7,\n",
       "       19.4, 50. , 22.8, 19.7, 24.7, 36.2, 14.2, 18.9, 18.3, 20.6, 24.6,\n",
       "       18.2,  8.7, 44. , 10.4, 13.2, 21.2, 37. , 30.7, 22.9, 20. , 19.3,\n",
       "       31.7, 32. , 23.1, 18.8, 10.9, 50. , 19.6,  5. , 14.4, 19.8, 13.8,\n",
       "       19.6, 23.9, 24.5, 25. , 19.9, 17.2, 24.6, 13.5, 26.6, 21.4, 11.9,\n",
       "       22.6, 19.6,  8.5, 23.7, 23.1, 22.4, 20.5, 23.6, 18.4, 35.2, 23.1,\n",
       "       27.9, 20.6, 23.7, 28. , 13.6, 27.1, 23.6, 20.6, 18.2, 21.7, 17.1,\n",
       "        8.4, 25.3, 13.8, 22.2, 18.4, 20.7, 31.6, 30.5, 20.3,  8.8, 19.2,\n",
       "       19.4, 23.1, 23. , 14.8, 48.8, 22.6, 33.4, 21.1, 13.6, 32.2, 13.1,\n",
       "       23.4, 18.9, 23.9, 11.8, 23.3, 22.8, 19.6, 16.7, 13.4, 22.2, 20.4,\n",
       "       21.8, 26.4, 14.9, 24.1, 23.8, 12.3, 29.1, 21. , 19.5, 23.3, 23.8,\n",
       "       17.8, 11.5, 21.7, 19.9, 25. , 33.4, 28.5, 21.4, 24.3, 27.5, 33.1,\n",
       "       16.2, 23.3, 48.3, 22.9, 22.8, 13.1, 12.7, 22.6, 15. , 15.3, 10.5,\n",
       "       24. , 18.5, 21.7, 19.5, 33.2, 23.2,  5. , 19.1, 12.7, 22.3, 10.2,\n",
       "       13.9, 16.3, 17. , 20.1, 29.9, 17.2, 37.3, 45.4, 17.8, 23.2, 29. ,\n",
       "       22. , 18. , 17.4, 34.6, 20.1, 25. , 15.6, 24.8, 28.2, 21.2, 21.4,\n",
       "       23.8, 31. , 26.2, 17.4, 37.9, 17.5, 20. ,  8.3, 23.9,  8.4, 13.8,\n",
       "        7.2, 11.7, 17.1, 21.6, 50. , 16.1, 20.4, 20.6, 21.4, 20.6, 36.5,\n",
       "        8.5, 24.8, 10.8, 21.9, 17.3, 18.9, 36.2, 14.9, 18.2, 33.3, 21.8,\n",
       "       19.7, 31.6, 24.8, 19.4, 22.8,  7.5, 44.8, 16.8, 18.7, 50. , 50. ,\n",
       "       19.5, 20.1, 50. , 17.2, 20.8, 19.3, 41.3, 20.4, 20.5, 13.8, 16.5,\n",
       "       23.9, 20.6, 31.5, 23.3, 16.8, 14. , 33.8, 36.1, 12.8, 18.3, 18.7,\n",
       "       19.1, 29. , 30.1, 50. , 50. , 22. , 11.9, 37.6, 50. , 22.7, 20.8,\n",
       "       23.5, 27.9, 50. , 19.3, 23.9, 22.6, 15.2, 21.7, 19.2, 43.8, 20.3,\n",
       "       33.2, 19.9, 22.5, 32.7, 22. , 17.1, 19. , 15. , 16.1, 25.1, 23.7,\n",
       "       28.7, 37.2, 22.6, 16.4, 25. , 29.8, 22.1, 17.4, 18.1, 30.3, 17.5,\n",
       "       24.7, 12.6, 26.5, 28.7, 13.3, 10.4, 24.4, 23. , 20. , 17.8,  7. ,\n",
       "       11.8, 24.4, 13.8, 19.4, 25.2, 19.4, 19.4, 29.1])"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_targets"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "The prices are typically between \\$10,000 and \\$50,000. If that sounds cheap, remember this was the mid-1970s, and these prices are not \n",
    "inflation-adjusted.\n",
    "\n",
    "房价大都在 10 000~50 000 美元。如果你觉得这很便宜，不要忘记当时是 20 世纪 70 年代中 期，而且这些价格没有根据通货膨胀进行调整。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Preparing the data\n",
    "\n",
    "\n",
    "It would be problematic to feed into a neural network values that all take wildly different ranges. The network might be able to \n",
    "automatically adapt to such heterogeneous data, but it would definitely make learning more difficult. A widespread best practice to deal \n",
    "with such data is to do feature-wise normalization: for each feature in the input data (a column in the input data matrix), we \n",
    "will subtract the mean of the feature and divide by the standard deviation, so that the feature will be centered around 0 and will have a \n",
    "unit standard deviation. This is easily done in Numpy:\n",
    "\n",
    "## 准备数据\n",
    "\n",
    "将取值范围差异很大的数据输入到神经网络中，这是有问题的。网络可能会自动适应这种取值范围不同的数据，但学习肯定变得更加困难。对于这种数据，普遍采用的最佳实践是对每个特征做标准化，即对于输入数据的每个特征（输入数据矩阵中的列），减去特征平均值，再除以标准差，这样得到的特征平均值为 0，标准差为 1。用 Numpy 可以很容易实现标准化。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "mean = train_data.mean(axis=0)\n",
    "train_data -= mean\n",
    "std = train_data.std(axis=0)\n",
    "train_data /= std\n",
    "\n",
    "test_data -= mean\n",
    "test_data /= std"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "Note that the quantities that we use for normalizing the test data have been computed using the training data. We should never use in our \n",
    "workflow any quantity computed on the test data, even for something as simple as data normalization.\n",
    "\n",
    "注意，用于测试数据标准化的均值和标准差都是在训练数据上计算得到的。在工作流程中，你不能使用在测试数据上计算得到的任何结果，即使是像数据标准化这么简单的事情也不行。\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Building our network\n",
    "\n",
    "\n",
    "Because so few samples are available, we will be using a very small network with two \n",
    "hidden layers, each with 64 units. In general, the less training data you have, the worse overfitting will be, and using \n",
    "a small network is one way to mitigate overfitting.\n",
    "\n",
    "## 构建网络\n",
    "\n",
    "由于样本数量很少，我们将使用一个非常小的网络，其中包含两个隐藏层，每层有 64 个单元。一般来说，训练数据越少，过拟合会越严重，而较小的网络可以降低过拟合。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "from keras import models\n",
    "from keras import layers\n",
    "\n",
    "def build_model():\n",
    "    # Because we will need to instantiate the same model multiple times,（因为需要将同一个模型多次实例化，）\n",
    "    # we use a function to construct it.（所以用一个函数来构建模型）\n",
    "    model = models.Sequential()\n",
    "    model.add(layers.Dense(64, activation='relu',\n",
    "                           input_shape=(train_data.shape[1],)))\n",
    "    model.add(layers.Dense(64, activation='relu'))\n",
    "    model.add(layers.Dense(1))\n",
    "    model.compile(optimizer='rmsprop', loss='mse', metrics=['mae'])\n",
    "    return model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "Our network ends with a single unit, and no activation (i.e. it will be linear layer). \n",
    "This is a typical setup for scalar regression (i.e. regression where we are trying to predict a single continuous value). \n",
    "Applying an activation function would constrain the range that the output can take; for instance if \n",
    "we applied a `sigmoid` activation function to our last layer, the network could only learn to predict values between 0 and 1. Here, because \n",
    "the last layer is purely linear, the network is free to learn to predict values in any range.\n",
    "\n",
    "Note that we are compiling the network with the `mse` loss function -- Mean Squared Error, the square of the difference between the \n",
    "predictions and the targets, a widely used loss function for regression problems.\n",
    "\n",
    "We are also monitoring a new metric during training: `mae`. This stands for Mean Absolute Error. It is simply the absolute value of the \n",
    "difference between the predictions and the targets. For instance, a MAE of 0.5 on this problem would mean that our predictions are off by \n",
    "\\$500 on average.\n",
    "\n",
    "网络的最后一层只有一个单元，没有激活，是一个线性层。这是标量回归（标量回归是预测单一连续值的回归）的典型设置。添加激活函数将会限制输出范围。例如，如果向最后一层添加 sigmoid 激活函数，网络只能学会预测 0~1 范围内的值。这里最后一层是纯线性的，所以网络可以学会预测任意范围内的值。\n",
    "\n",
    "注意，编译网络用的是 mse 损失函数，即均方误差（MSE，mean squared error），预测值与目标值之差的平方。这是回归问题常用的损失函数。\n",
    "\n",
    "在训练过程中还监控一个新指标：平均绝对误差（MAE，mean absolute error）。它是预测值与目标值之差的绝对值。比如，如果这个问题的 MAE 等于 0.5，就表示你预测的房价与实际价格平均相差 500 美元。\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Validating our approach using K-fold validation\n",
    "\n",
    "\n",
    "To evaluate our network while we keep adjusting its parameters (such as the number of epochs used for training), we could simply split the \n",
    "data into a training set and a validation set, as we were doing in our previous examples. However, because we have so few data points, the \n",
    "validation set would end up being very small (e.g. about 100 examples). A consequence is that our validation scores may change a lot \n",
    "depending on _which_ data points we choose to use for validation and which we choose for training, i.e. the validation scores may have a \n",
    "high _variance_ with regard to the validation split. This would prevent us from reliably evaluating our model.\n",
    "\n",
    "The best practice in such situations is to use K-fold cross-validation. It consists of splitting the available data into K partitions \n",
    "(typically K=4 or 5), then instantiating K identical models, and training each one on K-1 partitions while evaluating on the remaining \n",
    "partition. The validation score for the model used would then be the average of the K validation scores obtained.\n",
    "\n",
    "## 利用 K 折验证来验证你的方法\n",
    "\n",
    "为了在调节网络参数（比如训练的轮数）的同时对网络进行评估，你可以将数据划分为训 练集和验证集，正如前面例子中所做的那样。但由于数据点很少，验证集会非常小（比如大约100 个样本）。因此，验证分数可能会有很大波动，这取决于你所选择的验证集和训练集。也就是说，验证集的划分方式可能会造成验证分数上有很大的方差，这样就无法对模型进行可靠的评估。\n",
    "在这种情况下，最佳做法是使用 K 折交叉验证。这种方法将可用数据划分为 K 个分区（K 通常取 4 或 5），实例化 K 个相同的模型，将每个模型在 K-1 个分区上训练，并在剩下的一个分区上进行评估。模型的验证分数等于 K 个验证分数的平均值。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In terms of code, this is straightforward:\n",
    "\n",
    "这种方法的代码实现很简单。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "processing fold # 0\n",
      "processing fold # 1\n",
      "processing fold # 2\n",
      "processing fold # 3\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "\n",
    "k = 4\n",
    "num_val_samples = len(train_data) // k\n",
    "num_epochs = 100\n",
    "all_scores = []\n",
    "for i in range(k):\n",
    "    print('processing fold #', i)\n",
    "    # Prepare the validation data: data from partition # k（准备验证数据：第 k 个分区的数据）\n",
    "    val_data = train_data[i * num_val_samples: (i + 1) * num_val_samples]\n",
    "    val_targets = train_targets[i * num_val_samples: (i + 1) * num_val_samples]\n",
    "\n",
    "    # Prepare the training data: data from all other partitions（准备训练数据：其他所有分区的数据）\n",
    "    partial_train_data = np.concatenate(\n",
    "        [train_data[:i * num_val_samples],\n",
    "         train_data[(i + 1) * num_val_samples:]],\n",
    "        axis=0)\n",
    "    partial_train_targets = np.concatenate(\n",
    "        [train_targets[:i * num_val_samples],\n",
    "         train_targets[(i + 1) * num_val_samples:]],\n",
    "        axis=0)\n",
    "\n",
    "    # Build the Keras model (already compiled)（构建 Keras 模型（已编译））\n",
    "    model = build_model()\n",
    "    # Train the model (in silent mode, verbose=0)（训练模型（静默模式，）\n",
    "    model.fit(partial_train_data, partial_train_targets,\n",
    "              epochs=num_epochs, batch_size=1, verbose=0)\n",
    "    # Evaluate the model on the validation data（在验证数据上评估模型）\n",
    "    val_mse, val_mae = model.evaluate(val_data, val_targets, verbose=0)\n",
    "    all_scores.append(val_mae)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[2.1177661277279993, 2.3056187889363504, 2.93779449179621, 2.3707814741842816]"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "all_scores"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2.4329902206612104"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.mean(all_scores)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "As you can notice, the different runs do indeed show rather different validation scores, from 2.1 to 2.9. Their average (2.4) is a much more \n",
    "reliable metric than any single of these scores -- that's the entire point of K-fold cross-validation. In this case, we are off by \\$2,400 on \n",
    "average, which is still significant considering that the prices range from \\$10,000 to \\$50,000. \n",
    "\n",
    "Let's try training the network for a bit longer: 500 epochs. To keep a record of how well the model did at each epoch, we will modify our training loop \n",
    "to save the per-epoch validation score log:\n",
    "\n",
    "每次运行模型得到的验证分数有很大差异，从 2.1 到 2.9 不等。平均分数（2.4）是比单一分数更可靠的指标——这就是 K 折交叉验证的关键。在这个例子中，预测的房价与实际价格平均相差 2400 美元，考虑到实际价格范围在 10 000~50 000 美元，这一差别还是很大的。\n",
    "\n",
    "我们让训练时间更长一点，达到 500 个轮次。为了记录模型在每轮的表现，我们需要修改训练循环，以保存每轮的验证分数记录。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "from keras import backend as K\n",
    "\n",
    "# Some memory clean-up\n",
    "K.clear_session()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "processing fold # 0\n",
      "processing fold # 1\n",
      "processing fold # 2\n",
      "processing fold # 3\n"
     ]
    }
   ],
   "source": [
    "num_epochs = 500\n",
    "all_mae_histories = []\n",
    "for i in range(k):\n",
    "    print('processing fold #', i)\n",
    "    # Prepare the validation data: data from partition # k（准备验证数据：第 k 个分区的数据）\n",
    "    val_data = train_data[i * num_val_samples: (i + 1) * num_val_samples]\n",
    "    val_targets = train_targets[i * num_val_samples: (i + 1) * num_val_samples]\n",
    "\n",
    "    # Prepare the training data: data from all other partitions（准备训练数据：其他所有分区的数据）\n",
    "    partial_train_data = np.concatenate(\n",
    "        [train_data[:i * num_val_samples],\n",
    "         train_data[(i + 1) * num_val_samples:]],\n",
    "        axis=0)\n",
    "    partial_train_targets = np.concatenate(\n",
    "        [train_targets[:i * num_val_samples],\n",
    "         train_targets[(i + 1) * num_val_samples:]],\n",
    "        axis=0)\n",
    "\n",
    "    # Build the Keras model (already compiled)（构建 Keras 模型（已编译））\n",
    "    model = build_model()\n",
    "    # Train the model (in silent mode, verbose=0)（训练模型（静默模式，verbose=0））\n",
    "    history = model.fit(partial_train_data, partial_train_targets,\n",
    "                        validation_data=(val_data, val_targets),\n",
    "                        epochs=num_epochs, batch_size=1, verbose=0)\n",
    "    mae_history = history.history['val_mean_absolute_error']\n",
    "    all_mae_histories.append(mae_history)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can then compute the average of the per-epoch MAE scores for all folds:\n",
    "\n",
    "然后你可以计算每个轮次中所有折 MAE 的平均值。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "average_mae_history = [\n",
    "    np.mean([x[i] for x in all_mae_histories]) for i in range(num_epochs)]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's plot this:\n",
    "\n",
    "我们画图来看一下"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEKCAYAAAD9xUlFAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzt3Xd8VfX5B/DPc1cSkkAghD0CyBBkiigiZYgVxT3qXtXaap11V2sVtVXbat3W+dOqFfcWF4iACoYNArLCHgkhkEHmfX5/nJFz903ITYD7eb9eeeXec889+Z7k5jznu56vqCqIiIgAwNXcBSAiov0HgwIREdkYFIiIyMagQERENgYFIiKyMSgQEZGNQYGIiGwMCkREZGNQICIim6e5C1Bfbdu21dzc3OYuBhHRAWXevHmFqpoTa78DLijk5uYiLy+vuYtBRHRAEZH18ezH5iMiIrIxKBARkY1BgYiIbAwKRERkY1AgIiIbgwIREdkYFIiIyJY0QWHlthL868uVKCytbO6iEBHtt5ImKKzeUYonpq1GUVlVcxeFiGi/lTRBQcT47ldt3oIQEe3HkiYouMygwJhARBRZ0gQFwIgKrCkQEUWWNEFBWFMgIoopaYKCy4oKREQUUdIEBSsksPmIiCiypAkKLvNMGROIiCJLmqAg7GgmIoopeYKC1dHcvMUgItqvJVFQMKKCsqZARBRR8gQF8ztjAhFRZEkTFKwhqYwJRESRJU1QsHMf+RkWiIgiSbqgwJBARBRZ8gQFDkklIoopaYKCy+5pbtZiEBHt15ImKFhDUtmlQEQUWRIFBeO7sqpARBRR0gQFLrJDRBRbwoOCiLhFZIGIfBLmtUtFpEBEFppfVySwJADY0UxEFI2nCX7G9QCWA2gZ4fUpqnpNogvh4pBUIqKYElpTEJEuACYBeCGRPyfOsgBg7iMiomgS3Xz0bwC3AvBH2edMEVksIu+ISNdwO4jIlSKSJyJ5BQUFDSoI+xSIiGJLWFAQkZMA7FDVeVF2+xhArqoOAvA1gFfC7aSqz6nqcFUdnpOT07DygENSiYhiSWRNYRSAU0QkH8CbAMaLyGvOHVR1p6pWmk+fB3B4ogpjD0llVYGIKKKEBQVVvUNVu6hqLoBzAUxT1Qud+4hIR8fTU2B0SCcEcx8REcXWFKOPAojIZAB5qvoRgOtE5BQANQCKAFyasJ8LdjQTEcXSJEFBVb8F8K35+G7H9jsA3NEUZXC5rJ/ZFD+NiOjAlDQzmtnRTEQUW9IEBRdzHxERxZQ0QcFeeY0xgYgooqQJCmBHMxFRTEkTFOxFdoiIKKKkCQp1i+ywpkBEFEnSBAXmPiIiii1pggKHpBIRxZY8QYG5j4iIYkrCoNC85SAi2p8lUVAwh6Ry8hoRUURJExTY0UxEFFvSBAV2NBMRxZY0QYG5j4iIYkuaoADmPiIiiilpgoKLw4+IiGJKmqBgpT5iTYGIKLLkCQrCLKlERLEkTVCo62gmIqJIkiYocEgqEVFsyRMUzDNl8xERUWTJExTM74wJRESRJU1QcDH3ERFRTEkTFIST14iIYkqeoABrSGozF4SIaD+WPEGBuY+IiGJKvqDAmEBEFFHSBAUXZzQTEcWUNEGBuY+IiGJLeFAQEbeILBCRT8K8liIiU0RktYjMEZHcRJWjrqaQqJ9ARHTga4qawvUAlkd47XIAu1T1EACPAngoUYWoG5LKqEBEFElCg4KIdAEwCcALEXY5FcAr5uN3ABwrVjrTxi8LACbEIyKKJtE1hX8DuBWAP8LrnQFsBABVrQGwG0B2ogojwo5mIqJoEhYUROQkADtUdV603cJsC7lqi8iVIpInInkFBQUNLxPYp0BEFE3EoCAitzoenx302t/iOPYoAKeISD6ANwGMF5HXgvbZBKCreUwPgFYAioIPpKrPqepwVR2ek5MTx48OzyXCyWtERFFEqymc63h8R9BrE2MdWFXvUNUuqpprHmuaql4YtNtHAC4xH59l7pOwq7YIh6QSEUXjifKaRHgc7nncRGQygDxV/QjAiwD+KyKrYdQQzo365n0kImw+IiKKIlpQ0AiPwz2PSlW/BfCt+fhux/YKAGeHf1fjM/oUGBWIiCKJFhQGi8geGNfSNPMxzOepCS9ZAhh9CkREFEnEoKCq7qYsSFMQAfzsVCAiiqheQ1JFJF1ELhCRTxNVoEQScPIaEVE0MYOCiPhE5DQReQvAVgATADyb8JIlgIsdzUREUUVsPhKR4wCcB+B4ANMB/BfACFW9rInK1viEuY+IiKKJ1tH8BYCZAI5R1XUAICKPNUmpEsSVmLRKREQHjWhB4XAY8wa+FpG1MGYlH9Cdz8KaAhFRVBH7FFR1garepqq9ANwDYCgAn4h8LiJXNlUBGxP7FIiIootr9JGqzlbVa2BkNf03gJEJLVWCCFhTICKKJlpH87AILxUAeCIxxUksEQ5JJSKKJlqfQh6AZTCCABCY70gBjE9UoRKFuY+IiKKLFhRuAnAmgL0wOpnfV9XSJilVgjD3ERFRdNE6mh9V1WMAXANjzYNvROQtERnSZKVrZOxoJiKKLmZHszlH4UMAXwIYAaBPoguVKBySSkQUXbSO5p4w5imcCmMd5TcBPGCmuz4gMUsqEVF00foUVgNYDKOWsAdANwBXizkrWFUfSXjpEoA1BSKiyKIFhcmoG8GZ0QRlSThhmlQioqiiradwTxOWo0mw+YiIKLp6radwoGNHMxFRdEkVFDgklYgouqQKCsx9REQUXbSOZgCAiKTAmNmc69xfVScnrliJwdxHRETRxQwKMIak7gYwD0BlYouTWEbuI4YFIqJI4gkKXVR1YsJL0gSM3EfNXQoiov1XPH0K34vIwISXpAmwo5mIKLp4agrHALhURNbBaD4yk43qoISWLAE4JJWIKLp4gsIJCS9FExFOXiMiiiqeLKnrAWQBONn8yjK3HXC4ngIRUXQxg4KIXA/gdQDtzK/XROTaRBcsEVwudjQTEUUTT/PR5QCOVNUyABCRhwD8gANwnWaBsE+BiCiKeEYfCYBax/NaBK7XHP5NIqkiMldEFonIMhG5N8w+l4pIgYgsNL+uiL/o9cfJa0RE0cVTU3gZwBwRed98fhqAF+N4XyWA8apaKiJeALNE5HNV/TFovymqek38RW444ZBUIqKoYgYFVX1ERL6FMTRVAFymqgvieJ8CKDWfes2vZr0kM/cREVF00ZbjbKmqe0SkDYB888t6rY2qFsU6uIi4YaTHOATAU6o6J8xuZ4rIrwD8AuBGVd1Yv1OInytmoxcRUXKL1qfwhvl9HoA8x5f1PCZVrVXVIQC6ABghIocF7fIxgFxzItzXAF4JdxwRuVJE8kQkr6CgIJ4fHZYIO5qJiKKJGBRU9STzew9V7en46qGqPevzQ1S1GMC3ACYGbd+pqlaSvecBHB7h/c+p6nBVHZ6Tk1OfHx3AJRySSkQUTTzzFL6JZ1uYfXJEJMt8nAZgAoAVQft0dDw9BcDyWMfdFxySSkQUXbQ+hVQALQC0FZHWqBuG2hJApziO3RHAK2a/ggvAW6r6iYhMBpCnqh8BuE5ETgFQA6AIwKUNPpN4sKZARBRVtNFHvwdwA4wAMA91QWEPgKdiHVhVFwMYGmb73Y7HdwC4ox7l3ScuAfwMCkREEUUMCqr6GIDHRORaVT3gZi+HIxCo+pu7GERE+6145ik8YY4a6g8g1bH91UQWLBFcLkBrY+9HRJSs4lmj+a8AxsIICp/BSKU9C8ABFxTY0UxEFF08uY/OAnAsgG2qehmAwQBSElqqBHG5BLWMCUREEcUTFPaq0RBfIyItAewAUK95CvsLn1tQXcM+BSKiSOJJiJdnzjd4HsYopFIAcxNaqgTxeVyormVQICKKJJ6O5qvNh8+KyFQALc3hpgccr5tBgYgommiT14ZFe01V5yemSInjdbtQxeYjIqKIotUU/mV+TwUwHMAiGBPYBgGYAyOV9gHF53Ghij3NREQRRUuIN05VxwFYD2CYmZDucBizlFc3VQEbk4/NR0REUcUz+qifqi6xnqjqUgBDElekxPG6hUGBiCiKeEYfLReRFwC8BmPltAuR4GymicI+BSKi6OIJCpcBuArA9ebz7wA8k7ASJZDP40KNX+H3K1xcho2IKEQ8Q1IrADxqfh3QvG6jtaza70eKy93MpSEi2v9EG5L6lqr+RkSWwGg2CmAuoXlA8ZlBoarGjxQPgwIRUbBoNQWrueikpihIU/B5zJoCh6USEYUVbT2Freb39U1XnMSym484AomIKKxozUclCNNsBGMCm6pqy4SVKkG8bqNzmSOQiIjCi1ZTyGzKgjQFq/moijUFIqKw4hmSCgAQkXYIXHltQ0JKlEA+Nh8REUUVc0aziJwiIqsArAMwA0A+gM8TXK6EsPsUatjRTEQUTjxpLu4DcBSAX1S1B4xV2GYntFQJUtd8xIWaiYjCiScoVKvqTgAuEXGp6nQcsLmPrHkKrCkQEYUTT59CsYhkwEhv8bqI7ABQk9hiJYbPY4w+Yp8CEVF48dQUTgWwF8CNAKYCWAPg5EQWKlF8bmMWM4ekEhGFF22ewpMA3lDV7x2bX0l8kRLHy5oCEVFU0WoKqwD8S0TyReQhETkg+xGc7D4FBgUiorCirbz2mKqOBDAGQBGAl0VkuYjcLSJ9mqyEjahungI7momIwonZp6Cq61X1IVUdCuB8AKfjAF1kxx6Syj4FIqKw4pm85hWRk0XkdRiT1n4BcGbCS5YATIhHRBRdtI7m4wCcB2ASgLkA3gRwpaqWxXNgEUmFMYw1xfw576jqX4P2SQHwKoDDAewEcI6q5tf/NOJjJcRjUCAiCi/aPIU/A3gDwM2qWtSAY1cCGK+qpSLiBTBLRD5X1R8d+1wOYJeqHiIi5wJ4CMA5DfhZcbGajyrZfEREFFa0LKnj9uXAqqoASs2nXvMruIf3VAD3mI/fAfCkiIj53kbndbH5iIgomngmrzWYiLhFZCGAHQC+UtU5Qbt0BrARAFS1BsBuANlhjnOliOSJSF5BQUGDy+NyCTwuYVAgIoogoUFBVWtVdQiALgBGiMhhQbtIuLeFOc5zqjpcVYfn5OTsU5l8HhdHHxERRZDQoGBR1WIA3wKYGPTSJgBdAUBEPABawZgTkTBet4vzFIiIIkhYUBCRHBHJMh+nAZgAYEXQbh8BuMR8fBaAaYnqT7B43S7OaCYiiiDuldcaoCOAV0TEDSP4vKWqn4jIZAB5qvoRgBcB/FdEVsOoIZybwPIAAFI8LlSz+YiIKKyEBQVVXQxgaJjtdzseVwA4O1FlCMfrFtYUiIgiaJI+hf2J0afAoEBEFE7SBQVj9BE7momIwkm6oMCOZiKiyJIuKPjc7GgmIook6YKC18MZzUREkSRdUPCxo5mIKKKkCwpet4tZUilhlmzajQTPvyRKqOQLCh7WFPZntX7FszPWYG9VbXMXpd6mLt2Kk5+chfcXbG7uotBBpLrWj1p/091oJF1QSGHuoyazu7y63u/5eNEWPPj5Cvz7618SUKLEWr3DyBS/akdpjD2J4jfqwWk4//kfY+/YSJIuKHjdLmwoKsdLs9Y1d1EOanPW7sTgyV9i2ort9XpfRbVRQyhuQEBpblar0UcLt2BL8d59OlZFdS3eX7CJTVFJrqSiGjtKKjFnXULzhAZIuqBgmfzJz81dhIPa/A3FAFDvD7NLjGzqtQfgxdAq8ebivTj3uX27s/vXlytx45RF+G5V4b4XjBrd6gTXBmtq/Xjtx/X4Yc1Oe1txeVVCf6Yl6YLC1j0VzV2E5FLPa7vLZQQFfxO2oTYWvyOQbSgq36djbdpl1DRKK2r26Tj7qz+9tRBPfLOquYvRIF8s24YJj8zA1KVbw76+q6wKf/9sOSprGtYv5vcrnv52De76YCmu/O88e/vkj3/Gkk27G3TM+ki6oLBpH/9ZqX7qe2l3m5/Ipqop7N5bjd17wzdVzV1XhNLKuovynopqnPrUbKzeURJ2/8YsstWx6HaFW4eq4XbsqcCDn69o0o5LS0FJpT2A4L35m/Gvr/b/fqOtu/dic1BT4LLNxoX5563hPwcPTV2B/3y3Fvd98jNuemtRvZoAVRW3vbsYjzh+NxMHdAAAvLdgMz6PEIgaU9IFBZ/HOOUUT9KdepOSBl7LxFyMrykuWhXVtRh875e4+MXgVWKNttzf/OcHXPVa3Z3ajJUFWLSxOOAf1qkxSnzXB0vwwKc/27WOhv4eI7n13cV4dsYazFu/q3EPHIdRD07DsPu+arZ+kpmrCjD54/o1G4/8+zSMenBawDar9JHi9c4yo5nntR834N35m7B6Rylenr0OO+Jopbj57cV4e96mgG2DurayH+dkpsRf+AZKuivj8xcPR5fWaajxq91Ecd3/FqDvXZ83c8kOLg29lll5qZxNMet3luGKV/JQVtm4TSlfLNsGAFgUpkpebt7Rzllr9IkUl1fhtR/XRz1efZoLPly4GfPWh/a3vPbjBjw/c50dFCuqa7FqewkKSyvjPnY01u+wsYNNOCu27cGrP+Tbz6tq/dhbXYuf8hs3IO2pqMbDU1egqsaPn7fsQe7tn2LV9tC7+ItenIuXZq/bp6D05twNeGLaagB1/V+Wmlo/Hpq6Apt3BdYszn3uR9z78c/439yNAduXbt4d0kz67vzAgAAAnVql2Y/bZjAoNLqubVrgslE9UOtX7Kkwmg0+WrSFE9oSJNY/4E/5RfjTlIX2ftb62c6awrX/W4Cvl2/Hoo3F+1SWNQWlmL26ruN2u3nn1irNG7KvdfG0gtRdHyy1O82t2syijcW4/s0F9l13RZS5FR8v2oKLX5oLwGgzvv7NhTjzmR8AABuLyrGjpALPfbfG3t8aNV1eVYvjHv0OYx6eHnLMX7aXIL+wLGR7eVWN/dkGgPzCMvz9s+Xw+xU15u81VkyYv2EX3s4zLmKrd5Ti08X1b7Y45YnZuPvDZah13IABRrBoLAs3FuPhqSvw9Ldr8MniLXjLLPNXyyOPequobvj/urOWWBM03+mn/F145ts1+Hlr4PlZNYeCUuPztrGoHKc+NRsnPTEL93y8DFOXbsVnS7ZiY1F52JqA8/PZFDWFRK68tt/KTvcBMP5YWS18zVya+CzZtBsiwGGdW8Xc96i/fYOsFl5MveFXTVCy8OK9F7vghTmoqvHj3lMH4NUf1mPbbuMfx/n/tti8k6/ZxyalK1/Nw5qCMsy8dRy6tmlh/7OGa6oqD7rAB/Q7mFfU9xdsxocLt8AlgsO7tw55j6pCzLvJa/+3wDiHWn9AG7Xfr7j69fnITPXge8dIkzlrjcdWG3xZmIDz60e/AwDkPzgJny7eimVbdiMz1Yu38jZiXWEZ8h+cBAC4/s0FWLRpN04f1tlRA4l+YTzj6e8BAGcP74oJj8wAAEwaNCnqe4JZAXVXeZXdbAsAawvqAtmW4r3olJWGssoa7CqvQpfWLQKPUeOH1y3277GorAoXvDAHN0zojd17q3HrO4vtfWv8ihKzY/7LZdtxzvCuyA5zZ11SWY00nztiuTftKscxD03HG7870t724cLNqPUr0lM8QIlRa1uxrQSXvjwXmaleXDYqF+fFmEuwfU8lqmr8uOr1eVi62Qgcr/6wHq/+EFgDPXFgB3y2ZJv9PDO17jLdFDWFpAwK1i92x55K9MrJaObSxOfkJ2cBgP2PHs22PRXYVs9RVtv3VOA/M9bijhP7weve9wqkdfGJVVO3agZrCsrwjy9W2tv9qpi/YVdAFd1599sQVlE+WLAZLpdgU5E5wqeyBjW1fngc5x18gW/hvIiYB9q623h/YWklvl9dGNIWXFZVi4yUwH+x0soaLHfcSa7bWYbNxXuxK2i4oVVz3Vsdu0mq1q945KuVWFMQWGuwzsmKeXuralFjVkHiOS4QOAzy1ncW4fShXdCldRqe/nYN/nLSoWjhi30JGX7/1/jwj6Ps5/k768p59IPTkP/gJFzzxnxMX1mAVQ+cYH/+SiqqMfCeL3HbxH64amwvAEatZfnWPfi9Y1SOxSWCjeZAkoUbi3HjW4vw6m9HhOxXWlGDdpmRy/vlMqOW8fT0uprb9W8uDN3v57rayNc/B9ZMzj2iK349oD18bjf+7/t8bN29Fws3FmPYfV8FDF4Ip3/HlgFBoWUT1xSSrvkIALq0NtroNu0KHIkUXB08mFz04hw8/91azN+wK2wKiRm/FOCl2evwS5i22IaoNC86zpvwtQWlKIlwYV+yObBdv9avOOPp73HaU7PtbSX7ODwzxWNc2N/8aSP+8cVKfLqkrklkT9Cxy6rqnn+zfDu+X113F2/9U281azUzVxXi/BdCO6uPf/S7kM7FkoqagDHuq3eUYld5VcTg6byAOJvinI/fm78Ja8M0I/2wdid2l1cj1euyj2UF63iDwuiH6pqt3srbhPOe/xGjH56O/83dUK/O6g8W1qX+WBdUVlXFLLNZ7+ctdQHTqjW+MbfuTjo4eDpV1tQGfI7WFpTax//M8bcuqajBqu0l9u/i/QWbkJdf179jNStWBP2ORvRoE/FnB/8+zx3RDeP7tccxvdvihUuGo1+HligoqYwZEADg0I4tA547awotUxN/H5+UNYVOWWkQATYGdQhV1PiR0Qh3yfujmasKMdOcCDVpYEc8dcGwgNeti3hh6b5PkNlVVmXfaVudrzW1foz/1wyM7t0W/73cqJYXldX9rMVB/QXh8lPtCRo66vcr3p63EWcM6xJX7WaX3bYb2mm7e2815q3fhbLKGpw2tDPKK+v+yS9/JS9g31XbS1BRXYstxZFrY7nZLZC/sxwzVxXi6EOy7e2vfJ8fcG4rt5VErU0VOf4eRWVVdnOIsyZzi6MJxemiF+eibUYKDu1o3BbvKq+2h/pG6/9wKolyEYs0lPe2dxbjuP7tA7Y5/9brdwbejP2wZqedeiZv/S4M7poFoO7vtLFoLz5ZvAW52el2TSCcmb8UBlycrd/rlJ824vb3ltjbTzVvNB47dwiO7tUWN05ZBABYeu/xyEjxYJkZmJx9A20zUtArJwNzw0zGbJXmDfld9MxJD3jevmX8d/g92ga+t2WqF6/8dgR+WldkN6Ml0sF5BYzB53GhY8vUkDkLwXcGB6uFYTpsrTbmwpJ9G+Wyp6IaQ+/7Ci+YaUSsZhDr7tB5dzn2H3V3oXPzA//Zwt3JBjcfvTt/E257dwmen7k2YPv8DbvwU9DxVBVF5l1mlWNQgXUXdvFLc/C7V/NwwxSjmcBZUwi2ZXcFbnt3ccCIoLF9cwL2+fCaYwAA20sqMPLvdUMaX5i1Dq/P2YDOWUZt1XlnHM6UvLoRK1YQUlVc88Z8ALEvNoWllXYNqbi8yr47Lo9yfvHaGXQDYTXtTMnbiCteDQykKyKM6QcQUMu675OfcYUZhAscn8Vr3liAk56Yhfs/XR7w3kuPzrUfT1uxAyLAp9cdg+MHtMfm4r0orazBZ0u3IZz1O8sD0pFs212B3eXVmLPOqBU6A+/IXtnIahE6IAEA3v7DSPx4x7H28wuP6oaWqYH7tm+Zaj/+Iqivz2q5sHTKCnzewufGmD45uPn4vmF/fmNLyqAAGKOQgqvcyRIUnMM9b3l7ER6eusK+CO/r0Mfgu3krKKzYZlwUnP8Aziab4LvHcE1cwc1HVlNC8MXpjKe/x9nP/hCwrbyqNiAYWHqad2UbiwJrjeXmHfIfx/UKeQ8AfGKOxsnNNjpG+7QPbKRuleZFZqoH23eH1iZq/IoOrVLRuoUXy82ROG3SYw94WLSpGKu2l+Cc//yI6SsLAAAnD+oU830pZvNRUVkVavxWX0Xo76LWr6isqY17aO1O87NSU+vHkX/7GhMemYETHpsZdt+VZrNkpAur09fLt+OCF34M+UyE47yrrqr1o3ubFhjQqRWONyd85ReW4UdHB74lM8WDorIqu18IAHaUVGBK3gZU16r9uTi8e2s8cd5QPHTmQGSZbftH9WyD04d2BgDcdFwf9G6XgbYZdX+/+08bGPLz2jn6AoJrAteOPyTgearXjcfPG2o/b4ragVPSBoXRvdti4cZiTF+xw962L0PV9mfBw0KdQeHteZvw9Ldr7IC4r0EhuIPWapZaaQaFDq3SIg5TdXbmhgvQwQHH4jxcuCGaQGDzhVO37PSQbbe9s9ge7XOJ404UgP3Pat1xdzff3yrNi1uC7uQ6tEzF9j3hf5/Z6T60SffZF77Jpw4IeD24gxowhu9e8WpeQK3qiCjt3BYrGBaXVwd0NKsqXp+zHl+a8zX+9NZC9L1rKvreNTXmMQHg8WmrkXv7p1ixrSTieQazhvr6YjT3zV69M2SSYLj3dM8OHK1kBWerme3NnzagqtaPEw7rELBf28wUbNtdgZdm59vb1uwoxWNfr8K4vjm4fHQPAMAJh3XAyYM7oYXPYwe0Wr/i0XOGIP/BSbj22N4QkYBBCuG0c9QUfB4XfrhjvP18UJeskP1PGdwJoxzNjk0paYPC78f0gtslAc0ZB2tNoSqofT5cf7rdfBSlT2HhxuKAFA9LN+8OmJwEhN7NV5gXJCsX0MaicvS44zN8EGbNgW5t6v7Bg+/+gcCaxYcLN6OoLDBI7N5bjbH//NZ+7hxq+tXP4cetj+yZHdL0MyVvI9YUlMLjEuRkpOA1sw+kdQsvThncCaN7twVgBLHbT+iHdJ8bpwzuhD+OC7zja98yFdtLKuB1h97pZWek2KOFvG7BhEPb45ubxiDdDIyDugQOPc5I8WDaih0hOZV6tk1Hj7bpIefgZH2ui8qq7I7OiupafL18B+5838ivo6r4cOGWiMcI5gzgs1ZHTtr37IWH2yOAvG7ByYONms1lx+TG/bMAo4P1/357RMj23KCg3reDGRTMmtdrP25A6xZejO/XLmC/7HQfpi7bFtBH8NT0NSirqsXNx/fFBUd2x9q/nYgrRve0X2+VZhyzIan32wWNGuromJDWr0Mm5t01IeQ9r/72SKx64IR6/6x9lbRBwet2oV1mCrY4qo8NTWCVaPuaHC642cQf5k69oiZ2TeG0p2ZjwiPf2c9PemIW7v5wWcCdf/CsY6umYFXTrb4Fa5JRL0eHnLNpKbiDs3NWmj1yqaisCte/uRAP5518AAAZuUlEQVTPzjCGDL40ex2++6UA7wfNBt1ZVte88e+vf0Fmqgf9OgQ287RJ9+EPY0KbiJZu3o0WPjdExG7asX6Prc25LQM6tcShHVti2eSJ6GoGtG9uGmOPb2/fMhUbi8pRXau4LqiJIDvdh5t/3QeHdW6JX+4/AaleN3rlZODJ840BAMF3njce1wclFTUBtaILj+qGHm3TMf3msXjy/GEBwxWdKRisAQbri8rtoPDcd2sD0ppvLNprX0jj0d8xQmbmKqMp61jzwpvuc+OMYZ0xIrcNjh/QHr/qk4OFdx+Hr24cg4fOHISl9x5v96lEkpnqwcmDO9m5n769ZRxG9gy9c+7SOg0ex8kO6GSUK9vRnPO30wdi0qCOOH5AXed3cHOdz+PCtj0V6Ns+EwM6GQHZFZTHwqopWE1w9dEuTN/PJ9ceg/tOHQARQXZGCo7s0Sagtul2SaMMD6+vpA0KgNGh8978ujvW/bX5qD6zrcMNq40UFJyjYKy7yYIGdDR/sWwb3p1n5HgJHnJnHTd4pI7VPHPqkM72NquDrW/70EHknbPSsGevcexwwxIvfmku5m0oRuesNNxzcn8AxjyUF2etwxnPfI89FTV4+MxBIe252Rm+sBfDX7aXGhOVUHcBsWpcrc2LQ/CxAKBXTgaO7tXWPJ9Uu+aV42g+AIDW6T5cM743Prl2dECbsfU38boEn153jL19sKPmcM/J/ZH/4CTcf9pAO3hkpHgw67Zx9j5W57LToo3FAUHFmXZhwqMzsLOsCn86rg+GdgttznDOj3nltyPQ2RHAZ6/eib7tM/HipUfgs+tG46s/jcEjvxmCKb8/yj63rBY+5LZNh9ftQkaKB6mO8r31+5G44pgeARf9rq1b4InzhmLRX3+Nd/4wEm3SfRARvPG7I/H+1Ucbv9PMFHjcLlw9ti6oD+veGkDgRX90nxy08Hnwn4uG45TBnfDoOYOxw/ycX3dsb3x546+QYzY3WUElHKtJryZCTeG7W8Zh5q3jwr4W7u9xWOdWuGhkrv18yu9HhtQ2m0NSDkm1BKc32F+bj+pTruCmIiC0umvVPJxNPZVmQNwZoe09mj+8Nt9+/Kfj+gS8tmjTbrw4a5099tvebo6Act4xWh15Xo8gzesOGIHUKSsVG3eVo6K6Fv+ZsQbhLN28G73aZWCQNaSxpBL3OdbNOPqQtiHpD9qk+8KmuQDqPh+t043v1ggS6+6ta9Ds22BdHc1h1rHG9MnBOUd0xahD2oZ9z+HdWyPF48JVY3vZd6wAAmbgtokwq9V54Un1uuKaizC6d1vMWl1o3zh0bJVqN3X8/YyBuMMxlNMypk9OyBwF63z6Oy6q0TpIrZfOGNoZI3q0wYgebXD163UT0gaaM/czUjwYnlvXb2IF3JcuHY6+HYyf9adf98XjZj6idpnG38j5u3D2z1h9QjNXFWLhxmJcMrI7sjNS0LdDJjYX70VumEBvsW4SInWWd8uO/nlwntf+LKmDQvCY53gn9DS1+tQUwo2wCa0pGN+dHbd7He3Ofr+GVJ3jndj30uzQFe0e+/oX1PgVPo8rpCwdWtXdQWek1t2JpfkCg0Kb9BTs3luNm99eZI/8CbausAxj+uSgixlonMNSB3ZuhVZpXqQHzcDNTvchMzX8P/kR5sUoxePGY+cOwbBuxl2oFUyzYjS3OINGVpoXyydPhNcdvVMyOyMFK+8PbUdO87rhcQlq/Io2MVKzDOjUEg+dOQhnPPN92M+D09BurTF7daE927t7drp9YY2WgLBH28AL4JgofRrhWLVV5+fshgl9sH5nOf4wplfIPIdg4/sFvj7jlrH1+v+dfOphuPJXPe0O6YtHdse0FTsw3KxphJOb3QKTTx1gp7KurxX3TWz0VOiJkNRB4baJ/QLGU++vzUfOmkKtX6N+sJwXASv3TlVt4D+L9Q/prClY6Qxq/Ypd5VUhOWOczULhgkbdcUJHCFl31ofkZIQkC3PO3uycZVxoNhSV2+32ADC0WxYyUz0or6qNGBC6tWmBDUXlSE9xo13LVEwa2BFPf2vUKA5pl4EXLhkOILCD1O0StEz12ucyvl87tM3wIcXjxn9/XI+jHM0Zzmau7uYF8ZAYKVKcd46t0rxR8+3EkuZzI8XjQk1VbdThqz/dOQEZKR6k+dx48ZLhuOhFIwmf1y2orlWM79cOs1cXorLGj+k3j0X3Ni3wuLnYzVPnD8OIHm3w85a6WcGpXhey00NrJt3aBN5RHxOh5hOJdY/hdtQm+rTPxKfXja7XcSzdw4wie/jMQWgd4XeVkeJBvw51n72xfdth/l+Oi/q7FRFc7Gjuqa9Ub8P//k0pYUFBRLoCeBVABwB+AM+p6mNB+4wF8CEA6/byPVWdnKgyBZvQv31A8qn9tvnI0QFeXeuH2xX5w+WsVdT4FV63hNQ0au3mo7oLuLPZqLA0MCisKSjF4k11E97eW7AZD34eOInIKdXrQkW1H4O7ZqGiqtYeo967fWBQePScwQH/hEf0MO7Syqtq0TnLOMfzRnTDn0/sh7fyQlMKO/3t9IF4dsYanDmsCwBgQv92dhqLv5zU3276sYJb6xZedGiVZgcE512836+YeFgHHN0r/JDAK0f3xOHdWuPIMB2fTh0c/Qgds1Kj7BlbqscNn8eFsqraqGP9nZ3No3vn4IWLh+OKV/PQKSsNz100HL3bZaDUnLhmTbC65fi++McXK/FrsyP2wqO6Q2H87i8a2d3OCutkzc8Y3KUVHjh9YL3vgK2Z1e4wI7May2+O6Fqv/eOZK5IMEllTqAFwk6rOF5FMAPNE5CtVDV7lYqaqnpTAckTlTOiVyKDw5LRVmJK3ETNvHR975yCVjhpMda0/4I5DVfHEtNXo3S4DJwzsGNB5XFXjx9biipAmBKuz0TnEc2dpFTJSPCitrEFhaSVa7vagrLIGaT4Pjv3XjID33/z2oqjlzUjxYu6dY+Bzu3DXB0vtoBDcMWs1UwQ/79Qq1b6jP3t4F2Smeu2cL8F9DZYeOel47Yq6rJbOGsggRzuuNfrpn2cPxri+dcMUnXfxLpdEbPMHjJFBsQICYNREpt4wGuk+T8i51leaz40XLz0CL8/OD5gdG8sQs9PY7RJ7uGbwbNs/jjskoIPT43bhslE9Qo614C/H2bXM7IwUzL3zWGSnpzSoScQaetuWF+L9TsKCgqpuBbDVfFwiIssBdAZQv6WPEszZCRWp7d7vVyj2bWnEf35pTMRZtb0EvcOMronGGayCO413llXZk3wW3n1cQEfzP79ciZdn5+MfZw0KeI9fFSUV1QFzDkora9CvQ6Y5EakCl7w0t16pqr1uwdG92mLGLwXISHHbFx7rbjnF4wrp0LXuaqdceRTamo/n/vlY+DwufLZkGxZtWmIvMGK1+0fqQM0KOrYz+62zCeG3o3rgh7U7MTy3TcQmsMbkbKJoiOuP7Y3HvlmFFI8Lw7q1tvs14pWd7sOtE/vaM3z3RXBTzL4EulOHdEZpZQ3OqefdPCVekwxJFZFcAEMBhKaSBEaKyCIR+VxEBoR5PaGcbczONvbFm4rtETInPj4T/e+Ob5ZnJEPMETHTHDOo41VRE1hTcNrlaPYZMvkrfPdLgf38ZXO25ragVAu1qvjdq3l2oLL0aZ8Jn9uFldtL6r12wW0T+9l3os47WWvuQWWNH+ce0Q3PXliXiM8a5XJkz2z7It6uZSqyWvhw/pHdsPL+iXZHdMs0I3inet1hm09aBLXXe90u/O30gXjnDyMDtk/o3x7r/j4p4oij/c2Nx/VB/oOTGpzqQERw9dhD9rsU8W6X0T4fbqgmNa+EBwURyQDwLoAbVDU4+9d8AN1VdTCAJwB8EOEYV4pInojkFRQUhNulwQ5pV/fP4sx0eMqTs+1siiu2lQTUIqpq/GGX+3Oa8tOGgNEvVuK1hgz5/Mgx09TZFLSztBKbgjK9fv1zaNAJ/pmqwI9rQ7M9ZqR60Lt9Bv4zY23Ia7FkpHjszmHnMNNJgzoCMJqE0nxuTDysI84bYdwdxrowOy8YLe2aghszbhmHuXceG7BvuIvm+Ud2CxjOSESxJXT0kYh4YQSE11X1veDXnUFCVT8TkadFpK2qFgbt9xyA5wBg+PDhjbrq9+lDOyM7IwX3frQMu/dW4Z15m/D5kvAjXCx//Wgp/jd3I/LumhBxJaTb3jXGd1uTfqwEb/HkU7c8NX01VmwrwceL6oLC1KXbMKF/e/Rom47D7/865D2eMB130QLRy5cegcenrcKCDcXYs7caudnpdurgI3u0wcaicmwJk9QtWEaqx85C6lzNLjPVi2k3jQlYeev+0wbi7pMG1Ovu12q6y8lIMYOJF78b3QMV1f6o7f9EVD+JHH0kAF4EsFxVH4mwTwcA21VVRWQEjJpLaErDBBIRjOmTg1YtvPY4+Fi++8WIWWWVNXEvj2e1g9dn8XnnSmSWBz5bjgc+Wx5xBbZwzT47g1JX+NwuVNX6ce4RXTGuXzsc1rkVTntqNsb1NR4XllZic/FePHHeUNz+3pL4gkKKx+7A7hyUCrhnUNOF2yX1Hp7Zt30mrju2t13LAIA7J/Wv1zGIKLZE1hRGAbgIwBIRsday+zOAbgCgqs8COAvAVSJSA2AvgHM11krvCdIqzYvC0kqIRF5CUlVx89uL7TV2x/zjW3xy7TFxrZtsBYVSR7/FU9NX45vl23FIuww8fNbgqO8/4bAO+NyRFz7cIjRAaAAAQjOEHjegPT5dvNUOaDmZKZh9e92oqCm/r2uHDx6m95vhXfDhwi0hnfItfB5cenQuqmv9uPCoblHPpSFcLgmZLU1EjS9hfQqqOktVRVUHqeoQ8+szVX3WDAhQ1SdVdYCqDlbVo1T1+0SVJ5asNC/W7ywPCQjOkT8V1X68G5R07a8fLQs5Vri4VhGm+egfX6zE/A3FeCtvU9iLvHNwTHCWxUhppJ1ZTlO9LoiEZj61RurEk9s+OPvmw2cNDphxm2mO3krxuJCe4sENE/qw85DoAJbUCfGcWqV5w64B7Ox8DtcfsGBD6Dq14fIP2c1HVXWpi53KHCmNP1y4GaWVNQHrG7cLGpvuLFebdJ+dkti5vWWqF6qhmU+tdBLhFrIJdtKgTvjzif0ivn73yf3x2uVH2ksoEtGBjUEhjHRHe7fzglocJjtnuJGb4dJl1PUpGN/XFgQuBmMFpEtfnovr31yITxcH5rbPCeq7sC7+pwzuhFd/OwJPnDfUHvZqyYiwyPekgcaIoF/1iS9fTbSx9oO7ZuGY3uzoJTpYMCiYrFS6L1w8PKCjdKsj5fNxj34X8j4AIRkjg9dl8PvVDhRWbWNNQWnAPlYNwhoq+lN+4DFzMsMHhUuO7m73aQQP8cwMWr3rxIEd8MDph2FQlyzkPzgp7rv7cJ3CPXPS4fO4QpahJKIDW1InxHO6+fi+aJPuw5i+OXjMTBAGIGAN10jOfOZ7fHLtMejbIRNetysgLQUQOFPa6mjeEbRuQWlFTcDC9MGBJniUkxUUnIHAGuHZMycdawvKQmoKT50/rEGToNLMtBrOPo5Pr21Y4jIi2r+xpmDqlZOBB04fCK/bhVRv3a9lXWHsxcMB4NznfkTvOz8HEFpTKK+qS8K2t7oWtX7FztJKeFyCd68yRvqUVNZgs2Mi2rqgtYbTfIF/KqujuaUjKJw+tDOOH9Ae1x/bG4CRa+iqsb0w4dD2eON3RzZ4VqxVU3CmfE7zufcp6ycR7Z9YUwija5sWdvONc8nCaJxr3zr7FGavLrTv8ttmpGBXeTW2FO9FYWklsjN89kzd0ooarKsyAsFVY3vhGTPts3F3X7c+rGVDUbm5vS4onDqkM04d0hmqiqoaP04e3KlR0vVaKSQ8B0AueCLaNwwKYUw+9TAM69YaP+UXhV3M/C8n9Q9Y0cupqKwqoLnoghfm4LDORkdtx6w0rNpRitEPTwdgrHNrNfFc+78F9nt+O6oHvli6DWsLyzCiRxvkZKaEDHN9aXY+ju/fIezwTxHB2cMbL9GYz6whMCgQHfzYfBRGRooHFx7VPeLCIdFSwBtBIbD5aOnmurQRTileV0CWVsDoD2ib4cP7V4/CE+cNtTuYg5t+/Kq4c9KhcZ3Pvspq4cPh3VvjMXMpQyI6eLGmEMXo3uGHbIabh2ApLK2MOCP6iKDkbKu2l4YsD3nPyUZOoFYtvPbcg3DOGNolYA3gRHK7BO9edXST/Cwial6sKUTRoVUqzhvRFf8+Z0jA9hq/Yu6fj8X7V4deKPMLy+w0GEDgqKFDO2biluP74i7zDv/EgR1CcvpHW/3pjGF1S0KePbxL/U6GiCgOrCnE8PczjAVq7nx/CcqqajHh0Pa44MjuaJXmDbigP3PBMFz1+nzc83FgX8M/zx6ES1/+CYCRMdRa4eqsw7vYq771yknHGnMyW3ZG5KDwj7MG4+9nDERJRfyJ+IiI6oM1hTi9d/UoXHdsbzx/8eH2iB9nf8DEw0JXtpp56ziMNZd8DJ5tnNXCZ6eT/uamsfZ254L1wdwuQYrHzYBARAnDmkKc+nbItFcWs6SYF/Vrxx8CEcFtE/thSNcsnPf8jwDqhnIuu/f4sOschNMYQ0iJiBqKQWEfiEjAugZXje0V8HqKeYFPT+GvmYgODLxaJYDbJaj1q12TiMc9J/cPSX1BRNTUGBQS4LPrRmP6yh3wuuMPCpeO6pHAEhERxYdBIQHC9T8QER0IOPqIiIhsDApERGRjUCAiIhuDAhER2RgUiIjIxqBAREQ2BgUiIrIxKBARkU2Cl3nc34lIAYD1DXx7WwCFjVicAwHPOTnwnJPDvpxzd1UNv3KYwwEXFPaFiOSp6vDmLkdT4jknB55zcmiKc2bzERER2RgUiIjIlmxB4bnmLkAz4DknB55zckj4OSdVnwIREUWXbDUFIiKKIimCgohMFJGVIrJaRG5v7vI0FhF5SUR2iMhSx7Y2IvKViKwyv7c2t4uIPG7+DhaLyLDmK3nDiUhXEZkuIstFZJmIXG9uP2jPW0RSRWSuiCwyz/lec3sPEZljnvMUEfGZ21PM56vN13Obs/z7QkTcIrJARD4xnx/U5ywi+SKyREQWikieua1JP9sHfVAQETeApwCcAKA/gPNEpH/zlqrR/B+AiUHbbgfwjar2BvCN+Rwwzr+3+XUlgGeaqIyNrQbATap6KICjAPzR/HsezOddCWC8qg4GMATARBE5CsBDAB41z3kXgMvN/S8HsEtVDwHwqLnfgep6AMsdz5PhnMep6hDH0NOm/Wyr6kH9BWAkgC8cz+8AcEdzl6sRzy8XwFLH85UAOpqPOwJYaT7+D4Dzwu13IH8B+BDAccly3gBaAJgP4EgYk5g85nb7cw7gCwAjzccecz9p7rI34Fy7wLgIjgfwCQBJgnPOB9A2aFuTfrYP+poCgM4ANjqebzK3Hazaq+pWADC/tzO3H3S/B7OJYCiAOTjIz9tsRlkIYAeArwCsAVCsqjXmLs7zss/ZfH03gOymLXGj+DeAWwH4zefZOPjPWQF8KSLzRORKc1uTfraTYY1mCbMtGYdcHVS/BxHJAPAugBtUdY9IuNMzdg2z7YA7b1WtBTBERLIAvA/g0HC7md8P+HMWkZMA7FDVeSIy1tocZteD5pxNo1R1i4i0A/CViKyIsm9CzjkZagqbAHR1PO8CYEszlaUpbBeRjgBgft9hbj9ofg8i4oUREF5X1ffMzQf9eQOAqhYD+BZGf0qWiFg3ds7zss/ZfL0VgKKmLek+GwXgFBHJB/AmjCakf+PgPmeo6hbz+w4YwX8EmviznQxB4ScAvc1RCz4A5wL4qJnLlEgfAbjEfHwJjDZ3a/vF5oiFowDstqqkBxIxqgQvAliuqo84Xjpoz1tEcswaAkQkDcAEGJ2v0wGcZe4WfM7W7+IsANPUbHQ+UKjqHaraRVVzYfzPTlPVC3AQn7OIpItIpvUYwK8BLEVTf7abu2OliTpvTgTwC4x22DubuzyNeF7/A7AVQDWMu4bLYbSjfgNglfm9jbmvwBiFtQbAEgDDm7v8DTznY2BUkRcDWGh+nXgwnzeAQQAWmOe8FMDd5vaeAOYCWA3gbQAp5vZU8/lq8/WezX0O+3j+YwF8crCfs3lui8yvZda1qqk/25zRTEREtmRoPiIiojgxKBARkY1BgYiIbAwKRERkY1AgIiIbgwKRSURqzeyU1lejZdQVkVxxZLMl2l8lQ5oLonjtVdUhzV0IoubEmgJRDGaO+4fMNQ3misgh5vbuIvKNmcv+GxHpZm5vLyLvm+sfLBKRo81DuUXkeXNNhC/N2ckQketE5GfzOG8202kSAWBQIHJKC2o+Osfx2h5VHQHgSRg5eGA+flVVBwF4HcDj5vbHAcxQY/2DYTBmpwJG3vunVHUAgGIAZ5rbbwcw1DzOHxJ1ckTx4IxmIpOIlKpqRpjt+TAWuVlrJuPbpqrZIlIII399tbl9q6q2FZECAF1UtdJxjFwAX6mxUApE5DYAXlW9X0SmAigF8AGAD1S1NMGnShQRawpE8dEIjyPtE06l43Et6vr0JsHIYXM4gHmOLKBETY5BgSg+5zi+/2A+/h5GBk8AuADALPPxNwCuAuzFcVpGOqiIuAB0VdXpMBaUyQIQUlshaiq8IyGqk2aubmaZqqrWsNQUEZkD40bqPHPbdQBeEpFbABQAuMzcfj2A50Tkchg1gqtgZLMNxw3gNRFpBSPr5aNqrJlA1CzYp0AUg9mnMFxVC5u7LESJxuYjIiKysaZAREQ21hSIiMjGoEBERDYGBSIisjEoEBGRjUGBiIhsDApERGT7f9Y2NwQ49wt9AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1e025320c50>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "\n",
    "plt.plot(range(1, len(average_mae_history) + 1), average_mae_history)\n",
    "plt.xlabel('Epochs')\n",
    "plt.ylabel('Validation MAE')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "It may be a bit hard to see the plot due to scaling issues and relatively high variance. Let's:\n",
    "\n",
    "* Omit the first 10 data points, which are on a different scale from the rest of the curve.\n",
    "* Replace each point with an exponential moving average of the previous points, to obtain a smooth curve.\n",
    "\n",
    "因为纵轴的范围较大，且数据方差相对较大，所以难以看清这张图的规律。我们来重新绘制一张图。\n",
    "* 删除前 10 个数据点，因为它们的取值范围与曲线上的其他点不同。\n",
    "* 将每个数据点替换为前面数据点的指数移动平均值，以得到光滑的曲线。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYwAAAEKCAYAAAAB0GKPAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzt3XecXGX1+PHP2d5LtiXZ7GbTC5DGUkIIvSqKCIqgiKDfWFDh+0Us+BUVFUX8IfgFRaQJgog0EWkBQiedkLbpPdls772c3x/3zuzsZnZ3drOzs+W8X6997cyde+88Nyxz5rnP85wjqooxxhjTm7BQN8AYY8zwYAHDGGNMQCxgGGOMCYgFDGOMMQGxgGGMMSYgFjCMMcYExAKGMcaYgFjAMMYYExALGMYYYwISEeoGDKT09HTNy8sLdTOMMWbYWLNmTamqZgSy74gKGHl5eaxevTrUzTDGmGFDRPYGuq/dkjLGGBMQCxjGGGMCYgHDGGNMQCxgGGOMCYgFDGOMMQGxgGGMMSYgFjCMMcYEZEStwzDGmNHkgx2l7K+oZ2xyLKdOTSc8TIL6fhYwjDFmGFJVrnxgBQDpCdGsvPnsoL+n3ZIyxphhaEdxrffxCXmphAW5dwEWMIwxZlhauafc+/iEvDGD8p52S8oYY4ah3SV1xESG8eer8lk8NX1Q3tMChjHGDEOHqhoYnxLL6dMDSjQ7IOyWlDHGDEMHKxrITokd1Pe0gGGMMcPQwcpGCxjGGDMaqCp/eWcXO0tqe9+5i8aWNkprmxhvAcMYY0a+9Qeq+NVLBfz1gz19PnZfeT0AuWPiBrhVPbOAYYwxIfDcRwcBWLuvos/Hbi9yeiXTshIGtE29CVrAEJEcEVkmIgUisklErvezz00iss792SgibSIyxn3tAhHZKiI7ROSHwWqnMcYMtuKaRp5YsY8wgYLCGuqbW/t0/LaiGkRgSsYICRhAK3Cjqs4CTgauE5HZvjuo6h2qOk9V5wE/At5W1XIRCQfuBS4EZgNXdD3WGGOGo3e2lXDSbW/Q3NbOxfOyaWtXDlY0BHz8vz8+xGubi8gdE0dMZHgQW3qkoAUMVS1U1bXu4xqgAMju4ZArgL+7j08EdqjqLlVtBp4ELg5WW40xZrA8u/YAMRHhPPbVE/l8fg4AxTVNAR3b0tbOd/7+EQWF1UzLTAxmM/0alDEMEckD5gMrunk9DrgAeMbdlA3s99nlAD0HG2OMGRbWH6zi1GnpLJ6WwdjkGACKqht7PU5VeXLlPu/zwR6/gEEIGCKSgBMIblDV6m52+xTwvqp6kqP4y6Kl3Zx/iYisFpHVJSUlR99gY4wJkprGFnaV1HFcdjIAmYnRABRV997D+GBnGT/51ybv80np8cFpZA+CGjBEJBInWDyuqs/2sOsX6LgdBU6PIsfn+QTgkL8DVfV+Vc1X1fyMjMFbIm+MMX21crfznXhBbioA8dERJEZHUFTdiKqyraim22N3l9Z1eh6KgBG0XFIiIsCDQIGq3tnDfsnA6cCXfDavAqaJyCTgIE5AuTJYbTXGmMHw7vZSYiLDyM9L9W7LSIrmyVX7eMRdj3HSpDH86pJjmdpljMI3nfnT31hI/iBlqPUVzB7GIuAq4CyfqbOfEJFviMg3fPa7BHhNVb3hU1VbgW8Dr+IMlj+lqpswxphham9ZHU+t3s8Z0zM7zW66dMEE5ud0BJAVu8v58XMbjzh+6+EapmTE8+aNp4ckWEAQexiq+h7+xyK67vcI8Iif7S8BLw14w4wxJgSWbi6ivrmNH39yVqft1505levOnMpLGwr51uNrAWhrV5ZtLeaDHaX8+JOzaW5t5+MDlVy6YAKTB3nthS9Lb26MMYNgb1k9STERTEj1n//pE8eN4+GvnMA1j6xi9d4Krnl4FQAT0+KZMTaR+uY2Fk1NG8wmH8FSgxhjzCDYU1ZHXno8zvCuf2fOzOSiOeM6bfvf5zdy77IdAJw0yQKGMSPW3rI6fvD0eh77cA81jS2hbo4ZZKpKY0sb4AaMtN5nNn3nrGlccWIO//nuqd5tb20tITslltT4qKC1NRAWMIwJol+8WMA/Vu/nJ//axC3/snkbo83P/72ZmT95hcKqBg5WNAQ0FXbG2ER+/dk5HDM+mb98Od+7febYwV/Z3ZWNYRgTRAcq6jlnVhaltU0UFHa3btWMBA3NbRysrGdqZiIPv7+bf607xLr9lQAs/PWbgHPLqS/OnZ3FrHFJFBRWM8MChjEjW1F1I/l5qeSlxfHY8r20tythYb1OHjTD0O9e28qD7+32Pk+OjUQE1M1RkZkYzdwJyX0+7xdOyOG1zYf5zPzQZ0eygGFMkBxzyyvUNbeRmRhDekI0Ta3tHKxsIKcfRW9UlXaFcAs2Q1bXldiv3LCY5NhIZt/yKgDv/eCsHge8u3P1KXlcfUreQDTxqFnAMCYIWtraqWt2BjuzkqKZ6A527i6t61fAuOaRVXyws4xtv7xwQNtpBo66XYlrFuWROyaOcckd02ePzU4iKmL4DxlbwDAmCKoaOmZEZSRGM87NSno4gKyk/ry11Ums2dzaPiI+eEaiwqpGzpmVxU8/dUyn7etuOZfoiMGtWxEs9pdnTBBU1jd7HyfHRpGZ6ASMkgDrHnRnZ0lt7zuZQVPf3MrCX7/B4yv2UljVyPiUmCP2SYmLIjbKAoYxphuV9U4P48sLJ7IgN4XYqHASYyIo7mMPw5PF1GPTIZtp1V/VjS1UD/BamL9+4ASKHz+3kaqGFm99i5HKAoYxQeAJGJcdP8E70JmZGB1wZTWAHcU1nHTbG51m3vx+6TaaW9sHtrFDxIc7y6iqD97ixiWPrmbOz17rlPX1aP1r3cFOz88/ZuyAnXsosoBhTBBUuLekUmI7VuZmJcUEVFnNw/PB9sv/FAAwPzeFg5UNFFYFXv95uKhvbuWKvyxnyWOrg3L+oupGlu9yalG8tbV4QM65v7yeLYc76lf89znTmRLCxICDwQKGMQOsvV15fIVTSjMlPtK7vS89jNqmVh5+f0+nbbPHJQFQVtfs54jhzVNxbuPBqgE/t6ryP0+t8z4/UDEwAXfp5iIAjhnv/HfxN34x0ljAMGaAvbb5sHeFb2J0x0TErKQYimuaOo1JdOd3r25lxe7yTtsWTnESz63cXT7i8lIdrnJ6XgOxqLGtXalubOG4n73Ksi3FlNc18/6OMq5dNInpWQkcquwcMFbuLqe1re+3+ZZuLmJaZgJfWzwJgLk5KUfd9qHOptUaM8Be+NipJpwSF9lpoVZGYjTNre1UNbSQEtc5iZxn9lSGW+P5QEV9p9fX3XKud13Hb17ewr1v7mDNT84d9lNs29qVuuZWimucgBFxlAGjvV05/Y5l3l7Eb1/dyq0XO9NcF09PZ3dpLQd9AsbmQ9V8/s8f8l+LJ5GVFENJbRM/unCW33P7qqxvZuWecr5+2mQumT+Bs2ZkkRwX2etxw93w/mszZgjZW1bHsq3FvLapiGsW5bH8R2d3ej0zybllcaiykV++uJn95R1B4YRfvc7CX78BOL2L1wuc++w/unAmL37nVFLiokjzyVRa09TK+ztKe2zP9qIaXtl4eECuLVjueHUrc372GjtLnFXSFfUt/Pntnf0+3+6yuk63nMIEdrlTkaekJ5CdGtspYHh6G395dze//E8Bjy/f12sPsLWtnS/cv5y2duXc2VkAoyJYQBADhojkiMgyESkQkU0icn03+53hlm/dJCJv+2zfIyIb3NeCMxJmzFF4d3sJ+8vrqW9uBeCTf3iPax5eRWu7cu2iSZ3KcAJkub2HpZuLeOC93fzw2fVAx4dWa7uy+VA197i1D3504Uy+fvoUjs128g/5ni8mMqzHYPCLFzdz7u/f4Rt/W4Oqsqe0rl+3XYLtxfVOb+yxD/d4t/365S2A01t4bPneI24h9WTDgc5jIM2t7ewqqSMqIozs1FhmZCVSWd/CtqIayuua+e2rW7z7RoYLtU2tvS6uPFDRwJbDNcwcm8jcCSP/NpSvYPYwWoEbVXUWcDJwnYjM9t1BRFKAPwKfVtVjgM91OceZqjpPVfMxZgipbmzhqgdXsvi3y7j52Q2AM1ANMCMr0W/6D08P4/UCZ7DUM/X2PZ+ewq9fdmZEPfG1k7j21Endvv8Z0zN5b0ep32/Dza3tnabi/vGtnZzxu7d4Zu2BPl3jYEhLcIJoRZfptN/758d88YEV/OT5jTzw7m5/h/q1ak85MZEdH2t7y+t5f2cps8YlER4mXHDsOMLDhOc+OsjtL29hW5HT+0iJi+SXnzkWgJ3FdX7P7XHInaV2y0WzR10iyaAFDFUtVNW17uMaoADomm7xSuBZVd3n7jcw892MCbLCyo5voR/4rB9IjI7gz1cd7/eYTLeHscGdCbTpUDX/WneQXSV1hIcJeWlxvLu9lJjIME6anEZk+JH/e84cm8iJk8awaGoaBysb2F/e+dt3cXWjd/aOxx2vbgVg3f6Bn4F0tPaX15OVFE1cVDif9cnG+vSaA3y4qwyAw9W99zCW7yrj8/d9yD9XH+ATx47jlRsWc8M502hubWfjwWrOcdOKZyRGs3haOn96aydPrdnfcfyPzvamHt9WVOP3PTwOuf/tx6X4L7U6kg3KGIaI5AHzgRVdXpoOpIrIWyKyRkS+7POaAq+525cMRjuNCZTntsWcCcnUNbWyrdj5kPnDFfPJ66ZITnx0BPkTUztt+9/nNrKntI6JaXF89+xpAMwcm9RtVtpXbjiNp76+kBljnamcu8s6fxs+/653uO6JtQBM7tKOv6/cd8RCs1CqbmyhvK6ZaxZN4qNbzuXOy+ex87ZPcN+XFniDKzjf+HsbV/ivv65m5Z5ymtvauWrhRGaOTeIbp0/xvv6pueO9jy9xA5MqnJg3hqsXTiQmMpyMhGiyU2JZ7gaq7hS6t8jGjfBV3f4EfZaUiCQAzwA3qGrXvAYRwPHA2UAs8KGILFfVbcAiVT0kIpnAUhHZoqrv+Dn/EmAJQG5ubjAvxRivInca6ILcVNYfqGKlOwV2WlbPC7ee/uYpbDlcTVF1Ez//9yb2ldWzubCaaZkJfHrueP7y7m4WT0vv9f09H1aedgCs21/pvbWTPzGVf3x9IVNufqnTcdc/uY6L54W+rgLAQXdwekJqrDc5n+e2UXVjK99/ej3TMhPYWlTDzc9t5NefPa7bc9W7ZVCBTmM+//zGQpJiIjsF8U/PHY8qJMZEcPasLO92EeG06Rn8++NDNLa0cdl9H3DRnPGdAg84t6TS4qOOGKMaDYLawxCRSJxg8biqPutnlwPAK6pap6qlwDvAXABVPeT+LgaeA0709x6qer+q5qtqfkZGRjAuw5gjeHoYx7s9hjcKioiPCic7gNsUM8cmcfr0DG65aDat7cq+8nry0uOJCA/jpe+eyo3nzej1HJlJzjfwQp+A8egHe0iIjuD+q47nkWtPJDxMuOz4CQBcusD5nRA9dGbSe/4NfdOAe3zu+Am8fP1ibr3YGVf4+8p93Z6nsr6ZtvaOHojvrbwT8sYcUalORPjM/OxOwcLjojnjqG1q5Tt//4iNB6t5bVPHxIIPd5Zx83MbWLGrvNte5EgXtL8ecSagPwgUqOqd3ez2L+AeEYkAooCTgN+LSDwQpqo17uPzgFuD1VZj+kJVWbm7nDHxUd4exdp9lczNSelTgZzjfW5PeRZ9BXp8dEQ4afFR3g/dtnbl5Y1OVbbzfPIZ/fbSOVyzKI9ZY5NIiA7n8RX7aG1rJ8LP+Mhg8yzW85ewT0SY5a5sv+n8Gdzx6laqGlpIjo2kuKaR9PhoRJwBfc/q8CtOzOGzbmDsr1OmpHFCXqp3HGjjwWqaWtuIjgjn8RV7eXF9IQB3nzPtqN5nuArm141FwFXABhHxrMu/GcgFUNX7VLVARF4B1gPtwAOqulFEJgPPuf/zRABPqOorQWyrMQF7f0cZ7+0o5fiJqczISmR8cgyHqhqZmdW3msuJMR1z909xV3H3RVZSjDev1KHKBhpa2o4oARoWJhwz3tk2e3wSre1KYVVjv4o49eTNLUUkx0Zy/MQxve77x7d28PxHB5k1LgkROo1X+ONJiXLpnz7g4a+cwOLfLgOOTLVywznTyUo6unEFEeGiOeNZtacCgOa2dgoKa5iXk8K2ohpOmjSGry2ezDmz+labe6QIWsBQ1feAXr8uqeodwB1dtu3CvTVlzFDS2NLmnQb7f1fMR0S46YIZ/HP1Ab515pRejj7SLy4+hjV7K0hP6PlD05+JaXG8vPEwJ9/2BjPHOcFqcg/J7zxV/7YV1QxowFBVrn3EWSq15zef7HX/J1fuZ195PduKaslIjPY7G8xXfl4qJ08ew/Jd5Xz2Tx94txfXNDEjK5Gt7qym3gJPoC6eN57XC4q46uSJLHlsDdsO1zB7XBK7SupYclqWd7HeaDR0bmgaMwxc9/ha3thSTESYeAeeL5k/gUvm9+9WyFUL87hqYV6/jv3fi2bT1q68trnIe2tqSkb399bn5aQQHxXO6wVFfu/f91fXldPjexjHaWpto7CqgeOyk9l4qIqxAfQIEmMieXLJQn7z8hbu81kF/sTXTuKESWO46/Vt1DW19atetj8pcVE89tWTaG9XYiLD2FpUw7aiGlrb9YjxkNHGAoYxffDGFmepUGu7DtgHVH9lp8Ry/5fz+fFzG3h8xT5iIsMYEx/V7f4xkeGcMTOTVzYe5qyZWZwzK7Pf16CqzL7lVb566iTvWAPA6r0VfLqHgLHk0TW0tCnfOH0KDS1txPWhEt03T5/iDRi+PZmbzp/ZjyvoXViYMD0rkS2Hq3lxfRjhYcKiqb3PYBvJLGAY4+Mv7+xia1ENv/vckXdEnx2CK6UBfnXJcdx68bE0tfb+LXtKRgL/WV/Ifz26mlsvPoYvL8yjrLbJu+I6UEXVTTS0tHHPsh1M9unV+ObH6kpVWb2nnNS4SM6amdnnsqXJcZG89N3F3lQsg+GUKenc9/ZO3t9RxjmzMvt163AkCf1UCWOGiKfXHOBXLxXw9JoDR+RdUlV++8pWpruzogJZKzGYwsOEuKjev/+N95mR9NTq/Xywo5Tjf/l6n4oK1TS2sOj2N73Pd5XUMWdCMukJ0ewr6z5gVDe2UtfcxrfOmNrvGtezxyeRn9f7wPpAue7MKZw2PYNFU9O47ZLu14GMFtbDMMb1m5c7EtHtLa/vVD1tT1k9h6sbue6sY7nw2LFDaj1DX/hOYd1VUudNU/LmlmLOmBHYzJ+P91d1WvcAsGhqOit2lbG1qIb2dvWbY8mTRLCnMY6hJjEmkkev9bsEbFSyHoYZVpz79XsH/Lxt7Up5XROnT3cWf375wZXc4ZPJ9MOdTrqIU6akkZ4QPWxX+fp+WNc3t3nrcJT3oYpf11odnzt+AjecM42cMXGs21/Jfe8cmZ68qr6FLYedRA/jRkFlupHKAoYZNlSd0qc/fm7jER9aR6u8rpl2hZMnO+shDlY2cO+yjg++D3eVkZkYfUR+puGm6yI5T1W/kgBLxwLsr6gnPEx48GonifTXFk8mOiKcaxc52XVXu2sYPFSVy+//kP/+x8cAAa2GN0OTBQwzbFT6pMDeUVw7oOcurXU+MCemxXVKKlda65RUXb6rjIVT0kI+M+poJcVE8rnjJ3DPlfOBjsy56/ZXelde92ZfeQPjU2I4e1YW2391oXeq6dycFC6aM47txZ2zvW44WMWWw862aZkJo37geDizgGGGDd/CNhX1gd9CCYQnYKQnRPPQV07wTk/N/+XrlNY2U1LTxHHZyT2dYti443NzuWjOeHLdxXsJ0RG0tit/Wx7Yrb795fXkpDrHdl10Ny0zkQMVDd6ZTPvK6nm9oJgwgY9vOY+l/3N6t5l4zdBnAcMMWZsOVfH2thLa3QFW34BRVjuwAcNzSyY9IYpZ45J4/wdn4flc8+QVGmnfjD0BcPa4JOZOSOb9nT2XfG1saSPvh/9h3f7KbpPvzRibgCoUFNbw/o5STrtjGfe9tZPZ45NGTRnTkcwChhmyvvG3NVz90Ep+8IxTyrTYJ2D0ZZA2EJ4eRoabXiI2Kpx1Pz2P8DDh+Y+cGhI9LYobjs6Z7cyKyhkTx8Ipaaw/UOWtGujP1sMdt5q6G8vx5JJ6atV+vv3EWqIjwmhpb+fUqZZJeiQYnnMDzahQVO18iP9zzQH+ueYAV508EXDKaQ50wCipaSI6IqzTdNmkmEiOGZ/Eyj3OwHBawsgKGJfMn8C0zESykmLYeriGe5ft5IkVe0mJi+Lz+Tmd9lVV3tzSsVZjSqb/nFUZidFMyYjnH6udanY3nT+DC48dO6ym0pruWcAwQ1J7u9Lc2k5CdIT3W+9LGwrJSIwmLT4qCD2MZtIToo8Y1D42O5n1B5yB4ZF2Swo6ig15AuVtLzlTiccmxXDa9I5ewYe7yrj7je3e55PSup8tduVJE/nFi5u5PD+H686cGoxmmxCxgGGGpDp30HTG2ETW7HWmaZbVNTM/N4XYyPCg3JJK95Pt1HegOzVuZPUwfMVGhXPu7CzW7K2gvK6Zf6ze3ylg7CxxSsFOz0rgSydPZGJa99lurzklzw04Q2s1vDl6FjDMkOTpVcz0CRgAE1LjCJMj5/ofrZKaJiakHvkh6FunIipiZA/5/fGLC4gIE77+2BoKCjuqKe8rq2fb4RrCBF767uJeiy+FhQmfnDMu2M01ITCy/w8ww9Ldr2/nJ89vBGCmTyZUcGoeHJedzMHKBoqqA1s34E9jSxt3Lt3mnf5ZWttERuKRPYiJafFMTIsjKWbkf7eKDA9DRJg9PondpXXUN7fy8f5KTrtjGY8t38uY+OghUanPhE4wS7TmAI8CY3Gq6d2vqnf72e8M4C4gEihV1dPd7RcAdwPhOJX4fhOstpqh5fevb/M+npAay08umk1VQwt/eGM7EWHCCW7yudV7Kvr9TfbpNQf4wxvbaW1r5/TpGZTXNXc7RvHaf59Ga5v6fW0kOmZ8Mqrw4vpC/uOWJAWobmzp4SgzGgTza1MrcKOqrhWRRGCNiCxV1c2eHUQkBfgjcIGq7hORTHd7OHAvcC5wAFglIi/4HmtGh8ToCL566iRa29pJjI7g8/k5xEWHExsZzqo95f0OGI0tbYBTE/qPbzkpQLoLGNER4QzTXIP9cvr0DObnpvD9p53pzNcumsRD7++mubW9lyPNSNdt/1JEvu/z+HNdXruttxOraqGqrnUf1wAFQHaX3a4EnlXVfe5+nnl7JwI7VHWXqjYDTwIX9345Zrir6fItNsG9FRQRHsZ/nTaZ5LhIIsPDmJ+bwuq95f1+H980Ix6jvZqaR1REGN88vaPc7E8umgXApQv6V1XQjBw93ZD8gs/jH3V57YK+vImI5AHzgRVdXpoOpIrIWyKyRkS+7G7PBvb77HeAI4ONGYEKu+Qz6i6NeP7EVDYfqqain7OlDvmUFfWYl5PSr3ONRKfPyOCEvFT+fNXxiAhbf3kBd1w2J9TNMiHWU8CQbh77e979SUQSgGeAG1S1usvLEcDxwCeB84GfiMj0bs7v9yayiCwRkdUisrqkpCTQZpkh6mCXD/LEaP/pJD45ZzztSr9TnXd9nxlZicM2ZXkwREeE889vnML5x4z1PvdX48KMLj0FDO3msb/nfolIJE6weFxVn/WzywHgFVWtU9VS4B1grrvdd6npBOCQ30aq3q+q+aqan5Fh6QeGu64V2+Kj/X+IzxibyILcFN7e1r8vCYVVjd7eS2pcJC9dv7hf5zFmNOkpYMwVkWoRqQHmuI89z3utVSjOktkHgQJVvbOb3f4FLBaRCBGJA07CGetYBUwTkUkiEoVze+yFPlyXGaY2HaoiLT6K9394FvdcOb/HaZxTMhLY10MN6e6oKiU1TVy6IJvoiDD+59zplkHVmAB0O/dDVY+2f74IuArYICLr3G03A7nu+e9T1QIReQVYjzP19gFV3QggIt8GXsWZVvuQqm46yvaYIe5wVSMbD1ZzTHYy2SmxvRbayR0TR1F1E0XVjWQlHVnFra1d+ekLG/nCCbneFBgAdc1tNLS0MT4llq2/vHDAr8OYkapPkwVFJB74DHClqn6yp31V9T0CGOtQ1TuAO/xsfwl4qS/tM8PX8l1lfOH+5QB8a8aUXvZ25LrpKU667Q3W3XIuKV1Sd2wvruFvy/fxt+X72HXbJ7z34EtrOmpfGGMC1+uyTRGJEpHPiMhTQCFwDnBf0FtmRpU9pXXex8cGWKhoQmpHD+Th9/fQ0tZ5nYBveovlu8u8j0u6pDI3xgSmp3UY54rIQ8Bu4DLgMaBcVa9R1X8PVgPN6NDS3jGP4tjxgQWMeTmp/PDCmaTFR3H3G9uZ9/PXOr1eUOjUb4iNDOeFdR1zJjzFkixgGNM3PfUwXgWmAKeq6pfcIGFLPU1QlLnf+gFyxgRWOyE8TPjG6VOYPd7JN1XX3OZ9TVV5d3sp83JSOGHSGG/tajiyWJIxJjA9BYzjgeXA6yKyVES+ijMAbcyA85RcfeabC4+oSdEb38HxqgZnBfeqPRUUFFZzxYk5TM9MYEdxLW1uL2b1ngpS4iIZM4LTlRsTDN0GDFX9SFV/oKpTgJ/hrNSOEpGXRWTJYDXQjA7ldc1MyYj3lvjsC9+0479+qYCqhhZeLygiKjyMT80dz/SxiTS1trO/vJ765laWbi7iwmPH2UI0Y/oooFlSqvo+8L6IfBcnIeAXgPuD2TAzupTWNpE2ALOWnly1n3e3l1LT2EJ+XipxURHMdHNErdxTTkxkOA0tbXx67vijfi9jRptuA4aILOjmpRLg/4LTHDNaldQ2MSOrf8n/vn3WVPaV1/PW1hIiwsSb9uPc2VmAk64b8GZfzUqK5sRJfe/JGDPa9dTDWA1swgkQ0HlNhQJnBatRZnSpqm9hd2ldv7/1ZybG8Mg1J7JmbwXHjE/im39bw7KtJVx2vJNdNTxM+Hz+BJ5afYD0hGh+97m5trLbmH7oKWDcCFwKNOCkF39OVWsHpVVmVFm5pxxVOHlyWu879+D4iakA/PGLx1PZ0ExiTEfiwlsvPpabzp9pM6OMOQo9DXr/XlVPBb6NkwjwDRF5SkTmDVrrzIi0ek85O4prvM/XH6gkTAYuvXhsVDityy+GAAAgAElEQVTjkjtPzY2JDLdgYcxR6nWlt6ruxkkS+BpOYaPpwW6UGdkuu+9DzrnzHSrrm2lpa+dARQPjkmMtvbgxQ1xPg96TcWZDXYxTzOhJ4Feq2tjdMcb0RrVjRfe8W5dy6YIJHKxo6DXRoDEm9Hoaw9iBk0X2X0A1TpbZb3kWVfWQstyYblV0KY36zNoDZKfEcpLNWjJmyOspYNxKR6GkhEFoixnBWtraiQgTDlcd2UE9WNlAdqr1MIwZ6nqqh/GzQWyHGcGqGlqY+/PX+N55071rIrqa3s81GMaYwdProLcxR+NwVSNz3Syy97+zi6Jqp4fxyg2L+d3n5pIcG8mfrzqei+aMC2UzjTEB6FMBJWP6an9FRwlVdZ9HhAlTMhKYOTaJSxdk9znZoDEmNILWwxCRHBFZJiIFIrJJRK73s88ZIlIlIuvcn1t8XtsjIhvc7auD1U4TXHVNrQBcnp9DTWMrb24pIXdMHJFurW4LFsYMH732MEQkGmfFd57v/qp6ay+HtgI3qupaEUkE1ojIUlXd3GW/d1X1om7OcaaqlvbWRjN0Nbg1Kk6eMoZ/rN5PQWE158zKDHGrjDH9EcgtqX8BVcAaoKmXfb1UtRCnpCuqWiMiBUA20DVgmBHMU9Ro7oQUwgTaFSZn2KQ7Y4ajQALGBFW94GjeRETycOpprPDz8kIR+Rg4BHxPVTe52xV4TUQU+LOq+k2n7tbmWAKQm5t7NM00QdDQ7NySSoqNJC8tnl2ldeS7OZ+MMcNLIAHjAxE5TlU39OcNRCQBeAa4QVWru7y8FpioqrUi8gngeWCa+9oiVT0kIpnAUhHZoqrvdD2/G0juB8jPz9eur5vQ8vQw4qLCufeLCyitbWLxtIwQt8oY0x+BDHqfijP+sFVE1rsD0esDObmIROIEi8dV9dmur6tqtScDrqq+BESKSLr7/JD7uxh4DiePlRlm6ptaEYGYiHBmjUuyYGHMMBZID+PC/pxYnOkvDwIF3aUREZGxQJGqqoiciBPAykQkHghzxz7igfNwVp6bYaa+uY24yHArh2rMCNBrwFDVvSIyF1jsbnpXVT8O4NyLgKuADSKyzt12M05OKlT1PuAy4Jsi0opTd+MLbvDIAp5zp1xGAE+o6it9uC4zRNQ1txEbZct9jBkJAplWez3wX4DnltLfROR+Ve2xTKuqvkfnKn3+9rkHuMfP9l3A3N7aZoa+huZW4qMtbbkxI0EgX/2+CpykqnUAInI78CFW19sEoK65jTjrYRgzIgQy6C1Am8/zNnrpORjj0dDcRlyU9TCMGQkC+er3MLBCRJ5zn38GZzDbmF7VNbeSEG09DGNGgkBKtN4JXAOUAxXANap6V7AbZoa/vWV1bC+qJT3BamkbMxL0VKI1SVWrRWQMsMf98bw2RlXLg988M5z99YO9NLe18z/nWhl4Y0aCnu4VPAFchJNDyncFtbjPJwexXWYEeGtrMSdPTiNnTFyom2KMGQA9Vdy7yP09afCaY0aKirpmdpXWcfkJOaFuijFmgPQ6hiEibwSyzRhfZXVOYuOxyTEhbokxZqD0NIYRA8QB6SKSSsdU2iRg/CC0zYTISxsKmZwRz8yxSf0+R1ltMwBp8TbgbcxI0dMYxteBG3CCwxo6AkY1cG+Q22VCZE9pHd96fC0AH//0PJJjI/t1nvI6J2CMiY8asLYZY0Kr21tSqnq3O37xPVWdrKqT3J+5bkoPMwLtLqvzPv5wZ1m/zvHBjlI2HKwCIC3BAoYxI0UgyQf/T0SOBWYDMT7bHw1mwwbT5kPVLHlsNRfNGc8PL5wZ6uaE1L6yeu/jt7eVcMGxY/t0fFNrG1c+0FEnKzXOAoYxI0Ugg94/xckb9X/AmcBvgU8HuV2DanNhNQcqGvjLu7tC3ZSQ21tWT1xUOBfNGcfLGwtpbm3v0/EbD3bUyEqMiSAqIpDsM8aY4SCQ/5svA84GDqvqNThZZEfUSGZdk1NGtK1dUR3dRfv2ldeTOyaOzy7IprK+hfd3lPbp+DV7O9ZzRoVbsDBmJAnk/+gGVW0HWkUkCShmhC3aq3UDRtfHo1FZXRMZidGcMiUdgGseWcU/V+/ncFVjQMdvKawBYNHUNK4/Z1ovextjhpNAssKtFpEU4C84s6VqgZVBbdUgq2/uCBJF1Y0kxvRvZtBIUN3QwvjkWGIiwzljRgZvbS3hpqfXk5EYzds3ndFrqvKdpXUsmprG4187eZBabIwZLIEkH/yWqla6FfLOBa52b031SERyRGSZiBSIyCa3EFPXfc4QkSoRWef+3OLz2gVuHfEdIvLDvl5YX9Q1dWRvL6puCuZbDXk1ja0kxjhB4a7L5/GnLy4gKymakpomXlxf2OOxqsquklompycMRlONMYOsp4V7C3p6TVXX9nLuVuBGVV0rIonAGhFZqqqbu+z3ricNic/5w3HWepwLHABWicgLfo4dEL63oQK99TJSVTe2kOSuvUiJi+LC48Zx/jFjmfPz1/j+0+sZExfFObOz/B5bWttMTWMrkzPiB7PJxphB0lMP4/+5P/cCK4D7cW5LrQD+0NuJVbXQE1RUtQYoALIDbNeJwA5V3aWqzcCTwMUBHttndU2tTEyLI0xgb3l97weMUM2t7TS2tJMU0/l7RFiYeHsdX3t0tXf7vct2cOrtb3onCryy0emBzM1JGaQWG2MGU0/JB88EEJEngSWqusF9fizwvb68iYjkAfNxgk1XC0XkY+AQziLBTTiBZb/PPgeAk/rynn1R29RKalwU4SLsKK4J1tsMeTWNLQB+x3CykmIorGpkYlocNz+3gc2Hqlm3vxKA6gbnNtbTaw4wZ0Iy8y1gGDMiBTLoPdMTLABUdaOIzAv0DUQkAXgGuEFVq7u8vBaYqKq1IvIJ4HlgGv5LwPqd7yoiS4AlALm5uYE2q5P65jYSoiPITIxme1Ftv84xEtQ0OrfmkmKP/LO458r5nHr7MlrblCdW7Ov02nVPrGVHcS2VDc187vgcRKyCrzEjUSDTagtE5AF3gPp0EfkLzu2lXolIJE6weFxVn+36uqpWq2qt+/glIFJE0nF6FL55sSfg9ECOoKr3q2q+quZnZGQE0qwj1DW1EhcVztTMBHaX1vV5sdpwd7iqkbN+9xa3vLAJgMToI3sYE1KdtRkHKxuOeO29HaUcrm6ksaWdCamxQW+vMSY0AgkY1wCbgOtxkhFudrf1SJyvmQ8CBW6ZV3/7jHX3Q0ROdNtTBqwCponIJBGJAr4AvBBAW/ultsmpOz0tK4HWdmWvTz6l0aDgcDW7Sut4Z1sJgHfQu6sxPmk+zpyRwbfOmHLEPhNSrViSMSNVILmkGoHfuz99sQi4CtggIuvcbTcDue5578NZRf5NEWkFGoAvqDOC2ioi3wZeBcKBh9yxjaCoa2olPjqCaZmJAGwvrmVaVmKw3m7Iqaxv7vQ8Idr/n0WqT+bZn1w0m0np8fz1gz3UNXdMS7YehjEjV0/Tap9S1c+LyAb8jB+o6pyeTqyq7+F/LMJ3n3sAv5lv3VtUL/V0/ECpa24jPjqCKRkJiOCMYxw3GO88NFTUOYPdj157Ig+8t5tJ6f6nxfomEsxMikFEyE6NZZvPuI+VYzVm5Oqph+FZaHdRD/uMCJt+fj7tqkRHhDMpLZ4XPj7ItafmjZoV35X1zYjAoqnpnDa9+3Gg1LiOfw9PL8QpkFTLbz57HIunZ1j9C2NGsJ7qYRS6v/f6+xm8JgZfZHgY0RHhAPzkU7PZWVLX66rmkaSivoXk2EjCw3qe3eRZXzFnQrJ32xi33kVYmJCdYrejjBnJerolVYP/qawCqKr2v37nEHbG9AyyU2J5o6CYK07s3zTd4aaivjmguhXjU2LZfOv5tLV3/FlcvTCP/6wv5IS8McFsojFmCOhp4d7oGfX1ISKcOjWdpQVFoW7KoKmobyYlLrDbb12TD544aQx7fvPJYDTLGDPEBLJwDwARyaRzxb19Pew+rGWnxlJe10xza/uoKABUUdfC2OSY3nc0xoxqgVTc+7SIbAd2A28De4CXg9yukMpMdOpDldSOzMy1j324h7k/f40fP7eByvpmdpTUkpdmCQONMT0LpIfxC+Bk4HVVnS8iZwJXBLdZoZWV5HzbLqpuHJEDuR/tr6SqoYXHV+xjT5mzsv1Tc8eFulnGmCEukPstLapaBoSJSJiqLgMCziU1HGW4PYwH3909Iku21ja2eq/x/R1lnDs7i3mWMNAY04tAAkalm0DwHeBxEbkbp9bFiOXpYfxnQ2GnRWkjRW1TKxN9FtjddslxljDQGNOrQALGxThpO/4beAXYCXwqmI0KtTSfxWeH/CTbG+5qm1pJiIngujOncHl+jre3YYwxPelpHcY9wBOq+oHP5r8Gv0mhFxYmPHzNCVzz8Cq/2VmHq9qmVrYerqG2sZWJafHcdP7MUDfJGDOM9NTD2A78PxHZIyK396UGxkhw2rQMwsOEwqqREzD+uXo/l933AQcrG7pNMGiMMd3pKTXI3aq6EDgdKAceFpECEblFRKYPWgtDJDxMGJsUQ2HlyKnxXV7XjCo0tbZ7S64aY0ygeh3DcHNH3a6q84ErgUsIsIDScDcuOWbE3ZLySLQehjGmjwJZuBcpIp8SkcdxFuxtAy4NesuGgPEpsRRWDf8exs9e2MSbW4qo8wkYCdbDMMb0UbcBQ0TOFZGHcMqlLsGpTTFFVS9X1ecHq4GhNC4lhsKqBtrbh+9ajKqGFh75YA/XPrK6Uw/DxjCMMX3V06fGzcATwPdUtXyQ2jOkZKfE0tKmlNY1kZk4PHMtFRRWex/XNnVUxrMxDGNMX/WUrfbMozmxiOQAjwJjgXbgflW9u5t9TwCWA5er6tPutjZgg7vLPlX99NG0pz/GJTtpQQ5VNg67gLG3rI6n1xwg2ac+d21jC/NzUzg+N5WFU9JD2DpjzHAUzK+ZrcCNqrpWRBKBNSKyVFU3++4kIuHA7Tj1u301qGpIp/KOT3GCRGFlw7BLnfH9p9ezYnc5UzMTvNu2Hq7hlKnp/O9Fs0PYMmPMcBW03N2qWqiqa93HNTgzq7L97Pod4BmgOFht6a+xboqQw9XDb+A7OtKpILijuJZjs51aV3XNbTY7yhjTb4NS7EFE8oD5wIou27Nxpune5+ewGBFZLSLLReQzPZx7ibvf6pKSkgFsNaTGRREeJpQOwzTn45I6bqFdcWIui6c5t6DiLWAYY/op6AHDTVz4DHCDqlZ3efku4Aeq2nbkkeSqaj7O2o+7RGSKv/Or6v2qmq+q+RkZGQPa9rAwIS0+itKa5gE972DwLfx03uyxnGglVI0xRymoXzdFJBInWDyuqs/62SUfeNLNlJoOfEJEWlX1eVU9BKCqu0TkLZweys5gttef9IToYdnDqG92YvDXT59MRmI008c6FXf3lteHslnGmGEsaD0McaLAg0CBqt7pbx9VnaSqeaqaBzwNfEtVnxeRVBGJds+TDiwCNvs7R7ClJ0YPy8p7DS2tTMmI50cXzgLg2OxkAI4ZnxTKZhljhrFg9jAWAVcBG0RknbvtZiAXQFX9jVt4zAL+LCLtOEHtN11nVw2W9IQodhTVhOKtj0p9cxtxUR3/ebNTYnn7pjO8U4WNMaavghYwVPU9IOCqPKr6FZ/HHwDHBaFZfZaRGE1pbTOqOqyKDNU3txEbFd5p20Sr222MOQqDMktqOBubFENzWzu/e20rTa3+xuaHlrte38Y720poaG4jrkvAMMaYo2FzLHuRneLcwrl32U7SE6K5ZtGkELeoe02tbdz1+nYApmTEkxNlt5+MMQPHehi9yE7t+NCtbQxtKfNDlQ1sPFjV7eu7S+u8j+ub24iNtO8DxpiBYwGjFxNS47yPK+pbQtgSuPDud7no/95D1X/23O1Ftd7HhVWNdkvKGDOgLGD0wjd534GK0K5hqGpwApa/VCX1za3c9lLnulYWMIwxA8kCRgCuP3saAPsrQlt9z1PDYsOBI29LFRRWU1jVyOfzJzAxzekV+a72NsaYo2WfKAH473On86WTczlcFdqA4QkEWw4fuS7EUxnw2lMn8eWFeQCMiY8atLYZY0Y+GxUNUHpCNBX1LbS0tRMZHto4W+gncBVWOgFjXHIsXz11EidNGtMptbkxxhwt62EEKCMxGoCy2tAlImxw80O9s62UHcW1nV7zDHInuZX0js1OJibSxjCMMQPHAkaA0hOcgBHKRIQNLU7AOFjZwDl3vt3ptcKqBsYlxwyr1ejGmOHFAkaAPD2MkprQBQxPBlp/CqsaLU+UMSaoLGAEKMPtYYQyc62nh+HR1t6xHuNwVSPjkodX3XFjzPBiASNAnh7Gx/srQ/L+be1Kc2s7p05N927z3B5rbWunuMYChjEmuCxgBCgmMpxLF0zg8RX7OFjZwKubDrP5UNcCgsHj6V2cNj2dB76cDzipQgCKa5poVxiXYrekjDHBYwGjDz67IBuAvWV1fP2xNXziD+/ywY5SKuuDP3PKM0MqNjKc8W5gOOgGDM8027HWwzDGBJEFjD7w3PK587Vt3m1XPrCCP74V/MqxnoARExlOXnocInin1noW7Y23QW9jTBAFs0RrjogsE5ECEdkkItf3sO8JItImIpf5bLtaRLa7P1cHq5194ZmFtHpvRaftVYOQlNBzSyouKoK4qAhyx8Sx1V3x/dKGQuKjwskZYwHDGBM8wVzp3QrcqKprRSQRWCMiS7uWWhWRcOB24FWfbWOAnwL5gLrHvqCqnT+pB1nXCnYelQ3BvyVV39zqtsGJ8TPHJrL1cA0bDlTx0obDXH/2tE4lWY0xZqAFrYehqoWqutZ9XAMUANl+dv0O8AxQ7LPtfGCpqpa7QWIpcEGw2nq0BmNtRl2TZwzDCQqzxiWxu6yOe5ZtJzk2kq8tHrqFnYwxI8OgfCUVkTxgPrCiy/Zs4BLgLOAEn5eygf0+zw/gP9gMuoe/cgKKcuaMTNYfqOLh93ezZl/wOz57ypziSLluAsJ5OSmowqubijhtegaJMZE9HW6MMUct6AFDRBJwehA3qGrXeah3AT9Q1bYuKS385bfwWzVIRJYASwByc3OPvsG9OHNmpvfx3JwU0hOiKalpQlWDmpZjR3EtcVHhjHcH3uflpHhfy0uL6+4wY4wZMEGdJSUikTjB4nFVfdbPLvnAkyKyB7gM+KOIfAanR5Hjs98E4JC/91DV+1U1X1XzMzIyBrT9gchIjKaxpZ21+4K7oG9HcS1TMxO8QSklLopMdzHhxLT4oL63McZAcGdJCfAgUKCqd/rbR1UnqWqequYBTwPfUtXncQbAzxORVBFJBc7DZ1B8KDl5choAv/rP5l72PDo7S2qZktE5XfkJk8YAkJ5gdS+MMcEXzFtSi4CrgA0iss7ddjOQC6Cq93V3oKqWi8gvgFXupltVtTyIbe23uTkpXLtoEk+s3EtbuxIeNvC3pZpb2zlc3UjOmM63nn7+6WNIiong7FlZA/6exhjTVdAChqq+h/+xiO72/0qX5w8BDw1ws4Li2OwkGlva2VVSy7SsxAE//+GqRlRhQmrndRbpCdH8+rNzBvz9jDHGH1vpPQCOGZ8MwKYg5ZY6UFEPwATLFWWMCSELGANgUno84WHCzpLa3nfuo7qmVq58wJmNPCHVZkMZY0LHAsYAiIoIY+KYuKAEjO0+pVgtuaAxJpQsYAyQyRkJQbkl5Ulh/qcvLiAqwv5zGWNCxz6BBsjcCcnsLavn4fd39/nY+9/ZyeV//tDvawcrnIBxik/hJGOMCQULGANkyemTSY2L5IOdZX0+9raXtrBidzmNLUfW7D5Y2UBCdARJMZZY0BgTWhYwBkh0RDjzc1M54PYI+mNXSd0R2w5WNpCdEhvUtCPGGBMICxgDaEJqLAfdKbCB8qQtB9jhZ9D8YEUD41NssNsYE3oWMAZQdkos1Y2tVDf6L6hUXNPIKb9+g2VbOzK5bzhQ5X288WDVEcccrGwgO9XWXxhjQs8CxgDyrJM4UO7/ttTWwzUcqmrkmodXebc99P5ukmMjWTQ1jb+v3EdtU0ePo7aplaqGFrJTbP2FMSb0LGAMoMkZTtZYf7eWACp8Srl6psuu2lPBhceO5eqFedQ0trKjuBZV5R+r9nHDk04KLrslZYwZCmzqzQCanOGs+N52uAbmHvl6eW1HZb7Veys4IyaC8rpm8tLjGe+m/Thc1UhNYws/eGaDd99sSwlijBkCrIcxgKIjwpmcHs+WwzV+Xy+vc2p/J0RH8PD7u9npruLOS4sjK8npRRRVN7Jqt5OY91eXHMvYpBimZQ58QkNjjOkrCxgDbPrYRLYV+Q8YpXXNjImP4tefPY6P9lXyixedGhq5Y+JJi48iIkwoqm7ko/2VzBqXxBdPmsjym88mOc7KrxpjQs8CxgCbkZXIvvJ66nwGrz3Ka5tJi4/iU3PHs3ByGmv3VSICE9PiCAsTMhOjKaxqZN2+Shbkpvg5uzHGhI4FjAE2Y6xz+8g3aaBHudvDADh3tlP0KC0+ivhoZygpKzmG5z46SE1TKwtyUwepxcYYExgLGANshltAaevhIxMRltd3BIyTJjvlVWe7tTQArjgx1/t4vvUwjDFDTNBmSYlIDvAoMBZoB+5X1bu77HMx8Av39VbgBrdSHyLSBnimCu1T1U8Hq60DKXdMHAnREazaU8GHO8u46YKZ3llO1Q0tJMc64xGzxyVx2yXHcd4xHeVVP5+fQ0ZiNMt3lTEpPT4k7TfGmO4Ec1ptK3Cjqq4VkURgjYgsVdXNPvu8Abygqioic4CngJnuaw2qOi+I7QuKsDBhzoRknl5zAIDS2mb+9rWTAKhqaCHJDRgiwpUn5R5x/JkzMjlzRubgNdgYYwIUtFtSqlqoqmvdxzVAAZDdZZ9aVVX3aTygjADzcjpuJ3lmTDW2tNHU2u7tYRhjzHAzKGMYIpIHzAdW+HntEhHZAvwHuNbnpRgRWS0iy0XkMz2ce4m73+qSkpIBbnn/TMtK8D4urmni4fd3U+mu8rY05caY4Sron14ikgA8gzM+ccRIsKo+BzwnIqfhjGec476Uq6qHRGQy8KaIbFDVnX6Ovx+4HyA/P39I9FDGJXdemf3zf2/2pi5Psh6GMWaYCmoPQ0QicYLF46r6bE/7quo7wBQRSXefH3J/7wLewumhDAvjfQLGS99dTHZKLI8t3wtYwDDGDF9BCxjiVPx5EChQ1Tu72Wequx8isgCIAspEJFVEot3t6cAiYLO/cwxFWcnR3sezxyfxpZMnep/bGIYxZrgK5i2pRcBVwAYRWeduuxnIBVDV+4BLgS+LSAvQAFzuzpiaBfxZRNpxgtpvusyuGtKiI8I7PZ+b07HWIinGAoYxZngKWsBw11P0WFdUVW8Hbvez/QPguCA1bVCEibMmA2DOhI5ZU0mxNuhtjBme7NMrSDbfegGeMtwJ0RE8+61TeG1TERkJ0T0faIwxQ5QFjCCJiex8W2pBbqrlhzLGDGuWS8oYY0xALGAYY4wJiAUMY4wxAbGAYYwxJiAWMIwxxgTEAoYxxpiAWMAwxhgTEAsYxhhjAiId9YuGPxEpAfb249B0oHSAmzMcjNbrhtF77Xbdo0sg1z1RVTMCOdmIChj9JSKrVTU/1O0YbKP1umH0Xrtd9+gy0Ndtt6SMMcYExAKGMcaYgFjAcNwf6gaEyGi9bhi9127XPboM6HXbGIYxxpiAWA/DGGNMQEZ9wBCRC0Rkq4jsEJEfhro9A0lEHhKRYhHZ6LNtjIgsFZHt7u9Ud7uIyB/cf4f1bo31YUlEckRkmYgUiMgmEbne3T6ir11EYkRkpYh87F73z93tk0RkhXvd/xCRKHd7tPt8h/t6Xijbf7REJFxEPhKRF93no+W694jIBhFZJyKr3W1B+Vsf1QFDRMKBe4ELgdnAFSIyO7StGlCPABd02fZD4A1VnQa84T4H599gmvuzBPjTILUxGFqBG1V1FnAycJ3733WkX3sTcJaqzgXmAReIyMk4ZZB/7153BfBVd/+vAhWqOhX4PX7KJQ8z1wMFPs9Hy3UDnKmq83ym0Abnb11VR+0PsBB41ef5j4AfhbpdA3yNecBGn+dbgXHu43HAVvfxn4Er/O033H+AfwHnjqZrB+KAtcBJOAu3Itzt3r954FVgofs4wt1PQt32fl7vBPeD8SzgRUBGw3W717AHSO+yLSh/66O6hwFkA/t9nh9wt41kWapaCOD+znS3j8h/C/d2w3xgBaPg2t3bMuuAYmApsBOoVNVWdxffa/Net/t6FZA2uC0eMHcB3wfa3edpjI7rBlDgNRFZIyJL3G1B+Vsf7TW9xc+20TptbMT9W4hIAvAMcIOqVov4u0RnVz/bhuW1q2obME9EUoDngFn+dnN/j4jrFpGLgGJVXSMiZ3g2+9l1RF23j0WqekhEMoGlIrKlh32P6tpHew/jAJDj83wCcChEbRksRSIyDsD9XexuH1H/FiISiRMsHlfVZ93No+LaAVS1EngLZwwnRUQ8Xw59r8173e7ryUD54LZ0QCwCPi0ie4AncW5L3cXIv24AVPWQ+7sY50vCiQTpb320B4xVwDR3NkUU8AXghRC3KdheAK52H1+Nc3/fs/3L7iyKk4EqT5d2uBGnK/EgUKCqd/q8NKKvXUQy3J4FIhILnIMzCLwMuMzdret1e/49LgPeVPfG9nCiqj9S1Qmqmofz//CbqvpFRvh1A4hIvIgkeh4D5wEbCdbfeqgHbEL9A3wC2IZzr/fHoW7PAF/b34FCoAXnm8VXce7VvgFsd3+PcfcVnBljO4ENQH6o238U130qTjd7PbDO/fnESL92YA7wkXvdG4Fb3O2TgZXADuCfQLS7PcZ9vsN9fXKor2EA/g3OAF4cLdftXuPH7s8mz2dYsP7WbaW3McaYgIz2W1LGGGMCZAHDGGNMQCxgGGOMCYgFDGOMMQGxgGGMMSYgFjCM6YWItLmZQEtKMlcAAAHZSURBVD0/A5bVWETyxCebsDFD2WhPDWJMIBpUdV6oG2FMqFkPw5h+cusQ3O7WoFgpIlPd7RNF5A233sAbIpLrbs8SkefcehUfi8gp7qnCReQvbg2L19xV2ojId0Vks3ueJ0N0mcZ4WcAwpnexXW5JXe7zWrWqngjcg5O/CPfxo6o6B3gc+IO7/Q/A2+rUq1iAszIXnNoE96rqMUAlcKm7/YfAfPc83wjWxRkTKFvpbUwvRKRWVRP8bN+DU7Bol5vs8LCqpolIKU6NgRZ3e6GqpotICTBBVZt8zpEHLFWn0A0i8gMgUlV/KSKvALXA88Dzqlob5Es1pkfWwzDm6Gg3j7vbx58mn8dtdIwtfhIn78/xwBqfzKvGhIQFDGOOzuU+vz90H3+AkzUV4IvAe+7jN4BvgrfQUVJ3JxWRMCBHVZfhFAZKAY7o5RgzmOwbizG9i3Wr2Hm8oqqeqbXRIrIC58vXFe627wIPichNQAlwjbv9euB+EfkqTk/imzjZhP0JB/4mIsk4GUZ/r06NC2NCxsYwjOkndwwjX1VLQ90WYwaD3ZIyxhgTEOthGGOMCYj1MIwxxgTEAoYxxpiAWMAwxhgTEAsYxhhjAmIBwxhjTEAsYBhjjAnI/wevx+R4TyQzHwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1e02519af28>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "def smooth_curve(points, factor=0.9):\n",
    "  smoothed_points = []\n",
    "  for point in points:\n",
    "    if smoothed_points:\n",
    "      previous = smoothed_points[-1]\n",
    "      smoothed_points.append(previous * factor + point * (1 - factor))\n",
    "    else:\n",
    "      smoothed_points.append(point)\n",
    "  return smoothed_points\n",
    "\n",
    "smooth_mae_history = smooth_curve(average_mae_history[10:])\n",
    "\n",
    "plt.plot(range(1, len(smooth_mae_history) + 1), smooth_mae_history)\n",
    "plt.xlabel('Epochs')\n",
    "plt.ylabel('Validation MAE')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "According to this plot, it seems that validation MAE stops improving significantly after 80 epochs. Past that point, we start overfitting.\n",
    "\n",
    "Once we are done tuning other parameters of our model (besides the number of epochs, we could also adjust the size of the hidden layers), we \n",
    "can train a final \"production\" model on all of the training data, with the best parameters, then look at its performance on the test data:\n",
    "\n",
    "从图可以看出，验证 MAE 在 80 轮后不再显著降低，之后就开始过拟合。\n",
    "\n",
    "完成模型调参之后（除了轮数，还可以调节隐藏层大小），你可以使用最佳参数在所有训练数据上训练最终的生产模型，然后观察模型在测试集上的性能。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "102/102 [==============================] - 0s 332us/step\n"
     ]
    }
   ],
   "source": [
    "# Get a fresh, compiled model.\n",
    "model = build_model()\n",
    "# Train it on the entirety of the data.\n",
    "model.fit(train_data, train_targets,\n",
    "          epochs=80, batch_size=16, verbose=0)\n",
    "test_mse_score, test_mae_score = model.evaluate(test_data, test_targets)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2.686123801212685"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test_mae_score"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We are still off by about \\$2,550.\n",
    "\n",
    "你预测的房价还是和实际价格相差约 2550 美元。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Wrapping up\n",
    "\n",
    "\n",
    "Here's what you should take away from this example:\n",
    "\n",
    "* Regression is done using different loss functions from classification; Mean Squared Error (MSE) is a commonly used loss function for \n",
    "regression.\n",
    "* Similarly, evaluation metrics to be used for regression differ from those used for classification; naturally the concept of \"accuracy\" \n",
    "does not apply for regression. A common regression metric is Mean Absolute Error (MAE).\n",
    "* When features in the input data have values in different ranges, each feature should be scaled independently as a preprocessing step.\n",
    "* When there is little data available, using K-Fold validation is a great way to reliably evaluate a model.\n",
    "* When little training data is available, it is preferable to use a small network with very few hidden layers (typically only one or two), \n",
    "in order to avoid severe overfitting.\n",
    "\n",
    "This example concludes our series of three introductory practical examples. You are now able to handle common types of problems with vector data input:\n",
    "\n",
    "* Binary (2-class) classification.\n",
    "* Multi-class, single-label classification.\n",
    "* Scalar regression.\n",
    "\n",
    "In the next chapter, you will acquire a more formal understanding of some of the concepts you have encountered in these first examples, \n",
    "such as data preprocessing, model evaluation, and overfitting.\n",
    "\n",
    "## 小结\n",
    "下面是你应该从这个例子中学到的要点。\n",
    "* 回归问题使用的损失函数与分类问题不同。回归常用的损失函数是均方误差（MSE）。\n",
    "* 同样，回归问题使用的评估指标也与分类问题不同。显而易见，精度的概念不适用于回归问题。常见的回归指标是平均绝对误差（MAE）。\n",
    "* 如果输入数据的特征具有不同的取值范围，应该先进行预处理，对每个特征单独进行缩放。\n",
    "* 如果可用的数据很少，使用 K 折验证可以可靠地评估模型。\n",
    "* 如果可用的训练数据很少，最好使用隐藏层较少（通常只有一到两个）的小型网络，以避免严重的过拟合。\n",
    "\n",
    "在下一章中，您将获得对这些第一个示例中遇到的一些概念的更直观的理解，例如数据预处理，模型评估和过度拟合。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}