visual-image-search-gluoncv.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Visual image search\n",
    "_**Using a Convolutional Neural Net and Elasticsearch k-Nearest Neighbors Index to retrieve visually similar images**_\n",
    "\n",
    "---\n",
    "\n",
    "---\n",
    "\n",
    "## Contents\n",
    "\n",
    "\n",
    "1. [Background](#Background)\n",
    "1. [Setup](#Setup)\n",
    "1. [GluonCV(MXNet) Model Preparation](#GluonCV(MXNet)-Model-Preparation)\n",
    "1. [SageMaker Model Hosting](#Hosting-Model)\n",
    "1. [SageMaker Model Hosting (BYOC)](#Hosting-Model-BYOC)\n",
    "1. [Build a KNN Index in Elasticsearch](#ES-KNN)\n",
    "1. [Evaluate Index Search Results](#Searching-with-ES-k-NN)\n",
    "1. [Extensions](#Extensions)\n",
    "\n",
    "## Background\n",
    "\n",
    "In this notebook, we'll build the core components of a visual image search application. Visual image search is used in interfaces where instead of asking for something by voice or text, you show a photographic example of what you are looking for.\n",
    "\n",
    "One of the core components of visual image search is a convolutional neural net (CNN) model that generates “feature vectors” representing both a query image and the reference item images to be compared against the query. The reference item feature vectors typically are generated offline and must be stored in a database of some sort, so they can be efficiently searched. For small reference item datasets, it is possible to use a brute force search that compares the query against every reference item. However, this is not feasible for large data sets where brute force search would become prohibitively slow. \n",
    "\n",
    "To enable efficient searches for visually similar images, we'll use Amazon SageMaker to generate “feature vectors” from images and use KNN algorithm in Amazon Elasticsearch Service. KNN for Amazon Elasticsearch Service lets you search for points in a vector space and find the \"nearest neighbors\" for those points by Euclidean distance or cosine similarity(default is Euclidean distance). Use cases include recommendations (for example, an \"other songs you might like\" feature in a music application), image recognition, and fraud detection.\n",
    "\n",
    "Here are the steps we'll follow to build the visual image search: After some initial setup, we'll prepare a model using TensorFlow for generating feature vectors, then generate feature vectors of Fashion Images from *__feidegger__*, a *__zalandoresearch__* dataset. Those feature vectors will be imported in Amazon Elasticsearch KNN Index. Next, we'll explore some test image queries, and visualize the results.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Install tqdm to have progress bar\n",
    "!pip install tqdm # -i https://opentuna.cn/pypi/web/simple\n",
    "\n",
    "#install necessary pkg to make connection with elasticsearch domain\n",
    "!pip install elasticsearch==7.13.0 # -i https://opentuna.cn/pypi/web/simple\n",
    "!pip install requests # -i https://opentuna.cn/pypi/web/simple\n",
    "!pip install requests-aws4auth # -i https://opentuna.cn/pypi/web/simple\n",
    "\n",
    "!pip install gluoncv # -i https://opentuna.cn/pypi/web/simple"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {},
   "outputs": [],
   "source": [
    "import boto3\n",
    "import re\n",
    "import sagemaker\n",
    "from sagemaker import get_execution_role\n",
    "\n",
    "role = get_execution_role()\n",
    "\n",
    "s3_resource = boto3.resource(\"s3\")\n",
    "s3 = boto3.client('s3')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'esDomainName': 'vis-sea-domain-1pq3yx0uhzggz',\n",
       " 'S3BucketSecureURL': 'https://vis-search-s3buckethosting-woeidphm71gm.s3.amazonaws.com',\n",
       " 'esHostName': 'search-vis-sea-domain-1pq3yx0uhzggz-li37dpfinuuwe6jfp4mmd3l62q.us-east-1.es.amazonaws.com',\n",
       " 'SageMakerNotebookURL': 'https://console.aws.amazon.com/sagemaker/home?region=us-east-1#/notebook-instances/openNotebook/NotebookInstance-SweLJWFOINHp?view=classic',\n",
       " 's3BucketTraining': 'vis-search-s3buckettraining-1v5sjtzh20cxd',\n",
       " 's3BucketHostingBucketName': 'vis-search-s3buckethosting-woeidphm71gm'}"
      ]
     },
     "execution_count": 64,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cfn = boto3.client('cloudformation')\n",
    "\n",
    "def get_cfn_outputs(stackname):\n",
    "    outputs = {}\n",
    "    for output in cfn.describe_stacks(StackName=stackname)['Stacks'][0]['Outputs']:\n",
    "        outputs[output['OutputKey']] = output['OutputValue']\n",
    "    return outputs\n",
    "\n",
    "## Setup variables to use for the rest of the demo\n",
    "cloudformation_stack_name = \"vis-search\"\n",
    "\n",
    "outputs = get_cfn_outputs(cloudformation_stack_name)\n",
    "\n",
    "bucket = outputs['s3BucketTraining']\n",
    "es_host = outputs['esHostName']\n",
    "\n",
    "outputs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!wget -O images.zip \"https://datalab.s3.amazonaws.com/data/images.zip?AWSAccessKeyId=AKIAYNUCDPLSDWHHQJ7Y&Signature=qT76xYpYiFP%2B6CkX9ZB%2FgwSf4EE%3D&Expires=1648453451\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!unzip -q images.zip"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "!aws s3 cp --recursive images s3://$bucket/images"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## GluonCV(MXNet) Model Preparation\n",
    "\n",
    "We'll use MXNet backend to prepare a model for \"featurizing\" images into feature vectors. MXNet has a native Module API, as well as a higher level Gluon API. \n",
    "\n",
    "We will start with a pretrained model, avoiding spending time and money training a model from scratch. Accordingly, as a first step in preparing the model, we'll import a pretrained model from GluonCV application. Researchers have experimented with various pretrained CNN architectures with different numbers of layers, discovering that there are several good possibilities.\n",
    "\n",
    "In this notebook, we'll select a model based on the ResNet architecture, a commonly used choice. Of the various choices for number of layers, ranging from 18 to 152, we'll use 50 layers. This also is a common choice that balances the expressiveness of the resulting feature vectors (embeddings) against computational efficiency (lower number of layers means greater efficiency at the cost of less expressiveness)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!wget -O endpoint/model.zip \"https://datalab.s3.amazonaws.com/data/model.zip?AWSAccessKeyId=AKIAYNUCDPLSDWHHQJ7Y&Signature=n1mr3WUiLSXIrnK64%2FnvFhf8sC8%3D&Expires=1648453473\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!cd endpoint && unzip -q model.zip"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {},
   "outputs": [],
   "source": [
    "import mxnet as mx\n",
    "from mxnet.gluon import nn\n",
    "from mxnet import gluon, image, init, nd\n",
    "from gluoncv.model_zoo import get_model\n",
    "from gluoncv.data.transforms.presets.imagenet import transform_eval"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {},
   "outputs": [],
   "source": [
    "num_gpus = 0\n",
    "ctx = [mx.gpu(i) for i in range(num_gpus)] if num_gpus > 0 else [mx.cpu()]\n",
    "\n",
    "model_name = 'ResNet50_v2'\n",
    "classes = 5\n",
    "\n",
    "finetune_net = get_model(model_name, pretrained=True)\n",
    "with finetune_net.name_scope():\n",
    "    finetune_net.output = nn.Dense(classes)\n",
    "finetune_net.output.initialize(init.Xavier(), ctx = ctx)\n",
    "finetune_net.collect_params().reset_ctx(ctx)\n",
    "finetune_net.hybridize()\n",
    "\n",
    "finetune_net.load_parameters('endpoint/model/model-0000.params')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[2021-09-16 13:26:49.465 ip-172-16-191-1:1219 INFO utils.py:27] RULE_JOB_STOP_SIGNAL_FILENAME: None\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "\n",
       "[[ 0.25584534  2.7084174  -2.5742846   1.2124145  -3.8928082 ]]\n",
       "<NDArray 1x5 @cpu(0)>"
      ]
     },
     "execution_count": 67,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "input_pic = 'endpoint/1.jpg'\n",
    "\n",
    "# Load Images\n",
    "img = image.imread(input_pic)\n",
    "\n",
    "# Transform\n",
    "img = transform_eval(img).copyto(ctx[0])\n",
    "    \n",
    "finetune_net(img)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we'll get a reference ResNet50 model which is trained on Imagenet dataset to extract the feature without the actual clssifier. More specifically, we'll use that layer to generate a row vector of floating point numbers as an \"embedding\" or representation of the features of the image. We'll also save the model as *SavedModel* format under **export/Servo/1** to serve from SageMaker MXNet serving API."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# !mkdir -p endpoint/model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# finetune_net.save_parameters('endpoint/model/model-0000.params')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## SageMaker Model Hosting\n",
    "\n",
    "After saving the feature extractor model we will deploy the model using Sagemaker Tensorflow Serving api which is a flexible, high-performance serving system for machine learning models, designed for production environments.TensorFlow Serving makes it easy to deploy new algorithms and experiments, while keeping the same server architecture and APIs. TensorFlow Serving provides out-of-the-box integration with TensorFlow models, but can be easily extended to serve other types of models and data. We will define **inference.py** to customize the input data to TensorFlow serving API. We also need to add **requirements.txt** file for aditional libraby in the tensorflow serving container."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# #check the actual content of inference.py\n",
    "# !pygmentize endpoint/inference.py"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# import tarfile\n",
    "\n",
    "# #zip the model .gz format\n",
    "# export_dir = ''\n",
    "# with tarfile.open('model.tar.gz', mode='w:gz') as archive:\n",
    "#     archive.add('export', recursive=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# #Upload the model to S3\n",
    "# sagemaker_session = sagemaker.Session()\n",
    "# inputs = sagemaker_session.upload_data(path='model.tar.gz', key_prefix='model')\n",
    "# inputs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "After we upload the model to S3 we will use TensorFlow serving container to host the model. We are using ml.p3.16xlarge instance type. You may need to raise support ticket to increase the Service quotas for SageMaker hosting instance type. We will use this endpoint to generate features and import into ElasticSearch. you can also choose small instance such as \"ml.m4.xlarge\" to save cost."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# #Deploy the model in Sagemaker Endpoint. This process will take ~10 min.\n",
    "# from sagemaker.tensorflow.serving import Model\n",
    "\n",
    "# sagemaker_model = Model(entry_point='inference.py', model_data = 's3://' + sagemaker_session.default_bucket() + '/model/model.tar.gz',\n",
    "#                                   role = role, framework_version='2.1.0', source_dir='./src' )\n",
    "\n",
    "# predictor = sagemaker_model.deploy(initial_instance_count=3, instance_type='ml.m5.xlarge')\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# # get the features for a sample image\n",
    "# payload = s3.get_object(Bucket=bucket,Key='image-embedding/1.jpg')['Body'].read()\n",
    "# predictor.content_type = 'application/x-image'\n",
    "# predictor.serializer   = None\n",
    "# features = predictor.predict(payload)['predictions'][0]\n",
    "\n",
    "# features\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## SageMaker Model Hosting (BYOC)\n",
    "```\n",
    "cd endpoint\n",
    "./build_and_push.sh image-embedding\n",
    "python create_endpoint.py\n",
    "docker run -v -d -p 8080:8080 image-embedding\n",
    "python test.py\n",
    "python test-x-image.py\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Build a KNN Index in Elasticsearch\n",
    "\n",
    "KNN for Amazon Elasticsearch Service lets you search for points in a vector space and find the \"nearest neighbors\" for those points by Euclidean distance or cosine similarity (default is Euclidean distance). Use cases include recommendations (for example, an \"other songs you might like\" feature in a music application), image recognition, and fraud detection.\n",
    "\n",
    "KNN requires Elasticsearch 7.1 or later. Full documentation for the Elasticsearch feature, including descriptions of settings and statistics, is available in the Open Distro for Elasticsearch documentation. For background information about the k-nearest neighbors algorithm\n",
    "\n",
    "In this step we'll get all the features zalando images and import those features into Elastichseach7.4 domain."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Define some utility function\n",
    "\n",
    "#return all s3 keys\n",
    "def get_all_s3_keys(bucket):\n",
    "    \"\"\"Get a list of all keys in an S3 bucket.\"\"\"    \n",
    "    keys = []\n",
    "\n",
    "    kwargs = {'Bucket': bucket}\n",
    "    while True:\n",
    "        resp = s3.list_objects_v2(**kwargs)\n",
    "        for obj in resp['Contents']:\n",
    "            if obj['Key'].endswith('.jpg'):\n",
    "                keys.append('s3://' + bucket + '/' + obj['Key'])\n",
    "\n",
    "        try:\n",
    "            kwargs['ContinuationToken'] = resp['NextContinuationToken']\n",
    "        except KeyError:\n",
    "            break\n",
    "\n",
    "    return keys"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# get all the zalando images keys from the bucket make a list\n",
    "s3_uris = get_all_s3_keys(bucket)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "len(s3_uris)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "s3_uris[:10]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# define a function to extract image features\n",
    "from time import sleep\n",
    "import json\n",
    "\n",
    "sm_client = boto3.client('sagemaker-runtime')\n",
    "ENDPOINT_NAME = 'image-embedding'  # predictor.endpoint\n",
    "\n",
    "def get_predictions(payload):\n",
    "    return sm_client.invoke_endpoint(EndpointName=ENDPOINT_NAME,\n",
    "                                           ContentType='application/json',  # 'application/x-image'\n",
    "                                           Body=payload)\n",
    "\n",
    "def extract_features(s3_uri):\n",
    "    key = s3_uri.replace(f's3://{bucket}/', '')\n",
    "    payload = json.dumps({'bucket' : bucket, 'image_uri' : key, 'content_type': \"application/json\"})  # s3.get_object(Bucket=bucket,Key=key)['Body'].read()\n",
    "    try:\n",
    "        response = get_predictions(payload)\n",
    "    except:\n",
    "        sleep(0.1)\n",
    "        response = get_predictions(payload)\n",
    "\n",
    "    del payload\n",
    "    response_body = json.loads((response['Body'].read()))\n",
    "    feature_lst = response_body  # response_body['predictions'][0]\n",
    "    \n",
    "    return s3_uri, feature_lst\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "resulti = extract_features(s3_uris[0])\n",
    "print(resulti)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%time\n",
    "# This process cell will take approximately 24-25 minutes on a t3.medium notebook instance\n",
    "# with 3 m5.xlarge SageMaker Hosted Endpoint instances\n",
    "from multiprocessing import cpu_count\n",
    "from tqdm.contrib.concurrent import process_map\n",
    "\n",
    "workers = 1  # TODO if your endpoint instance is large enough: 2 * cpu_count()\n",
    "result = process_map(extract_features, s3_uris, max_workers=workers)  # TODO use 100 images for sample, s3_uris[:100]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# setting up the Elasticsearch connection\n",
    "from elasticsearch import Elasticsearch, RequestsHttpConnection\n",
    "from requests_aws4auth import AWS4Auth\n",
    "region = 'us-east-1' # e.g. us-east-1/cn-northwest-1\n",
    "service = 'es'\n",
    "credentials = boto3.Session().get_credentials()\n",
    "awsauth = AWS4Auth(credentials.access_key, credentials.secret_key, region, service, session_token=credentials.token)\n",
    "\n",
    "es = Elasticsearch(\n",
    "    hosts = [{'host': es_host, 'port': 443}],\n",
    "    http_auth = awsauth,\n",
    "    use_ssl = True,\n",
    "    verify_certs = True,\n",
    "    connection_class = RequestsHttpConnection\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Define KNN Elasticsearch index maping\n",
    "knn_index = {\n",
    "    \"settings\": {\n",
    "        \"index.knn\": True\n",
    "    },\n",
    "    \"mappings\": {\n",
    "        \"properties\": {\n",
    "            \"zalando_img_vector\": {\n",
    "                \"type\": \"knn_vector\",\n",
    "                \"dimension\": 2048\n",
    "            }\n",
    "        }\n",
    "    }\n",
    "}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Creating the Elasticsearch index\n",
    "es.indices.create(index=\"idx_zalando\",body=knn_index,ignore=400)\n",
    "es.indices.get(index=\"idx_zalando\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "%%time\n",
    "# defining a function to import the feature vectors corrosponds to each S3 URI into Elasticsearch KNN index\n",
    "# This process will take around ~3 min.\n",
    "\n",
    "\n",
    "def es_import(i):\n",
    "    es.index(index='idx_zalando',\n",
    "             body={\"zalando_img_vector\": i[1]['predictions'][0], \n",
    "                   \"image\": i[0]}\n",
    "            )\n",
    "    \n",
    "_ = process_map(es_import, result, max_workers=workers)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Evaluate Index Search Results\n",
    "\n",
    "In this step we will use SageMaker SDK as well as Boto3 SDK to query the Elasticsearch to retrive the nearest neighbours. One thing to mention **zalando** dataset has pretty good similarity with Imagenet dataset. Now if you hav a very domain speific problem then then you need to train that dataset on top of pretrained feature extractor model such as VGG, Resnet, Xeception, Mobilenet etc and bulid a new feature extractor model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {},
   "outputs": [],
   "source": [
    "#define display_image function\n",
    "def display_image(bucket, key):\n",
    "    response = s3.get_object(Bucket=bucket,Key=key)['Body']\n",
    "    img = Image.open(response)\n",
    "    return display(img)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {},
   "outputs": [],
   "source": [
    "import requests\n",
    "import random\n",
    "from PIL import Image\n",
    "import io\n",
    "# urls = []\n",
    "# # yellow pattern dess\n",
    "# urls.append('https://fastly.hautelookcdn.com/products/D7242MNR/large/13494318.jpg')\n",
    "# # T shirt kind dress\n",
    "# urls.append('https://fastly.hautelookcdn.com/products/M2241/large/15658772.jpg')\n",
    "# #Dotted pattern dress\n",
    "# urls.append('https://fastly.hautelookcdn.com/products/19463M/large/14537545.jpg')\n",
    "\n",
    "# img_bytes = requests.get(random.choice(urls)).content\n",
    "# query_img = Image.open(io.BytesIO(img_bytes))\n",
    "# query_img"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "###### SageMaker SDK Method"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {},
   "outputs": [],
   "source": [
    "#SageMaker SDK approach\n",
    "# predictor.content_type = 'application/x-image'\n",
    "# predictor.serializer   = None\n",
    "# features = predictor.predict(img_bytes)['predictions'][0]\n",
    "\n",
    "# Boto3 approach\n",
    "tmp_bucket = bucket\n",
    "s3_uri = f's3://{bucket}/images/1,12,0,4,22432,3005,2000,8a28288b.jpg'\n",
    "key = s3_uri.replace(f's3://{tmp_bucket}/', '')\n",
    "payload = json.dumps({'bucket' : tmp_bucket, 'image_uri' : key, 'content_type': \"application/json\"})  # s3.get_object(Bucket=bucket,Key=key)['Body'].read()\n",
    "response = get_predictions(payload)\n",
    "response_body = json.loads((response['Body'].read()))\n",
    "features = response_body['predictions'][0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[0.533256471157074,\n",
       " 0.45109084248542786,\n",
       " 0.8576896786689758,\n",
       " 0.6522411704063416,\n",
       " 0.3429913818836212,\n",
       " 0.14514729380607605,\n",
       " 0.15180404484272003,\n",
       " 0.4039151072502136,\n",
       " 0.00027326808776706457,\n",
       " 0.13114456832408905,\n",
       " 0.042439576238393784,\n",
       " 0.003713396145030856,\n",
       " 0.00026224847533740103,\n",
       " 0.5788456797599792,\n",
       " 0.15356361865997314,\n",
       " 0.27260032296180725,\n",
       " 0.5810298323631287,\n",
       " 0.0,\n",
       " 0.7041717171669006,\n",
       " 0.42837369441986084,\n",
       " 1.1482781171798706,\n",
       " 0.12104740738868713,\n",
       " 0.4381811320781708,\n",
       " 0.5327453017234802,\n",
       " 0.4736502766609192,\n",
       " 0.3600580394268036,\n",
       " 0.0966571569442749,\n",
       " 0.22678743302822113,\n",
       " 0.061171408742666245,\n",
       " 0.6312631368637085,\n",
       " 2.5476937294006348,\n",
       " 0.8289904594421387,\n",
       " 0.7676099538803101,\n",
       " 0.2759179174900055,\n",
       " 0.04089886695146561,\n",
       " 0.17884649336338043,\n",
       " 1.3540477752685547,\n",
       " 1.293715000152588,\n",
       " 0.0694701224565506,\n",
       " 0.8315494060516357,\n",
       " 0.3874511420726776,\n",
       " 0.7624723315238953,\n",
       " 0.5087661147117615,\n",
       " 0.2931152880191803,\n",
       " 0.08615832775831223,\n",
       " 0.0003321461845189333,\n",
       " 0.0051552914083004,\n",
       " 1.2343103885650635,\n",
       " 0.48671695590019226,\n",
       " 0.0,\n",
       " 0.00034374321694485843,\n",
       " 0.7495232224464417,\n",
       " 1.1178133487701416,\n",
       " 0.19857566058635712,\n",
       " 0.5827820301055908,\n",
       " 1.325059413909912,\n",
       " 0.17168974876403809,\n",
       " 0.09844472259283066,\n",
       " 0.32864803075790405,\n",
       " 0.20508119463920593,\n",
       " 0.00412343256175518,\n",
       " 1.1531457901000977,\n",
       " 0.6753336787223816,\n",
       " 0.22080697119235992,\n",
       " 0.3201945424079895,\n",
       " 0.0012288554571568966,\n",
       " 0.0025910716503858566,\n",
       " 6.786407175241038e-05,\n",
       " 0.7835779786109924,\n",
       " 0.1258775144815445,\n",
       " 1.2374950647354126,\n",
       " 1.4607391357421875,\n",
       " 0.46579641103744507,\n",
       " 0.13544072210788727,\n",
       " 1.287553071975708,\n",
       " 0.0,\n",
       " 0.0014897012151777744,\n",
       " 0.9748057723045349,\n",
       " 0.0012154228752478957,\n",
       " 1.2281607389450073,\n",
       " 0.3627760112285614,\n",
       " 1.0561126470565796,\n",
       " 0.44295334815979004,\n",
       " 0.00022728403564542532,\n",
       " 0.6819890737533569,\n",
       " 0.29162728786468506,\n",
       " 0.046073153614997864,\n",
       " 0.3089126944541931,\n",
       " 0.14493058621883392,\n",
       " 1.205019474029541,\n",
       " 1.1063445806503296,\n",
       " 0.7461479902267456,\n",
       " 0.36688435077667236,\n",
       " 0.08201544731855392,\n",
       " 1.3290047645568848,\n",
       " 0.037890274077653885,\n",
       " 0.6790285706520081,\n",
       " 0.20376695692539215,\n",
       " 0.42662009596824646,\n",
       " 1.8448123931884766,\n",
       " 0.2463947981595993,\n",
       " 0.8142717480659485,\n",
       " 0.7949320673942566,\n",
       " 1.0736004114151,\n",
       " 0.0,\n",
       " 0.3425960838794708,\n",
       " 0.0376487672328949,\n",
       " 0.6417722702026367,\n",
       " 0.7135933637619019,\n",
       " 0.005796218290925026,\n",
       " 0.1277267038822174,\n",
       " 0.5186132192611694,\n",
       " 0.4711111783981323,\n",
       " 0.0013395800488069654,\n",
       " 0.0043968213722109795,\n",
       " 0.8491925597190857,\n",
       " 0.0010722343577072024,\n",
       " 0.00026776030426844954,\n",
       " 0.8069233894348145,\n",
       " 0.0992027074098587,\n",
       " 1.370776891708374,\n",
       " 0.6439936757087708,\n",
       " 1.2255728244781494,\n",
       " 0.6774559020996094,\n",
       " 0.8043306469917297,\n",
       " 1.1727279424667358,\n",
       " 0.0,\n",
       " 0.21128341555595398,\n",
       " 0.051566001027822495,\n",
       " 0.0022849692031741142,\n",
       " 0.5072377324104309,\n",
       " 0.16607294976711273,\n",
       " 0.39094972610473633,\n",
       " 0.770722508430481,\n",
       " 0.02837429940700531,\n",
       " 0.8697959184646606,\n",
       " 0.7556737661361694,\n",
       " 1.1874120235443115,\n",
       " 0.399985134601593,\n",
       " 0.3927336633205414,\n",
       " 0.5836097598075867,\n",
       " 0.588153064250946,\n",
       " 0.5505080819129944,\n",
       " 0.05202125012874603,\n",
       " 0.4676945209503174,\n",
       " 0.10210739076137543,\n",
       " 0.5659317970275879,\n",
       " 0.13835681974887848,\n",
       " 0.4231824278831482,\n",
       " 0.012052008882164955,\n",
       " 0.14349964261054993,\n",
       " 0.11269563436508179,\n",
       " 0.5588691234588623,\n",
       " 0.3335362374782562,\n",
       " 0.9624994397163391,\n",
       " 0.36945047974586487,\n",
       " 0.47815069556236267,\n",
       " 0.38557177782058716,\n",
       " 0.0031347565818578005,\n",
       " 0.15867049992084503,\n",
       " 0.4194525480270386,\n",
       " 0.005963793955743313,\n",
       " 0.6140438914299011,\n",
       " 1.787595510482788,\n",
       " 0.5481551289558411,\n",
       " 0.655231237411499,\n",
       " 1.1297907829284668,\n",
       " 0.07377888262271881,\n",
       " 0.048079170286655426,\n",
       " 0.4416748285293579,\n",
       " 0.02065194584429264,\n",
       " 0.3670605719089508,\n",
       " 0.1044405922293663,\n",
       " 1.035358190536499,\n",
       " 0.10085231065750122,\n",
       " 0.5042384266853333,\n",
       " 0.3595419228076935,\n",
       " 0.04934479668736458,\n",
       " 2.4788870811462402,\n",
       " 0.6231176257133484,\n",
       " 0.0002712191781029105,\n",
       " 0.2834644913673401,\n",
       " 0.3889419436454773,\n",
       " 0.7949384450912476,\n",
       " 0.32726746797561646,\n",
       " 0.3842029273509979,\n",
       " 0.8511233925819397,\n",
       " 0.01571783423423767,\n",
       " 0.32976770401000977,\n",
       " 0.00561324879527092,\n",
       " 1.3770548105239868,\n",
       " 0.4589087963104248,\n",
       " 0.16468964517116547,\n",
       " 0.06687336415052414,\n",
       " 0.6351706981658936,\n",
       " 0.8375117778778076,\n",
       " 0.6958543658256531,\n",
       " 0.1161256805062294,\n",
       " 0.0358969122171402,\n",
       " 0.15837228298187256,\n",
       " 0.3703593313694,\n",
       " 0.2663535475730896,\n",
       " 0.022125333547592163,\n",
       " 0.3110290467739105,\n",
       " 0.30527690052986145,\n",
       " 0.9884710311889648,\n",
       " 0.002660514786839485,\n",
       " 0.6452932953834534,\n",
       " 0.5015872716903687,\n",
       " 0.2171403467655182,\n",
       " 0.2602697014808655,\n",
       " 0.9202189445495605,\n",
       " 0.08293154835700989,\n",
       " 0.5305405855178833,\n",
       " 0.2049478441476822,\n",
       " 0.00011673056724248454,\n",
       " 0.39091992378234863,\n",
       " 0.6143710613250732,\n",
       " 0.9123033881187439,\n",
       " 0.7106661200523376,\n",
       " 0.5734330415725708,\n",
       " 0.36592942476272583,\n",
       " 0.0,\n",
       " 0.004487941041588783,\n",
       " 0.32642635703086853,\n",
       " 4.909192696800346e-08,\n",
       " 0.7064369320869446,\n",
       " 0.3386009633541107,\n",
       " 1.2526443004608154,\n",
       " 1.7008667782647535e-05,\n",
       " 0.2507213354110718,\n",
       " 0.0,\n",
       " 0.3384822607040405,\n",
       " 0.08979464322328568,\n",
       " 0.9241864681243896,\n",
       " 0.11021169275045395,\n",
       " 0.5714813470840454,\n",
       " 0.4745878577232361,\n",
       " 0.1637326329946518,\n",
       " 0.040454085916280746,\n",
       " 0.0019112814916297793,\n",
       " 0.00020706413488369435,\n",
       " 0.0006720978417433798,\n",
       " 0.12146603316068649,\n",
       " 0.018490733578801155,\n",
       " 1.1994212865829468,\n",
       " 0.7741742134094238,\n",
       " 0.22175592184066772,\n",
       " 1.2342965602874756,\n",
       " 0.5744172930717468,\n",
       " 0.7682962417602539,\n",
       " 0.2098255604505539,\n",
       " 0.8789461255073547,\n",
       " 0.5992861390113831,\n",
       " 0.016421247273683548,\n",
       " 0.5930314064025879,\n",
       " 0.21396072208881378,\n",
       " 0.7798491716384888,\n",
       " 0.4468354880809784,\n",
       " 1.4635865688323975,\n",
       " 0.0016668887110427022,\n",
       " 0.0006413470255210996,\n",
       " 0.3601899743080139,\n",
       " 1.0145243406295776,\n",
       " 0.33936208486557007,\n",
       " 0.17295005917549133,\n",
       " 0.7131184935569763,\n",
       " 0.5844976305961609,\n",
       " 0.5874797105789185,\n",
       " 0.07431647926568985,\n",
       " 0.6362172961235046,\n",
       " 0.4950890839099884,\n",
       " 1.0349007844924927,\n",
       " 0.00013938613119535148,\n",
       " 0.014332207851111889,\n",
       " 0.3440416157245636,\n",
       " 0.823393702507019,\n",
       " 1.0540834665298462,\n",
       " 0.6959354877471924,\n",
       " 0.003011449007317424,\n",
       " 0.2823817729949951,\n",
       " 0.8321273326873779,\n",
       " 0.3216943144798279,\n",
       " 0.0008767914841882885,\n",
       " 0.42103272676467896,\n",
       " 0.42384839057922363,\n",
       " 1.5288972854614258,\n",
       " 0.28960907459259033,\n",
       " 0.9349156618118286,\n",
       " 0.8124665021896362,\n",
       " 1.2560529708862305,\n",
       " 0.7872804999351501,\n",
       " 0.3742885887622833,\n",
       " 0.007639741990715265,\n",
       " 0.1983955204486847,\n",
       " 0.3950903117656708,\n",
       " 0.031626392155885696,\n",
       " 0.4377249777317047,\n",
       " 0.5101194381713867,\n",
       " 0.002527061849832535,\n",
       " 0.013807927258312702,\n",
       " 1.6385908126831055,\n",
       " 0.5318158864974976,\n",
       " 0.008602751418948174,\n",
       " 0.0717882513999939,\n",
       " 0.5559426546096802,\n",
       " 0.6488480567932129,\n",
       " 1.3939625024795532,\n",
       " 0.7504509091377258,\n",
       " 0.06595772504806519,\n",
       " 2.0935821533203125,\n",
       " 0.4111368954181671,\n",
       " 0.6278056502342224,\n",
       " 0.4341662526130676,\n",
       " 0.14340606331825256,\n",
       " 0.42573025822639465,\n",
       " 0.0043129087425768375,\n",
       " 0.8802734613418579,\n",
       " 0.0013541426742449403,\n",
       " 0.022398676723241806,\n",
       " 0.4057680368423462,\n",
       " 0.4793616235256195,\n",
       " 0.9745047688484192,\n",
       " 0.6952344179153442,\n",
       " 0.22163842618465424,\n",
       " 0.0018165832152590156,\n",
       " 0.3075743615627289,\n",
       " 0.5285030007362366,\n",
       " 0.1834735870361328,\n",
       " 0.48909667134284973,\n",
       " 0.8521119952201843,\n",
       " 0.7066921591758728,\n",
       " 0.24927973747253418,\n",
       " 0.6027971506118774,\n",
       " 0.0,\n",
       " 0.9094584584236145,\n",
       " 0.8237636089324951,\n",
       " 0.3835640251636505,\n",
       " 0.7039869427680969,\n",
       " 0.5190272331237793,\n",
       " 0.6297550201416016,\n",
       " 0.2508579194545746,\n",
       " 0.1945253163576126,\n",
       " 0.19323137402534485,\n",
       " 0.12235062569379807,\n",
       " 0.9925358295440674,\n",
       " 0.3208143413066864,\n",
       " 0.6431986689567566,\n",
       " 0.7392249703407288,\n",
       " 0.9499493837356567,\n",
       " 0.9757120013237,\n",
       " 0.28130489587783813,\n",
       " 0.3311665952205658,\n",
       " 0.5985075235366821,\n",
       " 0.4442497491836548,\n",
       " 1.0937334299087524,\n",
       " 0.0,\n",
       " 1.027873158454895,\n",
       " 0.2787414789199829,\n",
       " 0.00967907439917326,\n",
       " 0.6333281993865967,\n",
       " 0.2694951891899109,\n",
       " 0.3063129782676697,\n",
       " 0.17242828011512756,\n",
       " 1.1142505407333374,\n",
       " 0.05557602643966675,\n",
       " 0.13012897968292236,\n",
       " 1.155113935470581,\n",
       " 0.5757520198822021,\n",
       " 0.002325933426618576,\n",
       " 0.5440956950187683,\n",
       " 0.8391426205635071,\n",
       " 1.2641946077346802,\n",
       " 0.341616690158844,\n",
       " 0.047389332205057144,\n",
       " 0.6997438669204712,\n",
       " 0.031061068177223206,\n",
       " 0.9541701674461365,\n",
       " 0.7936429381370544,\n",
       " 1.0985995531082153,\n",
       " 0.8798773884773254,\n",
       " 1.0831516981124878,\n",
       " 0.6033618450164795,\n",
       " 0.011249225586652756,\n",
       " 0.2657080292701721,\n",
       " 0.5604649186134338,\n",
       " 0.0,\n",
       " 0.5400010943412781,\n",
       " 0.9975683093070984,\n",
       " 0.07079015672206879,\n",
       " 0.9062218070030212,\n",
       " 0.30532732605934143,\n",
       " 0.3425360321998596,\n",
       " 2.096527099609375,\n",
       " 0.5183120369911194,\n",
       " 0.24750211834907532,\n",
       " 1.0282946825027466,\n",
       " 1.6962631940841675,\n",
       " 0.38030675053596497,\n",
       " 0.2600509226322174,\n",
       " 0.0,\n",
       " 0.21451282501220703,\n",
       " 0.37248560786247253,\n",
       " 0.04789414629340172,\n",
       " 0.3106825053691864,\n",
       " 0.28620222210884094,\n",
       " 0.5416199564933777,\n",
       " 1.7775696516036987,\n",
       " 0.0,\n",
       " 0.32018187642097473,\n",
       " 0.052655551582574844,\n",
       " 0.000269084150204435,\n",
       " 0.10777375847101212,\n",
       " 0.3643902838230133,\n",
       " 1.0954327583312988,\n",
       " 0.5641278624534607,\n",
       " 0.0,\n",
       " 0.5868003964424133,\n",
       " 0.1348762959241867,\n",
       " 0.00024020831915549934,\n",
       " 0.7941403388977051,\n",
       " 0.335343599319458,\n",
       " 0.01114198099821806,\n",
       " 0.1735372394323349,\n",
       " 0.3427422046661377,\n",
       " 0.12352346628904343,\n",
       " 0.9195038080215454,\n",
       " 0.22820362448692322,\n",
       " 0.7953507304191589,\n",
       " 0.5978438258171082,\n",
       " 0.44093015789985657,\n",
       " 1.3094121217727661,\n",
       " 1.0850249528884888,\n",
       " 0.24999192357063293,\n",
       " 0.5117764472961426,\n",
       " 0.3844331204891205,\n",
       " 0.3998867869377136,\n",
       " 0.42859509587287903,\n",
       " 0.08629171550273895,\n",
       " 0.6079007387161255,\n",
       " 0.37941670417785645,\n",
       " 0.2837775945663452,\n",
       " 0.8428017497062683,\n",
       " 0.2566941976547241,\n",
       " 0.0008801198564469814,\n",
       " 0.3879944682121277,\n",
       " 0.6431930661201477,\n",
       " 0.5825594067573547,\n",
       " 0.047368720173835754,\n",
       " 0.7012120485305786,\n",
       " 1.6509588956832886,\n",
       " 0.46169689297676086,\n",
       " 0.6095099449157715,\n",
       " 1.322440266609192,\n",
       " 0.09729289263486862,\n",
       " 0.0005068388418294489,\n",
       " 0.8011930584907532,\n",
       " 0.8268580436706543,\n",
       " 1.4482795000076294,\n",
       " 0.945307195186615,\n",
       " 0.5055266618728638,\n",
       " 0.056784022599458694,\n",
       " 0.0005294315633364022,\n",
       " 0.6610623598098755,\n",
       " 1.2105064392089844,\n",
       " 0.33021020889282227,\n",
       " 0.15706145763397217,\n",
       " 0.0010166516294702888,\n",
       " 0.42915433645248413,\n",
       " 0.005042003933340311,\n",
       " 0.7510620951652527,\n",
       " 0.36985859274864197,\n",
       " 0.010711387731134892,\n",
       " 0.09019976109266281,\n",
       " 0.5224980115890503,\n",
       " 0.0007449821569025517,\n",
       " 0.3225822448730469,\n",
       " 0.6187273263931274,\n",
       " 1.0262205600738525,\n",
       " 1.1097097396850586,\n",
       " 0.13302963972091675,\n",
       " 0.3535382151603699,\n",
       " 0.7489171028137207,\n",
       " 0.11803651601076126,\n",
       " 0.002958200639113784,\n",
       " 0.2673937678337097,\n",
       " 0.6697341203689575,\n",
       " 0.15812283754348755,\n",
       " 1.1359353065490723,\n",
       " 0.45105448365211487,\n",
       " 1.3309204578399658,\n",
       " 0.7292571067810059,\n",
       " 0.0,\n",
       " 0.726231575012207,\n",
       " 0.10537643730640411,\n",
       " 1.4807088375091553,\n",
       " 1.3088774681091309,\n",
       " 0.5043421983718872,\n",
       " 0.40241602063179016,\n",
       " 0.5507286190986633,\n",
       " 0.35884156823158264,\n",
       " 0.5643624067306519,\n",
       " 0.20938315987586975,\n",
       " 0.24690304696559906,\n",
       " 0.6936085820198059,\n",
       " 0.005061938893049955,\n",
       " 0.3156950771808624,\n",
       " 0.3268910348415375,\n",
       " 0.3320757746696472,\n",
       " 0.37517523765563965,\n",
       " 0.4208179712295532,\n",
       " 0.2188175916671753,\n",
       " 0.0,\n",
       " 0.7065791487693787,\n",
       " 1.36247980594635,\n",
       " 0.6236240267753601,\n",
       " 0.44241902232170105,\n",
       " 1.2912044525146484,\n",
       " 0.8980101943016052,\n",
       " 0.7978052496910095,\n",
       " 0.047107044607400894,\n",
       " 0.48055970668792725,\n",
       " 0.8590283989906311,\n",
       " 0.01640348695218563,\n",
       " 0.220428466796875,\n",
       " 0.6950981616973877,\n",
       " 0.37351280450820923,\n",
       " 0.9866886734962463,\n",
       " 0.2175840437412262,\n",
       " 0.42379096150398254,\n",
       " 0.3116440773010254,\n",
       " 0.1467454731464386,\n",
       " 0.7453293204307556,\n",
       " 1.3046019077301025,\n",
       " 1.762666940689087,\n",
       " 0.41800040006637573,\n",
       " 1.1846466064453125,\n",
       " 0.7039514780044556,\n",
       " 0.019777199253439903,\n",
       " 0.4022800326347351,\n",
       " 0.5227957963943481,\n",
       " 0.19263197481632233,\n",
       " 0.4486430883407593,\n",
       " 0.31777137517929077,\n",
       " 0.11341241747140884,\n",
       " 1.6594773530960083,\n",
       " 0.7871348857879639,\n",
       " 0.3606396019458771,\n",
       " 0.13527880609035492,\n",
       " 0.8415892720222473,\n",
       " 0.4662119448184967,\n",
       " 0.6670605540275574,\n",
       " 0.33348530530929565,\n",
       " 0.38116517663002014,\n",
       " 0.5145919919013977,\n",
       " 1.5063625574111938,\n",
       " 0.07750417292118073,\n",
       " 0.5381072759628296,\n",
       " 1.2514615058898926,\n",
       " 0.5534154772758484,\n",
       " 0.3710813820362091,\n",
       " 0.3190401494503021,\n",
       " 0.6590027809143066,\n",
       " 0.018498459830880165,\n",
       " 0.0,\n",
       " 0.4347689747810364,\n",
       " 0.0,\n",
       " 0.3793843984603882,\n",
       " 0.7899636030197144,\n",
       " 0.25865089893341064,\n",
       " 0.42275699973106384,\n",
       " 0.7033287286758423,\n",
       " 0.21496494114398956,\n",
       " 1.1989811658859253,\n",
       " 0.6963076591491699,\n",
       " 0.8789097666740417,\n",
       " 0.8192143440246582,\n",
       " 0.3919254541397095,\n",
       " 0.24993261694908142,\n",
       " 0.00023031079035717994,\n",
       " 0.027588939294219017,\n",
       " 0.5131893754005432,\n",
       " 0.0004121669626329094,\n",
       " 0.2109658420085907,\n",
       " 0.33412715792655945,\n",
       " 0.6878293752670288,\n",
       " 0.5406124591827393,\n",
       " 0.11408263444900513,\n",
       " 0.00908728688955307,\n",
       " 0.6703639626502991,\n",
       " 0.014150734059512615,\n",
       " 0.04363773390650749,\n",
       " 0.19713333249092102,\n",
       " 0.0002399489312665537,\n",
       " 0.15960116684436798,\n",
       " 0.16956251859664917,\n",
       " 0.4574473202228546,\n",
       " 0.29633623361587524,\n",
       " 0.9255749583244324,\n",
       " 0.47399038076400757,\n",
       " 0.34986013174057007,\n",
       " 1.0494904518127441,\n",
       " 0.5957407355308533,\n",
       " 0.6432690024375916,\n",
       " 0.07513437420129776,\n",
       " 0.2481071501970291,\n",
       " 0.7667674422264099,\n",
       " 0.33291465044021606,\n",
       " 0.465974897146225,\n",
       " 0.24128587543964386,\n",
       " 0.8091239333152771,\n",
       " 0.0,\n",
       " 1.0700459480285645,\n",
       " 0.05498015508055687,\n",
       " 0.654393196105957,\n",
       " 1.215707778930664,\n",
       " 0.17051726579666138,\n",
       " 0.2575823962688446,\n",
       " 0.15195487439632416,\n",
       " 0.0001391823316225782,\n",
       " 0.0,\n",
       " 0.5838724374771118,\n",
       " 0.6348842978477478,\n",
       " 0.24665062129497528,\n",
       " 0.15615113079547882,\n",
       " 0.6368445158004761,\n",
       " 0.2577391564846039,\n",
       " 0.2232334166765213,\n",
       " 0.33250316977500916,\n",
       " 0.555662214756012,\n",
       " 0.0,\n",
       " 1.1722211837768555,\n",
       " 1.1041816473007202,\n",
       " 0.11621548980474472,\n",
       " 0.7033518552780151,\n",
       " 0.4587061405181885,\n",
       " 0.7509430050849915,\n",
       " 0.4022201597690582,\n",
       " 0.3898020088672638,\n",
       " 0.6409820914268494,\n",
       " 0.082003153860569,\n",
       " 0.0,\n",
       " 0.8365394473075867,\n",
       " 0.23977898061275482,\n",
       " 0.005034235306084156,\n",
       " 0.29351887106895447,\n",
       " 1.672879934310913,\n",
       " 0.44271889328956604,\n",
       " 0.8575884699821472,\n",
       " 0.48172950744628906,\n",
       " 1.0473227500915527,\n",
       " 0.6514292359352112,\n",
       " 0.5903353691101074,\n",
       " 0.9807459712028503,\n",
       " 0.9056824445724487,\n",
       " 0.11390653252601624,\n",
       " 0.32994386553764343,\n",
       " 1.4101518392562866,\n",
       " 0.5362820029258728,\n",
       " 0.6830962896347046,\n",
       " 1.3368698358535767,\n",
       " 0.6396023035049438,\n",
       " 0.25912755727767944,\n",
       " 0.03223642706871033,\n",
       " 0.19667471945285797,\n",
       " 0.6066287755966187,\n",
       " 0.01872958615422249,\n",
       " 0.14001312851905823,\n",
       " 0.697624921798706,\n",
       " 0.5355572700500488,\n",
       " 0.5627530813217163,\n",
       " 1.4827220439910889,\n",
       " 0.1705055683851242,\n",
       " 1.105568766593933,\n",
       " 0.40207862854003906,\n",
       " 0.13300791382789612,\n",
       " 0.8745979070663452,\n",
       " 0.0004380572645459324,\n",
       " 0.8241612315177917,\n",
       " 0.5943044424057007,\n",
       " 0.6696501970291138,\n",
       " 0.1817445009946823,\n",
       " 0.0,\n",
       " 0.9086406230926514,\n",
       " 1.0352563858032227,\n",
       " 0.3369729518890381,\n",
       " 0.12715984880924225,\n",
       " 0.6534799933433533,\n",
       " 0.23813427984714508,\n",
       " 1.1037805080413818,\n",
       " 0.6419286727905273,\n",
       " 1.0052870512008667,\n",
       " 1.1990611553192139,\n",
       " 0.5879715085029602,\n",
       " 0.043186742812395096,\n",
       " 0.07170340418815613,\n",
       " 0.48284807801246643,\n",
       " 1.0460354089736938,\n",
       " 0.16183669865131378,\n",
       " 0.14161916077136993,\n",
       " 0.21936994791030884,\n",
       " 0.6157968640327454,\n",
       " 0.1958009898662567,\n",
       " 0.5015934705734253,\n",
       " 0.5079047679901123,\n",
       " 0.5092726945877075,\n",
       " 0.4715528190135956,\n",
       " 0.5183640718460083,\n",
       " 0.8663540482521057,\n",
       " 0.4036208689212799,\n",
       " 0.18732397258281708,\n",
       " 0.28161463141441345,\n",
       " 0.055715467780828476,\n",
       " 0.831108570098877,\n",
       " 0.8396826982498169,\n",
       " 0.00039365983684547246,\n",
       " 0.6847853660583496,\n",
       " 0.6640549302101135,\n",
       " 0.3130927085876465,\n",
       " 0.004145281855016947,\n",
       " 0.13216020166873932,\n",
       " 0.5793322920799255,\n",
       " 0.9153851270675659,\n",
       " 1.27147376537323,\n",
       " 0.07512170076370239,\n",
       " 0.5592222213745117,\n",
       " 0.2968188226222992,\n",
       " 1.297088384628296,\n",
       " 0.9560844898223877,\n",
       " 2.3158957958221436,\n",
       " 0.7308446168899536,\n",
       " 0.951460063457489,\n",
       " 1.5627080202102661,\n",
       " 0.6890840530395508,\n",
       " 0.34691476821899414,\n",
       " 0.6573816537857056,\n",
       " 0.6797606945037842,\n",
       " 0.46929410099983215,\n",
       " 0.9514678120613098,\n",
       " 0.735045313835144,\n",
       " 0.4586474299430847,\n",
       " 0.8678262829780579,\n",
       " 0.4561341404914856,\n",
       " 0.6378411054611206,\n",
       " 0.4888385534286499,\n",
       " 0.45688706636428833,\n",
       " 0.29295340180397034,\n",
       " 1.5161035060882568,\n",
       " 0.0007140177767723799,\n",
       " 1.1624784469604492,\n",
       " 0.18612615764141083,\n",
       " 1.20686936378479,\n",
       " 1.2122130393981934,\n",
       " 0.6205127239227295,\n",
       " 0.0,\n",
       " 0.23543618619441986,\n",
       " 0.0007379972375929356,\n",
       " 0.4289412200450897,\n",
       " 0.15540321171283722,\n",
       " 0.6525804400444031,\n",
       " 0.23966668546199799,\n",
       " 1.1765005588531494,\n",
       " 0.11175025254487991,\n",
       " 0.6176701784133911,\n",
       " 0.07805076241493225,\n",
       " 0.331449419260025,\n",
       " 0.2928519546985626,\n",
       " 0.514656662940979,\n",
       " 0.9761031270027161,\n",
       " 0.6895237565040588,\n",
       " 0.422698974609375,\n",
       " 0.19250905513763428,\n",
       " 0.51077800989151,\n",
       " 0.448382169008255,\n",
       " 0.8400192856788635,\n",
       " 0.4188682436943054,\n",
       " 0.47678422927856445,\n",
       " 1.0194787979125977,\n",
       " 0.0038688755594193935,\n",
       " 0.1654711812734604,\n",
       " 1.3358888626098633,\n",
       " 0.11820819228887558,\n",
       " 0.0,\n",
       " 1.2910419702529907,\n",
       " 0.8281356692314148,\n",
       " 0.062408652156591415,\n",
       " 0.45097196102142334,\n",
       " 0.026339437812566757,\n",
       " 0.18578006327152252,\n",
       " 0.46442773938179016,\n",
       " 0.24015845358371735,\n",
       " 0.7004827260971069,\n",
       " 0.07818051427602768,\n",
       " 0.27698463201522827,\n",
       " 1.1812527179718018,\n",
       " 0.6581670641899109,\n",
       " 0.9232539534568787,\n",
       " 0.03703384846448898,\n",
       " 0.005453205667436123,\n",
       " 0.3124372065067291,\n",
       " 0.3710583746433258,\n",
       " 1.106669545173645,\n",
       " 0.578248918056488,\n",
       " 0.5201290845870972,\n",
       " 0.6193906664848328,\n",
       " 0.8840276598930359,\n",
       " 1.2341753244400024,\n",
       " 1.485943078994751,\n",
       " 0.5034968852996826,\n",
       " 0.638940155506134,\n",
       " 0.33753153681755066,\n",
       " 0.9817624092102051,\n",
       " 0.5793217420578003,\n",
       " 0.19574230909347534,\n",
       " 0.9130889773368835,\n",
       " 0.4903586208820343,\n",
       " 0.2635655999183655,\n",
       " 0.7935401201248169,\n",
       " 0.40225285291671753,\n",
       " 0.03938139230012894,\n",
       " 0.29749009013175964,\n",
       " 0.0046902671456336975,\n",
       " 1.853127360343933,\n",
       " 0.5384020805358887,\n",
       " 0.831140398979187,\n",
       " 0.6450527906417847,\n",
       " 0.04953538626432419,\n",
       " 0.1303177922964096,\n",
       " 0.3216671347618103,\n",
       " 0.9536880254745483,\n",
       " 1.3664799928665161,\n",
       " 0.8405963182449341,\n",
       " 0.73835289478302,\n",
       " 0.38603582978248596,\n",
       " 0.043011877685785294,\n",
       " 0.9739142656326294,\n",
       " 0.3216899037361145,\n",
       " 0.1795756220817566,\n",
       " 1.23698890209198,\n",
       " 0.6262202262878418,\n",
       " 0.39543697237968445,\n",
       " 0.3458735942840576,\n",
       " 0.28473666310310364,\n",
       " 0.08902963995933533,\n",
       " 0.00254464172758162,\n",
       " 0.742393970489502,\n",
       " 0.139676034450531,\n",
       " 0.12650522589683533,\n",
       " 0.3714555501937866,\n",
       " 1.0917805433273315,\n",
       " 0.3943408727645874,\n",
       " 0.9076893329620361,\n",
       " 0.48158255219459534,\n",
       " 0.6682363748550415,\n",
       " 1.0022238492965698,\n",
       " 0.37589171528816223,\n",
       " 1.0296791791915894,\n",
       " 0.6614089608192444,\n",
       " 1.4288350343704224,\n",
       " 0.5692470073699951,\n",
       " 1.0104531049728394,\n",
       " 0.5090394020080566,\n",
       " 0.1784231960773468,\n",
       " 0.0,\n",
       " 1.071455955505371,\n",
       " 0.48365411162376404,\n",
       " 0.3082663118839264,\n",
       " 0.886122465133667,\n",
       " 0.6105605959892273,\n",
       " 0.8901296854019165,\n",
       " 0.3202568292617798,\n",
       " 0.4276765286922455,\n",
       " 0.02041829563677311,\n",
       " 1.1510801315307617,\n",
       " 1.1626735925674438,\n",
       " 0.21319006383419037,\n",
       " 0.17340406775474548,\n",
       " 0.0,\n",
       " 0.38936248421669006,\n",
       " 0.14288589358329773,\n",
       " 0.14391452074050903,\n",
       " 0.3493160307407379,\n",
       " 0.7540140748023987,\n",
       " 1.9687683582305908,\n",
       " 0.4373131990432739,\n",
       " 0.21412312984466553,\n",
       " 0.7211458086967468,\n",
       " 0.31743356585502625,\n",
       " 1.1831144094467163,\n",
       " 0.27595630288124084,\n",
       " 0.27532878518104553,\n",
       " 0.7174000144004822,\n",
       " 0.6098393201828003,\n",
       " 0.79901522397995,\n",
       " 0.6710553765296936,\n",
       " 0.03074217028915882,\n",
       " 0.07120554894208908,\n",
       " 0.0039038073737174273,\n",
       " 0.8840492963790894,\n",
       " 0.16696493327617645,\n",
       " 0.2649458944797516,\n",
       " 1.058811068534851,\n",
       " 0.53566974401474,\n",
       " 0.3616521656513214,\n",
       " 0.9842564463615417,\n",
       " 0.683972179889679,\n",
       " 0.38802507519721985,\n",
       " 0.0,\n",
       " 0.48505309224128723,\n",
       " 0.3850397765636444,\n",
       " 0.5209197998046875,\n",
       " 1.0560063123703003,\n",
       " 0.3455544710159302,\n",
       " 0.0,\n",
       " 0.43841612339019775,\n",
       " 0.46857568621635437,\n",
       " 0.0,\n",
       " 0.8049222826957703,\n",
       " 0.39988529682159424,\n",
       " 0.5237670540809631,\n",
       " 0.0014657144201919436,\n",
       " 0.5940325856208801,\n",
       " 0.2671264410018921,\n",
       " 0.0,\n",
       " 0.2373233586549759,\n",
       " 0.43875110149383545,\n",
       " 1.0541996955871582,\n",
       " 1.00892174243927,\n",
       " 0.4645148813724518,\n",
       " 0.1864784210920334,\n",
       " 0.5555911660194397,\n",
       " 1.8060848712921143,\n",
       " 0.3065502643585205,\n",
       " 0.13863685727119446,\n",
       " 0.2614304721355438,\n",
       " 0.6729487180709839,\n",
       " 1.261940360069275,\n",
       " 1.3365087509155273,\n",
       " 0.14168508350849152,\n",
       " 0.004925834946334362,\n",
       " 1.4554188251495361,\n",
       " 1.1059141159057617,\n",
       " 0.7611795663833618,\n",
       " 0.7018170356750488,\n",
       " 0.10902909934520721,\n",
       " 0.08540849387645721,\n",
       " 0.4132474362850189,\n",
       " 0.3265051543712616,\n",
       " 0.23658503592014313,\n",
       " 0.567356526851654,\n",
       " 0.28448987007141113,\n",
       " 0.6324697136878967,\n",
       " 0.08788805454969406,\n",
       " 0.5104267001152039,\n",
       " 0.6579039692878723,\n",
       " 0.00418108468875289,\n",
       " 0.2428903579711914,\n",
       " 1.1230273246765137,\n",
       " 0.0,\n",
       " 0.33824846148490906,\n",
       " 0.0,\n",
       " 0.007141833193600178,\n",
       " 0.7860268354415894,\n",
       " 0.060500096529722214,\n",
       " 0.46053722500801086,\n",
       " 0.29712939262390137,\n",
       " 0.2319629043340683,\n",
       " 0.034766942262649536,\n",
       " 0.11341491341590881,\n",
       " 1.5207291841506958,\n",
       " 1.7288885116577148,\n",
       " 0.6329889297485352,\n",
       " 1.7400819063186646,\n",
       " 0.019262101501226425,\n",
       " 0.7675219178199768,\n",
       " 0.13163931667804718,\n",
       " 0.0003049922815989703,\n",
       " 0.2996949851512909,\n",
       " 0.00013240749831311405,\n",
       " 0.2666206657886505,\n",
       " 0.38631874322891235,\n",
       " 0.33551523089408875,\n",
       " 0.6794143915176392,\n",
       " 0.7824943661689758,\n",
       " 0.0,\n",
       " 0.8172945976257324,\n",
       " 0.9643930792808533,\n",
       " 0.8687180280685425,\n",
       " 0.30611300468444824,\n",
       " 0.0032894948963075876,\n",
       " 0.45209425687789917,\n",
       " 1.2664918899536133,\n",
       " 0.0,\n",
       " 0.8740046620368958,\n",
       " 0.7512905597686768,\n",
       " 0.4274863004684448,\n",
       " 0.5475674271583557,\n",
       " 0.05210009962320328,\n",
       " 0.002229429315775633,\n",
       " 0.6684722304344177,\n",
       " ...]"
      ]
     },
     "execution_count": 71,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "features"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {},
   "outputs": [
    {
     "ename": "AuthorizationException",
     "evalue": "AuthorizationException(403, '{\"message\":\"The security token included in the request is expired\"}')",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mAuthorizationException\u001b[0m                    Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-72-ef9c1e9045c3>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[1;32m      4\u001b[0m res = es.search(request_timeout=30, index=idx_name,\n\u001b[1;32m      5\u001b[0m                 body={'size': k, \n\u001b[0;32m----> 6\u001b[0;31m                       'query': {'knn': {'zalando_img_vector': {'vector': features, 'k': k}}}})\n\u001b[0m",
      "\u001b[0;32m~/anaconda3/envs/mxnet_p36/lib/python3.6/site-packages/elasticsearch/client/utils.py\u001b[0m in \u001b[0;36m_wrapped\u001b[0;34m(*args, **kwargs)\u001b[0m\n\u001b[1;32m    166\u001b[0m                 \u001b[0;32mif\u001b[0m \u001b[0mp\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mkwargs\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    167\u001b[0m                     \u001b[0mparams\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mp\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mkwargs\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mpop\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mp\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 168\u001b[0;31m             \u001b[0;32mreturn\u001b[0m \u001b[0mfunc\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mparams\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mparams\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mheaders\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mheaders\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    169\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    170\u001b[0m         \u001b[0;32mreturn\u001b[0m \u001b[0m_wrapped\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/anaconda3/envs/mxnet_p36/lib/python3.6/site-packages/elasticsearch/client/__init__.py\u001b[0m in \u001b[0;36msearch\u001b[0;34m(self, body, index, doc_type, params, headers)\u001b[0m\n\u001b[1;32m   1673\u001b[0m             \u001b[0mparams\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mparams\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1674\u001b[0m             \u001b[0mheaders\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mheaders\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1675\u001b[0;31m             \u001b[0mbody\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mbody\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   1676\u001b[0m         )\n\u001b[1;32m   1677\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/anaconda3/envs/mxnet_p36/lib/python3.6/site-packages/elasticsearch/transport.py\u001b[0m in \u001b[0;36mperform_request\u001b[0;34m(self, method, url, headers, params, body)\u001b[0m\n\u001b[1;32m    413\u001b[0m                         \u001b[0;32mraise\u001b[0m \u001b[0me\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    414\u001b[0m                 \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 415\u001b[0;31m                     \u001b[0;32mraise\u001b[0m \u001b[0me\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    416\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    417\u001b[0m             \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/anaconda3/envs/mxnet_p36/lib/python3.6/site-packages/elasticsearch/transport.py\u001b[0m in \u001b[0;36mperform_request\u001b[0;34m(self, method, url, headers, params, body)\u001b[0m\n\u001b[1;32m    386\u001b[0m                     \u001b[0mheaders\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mheaders\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    387\u001b[0m                     \u001b[0mignore\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mignore\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 388\u001b[0;31m                     \u001b[0mtimeout\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mtimeout\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    389\u001b[0m                 )\n\u001b[1;32m    390\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/anaconda3/envs/mxnet_p36/lib/python3.6/site-packages/elasticsearch/connection/http_requests.py\u001b[0m in \u001b[0;36mperform_request\u001b[0;34m(self, method, url, params, body, timeout, ignore, headers)\u001b[0m\n\u001b[1;32m    202\u001b[0m                 \u001b[0mraw_data\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    203\u001b[0m             )\n\u001b[0;32m--> 204\u001b[0;31m             \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_raise_error\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mresponse\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mstatus_code\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mraw_data\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    205\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    206\u001b[0m         self.log_request_success(\n",
      "\u001b[0;32m~/anaconda3/envs/mxnet_p36/lib/python3.6/site-packages/elasticsearch/connection/base.py\u001b[0m in \u001b[0;36m_raise_error\u001b[0;34m(self, status_code, raw_data)\u001b[0m\n\u001b[1;32m    329\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    330\u001b[0m         raise HTTP_EXCEPTIONS.get(status_code, TransportError)(\n\u001b[0;32m--> 331\u001b[0;31m             \u001b[0mstatus_code\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0merror_message\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0madditional_info\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    332\u001b[0m         )\n\u001b[1;32m    333\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mAuthorizationException\u001b[0m: AuthorizationException(403, '{\"message\":\"The security token included in the request is expired\"}')"
     ]
    }
   ],
   "source": [
    "import json\n",
    "k = 5\n",
    "idx_name = 'idx_zalando'\n",
    "res = es.search(request_timeout=30, index=idx_name,\n",
    "                body={'size': k, \n",
    "                      'query': {'knn': {'zalando_img_vector': {'vector': features, 'k': k}}}})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for i in range(k):\n",
    "    key = res['hits']['hits'][i]['_source']['image']\n",
    "    key = key.replace(f's3://{bucket}/','')\n",
    "    if key.startswith('s3'):\n",
    "        key = 'images'+key\n",
    "    print(key)\n",
    "    img = display_image(bucket,key)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Deploying a full-stack visual search application"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "s3_resource.Object(bucket, 'backend/template.yaml').upload_file('./backend/template.yaml', ExtraArgs={'ACL':'public-read'})\n",
    "\n",
    "\n",
    "sam_template_url = f'https://{bucket}.s3.amazonaws.com/backend/template.yaml'\n",
    "\n",
    "# Generate the CloudFormation Quick Create Link\n",
    "\n",
    "print(\"Click the URL below to create the backend API for visual search:\\n\")\n",
    "print((\n",
    "    'https://console.aws.amazon.com/cloudformation/home?region=us-east-1#/stacks/create/review'\n",
    "    f'?templateURL={sam_template_url}'\n",
    "    '&stackName=vis-search-api'\n",
    "    f'&param_BucketName={outputs[\"s3BucketTraining\"]}'\n",
    "    f'&param_DomainName={outputs[\"esDomainName\"]}'\n",
    "    f'&param_ElasticSearchURL={outputs[\"esHostName\"]}'\n",
    "    f'&param_SagemakerEndpoint=image-embedding'  # TODO predictor.endpoint\n",
    "))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now that you have a working Amazon SageMaker endpoint for extracting image features and a KNN index on Elasticsearch, you are ready to build a real-world full-stack ML-powered web app. The SAM template you just created will deploy an Amazon API Gateway and AWS Lambda function. The Lambda function runs your code in response to HTTP requests that are sent to the API Gateway."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Review the content of the Lambda function code.\n",
    "!pygmentize backend/lambda/app.py"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Once the CloudFormation Stack shows CREATE_COMPLETE, proceed to this cell below:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Save the REST endpoint for the search API to a config file, to be used by the frontend build\n",
    "\n",
    "import json\n",
    "api_endpoint = get_cfn_outputs('vis-search-api')['ImageSimilarityApi']\n",
    "\n",
    "with open('./frontend/src/config/config.json', 'w') as outfile:\n",
    "    json.dump({'apiEndpoint': api_endpoint}, outfile)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 2: Deploy frontend services"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# add NPM to the path so we can assemble the web frontend from our notebook code\n",
    "\n",
    "from os import environ\n",
    "\n",
    "npm_path = ':/home/ec2-user/anaconda3/envs/JupyterSystemEnv/bin'\n",
    "\n",
    "if npm_path not in environ['PATH']:\n",
    "    ADD_NPM_PATH = environ['PATH']\n",
    "    ADD_NPM_PATH = ADD_NPM_PATH + npm_path\n",
    "else:\n",
    "    ADD_NPM_PATH = environ['PATH']\n",
    "    \n",
    "%set_env PATH=$ADD_NPM_PATH"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%cd ./frontend/\n",
    "\n",
    "!npm install"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!npm run-script build"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "hosting_bucket = f\"s3://{outputs['s3BucketHostingBucketName']}\"\n",
    "\n",
    "!aws s3 sync ./build/ $hosting_bucket --acl public-read"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 3: Browse your frontend service, and upload an image"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print('Click the URL below:\\n')\n",
    "print(outputs['S3BucketSecureURL'] + '/index.html')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You should see the following page:\n",
    "\n",
    "![Website](pi3small.png)\n",
    "\n",
    "On the website, try pasting the following URL in the URL text field.\n",
    "\n",
    "`https://i4.ztat.net/large/VE/12/1C/14/8K/12/VE121C148-K12@10.jpg`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Extensions\n",
    "\n",
    "We have used pretrained Resnet50 model which is trained on Imagenet dataset. Now based on your use-case you can fine tune any pre-trained models, such as VGG, Inception, and MobileNet with your own dataset and host the model in Amazon SageMaker.\n",
    "\n",
    "You can also use Amazon SageMaker Batch transform job to have a bulk feaures extracted from your stored S3 images and then you can use AWS Glue to import that data into Elasticeearch domain.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Cleanup\n",
    "\n",
    "Make sure that you stop the notebook instance, delete the Amazon SageMaker endpoint and delete the Elasticsearch domain to prevent any additional charges."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Delete the endpoint\n",
    "# predictor.delete_endpoint()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Empty S3 Contents\n",
    "# training_bucket_resource = s3_resource.Bucket(bucket)\n",
    "# training_bucket_resource.objects.all().delete()\n",
    "\n",
    "# hosting_bucket_resource = s3_resource.Bucket(outputs['s3BucketHostingBucketName'])\n",
    "# hosting_bucket_resource.objects.all().delete()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "conda_mxnet_p36",
   "language": "python",
   "name": "conda_mxnet_p36"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}