Implment IIIFv3 and Lambda response streaming

- Convert Lambda to streaming function url
- Use IIIF v3 branch of node-iiif
- Add IIIF v3 tests and other supports
- Add service discovery endpoint at `/`
- Create Terraform module
- Create CloudFormation and Terraform examples
- Enhance docs

.gitignore

@@ -1,11 +1,26 @@
+# AWS SAM
 .aws
 .aws-sam
-coverage
 deploy.yaml
-node_modules
-samconfig.toml
 package.yml
+samconfig.toml
+
+# NodeJS Build
+coverage
+node_modules
+
+# Secrets
+.env
+env.json
+
+# Terraform
+.terraform*
+*.tfstate*
+*.tfvars
+
+# Misc
+.vscode
 *~undo-tree~
 .DS_Store
-.env
 sharp-lambda-layer.*
+*.log

README.md

@@ -4,16 +4,21 @@
 [![Maintainability](https://api.codeclimate.com/v1/badges/4ac80b539190cb5b082f/maintainability)](https://codeclimate.com/github/samvera/serverless-iiif/maintainability)
 [![Test Coverage](https://coveralls.io/repos/github/samvera/serverless-iiif/badge.svg)](https://coveralls.io/github/samvera/serverless-iiif)
 
+## Upgrade Note
+
+Previous versions of this application featured an optional [CloudFront](https://aws.amazon.com/cloudfront/) distribution that provided caching, custom domain/hostname mapping, and request/response pre/post-processing. However, the primary motivation for including this feature in the past was that it provided a complicated but effective way to skirt the hard 6 megabyte limit for Lambda function response payloads. Since then, AWS has introduced [AWS Lambda response streaming](https://aws.amazon.com/blogs/compute/introducing-aws-lambda-response-streaming/), which uses chunked responses to bypass the 6 megabyte limit. As this is a much more elegant solution to the problem, there's nothing about the CloudFront template that's specific to this project any more. Ongoing development and maintenance will therefore focus on the IIIF Lambda itself rather than the large, complicated template required for a flexible, customizable CloudFront deployment.
+
+While the CloudFront-enabled version of the application is no longer available in the Serverless Application Repository, the newly created [`examples`](./examples/README.md) directory includes sample [CloudFormation](https://aws.amazon.com/cloudformation/) templates and [Terraform](https://terraform.io/) manifests showing how to deploy the IIIF service as part of a larger application/infrastructure stack.
+
 ## Description
 
-A [IIIF 2.1 Image API](https://iiif.io/api/image/2.1/) compliant server written as an [AWS Serverless Application](https://aws.amazon.com/serverless/sam/).
+A IIIF [2.1](https://iiif.io/api/image/2.1/) and [3.0](https://iiif.io/api/image/3.0/) Image API compliant server written as an [AWS Serverless Application](https://aws.amazon.com/serverless/sam/).
 
 ## Components
 
 * A simple [Lambda Function](https://aws.amazon.com/lambda/) wrapper for the [iiif-processor](https://www.npmjs.com/package/iiif-processor) module.
 * A [Lambda Function URL](https://docs.aws.amazon.com/lambda/latest/dg/lambda-urls.html) that is used to invoke the IIIF API via HTTPS.
 * A [Lambda Layer](https://docs.aws.amazon.com/lambda/latest/dg/configuration-layers.html) containing all the dependencies for the Lambda Function.
-* An optional [CloudFormation](https://aws.amazon.com/cloudformation/) template describing the resources needed to deploy the application.
 
 ## Prerequisites
 
@@ -24,39 +29,33 @@ A [IIIF 2.1 Image API](https://iiif.io/api/image/2.1/) compliant server written
 
 ## Quick Start
 
-`serverless-iiif` comes in two flavors: *Standalone (Lambda-only)* and *Caching (CloudFront-enabled)*. The Standalone version is much simpler, but lacks the following features:
-
-- Custom Domain Name
-  - Standalone URLs are in the `lambda-url.AWS_REGION.on.aws` domain (e.g., `https://fu90293j0pj902j902c32j902.lambda-url.us-east-1.on.aws/iiif/2/`)
-  - Caching URLs *without* Custom Domains are in the `cloudfront.net` domain (e.g., `https://d3kmjdzzy1l5t3.cloudfront.net/iiif/2/`)
-- Responses larger than ~6MB
-- CloudFront function support (for pre/post-processing requests and responses)
+`serverless-iiif` is deployed as a Lambda Function URL, in the `lambda-url.AWS_REGION.on.aws` domain (e.g., `https://fu90293j0pj902j902c32j902.lambda-url.us-east-1.on.aws/iiif/2/`). In order to use a custom domain name, or other features like caching and pre/post-processing functions, you'll have to set up a [CloudFront distribution](https://aws.amazon.com/cloudfront/).
 
 ### Deploying via the AWS Serverless Application Repository
 
 `serverless-iiif` is distributed and deployed via the [AWS Serverless Application Repository](https://aws.amazon.com/serverless/serverlessrepo/). To deploy it using the AWS Console:
 
-1. Click one of the following links to deploy the desired application from the AWS Console:
-   - [Standalone (Lambda-Only) Version](https://console.aws.amazon.com/lambda/home#/create/app?applicationId=arn:aws:serverlessrepo:us-east-1:625046682746:applications/serverless-iiif-standalone)
-   - [Caching (CloudFront-Enabled) Version](https://console.aws.amazon.com/lambda/home#/create/app?applicationId=arn:aws:serverlessrepo:us-east-1:625046682746:applications/serverless-iiif-cloudfront)
+1. Find the [serverless-iiif application](https://console.aws.amazon.com/lambda/home#/create/app?applicationId=arn:aws:serverlessrepo:us-east-1:625046682746:applications/serverless-iiif) in the AWS Serverless Application Repository.
 2. Make sure your currently selected region (in the console's top navigation bar) is the one you want to deploy to.
 3. Scroll down to the **Application settings** section.
 4. Configure the deploy template:
    - Give your stack a unique **Application name**
    - Enter the name of the **SourceBucket** the service will serve images from
-   - Check the box acknowledging that the app will create a custom IAM roles and resource policies (and if deploying the Caching version, that it will also deploy a nested application)
-   - *Optional*: Enter or change any other parameters that apply to your desired configuration.
+   - Check the box acknowledging that the app will create a custom IAM roles and resource policies
+   - *Optional*: Enter or change any other parameters that apply to your desired configuration
 5. Click **Deploy**.
-6. When all the resources are properly created and configured, the new stack should be in the **CREATE_COMPLETE** stage. If there's an error, it will delete all the resources it created, roll back any changes it made, and eventually reach the **ROLLBACK_COMPLETE** stage.
+6. When all the resources are properly created and configured, the new stack should be in the **CREATE_COMPLETE** stage. If there's an error, 
+   it will delete all the resources it created, roll back any changes it made, and eventually reach the **ROLLBACK_COMPLETE** stage.
 7. Click the **CloudFormation stack** link.
-8. Click the **Outputs** tab to see (and copy) the IIIF Endpoint URL.
+8. Click the **Outputs** tab to see (and copy) the IIIF Endpoint URLs.
 
 ### Deploying via the Command Line
 
 1. Make sure you have the [SAM CLI](https://aws.amazon.com/serverless/sam/) and [AWS CLI](https://aws.amazon.com/cli/) installed.
-2. Make sure the AWS CLI is [properly configured](https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-configure.html) with credentials that have sufficient access to manage IAM, S3, Lambda, and (optionally) CloudFront resources.
+2. Make sure the AWS CLI is [properly configured](https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-configure.html) with 
+   credentials that have sufficient access to manage IAM, S3, and Lambda resources.
 3. Clone this repository.
-4. Copy .env.example to .env. Update the various values within.
+4. Copy `.env.example` to `.env`. Update the various values within.
 5. Build the application:
    ```shell
    $ npm run build
@@ -66,7 +65,7 @@ A [IIIF 2.1 Image API](https://iiif.io/api/image/2.1/) compliant server written
    $ npm run deploy
    ```
 
-  You'll be prompted for various configuration parameters, confirmations, and acknowledgments of specific issues (particularly the creation of IAM resources and the deployment of an open/unauthenticated Lambda Function URL).
+   You'll be prompted for various configuration parameters, confirmations, and acknowledgments of specific issues (particularly the creation of IAM resources and the deployment of an open/unauthenticated Lambda Function URL).
 7. Follow the prompts to complete the deployment process and get the resulting endpoint.
 
 ### Deleting the application
@@ -123,16 +122,16 @@ The default values will work in most circumstances, but if you need the IIIF ser
 
 ### Request/Response Functions
 
-The SAM deploy template takes several optional parameters to enable the association of [CloudFront Functions](https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/cloudfront-functions.html) or [Lambda@Edge Functions](https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/lambda-at-the-edge.html) with the CloudFront distribution. These functions can perform authentication and authorization functions, change how the S3 file and/or image dimensions are resolved, or alter the response from the lambda or cache. These parameters are:
+The IIIF service can be heavily customized through the use of [CloudFront Functions](https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/cloudfront-functions.html) or [Lambda@Edge Functions](https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/lambda-at-the-edge.html) attached to a CloudFront distribution in front of the service. It's important to understand the four stages of CloudFront processing in order to know where a given type of customization belongs.
 
-* `OriginRequestARN`: ARN of the Lambda@Edge Function to use at the origin-request stage
-* `OriginResponseARN`: ARN of the Lambda@Edge Function to use at the origin-response stage
-* `ViewerRequestARN`: ARN of the CloudFront or Lambda@Edge Function to use at the viewer-request stage
-* `ViewerRequestType`: Type of viewer-request Function to use (`CloudWatch Function` or `Lambda@Edge`)
-* `ViewerResponseARN`: ARN of the CloudFront or Lambda@Edge Function to use at the viewer-response stage
-* `ViewerResponseType`: Type of viewer-response Function to use (`CloudWatch Function` or `Lambda@Edge`)
-
-These functions, if used, must be created, configured, and published before the serverless application is deployed.
+- A `viewer-request` function will be called on every request, cached or not. This is the appropriate place to attach 
+  a function that performs authorization, authentication, or anything else whose result should *not* be cached.
+- An `origin-request` function will only be called when CloudFront refreshes the content from the origin (e.g., the IIIF server).
+  It's the appropriate place to attach a function that *should* be cached, such as S3 file resolution or the retrieval of
+  image dimensions.
+- Similarly, the `origin-response` and `viewer-response` functions are called after the IIIF server returns its response
+  and before CloudFront passes it on to the viewer, respectively. They can be used to alter the response in a way that is
+  either cached or ephemeral.
 
 #### Examples
 
@@ -178,13 +177,7 @@ For example, the following dimension values would all describe the same pyramida
 
 The `limit` calculator will keep going until both dimensions are _less than_ the limit, not _less than or equal to_. So a `limit: 512` on the third example above would generate a fourth page at `{ width: 256, height: 192 }`.
 
-*Note:* The SAM deploy template adds a `preflight=true` environment variable to the main IIIF Lambda if a preflight function is provided. The function will _only_ look for the preflight headers if this environment variable is `true`. This prevents requests from including those headers directly if no preflight function is present. If you do use a preflight function, make sure it strips out any `x-preflight-location` and `x-preflight-dimensions` headers that it doesn't set itself.
-
-## Notes
-
-Lambda Function URLs have a payload (request/response body) size limit of approximately 6MB in both directions. To overcome this limitation, the Lambda URL is configured behind an AWS CloudFront distribution with two origins - the API and a cache bucket. Responses larger than 6MB are saved to the cache bucket at the same relative path as the request, and the Lambda returns a `404 Not Found` response to CloudFront. CloudFront then fails over to the second origin (the cache bucket), where it finds the actual response and returns it.
-
-The cache bucket uses an S3 lifecycle rule to expire cached responses in 1 day.
+*Note:* If you plan to use CloudFront functions to add either of the above `x-preflight-` headers to incoming requests, you *must* set the value of the `Preflight` parameter to `true` when deploying `serverless-iiif`. The function will _only_ look for the preflight headers if this environment variable is `true`. This prevents requests from including those headers directly if no preflight function is present. If you do use a preflight function, make sure it strips out any `x-preflight-location` and `x-preflight-dimensions` headers that it doesn't set itself.
 
 ## License
 
@@ -197,7 +190,7 @@ The cache bucket uses an S3 lifecycle rule to expire cached responses in 1 day.
 * [Rob Kaufman](https://github.com/orangewolf)
 * [Edward Silverton](https://github.com/edsilv)
 * [Trey Pendragon](https://github.com/tpendragon)
-* [Dan Wolfe](https://github.com/danthewolfe)
+* [Theia Wolfe](https://github.com/theiawolfe)
 
 ## Contributing
 

bin/build

@@ -5,7 +5,7 @@ const Setup = require('./setup.js')
 
 var setup = new Setup();
 
-var cmd = `cd sam/${setup.type} && sam build --use-container`
+var cmd = `cd sam && sam build --use-container`
 
 console.log(cmd)
 

bin/deploy

@@ -25,12 +25,12 @@ var keys = [
 
 let parameter_overrides = ""
 keys.forEach(element => {
-  if(process.env[element] !== undefined) {
+  if (process.env[element] !== undefined) {
     parameter_overrides = parameter_overrides + ` ${element}="${process.env[element]}"`
   }
 })
 
-var cmd = `cd sam/${setup.type} && sam deploy --capabilities CAPABILITY_IAM CAPABILITY_AUTO_EXPAND --region ${process.env['AWS_REGION']} --profile ${process.env['AWS_PROFILE']} --stack-name ${process.env['AWS_STACK_NAME']} --resolve-s3 --parameter-overrides ${parameter_overrides}`
+var cmd = `sam deploy --capabilities CAPABILITY_IAM CAPABILITY_AUTO_EXPAND --region ${process.env['AWS_REGION']} --profile ${process.env['AWS_PROFILE']} --stack-name ${process.env['AWS_STACK_NAME']} --resolve-s3 --parameter-overrides ${parameter_overrides}`
 console.log(cmd)
 
 exec(cmd, (error, stdout, stderr) => {

bin/setup.js

@@ -1,11 +1,10 @@
 class Setup {
   constructor() {
-    this.type = process.env.DEPLOY_TYPE || "cloudfront"
-    if(process.env.LOCAL_SHARP){
+    if (process.env.LOCAL_SHARP) {
       var fs = require('fs');
       var path = require('path');
-      var file_path = path.resolve(__dirname, "..", "sam/standalone/template.yml");
-      fs.readFile(file_path, {encoding: 'utf8'}, function (err,data) {
+      var file_path = path.resolve(__dirname, "..", "sam/template.yml");
+      fs.readFile(file_path, {encoding: 'utf8'}, function (err, data) {
         var formatted = data.replace(/^\s+ContentUri: s3:\/\/bucket_name\/sharp-lambda-layer.*/g, "      ContentUri: ../../sharp-lambda-layer.x86_64.zip");
         fs.writeFile(file_path, formatted, 'utf8', function (err) {
           if (err) return console.log(err);

bin/update_version

@@ -12,10 +12,11 @@ for f in $root/package.json $root/src/package.json $root/dependencies/package.js
   jq --arg v "$version" '.version = $v' <<< $content > $f
 done
 
-for f in $root/sam/standalone/template.yml $root/sam/cloudfront/template.yml; do
-  if [[ "$OSTYPE" = "darwin*" ]]; then
-    sed -i '' -E "s/SemanticVersion: .+/SemanticVersion: $version/" $f
-  else
-    sed -i'' -E "s/SemanticVersion: .+/SemanticVersion: $version/" $f
-  fi
-done
+if [[ "$OSTYPE" = "darwin*" ]]; then
+  sedreplace="sed -i ''"
+else
+  sedreplace="sed -i''"
+fi
+
+$sedreplace -E "s/SemanticVersion: .+/SemanticVersion: $version/" $root/sam/template.yml
+$sedreplace -E "s/  serverless_iiif_app_version   =.+/  serverless_iiif_app_version   = \"$version\"" $root/extras/terraform/main.tf

dependencies/package.json

@@ -1,11 +1,11 @@
 {
   "name": "serverless-iiif-dependencies",
-  "version": "4.3.2",
+  "version": "5.0.0",
   "description": "Dependencies for serverless IIIF",
   "author": "Michael B. Klein",
   "license": "Apache-2.0",
   "dependencies": {
     "aws-sdk": "^2.368",
-    "iiif-processor": "^3.2.3"
+    "iiif-processor": "^4.0.0"
   }
 }

examples/cloudformation/README.md

@@ -0,0 +1,10 @@
+# serverless-iiif CloudFormation Examples
+
+This directory contains examples of how to use [CloudFormation](https://aws.amazon.com/cloudformation/) to deploy a serverless-iiif image server as part of a larger application stack. Please refer to the [CloudFormation documentation](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/Welcome.html) for more information on how to tailor these templates to your own needs and deploy them to AWS.
+
+## `custom_hostname.yml`
+
+The `custom_hostname` template will deploy a full application stack consisting of:
+
+- A serverless-iiif image server
+- A CloudFront distribution with a custom hostname and SSL certificate