Added server logging bucket (#15)

* Get logging with s3 bucket

* Get logging with s3 bucket

* Logging

* Added logging

* Added main

* Update main.tf

Co-Authored-By: Erik Osterman <erik@cloudposse.com>

* Added access logs bucket

README.md

@@ -8,6 +8,8 @@
 
 Terraform module to provision an S3 bucket with built in policy to allow [CloudTrail](https://aws.amazon.com/cloudtrail/) [logs](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/logging-management-and-data-events-with-cloudtrail.html).
 
+In addtion to the main S3 bucket the module creates S3 Bucket for access logs for the first one.
+
 This is useful if an organization uses a number of separate AWS accounts to isolate the Audit environment from other environments (production, staging, development).
 
 In this case, you create CloudTrail in the production environment (Production AWS account),
@@ -88,9 +90,23 @@ Available targets:
 
 | Name | Description | Type | Default | Required |
 |------|-------------|:----:|:-----:|:-----:|
+| access_logs_acl | Canned ACL to apply to the logs S3 bucket | string | `private` | no |
+| access_logs_expiration_days | Number of days after which to expunge the objects for access logs s3 bucket | string | `90` | no |
+| access_logs_force_destroy | A boolean that indicates the logs bucket can be destroyed even if it contains objects. These objects are not recoverable | string | `false` | no |
+| access_logs_glacier_transition_days | Number of days after which to move the data to the glacier storage tier for access logs s3 bucket | string | `60` | no |
+| access_logs_kms_master_key_arn | The AWS KMS master key ARN used for the SSE-KMS encryption. This can only be used when you set the value of sse_algorithm as aws:kms. The default aws/s3 AWS KMS master key is used if this element is absent while the sse_algorithm is aws:kms for access logs s3 bucket | string | `` | no |
+| access_logs_lifecycle_prefix | Prefix filter for access logs s3 bucket. Used to manage object lifecycle events | string | `` | no |
+| access_logs_lifecycle_rule_enabled | Enable lifecycle events on this logs bucket | string | `true` | no |
+| access_logs_lifecycle_tags | Tags filter for access logs s3 bucket. Used to manage object lifecycle events | map | `<map>` | no |
+| access_logs_noncurrent_version_expiration_days | Specifies when noncurrent object versions expire for access logs s3 bucket | string | `90` | no |
+| access_logs_noncurrent_version_transition_days | Specifies when noncurrent object versions transition for access logs s3 bucket | string | `30` | no |
+| access_logs_sse_algorithm | The server-side encryption algorithm to use. Valid values are AES256 and aws:kms for access logs s3 bucket | string | `AES256` | no |
+| access_logs_standard_transition_days | Number of days to persist in the standard storage tier before moving to the infrequent access tier for access logs s3 bucket | string | `30` | no |
+| access_logs_versioning_enabled | A state of versioning logs bucket. Versioning is a means of keeping multiple variants of an object in the same bucket | string | `true` | no |
 | acl | Canned ACL to apply to the S3 bucket | string | `log-delivery-write` | no |
 | attributes | Additional attributes (e.g. `logs`) | list | `<list>` | no |
 | delimiter | Delimiter to be used between `namespace`, `stage`, `name` and `attributes` | string | `-` | no |
+| enabled | Set to `false` to prevent the module from creating any resources | string | `true` | no |
 | expiration_days | Number of days after which to expunge the objects | string | `90` | no |
 | force_destroy | A boolean that indicates the bucket can be destroyed even if it contains objects. These objects are not recoverable | string | `false` | no |
 | glacier_transition_days | Number of days after which to move the data to the glacier storage tier | string | `60` | no |
@@ -113,9 +129,12 @@ Available targets:
 
 | Name | Description |
 |------|-------------|
+| access_logs_bucket_arn | Access logs bucket ARN |
+| access_logs_bucket_id | Access logs bucket name (aka ID) |
 | bucket_arn | Bucket ARN |
 | bucket_domain_name | FQDN of bucket |
-| bucket_id | Bucket ID |
+| bucket_id | Bucket Name (aka ID) |
+| enabled | Is module enabled |
 | prefix | Prefix configured for lifecycle rules |
 
 

README.yaml

@@ -46,6 +46,8 @@ related:
 description: |-
   Terraform module to provision an S3 bucket with built in policy to allow [CloudTrail](https://aws.amazon.com/cloudtrail/) [logs](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/logging-management-and-data-events-with-cloudtrail.html).
 
+  In addtion to the main S3 bucket the module creates S3 Bucket for access logs for the first one.
+
   This is useful if an organization uses a number of separate AWS accounts to isolate the Audit environment from other environments (production, staging, development).
 
   In this case, you create CloudTrail in the production environment (Production AWS account),

docs/terraform.md

@@ -2,9 +2,23 @@
 
 | Name | Description | Type | Default | Required |
 |------|-------------|:----:|:-----:|:-----:|
+| access_logs_acl | Canned ACL to apply to the logs S3 bucket | string | `private` | no |
+| access_logs_expiration_days | Number of days after which to expunge the objects for access logs s3 bucket | string | `90` | no |
+| access_logs_force_destroy | A boolean that indicates the logs bucket can be destroyed even if it contains objects. These objects are not recoverable | string | `false` | no |
+| access_logs_glacier_transition_days | Number of days after which to move the data to the glacier storage tier for access logs s3 bucket | string | `60` | no |
+| access_logs_kms_master_key_arn | The AWS KMS master key ARN used for the SSE-KMS encryption. This can only be used when you set the value of sse_algorithm as aws:kms. The default aws/s3 AWS KMS master key is used if this element is absent while the sse_algorithm is aws:kms for access logs s3 bucket | string | `` | no |
+| access_logs_lifecycle_prefix | Prefix filter for access logs s3 bucket. Used to manage object lifecycle events | string | `` | no |
+| access_logs_lifecycle_rule_enabled | Enable lifecycle events on this logs bucket | string | `true` | no |
+| access_logs_lifecycle_tags | Tags filter for access logs s3 bucket. Used to manage object lifecycle events | map | `<map>` | no |
+| access_logs_noncurrent_version_expiration_days | Specifies when noncurrent object versions expire for access logs s3 bucket | string | `90` | no |
+| access_logs_noncurrent_version_transition_days | Specifies when noncurrent object versions transition for access logs s3 bucket | string | `30` | no |
+| access_logs_sse_algorithm | The server-side encryption algorithm to use. Valid values are AES256 and aws:kms for access logs s3 bucket | string | `AES256` | no |
+| access_logs_standard_transition_days | Number of days to persist in the standard storage tier before moving to the infrequent access tier for access logs s3 bucket | string | `30` | no |
+| access_logs_versioning_enabled | A state of versioning logs bucket. Versioning is a means of keeping multiple variants of an object in the same bucket | string | `true` | no |
 | acl | Canned ACL to apply to the S3 bucket | string | `log-delivery-write` | no |
 | attributes | Additional attributes (e.g. `logs`) | list | `<list>` | no |
 | delimiter | Delimiter to be used between `namespace`, `stage`, `name` and `attributes` | string | `-` | no |
+| enabled | Set to `false` to prevent the module from creating any resources | string | `true` | no |
 | expiration_days | Number of days after which to expunge the objects | string | `90` | no |
 | force_destroy | A boolean that indicates the bucket can be destroyed even if it contains objects. These objects are not recoverable | string | `false` | no |
 | glacier_transition_days | Number of days after which to move the data to the glacier storage tier | string | `60` | no |
@@ -27,8 +41,11 @@
 
 | Name | Description |
 |------|-------------|
+| access_logs_bucket_arn | Access logs bucket ARN |
+| access_logs_bucket_id | Access logs bucket name (aka ID) |
 | bucket_arn | Bucket ARN |
 | bucket_domain_name | FQDN of bucket |
-| bucket_id | Bucket ID |
+| bucket_id | Bucket Name (aka ID) |
+| enabled | Is module enabled |
 | prefix | Prefix configured for lifecycle rules |
 

main.tf

@@ -6,6 +6,42 @@ module "label" {
   delimiter  = "${var.delimiter}"
   attributes = "${var.attributes}"
   tags       = "${var.tags}"
+  enabled    = "${var.enabled}"
+}
+
+module "access_logs_bucket" {
+  source = "git::https://github.com/cloudposse/terraform-aws-s3-log-storage.git?ref=tags/0.4.1"
+
+  enabled = "${var.enabled}"
+
+  namespace  = "${var.namespace}"
+  stage      = "${var.stage}"
+  name       = "${var.name}"
+  delimiter  = "${var.delimiter}"
+  attributes = "${concat(list("access", "logging"), var.attributes)}"
+  tags       = "${var.tags}"
+
+  region = "${var.region}"
+
+  acl = "${var.access_logs_acl}"
+
+  force_destroy = "${var.access_logs_force_destroy}"
+
+  versioning_enabled = "${var.access_logs_versioning_enabled}"
+
+  lifecycle_rule_enabled = "${var.access_logs_lifecycle_rule_enabled}"
+  lifecycle_prefix       = "${var.access_logs_lifecycle_prefix}"
+  lifecycle_tags         = "${var.access_logs_lifecycle_tags}"
+
+  noncurrent_version_expiration_days = "${var.access_logs_noncurrent_version_expiration_days}"
+  noncurrent_version_transition_days = "${var.access_logs_noncurrent_version_transition_days}"
+
+  standard_transition_days = "${var.access_logs_standard_transition_days}"
+  glacier_transition_days  = "${var.access_logs_glacier_transition_days}"
+  expiration_days          = "${var.access_logs_expiration_days}"
+
+  sse_algorithm      = "${var.access_logs_sse_algorithm}"
+  kms_master_key_arn = "${var.access_logs_kms_master_key_arn}"
 }
 
 data "aws_iam_policy_document" "default" {
@@ -53,27 +89,63 @@ data "aws_iam_policy_document" "default" {
   }
 }
 
-module "s3_bucket" {
-  source                             = "git::https://github.com/cloudposse/terraform-aws-s3-log-storage.git?ref=tags/0.4.1"
-  namespace                          = "${var.namespace}"
-  stage                              = "${var.stage}"
-  name                               = "${var.name}"
-  region                             = "${var.region}"
-  acl                                = "${var.acl}"
-  policy                             = "${data.aws_iam_policy_document.default.json}"
-  force_destroy                      = "${var.force_destroy}"
-  versioning_enabled                 = "${var.versioning_enabled}"
-  lifecycle_rule_enabled             = "${var.lifecycle_rule_enabled}"
-  lifecycle_prefix                   = "${var.lifecycle_prefix}"
-  lifecycle_tags                     = "${var.lifecycle_tags}"
-  noncurrent_version_expiration_days = "${var.noncurrent_version_expiration_days}"
-  noncurrent_version_transition_days = "${var.noncurrent_version_transition_days}"
-  standard_transition_days           = "${var.standard_transition_days}"
-  glacier_transition_days            = "${var.glacier_transition_days}"
-  expiration_days                    = "${var.expiration_days}"
-  sse_algorithm                      = "${var.sse_algorithm}"
-  kms_master_key_arn                 = "${var.kms_master_key_arn}"
-  delimiter                          = "${var.delimiter}"
-  attributes                         = "${var.attributes}"
-  tags                               = "${var.tags}"
+resource "aws_s3_bucket" "default" {
+  count         = "${var.enabled == "true" ? 1 : 0}"
+  bucket        = "${module.label.id}"
+  acl           = "${var.acl}"
+  region        = "${var.region}"
+  force_destroy = "${var.force_destroy}"
+  policy        = "${data.aws_iam_policy_document.default.json}"
+
+  versioning {
+    enabled = "${var.versioning_enabled}"
+  }
+
+  lifecycle_rule {
+    id      = "${module.label.id}"
+    enabled = "${var.lifecycle_rule_enabled}"
+
+    prefix = "${var.lifecycle_prefix}"
+    tags   = "${var.lifecycle_tags}"
+
+    noncurrent_version_expiration {
+      days = "${var.noncurrent_version_expiration_days}"
+    }
+
+    noncurrent_version_transition {
+      days          = "${var.noncurrent_version_transition_days}"
+      storage_class = "GLACIER"
+    }
+
+    transition {
+      days          = "${var.standard_transition_days}"
+      storage_class = "STANDARD_IA"
+    }
+
+    transition {
+      days          = "${var.glacier_transition_days}"
+      storage_class = "GLACIER"
+    }
+
+    expiration {
+      days = "${var.expiration_days}"
+    }
+  }
+
+  # https://docs.aws.amazon.com/AmazonS3/latest/dev/bucket-encryption.html
+  # https://www.terraform.io/docs/providers/aws/r/s3_bucket.html#enable-default-server-side-encryption
+  server_side_encryption_configuration {
+    rule {
+      apply_server_side_encryption_by_default {
+        sse_algorithm     = "${var.sse_algorithm}"
+        kms_master_key_id = "${var.kms_master_key_arn}"
+      }
+    }
+  }
+
+  logging {
+    target_bucket = "${module.access_logs_bucket.bucket_id}"
+  }
+
+  tags = "${module.label.tags}"
 }

outputs.tf

@@ -1,19 +1,34 @@
 output "bucket_domain_name" {
-  value       = "${module.s3_bucket.bucket_domain_name}"
+  value       = "${var.enabled == "true" ? join("", aws_s3_bucket.default.*.bucket_domain_name) : ""}"
   description = "FQDN of bucket"
 }
 
 output "bucket_id" {
-  value       = "${module.s3_bucket.bucket_id}"
-  description = "Bucket ID"
+  value       = "${var.enabled == "true" ? join("", aws_s3_bucket.default.*.id) : ""}"
+  description = "Bucket Name (aka ID)"
 }
 
 output "bucket_arn" {
-  value       = "${module.s3_bucket.bucket_arn}"
+  value       = "${var.enabled == "true" ? join("", aws_s3_bucket.default.*.arn) : ""}"
   description = "Bucket ARN"
 }
 
+output "access_logs_bucket_id" {
+  value       = "${module.access_logs_bucket.bucket_id}"
+  description = "Access logs bucket name (aka ID)"
+}
+
+output "access_logs_bucket_arn" {
+  value       = "${module.access_logs_bucket.bucket_arn}"
+  description = "Access logs bucket ARN"
+}
+
 output "prefix" {
-  value       = "${module.s3_bucket.prefix}"
+  value       = "${var.lifecycle_prefix}"
   description = "Prefix configured for lifecycle rules"
 }
+
+output "enabled" {
+  value       = "${var.enabled}"
+  description = "Is module enabled"
+}

variables.tf

@@ -1,3 +1,8 @@
+variable "enabled" {
+  description = "Set to `false` to prevent the module from creating any resources"
+  default     = "true"
+}
+
 variable "namespace" {
   description = "Namespace (e.g. `cp` or `cloudposse`)"
   type        = "string"
@@ -31,18 +36,18 @@ variable "tags" {
   description = "Additional tags (e.g. map(`BusinessUnit`,`XYZ`)"
 }
 
-variable "acl" {
-  type        = "string"
-  description = "Canned ACL to apply to the S3 bucket"
-  default     = "log-delivery-write"
-}
-
 variable "region" {
   type        = "string"
   default     = "us-east-1"
   description = "AWS Region for S3 bucket"
 }
 
+variable "acl" {
+  type        = "string"
+  description = "Canned ACL to apply to the S3 bucket"
+  default     = "log-delivery-write"
+}
+
 variable "force_destroy" {
   description = "A boolean that indicates the bucket can be destroyed even if it contains objects. These objects are not recoverable"
   default     = "false"
@@ -102,3 +107,69 @@ variable "kms_master_key_arn" {
   description = "The AWS KMS master key ARN used for the SSE-KMS encryption. This can only be used when you set the value of sse_algorithm as aws:kms. The default aws/s3 AWS KMS master key is used if this element is absent while the sse_algorithm is aws:kms"
   default     = ""
 }
+
+variable "access_logs_acl" {
+  type        = "string"
+  description = "Canned ACL to apply to the logs S3 bucket"
+  default     = "private"
+}
+
+variable "access_logs_force_destroy" {
+  description = "A boolean that indicates the logs bucket can be destroyed even if it contains objects. These objects are not recoverable"
+  default     = "false"
+}
+
+variable "access_logs_versioning_enabled" {
+  description = "A state of versioning logs bucket. Versioning is a means of keeping multiple variants of an object in the same bucket"
+  default     = "true"
+}
+
+variable "access_logs_lifecycle_rule_enabled" {
+  description = "Enable lifecycle events on this logs bucket"
+  default     = "true"
+}
+
+variable "access_logs_lifecycle_prefix" {
+  description = "Prefix filter for access logs s3 bucket. Used to manage object lifecycle events"
+  default     = ""
+}
+
+variable "access_logs_lifecycle_tags" {
+  description = "Tags filter for access logs s3 bucket. Used to manage object lifecycle events"
+  default     = {}
+}
+
+variable "access_logs_noncurrent_version_expiration_days" {
+  description = "Specifies when noncurrent object versions expire for access logs s3 bucket"
+  default     = "90"
+}
+
+variable "access_logs_noncurrent_version_transition_days" {
+  description = "Specifies when noncurrent object versions transition for access logs s3 bucket"
+  default     = "30"
+}
+
+variable "access_logs_standard_transition_days" {
+  description = "Number of days to persist in the standard storage tier before moving to the infrequent access tier for access logs s3 bucket"
+  default     = "30"
+}
+
+variable "access_logs_glacier_transition_days" {
+  description = "Number of days after which to move the data to the glacier storage tier for access logs s3 bucket"
+  default     = "60"
+}
+
+variable "access_logs_expiration_days" {
+  description = "Number of days after which to expunge the objects for access logs s3 bucket"
+  default     = "90"
+}
+
+variable "access_logs_sse_algorithm" {
+  description = "The server-side encryption algorithm to use. Valid values are AES256 and aws:kms for access logs s3 bucket"
+  default     = "AES256"
+}
+
+variable "access_logs_kms_master_key_arn" {
+  description = "The AWS KMS master key ARN used for the SSE-KMS encryption. This can only be used when you set the value of sse_algorithm as aws:kms. The default aws/s3 AWS KMS master key is used if this element is absent while the sse_algorithm is aws:kms for access logs s3 bucket"
+  default     = ""
+}