terraform-kubernetes-opensearch

SquareOps Technologies Your DevOps Partner for Accelerating cloud journey.

This module deploys OpenSearch. With this module, take the advantage of OpenSearch installation in your AWS account. OpenSearch is a scalable, flexible, and extensible open-source software suite for search, analytics, and observability applications licensed under Apache 2.0. Powered by Apache Lucene and driven by the OpenSearch Project community, OpenSearch offers a vendor-agnostic toolset you can use to build secure, high-performance, cost-efficient applications. Use OpenSearch as an end-to-end solution or connect it with your preferred open-source tools or partner projects.OpenSearch is a distributed, community-driven, Apache 2.0-licensed, 100% open-source search and analytics suite used for a broad set of use cases like real-time application monitoring, log analytics, and website search. OpenSearch provides a highly scalable system for providing fast access and response to large volumes of data with an integrated visualization tool, OpenSearch Dashboards, that makes it easy for users to explore their data. OpenSearch is powered by the Apache Lucene search library, and it supports a number of search and analytics capabilities such as k-nearest neighbors (KNN) search, SQL, Anomaly Detection, Machine Learning Commons, Trace Analytics, full-text search, and more.

Important Notes:

This module is compatible with all the terraform versions which is great news for users deploying the module on AWS running account. Reviewed the module's documentation, meet specific configuration requirements, and test thoroughly after deployment to ensure everything works as expected.

Usage Example

locals {
  region                         = ""
  custom_master_password         = "H2222@sbkQTX"
  custom_master_password_enabled = true
  additional_tags = {
    Owner      = "organization_name"
    Expires    = "Never"
    Department = "Engineering"
  }
}

module "aws_opensearch" {
  source         = "squareops/opensearch/aws"
  version        = "1.0.1"
  opensearch_enabled = true
  domain_name    = "skaf"
  engine_version = "2.7"
  cluster_config = [{
    instance_type            = "t3.medium.search"
    instance_count           = 1
#warm nodes depends on dedicated master type nodes. 
    dedicated_master_enabled = false
    dedicated_master_type    = "r6g.large.search"
    dedicated_master_count   = 3
    warm_enabled             = false
    zone_awareness_enabled   = false
    availability_zone_count  = 1
  }]

  custom_master_password_enabled    = local.custom_master_password_enabled
  custom_master_password            = local.custom_master_password
  advanced_security_options_enabled = true
  advanced_security_options = [{
    master_user_options = {
      master_user_name     = "admin"
      master_user_password = local.custom_master_password_enabled ? local.custom_master_password : ""
    }
  }]

  domain_endpoint_options = [{
    enforce_https            = true
    custom_endpoint_enabled  = false
  }]

  ebs_enabled = true
  ebs_options = [{
    volume_size = 10
    volume_type = "gp2"
    iops        = 3000
  }]

  #if you will not pass kms_key_id it will pick default key
  encrypt_at_rest = [{
    enabled = true
    #kms_key_id = "arn:aws:kms:us-east-2:271251951598:key/f1e2f1a9-686a-4e31-a5c8-38623e045e27"
  }]

  cloudwatch_log_enabled = false
  log_publishing_options = {
    es_application_logs = {
      enabled                          = true
      log_publishing_options_retention = 30
      cloudwatch_log_group_name        = "os_application_logs_dev"
    }
    audit_logs = {
      enabled                          = false
      log_publishing_options_retention = 30
      cloudwatch_log_group_name        = "os_audit_logs"
    }
  }

  node_to_node_encryption = [
    {
      enabled = true
    }
  ]

  snapshot_options = [{
    automated_snapshot_start_hour = 23
  }]
}

Requirements

Name	Version
aws	4.67.0

Providers

Name	Version
aws	4.67.0
random	n/a

Modules

No modules.

Resources

Name	Type
aws_cloudwatch_log_group.es_cloudwatch_log_group	resource
aws_cloudwatch_log_resource_policy.es_aws_cloudwatch_log_resource_policy	resource
aws_iam_service_linked_role.es	resource
aws_opensearch_domain.es_domain	resource
random_password.master_password	resource
aws_caller_identity.current	data source
aws_iam_policy_document.access_policy	data source
aws_kms_key.aws_es	data source
aws_region.current	data source

Inputs

Name	Description	Type	Default	Required
advanced_options	Key-value string pairs to specify advanced configuration options. Note that the values for these configuration options must be strings (wrapped in quotes) or they may be wrong and cause a perpetual diff, causing Terraform to want to recreate your Elasticsearch domain on every apply	map(string)	{}	no
advanced_security_options	Options for fine-grained access control	any	{}	no
advanced_security_options_create_random_master_password	Whether to create random master password for Elasticsearch master user	bool	false	no
advanced_security_options_enabled	Whether advanced security is enabled (Forces new resource)	bool	false	no
advanced_security_options_internal_user_database_enabled	Whether the internal user database is enabled. If not set, defaults to false by the AWS API.	bool	false	no
advanced_security_options_master_user_arn	ARN for the master user. Only specify if internal_user_database_enabled is not set or set to false)	string	null	no
advanced_security_options_master_user_password	The master user's password, which is stored in the Amazon Elasticsearch Service domain's internal database. Only specify if internal_user_database_enabled is set to true.	string	"Admin@2233"	no
advanced_security_options_master_user_username	The master user's username, which is stored in the Amazon Elasticsearch Service domain's internal database. Only specify if internal_user_database_enabled is set to true.	string	"admin"	no
advanced_security_options_random_master_password_length	Length of random master password to create	number	8	no
cloudwatch_log_enabled	Change to false to avoid deploying any Cloudwatch Logs resources	bool	true	no
cluster_config	Cluster configuration of the domain	any	{}	no
cluster_config_availability_zone_count	Number of Availability Zones for the domain to use with	number	1	no
cluster_config_cold_storage_options_enabled	Indicates whether to enable cold storage for an Elasticsearch domain	bool	false	no
cluster_config_dedicated_master_count	Number of dedicated master nodes in the cluster	number	1	no
cluster_config_dedicated_master_enabled	Indicates whether dedicated master nodes are enabled for the cluster	bool	false	no
cluster_config_dedicated_master_type	Instance type of the dedicated master nodes in the cluster	string	"t3.medium.search"	no
cluster_config_instance_count	Number of instances in the cluster	number	1	no
cluster_config_instance_type	Instance type of data nodes in the cluster	string	"t3.medium.search"	no
cluster_config_warm_count	The number of warm nodes in the cluster	number	null	no
cluster_config_warm_enabled	Indicates whether to enable warm storage	bool	false	no
cluster_config_warm_type	The instance type for the Elasticsearch cluster's warm nodes	string	null	no
cluster_config_zone_awareness_enabled	Indicates whether zone awareness is enabled. To enable awareness with three Availability Zones	bool	false	no
cognito_options	Options for Amazon Cognito Authentication for Kibana	any	{}	no
cognito_options_enabled	Specifies whether Amazon Cognito authentication with Kibana is enabled or not	bool	false	no
cognito_options_identity_pool_id	ID of the Cognito Identity Pool to use	string	""	no
cognito_options_role_arn	ARN of the IAM role that has the AmazonESCognitoAccess policy attached	string	""	no
cognito_options_user_pool_id	ID of the Cognito User Pool to use	string	""	no
create_a_record	create route 53 record	bool	false	no
create_service_link_role	Create service link role for AWS Elasticsearch Service	bool	true	no
domain_endpoint_options	Domain endpoint HTTP(S) related options.	any	{}	no
domain_endpoint_options_custom_endpoint	Fully qualified domain for your custom endpoint	string	null	no
domain_endpoint_options_custom_endpoint_certificate_arn	ACM certificate ARN for your custom endpoint	string	null	no
domain_endpoint_options_custom_endpoint_enabled	Whether to enable custom endpoint for the Elasticsearch domain	bool	false	no
domain_endpoint_options_enforce_https	Whether or not to require HTTPS	bool	true	no
domain_endpoint_options_tls_security_policy	The name of the TLS security policy that needs to be applied to the HTTPS endpoint. Valid values: Policy-Min-TLS-1-0-2019-07 and Policy-Min-TLS-1-2-2019-07	string	"Policy-Min-TLS-1-2-2019-07"	no
domain_name	Name of the domain	string	n/a	yes
ebs_enabled	Whether EBS volumes are attached to data nodes in the domain	bool	true	no
ebs_options	EBS related options, may be required based on chosen instance size	any	{}	no
ebs_options_iops	The baseline input/output (I/O) performance of EBS volumes attached to data nodes. Applicable only for the Provisioned IOPS EBS volume type	number	0	no
ebs_options_volume_size	The size of EBS volumes attached to data nodes (in GB). Required if ebs_enabled is set to true	number	10	no
ebs_options_volume_type	The type of EBS volumes attached to data nodes	string	"gp2"	no
enabled	Change to false to avoid deploying any AWS ElasticSearch resources	bool	true	no
encrypt_at_rest	Encrypt at rest options. Only available for certain instance types	any	{}	no
encrypt_at_rest_enabled	Whether to enable encryption at rest	bool	true	no
encrypt_at_rest_kms_key_id	The KMS key id to encrypt the Elasticsearch domain with. If not specified then it defaults to using the aws/es service KMS key	string	"alias/aws/es"	no
engine_version	The version of OpenSearch to deploy.	string	"2.7"	no
log_publishing_options	Options for publishing slow logs to CloudWatch Logs	any	{}	no
log_publishing_options_retention	Retention in days for the created Cloudwatch log group	number	60	no
node_to_node_encryption	Node-to-node encryption options	any	{}	no
node_to_node_encryption_enabled	Whether to enable node-to-node encryption	bool	true	no
snapshot_options	Snapshot related options	any	{}	no
snapshot_options_automated_snapshot_start_hour	Hour during which the service takes an automated daily snapshot of the indices in the domain	number	0	no
vpc_options	VPC related options, see below. Adding or removing this configuration forces a new resource	any	{}	no
vpc_options_security_group_ids	List of VPC Security Group IDs to be applied to the Elasticsearch domain endpoints. If omitted, the default Security Group for the VPC will be used	list(any)	[]	no
vpc_options_subnet_ids	List of VPC Subnet IDs for the Elasticsearch domain endpoints to be created in	list(any)	[]	no

Outputs

Name	Description
arn	Amazon Resource Name (ARN) of the domain
domain_id	Unique identifier for the domain
endpoint	Domain-specific endpoint used to submit index, search, and data upload requests
kibana_endpoint	Domain-specific endpoint for kibana without https scheme
master_password	Master password
master_username	Master username
vpc_options_availability_zones	If the domain was created inside a VPC, the names of the availability zones the configured subnet_ids were created inside
vpc_options_vpc_id	If the domain was created inside a VPC, the ID of the VPC

Contribution & Issue Reporting

To report an issue with a project:

1. Check the repository's issue tracker on GitHub
2. Search to see if the issue has already been reported
3. If you can't find an answer to your question in the documentation or issue tracker, you can ask a question by creating a new issue. Be sure to provide enough context and details so others can understand your problem.

License

Apache License, Version 2.0, January 2004 (http://www.apache.org/licenses/).

Support Us

To support a GitHub project by liking it, you can follow these steps:

1. Visit the repository: Navigate to the GitHub repository.

2. Click the "Star" button: On the repository page, you'll see a "Star" button in the upper right corner. Clicking on it will star the repository, indicating your support for the project.

3. Optionally, you can also leave a comment on the repository or open an issue to give feedback or suggest changes.

Starring a repository on GitHub is a simple way to show your support and appreciation for the project. It also helps to increase the visibility of the project and make it more discoverable to others.

Who we are

We believe that the key to success in the digital age is the ability to deliver value quickly and reliably. That’s why we offer a comprehensive range of DevOps & Cloud services designed to help your organization optimize its systems & Processes for speed and agility.

1. We are an AWS Advanced consulting partner which reflects our deep expertise in AWS Cloud and helping 100+ clients over the last 5 years.
2. Expertise in Kubernetes and overall container solution helps companies expedite their journey by 10X.
3. Infrastructure Automation is a key component to the success of our Clients and our Expertise helps deliver the same in the shortest time.
4. DevSecOps as a service to implement security within the overall DevOps process and helping companies deploy securely and at speed.
5. Platform engineering which supports scalable,Cost efficient infrastructure that supports rapid development, testing, and deployment.
6. 24*7 SRE service to help you Monitor the state of your infrastructure and eradicate any issue within the SLA.

We provide support on all of our projects, no matter how small or large they may be.

To find more information about our company, visit squareops.com, follow us on Linkedin, or fill out a job application. You can also checkout our Case-studies or Blogs to understand more about our solutions. If you have any questions or would like assistance with your cloud strategy and implementation, please don't hesitate to contact us.