Terraform IBM Observability agents module

This module deploys the following observability agents to an IBM Cloud Red Hat OpenShift Container Platform or Kubernetes cluster:

• Logs agent
• Monitoring agent

Overview

• terraform-ibm-observability-agents
• Submodules
  • logs-agent
• Examples
  • Monitoring agent + Cloud Logs agent on Kubernetes using CSE ingress endpoint with an apikey
  • Monitoring agent + Cloud Logs agent on OCP using VPE ingress endpoint with a Trusted Profile
• Contributing

terraform-ibm-observability-agents

Usage

# ############################################################################
# Init cluster config for helm
# ############################################################################

data "ibm_container_cluster_config" "cluster_config" {
  # update this value with the Id of the cluster where these agents will be provisioned
  cluster_name_id = "cluster_id"
}

# ############################################################################
# Config providers
# ############################################################################

provider "ibm" {
  # update this value with your IBM Cloud API key value
  ibmcloud_api_key = "XXXXXXXXXXXXXXXXX"
}

provider "helm" {
  kubernetes {
    host                   = data.ibm_container_cluster_config.cluster_config.host
    token                  = data.ibm_container_cluster_config.cluster_config.token
    cluster_ca_certificate = data.ibm_container_cluster_config.cluster_config.ca_certificate
  }
  # IBM Cloud credentials are required to authenticate to the helm repo
  registry {
    url = "oci://icr.io/ibm/observe/logs-agent-helm"
    username = "iamapikey"
    password = "XXXXXXXXXXXXXXXXX" # replace with an IBM cloud apikey
  }
}

# ############################################################################
# Install observability agents
# ############################################################################

module "observability_agents" {
  source                           = "terraform-ibm-modules/observability-agents/ibm"
  version                          = "X.X.X" # Replace "X.X.X" with a release version to lock into a specific release
  is_vpc_cluster                   = true # Change to false if target cluster is running on classic infrastructure
  cluster_id                       = "cluster id" # update this with your cluster id where the agents will be installed
  cluster_resource_group_id        = "resource group id" # update this with the Id of your IBM Cloud resource group
  cloud_monitoring_access_key      = "XXXXXXXX"
  cloud_monitoring_instance_region = "us-south"
  # Logs Agent variables
  logs_agent_trusted_profile  = "XXXXXXXX"
  cloud_logs_ingress_endpoint = "<cloud-logs-instance-guid>.ingress.us-south.logs.cloud.ibm.com"
  cloud_logs_ingress_port     = 443
}

Required IAM access policies

You need the following permissions to run this module.

• Service
  • Resource group only
    • Viewer access on the specific resource group
  • Kubernetes service
    • Viewer platform access
    • Manager service access

Requirements

Name	Version
terraform	>= 1.3.0
helm	>= 2.15.0, <3.0.0
ibm	>= 1.69.2, <2.0.0

Modules

Name	Source	Version
logs_agent	./modules/logs-agent	n/a

Resources

Name	Type
helm_release.cloud_monitoring_agent	resource
ibm_container_cluster.cluster	data source
ibm_container_cluster_config.cluster_config	data source
ibm_container_vpc_cluster.cluster	data source

Inputs

Name	Description	Type	Default	Required
cloud_logs_ingress_endpoint	The host for IBM Cloud Logs ingestion. Ensure you use the ingress endpoint. See https://cloud.ibm.com/docs/cloud-logs?topic=cloud-logs-endpoints_ingress.	string	null	no
cloud_logs_ingress_port	The target port for the IBM Cloud Logs ingestion endpoint. The port must be 443 if you connect by using a VPE gateway, or port 3443 when you connect by using CSEs.	number	3443	no
cloud_monitoring_access_key	Access key used by the IBM Cloud Monitoring agent to communicate with the instance	string	null	no
cloud_monitoring_add_cluster_name	If true, configure the cloud monitoring agent to attach a tag containing the cluster name to all metric data.	bool	true	no
cloud_monitoring_agent_name	Cloud Monitoring agent name. Used for naming all kubernetes and helm resources on the cluster.	string	"sysdig-agent"	no
cloud_monitoring_agent_namespace	Namespace where to deploy the Cloud Monitoring agent. Default value is 'ibm-observe'	string	"ibm-observe"	no
cloud_monitoring_agent_tags	List of tags to associate to all matrics that the agent collects. NOTE: Use the 'cloud_monitoring_add_cluster_name' variable to add the cluster name as a tag.	list(string)	[]	no
cloud_monitoring_agent_tolerations	List of tolerations to apply to Cloud Monitoring agent.	list(object({ key = optional(string) operator = optional(string) value = optional(string) effect = optional(string) tolerationSeconds = optional(number) }))	[ { "operator": "Exists" }, { "effect": "NoSchedule", "key": "node-role.kubernetes.io/master", "operator": "Exists" } ]	no
cloud_monitoring_container_filter	To filter custom containers, specify the Cloud Monitoring containers to include or to exclude. See https://cloud.ibm.com/docs/monitoring?topic=monitoring-change_kube_agent#change_kube_agent_filter_data.	list(object({ type = string parameter = string name = string }))	[]	no
cloud_monitoring_enabled	Deploy IBM Cloud Monitoring agent	bool	true	no
cloud_monitoring_endpoint_type	Specify the IBM Cloud Monitoring instance endpoint type (public or private) to use. Used to construct the ingestion endpoint.	string	"private"	no
cloud_monitoring_instance_region	The IBM Cloud Monitoring instance region. Used to construct the ingestion endpoint.	string	null	no
cloud_monitoring_metrics_filter	To filter custom metrics, specify the Cloud Monitoring metrics to include or to exclude. See https://cloud.ibm.com/docs/monitoring?topic=monitoring-change_kube_agent#change_kube_agent_inc_exc_metrics.	list(object({ type = string name = string }))	[]	no
cloud_monitoring_secret_name	The name of the secret which will store the access key.	string	"sysdig-agent"	no
cluster_config_endpoint_type	Specify which type of endpoint to use for for cluster config access: 'default', 'private', 'vpe', 'link'. 'default' value will use the default endpoint of the cluster.	string	"default"	no
cluster_id	The ID of the cluster you wish to deploy the agents in	string	n/a	yes
cluster_resource_group_id	The Resource Group ID of the cluster	string	n/a	yes
is_vpc_cluster	Specify true if the target cluster for the observability agents is a VPC cluster, false if it is a classic cluster.	bool	true	no
logs_agent_additional_log_source_paths	The list of additional log sources. By default, the Logs agent collects logs from a single source at /var/log/containers/*.log.	list(string)	[]	no
logs_agent_additional_metadata	The list of additional metadata fields to add to the routed logs.	list(object({ key = optional(string) value = optional(string) }))	[]	no
logs_agent_enable_scc	Whether to enable creation of Security Context Constraints in Openshift. When installing on an OpenShift cluster, this setting is mandatory to configure permissions for pods within your cluster.	bool	true	no
logs_agent_enabled	Whether to deploy the Logs agent.	bool	true	no
logs_agent_exclude_log_source_paths	The list of log sources to exclude. Specify the paths that the Logs agent ignores.	list(string)	[]	no
logs_agent_iam_api_key	The IBM Cloud API key for the Logs agent to authenticate and communicate with the IBM Cloud Logs. It is required if logs_agent_iam_mode is set to IAMAPIKey.	string	null	no
logs_agent_iam_environment	IAM authentication Environment: Production or PrivateProduction or Staging or PrivateStaging. Production specifies the public endpoint & PrivateProduction specifies the private endpoint.	string	"PrivateProduction"	no
logs_agent_iam_mode	IAM authentication mode: TrustedProfile or IAMAPIKey.	string	"TrustedProfile"	no
logs_agent_log_source_namespaces	The list of namespaces from which logs should be forwarded by agent. If namespaces are not listed, logs from all namespaces will be sent.	list(string)	[]	no
logs_agent_name	The name of the Logs agent. The name is used in all Kubernetes and Helm resources in the cluster.	string	"logs-agent"	no
logs_agent_namespace	The namespace where the Logs agent is deployed. The default value is ibm-observe.	string	"ibm-observe"	no
logs_agent_selected_log_source_paths	The list of specific log sources paths. Logs will only be collected from the specified log source paths. If no paths are specified, it will send logs from /var/log/containers.	list(string)	[]	no
logs_agent_tolerations	List of tolerations to apply to Logs agent. The default value means a pod will run on every node.	list(object({ key = optional(string) operator = optional(string) value = optional(string) effect = optional(string) tolerationSeconds = optional(number) }))	[ { "operator": "Exists" } ]	no
logs_agent_trusted_profile	The IBM Cloud trusted profile ID. Used only when logs_agent_iam_mode is set to TrustedProfile. The trusted profile must have an IBM Cloud Logs Sender role.	string	null	no
wait_till	To avoid long wait times when you run your Terraform code, you can specify the stage when you want Terraform to mark the cluster resource creation as completed. Depending on what stage you choose, the cluster creation might not be fully completed and continues to run in the background. However, your Terraform code can continue to run without waiting for the cluster to be fully created. Supported args are MasterNodeReady, OneWorkerNodeReady, IngressReady and Normal	string	"Normal"	no
wait_till_timeout	Timeout for wait_till in minutes.	number	90	no

Outputs

No outputs.

Contributing

You can report issues and request features for this module in GitHub issues in the module repo. See Report an issue or request a feature.

To set up your local development environment, see Local development setup in the project documentation.