From f51ba78bc156a7bc346050c15610c90df31e96a0 Mon Sep 17 00:00:00 2001 From: milldr Date: Mon, 21 Oct 2024 14:52:36 -0400 Subject: [PATCH 01/28] Add decision guide for default storage class in EKS clusters --- .../decide-on-default-storage-class.mdx | 73 +++++++++++++++++++ 1 file changed, 73 insertions(+) create mode 100644 docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx diff --git a/docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx b/docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx new file mode 100644 index 000000000..983fc5a2b --- /dev/null +++ b/docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx @@ -0,0 +1,73 @@ +--- +title: "Decide on Default Storage Class for EKS Clusters" +sidebar_label: "Default Storage Class" +description: Determine the default storage class for Kubernetes EKS clusters +--- +import Intro from '@site/src/components/Intro'; +import KeyPoints from '@site/src/components/KeyPoints'; + + +When provisioning EKS (Kubernetes) clusters, selecting the appropriate default storage class is crucial for ensuring optimal performance, scalability, and cost-efficiency of persistent storage for workloads. + + +## Default Storage Class Options + +We need to decide between **Amazon EFS (Elastic File System)** and **Amazon EBS (Elastic Block Store)** as the default storage class for our EKS clusters. Historically, **EFS** has been the recommended option, but there is a growing preference for **EBS** for customers. + +### Amazon EFS + +**Amazon EFS** provides a scalable, fully managed elastic NFS file system for use with AWS Cloud services and on-premises resources. + +#### Pros: +- **Scalability:** Automatically scales storage capacity as needed without manual intervention. +- **Shared Access:** Allows multiple pods to access the same file system concurrently, facilitating shared storage scenarios. +- **Managed Service:** Fully managed by AWS, reducing operational overhead for maintenance and scaling. +- **NFS Protocol Support:** Compatible with applications requiring NFS (Network File System) access. + +#### Cons: +- **Performance:** Generally offers lower performance compared to EBS, especially for high IOPS (Input/Output Operations Per Second) requirements. +- **Cost:** Can be more expensive for certain usage patterns due to its pricing structure. +- **Latency:** Higher latency compared to EBS, which may impact performance-sensitive applications. + +### Amazon EBS + +**Amazon EBS** provides high-performance block storage volumes for use with Amazon EC2 instances, suitable for a wide range of workloads. + +#### Pros: +- **Performance:** Offers high IOPS and low latency, making it ideal for performance-critical applications. +- **Cost-Effective:** Potentially lower costs for specific storage types and usage scenarios. +- **Integration:** Well-integrated with Kubernetes through the EBS CSI (Container Storage Interface) driver, facilitating seamless provisioning and management. +- **Snapshot and Backup:** Supports snapshotting for data backup, recovery, and cloning. + +#### Cons: +- **Single-Attach Limitation:** EBS volumes are typically attached to a single node at a time, limiting shared access across multiple pods. +- **Scalability:** Requires manual provisioning and management of storage capacity, which can introduce operational complexity. +- **Availability Zones:** EBS volumes are confined to a single Availability Zone, which may affect high availability and disaster recovery strategies. + +## Decision + +**Both Amazon EFS and Amazon EBS are excellent choices for the default storage class in EKS clusters, depending on specific use cases and requirements.** + +- **Choose Amazon EFS** if your applications require: + - Shared access to the same file system across multiple pods. + - Automatic scaling of storage without manual intervention. + - Compatibility with NFS protocols. + +- **Choose Amazon EBS** if your applications demand: + - High IOPS and low latency for performance-critical workloads. + - Cost-effective storage solutions for specific usage patterns. + - Advanced features like snapshotting and seamless integration with Kubernetes. + +This approach allows teams to select the most appropriate storage option based on the nature of their workloads, performance requirements, and operational considerations. + +## Consequences + +- **Flexibility:** Offering both EFS and EBS as options provides the flexibility to choose the storage solution that best fits the application's needs. +- **Complexity:** Managing multiple storage classes may introduce additional complexity in provisioning and maintaining storage resources. +- **Documentation:** Clear guidelines and best practices will be needed to help teams decide which storage class to use for different scenarios. +- **Resource Allocation:** Teams may need to allocate resources for evaluating and monitoring both storage options to ensure optimal usage and performance. +- **Potential Training:** Developers and operations teams might require training to effectively utilize and manage both EFS and EBS within their Kubernetes deployments. + +## Recommendations + +Either Amazon EFS and Amazon EBS as excellent choices as the default storage class. Choose the appropriate storage class based on the specific requirements of your applications and workloads. From 39f52b326545895dc3844ec693b21fa6409d65c2 Mon Sep 17 00:00:00 2001 From: milldr Date: Mon, 21 Oct 2024 14:56:40 -0400 Subject: [PATCH 02/28] Update default storage class recommendation --- .../decide-on-default-storage-class.mdx | 13 +++++-------- 1 file changed, 5 insertions(+), 8 deletions(-) diff --git a/docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx b/docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx index 983fc5a2b..cf704a4a3 100644 --- a/docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx +++ b/docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx @@ -25,8 +25,8 @@ We need to decide between **Amazon EFS (Elastic File System)** and **Amazon EBS - **NFS Protocol Support:** Compatible with applications requiring NFS (Network File System) access. #### Cons: -- **Performance:** Generally offers lower performance compared to EBS, especially for high IOPS (Input/Output Operations Per Second) requirements. -- **Cost:** Can be more expensive for certain usage patterns due to its pricing structure. +- **Performance:** Generally offers lower performance compared to EBS, with throughput as low as 100 MB/s, which may not meet the demands of high-performance applications. +- **Cost:** Can be significantly more expensive than EBS, potentially up to 3x the price depending on usage patterns. - **Latency:** Higher latency compared to EBS, which may impact performance-sensitive applications. ### Amazon EBS @@ -41,10 +41,11 @@ We need to decide between **Amazon EFS (Elastic File System)** and **Amazon EBS #### Cons: - **Single-Attach Limitation:** EBS volumes are typically attached to a single node at a time, limiting shared access across multiple pods. +- **Data Sharing:** Data cannot be easily shared across multiple instances, requiring additional configurations or solutions for shared access. +- **Availability Zones:** EBS volumes are confined to a single Availability Zone, which may affect high availability and disaster recovery strategies. Kubernetes has implemented several mitigations for this limitation, and ECS likely has similar solutions. - **Scalability:** Requires manual provisioning and management of storage capacity, which can introduce operational complexity. -- **Availability Zones:** EBS volumes are confined to a single Availability Zone, which may affect high availability and disaster recovery strategies. -## Decision +## Recommendation **Both Amazon EFS and Amazon EBS are excellent choices for the default storage class in EKS clusters, depending on specific use cases and requirements.** @@ -67,7 +68,3 @@ This approach allows teams to select the most appropriate storage option based o - **Documentation:** Clear guidelines and best practices will be needed to help teams decide which storage class to use for different scenarios. - **Resource Allocation:** Teams may need to allocate resources for evaluating and monitoring both storage options to ensure optimal usage and performance. - **Potential Training:** Developers and operations teams might require training to effectively utilize and manage both EFS and EBS within their Kubernetes deployments. - -## Recommendations - -Either Amazon EFS and Amazon EBS as excellent choices as the default storage class. Choose the appropriate storage class based on the specific requirements of your applications and workloads. From 9d27ce8e4481a7f71c26e4b5cbf1912972809d8a Mon Sep 17 00:00:00 2001 From: milldr Date: Mon, 21 Oct 2024 15:41:58 -0400 Subject: [PATCH 03/28] Decide on default storage class for EKS clusters --- .../decide-on-default-storage-class.mdx | 25 +++++++++++++------ 1 file changed, 18 insertions(+), 7 deletions(-) diff --git a/docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx b/docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx index cf704a4a3..c48dfac87 100644 --- a/docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx +++ b/docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx @@ -14,38 +14,49 @@ When provisioning EKS (Kubernetes) clusters, selecting the appropriate default s We need to decide between **Amazon EFS (Elastic File System)** and **Amazon EBS (Elastic Block Store)** as the default storage class for our EKS clusters. Historically, **EFS** has been the recommended option, but there is a growing preference for **EBS** for customers. -### Amazon EFS +## Top Considerations + +1. **Low Performance:** + - **EFS:** Can be mitigated by paying for additional bandwidth, but has routinely caused outages due to throttling even with low-performance applications. Additionally, poor lock performance makes EFS completely unsuitable for high-performance applications like RDBMS. + +2. **Cost:** + - **EFS:** At least 3x the cost of EBS since the storage is backed by an EBS volume in each of the 3 Availability Zones (AZs). + +3. **Integration:** + - Both **EFS** and **EBS** are now fully integrated into Kubernetes with CSI (Container Storage Interface) controllers, facilitating seamless provisioning and management. + +## Amazon EFS **Amazon EFS** provides a scalable, fully managed elastic NFS file system for use with AWS Cloud services and on-premises resources. -#### Pros: +### Pros: - **Scalability:** Automatically scales storage capacity as needed without manual intervention. - **Shared Access:** Allows multiple pods to access the same file system concurrently, facilitating shared storage scenarios. - **Managed Service:** Fully managed by AWS, reducing operational overhead for maintenance and scaling. - **NFS Protocol Support:** Compatible with applications requiring NFS (Network File System) access. -#### Cons: +### Cons: - **Performance:** Generally offers lower performance compared to EBS, with throughput as low as 100 MB/s, which may not meet the demands of high-performance applications. - **Cost:** Can be significantly more expensive than EBS, potentially up to 3x the price depending on usage patterns. - **Latency:** Higher latency compared to EBS, which may impact performance-sensitive applications. -### Amazon EBS +## Amazon EBS **Amazon EBS** provides high-performance block storage volumes for use with Amazon EC2 instances, suitable for a wide range of workloads. -#### Pros: +### Pros: - **Performance:** Offers high IOPS and low latency, making it ideal for performance-critical applications. - **Cost-Effective:** Potentially lower costs for specific storage types and usage scenarios. - **Integration:** Well-integrated with Kubernetes through the EBS CSI (Container Storage Interface) driver, facilitating seamless provisioning and management. - **Snapshot and Backup:** Supports snapshotting for data backup, recovery, and cloning. -#### Cons: +### Cons: - **Single-Attach Limitation:** EBS volumes are typically attached to a single node at a time, limiting shared access across multiple pods. - **Data Sharing:** Data cannot be easily shared across multiple instances, requiring additional configurations or solutions for shared access. - **Availability Zones:** EBS volumes are confined to a single Availability Zone, which may affect high availability and disaster recovery strategies. Kubernetes has implemented several mitigations for this limitation, and ECS likely has similar solutions. - **Scalability:** Requires manual provisioning and management of storage capacity, which can introduce operational complexity. -## Recommendation +## Decision **Both Amazon EFS and Amazon EBS are excellent choices for the default storage class in EKS clusters, depending on specific use cases and requirements.** From 039d2c41d46f4c43dfa72ebc318d4d404951e692 Mon Sep 17 00:00:00 2001 From: Dan Miller Date: Mon, 21 Oct 2024 16:02:54 -0400 Subject: [PATCH 04/28] Update docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx --- .../eks/design-decisions/decide-on-default-storage-class.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx b/docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx index c48dfac87..a1e6d5679 100644 --- a/docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx +++ b/docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx @@ -56,7 +56,7 @@ We need to decide between **Amazon EFS (Elastic File System)** and **Amazon EBS - **Availability Zones:** EBS volumes are confined to a single Availability Zone, which may affect high availability and disaster recovery strategies. Kubernetes has implemented several mitigations for this limitation, and ECS likely has similar solutions. - **Scalability:** Requires manual provisioning and management of storage capacity, which can introduce operational complexity. -## Decision +## Recommendation **Both Amazon EFS and Amazon EBS are excellent choices for the default storage class in EKS clusters, depending on specific use cases and requirements.** From 9aa638a18816010d8bdbac44890f8e26faff7bd4 Mon Sep 17 00:00:00 2001 From: Dan Miller Date: Mon, 21 Oct 2024 16:21:11 -0400 Subject: [PATCH 05/28] Update docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx --- .../design-decisions/decide-on-default-storage-class.mdx | 7 ------- 1 file changed, 7 deletions(-) diff --git a/docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx b/docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx index a1e6d5679..f0d9fe89a 100644 --- a/docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx +++ b/docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx @@ -72,10 +72,3 @@ We need to decide between **Amazon EFS (Elastic File System)** and **Amazon EBS This approach allows teams to select the most appropriate storage option based on the nature of their workloads, performance requirements, and operational considerations. -## Consequences - -- **Flexibility:** Offering both EFS and EBS as options provides the flexibility to choose the storage solution that best fits the application's needs. -- **Complexity:** Managing multiple storage classes may introduce additional complexity in provisioning and maintaining storage resources. -- **Documentation:** Clear guidelines and best practices will be needed to help teams decide which storage class to use for different scenarios. -- **Resource Allocation:** Teams may need to allocate resources for evaluating and monitoring both storage options to ensure optimal usage and performance. -- **Potential Training:** Developers and operations teams might require training to effectively utilize and manage both EFS and EBS within their Kubernetes deployments. From 91c080a06251fa4a4c59addebc5ff9a59f41f383 Mon Sep 17 00:00:00 2001 From: milldr Date: Mon, 21 Oct 2024 16:57:46 -0400 Subject: [PATCH 06/28] Decide on default storage class recommendations --- .../decide-on-default-storage-class.mdx | 48 ++++++------------- 1 file changed, 14 insertions(+), 34 deletions(-) diff --git a/docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx b/docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx index c48dfac87..ac1f8daed 100644 --- a/docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx +++ b/docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx @@ -12,18 +12,13 @@ When provisioning EKS (Kubernetes) clusters, selecting the appropriate default s ## Default Storage Class Options -We need to decide between **Amazon EFS (Elastic File System)** and **Amazon EBS (Elastic Block Store)** as the default storage class for our EKS clusters. Historically, **EFS** has been the recommended option, but there is a growing preference for **EBS** for customers. +We need to decide between **Amazon EFS (Elastic File System)** and **Amazon EBS (Elastic Block Store)** as the default storage class for our EKS clusters. -## Top Considerations - -1. **Low Performance:** - - **EFS:** Can be mitigated by paying for additional bandwidth, but has routinely caused outages due to throttling even with low-performance applications. Additionally, poor lock performance makes EFS completely unsuitable for high-performance applications like RDBMS. - -2. **Cost:** - - **EFS:** At least 3x the cost of EBS since the storage is backed by an EBS volume in each of the 3 Availability Zones (AZs). - -3. **Integration:** - - Both **EFS** and **EBS** are now fully integrated into Kubernetes with CSI (Container Storage Interface) controllers, facilitating seamless provisioning and management. + +- Availability Zone Lock-in: **EBS:** EBS volumes are locked to a single Availability Zone, which may affect high availability and disaster recovery strategies. This is the primary argument against EBS. However, sharing data across Pods is an uncommon use case. +- Performance: **EFS:** Can be mitigated by paying for additional bandwidth, but has routinely caused outages due to throttling even with low-performance applications. Additionally, poor lock performance makes EFS completely unsuitable for high-performance applications like RDBMS. +- Cost: **EFS:** Significantly more expensive than EBS, at least 3x the price per GB and potentially more depending on performance demands, although there may be some savings from not having to reserve size for future growth. + ## Amazon EFS @@ -36,8 +31,8 @@ We need to decide between **Amazon EFS (Elastic File System)** and **Amazon EBS - **NFS Protocol Support:** Compatible with applications requiring NFS (Network File System) access. ### Cons: -- **Performance:** Generally offers lower performance compared to EBS, with throughput as low as 100 MB/s, which may not meet the demands of high-performance applications. -- **Cost:** Can be significantly more expensive than EBS, potentially up to 3x the price depending on usage patterns. +- **Performance:** Generally offers lower performance compared to EBS, with throughput as low as 100 MB/s, which may not meet the demands of even modestly demanding applications. +- **Cost:** Significantly more expensive than EBS, at least 3x the price per GB and potentially more depending on performance demands, although there may be some savings from not having to reserve size for future growth. - **Latency:** Higher latency compared to EBS, which may impact performance-sensitive applications. ## Amazon EBS @@ -51,31 +46,16 @@ We need to decide between **Amazon EFS (Elastic File System)** and **Amazon EBS - **Snapshot and Backup:** Supports snapshotting for data backup, recovery, and cloning. ### Cons: -- **Single-Attach Limitation:** EBS volumes are typically attached to a single node at a time, limiting shared access across multiple pods. +- **Single-Attach Limitation:** EBS volumes may only be attached to a single node at a time, limiting shared access across multiple pods. - **Data Sharing:** Data cannot be easily shared across multiple instances, requiring additional configurations or solutions for shared access. - **Availability Zones:** EBS volumes are confined to a single Availability Zone, which may affect high availability and disaster recovery strategies. Kubernetes has implemented several mitigations for this limitation, and ECS likely has similar solutions. - **Scalability:** Requires manual provisioning and management of storage capacity, which can introduce operational complexity. -## Decision - -**Both Amazon EFS and Amazon EBS are excellent choices for the default storage class in EKS clusters, depending on specific use cases and requirements.** - -- **Choose Amazon EFS** if your applications require: - - Shared access to the same file system across multiple pods. - - Automatic scaling of storage without manual intervention. - - Compatibility with NFS protocols. - -- **Choose Amazon EBS** if your applications demand: - - High IOPS and low latency for performance-critical workloads. - - Cost-effective storage solutions for specific usage patterns. - - Advanced features like snapshotting and seamless integration with Kubernetes. +## Recommendations -This approach allows teams to select the most appropriate storage option based on the nature of their workloads, performance requirements, and operational considerations. +Use **Amazon EBS** unless there is a good reason not to. -## Consequences +For example, use Amazon EFS when: -- **Flexibility:** Offering both EFS and EBS as options provides the flexibility to choose the storage solution that best fits the application's needs. -- **Complexity:** Managing multiple storage classes may introduce additional complexity in provisioning and maintaining storage resources. -- **Documentation:** Clear guidelines and best practices will be needed to help teams decide which storage class to use for different scenarios. -- **Resource Allocation:** Teams may need to allocate resources for evaluating and monitoring both storage options to ensure optimal usage and performance. -- **Potential Training:** Developers and operations teams might require training to effectively utilize and manage both EFS and EBS within their Kubernetes deployments. +- **Multiple Pods need read/write access to the same data at the same time.** +- **Persistent data needs to be available in multiple Availability Zones and the underlying application does not support replication.** From aa634088d263a6237ca3f684658057dc7ea3ef28 Mon Sep 17 00:00:00 2001 From: milldr Date: Tue, 22 Oct 2024 09:42:03 -0400 Subject: [PATCH 07/28] Update storage class comparison in EKS design decisions --- .../design-decisions/decide-on-default-storage-class.mdx | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx b/docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx index c31c5d031..64130c7b9 100644 --- a/docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx +++ b/docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx @@ -15,9 +15,9 @@ When provisioning EKS (Kubernetes) clusters, selecting the appropriate default s We need to decide between **Amazon EFS (Elastic File System)** and **Amazon EBS (Elastic Block Store)** as the default storage class for our EKS clusters. -- Availability Zone Lock-in: **EBS:** EBS volumes are locked to a single Availability Zone, which may affect high availability and disaster recovery strategies. This is the primary argument against EBS. However, sharing data across Pods is an uncommon use case. -- Performance: **EFS:** Can be mitigated by paying for additional bandwidth, but has routinely caused outages due to throttling even with low-performance applications. Additionally, poor lock performance makes EFS completely unsuitable for high-performance applications like RDBMS. -- Cost: **EFS:** Significantly more expensive than EBS, at least 3x the price per GB and potentially more depending on performance demands, although there may be some savings from not having to reserve size for future growth. +- Availability Zone Lock-in: EBS volumes are locked to a single Availability Zone, which may affect high availability and disaster recovery strategies. This is the primary argument against EBS. Sharing data across Pods is an uncommon use case. +- Performance: EFS generally offers lower performance when compared to EBS. This can be mitigated by paying for additional bandwidth but has routinely caused outages due to throttling even with low-performance applications. Additionally, poor lock performance makes EFS completely unsuitable for high-performance applications like RDBMS. +- Cost: EFS is significantly more expensive than EBS, at least 3x the price per GB and potentially more depending on performance demands, although there may be some savings from not having to reserve size for future growth. ## Amazon EFS From 594a83133cb250b13f82dc808ab961ab0cef50e7 Mon Sep 17 00:00:00 2001 From: milldr Date: Tue, 22 Oct 2024 09:43:16 -0400 Subject: [PATCH 08/28] Update default storage class decision key points --- .../eks/design-decisions/decide-on-default-storage-class.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx b/docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx index 64130c7b9..1abaede21 100644 --- a/docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx +++ b/docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx @@ -14,7 +14,7 @@ When provisioning EKS (Kubernetes) clusters, selecting the appropriate default s We need to decide between **Amazon EFS (Elastic File System)** and **Amazon EBS (Elastic Block Store)** as the default storage class for our EKS clusters. - + - Availability Zone Lock-in: EBS volumes are locked to a single Availability Zone, which may affect high availability and disaster recovery strategies. This is the primary argument against EBS. Sharing data across Pods is an uncommon use case. - Performance: EFS generally offers lower performance when compared to EBS. This can be mitigated by paying for additional bandwidth but has routinely caused outages due to throttling even with low-performance applications. Additionally, poor lock performance makes EFS completely unsuitable for high-performance applications like RDBMS. - Cost: EFS is significantly more expensive than EBS, at least 3x the price per GB and potentially more depending on performance demands, although there may be some savings from not having to reserve size for future growth. From 7cc39df42448f6ae3cbc1d7a17af607484ea3e58 Mon Sep 17 00:00:00 2001 From: Dan Miller Date: Tue, 22 Oct 2024 17:24:35 -0400 Subject: [PATCH 09/28] Update docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx Co-authored-by: Erik Osterman (CEO @ Cloud Posse) --- .../eks/design-decisions/decide-on-default-storage-class.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx b/docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx index 1abaede21..c17f8ffbf 100644 --- a/docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx +++ b/docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx @@ -15,7 +15,7 @@ When provisioning EKS (Kubernetes) clusters, selecting the appropriate default s We need to decide between **Amazon EFS (Elastic File System)** and **Amazon EBS (Elastic Block Store)** as the default storage class for our EKS clusters. -- Availability Zone Lock-in: EBS volumes are locked to a single Availability Zone, which may affect high availability and disaster recovery strategies. This is the primary argument against EBS. Sharing data across Pods is an uncommon use case. +- Availability Zone Lock-in: EBS volumes are locked to a single Availability Zone, which may affect high availability and disaster recovery strategies. This is the primary argument against EBS. Sharing data across Pods is an uncommon use case, where EFS is a more suitable option. - Performance: EFS generally offers lower performance when compared to EBS. This can be mitigated by paying for additional bandwidth but has routinely caused outages due to throttling even with low-performance applications. Additionally, poor lock performance makes EFS completely unsuitable for high-performance applications like RDBMS. - Cost: EFS is significantly more expensive than EBS, at least 3x the price per GB and potentially more depending on performance demands, although there may be some savings from not having to reserve size for future growth. From 309f0d0802280403e1138ec048903dd176cdd0a0 Mon Sep 17 00:00:00 2001 From: Dan Miller Date: Tue, 22 Oct 2024 17:27:42 -0400 Subject: [PATCH 10/28] Apply suggestions from code review Co-authored-by: Erik Osterman (CEO @ Cloud Posse) --- .../eks/design-decisions/decide-on-default-storage-class.mdx | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx b/docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx index c17f8ffbf..885e0860d 100644 --- a/docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx +++ b/docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx @@ -15,9 +15,10 @@ When provisioning EKS (Kubernetes) clusters, selecting the appropriate default s We need to decide between **Amazon EFS (Elastic File System)** and **Amazon EBS (Elastic Block Store)** as the default storage class for our EKS clusters. -- Availability Zone Lock-in: EBS volumes are locked to a single Availability Zone, which may affect high availability and disaster recovery strategies. This is the primary argument against EBS. Sharing data across Pods is an uncommon use case, where EFS is a more suitable option. +- Availability Zone Lock-in: EBS volumes are restricted to a single Availability Zone, which can impact high availability and disaster recovery strategies. This is a key drawback of using EBS. If you need a Pod to recover across multiple AZs, EFS is a more suitable option, though it comes at a higher cost. - Performance: EFS generally offers lower performance when compared to EBS. This can be mitigated by paying for additional bandwidth but has routinely caused outages due to throttling even with low-performance applications. Additionally, poor lock performance makes EFS completely unsuitable for high-performance applications like RDBMS. - Cost: EFS is significantly more expensive than EBS, at least 3x the price per GB and potentially more depending on performance demands, although there may be some savings from not having to reserve size for future growth. +- Concurrent Access: EBS volumes can only be attached to one instance at a time within the same Availability Zone, making them unsuitable for scenarios that require concurrent access from multiple instances. In contrast, EFS allows multiple instances or Pods to access the same file system concurrently, which is useful for distributed applications or workloads requiring shared storage across multiple nodes. ## Amazon EFS From 89a33134e5bb22719194526b391a1bfe7fdbb215 Mon Sep 17 00:00:00 2001 From: Dan Miller Date: Tue, 22 Oct 2024 18:01:51 -0400 Subject: [PATCH 11/28] Update docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx Co-authored-by: Erik Osterman (CEO @ Cloud Posse) --- .../eks/design-decisions/decide-on-default-storage-class.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx b/docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx index 885e0860d..fa29767dd 100644 --- a/docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx +++ b/docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx @@ -23,7 +23,7 @@ We need to decide between **Amazon EFS (Elastic File System)** and **Amazon EBS ## Amazon EFS -**Amazon EFS** provides a scalable, fully managed elastic NFS file system for use with AWS Cloud services and on-premises resources. +**Amazon EFS** provides a scalable, fully managed, elastic file system with NFS compatibility, designed for use with AWS Cloud services and on-premises resources. ### Pros: - **Scalability:** Automatically scales storage capacity as needed without manual intervention. From 30850e217edc8a1f00d89e7661b6a2fdcb151d5c Mon Sep 17 00:00:00 2001 From: Dan Miller Date: Thu, 24 Oct 2024 15:56:17 -0400 Subject: [PATCH 12/28] Update docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx Co-authored-by: Erik Osterman (CEO @ Cloud Posse) --- .../eks/design-decisions/decide-on-default-storage-class.mdx | 1 - 1 file changed, 1 deletion(-) diff --git a/docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx b/docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx index fa29767dd..fa8c55acb 100644 --- a/docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx +++ b/docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx @@ -29,7 +29,6 @@ We need to decide between **Amazon EFS (Elastic File System)** and **Amazon EBS - **Scalability:** Automatically scales storage capacity as needed without manual intervention. - **Shared Access:** Allows multiple pods to access the same file system concurrently, facilitating shared storage scenarios. - **Managed Service:** Fully managed by AWS, reducing operational overhead for maintenance and scaling. -- **NFS Protocol Support:** Compatible with applications requiring NFS (Network File System) access. ### Cons: - **Performance:** Generally offers lower performance compared to EBS, with throughput as low as 100 MB/s, which may not meet the demands of even modestly demanding applications. From c8beb629b4cc167cdf812e289526b12adee649ca Mon Sep 17 00:00:00 2001 From: Dan Miller Date: Thu, 24 Oct 2024 15:56:31 -0400 Subject: [PATCH 13/28] Update docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx Co-authored-by: Erik Osterman (CEO @ Cloud Posse) --- .../design-decisions/decide-on-default-storage-class.mdx | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx b/docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx index fa8c55acb..6458de107 100644 --- a/docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx +++ b/docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx @@ -31,9 +31,9 @@ We need to decide between **Amazon EFS (Elastic File System)** and **Amazon EBS - **Managed Service:** Fully managed by AWS, reducing operational overhead for maintenance and scaling. ### Cons: -- **Performance:** Generally offers lower performance compared to EBS, with throughput as low as 100 MB/s, which may not meet the demands of even modestly demanding applications. -- **Cost:** Significantly more expensive than EBS, at least 3x the price per GB and potentially more depending on performance demands, although there may be some savings from not having to reserve size for future growth. -- **Latency:** Higher latency compared to EBS, which may impact performance-sensitive applications. +- **Lower Performance:** Generally offers lower performance compared to EBS, with throughput as low as 100 MB/s, which may not meet the demands of even modestly demanding applications. +- **Higher Cost:** Significantly more expensive than EBS, at least 3x the price per GB and potentially more depending on performance demands, although there may be some savings from not having to reserve size for future growth. +- **Higher Latency:** Higher latency compared to EBS, which may impact performance-sensitive applications. ## Amazon EBS From ab98f7aba33f3a76e066744dbad0b3ba4353f8f2 Mon Sep 17 00:00:00 2001 From: Dan Miller Date: Thu, 24 Oct 2024 15:57:25 -0400 Subject: [PATCH 14/28] Update docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx Co-authored-by: Erik Osterman (CEO @ Cloud Posse) --- .../eks/design-decisions/decide-on-default-storage-class.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx b/docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx index 6458de107..10d5b936b 100644 --- a/docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx +++ b/docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx @@ -26,7 +26,7 @@ We need to decide between **Amazon EFS (Elastic File System)** and **Amazon EBS **Amazon EFS** provides a scalable, fully managed, elastic file system with NFS compatibility, designed for use with AWS Cloud services and on-premises resources. ### Pros: -- **Scalability:** Automatically scales storage capacity as needed without manual intervention. +- **Unlimited Disk Space:** Automatically scales storage capacity as needed without manual intervention. - **Shared Access:** Allows multiple pods to access the same file system concurrently, facilitating shared storage scenarios. - **Managed Service:** Fully managed by AWS, reducing operational overhead for maintenance and scaling. From 9759c5dae23d639ae7439beb7703314576ab1ae6 Mon Sep 17 00:00:00 2001 From: Dan Miller Date: Thu, 24 Oct 2024 15:57:44 -0400 Subject: [PATCH 15/28] Update docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx Co-authored-by: Erik Osterman (CEO @ Cloud Posse) --- .../eks/design-decisions/decide-on-default-storage-class.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx b/docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx index 10d5b936b..bc4a621dd 100644 --- a/docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx +++ b/docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx @@ -40,7 +40,7 @@ We need to decide between **Amazon EFS (Elastic File System)** and **Amazon EBS **Amazon EBS** provides high-performance block storage volumes for use with Amazon EC2 instances, suitable for a wide range of workloads. ### Pros: -- **Performance:** Offers high IOPS and low latency, making it ideal for performance-critical applications. +- **Higher Performance:** Offers high IOPS and low latency, making it ideal for performance-critical applications. - **Cost-Effective:** Potentially lower costs for specific storage types and usage scenarios. - **Integration:** Well-integrated with Kubernetes through the EBS CSI (Container Storage Interface) driver, facilitating seamless provisioning and management. - **Snapshot and Backup:** Supports snapshotting for data backup, recovery, and cloning. From f84ecd0fb208449a868f2b69b1e262b6efd83fbc Mon Sep 17 00:00:00 2001 From: Dan Miller Date: Thu, 24 Oct 2024 15:57:55 -0400 Subject: [PATCH 16/28] Update docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx Co-authored-by: Erik Osterman (CEO @ Cloud Posse) --- .../eks/design-decisions/decide-on-default-storage-class.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx b/docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx index bc4a621dd..adef2d03f 100644 --- a/docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx +++ b/docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx @@ -42,7 +42,7 @@ We need to decide between **Amazon EFS (Elastic File System)** and **Amazon EBS ### Pros: - **Higher Performance:** Offers high IOPS and low latency, making it ideal for performance-critical applications. - **Cost-Effective:** Potentially lower costs for specific storage types and usage scenarios. -- **Integration:** Well-integrated with Kubernetes through the EBS CSI (Container Storage Interface) driver, facilitating seamless provisioning and management. +- **Native EKS Integration:** Well-integrated with Kubernetes through the EBS CSI (Container Storage Interface) driver, facilitating seamless provisioning and management. - **Snapshot and Backup:** Supports snapshotting for data backup, recovery, and cloning. ### Cons: From e7a5b3a92c3d953eceb14aa843705afd4bc839a0 Mon Sep 17 00:00:00 2001 From: Dan Miller Date: Thu, 24 Oct 2024 15:58:16 -0400 Subject: [PATCH 17/28] Update docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx Co-authored-by: Erik Osterman (CEO @ Cloud Posse) --- .../eks/design-decisions/decide-on-default-storage-class.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx b/docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx index adef2d03f..3158bb5be 100644 --- a/docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx +++ b/docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx @@ -43,7 +43,7 @@ We need to decide between **Amazon EFS (Elastic File System)** and **Amazon EBS - **Higher Performance:** Offers high IOPS and low latency, making it ideal for performance-critical applications. - **Cost-Effective:** Potentially lower costs for specific storage types and usage scenarios. - **Native EKS Integration:** Well-integrated with Kubernetes through the EBS CSI (Container Storage Interface) driver, facilitating seamless provisioning and management. -- **Snapshot and Backup:** Supports snapshotting for data backup, recovery, and cloning. +- **Supports Snapshot and Backup:** Supports snapshotting for data backup, recovery, and cloning. ### Cons: - **Single-Attach Limitation:** EBS volumes may only be attached to a single node at a time, limiting shared access across multiple pods. From 13aabdb30e0f124913974f4c13916c8654c2a581 Mon Sep 17 00:00:00 2001 From: Dan Miller Date: Thu, 24 Oct 2024 16:01:27 -0400 Subject: [PATCH 18/28] Update docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx Co-authored-by: Erik Osterman (CEO @ Cloud Posse) --- .../eks/design-decisions/decide-on-default-storage-class.mdx | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx b/docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx index 3158bb5be..2b79a5497 100644 --- a/docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx +++ b/docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx @@ -34,6 +34,7 @@ We need to decide between **Amazon EFS (Elastic File System)** and **Amazon EBS - **Lower Performance:** Generally offers lower performance compared to EBS, with throughput as low as 100 MB/s, which may not meet the demands of even modestly demanding applications. - **Higher Cost:** Significantly more expensive than EBS, at least 3x the price per GB and potentially more depending on performance demands, although there may be some savings from not having to reserve size for future growth. - **Higher Latency:** Higher latency compared to EBS, which may impact performance-sensitive applications. +- **No Native Backup Support:** EFS lacks a built-in, straightforward backup and recovery solution for EKS. Kubernetes-native tools don’t support EFS backups directly, requiring the use of alternatives like AWS Backup. Recovery, however, can be non-trivial and may involve complex configurations to restore data effectively. ## Amazon EBS From 53c6383ba20c29999fc9b8917356d8a3080e6cd8 Mon Sep 17 00:00:00 2001 From: Dan Miller Date: Thu, 24 Oct 2024 16:03:07 -0400 Subject: [PATCH 19/28] Update docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx Co-authored-by: Erik Osterman (CEO @ Cloud Posse) --- .../eks/design-decisions/decide-on-default-storage-class.mdx | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx b/docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx index 2b79a5497..6cc0f8818 100644 --- a/docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx +++ b/docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx @@ -29,6 +29,7 @@ We need to decide between **Amazon EFS (Elastic File System)** and **Amazon EBS - **Unlimited Disk Space:** Automatically scales storage capacity as needed without manual intervention. - **Shared Access:** Allows multiple pods to access the same file system concurrently, facilitating shared storage scenarios. - **Managed Service:** Fully managed by AWS, reducing operational overhead for maintenance and scaling. +- **Availability Zone Failover**: For workloads that require failover across multiple Availability Zones, EFS is a more suitable option. It provides multi-AZ durability, ensuring that Pods can recover and access persistent storage seamlessly across different AZs. ### Cons: - **Lower Performance:** Generally offers lower performance compared to EBS, with throughput as low as 100 MB/s, which may not meet the demands of even modestly demanding applications. From 591f3c658ef97278bc4b20bef64ad9c6618840a2 Mon Sep 17 00:00:00 2001 From: Dan Miller Date: Thu, 24 Oct 2024 16:11:16 -0400 Subject: [PATCH 20/28] Apply suggestions from code review Co-authored-by: Erik Osterman (CEO @ Cloud Posse) --- .../eks/design-decisions/decide-on-default-storage-class.mdx | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx b/docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx index 6cc0f8818..5b86d0e42 100644 --- a/docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx +++ b/docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx @@ -50,8 +50,8 @@ We need to decide between **Amazon EFS (Elastic File System)** and **Amazon EBS ### Cons: - **Single-Attach Limitation:** EBS volumes may only be attached to a single node at a time, limiting shared access across multiple pods. - **Data Sharing:** Data cannot be easily shared across multiple instances, requiring additional configurations or solutions for shared access. -- **Availability Zones:** EBS volumes are confined to a single Availability Zone, which may affect high availability and disaster recovery strategies. Kubernetes has implemented several mitigations for this limitation, and ECS likely has similar solutions. -- **Scalability:** Requires manual provisioning and management of storage capacity, which can introduce operational complexity. +- **Availability Zones:** EBS volumes are confined to a single Availability Zone, limiting high availability and disaster recovery across zones. This limitation can be mitigated by configuring a `StatefulSet` with replicas spread across multiple AZs. However, for workloads using EBS-backed Persistent Volume Claims (PVCs), failover to a different AZ requires manual intervention, including provisioning a new volume in the target zone, as EBS volumes cannot be moved between zones. +- **Non-Elastic Storage:** EBS volumes can be manually resized, but this process is not fully automated in EKS. After resizing an EBS volume, additional manual steps are required to expand the associated Persistent Volume (PV) and Persistent Volume Claim (PVC). This introduces operational complexity, especially for workloads with dynamic storage needs, as EBS lacks automatic scaling like EFS. ## Recommendation From 34bde7a51a7f24bf76a13adf0a4dd0863d423408 Mon Sep 17 00:00:00 2001 From: Dan Miller Date: Thu, 24 Oct 2024 16:15:21 -0400 Subject: [PATCH 21/28] Update docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx Co-authored-by: Erik Osterman (CEO @ Cloud Posse) --- .../decide-on-default-storage-class.mdx | 18 ++++++++++++++---- 1 file changed, 14 insertions(+), 4 deletions(-) diff --git a/docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx b/docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx index 5b86d0e42..c4a0ebe60 100644 --- a/docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx +++ b/docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx @@ -55,9 +55,19 @@ We need to decide between **Amazon EFS (Elastic File System)** and **Amazon EBS ## Recommendation -Use **Amazon EBS** unless there is a good reason not to. +Use **Amazon EBS** as the primary storage option when: -Use **Amazon EFS** when: + • High performance, low-latency storage is required for workloads confined to a single Availability Zone. + • The workload doesn’t require shared access across multiple Pods. + • You need cost-effective storage with support for snapshots and backups. + • Manual resizing of volumes is acceptable for capacity management, recognizing that failover across AZs requires manual intervention and provisioning. -- Multiple Pods need read/write access to the same data at the same time. -- Persistent data needs to be available in multiple Availability Zones and the underlying application does not support replication. +Consider **Amazon EFS** when: + + • Multiple Pods need concurrent read/write access to shared data across nodes. + • Workloads must persist data across multiple Availability Zones for high availability, and the application does not support native replication. + • Elastic, automatically scaling storage is necessary to avoid manual provisioning, especially for workloads with unpredictable growth. + • You are willing to trade off higher costs and lower performance for multi-AZ durability and easier management of shared storage. + +> ![IMPORTANT] +> EFS should never be used as backend storage for performance-sensitive applications like databases, due to its high latency and poor performance under heavy load. From 10031d1eb0e9f79896941573f92d2de58c5c2869 Mon Sep 17 00:00:00 2001 From: milldr Date: Thu, 24 Oct 2024 16:16:22 -0400 Subject: [PATCH 22/28] Update storage class decision considerations --- .../decide-on-default-storage-class.mdx | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx b/docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx index c4a0ebe60..1595d5432 100644 --- a/docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx +++ b/docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx @@ -57,17 +57,17 @@ We need to decide between **Amazon EFS (Elastic File System)** and **Amazon EBS Use **Amazon EBS** as the primary storage option when: - • High performance, low-latency storage is required for workloads confined to a single Availability Zone. - • The workload doesn’t require shared access across multiple Pods. - • You need cost-effective storage with support for snapshots and backups. - • Manual resizing of volumes is acceptable for capacity management, recognizing that failover across AZs requires manual intervention and provisioning. +- High performance, low-latency storage is required for workloads confined to a single Availability Zone. +- The workload doesn’t require shared access across multiple Pods. +- You need cost-effective storage with support for snapshots and backups. +- Manual resizing of volumes is acceptable for capacity management, recognizing that failover across AZs requires manual intervention and provisioning. Consider **Amazon EFS** when: - • Multiple Pods need concurrent read/write access to shared data across nodes. - • Workloads must persist data across multiple Availability Zones for high availability, and the application does not support native replication. - • Elastic, automatically scaling storage is necessary to avoid manual provisioning, especially for workloads with unpredictable growth. - • You are willing to trade off higher costs and lower performance for multi-AZ durability and easier management of shared storage. +- Multiple Pods need concurrent read/write access to shared data across nodes. +- Workloads must persist data across multiple Availability Zones for high availability, and the application does not support native replication. +- Elastic, automatically scaling storage is necessary to avoid manual provisioning, especially for workloads with unpredictable growth. +- You are willing to trade off higher costs and lower performance for multi-AZ durability and easier management of shared storage. > ![IMPORTANT] > EFS should never be used as backend storage for performance-sensitive applications like databases, due to its high latency and poor performance under heavy load. From a963698b74d2c18d920fa97731927621ba4c772c Mon Sep 17 00:00:00 2001 From: milldr Date: Thu, 24 Oct 2024 16:17:31 -0400 Subject: [PATCH 23/28] Decide on default storage class guidelines updated --- .../decide-on-default-storage-class.mdx | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx b/docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx index 1595d5432..8d4f3b725 100644 --- a/docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx +++ b/docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx @@ -57,17 +57,17 @@ We need to decide between **Amazon EFS (Elastic File System)** and **Amazon EBS Use **Amazon EBS** as the primary storage option when: -- High performance, low-latency storage is required for workloads confined to a single Availability Zone. -- The workload doesn’t require shared access across multiple Pods. -- You need cost-effective storage with support for snapshots and backups. -- Manual resizing of volumes is acceptable for capacity management, recognizing that failover across AZs requires manual intervention and provisioning. +- High performance, low-latency storage is required for workloads confined to a single Availability Zone. +- The workload doesn’t require shared access across multiple Pods. +- You need cost-effective storage with support for snapshots and backups. +- Manual resizing of volumes is acceptable for capacity management, recognizing that failover across AZs requires manual intervention and provisioning. Consider **Amazon EFS** when: -- Multiple Pods need concurrent read/write access to shared data across nodes. -- Workloads must persist data across multiple Availability Zones for high availability, and the application does not support native replication. -- Elastic, automatically scaling storage is necessary to avoid manual provisioning, especially for workloads with unpredictable growth. -- You are willing to trade off higher costs and lower performance for multi-AZ durability and easier management of shared storage. +- Multiple Pods need concurrent read/write access to shared data across nodes. +- Workloads must persist data across multiple Availability Zones for high availability, and the application does not support native replication. +- Elastic, automatically scaling storage is necessary to avoid manual provisioning, especially for workloads with unpredictable growth. +- You are willing to trade off higher costs and lower performance for multi-AZ durability and easier management of shared storage. > ![IMPORTANT] > EFS should never be used as backend storage for performance-sensitive applications like databases, due to its high latency and poor performance under heavy load. From 8061a934d83f59b3ca40687c8ffab0438b23ff04 Mon Sep 17 00:00:00 2001 From: milldr Date: Thu, 24 Oct 2024 16:18:53 -0400 Subject: [PATCH 24/28] Update EFS usage warning in storage class decision --- .../eks/design-decisions/decide-on-default-storage-class.mdx | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx b/docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx index 8d4f3b725..27d026a65 100644 --- a/docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx +++ b/docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx @@ -69,5 +69,6 @@ Consider **Amazon EFS** when: - Elastic, automatically scaling storage is necessary to avoid manual provisioning, especially for workloads with unpredictable growth. - You are willing to trade off higher costs and lower performance for multi-AZ durability and easier management of shared storage. -> ![IMPORTANT] -> EFS should never be used as backend storage for performance-sensitive applications like databases, due to its high latency and poor performance under heavy load. +:::important +EFS should never be used as backend storage for performance-sensitive applications like databases, due to its high latency and poor performance under heavy load. +::: From 04316d47c00ebb56fa3fffe11a673ef3dc1b7f36 Mon Sep 17 00:00:00 2001 From: Dan Miller Date: Thu, 24 Oct 2024 16:19:42 -0400 Subject: [PATCH 25/28] Update docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx Co-authored-by: Erik Osterman (CEO @ Cloud Posse) --- .../eks/design-decisions/decide-on-default-storage-class.mdx | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx b/docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx index 27d026a65..8deb32ba0 100644 --- a/docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx +++ b/docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx @@ -7,9 +7,10 @@ import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; -When provisioning EKS (Kubernetes) clusters, selecting the appropriate default storage class is crucial for ensuring optimal performance, scalability, and cost-efficiency of persistent storage for workloads. +When provisioning EKS (Kubernetes) clusters, there is no one-size-fits-all default storage class. The right choice depends on your specific workload requirements, including performance, scalability, and cost-efficiency. Storage classes are not mutually exclusive, and in many cases, the best solution might involve using a combination of options to meet different needs. + ## Default Storage Class Options We need to decide between **Amazon EFS (Elastic File System)** and **Amazon EBS (Elastic Block Store)** as the default storage class for our EKS clusters. From cce7fea5c917e864d7f151516a2083c35a8d5354 Mon Sep 17 00:00:00 2001 From: Dan Miller Date: Thu, 24 Oct 2024 16:21:06 -0400 Subject: [PATCH 26/28] Update docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx Co-authored-by: Erik Osterman (CEO @ Cloud Posse) --- .../eks/design-decisions/decide-on-default-storage-class.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx b/docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx index 8deb32ba0..8b726260c 100644 --- a/docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx +++ b/docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx @@ -7,7 +7,7 @@ import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; -When provisioning EKS (Kubernetes) clusters, there is no one-size-fits-all default storage class. The right choice depends on your specific workload requirements, including performance, scalability, and cost-efficiency. Storage classes are not mutually exclusive, and in many cases, the best solution might involve using a combination of options to meet different needs. +When provisioning EKS (Kubernetes) clusters, there is no one-size-fits-all recommendation for the default storage class. The right choice depends on your workload’s specific requirements, such as performance, scalability, and cost-efficiency. While only one storage class can be set as the default, storage classes are not mutually exclusive, and the best solution may often involve using a combination of classes to meet various needs. From cb845e9af8d9e17cb9c7f045779a8bda05f73186 Mon Sep 17 00:00:00 2001 From: Dan Miller Date: Thu, 24 Oct 2024 16:22:20 -0400 Subject: [PATCH 27/28] Update docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx Co-authored-by: Erik Osterman (CEO @ Cloud Posse) --- .../design-decisions/decide-on-default-storage-class.mdx | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx b/docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx index 8b726260c..6ef156360 100644 --- a/docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx +++ b/docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx @@ -16,10 +16,10 @@ When provisioning EKS (Kubernetes) clusters, there is no one-size-fits-all recom We need to decide between **Amazon EFS (Elastic File System)** and **Amazon EBS (Elastic Block Store)** as the default storage class for our EKS clusters. -- Availability Zone Lock-in: EBS volumes are restricted to a single Availability Zone, which can impact high availability and disaster recovery strategies. This is a key drawback of using EBS. If you need a Pod to recover across multiple AZs, EFS is a more suitable option, though it comes at a higher cost. -- Performance: EFS generally offers lower performance when compared to EBS. This can be mitigated by paying for additional bandwidth but has routinely caused outages due to throttling even with low-performance applications. Additionally, poor lock performance makes EFS completely unsuitable for high-performance applications like RDBMS. -- Cost: EFS is significantly more expensive than EBS, at least 3x the price per GB and potentially more depending on performance demands, although there may be some savings from not having to reserve size for future growth. -- Concurrent Access: EBS volumes can only be attached to one instance at a time within the same Availability Zone, making them unsuitable for scenarios that require concurrent access from multiple instances. In contrast, EFS allows multiple instances or Pods to access the same file system concurrently, which is useful for distributed applications or workloads requiring shared storage across multiple nodes. +- **Availability Zone Lock-in:** EBS volumes are restricted to a single Availability Zone, which can impact high availability and disaster recovery strategies. This is a key drawback of using EBS. If you need a Pod to recover across multiple AZs, EFS is a more suitable option, though it comes at a higher cost. +- **Performance:** EFS generally offers lower performance when compared to EBS. This can be mitigated by paying for additional bandwidth but has routinely caused outages due to throttling even with low-performance applications. Additionally, poor lock performance makes EFS completely unsuitable for high-performance applications like RDBMS. +- **Cost:** EFS is significantly more expensive than EBS, at least 3x the price per GB and potentially more depending on performance demands, although there may be some savings from not having to reserve size for future growth. +- **Concurrent Access:** EBS volumes can only be attached to one instance at a time within the same Availability Zone, making them unsuitable for scenarios that require concurrent access from multiple instances. In contrast, EFS allows multiple instances or Pods to access the same file system concurrently, which is useful for distributed applications or workloads requiring shared storage across multiple nodes. ## Amazon EFS From b44934f37eb78bacf5b74a62e447c14ee95b208c Mon Sep 17 00:00:00 2001 From: Dan Miller Date: Thu, 24 Oct 2024 16:27:35 -0400 Subject: [PATCH 28/28] Apply suggestions from code review Co-authored-by: Erik Osterman (CEO @ Cloud Posse) --- .../eks/design-decisions/decide-on-default-storage-class.mdx | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx b/docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx index 6ef156360..1ad28651d 100644 --- a/docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx +++ b/docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx @@ -10,6 +10,7 @@ import KeyPoints from '@site/src/components/KeyPoints'; When provisioning EKS (Kubernetes) clusters, there is no one-size-fits-all recommendation for the default storage class. The right choice depends on your workload’s specific requirements, such as performance, scalability, and cost-efficiency. While only one storage class can be set as the default, storage classes are not mutually exclusive, and the best solution may often involve using a combination of classes to meet various needs. +A `StorageClass` in Kubernetes defines the type of storage (e.g., EBS, EFS, etc.) and its parameters (e.g., performance, replication) for dynamically provisioning Persistent Volumes. The default `StorageClass` is automatically used when a `PersistentVolumeClaim` (PVC) is created without specifying a specific storage class, but its configuration varies depending on the cluster setup and cloud provider. Storage classes available on AWS differ from other clouds. ## Default Storage Class Options @@ -49,8 +50,7 @@ We need to decide between **Amazon EFS (Elastic File System)** and **Amazon EBS - **Supports Snapshot and Backup:** Supports snapshotting for data backup, recovery, and cloning. ### Cons: -- **Single-Attach Limitation:** EBS volumes may only be attached to a single node at a time, limiting shared access across multiple pods. -- **Data Sharing:** Data cannot be easily shared across multiple instances, requiring additional configurations or solutions for shared access. +- **Single-Attach Limitation:** EBS volumes can only be attached to a single node at a time, limiting shared access across multiple Pods or instances. Additional configurations or alternative storage solutions are required for scenarios needing concurrent access. - **Availability Zones:** EBS volumes are confined to a single Availability Zone, limiting high availability and disaster recovery across zones. This limitation can be mitigated by configuring a `StatefulSet` with replicas spread across multiple AZs. However, for workloads using EBS-backed Persistent Volume Claims (PVCs), failover to a different AZ requires manual intervention, including provisioning a new volume in the target zone, as EBS volumes cannot be moved between zones. - **Non-Elastic Storage:** EBS volumes can be manually resized, but this process is not fully automated in EKS. After resizing an EBS volume, additional manual steps are required to expand the associated Persistent Volume (PV) and Persistent Volume Claim (PVC). This introduces operational complexity, especially for workloads with dynamic storage needs, as EBS lacks automatic scaling like EFS.