feat: Decide on Default Storage Class for EKS Clusters

docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx

@@ -0,0 +1,63 @@
+---
+title: "Decide on Default Storage Class for EKS Clusters"
+sidebar_label: "Default Storage Class"
+description: Determine the default storage class for Kubernetes EKS clusters
+---
+import Intro from '@site/src/components/Intro';
+import KeyPoints from '@site/src/components/KeyPoints';
+
+<Intro>
+When provisioning EKS (Kubernetes) clusters, selecting the appropriate default storage class is crucial for ensuring optimal performance, scalability, and cost-efficiency of persistent storage for workloads.
+</Intro>
+
+## Default Storage Class Options
+
+We need to decide between **Amazon EFS (Elastic File System)** and **Amazon EBS (Elastic Block Store)** as the default storage class for our EKS clusters.
+
+<KeyPoints title="Top Considerations">
+- Availability Zone Lock-in: EBS volumes are restricted to a single Availability Zone, which can impact high availability and disaster recovery strategies. This is a key drawback of using EBS. If you need a Pod to recover across multiple AZs, EFS is a more suitable option, though it comes at a higher cost.
+- Performance: EFS generally offers lower performance when compared to EBS. This can be mitigated by paying for additional bandwidth but has routinely caused outages due to throttling even with low-performance applications. Additionally, poor lock performance makes EFS completely unsuitable for high-performance applications like RDBMS.
+- Cost: EFS is significantly more expensive than EBS, at least 3x the price per GB and potentially more depending on performance demands, although there may be some savings from not having to reserve size for future growth.
+- Concurrent Access: EBS volumes can only be attached to one instance at a time within the same Availability Zone, making them unsuitable for scenarios that require concurrent access from multiple instances. In contrast, EFS allows multiple instances or Pods to access the same file system concurrently, which is useful for distributed applications or workloads requiring shared storage across multiple nodes.
+</KeyPoints>
+
+## Amazon EFS
+
+**Amazon EFS** provides a scalable, fully managed, elastic file system with NFS compatibility, designed for use with AWS Cloud services and on-premises resources.
+
+### Pros:
+- **Unlimited Disk Space:** Automatically scales storage capacity as needed without manual intervention.
+- **Shared Access:** Allows multiple pods to access the same file system concurrently, facilitating shared storage scenarios.
+- **Managed Service:** Fully managed by AWS, reducing operational overhead for maintenance and scaling.
+- **Availability Zone Failover**: For workloads that require failover across multiple Availability Zones, EFS is a more suitable option. It provides multi-AZ durability, ensuring that Pods can recover and access persistent storage seamlessly across different AZs.
+
+### Cons:
+- **Lower Performance:** Generally offers lower performance compared to EBS, with throughput as low as 100 MB/s, which may not meet the demands of even modestly demanding applications.
+- **Higher Cost:** Significantly more expensive than EBS, at least 3x the price per GB and potentially more depending on performance demands, although there may be some savings from not having to reserve size for future growth.
+- **Higher Latency:** Higher latency compared to EBS, which may impact performance-sensitive applications.
+- **No Native Backup Support:** EFS lacks a built-in, straightforward backup and recovery solution for EKS. Kubernetes-native tools don’t support EFS backups directly, requiring the use of alternatives like AWS Backup. Recovery, however, can be non-trivial and may involve complex configurations to restore data effectively.
+
+## Amazon EBS
+
+**Amazon EBS** provides high-performance block storage volumes for use with Amazon EC2 instances, suitable for a wide range of workloads.
+
+### Pros:
+- **Higher Performance:** Offers high IOPS and low latency, making it ideal for performance-critical applications.
+- **Cost-Effective:** Potentially lower costs for specific storage types and usage scenarios.
+- **Native EKS Integration:** Well-integrated with Kubernetes through the EBS CSI (Container Storage Interface) driver, facilitating seamless provisioning and management.
+- **Supports Snapshot and Backup:** Supports snapshotting for data backup, recovery, and cloning.
+
+### Cons:
+- **Single-Attach Limitation:** EBS volumes may only be attached to a single node at a time, limiting shared access across multiple pods.
+- **Data Sharing:** Data cannot be easily shared across multiple instances, requiring additional configurations or solutions for shared access.
+- **Availability Zones:** EBS volumes are confined to a single Availability Zone, which may affect high availability and disaster recovery strategies. Kubernetes has implemented several mitigations for this limitation, and ECS likely has similar solutions.
+- **Scalability:** Requires manual provisioning and management of storage capacity, which can introduce operational complexity.
+
+## Recommendation
+
+Use **Amazon EBS** unless there is a good reason not to.
+
+Use **Amazon EFS** when:
+
+- Multiple Pods need read/write access to the same data at the same time.
+- Persistent data needs to be available in multiple Availability Zones and the underlying application does not support replication.