Merge pull request #71 from Cameronsplaze/feature/multiple-similar-games

[feature/multiple-similar-games] Change Examples (i.e `Minecraft-example.yaml` to `Minecraft.java.example.yaml`)

.github/pull_request_template.md

@@ -25,6 +25,6 @@
 <!--- If you're unsure about any of these, don't hesitate to ask. We're here to help! -->
 - [ ] My code follows the code style of this project.
 - [ ] My change requires a change to the documentation.
-- [ ] I have updated the documentation accordingly.
-- [ ] I have added tests to cover my changes.
+  - [ ] I have updated the documentation accordingly.
 - [ ] All new and existing tests passed.
+  - [ ] I have added tests to cover my changes.

.github/workflows/README.md

@@ -1,20 +1,20 @@
 # GitHub Actions
 
-GitHub Actions Docs/References:
+**GitHub Actions Docs/References:**
 
 - The GOOD docs on [writting workflows](https://docs.github.com/en/actions/writing-workflows/workflow-syntax-for-github-actions) that I can never find when I need.
 - Docs on [context](https://docs.github.com/en/actions/writing-workflows/choosing-what-your-workflow-does/accessing-contextual-information-about-workflow-runs)'s (i.e ${{ github.* }}, and other built-in variables)
 
-## Dependabot auto updates
+## Dependabot Auto-Updates
 
 There's a lot of tweaks I had to do to get this working. Will come back to at some point and re-write this readme. To include:
 
 - Disabled PR code having to be up to date with base. If two dependabot PRs are open at the same time, the second one will fail to merge.
   - We can technically get around this by setting the max-pr to 1 in each config block, then having each block run a different day of the week. This feels really hacky though...
 - Added [CODEOWNERS](../CODEOWNERS) file, but had to disable on the files dependabot touches. There's no way to add apps to CODEOWNERS. (More details in the CODEOWNERS file itself.).
-- Added a list of required actions to main. At first didn't want to maintain the list, but you get a nice clear label on all the actions, so you can easily see if you add an action later and forget to add it to the list. (I tried having the action run a `gh pr` command to wait, but it ended up waiting on itself...).
+- Added a list of required actions to Branch Protections Rules for `main`. When dependabot waits to auto-merge, it'll only wait for **required** actions to finish.
+  - They also can't contain the `on.<trigger>.paths` key. If they do, and the paths isn't in the PR, you won't be able to merge it. (It just freezes saying "Expected — Waiting for status to be reported")
 - In [dependabot.yml](../dependabot.yml), added a `groups` section. This makes all updates happen under a single PR. That way if 3 updates get created, you don't merge all 3 at the same time and try to do 3 deployment updates at once.
-- Any actions you want to block dependabot updates from merging, MUST be **required workflows** in the repo settings. They also can't contain the `on.<trigger>.paths` key. If they do, and the paths isn't in the PR, you won't be able to merge it. (It just freezes saying "Expected — Waiting for status to be reported")
 
 ## main-pipeline-cdk.yml
 
@@ -30,13 +30,18 @@ I made this different than `cdk-synth`. For synth, it should run on EVERY config
 1) Add a new line to the **Github Variable** `DEPLOY_EXAMPLES`. Each line is the filename for a config **inside** `./Examples/` in this repo. For example, it might contain:
 
     ```txt
-    Minecraft-example.yaml
-    Valheim-example.yaml
+    Minecraft.java.example.yaml
+    Valheim.example.yaml
     ```
 
-2) Create a new Github Environment, with the same name as the line you added. For example, `./Minecraft-example.yaml`.
+    **NOTE**: If you deploy *manually*, the `container-id` will be `minecraft.java.example`. If the *pipeline* does it though, I made it default to everything left of the first period (i.e here, just `minecraft`). This is to keep the urls short, and also not conflict with manual deployments by default.
 
-3) Inside that environment, you can create any number of variables / secrets specific to that deployment. Since they're apart of the environment, they won't be exposed to the other containers either. They'll be environment variables when the action deploys, so reference them directly in the `./Examples/<container>` yaml file. I.e:
+    - This means `minecraft.java.example` and `minecraft.bedrock.example` will conflict by default. I figured no one wants both running at once, and there's away to override this if you do.
+    - You can override this by adding `CONTAINER_ID` as either a **secret** or **variable** in your Github Environment.
+
+2) Create a new Github Environment, with the same name as the line you added. For example, `Minecraft.java.example.yaml`.
+
+3) Inside that environment, you can create any number of variables / secrets specific to that deployment. Since they're apart of the environment, they won't be exposed to the other containers either. They'll be *environment variables* when the action deploys, so reference them directly in the `./Examples/<container>` yaml file. I.e:
 
     ```txt
     Github Secret: ${{ secrets.SERVER_PASS }}
@@ -50,7 +55,7 @@ I made this different than `cdk-synth`. For synth, it should run on EVERY config
     !ENV ${SERVER_NAME}
     ```
 
-    Technically to do this, all secrets/variables are turned into environment variables in the action. However, even if the container is untrusted, if you don't pass the env-vars to the container in the `./Examples/<container>` yaml file, it won't be able to see them.
+    Technically to do this, **all** secrets/variables are turned into environment variables in the action. However, even if the container is untrusted, if you don't pass the env-vars to the container in the `./Examples/<container>` yaml file, it won't be able to see them.
 
 **Finally**: If you decide to remove it to save money, you can just remove the one line from `DEPLOY_EXAMPLES`, then delete the stack. This lets you keep the environment around with all the variables, in case you want to re-deploy it later.
 
@@ -60,22 +65,24 @@ I specifically designed all the automation, so that if it's forked, you can re-u
 
 1) Setup [AWS OIDC](https://docs.github.com/en/actions/security-for-github-actions/security-hardening-your-deployments/configuring-openid-connect-in-amazon-web-services). How I do it personally is at [Cameronsplaze/AWS-OIDC](https://github.com/Cameronsplaze/AWS-OIDC). It's only one-per-account, so it can't be in this repo.
 
-2) **Secrets and variables: Actions Secrets** you'll want declared (in the 'core', NOT in any environment):
+2) There's a couple steps inside [Dependabot Auto-updates](#dependabot-auto-updates) above you'll want to follow, like setting up branch protections with required actions.
+
+3) **Secrets and variables: Actions Secrets** you'll want declared (in the 'core', NOT in any environment):
 
     - **AWS_ACCOUNT_ID**: Your AWS Account ID.
     - **DOMAIN_NAME**: The domain name of your Route53 Hosted Zone.
     - **HOSTED_ZONE_ID**: The ID of your Route53 Hosted Zone.
     - **EMAIL**: The email for receiving alerts for everything. (Might turn this into a list at some point, the yaml config supports that already anyways...)
     - **PAT_AUTOMERGE_PR**: The Personal Access Token for automerging PRs. If you used `GITHUB_TOKEN` instead, it wouldn't trigger other workflows when merged. Go to `Profile` -> `Settings` -> `Developer settings` -> `Personal access tokens`+`Fine-grained tokens` -> `Generate new token`. For permissions, only give it access to your fork, and you only need `Read & Write for Pull requests`.
 
-3) **Secrets and variables: *Actions Variables*** you'll want declared (in the 'core', NOT in any environment):
+4) **Secrets and variables: *Actions Variables*** you'll want declared (in the 'core', NOT in any environment):
 
     - **AWS_DEPLOY_ROLE**: The role to use with OIDC. (If you're using my repo, it's `github_actions_role`).
     - **AWS_REGION**: The region to deploy to. (Some HAS to be deployed to `us-east-1`, this is everything else that's not restricted).
     - **DEPLOY_EXAMPLES**: The list of container config paths to deploy, each on their own line. (See '[Automatic Deployments](#automatic-deployments-whitelistingadding-a-container)' for details).
 
-4) **Secrets and variables: *Dependabot Secrets***: These are only accessible to dependabot, BUT dependabot can't access any of the github secrets above.
+5) **Secrets and variables: *Dependabot Secrets***: These are only accessible to dependabot, BUT dependabot can't access any of the github secrets above.
 
     - **PAT_AUTOMERGE_PR**: A *classic* PAT with ONLY `repo:public_repo` permissions. (Has to be classic until [this issue](https://github.com/cli/cli/issues/9166) is fixed. You'll get permission denied otherwise.) Create this under `Account` -> `Settings` -> `Developer Settings` -> `Personal Access Tokens` -> `Tokens (classic)`
 
-5) Follow '[Automatic Deployments](#automatic-deployments-whitelistingadding-a-container)' for adding a new container to deploy. Can go through any number of times.
+6) Follow '[Automatic Deployments](#automatic-deployments-whitelistingadding-a-container)' for adding a new container to deploy. Can go through any number of times.

.github/workflows/dispatch-delete-leaf-stack.yaml

@@ -35,8 +35,10 @@ jobs:
           role-to-assume: "arn:aws:iam::${{ secrets.AWS_ACCOUNT_ID }}:role/${{ vars.AWS_DEPLOY_ROLE }}"
 
       ## Destroy the Leaf Stack:
+      # CONTAINER_ID: If not defined in Environment, default to everything before the first period in the filename.
       - name: "Destroying: ${{ inputs.environment }}"
         run: |
+          CONTAINER_ID=${CONTAINER_ID:=$(echo "${{ inputs.environment }}" | sed 's/\..*//i')}
           make cdk-destroy-leaf \
             config-file="${{ env.EXAMPLES_PATH }}/${{ inputs.environment }}" \
-            container-id=$(echo "${{ inputs.environment }}" | sed -E 's/-example\.ya?ml$//i')
+            container-id=${CONTAINER_ID}

.github/workflows/main-pipeline-cdk.yml

@@ -166,8 +166,9 @@ jobs:
 
       ## Deploy the Leaf Stack:
       - name: "Deploying: ${{ matrix.deploy-config }}"
-        # container-id: Take off the end '-example.yaml', and make all lowercase, to use the example file directly.
+        # CONTAINER_ID: If not defined in Environment, default to everything before the first period in the filename.
         run: |
+          CONTAINER_ID=${CONTAINER_ID:=$(echo "${{ matrix.deploy-config }}" | sed 's/\..*//i')}
           make cdk-deploy-leaf \
             config-file="${{ env.EXAMPLES_PATH }}/${{ matrix.deploy-config }}" \
-            container-id=$(echo "${{ matrix.deploy-config }}" | sed -E 's/-example\.ya?ml$//i')
+            container-id=${CONTAINER_ID}

ContainerManager/leaf_stack/NestedStacks/AsgStateChangeHook.py

@@ -38,6 +38,7 @@ def __init__(
         **kwargs,
     ) -> None:
         super().__init__(scope, "AsgStateChangeHook", **kwargs)
+        container_id_alpha = "".join(e for e in container_id.title() if e.isalpha())
 
         ## Log group for the lambda function:
         # https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_logs.LogGroup.html
@@ -71,7 +72,7 @@ def __init__(
         self.lambda_asg_state_change_hook = aws_lambda.Function(
             self,
             "AsgStateChangeHook",
-            description=f"{container_id}-ASG-StateChange: Triggered by ec2 state changes. Starts/Stops the management logic",
+            description=f"{container_id_alpha}-ASG-StateChange: Triggered by ec2 state changes. Starts/Stops the management logic",
             code=aws_lambda.Code.from_asset("./ContainerManager/leaf_stack/lambda/instance-StateChange-hook/"),
             handler="main.lambda_handler",
             runtime=aws_lambda.Runtime.PYTHON_3_12,

ContainerManager/leaf_stack/NestedStacks/Container.py

@@ -29,6 +29,7 @@ def __init__(
         **kwargs
     ) -> None:
         super().__init__(scope, "ContainerNestedStack", **kwargs)
+        container_id_alpha = "".join(e for e in container_id.title() if e.isalpha())
 
         ## The details of a task definition run on an EC2 cluster.
         # https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_ecs.TaskDefinition.html
@@ -56,7 +57,7 @@ def __init__(
         ## And what it returns:
         # https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_ecs.ContainerDefinition.html
         self.container = self.task_definition.add_container(
-            container_id.title(),
+            container_id_alpha,
             image=ecs.ContainerImage.from_registry(container_config["Image"]),
             port_mappings=container_config["Ports"],
             ## Hard limit. Won't ever go above this

ContainerManager/leaf_stack/NestedStacks/Dashboard.py

@@ -37,6 +37,7 @@ def __init__(
         **kwargs
     ) -> None:
         super().__init__(scope, "DashboardNestedStack", **kwargs)
+        container_id_alpha = "".join(e for e in container_id.title() if e.isalpha())
 
         #######################
         ### Dashboard stuff ###
@@ -246,7 +247,7 @@ def __init__(
         self.dashboard = cloudwatch.Dashboard(
             self,
             "CloudwatchDashboard",
-            dashboard_name=f"{application_id}-{container_id}-Dashboard",
+            dashboard_name=f"{application_id}-{container_id_alpha}-Dashboard",
             period_override=cloudwatch.PeriodOverride.AUTO,
             default_interval=dashboard_config["IntervalMinutes"],
             widgets=[dashboard_widgets],

ContainerManager/leaf_stack/NestedStacks/Watchdog.py

@@ -38,6 +38,7 @@ def __init__(
         **kwargs,
     ) -> None:
         super().__init__(scope, "WatchdogNestedStack", **kwargs)
+        container_id_alpha = "".join(e for e in container_id.title() if e.isalpha())
 
         ## Scale down ASG if this is ever triggered:
         # https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_autoscaling.StepScalingAction.html
@@ -174,7 +175,7 @@ def __init__(
         self.lambda_watchdog_container_activity = aws_lambda.Function(
             self,
             "WatchdogContainerActivity",
-            description=f"{container_id}-Watchdog: Counts the number of connections to the container, and passes it to a CloudWatch Alarm.",
+            description=f"{container_id_alpha}-Watchdog: Counts the number of connections to the container, and passes it to a CloudWatch Alarm.",
             code=aws_lambda.Code.from_asset("./ContainerManager/leaf_stack/lambda/watchdog-container-activity/"),
             handler="main.lambda_handler",
             runtime=aws_lambda.Runtime.PYTHON_3_12,

Examples/README.md

@@ -2,7 +2,13 @@
 
 ## Creating Configs
 
-These are config options when you deploy, for a single leaf. (The file's name becomes the sub-domain for the stack, so one file for one stack. I.e `Minecraft-example.yaml` -> `minecraft-example.my-domain.com`). See any `*-example.yaml` in this directory for examples. (If you need to override the domain name to something new when deploying, use the `maturity=` key. See the developer section in the [root README.md](../README.md#different-maturities) for more details.)
+These are config options when you deploy, for a single leaf. (The file's name becomes the sub-domain for the stack, so one file for one stack. I.e `Minecraft.java.example.yaml` -> `minecraft.java.example.my-domain.com`). See any `*.example.yaml` in this directory for examples. (If you need to override the domain name to something new when deploying, use the `container_id=` key. See the developer section in the [root README.md](../README.md#different-maturities) for more details.)
+
+If you're looking at automating updates to the deployments, or just deploying with GitHub, see the [GitHub Actions README](../.github/workflows/README.md).
+
+### Config File Options
+
+You can also look at the yaml's in the [./Examples](./) directory here to see how each of these are used directly.
 
 - `Ec2`: (dict)
 
@@ -137,8 +143,16 @@ These are config options when you deploy, for a single leaf. (The file's name be
 
 ## Gotchas
 
-- **For backups**: We use EFS behind the scenes. Use the `Volume.EnableBackups` if you want backups. **IF you do it inside the container**, you'll be doing backups of backups, and pay a lot more for storage. Plus if your container gets hacked, they'll have access to the backups too.
+- **For backups**: We use EFS behind the scenes. Use the `Volume.EnableBackups` if you want backups. **IF you do it inside the container**, you'll be doing backups of backups, and pay a lot more for storage. Plus if your container gets hacked, they'll have access to the backups too. Always use flags for the container to disable backups, and use EFS if you want them.
 - **For updating the server**: Since the container is only up when someone is connected, any "idle update" strategy won't work. The container has to check for updates when it first spins up. Then what to do depends on the game.
   - For minecraft, it won't let anyone connect until after it finishes. It handles everything for you.
   - For Valheim, it'll let you connect, then everyone will get kicked when it finishes so it can restart (3-4min into playing). OR you can have it *not* restart, and you'll get the update after everyone disconnects for the night.
 - **Whitelist users inside of the Configs**: All the containers I've tested so far provide some form of this. You can use it, but it means you have to re-deploy this project every time you make a change. It takes forever, and (might?) kick everyone for a bit. If you can, use the game's built-in whitelist feature instead. (Unless maybe you don't expect it changing often, like with an admin list.)
+
+## Adding a new Example to the Repo
+
+1) Create a new file in this [./Examples](./) directory. Make sure it ends with `*.example.yaml`.
+2) Make sure it correctly Synths. (If you're doing a PR, it'll happen automatically)
+3) Once it Synths, add it to the "Settings -> Branches -> `main` -> Required Status Checks" list. (If you don't have permissions, remind me to inside the PR please).
+
+And that's it!

README.md

@@ -48,14 +48,14 @@ make cdk-deploy-base
 
 #### Leaf Stack
 
-The config examples are in `./Examples/*-example.yaml`. Info on each config option and writing your own config is in [./Examples/README.md](./Examples/README.md). For a quickstart, just run:
+The config examples are in `./Examples/*.example.yaml`. Info on each config option and writing your own config is in [./Examples/README.md](./Examples/README.md). For a quickstart, just run:
 
 ```bash
 # IF a new shell
 source .venv/bin/activate
 source vars.env
 # Edit the config to what you want:
-cp ./Examples/Minecraft-example.yaml ./Minecraft.yaml
+cp ./Examples/Minecraft.java.example.yaml ./Minecraft.yaml
 nano ./Minecraft.yaml
 # Actually deploy:
 make cdk-deploy-leaf config-file=./Minecraft.yaml
@@ -212,15 +212,15 @@ For example, you can have GH Actions deploy to prod, but use devel locally. Both
 ```bash
 # To deploy to prod, it'll look like:
 #    (You can have `maturity=prod` if you want, but it's the default).
-make cdk-deploy-leaf config-file=./Examples/Minecraft-example.yaml container-id=Minecraft
+make cdk-deploy-leaf config-file=./Examples/Minecraft.java.example.yaml container-id=Minecraft
 # And then manually deploying to devel could look like:
-make cdk-deploy-leaf config-file=./Examples/Minecraft-example.yaml maturity=devel
+make cdk-deploy-leaf config-file=./Examples/Minecraft.java.example.yaml maturity=devel
 ```
 
-This would still give you two stacks, each with a different base stack. They won't conflict since the first command got overridden to `minecraft`, and the second one is using the default `minecraft-example` from the filename:
+This would still give you two stacks, each with a different base stack. They won't conflict since the *first command* got overridden to `minecraft`, and the *second one* is using the default `minecraft.java.example` from the filename:
 
 - `minecraft.<DOMAIN>`: On the normal prod stack.
-- `minecraft-example.<DOMAIN>`: In the devel stack.
+- `minecraft.java.example.<DOMAIN>`: In the devel stack.
 
 > [!NOTE]
 > If you want to update an existing stack, you MUST pass in the same exact flags you deployed with! Otherwise it's going to try to create a new stack entirely.