diff --git a/README.md b/README.md index edfdb6b..29ab913 100644 --- a/README.md +++ b/README.md @@ -2,15 +2,25 @@ Luther Systems infrastructure management tool. -# Setup +# Installation -Build the container +Clone this repository. For ease of access, alias the name `mars` to the +`mars_macos.sh` script in this repo. Add the following snippet to your shell +configuration, substituting the correct path to this repository: ``` -make +alias mars="$HOME/mars/mars_macos.sh" ``` -Make sure ssh-agent has your key private key +## Additional requirements for ansible + +If running ansible locally, you will also need ssh agent access through mars. +This is currently accomplished using the `pinata-ssh-forward` tool from +[uber-common/docker-ssh-forward](https://github.com/uber-common/docker-ssh-agent-forward). +Follow the installation instructions there to install the tool and make sure +`pinata-ssh-forward` is in your path. + +In addition, make sure ssh-agent has your ssh key loaded: ``` ssh-add -l @@ -19,79 +29,6 @@ ssh-add -l If ssh-add doesn't print anything, it doesn't have you key. Run `ssh-add` without arguments or point it to the appropriate key file in special cases. -# Usage - -Place a .mars-version file at the root of the repository to ensure that the -behavior of mars is the same for all team members. - -``` -echo $MARS_VERSION > $(git rev-parse --show-toplevel)/.mars-version -``` - -Ensure all terraform projects have a `.terraform-version` file specifying which -terraform version to use (technically optional). See -[tfenv](https://github.com/kamatama41/tfenv) for more details. - -``` -echo 0.11.3 > .terraform-version -``` - -MacOS users need a method of forwarding the host's ssh-agent to the docker -container is necessary (see long discussions spawning from [this thread]( -https://forums.docker.com/t/can-we-re-use-the-osx-ssh-agent-socket-in-a-container/8152)). -It seems like -[uber-common/docker-ssh-forward](https://github.com/uber-common/docker-ssh-agent-forward) -is the defacto best option as the -[avsm/docker-ssh-forward](https://github.com/avsm/docker-ssh-agent-forward) -project recommended from the linked thread seems to be unmaintained now (years -later). - -To ease working with the docker image install a symlink pointing to the shell -script in this project to make running the container less painful. - -MacOS users should symlink the macOS specific script `mars_macos.sh` which will -make use of `pinata-ssh-mount` to forward the host ssh agent (see note above). - -``` -ln -s $(pwd)/mars_macos.sh ~/bin/mars -mars -h -``` - -## Technical details - -If `$SSH_AUTH_SOCK` is mountable on docker containers (on Linux -- not macOS), -general usage has the following form. - -```sh -LOCAL_CACHE="$HOME/.mars/tfenv/versions" -END_USER=$(id -u $USER):$(id -g $USER) -DOCKER_WORKDIR=/terraform - -PROJECT_PATH=$(pwd) -docker run --rm -it \ - -u $END_USER \ - -v "$LOCAL_CACHE:/opt/tfenv/versions" \ - -v "$PROJECT_PATH:/terraform" \ - -v "$HOME/.aws/:/opt/home/.aws" \ - -v "$SSH_AUTH_SOCK:$SSH_AUTH_SOCK" \ - -e "SSH_AUTH_SOCK=$SSH_AUTH_SOCK" \ - luthersystems/mars COMMAND [FLAGS] ARGS -``` - -The user (`-u`) is set so that state files in the project's .terraform -directory will be owned by the correct user (not root). AWS credentials must -be mounted into /opt/home, which is containers value for environment variable -`HOME`. Finally, in order for ssh to work the ssh-agent's socket must be -mounted into the container from the host and variable `SSH_AUTH_SOCK` has to be -set, telling the container where to find the unix socket. - -Mounting to path /opt/tfenv/versions is not a requirement but will prevent -containers run with `--rm` from continuously needing to download versions of -terraform not included in the default installation. The mount, or -alternatively running with `--rm`, will bypass this issue. For now, `--rm` and -mounting the cache in the docker command fits our workflow better so we -typically do that. - ## Terraform If you need to run a raw terraform command using the `terraform` binary @@ -104,46 +41,29 @@ meant for terraform. mars dev terraform -- providers --help ``` -## Packer +# Setting up managed repositories -Running packer currently expects a specific directory structure. +Place a .mars-version file at the root of the mars managed infrastructure +repository to pin the version of mars for the repository. For example, to set +the mars version to v0.81.1, run the following from the target repo. ``` -/IMAGE/packer.json -/... +echo v0.82.1 > $(git rev-parse --show-toplevel)/.mars-version ``` -A packer.json file is expected to be nested under a directory with the name of -the output AMI. This structure allows multiple pcaker AMIs to be built using -common ansible roles. Run packer commands by specifying the image and then the -desired command. +Ensure all terraform projects have a `.terraform-version` file specifying which +terraform version to use. See [tfenv](https://github.com/kamatama41/tfenv) for +more details. ``` -mars IMAGE packer-validate -mars IMAGE packer-build +echo 1.7.3 > .terraform-version ``` -## ALB - -A utility is provided to locate DNS names for load balancers created by -kubernetes ingress objects (via the alb-ingress-controller) using tags. +# Building -Create a file that defines environment variables `PROJECT_NAME` and -`AWS_REGION`. For example, +To build the container, run: ``` -cat > mars.env -PROJECT_NAME=xyz -AWS_REGION=eu-west-2 -^D -``` - -Then invoke the utility specify an optional component and org. - -``` -mars ENV alb-dns --component COMPONENT --org ORG +make ``` -The utility will print the DNS name for all matching ALBs. Any number of ALBs -may be found if the provided parameters aren't specific enough or if no ALB has -tags matching the input values. diff --git a/mars_macos.sh b/mars_macos.sh index 0ddbb42..d1e4e5a 100755 --- a/mars_macos.sh +++ b/mars_macos.sh @@ -19,7 +19,9 @@ fullpath() { } getroot() { - GIT_ROOT=$(git rev-parse --show-toplevel 2>/dev/null) + if ! GIT_ROOT=$(git rev-parse --show-toplevel 2>/dev/null); then + return + fi if [ -z "$PROJECT_PATH"]; then if [ -n "$GIT_ROOT" ]; then PROJECT_PATH="$GIT_ROOT" @@ -68,9 +70,14 @@ if [ -n "${TF_LOG+x}" ]; then ENV_VARS="-e TF_LOG=$TF_LOG $ENV_VARS" fi -if [ -z "$(docker ps | grep pinata-sshd)" ]; then - echo 2>&1 "pinata-sshd not found; starting..." - pinata-ssh-forward + +PINATA_OPTS="" +if command -v pinata-ssh-forward > /dev/null; then + PINATA_OPTS="$(pinata-ssh-mount)" + if [ -z "$(docker ps | grep pinata-sshd)" ]; then + echo 2>&1 "pinata-sshd not found; starting..." + pinata-ssh-forward + fi fi DOCKER_TERM_VARS=-i @@ -86,7 +93,7 @@ if [[ "$MARS_SHELL" == "true" ]]; then fi # include GitHub credentials if available -if type -ap gh &> /dev/null; then +if command -v gh > /dev/null; then export GITHUB_TOKEN="$(gh auth token)" fi @@ -111,5 +118,5 @@ docker run --rm $DOCKER_TERM_VARS \ -e ANSIBLE_LOAD_CALLBACK_PLUGINS=yes \ -e ANSIBLE_STDOUT_CALLBACK=yaml \ $SHELL_OPTS \ - $(pinata-ssh-mount) \ + $PINATA_OPTS \ "$DOCKER_IMAGE:$MARS_VERSION" "$@"