Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Simplify installation #39

Merged
merged 3 commits into from
Mar 22, 2024
Merged

Simplify installation #39

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
5a645b7
remove pinata as a requirement
jeremy-asher Mar 22, 2024
2d17a9f
don't crash outside of a git repo
jeremy-asher Mar 22, 2024
92d8c9a
first pass at updating README
jeremy-asher Mar 22, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
132 changes: 26 additions & 106 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,15 +2,25 @@

Luther Systems infrastructure management tool.

# Setup
# Installation

Build the container
Clone this repository. For ease of access, alias the name `mars` to the
`mars_macos.sh` script in this repo. Add the following snippet to your shell
configuration, substituting the correct path to this repository:

```
make
alias mars="$HOME/mars/mars_macos.sh"
```

Make sure ssh-agent has your key private key
## Additional requirements for ansible

If running ansible locally, you will also need ssh agent access through mars.
This is currently accomplished using the `pinata-ssh-forward` tool from
[uber-common/docker-ssh-forward](https://github.com/uber-common/docker-ssh-agent-forward).
Follow the installation instructions there to install the tool and make sure
`pinata-ssh-forward` is in your path.

In addition, make sure ssh-agent has your ssh key loaded:

```
ssh-add -l
Expand All @@ -19,79 +29,6 @@ ssh-add -l
If ssh-add doesn't print anything, it doesn't have you key. Run `ssh-add`
without arguments or point it to the appropriate key file in special cases.

# Usage

Place a .mars-version file at the root of the repository to ensure that the
behavior of mars is the same for all team members.

```
echo $MARS_VERSION > $(git rev-parse --show-toplevel)/.mars-version
```

Ensure all terraform projects have a `.terraform-version` file specifying which
terraform version to use (technically optional). See
[tfenv](https://github.com/kamatama41/tfenv) for more details.

```
echo 0.11.3 > .terraform-version
```

MacOS users need a method of forwarding the host's ssh-agent to the docker
container is necessary (see long discussions spawning from [this thread](
https://forums.docker.com/t/can-we-re-use-the-osx-ssh-agent-socket-in-a-container/8152)).
It seems like
[uber-common/docker-ssh-forward](https://github.com/uber-common/docker-ssh-agent-forward)
is the defacto best option as the
[avsm/docker-ssh-forward](https://github.com/avsm/docker-ssh-agent-forward)
project recommended from the linked thread seems to be unmaintained now (years
later).

To ease working with the docker image install a symlink pointing to the shell
script in this project to make running the container less painful.

MacOS users should symlink the macOS specific script `mars_macos.sh` which will
make use of `pinata-ssh-mount` to forward the host ssh agent (see note above).

```
ln -s $(pwd)/mars_macos.sh ~/bin/mars
mars -h
```

## Technical details

If `$SSH_AUTH_SOCK` is mountable on docker containers (on Linux -- not macOS),
general usage has the following form.

```sh
LOCAL_CACHE="$HOME/.mars/tfenv/versions"
END_USER=$(id -u $USER):$(id -g $USER)
DOCKER_WORKDIR=/terraform

PROJECT_PATH=$(pwd)
docker run --rm -it \
-u $END_USER \
-v "$LOCAL_CACHE:/opt/tfenv/versions" \
-v "$PROJECT_PATH:/terraform" \
-v "$HOME/.aws/:/opt/home/.aws" \
-v "$SSH_AUTH_SOCK:$SSH_AUTH_SOCK" \
-e "SSH_AUTH_SOCK=$SSH_AUTH_SOCK" \
luthersystems/mars COMMAND [FLAGS] ARGS
```

The user (`-u`) is set so that state files in the project's .terraform
directory will be owned by the correct user (not root). AWS credentials must
be mounted into /opt/home, which is containers value for environment variable
`HOME`. Finally, in order for ssh to work the ssh-agent's socket must be
mounted into the container from the host and variable `SSH_AUTH_SOCK` has to be
set, telling the container where to find the unix socket.

Mounting to path /opt/tfenv/versions is not a requirement but will prevent
containers run with `--rm` from continuously needing to download versions of
terraform not included in the default installation. The mount, or
alternatively running with `--rm`, will bypass this issue. For now, `--rm` and
mounting the cache in the docker command fits our workflow better so we
typically do that.

## Terraform

If you need to run a raw terraform command using the `terraform` binary
Expand All @@ -104,46 +41,29 @@ meant for terraform.
mars dev terraform -- providers --help
```

## Packer
# Setting up managed repositories

Running packer currently expects a specific directory structure.
Place a .mars-version file at the root of the mars managed infrastructure
repository to pin the version of mars for the repository. For example, to set
the mars version to v0.81.1, run the following from the target repo.

```
/IMAGE/packer.json
/...
echo v0.82.1 > $(git rev-parse --show-toplevel)/.mars-version
```

A packer.json file is expected to be nested under a directory with the name of
the output AMI. This structure allows multiple pcaker AMIs to be built using
common ansible roles. Run packer commands by specifying the image and then the
desired command.
Ensure all terraform projects have a `.terraform-version` file specifying which
terraform version to use. See [tfenv](https://github.com/kamatama41/tfenv) for
more details.

```
mars IMAGE packer-validate
mars IMAGE packer-build
echo 1.7.3 > .terraform-version
```

## ALB

A utility is provided to locate DNS names for load balancers created by
kubernetes ingress objects (via the alb-ingress-controller) using tags.
# Building

Create a file that defines environment variables `PROJECT_NAME` and
`AWS_REGION`. For example,
To build the container, run:

```
cat > mars.env
PROJECT_NAME=xyz
AWS_REGION=eu-west-2
^D
```

Then invoke the utility specify an optional component and org.

```
mars ENV alb-dns --component COMPONENT --org ORG
make
```

The utility will print the DNS name for all matching ALBs. Any number of ALBs
may be found if the provided parameters aren't specific enough or if no ALB has
tags matching the input values.
19 changes: 13 additions & 6 deletions mars_macos.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,9 @@ fullpath() {
}

getroot() {
GIT_ROOT=$(git rev-parse --show-toplevel 2>/dev/null)
if ! GIT_ROOT=$(git rev-parse --show-toplevel 2>/dev/null); then
return
fi
if [ -z "$PROJECT_PATH"]; then
if [ -n "$GIT_ROOT" ]; then
PROJECT_PATH="$GIT_ROOT"
Expand Down Expand Up @@ -68,9 +70,14 @@ if [ -n "${TF_LOG+x}" ]; then
ENV_VARS="-e TF_LOG=$TF_LOG $ENV_VARS"
fi

if [ -z "$(docker ps | grep pinata-sshd)" ]; then
echo 2>&1 "pinata-sshd not found; starting..."
pinata-ssh-forward

PINATA_OPTS=""
if command -v pinata-ssh-forward > /dev/null; then
PINATA_OPTS="$(pinata-ssh-mount)"
if [ -z "$(docker ps | grep pinata-sshd)" ]; then
echo 2>&1 "pinata-sshd not found; starting..."
pinata-ssh-forward
fi
fi

DOCKER_TERM_VARS=-i
Expand All @@ -86,7 +93,7 @@ if [[ "$MARS_SHELL" == "true" ]]; then
fi

# include GitHub credentials if available
if type -ap gh &> /dev/null; then
if command -v gh > /dev/null; then
export GITHUB_TOKEN="$(gh auth token)"
fi

Expand All @@ -111,5 +118,5 @@ docker run --rm $DOCKER_TERM_VARS \
-e ANSIBLE_LOAD_CALLBACK_PLUGINS=yes \
-e ANSIBLE_STDOUT_CALLBACK=yaml \
$SHELL_OPTS \
$(pinata-ssh-mount) \
$PINATA_OPTS \
"$DOCKER_IMAGE:$MARS_VERSION" "$@"
Loading