GitHub Action
Run Elementary
This action allows you to run edr as a GitHub Action.
In order to use this action, you will need to create a workflow within your repository.
To create a new workflow, simply create .github/workflows/elementary.yml within your repository.
If you have not yet installed Elementary's dbt package, please refer to this guide.
If you already have a profiles.yml in your repository,
checkout the repository and pass the environment variables that are used within it to the action.
edr will automatically pick up on the profiles.yml file.
- name: Checkout dbt project
uses: actions/checkout@v3
- name: Run Elementary
uses: elementary-data/run-elementary-action@v1
env:
USER: ${{ secrets.USER }}
PASSWORD: ${{ secrets.PASSWORD }}
with:
edr-command: ...If you do not have a profiles.yml in your repository, you can pass the credentials directly to the action.
In order to generate the profiles.yml that is needed by edr to operate, run the following command within your dbt
project:
dbt run-operation --args '{"method": "github-actions"}' elementary.generate_elementary_cli_profileAfterwards, fill in the missing fields and copy the output to a secret in your repository. Once you've done that, you can pass the secret to the action.
- name: Checkout dbt project
uses: actions/checkout@v3
- name: Run Elementary
uses: elementary-data/run-elementary-action@v1
with:
profiles-yml: ${{ secrets.ELEMENTARY_PROFILES_YML }}
edr-command: ...Below is a basic example of an Elementary workflow file.
For more information on how to
use GitHub workflows.
name: Run Elementary
on:
# Run the action when a push to the main branch of the repository is made.
push:
branches: [ "main", "master" ]
# Run the action when a pull request to the main branch is opened.
pull_request:
branches: [ "main", "master" ]
# Run the action in a scheduled manner every hour.
schedule:
- cron: '0 * * * *'
# Allows you to run this workflow manually from the Actions tab.
workflow_dispatch:
jobs:
elementary:
runs-on: ubuntu-latest
steps:
- name: Checkout dbt project
uses: actions/checkout@v3
- name: Run Elementary
uses: elementary-data/run-elementary-action@v1
with:
warehouse-type: bigquery # Type of warehouse to use (bigquery, snowflake, redshift, etc.)
profiles-yml: ${{ secrets.ELEMENTARY_PROFILES_YML }} # Content of ~/.dbt/profiles.yml, should have an `elementary` profile.
edr-command: |
edr monitor --slack-token ${{ secrets.SLACK_TOKEN }} --slack-channel-name ${{ secrets.SLACK_CHANNEL_NAME }} &&
edr monitor report --file-path report.html &&
edr monitor send-report \
--slack-token ${{ secrets.SLACK_TOKEN }} --slack-channel-name ${{ secrets.SLACK_CHANNEL_NAME }} \
--aws-access-key-id ${{ secrets.AWS_ACCESS_KEY_ID }} --aws-secret-access-key ${{ secrets.AWS_SECRET_ACCESS_KEY }} --s3-bucket-name ${{ secrets.S3_BUCKET_NAME }} \
--google-service-account-path /tmp/gcs_keyfile.json --gcs-bucket-name ${{ secrets.GCS_BUCKET_NAME }} \
--update-bucket-website true
bigquery-keyfile: ${{ secrets.BIGQUERY_KEYFILE }} # If using BigQuery, the content of its keyfile.
gcs-keyfile: ${{ secrets.GCS_KEYFILE }} # If using GCS, the content of its keyfile.
- name: Upload report
uses: actions/upload-artifact@v3
with:
name: report.html
path: report.html
- name: Upload log
if: always()
uses: actions/upload-artifact@v3
with:
name: edr.log
path: edr.logNote that the profiles-yml argument needs to be the content of profiles.yml rather than a path to it.
You can create a secret and then pass it.
profiles-yml: ${{ secrets.PROFILES_YML }}
If you're using BigQuery with a key file,
supply the bigquery-keyfile argument to the action and make sure your keyfile in the profiles-yml
is /tmp/bigquery_keyfile.json.
If you want to upload your report to a Google Cloud Storage bucket using send-report,
supply the gcs-keyfile argument to the action with the content of your Google service account keyfile.
Afterwards, use edr monitor send-report --google-service-account-path /tmp/gcs_keyfile.json to upload the report.
Please contact us on Slack, we're here to help!