Auth0 - Logs to S3

This extension will take all of your Auth0 logs and export them to an AWS S3 Bucket.

Configure Amazon Web Services

First you'll need to create an Amazon Web Services account.

You will need to setup an IAM user with the following policy (replace your-bucket-name with an actual bucket name):

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "AllowedActions",
            "Effect": "Allow",
            "Action": [
                "s3:PutObject"
            ],
            "Resource": [
                "arn:aws:s3:::your-bucket-name/*"
            ]
        }
    ]
}

You will also need to create an S3 Bucket to put the logs into if you have not already done so.

Automatic Item Expiry

You can configure the S3 bucket with a lifecycle rule to enforce automatic deletion after X days. See more information here.

Configure Webtask

If you haven't configured Webtask on your machine run this first:

npm i -g wt-cli
wt init

Requires at least node 0.10.40 - if you're running multiple version of node make sure to load the right version, e.g. "nvm use 0.10.40"

Deploy to Webtask.io

To run it on a schedule (run every 5 minutes for example):

wt cron schedule \
    --name auth0-logs-to-s3 \
    --secret AUTH0_DOMAIN="YOUR_AUTH0_DOMAIN" \
    --secret AUTH0_CLIENT_ID="YOUR_AUTH0_GLOBAL_CLIENT_ID" \
    --secret AUTH0_CLIENT_SECRET="YOUR_AUTH0_GLOBAL_CLIENT_SECRET" \
    --secret AWS_ACCESS_KEY_ID="YOUR_AWS_ACCESS_KEY_ID" \
    --secret AWS_SECRET_ACCESS_KEY="YOUR_AWS_SECRET_ACCESS_KEY" \
    --secret AWS_BUCKET_NAME="YOUR_AWS_BUCKET_NAME" \
    --json \
    "*/5 * * * *" \
    ./build/bundle.js

There are optional properties you can set (here they are with their defaults).

    --secret AWS_REGION="us-west-2" \
    --secret BATCH_SIZE="100" \

You can get your Global Client Id/Secret here.

Usage

Use any Storage Explorer to access your logs. Each record will be saved in the following structure:

{YOUR_AWS_BUCKET_NAME}/YYYY/MM/DD/HH/{LOG_ISO_8601_DATE_STRING}-{LOG_ID}.json

This will allow you to process the logs with Stream Analytics for example (which in turn can export them to Power BI).

Issue Reporting

If you have found a bug or if you have a feature request, please report them at this repository issues section. Please do not report security vulnerabilities on the public GitHub issue tracker. The Responsible Disclosure Program details the procedure for disclosing security issues.

Author

Seth Westphal

What is Auth0?

Auth0 helps you to:

• Add authentication with multiple authentication sources, either social like Google, Facebook, Microsoft Account, LinkedIn, GitHub, Twitter, Box, Salesforce, amont others, or enterprise identity systems like Windows Azure AD, Google Apps, Active Directory, ADFS or any SAML Identity Provider.
• Add authentication through more traditional username/password databases.
• Add support for linking different user accounts with the same user.
• Support for generating signed Json Web Tokens to call your APIs and flow the user identity securely.
• Analytics of how, when and where users are logging in.
• Pull data from other sources and add it to the user profile, through JavaScript rules.

Create a free Auth0 Account

1. Go to Auth0 and click Sign Up.
2. Use Google, GitHub or Microsoft Account to login.

License

This project is licensed under the MIT license. See the LICENSE file for more info.

The original sources were heavily influenced by auth0/auth0-logs-to-azure-blob-storage.