Log Export Container

Thank you for your interest in Log Export Container. As a result of the positive feedback we have received about the LEC project, we want to further invest in our core product offering to deliver functionality that will address similar challenges. We will give time and attention to the development of these feature sets over the next few quarters.

In the meantime, we are freezing the current version of Log Export Container v1.0.53 and have halted any non-critical bug fixes. We’ll continue to support the existing version while we work to build out the core product to meet similar use cases to those that LEC serves. Let us know if you have any questions about how this might impact you or the projects you are working on.

A docker container that can be easily deployed and configured to export strongDM query logs.

The container acts as a syslog concentrator. Customers that want to export their strongDM query logs to a third party logging service can use the container to do so. They configure the container for the appropriate target. Deploy the container. Configure their strongDM gateways to logs to a syslog destination and set the destination to the address of the logging container.

The container uses fluentd for processing and routing your logs. Currently supports routing to: stdout, remote syslog, S3, CloudWatch, Splunk HEC, Datadog, Azure Log Analytics, Sumo Logic, Kafka, Mongo, Logz, ElasticSearch and BigQuery. You could configure multiple storages in one container, for example: stdout s3.

A curated version of the documentation can be found here

Table of Contents

• Getting Started
• Dev Setup
• Contributing
• Support

Getting Started

The Log Export Container is available as a Ruby Gem and as a Docker Image you can use for spinning up multiple containers.

Ruby Gem

To use the log-export-container Ruby Gem, first you should have Ruby 3 installed. Then, you can install log-export-container gem by running gem install log-export-container.

NOTE: In case you face the error undefined class/module YAML when trying to install the gem, enter the following commands to solve the error:

$ gem install rubygems-update
$ update_rubygems

After that, you can use log-export-container globally as a command:

$ log-export-container

Docker Image

To use the log-export-container Docker Image, follow the next steps:

1. Download the docker-compose.yml file from the Github repo onto your machine (or copy-paste its contents into a file you created directly on the machine with the same name).
   • Make sure that the 'Required variables' in the .yml file are set appropriately based on your desired log format and output destination.
2. Run with your preferred container orchestrator (with docker, you can simply run docker-compose up)
3. Log into the strongDM Admin UI and go to the Settings page, then the Log Encryption & Storage tab.
4. Set "Log locally on relays?" to 'Yes'
5. Set "Local storage?" to "Syslog" and enter the IP address of the machine running the Log Export Container along with port 5140
   • 
   • Make sure that port 5140 on the machine hosting the container is accesible from your gateways. You can also host the container on your gateways themselves.
6. Set "Local format?" to match the input format you specified in the .yml file.
7. Click "Update" and you're done!
8. If you notice that your strongDM client is stuck in reconnecting mode after hitting update, then it's possible that something went wrong during the setup process. Check your docker settings, security settings, port availability, IP address, and that the Docker image is running correctly.

Here's a gif demonstrating the setup process:

Configure Your LEC Instance

For configuration details, please refer to CONFIGURE_LOG_EXPORT_CONTAINER.md.

Dev Tools

If you want to modify the container and quickly see the changes in your local, you could start the container locally and forward the local port to your gateway using:

./dev-tools/start-container.sh

You could also run the project in your local without docker, please refer to CONFIGURE_LOCAL_ENV

Monitoring

Currently the application supports Prometheus Metrics about the received and forwarded logs. For more details, please see CONFIGURE_PROMETHEUS

Contributing

Refer to the contributing guidelines or dump part of the information here.

Support

Refer to the support guidelines or dump part of the information here.