README.md

Wombat is a high performance and resilient stream processor, able to connect various sources and sinks in a range of brokering patterns and perform hydration, enrichments, transformations and filters on payloads.

It comes with a powerful mapping language, is easy to deploy and monitor, and ready to drop into your pipeline either as a static binary or docker image, making it cloud native as heck.

Wombat is declarative, with stream pipelines defined in as few as a single config file, allowing you to specify connectors and a list of processing stages:

input:
  gcp_pubsub:
    project: foo
    subscription: bar

pipeline:
  processors:
    - mapping: |
        root.message = this
        root.meta.link_count = this.links.length()
        root.user.age = this.user.age.number()

output:
  redis_streams:
    url: tcp://TODO:6379
    stream: baz
    max_in_flight: 20

Why Fork?

First of all, this project is not a full fork. We still use the MIT licensed RedPanda Benthos project as our base. We even still use a large part of the Apache2 Licensed RedPanda Connect project. We did however fork some of the components RedPanda made proprietary and added some of our own.

The idea behind this move is to allow anyone to experience wombat without having to rely on a commercial entity behind it. This is a community project and always will be.

Documentation

Take a look at the documentation for more information on how to get started

Install

We're working on the release process, but you can either compile from source or pull the docker image:

docker pull ghcr.io/wombatwisdom/wombat

For more information check out the getting started guide.

Run

wombat -c ./config.yaml

Or, with docker:

# Using a config file
docker run --rm -v /path/to/your/config.yaml:/wombat.yaml ghcr.io/wombatwisdom/wombat

# Using a series of -s flags
docker run --rm -p 4195:4195 ghcr.io/wombatwisdom/wombat \
  -s "input.type=http_server" \
  -s "output.type=kafka" \
  -s "output.kafka.addresses=kafka-server:9092" \
  -s "output.kafka.topic=wombat_topic"

Honorable Mentions

I can't in all good faith take credit for the enormous amount of work that went into this project. Most of that is on Ash and the rest of the community behind the old Benthos project. I'm just a guy who forked it and made it worse.

For those of you who miss Ash too much, here are some links to some of the old content still available:

• Delivery guarantees can be a dodgy subject

Contributing

Contributions are welcome, please read the guidelines, and watch your back.