Bloblang for field values

[codingconcepts]

It would be great to have a list of fields that support Bloblang as described in the Interpolation page.

I'm hoping to spin up multiple Kafka consumers with different client_ids but the client_id doesn't seem to support Bloblang:

input:
    kafka_balanced:
        addresses:
            - localhost:9092
        topics: [ topic.1 ]
        client_id: ${! uuid_v4() }
        consumer_group: example

output:
  stdout: {}

The output of running against the above configuration is as follows:

{"@timestamp":"2020-11-02T20:21:19Z","@service":"benthos","level":"ERROR","component":"benthos.input","message":"Failed to connect to kafka_balanced: kafka: invalid configuration (ClientID is invalid)"}

Does this field support Bloblang, or have I misunderstood something?

[Jeffail]

Hey @codingconcepts, unfortunately client_id doesn't support interpolations, they're mostly limited to fields that are useful to change based on messages passing through rather than cases like this that would be evaluated once on start up. A dirty work around might be to use an env var instead with client_id: ${CLIENT_ID}, and then when you execute the config set that var with something like CLIENT_ID=$(cat /proc/sys/kernel/random/uuid) benthos -c ./foo.yaml.

The fields that support interpolation already have notes on their docs, but it's not particularly clear and there's currently no way to quickly scan all interpolation fields. I'll mark this as a documentation enhancement and think about how this could be made more user friendly.

[codingconcepts]

That'd be brilliant, thanks very much, @Jeffail!

Thanks for the suggestion of using env vars too, I'll see how this feels once implemented 👍