Collector/agent in an embedded system

[utezduyar]

Hi. We have an embedded platform that is powerful enough to run deep learning, linux, systemd, apache web server etc. We have different products but what we are talking about is around 1 GB volatile and persistent memory.
Arm64/linux version of the collector (otelcol 0.68.0) is 87.9 MB which is quite large for our system.

I am aware that collector can run in agent mode. Is there an agent binary that can be built/downloaded or is agent a runtime configuration which in my case is not going to help with the size problem.
Otherwise what are my options to take down the size of the collector?

[jpkrohling]

arm64 linux for contrib is currently at 45.1 MB, core is at 21.5 MB. If that's still too big for you, you can have your own distribution of the collector and include only the components you need.

For comparison, a pure OTLP distribution (otlp receiver and otlp exporter, plus health check extension) is at 5.93 MB. Reference: https://github.com/grafana/opentelemetry-collector-components/blob/main/distributions/sidecar/manifest.yaml

[utezduyar]

Note that the numbers you are reporting are for the compressed files.
Thanks for the tips about creating my own distribution.

[jpkrohling]

Which reminded me of something: you can also use upx to compress the binaries even further if space is the biggest concern. I remember using it with great results, except that using it on binaries for Mac would yield weird runtime errors, but perhaps you can use it safely given proper testing?

https://upx.github.io/

[utezduyar]

Thanks for the tip and we indeed use upx on mac and linux desktop applications. We also use it in some applications in our embedded linux product while keeping in mind the tradeoffs of UPX.
UPX helps very much on the persistent storage size but when it is extracted to the volatile memory (RAM), the pages of the binary are not marked as if they belong to text section of the program therefore they are never thrown away (we don't have swap) in memory pressure.

[utezduyar]

@jpkrohling Hi. You have written on Slack channel that your custom collector is 8 MB. Was this with UPX? I am struggling to get lower than 18MB.

20829730  otelcol-custom (Darwin binary)
18284544  otelcol-custom-linux (ARM64 on linux, aarch64, stripped)

./otelcol-custom components
buildinfo:
    command: otelcol-custom
    description: Custom OpenTelemetry Collector distribution
    version: 1.0.0
receivers:
    - otlp
processors: []
exporters:
    - logging
extensions: []

[jpkrohling]

Sorry, as others pointed out (in Slack, likely), the numbers I was providing was the tar.gz, not the actual binary. The actual Darwin ARM64 I have is 22MiB.

Here's the reference I have: https://github.com/grafana/opentelemetry-collector-components/releases/tag/v0.0.2