Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[inputs.tail] Behaviour when from_beginning and statefile coexist is not as expected. #16237

Open
squ opened this issue Nov 26, 2024 · 0 comments · May be fixed by #16240
Open

[inputs.tail] Behaviour when from_beginning and statefile coexist is not as expected. #16237

squ opened this issue Nov 26, 2024 · 0 comments · May be fixed by #16240

Comments

@squ
Copy link

squ commented Nov 26, 2024 •
edited
Loading

Relevant telegraf.conf

[agent]
  statefile = "tail.state"

[[inputs.tail]]
  files = ["c:\\tmp\\mytest.log"]
  from_beginning = true
  watch_method = "poll"
  data_format = "value"
  data_type = "string"

Logs from Telegraf

2024-11-26T02:38:49Z I! Loading config: mytest.conf
2024-11-26T02:38:49Z I! Starting Telegraf 1.33.0-eb2afadc (customized) brought to you by InfluxData the makers of InfluxDB
2024-11-26T02:38:49Z I! Available plugins: 3 inputs, 0 aggregators, 0 processors, 2 parsers, 1 outputs, 0 secret-stores
2024-11-26T02:38:49Z I! Loaded inputs: tail
2024-11-26T02:38:49Z I! Loaded aggregators:
2024-11-26T02:38:49Z I! Loaded processors:
2024-11-26T02:38:49Z I! Loaded secretstores:
2024-11-26T02:38:49Z I! Loaded outputs: http
2024-11-26T02:38:49Z I! Tags enabled: host=WX-C-002TK
2024-11-26T02:38:49Z I! [agent] Config: Interval:10s, Quiet:false, Hostname:"WX-C-002TK", Flush Interval:10s
2024-11-26T02:38:49Z D! [agent] Initializing plugins
2024-11-26T02:38:49Z D! [agent] Initializing plugin states
2024-11-26T02:38:49Z D! [agent] Connecting outputs
2024-11-26T02:38:49Z D! [agent] Attempting connection to [outputs.http]
2024-11-26T02:38:49Z D! [agent] Successfully connected to outputs.http
2024-11-26T02:38:49Z D! [agent] Starting service inputs
2024-11-26T02:38:49Z D! [inputs.tail] Tail added for "c:\\tmp\\mytest.log"
2024-11-26T02:38:58Z D! [agent] Stopping service inputs
2024-11-26T02:38:58Z D! [inputs.tail] Tail removed for "c:\\tmp\\mytest.log"
2024-11-26T02:38:58Z D! [agent] Input channel closed
2024-11-26T02:38:58Z I! [agent] Hang on, flushing any cached metrics before shutdown
2024-11-26T02:38:58Z D! [outputs.http] Wrote batch of 30 metrics in 85.5889ms
2024-11-26T02:38:58Z D! [outputs.http] Buffer fullness: 0 / 10000 metrics
2024-11-26T02:38:58Z I! [agent] Stopping running outputs
2024-11-26T02:38:58Z D! [agent] Persisting plugin states
2024-11-26T02:38:58Z D! [agent] Stopped Successfully

System info

Windows 11, Telegraf 1.33.0-eb2afadc (git: master@eb2afadc)

Docker

N/A

Steps to reproduce

  1. set from_beginning = true, set a statefile to persist the plugin state.
  2. Start telegraf with above config and Stop
  3. Start telegraf again. It will send the log from the beginning, regardless of the statefile record. In fact the statefile is also empty. Statefile content: {"0f3361e8c8701c3bbb9ce162820ea26a2211a3ab9bb0443ec6469029cd43fc39":"e30="}

Expected behavior

These two options are able to co-exist. When I set both from_begin and statefile:

  • If there is a record position in the statefile, it reads from the record position.
  • If the statefile doesn't exist or there is no record location, it reads from the beginning.

Actual behavior

With from_begin=true set, it doesn't log the statefile. It looks like there's a conflict between these two options.

For the original post

As my test with the latest version, Test2 was fixed, but Test1 was not. And my expectation is still there:

This should be fixed by #16123 in Telegraf 1.32.3, since you were able to test that it is fixed on master I will close this issue for now, feel free to reopen if this isn't the case!

Originally posted by @squ in #16225 (comment)
@squ squ changed the title As my test with the latest version, Test2 was fixed, but Test1 was not. And my expectation is still there: [inputs.tail] Behaviour when from_beginning and statefile coexist is not as expected. Nov 26, 2024
This was referenced Nov 28, 2024
Closed
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
1 participant
@squ