Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

COMBINEDAPACHELOG leading to grokparsefailures due to Email as Username #196

Open
bne86 opened this issue Feb 8, 2017 · 0 comments
Open
Labels

Comments

@bne86
Copy link

bne86 commented Feb 8, 2017

Hi,
i have issues with our ELK stack analyzing some httpd logs. For users that are authenticated with their email address (see sample data) the filter does not work. Some info on the environment:

  • Version: all 5.2
  • Operating System: Docker containers running on CentOS 7
  • Config File:
   mutate { replace => { type => "apache_access" } }
   grok {
      match => [
      "message" , "%{COMBINEDAPACHELOG}+%{GREEDYDATA:extra_fields}",
      "message" , "%{COMBINEDAPACHELOG}",
      "message" , "%{COMMONAPACHELOG}+%{GREEDYDATA:extra_fields}"
      ]
      overwrite => [ "message" ]
   }
   geoip {
      source => "clientip"
      target => "geoip"
      add_field => [ "[geoip][coordinates]", "%{[geoip][longitude]}" ]
      add_field => [ "[geoip][coordinates]", "%{[geoip][latitude]}"  ]
    }
    mutate {
      convert => ["response", "integer"]
      convert => ["bytes", "integer"]
      convert => ["responsetime", "float"]
      convert => [ "[geoip][coordinates]", "float"]
   }
   useragent {
      source => "agent"
   }
  • Sample Data:
    Working sample data:
    123.123.123.123 - bvonstvieth [08/Feb/2017:14:32:15 +0100] "PROPFIND /remote.php/webdav/ HTTP/1.1" 207 408 "-" "Mozilla/5.0 (Macintosh) mirall/2.2.4 (build 3709)"
    Not working data leading to _grokparsefailure:
    123.123.123.123 - b.von.st.vieth@example.com [08/Feb/2017:13:50:14 +0100] "PROPFIND /remote.php/webdav/ HTTP/1.1" 207 389 "-" "Mozilla/5.0 (Macintosh) mirall/2.2.4 (build 3709)"
  • Steps to Reproduce:
    Put logstash on /var/log/httpd and see what happens.

Taking a look at logstash-patterns-core and copying it to my grok filter makes it working, but shouldn't it be already in place?
match => [ "message" , "%{IPORHOST:clientip} %{HTTPDUSER:ident} %{HTTPDUSER:auth} \[%{HTTPDATE:timestamp}\] \"(?:%{WORD:verb} %{NOTSPACE:request}(?: HTTP/%{NUMBER:httpversion})?|%{DATA:rawrequest})\" %{NUMBER:response} (?:%{NUMBER:bytes}|-)" ]

@kares kares added the bug label Jul 30, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants