WIP: Add ansible runner logs to ELK #372
Conversation
|
Build failed. |
ConsoleCatzirl
left a comment
ConsoleCatzirl
left a comment
There was a problem hiding this comment.
A good start, but we need a little more than this.
bastion.yml
Outdated
| - input_type: log | ||
| document_type: ansible_runner | ||
| paths: | ||
| - "/var/www/html/cron-logs/$env/*.log" |
There was a problem hiding this comment.
We should only track either the current or the latest log, but if we track all logs in the directory then we will get a bunch of duplicate entries after every log rotation. Also there is nothing setting $env, so that should be another * instead.
Also, the document type is used by logstash to know how to parse the logs. Using ansible_runner for the type is correct, but doesn't have a matching logstash filter yet, we will need to add that as well. We will need a change similar to 808c70c to add the new log parsing. And we can probably reuse the exact same multiline configuration from that change since it is based on lines starting with a timestamp. The tricky part is the grok pattern, we can chat about that in irc or slack.
|
Build failed. |
|
Build failed. |
| filter { | ||
| if [type] == "ansible-runner" { | ||
| grok { | ||
| match => { "message" => "%{SUPERAWESOMEANSIBLELOG}"} |
There was a problem hiding this comment.
I'll take a stab at writing our super-awesome ansible-log pattern
Make the filebeat document-type match the logstash filter type. Remove the date filter since we don't have timestamps to parse, which means we have to rely on logstash received time for event timestamps. Create multiline events split by empty lines in the log file. Add multiple grok patterns for the various types of output we get from ansible-playbook. Signed-off-by: K Jonathan Harker <Jonathan.Harker@ibm.com>
Better ansible grokking
|
Build failed. |
3 participants
fixes: BonnyCI/projman#28