netflow.rb: cleanup even if there isn't a file#212
Open
Cireo wants to merge 1 commit intologstash-plugins:mainfrom
Open
netflow.rb: cleanup even if there isn't a file#212Cireo wants to merge 1 commit intologstash-plugins:mainfrom
Cireo wants to merge 1 commit intologstash-plugins:mainfrom
Conversation
This was a change that I've been patching locally for a couple of years.
Some details are ommited, but when packets are parsed by
logstash using the netflow codec, there are two issues.
storage of templates will slowly leak memory
The cache ttl is never checked, so it grows without bounds.
There is a workaround of doing cache cleanup by specifing a cache file.
netflow will stop processing packets in burst
This is probably not resolved, since it existed before this change.
However, the same issue of (unbounded recv-q) persist when we use a cache file.
The issue with the cache file is multi-part:
- data is keyed by template but doesn't get passed metadata host/port
- this means that sources clobber each other's templates
- this is also all for the best, or we would store 3GB of data instead of 2MB
- aside: this could even be resolved by having a multi-part lookup
template.cache.definitions :: {hash -> template} // 4 * 600 bytes, 2KB
template.cache.keys :: {key -> hash} // 3k * 30 bytes, 10KB
and then we could even have the different flow exporters not collide
- the cache is rewritten on every new template
- these come constantly (every X minutes times 2k source)
- when the 2MB cache is rewritten it takes a mutex lock
- the file just flickers in and out of existence as fast as possible
This grinds the entire processing to a halt. Even when changing the
cache to only be rewritten if a new key is present it didn't resolve the
core issues.
|
💚 CLA has been signed |
Author
I did sign this while creating the commit, perhaps a timing issue? Please re-trigger. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This was a change that I've been patching locally for a couple of years.
Some details are ommited, but when packets are parsed by logstash using the netflow codec, there are two issues.
The issue with the cache file is multi-part:
template.cache.definitions :: {hash -> template} // 4 * 600 bytes, 2KBtemplate.cache.keys :: {key -> hash} // 3k * 30 bytes, 10KBand then we could even have the different flow exporters not collide
This grinds the entire processing to a halt. Even when changing the cache to only be rewritten if a new key is present it didn't resolve the core issues.