Skip to content

Raise exception when we can't get get stats or info from a node#5

Open
bender-the-greatest wants to merge 5 commits intodevelopfrom
stop_on_bad_stats
Open

Raise exception when we can't get get stats or info from a node#5
bender-the-greatest wants to merge 5 commits intodevelopfrom
stop_on_bad_stats

Conversation

@bender-the-greatest
Copy link
Contributor

No description provided.

@akatrevorjay
Copy link
Contributor

This is going to happen sometimes if a node is down or even memory hogged, so that's why it continues.

@bender-the-greatest
Copy link
Contributor Author

I updated the node.py to keep track of how many times a node fails to populate in a row, and raise an exception if the count exceeds 10 times.

'''
self.cluster = cluster
self.node_id = node_id
self.badcount = 0
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This desperately needs a better name. What are we trying to say with this? We're trying to indicate that we've had unsuccessful or bad attempts to get info, so we should say something to that affect.

unsuccessful_cluster_info_retrieval_attempts = 0

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with this fully (although there has to be a smaller name) at the very least keep it pythonic ;)

self.bad_result_count = 0
except Exception as e:
self.bad_result_count += 1
_LOG.warning("{0} (failure {1} of {2}".format(str(e), self.bad_result_count, self.bad_result_maximum))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use sprintf that's already being handled by logging, otherwise it looks ugly (and is two different format string methods being applied)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also the format string has a left parentheses in there but no right?

@akatrevorjay
Copy link
Contributor

FYI, the reasoning for it not failing upon highstate failure, was because the dpkg post-install script from Elastic themselves failed out upon upgrade once upon a time.

@akatrevorjay
Copy link
Contributor

So the check was removed on purpose and never added back in.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants