fix: after the machine reboot, vNPU cannot be deleted, resulting in idle resource occupation

[dartagnanli]

After the machine restarts, there is no time to execute the destroy method of Ascend, resulting in the vNPU being idle. When the model is rescheduled, a new vNPU will be created. This situation can lead to the model service pod not starting up when resources are insufficient. It is necessary to manually delete the vNPU that was not deleted before the last restart. Here, I have added a logic to periodically check for idle vNPU. When an idle vNPU is found, the deletion logic will be executed.

In the image below, the vNPU with a status of 0 represents the idle vNPU left after a restart. These vNPU were used before the model restart, and kubelet did not have enough time to execute the post-hook method of the container during the restart

[hami-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: dartagnanli
Once this PR has been reviewed and has the lgtm label, please assign archlitchi for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[hami-robot]

Welcome @dartagnanli! It looks like this is your first PR to Project-HAMi/ascend-device-plugin 🎉

[DSFans2014]

please sign-off your commit