perf: convert m_nodes_mutex to a non-recursive mutex

[PastaPastaPasta]

Issue being fixed or feature implemented

If we are able to get this merged, I'll probably try to upstream this.

m_nodes_mutex is heavily contended; likely more-so in dash than bitcoin, due to master nodes aggressively sending out signature shares and related messages, and additional logic that uses stuff like ForEachNode.

Long term, I may want to convert the non-recursive mutex into a shared mutex, similar to the logic in 6468 as m_nodes is only actually mutated (hence needing an exclusive lock) when nodes are added or removed. CNode is internally thread safe.

As you can see below, m_nodes_mutex represents ~90% of all contention on ~latest nightly on testnet during tx/islock spamming. Making this into a non-recursive mutex will likely significantly reduce the time a contention takes, although, the contention itself likely won't go away.

What was done?

Convert m_nodes_mutex from RecursiveMutex -> Mutex and fix all compilation errors.

How Has This Been Tested?

This should be tested with a tsan reindex and otherwise tsan test suite, which has not been done yet.

Breaking Changes

N/a

Checklist:

Go over all the following points, and put an x in all the boxes that apply.

• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas
• I have added or updated relevant unit/integration/functional/e2e tests
• I have made corresponding changes to the documentation
• I have assigned this pull request to a milestone (for repository code-owners and collaborators only)

[github-actions]

This pull request has conflicts, please rebase.

[UdjinM6]

CI is pretty unhappy...

[PastaPastaPasta]

I had changes that fixed this, but they're on my dead system... will have to wait a week or so before I can recover them.

[github-actions]

This pull request has conflicts, please rebase.

[DashCoreAutoGuix]

ℹ️ Backport Verification - Not Applicable

This PR is not a Bitcoin Core backport and therefore does not require backport verification.

Analysis Results:

• Original Bitcoin commit: Not applicable (original Dash change)
• Reviewed commit hash: fb561ef-verify-1753727727
• PR Type: Original Dash performance improvement to m_nodes_mutex

Summary:
PR #6473 is an original Dash Core change focused on converting m_nodes_mutex from a recursive to non-recursive mutex for performance improvements. This type of change should follow the standard Dash Core review process rather than the Bitcoin backport verification workflow.

The backport verification system is designed exclusively for Bitcoin Core backports to Dash Core. Original Dash changes should be reviewed by the Dash Core development team using the standard review process.

Recommendation: Continue with standard Dash Core review process.

[PastaPastaPasta]

superseded by: #6912