How to safely restart primary without triggering failover (e.g. to lower max_connections)?

[patmaddox]

As I understand it, Postgres max_connections has two properties that I am struggling to make work with pg_auto_failover:

1. replica values must be >= the primary
2. changing it requires restarting the server

There are a number of settings like this, where if you want to reduce them then you need to do so on the primary first, followed by the replicas. Set the value lower on the replica first and restart, and it will fail to replay the WAL file.

In other words, the sequence that you must follow for these settings is:

1. lower it in primary config file
2. restart primary
3. lower it in replicas and restart

If I'm wrong about that, please let me know!

At first I thought that PG_AUTOCTL_DEBUG=1 pg_autoctl do service restart postgres would do the trick. Sometimes it restarts fine, but other times it triggers a failover - and then the original primary is on a new timeline and needs to be rebuilt.

The best idea I can come up with is to set candidate-priority of all nodes to 0 before restarting pg_auto_failover on the primary. While it's down, the monitor node repeatedly reports something like: 01:28:30 8262 INFO Failover still in progress after all 3 candidate nodes reported their LSN and we failed to select one of them; activeNode is node 13 "my-node" (10.0.0.12:5432) and reported state "report_lsn" until it finishes starting up.

Is that the approach? I believe I've read the documentation thoroughly, and overall pgaf has been extremely positive. It's this one specific situation that I'm not quite sure about. While "restart the primary" is not a frequent case, it is part of managing a postgres cluster - and so I think it would fit within pg_auto_failover functionality. I don't know if I've overlooked it, or if it's more inferred, and done by setting all priorities to 0 so no failover takes place.