Binaries:
polkadot 0.9.23-a7e188cd966 from default apt repo https://releases.parity.io/deb
prometheus, version 2.34.0
grafana from apt repo version 8.5.5 (commit: d32ae18909, branch: HEAD)

Server:
Ubuntu 20.04 LTS 5.4.0-113-generic #127-Ubuntu SMP x86_64 on a dedicated AMD server, 64 MB ram.

Description:
When getting in the active validator set the prometheus metric endpoint "polkadot_node_is_authority" switches to 1, but when you get out of the active set it does not switch back to 0. If you restart the node it goes to 0 and stays at 0. Witnessed this on 0.9.23 and also before on 0.9.22. The "polkadot_node_is_parachain_validator" endpoint does switch on and off depending on the paravalidating state. Heard rumors that if you end the authority session in a parachain validator state it also stays on 1 but can't confirm because all my parachain sessions ended before the authority ended.

Expected behaviour: when you get out of the active set and are not longer authority I would expect the "polkadot_node_is_authority" switch back to 0.

I've heard in the 1000 validator matrix group it is a known problem but I could not find a previous bug report, if it does exist my excuses for the duplicate. A fellow validator uses the "overseer rate" rate but this seems to change between version so is not perfect.

It seems to be included here #4699 by @sandreim