When we send an invite we need to check the returned invite's signatures. This eventually boils down to:

Which then tries and checks that the server has in fact signed it and its a valid signature.

Seemingly, for servers that have disabled perspectives, the server then tries and to fetch its own verify keys (which in the particular case I saw failed to fetch due to a mismatch in TLS certificates). It probably shouldn't bother trying to fetch its own keys.