Once LL members are fetched, they are never removed or updated until you leave the room yourself. This is not a problem as they will never replace memberships known to the sync accumulator (minus potential synapse bug causing #7450), but it keeps more members in the store than needed.