Perhaps allow users /organizations to create a follow-up conversation and send notifications to the existing participants?