Skip to content

handling SIGTERMs when dealing with long requests  #932

@everplays

Description

@everplays

Hi there,

TL;DR:

How can we receive SIGTERMs in the app itself and still let puma gracefully exit after request was processed?

Long version:

We are using Amazon SQS as backend of our job queue. Essentially, it runs the app just as it would run a web app instance but instead of directing normal traffic, it makes requests that are background jobs. Everything is working fine until we deploy or elastic beanstalk tries to scale. In either of the scenarios, they issue a SIGTERM to let the app finish what it is doing before SIGKILL but it's a long running request, it's meant to run for a few minutes! Therefore, puma doesn't get a chance to do a graceful stop as the execution is still in the app. What I am looking for is a way to communicate to Puma from within the app. Either of the following can work for us:

‌1. App asks puma if it has received a SIGTERM and if that was the case, stops what it's doing:

def I_am_an_action_that_takes_5_minutes_to_run
  (1..300).each do
    sleep 1
    if env['puma.control'].status == :stop
      put_the_job_back_to_queue!
      break
    end
  end
end

‌2. App traps the signals (which prevents Puma from receiving them), handles the situation, and tells puma to stop:

def I_am_an_action_that_takes_5_minutes_to_run
  (1..300).each do
    sleep 1
    SignalHandler.check!
  end
rescue SignalException => e
  put_the_job_back_to_queue!
end

# and SignalHandler.check! does something like the following in case of SIGTERM
if received_sigterm?
  # this is essential, otherwise, puma keeps running after request.
  env['puma.control'].set_status(:stop)
  raise SignalException, 15
end

Please let me know what you think and if you have a better suggestion. I would like to make a PR to address this.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions