-
-
Notifications
You must be signed in to change notification settings - Fork 367
Closed
Description
Right now, run_in_worker_thread
just always spawns a new thread for the operation, and then kills it after. This might sound ridiculous, but it's not so obviously wrong as it looks! There's a large comment in trio._threads
talking about some of the issues.
Questions:
- Should there be any global control over worker thread spawning?
- If there is, should it be a strict limit on the number of threads, or something more subtle like a limiter on the rate at which they spawn?
- How do administrators configure this stuff? Or instrument it?
- What should the API to interact with it look like, e.g. do we need a way for a specific
run_in_worker_thread
to say that it shouldn't block waiting for a thread because it might unblock a thread?
Prior art: https://twistedmatrix.com/trac/ticket/5298
Interacting with the products at Rackspace which use Twisted, I've seen problems caused by thread-pool maximum sizes with some annoying regularity. The basic problem is this: if you have a hard limit on the number of threads, it is not possible to write a correct program which may require starting a new thread to un-block a blocked pool thread - glyph