Skip to content

Pool grows beyond max_size, hits open files limit #171

@liskin

Description

@liskin

Here's a straightforward reproducer:

use std::{panic, thread, time::Duration};

fn main() {
    delay_main_panic();

    for _ in 1..250 {
        let srv = mockito::Server::new();
        std::mem::drop(srv);
    }

    println!("All good!");

    // maybe some fds will get closed? probably not.
    thread::sleep(Duration::from_secs(5));
}

// workaround for panic messages from different threads getting mixed together
fn delay_main_panic() {
    let def_panic_hook = panic::take_hook();
    panic::set_hook(Box::new(move |info| {
        if thread::current().name() == Some("main") {
            thread::sleep(Duration::from_secs(1));
        }
        def_panic_hook(info)
    }));
}

This results in the following on a Linux system with ulimit -n being 1024:

thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: Error { kind: ServerFailure, context: Some("Too many open files (os error 24)") }', /home/tomi/src/mockito/src/server.rs:238:56
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Error { kind: ServerFailure, context: Some("the server is not running (context: receiving on a closed channel)") }', src/main.rs:7:19

liskin@ea7e583 demonstrates why this is happening:

  • self.created is never being incremented, therefore self.created < self.max_size always evaluates to true
  • thus a new Server is always created and added to the pool whenever a Server is requested, regardless of the pool size and regardless of whether any recycled servers are available
  • every server uses 5 file descriptors (and at least one thread), so ~200 calls to mockito::Server::new exhaust the limit (on MacOS, the default limit is much lower, so just 50 calls exhausts it)

Ideally, the pool would first try to satisfy a request using any available (recycled) servers, thus keeping the resource (file descriptor, thread, memory) usage minimal. If there are no available servers, only then would it create new ones on demand.

(I believe this is how deadpool behaves. What was wrong with it?)

(Unrelated, but when playing around with the code I did this and it compiles and tests pass. Being quite new to Rust, I'm guessing I'm missing an important reason for those clones… What is it?)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions