Skip to content

fd starts slowly on systems with very many cores #1203

@tavianator

Description

@tavianator

Originally posted by @pmarreck in #1131 (comment)

I use fd in some test suites and I noticed a slowdown. I investigated and I found that simply running fd with no arguments in an empty directory on /tmp (which is in-memory!) was taking 0.2 seconds. This is ridiculously slow on my machine (Ryzen Threadripper 3990x with 64 cores/128 threads running on NVMe). Multiplied by the number of times I was calling fd in my suite, this immediately explained why multiple seconds had been added at some point.

When I add the -j1 argument, as someone else indicated, the boot runtime of running fd alone drops to 0.01 seconds, which is 20x faster and far more reasonable an expectation.

I'm on NixOS unstable running on ZFS root.

@matu3ba & @tavianator , this is NOT a filesystem problem (IMHO). This looks like a heuristics bug regarding non-specification of the number of concurrent jobs to run (per the mandocs, not specifying it "uses heuristics")

At least I now know a workaround that works!

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions