git_status_list is slower than git status #4230
Description
Reproduction steps
Build provided libgit2status tool (I'll post it source in the next comment to keep initial report easier to read) and run it in the root of large enough repo. Compare it speed with git status.
% go version
go version go1.8.1 linux/amd64
% git --version
git version 2.10.2
% git clone https://github.com/gentoo/gentoo.git
% cd gentoo
% find -type f | wc -l
99540
% find -type d | wc -l
27306
% git config core.untrackedCache false
% time git status --short
0.257 real 0.128 user 0.206 sys 32MB RAM
% git config core.untrackedCache true
% time git status --short
0.097 real 0.067 user 0.110 sys 38MB RAM
% time libgit2status
{AddedFiles:0 ModifiedFiles:0 DeletedFiles:0 RenamedFiles:0 UnmergedFiles:0 HasUntrackedFiles:false}
0.626 real 0.340 user 0.282 sys 154MB RAM
% touch profiles/base/testfile.added
% touch profiles/base/testfile.new
% git add profiles/base/testfile.added
% time git status --short
A profiles/base/testfile.added
?? profiles/base/testfile.new
0.115 real 0.082 user 0.108 sys 38MB RAM
% git config core.untrackedCache false
% time git status --short
A profiles/base/testfile.added
?? profiles/base/testfile.new
0.255 real 0.134 user 0.193 sys 32MB RAM
% time libgit2status
{AddedFiles:1 ModifiedFiles:0 DeletedFiles:0 RenamedFiles:0 UnmergedFiles:0 HasUntrackedFiles:true}
0.611 real 0.358 user 0.250 sys 154MB RAM
Expected behavior
Well… in my dream git2go should works at least as fast as git status… actually faster if we just needs to know is workdir is dirty or is it has any untracked files - without needs to know full list of all added/modified/untracked/etc. files output by git status.
Actual behavior
It's nearly 2.5 times slower than git status on my system (HDD, i7-2600K overcloked @ 4.5GHz) without untrackedCache and 5-6 times slower with untrackedCache.
While strace-ing git status I've noticed it calls lstat from 2 or 3 threads - can this be the reason?
For all timed runs above I take average result from several runs and my system has enough RAM to keep dircache for the full repo, so there was no (noticeable, at least) disk accesses, and thus HDD/SSD shouldn't make any difference.
Version of libgit2 (release number or SHA1)
This version is vendored by git2go. I've tried to update to current master but get this error:
./submodule.go:339: ptr.clone_checkout_strategy undefined (type *C.struct_git_submodule_update_options has no field or method clone_checkout_strategy)
If something was just renamed then it probably can be easily fixed to allow testing with latest master.
Operating system(s) tested
Hardened Gentoo Linux, amd64.