x/tools/gopls: textDocument/definition is slow in large codebase #55293
Description
gopls version
golang.org/x/tools/gopls v0.9.4
golang.org/x/tools/gopls@v0.9.4 h1:YhHOxVi++ILnY+QnH9FGtRKZZrunSaR7OW8/dCp7bBk=
github.com/BurntSushi/toml@v1.2.0 h1:Rt8g24XnyGTyglgET/PRUNlrUeu9F5L+7FilkXfZgs0=
github.com/google/go-cmp@v0.5.8 h1:e6P7q2lk1O+qJJb4BtCQXlK8vWEO8V1ZeuEdJNOqZyg=
github.com/sergi/go-diff@v1.1.0 h1:we8PVUC3FE2uYfodKH/nBHMSetSfHDR6scGdBi+erh0=
golang.org/x/exp/typeparams@v0.0.0-20220722155223-a9213eeb770e h1:7Xs2YCOpMlNqSQSmrrnhlzBXIE/bpMecZplbLePTJvE=
golang.org/x/mod@v0.6.0-dev.0.20220419223038-86c51ed26bb4 h1:6zppjxzCulZykYSLyVDYbneBfbaBIQPYMevg0bEwv2s=
golang.org/x/sync@v0.0.0-20220722155255-886fb9371eb4 h1:uVc8UZUe6tr40fFVnUP5Oj+veunVezqYl9z7DYw9xzw=
golang.org/x/sys@v0.0.0-20220722155257-8c9f86f7a55f h1:v4INt8xihDGvnrfjMDVXGxw9wrfxYyCjk0KbXjhR55s=
golang.org/x/text@v0.3.7 h1:olpwvP2KacW1ZWvsR7uQhoyTYvKAupfQrRGBFM352Gk=
golang.org/x/tools@v0.1.13-0.20220812184215-3f9b119300de h1:b68wxF4nfQjj1XTRHtjVjCximbhAwjztuzDEFGU+n9o=
honnef.co/go/tools@v0.3.2 h1:ytYb4rOqyp1TSa2EPvNVwtPQJctSELKaMyLfqNP4+34=
mvdan.cc/gofumpt@v0.3.1 h1:avhhrOmv0IuvQVK7fvwV91oFSGAk5/6Po8GXTzICeu8=
mvdan.cc/xurls/v2@v2.4.0 h1:tzxjVAj+wSBmDcF6zBB7/myTy3gX9xvi8Tyr28AuQgc=
go: go1.17.6
go env
GO111MODULE="on"
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/ubuntu/.cache/go-build"
GOENV="/home/ubuntu/.config/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS="-mod=readonly"
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/ubuntu/co/backend/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/ubuntu/co/backend/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/home/ubuntu/co/backend/opt/go1.17.6"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/home/ubuntu/co/backend/opt/go1.17.6/pkg/tool/linux_amd64"
GOVCS=""
GOVERSION="go1.17.6"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/dev/null"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build457205370=/tmp/go-build -gno-record-gcc-switches"
What Happened
I noticed that go-to-definition in vscode on a go file was taking a long time, and consistently saw the following log when navigating within a package (specifically, I was using go to definition to navigate within a package. I'm operating within a large monorepo, with some heavily interlinked packages)
[Trace - 23:17:36.892 PM] Received response 'textDocument/definition - (213)' in 11214ms.
I pulled up the "Trace Information" tab in the debug server, and found a log for textDocument/Definition RPC. This log looks to consist of cache.parseGo and cache.TypeCheck events, but many of these events are replicated, seemly indicating that duplicate work is build done. I see multiple packages being parsed many times (up to ~20, looks to be dependent on imports), with all of its files listed underneath
0s textDocument/definition 13.862321388s start="textDocument/definition" method="textDocument/definition" direction="in" id="#20"
13.862319081s label= status.code="OK"
74.474µs queued 27.786µs start="queued"
178.05µs source.Identifier 13.861909568s start="source.Identifier"
...
3.12062168s cache.typeCheck 192.827411ms start="cache.typeCheck" package="foo [bar.test]"
3.120790449s cache.parseGo 196.421µs start="cache.parseGo" file="foo/file_a.go"
.... <package files> ...
...
3.590160298s cache.typeCheck 317.260966ms start="cache.typeCheck" package="foo [baz.test]"
3.590357772s cache.parseGo 274.427µs start="cache.parseGo" file="foo/file_a.go"
.... <package files> ...
...
5.043508487s cache.typeCheck 175.931042ms start="cache.typeCheck" package="foo [foo2.test]"
5.492814972s cache.parseGo 351.925µs start="cache.parseGo" file="foo/file_a.go"
.... <package files> ...
...
Appears that there's duplicate work being done - I looked through the codebase for these events and found many todos added in golang/tools@f79f3aa. (eg a b). Is there any planned work to speed up the cache.typeCheck and cache.parseGo operations?
Happy to DM the full trace file to someone if that would help!