Go "tinker mode" proposal.

Last updated: 2021/04/11

Rationale:

@cosban wrote: I need to be able to, without being forced to commit code that is not fully vetted, build and test our modifications. (@cosban)

@rsc wrote: GOPATH is holding back the ecosystem and the toolchain. It's time to retire it. [...] we're happy to listen. But GOPATH needs to go. (@rsc)

@bcmills wrote: Many module issues and questions seem to center on editing, testing, and deploying multiple (possibly mutually-interdependent, possibly cyclic) modules (@bcmills)

The main workaround at the moment is to add replace directives among the modules to be edited, but maintaining those directives is tedious and error-prone. @rogpeppe's gohack tool automates away some of the tedium, but doesn't seem to remove the risk of accidentally checking in a go.mod with what were intended to be local, temporary replacements.

Related: #26640, #37755, #26377, #25053.

If an environment variable named GOTINKER is defined and set to an absolute path to the existing location on the local filesystem, and the import path of an include can be found under $GOTINKER/ directory, then build commands treat the $GOTINKER/src/import/path as a final authoritative source of the import; foregoing both vendor/ and any go.mod directive perpeting to this import/path.

Ie. build commands like 'go build' and 'go test' will compile modules present in the $GOTINKER directory instead of accessing the network, local module cache, or vendor directory.

Under tinker mode GOBIN, GOCACHE, GOMODCACHE, and GOENV are bound to locations relative to the GOTINKER: $GOTINKER/bin, $GOTINKER/cache, $GOTINKER/pkg/mod, and $GOTINKER/goenv - respectively.

$GOTINKER tree should be populated by the user. For yet some time to come the last version of Go to support GOPATH can be used to ease this task, ie. GOROOT=/where/go1.16 GO111MODULE=off GOPATH="$GOTINKER" go1.16 get

GOTINKER path last element may start with an underscore character so experiments can be kept inside any project tree.

Security considerations

Both object code and the executable built under the tinker mode should not accidentally leak to the production environment. Ie. while objects are built and cached under $GOTINKER, then built executable MUST be amended (by the compiler) to refuse to run in an environment where GOTINKER is not set, or it is set but does not match the last element of the GOTINKER path that was compiled in.

Ie. "tinkered with" executable preserves the last part of the GOTINKER path then matches it to the last part of GOTINKER string where it runs. If these do not match, exectutable exits immediately with "Experimental but GOTINKER is not set or did not match" error message.

How GOTINKER workflow is different to GOPATH''s one?

GOPATH is all-or-nothing regarding versions. Ie. under GOPATH tools operate on code as-is. If we are about debugging or changing interdependent code, it is up to us to do proper checkouts of everything our — possibly big — app or service uses.

In proposed GOTINKER mode tools operate on modules as usual, so anything else but code we pulled under GOTINKER path is kept at version specified in respective go.mod. All bookkeeping is done for us, nothing will drift apart or leak accidentally.

edits:

• 2021/02/17 added "match compiled in GOTINKER with run one" in Security
• 2021/02/18 be explicit about possibility to keep GOTINKER tree inside a module tree, eg. in a _tinker/ subdirectory.
• 2021/04/11 compare GOPATH and GOTINKER workflows.