panic when checking lots of files #166
Description
If I create a file with all the .go files I want to check, and use xargs thus:
xargs < .all-files gofumpt -d
as of a few minutes ago gofumpt will panic with
panic: invalid Pos value 1 (should be in [774665, 781950]) [recovered]
panic: invalid Pos value 1 (should be in [774665, 781950])
goroutine 1 [running]:
golang.org/x/tools/go/ast/astutil.Apply.func1()
…/pkg/mod/golang.org/x/tools@v0.1.8-0.20211102182255-bb4add04ddef/go/ast/astutil/rewrite.go:49 +0x89
panic({0x60bb20, 0xc000540770})
/home/john/sdk/go1.17.1/src/runtime/panic.go:1038 +0x215
go/token.(*File).Offset(0xc00054a3c0, 0x2)
/home/john/sdk/go1.17.1/src/go/token/position.go:281 +0xeb
mvdan.cc/gofumpt/format.(*fumpter).addNewline(0xc0001875c0, 0xc000540430)
…/pkg/mod/mvdan.cc/gofumpt@v0.2.0/format/format.go:213 +0x3a
mvdan.cc/gofumpt/format.(*fumpter).applyPre(0xc0001875c0, 0xc00054a4f0)
…/pkg/mod/mvdan.cc/gofumpt@v0.2.0/format/format.go:558 +0x12f5
mvdan.cc/gofumpt/format.File.func1(0xc00054a4f0)
…/pkg/mod/mvdan.cc/gofumpt@v0.2.0/format/format.go:104 +0x3a
golang.org/x/tools/go/ast/astutil.(*application).apply(0xc00054a4e0, {0x696ca0, 0xc000184a50}, {0x649e41, 0x18}, 0xc0001c7378, {0x6969d0, 0xc000184a20})
…/pkg/mod/golang.org/x/tools@v0.1.8-0.20211102182255-bb4add04ddef/go/ast/astutil/rewrite.go:200 +0x202
golang.org/x/tools/go/ast/astutil.(*application).apply(0xc00054a4e0, {0x696c50, 0xc000431180}, {0x64a094, 0xc000431180}, 0x0, {0x696ca0, 0xc000184a50})
…/pkg/mod/golang.org/x/tools@v0.1.8-0.20211102182255-bb4add04ddef/go/ast/astutil/rewrite.go:424 +0x17fd
golang.org/x/tools/go/ast/astutil.(*application).applyList(0xc00054a4e0, {0x696c50, 0xc000431180}, {0x64a094, 0x5})
…/pkg/mod/golang.org/x/tools@v0.1.8-0.20211102182255-bb4add04ddef/go/ast/astutil/rewrite.go:479 +0xae
golang.org/x/tools/go/ast/astutil.(*application).apply(0xc00054a4e0, {0x6974c0, 0xc000540740}, {0x649ea5, 0x18}, 0x18, {0x696c50, 0xc000431180})
…/pkg/mod/golang.org/x/tools@v0.1.8-0.20211102182255-bb4add04ddef/go/ast/astutil/rewrite.go:430 +0xb0d
golang.org/x/tools/go/ast/astutil.Apply({0x696c50, 0xc000431180}, 0xc0005458c0, 0xc0005458d8)
…/pkg/mod/golang.org/x/tools@v0.1.8-0.20211102182255-bb4add04ddef/go/ast/astutil/rewrite.go:54 +0x166
mvdan.cc/gofumpt/format.File(0xc0000b6440, 0xc000431180, {{0xc0000240f0, 0xc00016cf00}, 0xa0})
…/pkg/mod/mvdan.cc/gofumpt@v0.2.0/format/format.go:152 +0x335
main.processFile({0x7fff6f0bca65, 0x36}, {0x0, 0x0}, {0x692e00, 0xc0000b2008}, 0x0)
…/pkg/mod/mvdan.cc/gofumpt@v0.2.0/gofmt.go:130 +0x388
main.gofumptMain()
…/pkg/mod/mvdan.cc/gofumpt@v0.2.0/gofmt.go:242 +0x458
main.main()
…/pkg/mod/mvdan.cc/gofumpt@v0.2.0/gofmt.go:195 +0x19
however feeding it all the files one at a time (i.e. xargs -n1) does not trigger this behaviour. Larger batch sizes also work.
Shuffling the list of files will trigger it at a different place, e.g. while
xargs -n 1061 gofumpt -d < .all-files
will work (and -n1062 will fail),
shuf .all-files | xargs -n 1061 gofumpt -d
will fail more often than not.