cut test time with coverage by half

Per the inline comment, we want to build every package in the test suite
at least once, otherwise we won't get proper code coverage information.
For example, some code only triggers when obfuscating the runtime,
and if I run "go test" twice in a row, the second might reuse the
first's obfuscation of the runtime and skip the code entirely.

However, per the TODO, we used to solve this in a rather wasteful way,
by making each test use its own separate GOCACHE directory.
Instead, use a new GOCACHE directory, but share it between tests.
This is still enough, as we still build each package at least once.

Running the command below twice in a row with Go 1.20:

	go test -cover

took about 1m55s on my laptop before, and now takes about 1m10s.
Not great, but a noticeable improvement.
Both report a total coverage of 88.7%.

While here, do the same for GARBLE_CACHE_DIR,
and use testscript.Env.Setenv for simplicity.
pull/682/head
Daniel Martí 1 year ago
parent 86b7e334ba
commit bb69facbd8

@ -61,7 +61,9 @@ func TestScript(t *testing.T) {
t.Fatal(err)
}
userCacheDir, err := os.UserCacheDir()
tempCacheDir := t.TempDir()
hostCacheDir, err := os.UserCacheDir()
if err != nil {
t.Fatal(err)
}
@ -69,38 +71,36 @@ func TestScript(t *testing.T) {
p := testscript.Params{
Dir: filepath.Join("testdata", "script"),
Setup: func(env *testscript.Env) error {
env.Vars = append(env.Vars,
// Use testdata/mod as our module proxy.
"GOPROXY="+proxyURL,
// We use our own proxy, so avoid sum.golang.org.
"GONOSUMDB=*",
// "go build" starts many short-lived Go processes,
// such as asm, buildid, compile, and link.
// They don't allocate huge amounts of memory,
// and they'll exit within seconds,
// so using the GC is basically a waste of CPU.
// Turn it off entirely, releasing memory on exit.
//
// We don't want this setting always on,
// as it could result in memory problems for users.
// But it helps for our test suite,
// as the packages are relatively small.
"GOGC=off",
"gofullversion="+runtime.Version(),
"EXEC_PATH="+execPath,
"GARBLE_CACHE_DIR="+userCacheDir,
)
// Use testdata/mod as our module proxy.
env.Setenv("GOPROXY", proxyURL)
// We use our own proxy, so avoid sum.golang.org.
env.Setenv("GONOSUMDB", "*")
// "go build" starts many short-lived Go processes,
// such as asm, buildid, compile, and link.
// They don't allocate huge amounts of memory,
// and they'll exit within seconds,
// so using the GC is basically a waste of CPU.
// Turn it off entirely, releasing memory on exit.
//
// We don't want this setting always on,
// as it could result in memory problems for users.
// But it helps for our test suite,
// as the packages are relatively small.
env.Setenv("GOGC", "off")
env.Setenv("gofullversion", runtime.Version())
env.Setenv("EXEC_PATH", execPath)
if os.Getenv("GOCOVERDIR") != "" {
// Don't reuse the build cache if we want to collect
// code coverage. Otherwise, many toolexec calls would
// be avoided and the coverage would be incomplete.
// TODO: to not make "go test" insanely slow, we could still use
// an empty GOCACHE, but share it between all the test scripts.
env.Vars = append(env.Vars, "GOCACHE="+filepath.Join(env.WorkDir, "go-cache-tmp"))
// Don't share cache dirs with the host if we want to collect code
// coverage. Otherwise, the coverage info might be incomplete.
env.Setenv("GOCACHE", filepath.Join(tempCacheDir, "go-cache"))
env.Setenv("GARBLE_CACHE_DIR", filepath.Join(tempCacheDir, "garble-cache"))
} else {
// GOCACHE is initialized by gotooltest to use the host's cache.
env.Setenv("GARBLE_CACHE_DIR", hostCacheDir)
}
return nil
},

Loading…
Cancel
Save