|
|
|
|
env GOPRIVATE=match-absolutely/nothing
|
|
|
|
|
! garble build -o=out ./standalone
|
|
|
|
|
stderr '^GOPRIVATE="match-absolutely/nothing" does not match any packages to be built$'
|
|
|
|
|
|
|
|
|
|
env GOPRIVATE=test/main/imported
|
|
|
|
|
! garble build ./importer
|
|
|
|
|
stderr '^public package "test/main/importer" can''t depend on obfuscated package "test/main/imported" \(matched via GOPRIVATE="test/main/imported"\)$'
|
|
|
|
|
|
|
|
|
|
[short] stop # rebuilding std is slow
|
|
|
|
|
|
|
|
|
|
env GOPRIVATE='*'
|
|
|
|
|
|
|
|
|
|
# Try garbling all of std, given some std packages.
|
|
|
|
|
# No need for a main package here; building the std packages directly works the
|
|
|
|
|
# same, and is faster as we don't need to link a binary.
|
|
|
|
|
# This used to cause multiple errors, mainly since std vendors some external
|
|
|
|
|
# packages so we must properly support ImportMap.
|
|
|
|
|
# Plus, some packages like net make heavy use of complex features like Cgo.
|
|
|
|
|
# Note that we won't obfuscate a few std packages just yet, mainly those around runtime.
|
|
|
|
|
garble build std
|
|
|
|
|
|
|
|
|
|
# Link a binary importing net/http, which will catch whether or not we
|
|
|
|
|
# support ImportMap when linking.
|
|
|
|
|
garble build -o=out ./stdimporter
|
|
|
|
|
|
avoid reproducibility issues with full rebuilds
We were using temporary filenames for modified Go and assembly files.
For example, an obfuscated "encoding/json/encode.go" would end up as:
/tmp/garble-shared123/encode.go.456.go
where "123" and "456" are random numbers, usually longer.
This was usually fine for two reasons:
1) We would add "/tmp/garble-shared123/" to -trimpath, so the temporary
directory and its random number would be invisible.
2) We would add "//line" directives to the source files, replacing
the filename with obfuscated versions excluding any random number.
Unfortunately, this broke in multiple ways. Most notably, assembly files
do not have any line directives, and it's not clear that there's any
support for them. So the random number in their basename could end up in
the binary, breaking reproducibility.
Another issue is that the -trimpath addition described above was only
done for cmd/compile, not cmd/asm, so assembly filenames included the
randomized temporary directory.
To fix the issues above, the same "encoding/json/encode.go" would now
end up as:
/tmp/garble-shared123/encoding/json/encode.go
Such a path is still unique even though the "456" random number is gone,
as import paths are unique within a single build.
This fixes issues with the base name of each file, so we no longer rely
on line directives as the only way to remove the second original random
number.
We still rely on -trimpath to get rid of the temporary directory in
filenames. To fix its problem with assembly files, also amend the
-trimpath flag when running the assembler tool.
Finally, add a test that reproducible builds still work when a full
rebuild is done. We choose goprivate.txt for such a test as its
stdimporter package imports a number of std packages, including uses of
assembly and cgo.
For the time being, we don't use such a "full rebuild" reproducibility
test in other test scripts, as this step is expensive, rebuilding many
packages from scratch.
This issue went unnoticed for over a year because such random numbers
"123" and "456" were created when a package was obfuscated, and that
only happened once per package version as long as the build cache was
kept intact.
When clearing the build cache, or forcing a rebuild with -a, one gets
new random numbers, and thus a different binary resulting from the same
build input. That's not something that most users would do regularly,
and our tests did not cover that edge case either, until now.
Fixes #328.
5 years ago
|
|
|
# Also check that a full rebuild is reproducible,
|
|
|
|
|
# with -a to rebuild all packages.
|
|
|
|
|
# This is slow, but necessary to uncover bugs hidden by the build cache.
|
|
|
|
|
garble build -o=out_rebuild -a ./stdimporter
|
|
|
|
|
bincmp out_rebuild out
|
|
|
|
|
|
|
|
|
|
-- go.mod --
|
|
|
|
|
module test/main
|
|
|
|
|
|
|
|
|
|
go 1.16
|
|
|
|
|
-- standalone/main.go --
|
|
|
|
|
package main
|
|
|
|
|
|
|
|
|
|
func main() {}
|
|
|
|
|
-- importer/importer.go --
|
|
|
|
|
package importer
|
|
|
|
|
|
|
|
|
|
import "test/main/imported"
|
|
|
|
|
|
|
|
|
|
var _ = imported.Name
|
|
|
|
|
-- imported/imported.go --
|
|
|
|
|
package imported
|
|
|
|
|
|
|
|
|
|
var Name = "value"
|
|
|
|
|
-- stdimporter/main.go --
|
|
|
|
|
package main
|
|
|
|
|
|
|
|
|
|
import "net/http"
|
|
|
|
|
|
|
|
|
|
func main() {
|
|
|
|
|
http.ListenAndServe("", nil)
|
|
|
|
|
}
|