You cannot select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
d8de5a4306
We were using temporary filenames for modified Go and assembly files. For example, an obfuscated "encoding/json/encode.go" would end up as: /tmp/garble-shared123/encode.go.456.go where "123" and "456" are random numbers, usually longer. This was usually fine for two reasons: 1) We would add "/tmp/garble-shared123/" to -trimpath, so the temporary directory and its random number would be invisible. 2) We would add "//line" directives to the source files, replacing the filename with obfuscated versions excluding any random number. Unfortunately, this broke in multiple ways. Most notably, assembly files do not have any line directives, and it's not clear that there's any support for them. So the random number in their basename could end up in the binary, breaking reproducibility. Another issue is that the -trimpath addition described above was only done for cmd/compile, not cmd/asm, so assembly filenames included the randomized temporary directory. To fix the issues above, the same "encoding/json/encode.go" would now end up as: /tmp/garble-shared123/encoding/json/encode.go Such a path is still unique even though the "456" random number is gone, as import paths are unique within a single build. This fixes issues with the base name of each file, so we no longer rely on line directives as the only way to remove the second original random number. We still rely on -trimpath to get rid of the temporary directory in filenames. To fix its problem with assembly files, also amend the -trimpath flag when running the assembler tool. Finally, add a test that reproducible builds still work when a full rebuild is done. We choose goprivate.txt for such a test as its stdimporter package imports a number of std packages, including uses of assembly and cgo. For the time being, we don't use such a "full rebuild" reproducibility test in other test scripts, as this step is expensive, rebuilding many packages from scratch. This issue went unnoticed for over a year because such random numbers "123" and "456" were created when a package was obfuscated, and that only happened once per package version as long as the build cache was kept intact. When clearing the build cache, or forcing a rebuild with -a, one gets new random numbers, and thus a different binary resulting from the same build input. That's not something that most users would do regularly, and our tests did not cover that edge case either, until now. Fixes #328. |
5 years ago | |
|---|---|---|
| .. | ||
| bench-cache | 5 years ago | |
| bench-nocache | 5 years ago | |
| mod | 5 years ago | |
| scripts | 5 years ago | |