Go 1.20 is starting to deprecate the use of math/rand's global state,
per https://go.dev/issue/56319 and https://go.dev/issue/20661.
The reasoning is sound:
Deprecated: Programs that call Seed and then expect a specific sequence
of results from the global random source (using functions such as Int)
can be broken when a dependency changes how much it consumes from the
global random source. To avoid such breakages, programs that need a
specific result sequence should use NewRand(NewSource(seed)) to obtain a
random generator that other packages cannot access.
Aside from the tests, we used math/rand only for obfuscating literals,
which caused a deterministic series of calls like Intn. Our call to Seed
was also deterministic, per either GarbleActionID or the -seed flag.
However, our determinism was fragile. If any of our dependencies or
other packages made any calls to math/rand's global funcs, then our
determinism could be broken entirely, and it's hard to notice.
Start using separate math/rand.Rand objects for each use case.
Also make uses of crypto/rand use "cryptorand" for consistency.
Note that this requires a bit of a refactor in internal/literals
to start passing around Rand objects. We also do away with unnecessary
short funcs, especially since math/rand's Read never errors,
and we can obtain a byte via math/rand's Uint32.
Even if diffoscope is installed, because further investigation might be
needed, and some failures are rare or hard to reproduce.
Make GitHub Actions upload those artifacts,
so that a failed CI run on Windows or Mac due to bincmp
allows us to download and inspect those binaries locally.
The generics issue has been fixed for the upcoming Go 1.20.
Include that version as a reminder for when we can drop Go 1.19.
The fs.SkipAll proposal is also implemented for Go 1.20.
The BinaryContentID comment was a little bit trickier.
We did get stamped VCS information some time ago,
but it only provides us with the current commit info and a dirty bit.
That is not enough for our use of the build cache,
because we want any uncommitted changes to garble to cause rebuilds.
I don't think we'll get any better than using garble's own build ID.
Reword the quasi-TODO to instead explain what we're doing and why.
Following the best practices from upstream.
In particular, the "txt" extension is somewhat ambiguous.
This may cause some conflicts due to the git diff noise,
but hopefully we won't ever do this again.
Every now and then, I get test failures in the goenv test like:
> [!windows] cp $EXEC_PATH $NAME/garble$exe
> [!windows] exec $NAME/garble$exe build
[fork/exec with"double"quotes/garble: text file busy]
FAIL: testdata/scripts/goenv.txt:21: unexpected command failure
The root cause is https://go.dev/issue/22315, which isn't going to be
fixed anytime soon, as it is a race condition in Linux itself, triggered
by how heavily concurrent Go tends to be.
For now, try to make the race much less likely to happen.
Back in February 2021, we changed the obfuscation logic so that the
entire `garble build` process would use one shared temporary directory
across all package builds, reducing the amount of files we created in
the top-level system temporary directory.
However, we made one mistake: we didn't swap os.Remove for os.RemoveAll.
Ever since then, we've been leaving temporary files behind.
Add regression tests, which failed before the fix, and fix the bug.
Note that we need to test `garble reverse` as well, as it calls
toolexecCmd separately, so it needs its own cleanup as well.
The cleanup happens via the env var, which doesn't feel worse than
having toolexecCmd return an extra string or cleanup func.
While here, also test that we support TMPDIRs with special characters.
If we don't quote it, paths containing spaces or quote characters will
fail. For instance, the added test without the fix fails:
> env NAME='with spaces'
> mkdir $NAME
> cp $EXEC_PATH $NAME/garble$exe
> exec $NAME/garble$exe build main.go
[stderr]
go tool compile: fork/exec $WORK/with: no such file or directory
exit status 1
Luckily, the fix is easy: we bundle Go's cmd/internal/quoted package,
which implements a QuotedJoin API for this very purpose.
Fixes#544.
I recently added TODOs for bits of code we should cover in the tests.
I was looking at that again just now, and was puzzled;
we do indeed have test cases for many of them already.
We just weren't counting them towards code coverage due to a bug.
errJustExit works as expected, except that it calls os.Exit directly,
whereas testscript wants a non-zero return to run its "after" code.
Part of that code is what handles joining code coverage files.
The total code coverage jumps from 86.2% to 87.6%.
The default behavior of garble is to seed via the build inputs,
including the build IDs of the entire Go build of each package.
This works well as a default, and does give us determinism,
but it means that building for different platforms
will result in different obfuscation per platform.
Instead, when -seed is provided, don't use any other hash seed or salt.
This means that a particular Go name will be obfuscated the same way
as long as the seed, package path, and name itself remain constant.
In other words, when the user supplies a custom -seed,
we assume they know what they're doing in terms of storage and rotation.
Expand the README docs with more examples and detail.
Fixes#449.
Note that this cross-compilation disables cgo by default,
and so the cgo.txt test script isn't run on GOARCH=386.
That seems fine for now, as the test isn't arch-specific.
This testing uncovered one build failure in internal/literals;
the comparison between int and math.MaxUint32 is invalid on 32-bit.
To fix that build failure, use int64 consistently.
One test also incorrectly assumed amd64; it now supports 386 too.
For any other architecture, it's being skipped for now.
I also had to increase the -race test timeout,
as it usually takes 8-9m on GitHub Actions,
and the timeout would sometimes trigger.
Finally, use "go env" rather than "go version" on CI,
which gives us much more useful information,
and also includes Go's own version now via GOVERSION.
Fixes#426.
While investigating a bug report,
I noticed that garble was writing to the same temp file twice.
At best, writing to the same path on disk twice is wasteful,
as the design is careful to be deterministic and use unique paths.
At worst, the two writes could cause races at the filesystem level.
To prevent either of those situations,
we now create files with os.OpenFile and os.O_EXCL,
meaning that we will error if the file already exists.
That change uncovered a number of such unintended cases.
First, transformAsm would write obfuscated Go files twice.
This is because the Go toolchain actually runs:
[...]/asm -gensymabis [...] foo.s bar.s
[...]/asm [...] foo.s bar.s
That is, the first run is only meant to generate symbol ABIs,
which are then used by the compiler.
We need to obfuscate at that first stage,
because the symbol ABI descriptions need to use obfuscated names.
However, having already obfuscated the assembly on the first stage,
there is no need to do so again on the second stage.
If we detect gensymabis is missing, we simply reuse the previous files.
This first situation doesn't seem racy,
but obfuscating the Go assembly files twice is certainly unnecessary.
Second, saveKnownReflectAPIs wrote a gob file to the build cache.
Since the build cache can be kept between builds,
and since the build cache uses reproducible paths for each build,
running the same "garble build" twice could overwrite those files.
This could actually cause races at the filesystem level;
if two concurrent builds write to the same gob file on disk,
one of them could end up using a partially-written file.
Note that this is the only of the three cases not using temporary files.
As such, it is expected that the file may already exist.
In such a case, we simply avoid overwriting it rather than failing.
Third, when "garble build -a" was used,
and when we needed an export file not listed in importcfg,
we would end up calling roughly:
go list -export -toolexec=garble -a <dependency>
This meant we would re-build and re-obfuscate those packages.
Which is unfortunate, because the parent process already did via:
go build -toolexec=garble -a <main>
The repeated dependency builds tripped the new os.O_EXCL check,
as we would try to overwrite the same obfuscated Go files.
Beyond being wasteful, this could again cause subtle filesystem races.
To fix the problem, avoid passing flags like "-a" to nested go commands.
Overall, we should likely be using safer ways to write to disk,
be it via either atomic writes or locked files.
However, for now, catching duplicate writes is a big step.
I have left a self-assigned TODO for further improvements.
CI on the pull request found a failure on test-gotip.
The failure reproduces on master, so it seems to be related to gotip,
and not a regression introduced by this change.
For now, disable test-gotip until we can investigate.
That is, use a very specific build tag and git commit,
and ensure that neither ends up in the binary.
Luckily, we have nothing to do here.
We were already removing _gomod_.go from the build entirely,
and that is still the mechanism that "go build" uses to bundle the data.
Note that the test will still work if git is not installed,
but it will simply not check the VCS side.
Finally, we use "go version -m" to check the existing fields,
which is easier than calling the Go APIs directly.
It seems like "go test" passes on yesterday's Go master, now.
So, enable test-gotip again with that commit hash.
Fixes#385.
In particular, internal/abi now has some actual code,
so obfuscating those literals was breaking as expected.
Document how to update the list in the future as well.
The change above gets "go test" to just one test failure on:
go version devel go1.18-578ada410d Tue Nov 9 22:58:24 2021 +0000 linux/amd64
We also move the doc about why we disable GarbleLiterals,
so that it's next to where the disabling happens.
While here, we also rename GarbleLiterals to ObfuscateLiterals,
as we have been trying to move away from "to garble" as a verb.
Finally, limit the verbosity of diffoscope.
One test was failing for me, and diffoscope printed thousands of lines.
Not particularly useful when I'm trying to skim test results.
Usually, seeing a few dozen lines of output is enough.
Updates #385.
Spotted by our friend "go test -race":
WARNING: DATA RACE
Write at 0x0000010522d8 by goroutine 69:
mvdan.cc/garble.readFile()
garble/main_test.go:124 +0x23a
mvdan.cc/garble.binsubstr()
garble/main_test.go:141 +0xc4
github.com/rogpeppe/go-internal/testscript.(*TestScript).run()
github.com/rogpeppe/go-internal@v1.8.1-0.20211023094830-115ce09fd6b4/testscript/testscript.go:496 +0x9e8
[...]
Previous write at 0x0000010522d8 by goroutine 60:
mvdan.cc/garble.readFile()
garble/main_test.go:124 +0x23a
mvdan.cc/garble.binsubstr()
garble/main_test.go:141 +0xc4
github.com/rogpeppe/go-internal/testscript.(*TestScript).run()
github.com/rogpeppe/go-internal@v1.8.1-0.20211023094830-115ce09fd6b4/testscript/testscript.go:496 +0x9e8
[...]
This wasn't a data race that we spotted via failures in practice,
as it only affected test code since July.
The race is due to the fact that each test script runs as a parallel
sub-test within the same Go program, sharing all globals.
As such, a single "cached binary" global is read and written with races.
Moreover, note that the caching always missed.
I briefly rewrote the code to avoid the race via a sync.Map keyed by
absolute filenames, and while that removed the data race,
the caching never actually hit.
To have a cache hit, we need an absolute path to already be in the cache
and for it to not have been modified since it was last cached. That is:
modify-bin-1 foo
binsubstr foo 'abc' # miss
binsubstr foo 'def' # hit; use the cached "/tmp/[...]/foo" entry
modify-bin-2 foo
binsubstr foo 'abc' # miss
However, the test scripts don't do contiguous binsubstr calls like
these. Instead, they join repeated binsubstr calls:
modify-bin-1 foo
binsubstr foo 'abc' 'def' # miss
modify-bin-2 foo
binsubstr foo 'abc' # miss
For that reason, remove the extra code entirely.
I didn't notice any change to the performance of "go test -short"
with a warm build cache, with:
go test -c
./garble.test -test.short #warm cache
benchcmd -n 5 TestShort ./garble.test -test.short
name old time/op new time/op delta
TestShort 4.62s ±12% 4.35s ±12% ~ (p=0.310 n=5+5)
name old user-time/op new user-time/op delta
TestShort 16.8s ± 3% 16.7s ± 3% ~ (p=0.690 n=5+5)
name old sys-time/op new sys-time/op delta
TestShort 7.28s ± 1% 7.26s ± 2% ~ (p=0.841 n=5+5)
name old peak-RSS-bytes new peak-RSS-bytes delta
TestShort 305MB ± 0% 306MB ± 0% ~ (p=0.421 n=5+5)
Finally, start using "go test -race" on Linux on CI,
which should have made the PR back in July red before merging.
See the added comment for the rationale. For that same reason, I always
build Go itself via "GOGC=off ./make.bash", as it's noticeably faster.
Before this change:
$ go clean -cache && go test -short
PASS
ok mvdan.cc/garble 35.298s
$ go test -short
PASS
ok mvdan.cc/garble 2.703s
With the change:
$ go clean -cache && go test -short
PASS
ok mvdan.cc/garble 25.323s
$ go test -short
PASS
ok mvdan.cc/garble 2.469s
Incremental test runs with a warm cache are largely unaffected, as those
would run very few of those short-lived and allocation-heavy programs.
However, when the build cache isn't warm (such as when garble itself is
modified), we easily see savings of 20-30%.
We might revisit this in the future if Go's GC gets better in these
situations, which should make "go build" faster. For now, we run our
tests very often, so having them burn a bit less CPU is nice.
This mainly cleans up the few bits of code where we explicitly kept
support for Go 1.15.x. With v0.1.0 released, we can drop support now,
since the next v0.2.0 release will only support Go 1.16.x.
Also updates all modules, including test ones, to 'go 1.16'.
Note that the TOOLEXEC_IMPORTPATH refactor is not done here, despite all
the TODOs about doing so when we drop 1.15 support. This is because that
refactor needs to be done carefully and might have side effects, so it's
best to keep it to a separate commit.
Finally, update the deps.
binsubint and binsubfloat haven't been used since 388ff7d1a4 over half a
year ago, and they're a significant amount of code.
Remove them for now; we can always re-add them from the git history if
needed.
testscript already included magic to also account for commands in the
total code coverage. That does not happen with plain tests, since those
only include coverage from the main test process.
The main problem was that, before, indirectly executed commands did not
properly save their coverage profile anywhere for testscript to collect
it at the end. In other words, we only collected coverage from direct
garble executions like "garble -help", but not indirect ones like "go
build -toolexec=garble".
$ go test -coverprofile=cover.out
PASS
coverage: 3.6% of statements
total coverage: 16.6% of statements
ok mvdan.cc/garble 6.453s
After the delicate changes to testscript, any direct or indirect
executions of commands all go through $PATH and properly count towards
the total coverage:
$ go test -coverprofile=cover.out
PASS
coverage: 3.6% of statements
total coverage: 90.5% of statements
ok mvdan.cc/garble 33.258s
Note that we can also get rid of our code to set up $PATH, since
testscript now does it for us.
goversion.txt needed minor tweaks, since we no longer set up $WORK/.bin.
Finally, note that we disable the reuse of $GOCACHE when collecting
coverage information. This is to do "full builds", as otherwise the
cached package builds would result in lower coverage.
Fixes#35.
It's common for asset bundling code generators to produce huge literals,
for example in strings. Our literal obfuscators are meant for relatively
small string-like literals that a human would write, such as URLs, file
paths, and English text.
I ran some quick experiments, and it seems like "garble build -literals"
appears to hang trying to obfuscate literals starting at 5-20KiB. It's
not really hung; it's just doing a lot of busy work obfuscating those
literals. The code it produces is also far from ideal, so it also takes
some time to finally compile.
The generated code also led to crashes. For example, using "garble build
-literals -tiny" on a package containing literals of over a megabyte,
our use of asthelper to remove comments and shuffle line numbers could
run out of stack memory.
This all points in one direction: we never designed "-literals" to deal
with large sizes. Set a source-code-size limit of 2KiB.
We alter the literals.txt test as well, to include a few 128KiB string
literals. Before this fix, "go test" would seemingly hang on that test
for over a minute (I did not wait any longer). With the fix, those large
literals are not obfuscated, so the test ends in its usual 1-3s.
As said in the const comment, I don't believe any of this is a big
problem. Come Go 1.16, most developers should stop using asset-bundling
code generators and use go:embed instead. If we wanted to somehow
obfuscate those, it would be an entirely separate feature.
And, if someone wants to work on obfuscating truly large literals for
any reason, we need good tests and benchmarks to ensure garble does not
consume CPU for minutes or run out of memory.
I also simplified the generate-literals test command. The only argument
that matters to the script is the filename, since it's used later on.
Fixes#178.
In Go 1.15, if a dependency is required but not listed in go.mod/go.sum,
it's resolved and added automatically.
This is changing in 1.16. From that release, one will have to explicitly
update the mod files via 'go mod tidy' or 'go get'.
To get ahead of the curve, start using -mod=readonly to get the same
behavior in 1.15, and fix all existing tests.
The only tests that failed were imports.txt and syntax.txt, the only
ones to require other modules. But since we're here, let's add the 'go'
line to all go.mod files as well.
Many files were missing copyright, so also add a short script to add the
missing lines with the current year, and run it.
The AUTHORS file is also self-explanatory. Contributors can add
themselves there, or we can simply update it from time to time via
git-shortlog.
Since we have two scripts now, set up a directory for them.
If the flags list included ["-o" "binary"], we would properly skip "-o",
but we wouldn't skip "binary".
Thus, 'go list' would receive "binary" as the first argument, and assume
that's the first parameter and the end of the flags.
And add a unit test case.
Fixes#82, again.
Otherwise any build flags like -tags won't be used, and we might easily
end up with errors or incorrect packages.
The common case with -tags is covered by one of the integration test
scripts. On top of that, we add a table-driven unit test to cover all
edge cases, since there are many we can do quickly in a unit test.
Fixes#82.
Instead of doing a 'go list' call every time we need to fetch a
dependency's export file, we now do a single 'go list' call before the
build begins. With the '-deps' flag, it gives us all the dependency
packages recursively.
We store that data in the gob format in a temporary file, and share it
with the future garble sub-processes via an env var.
This required lazy parsing of flags for the 'build' and 'test' commands,
since now we need to run 'go list' with the same package pattern
arguments.
Fixes#63.
First, unindent some of the AST code.
Second, genRandInt is unused; delete it.
Third, genRandIntn is really just mathrand.Intn. Just use it directly.
Fourth, don't use inline comments if they result in super long lines.
This requires a bit of extra magic to replace one constant in
runtime/internal/sys, but that was simple enough given that we can reuse
a lot of the code to parse the files and write them to a temporary dir.
We can also drop the -X flags, as runtime.buildVersion is based on the
constant that we replace here.
Fixes#44, again.
Carefully select a default that will do the right thing when inside a
module, as well as when building ad-hoc packages.
This means we no longer need to look at the compiler's -std flag, which
is nice.
Also replace foo.com/ with test/, as per golang/go#37641.
Fixes#7.
We use 'go list -json -export' to locate required modules. This works
fine to locate direct module dependencies; since we're building in the
current module, we run 'go list' in the correct directory.
However, if we're building one of those module dependencies, and it has
other module dependencies of its own, we would fail with cryptic errors
like:
typecheck error: [...] go list error: updates to go.sum needed, disabled by -mod=readonly
This is because we would try to run 'go list' outside of the main
module, probably inside the module cache. Instead, use a $GARBLE_DIR env
var from the top-level 'garble build' call to always run 'go list' in
the original directory.
We add a few small modules to properly test this.
Updates #9.
This is important, because "-std -foo" and "-buildid -foo" are entirely
different cases. The first is equivalent to "-std=true -foo" since the
flag is boolean, but the second is equivalent to "-buildid=-foo" since
the flag isn't boolean.
We can keep track of which of the flags we're interested in are boolean,
which isn't much extra work. Also add unit tests; the build ID is a
hash, so it's very hard to write an end-to-end test that reliably has an
ID starting with a dash.
The problem with the "grep" built-in command is that it prints the
entire data if there is an error. We don't want megabytes of binary
output for a test.
Daniel Martí
Nick
lu4p
pagran