Commit Graph

316 Commits (f85492a728f95182ff9dc8c867caee2c80672fa8)

Author SHA1 Message Date
Daniel Martí f85492a728 split typecheck and loadPkgCache from transformer
Neither of them has anything to do with transforming Go code;
they simply load or compute the information necessary for doing so.

Split typecheck into two functions as well.
The new typecheck function only does typechecking and nothing else.
A new comptueFieldToStruct func fills the fieldToStruct map,
which depends on typecheck, but is not needed when computing pkgCache.

This isolation also forces us to separate the code that fills pkgCache
from the code that fills the in-memory-only maps in transformer,
removing the need for the NOTE that we had left around as a reminder.
1 year ago
Daniel Martí 4b0b2acf6f isolate reflect.go from updating globals directly
That is, stop reusing "transformer" as the receiver on methods,
and stop writing the results to the global curPkgCache struct.

Soon we will need to support computing pkgCache for any dependency,
not just the current package, to make the caching properly robust.
This allows us to fill reflectInspector with different values.

The explicit isolation also helps prevent bugs.
For instance, we were calling recursivelyRecordAsNotObfuscated from
transformCompile, which happens after we have loaded or saved pkgCache.
Meaning, the current package sees a larger pkgCache than its dependents.
In this particular case it wasn't causing any bugs,
since the two reflect types in question only had unexported fields,
but it's still good to treat pkgCache as read-only in transformCompile.
1 year ago
Daniel Martí d108f21846 apply TODO to rename "cannot obfuscate" APIs
They have been exclusively about reflect for over a year now.
Make that clearer, and update the docs as well.
1 year ago
Daniel Martí e079c0af43 declare a type for cachedOutput
To properly make our cache robust, we'll need to be able to compute
cache entries for dependencies as needed if they are missing.
So we'll need to create more of these struct values in the code.

Rename cachedOutput to curPkgCache, to clarify that it relates
to the current package.

While here, remove the "known" prefix on all pkgCache fields.
All of the names still make perfect sense without it.
1 year ago
Daniel Martí 0f2b59d794 merge the two "known cannot obfuscate" maps
Per the TODOs that I left myself in the last commit.
As expected, this change allows tidying up the code a bit,
makes our use of caching a bit more consistent,
and also allows us to load the current package from the cache.
1 year ago
Daniel Martí 7d1bd13778 replace our caching inside GOCACHE with GARBLE_CACHE
For each Go package we obfuscate, we need to store information about
how we obfuscated it, which is needed when obfuscating its dependents.
For example, if A depends on B to use the type B.Foo, A needs to know
whether or not B.Foo was obfuscated; it depends on B's use of reflect.

We record this information in a gob file, which is cached on disk.
To avoid rolling our own custom cache, and since garble is so closely
connected with cmd/go already, we piggybacked off of Go's GOCACHE.
In particular, for each build cache entry per `go list`'s Export field,
we would store a "garble" sibling file with that gob content.

However, this was brittle for two reasons:

1) We were doing this without cmd/go's permission or knowledge.
   We were careful to use filename suffixes similar to Export files,
   meaning that `go clean` and other commands would treat them the same.
   However, this could confuse cmd/go at any point in the future.

2) cmd/go trims cache entries in GOCACHE regularly, to keep the size of
   the build and test caches under control. Right now, this means that
   every 24h, any file not accessed in the last five days is deleted.
   However, that trimming heuristic is done per-file.
   If the trimming removed Garble's sibling file but not the original
   Export file, this could cause errors such as
   "cannot load garble export file" which users already ran into.

Instead, start using github.com/rogpeppe/go-internal/cache,
an exported copy of cmd/go's own cache implementation for GOCACHE.
Since we need an entirely separate directory, we introduce GARBLE_CACHE,
defaulting to the "garble" directory inside the user's cache directory.
For example, on Linux this would be ~/.cache/garble.

Inside GARBLE_CACHE, our gob file cache will be under "build",
which helps clarify that this cache is used when obfuscating Go builds,
and allows placing other kinds of caches inside GARBLE_CACHE.
For example, we already have a need for storing linker binaries,
which for now still use their own caching mechanism.

This commit does not make our cache properly resistant to removed files.
The proof is that our seed.txtar testscript still fails the second case.
However, we do rewrite all of our caching logic away from Export files,
which in itself is a considerable refactor, and we add a few TODOs.

One notable change is how we load gob files from dependencies
when building the cache entry for the current package.
We used to load the gob files from all packages in the Deps field.
However, that is the list of all _transitive_ dependencies.
Since these gob files are already flat, meaning they contain information
about all of their transitive dependencies as well, we need only load
the gob files from the direct dependencies, the Imports field.

Performance is largely unchanged, since the behavior is similar.
However, the change from Deps to Imports saves us some work,
which can be seen in the reduced mallocs per obfuscated build.

It's unclear why the binary size isn't stable.
When reverting the Deps to Imports change, it then settles at 5.386Mi,
which is almost exactly in between the two measurements below.
I'm not sure why, but that metric appears to be slightly unstable.

    goos: linux
    goarch: amd64
    pkg: mvdan.cc/garble
    cpu: AMD Ryzen 7 PRO 5850U with Radeon Graphics
            │    old     │             new              │
            │   sec/op   │   sec/op    vs base          │
    Build-8   11.09 ± 1%   11.08 ± 1%  ~ (p=0.796 n=10)

            │     old      │                 new                 │
            │    bin-B     │    bin-B      vs base               │
    Build-8   5.390Mi ± 0%   5.382Mi ± 0%  -0.14% (p=0.000 n=10)

            │      old      │               new               │
            │ cached-sec/op │ cached-sec/op  vs base          │
    Build-8     415.5m ± 4%     421.6m ± 1%  ~ (p=0.190 n=10)

            │     old     │                new                 │
            │ mallocs/op  │ mallocs/op   vs base               │
    Build-8   35.43M ± 0%   34.05M ± 0%  -3.89% (p=0.000 n=10)

            │    old     │             new              │
            │ sys-sec/op │ sys-sec/op  vs base          │
    Build-8   5.662 ± 1%   5.701 ± 2%  ~ (p=0.280 n=10)
1 year ago
Daniel Martí cee53a7868 make GarbleActionID a full sha256 hash
This is in preparation for the switch to Go's cache package,
whose ActionID type is also a full sha256 hash with 32 bytes.
We were using "short" hashes as shown by `go tool buildid`,
since that was consistent and 15 bytes was generally enough.
1 year ago
Daniel Martí 414e3b7f70 tidy our build ID hash code a bit
First, rename "component" to "hash", since it's shorter and more useful.
A full build ID is two or four hashes joined with slashes.

Second, add sanity checks that buildIDHashLength is being followed.
Otherwise the use of []byte could lead to human error.

Third, move all the hash encoding and decoding logic together.
1 year ago
Daniel Martí 0c9a59127a rename cache global to sharedCache
Since we will start importing github.com/rogpeppe/go-internal/cache,
and I don't want to have to rename it or leave confusion around.
1 year ago
Daniel Martí 9f50e1a8a5 tweak when we read and write cachedOutput files
We first called the typecheck method, which starts filling cachedOutput
with information from the current package, and later we would load the
gob files for all dependencies via loadCachedOutputs.

This was a bit confusing; instead, load the cached gob files first,
and then do all the operations which fill information for curPkg.

Similarly, we were waiting until the very end of transformCompile to
write curPkg's cachedOutput gob file to the disk cache.
We can write the file at an earlier point, before we have obfuscated and
re-printed all Go files for the current package.
We can also write the file before other work like processImportCfg.

None of these changes should affect garble's behavior,
but they will make the cache redesign for #708 easier.
1 year ago
Daniel Martí 744e9a375a suggest a command when asking the user to rebuild garble
See a user's apparent confusion in #738.
1 year ago
lu4p 1526ce7fd2
rework reflection detection with ssa (#732)
This is significantly more robust, than the ast based detection and can
record very complex cases of indirect parameter reflection.

Fixes #554
1 year ago
Daniel Martí b0ff2fb133 fix encoding/asn1 marshaling of pkix types
See the added comment and test, which failed before the fix when
the encoded certificate was decoded.

It took a while to find the culprit in asn1, but in hindsight it's easy:

	case reflect.Slice:
		if t.Elem().Kind() == reflect.Uint8 {
			return false, TagOctetString, false, true
		}
		if strings.HasSuffix(t.Name(), "SET") {
			return false, TagSet, true, true
		}
		return false, TagSequence, true, true

Fixes #711.
1 year ago
Daniel Martí 4d7546703a update gotip in CI and fix -tiny on the latest tip
printOneCgoTraceback now returns a boolean rather than an int.
Since we need to have different logic based on the Go version,
and toolchainVersionSemver was only set for the main process,
move the string to the shared cache global.
This is a nice thing to do anyway, to reduce the number of globals.

While here, update actions/setup-go to v4, which starts caching
GOMODCACHE and GOCACHE by default now.
Disable it, because it still doesn't help in our case,
and GitHub's Actions caching is still really inefficient.

And update staticcheck too.
1 year ago
Daniel Martí cd003eade0 support `garble run`
It's the third time I've wanted it to quickly test out standalone
reproducers when people submit bugs.

Fixes #661.
1 year ago
Daniel Martí d1da066120 print chosen seed when building with -seed=random
The seedFlag.random field had never worked,
as my refactor in December 2021 never set it to true.

Even if the boolean was working, we only printed the random seed
when we failed. It's still useful to see it when a build succeeds,
for example when wanting to reproduce the same binary
or when wanting to reverse a panic from the produced binary.

Add a test this time.

Fixes #696.
1 year ago
Daniel Martí 9d04637009 ensure the alignment of sync/atomic types works
Added in Go 1.19, types like sync/atomic.Uint64 are handy,
because they ensure proper alignment even on 32-bit GOOSes.
However, this was done via a magic `type align64 struct{}`,
which the compiler spotted by name.

To keep that magic working, do not obfuscate the name.
Neither package path was being obfuscated,
as both packages contain compiler intrinsics already.

Fixes #686.
1 year ago
Daniel Martí cf49f7f3b1 use the host's GOEXE when building the linker
We're building the linker binary for the host GOOS,
not the target GOOS that we happen to be building for.

I noticed that, after running `go test`, my garble cache
would contain both link and link.exe, which made no sense
as I run linux and not windows.

`go env` has GOHOSTOS to mirror GOOS, but there is no
GOHOSTEXE to mirror GOEXE, so we reconstruct it from
runtime.GOOS, which is equivalent to GOHOSTOS.

Add a regression test as well.
1 year ago
Daniel Martí 059c1d68e2 use fewer build flags when building std or cmd
When we use `go list` on the standard library, we need to be careful
about what flags are passed from the top-level build command,
because some flags are not going to be appropriate.
In particular, GOFLAGS=-modfile=... resulted in a failure,
reproduced via the GOFLAGS variable added to linker.txtar:

	go: inconsistent vendoring in /home/mvdan/tip/src:
		golang.org/x/crypto@v0.5.1-0.20230203195927-310bfa40f1e4: is marked as explicit in vendor/modules.txt, but not explicitly required in go.mod
		golang.org/x/net@v0.7.0: is marked as explicit in vendor/modules.txt, but not explicitly required in go.mod
		golang.org/x/sys@v0.5.1-0.20230208141308-4fee21c92339: is marked as explicit in vendor/modules.txt, but not explicitly required in go.mod
		golang.org/x/text@v0.7.1-0.20230207171107-30dadde3188b: is marked as explicit in vendor/modules.txt, but not explicitly required in go.mod

		To ignore the vendor directory, use -mod=readonly or -mod=mod.
		To sync the vendor directory, run:
			go mod vendor

To work around this problem, reset the -mod and -modfile flags when
calling "go list" on the standard library, as those are the only two
flags which alter how we load the main module in a build.

The code which builds a modified cmd/link has a similar problem;
it already reset GOOS and GOARCH, but it could similarly run into
problems if other env vars like GOFLAGS were set.
To be on the safe side, we also disable GOENV and GOEXPERIMENT,
which we borrow from Go's bootstrapping commands.
1 year ago
pagran 86b7e334ba
implement funcInfo.entryoff encryption
At linker stage, we now encrypt funcInfo.entryoff value with a simple algorithm (1 xor + 1 mul). 
This makes it harder to relate function metadata (e.g. name) to function itself in binary, almost without affecting performance.
1 year ago
Daniel Martí 89b27fa7f9 tweaks thanks to code coverage info
The test package hack still appears to be needed as of Go 1.20.

Interestingly, the cgo filename check does not trigger at all anymore.
Presumably this means that it's not needed at all, and obfuscating code
generated by cgo appears to be fine. Go with that for now.
1 year ago
Daniel Martí 658060851d drop bits of code to support Go 1.19 1 year ago
Daniel Martí b322876efe drop support for Go 1.19
Now that we're done with garble v0.9.x,
v0.10 will only support Go 1.20 as a minimum version.
1 year ago
Daniel Martí 5effe20c19 avoid breaking github.com/davecgh/go-spew
It assumes that reflect.Value has a field named "flag",
which wasn't the case with obfuscated builds as we obfuscated it.

We already treated the reflect package as special,
for instance when not obfuscating Method or MethodByName.
In a similar fashion, mark reflect's rtype and Value types to not be
obfuscated alongside their field names. Note that rtype is the
implementation behind the reflect.Type interface.

This fix is fairly manual and repetitive.
transformCompile, transformLinkname, and transformAsm should all
use the same mechanism to tell if names should be obfuscated.
However, they do not do that right now, and that refactor feels too
risky for a bugfix release. We add more TODOs instead.

We're not adding go-spew to scripts/check-third-party.sh since the
project is largely abandoned. It's not even a Go module yet.
The only broken bit from it is what we've added to our tests.

Fixes #676.
1 year ago
Daniel Martí 95ef0357da support inline and non-spaced comments in assembly
Go contains such inline comments, like:

	crypto/aes/gcm_ppc64x.s:129: VPMSUMD IN, HL, XL // H.lo·H.lo

We didn't notice since these comments were somewhat rare.
While here, "//" does not need to be followed by a space.

The code turns out to be pretty easy with strings.Cut.

Fixes #672.
1 year ago
Daniel Martí 6d54a77771 update flag tables with Go 1.20
`go help build` now has -C, -cover, -coverpkg, and -pgo.

There are no new boolean flags, but note that -cover is now a common
build flag rather than just a test flag.

While here, sort the lists so that they are easier to skim for missing
items in the future.
1 year ago
Daniel Martí 4919be0658 switch to final Go 1.20 release and re-enable gotip in CI
Go master, the upcoming Go 1.21, has had its merge window open for over
two weeks at this point, and it seems calmer at this point.

ALso update staticcheck to its latest release, which supports Go 1.20.
1 year ago
Daniel Martí 2ee9cf7a43 support go:linkname directives pointing at methods
This is not common, but it is done by a few projects.
Namely, github.com/goccy/go-json reached into reflect's guts,
which included a number of methods:

	internal/runtime/rtype.go
	11://go:linkname rtype_Align reflect.(*rtype).Align
	19://go:linkname rtype_FieldAlign reflect.(*rtype).FieldAlign
	27://go:linkname rtype_Method reflect.(*rtype).Method
	35://go:linkname rtype_MethodByName reflect.(*rtype).MethodByName
	[...]

Add tests for such go:linkname directives pointing at methods.
Note that there are two possible symbol string variants;
"pkg/path.(*Receiver).method" for methods with pointer receivers,
and "pkg/path.Receiver.method" for the rest.

We can't assume that the presence of two dots means a method either.
For example, a package path may be "pkg/path.with.dots",
and so "pkg/path.with.dots.SomeFunc" is the function "SomeFunc"
rather than the method "SomeFunc" on a type "dots".
To account for this ambiguity, rather than splitting on the last dot
like we used to, try to find a package path prefix by splitting on an
increasing number of first dots.

This can in theory still be ambiguous. For example,
we could have the package "pkg/path" expose the method "foo.bar",
and the package "pkg/path.foo" expose the func "bar".
Then, the symbol string "pkg/path.foo.bar" could mean either of them.
However, this seems extremely unlikely to happen in practice,
and I'm not sure that Go's toolchain would support it either.

I also noticed that goccy/go-json still failed to build after the fix.
The reason was that the type reflect.rtype wasn't being obfuscated.
We could, and likely should, teach our assembly and linkname
transformers about which names we chose not to obfuscate due to the use
of reflection. However, in this particular case, reflect's own types
can be obfuscated safely, so just do that.

Fixes #656.
1 year ago
Daniel Martí 09a17375e3 reduce verbosity in the last change
Declare the ast.GenDecl node once,
which allows us to deduplicate and inline code.
resolveImportSpec doesn't need to be separate either.

We also don't need explicit panics for unexpected nils;
we can rely on nil checks inserted by the compiler.

Finally, move the bulk of the code outside of the scope.Names loop,
which unindents most of the logic.

While here, add a few more inline docs.
1 year ago
pagran 7c50979899
ensure that all imports are used after obfuscation
garble's -literals flag and its patching of the runtime may leave unused imports.
We used to try to detect those and remove the imports,
but that was still buggy with edge cases like dot imports or renamed imports.

Moreover, it was potentially incorrect.
Completely removing an import from a package means we don't run its init funcs,
which could have side effects changing the behavior of a program.
As an example, database/sql drivers are registered at init time.

Instead, for each import in an obfuscated Go file,
add an unnamed declaration which references the imported package.
This may not be necessary for all imported packages,
as only a minority become unused due to garble,
but it's also relatively harmless to do so.

Fixes #658.
1 year ago
Daniel Martí 0ec363d9c8 avoid breaking intrinsics when obfuscating names
We obfuscate import paths as well as their declared names.
The compiler treats some packages and APIs in special ways,
and the way it detects those is by looking at import paths and names.

In the past, we have avoided obfuscating some names like embed.FS or
reflect.Value.MethodByName for this reason. Otherwise,
go:embed or the linker's deadcode elimination might be broken.

This matching by path and name also happens with compiler intrinsics.
Intrinsics allow the compiler to rewrite some standard library calls
with small and efficient assembly, depending on the target GOARCH.
For example, math/bits.TrailingZeros32 gets replaced with ssa.OpCtz32,
which on amd64 may result in using the TZCNTL instruction.

We never noticed that we were breaking many of these intrinsics.
The intrinsics for funcs declared in the runtime and its dependencies
still worked properly, as we do not obfuscate those packages yet.
However, for other packages like math/bits and sync/atomic,
the intrinsics were being entirely disabled due to obfuscated names.

Skipping intrinsics is particularly bad for performance,
and it also leads to slightly larger binaries:

			 │      old      │                 new                 │
			 │     bin-B     │     bin-B      vs base              │
	Build-16   5.450Mi ± ∞ ¹   5.333Mi ± ∞ ¹  -2.15% (p=0.029 n=4)

Finally, the main reason we noticed that intrinsics were broken
is that apparently GOARCH=mips fails to link without them,
as some symbols end up being not defined at all.
This patch fixes builds for the MIPS family of architectures.

Rather than building and linking all of std for every GOARCH,
test that intrinsics work by asking the compiler to print which
intrinsics are being applied, and checking that math/bits gets them.

This fix is relatively unfortunate, as it means we stop obfuscating
about 120 function names and a handful of package paths.
However, fixing builds and intrinsics is much more important.
We can figure out better ways to deal with intrinsics in the future.

Fixes #646.
1 year ago
Daniel Martí 598d5182fb update x/tools version used in go:generate
Fixes running this go:generate line with Go tip.
1 year ago
pagran d3c6ed6729
add support for obfuscation of dot-imported string constants 1 year ago
Daniel Martí 5dd6b2dc43 support assembly references to package names
Go's package runtime/internal/atomic contains references to functions
which refer to the current package by its package name alone:

	src/runtime/internal/atomic/atomic_loong64.s:   JMP atomic·Load(SB)
	src/runtime/internal/atomic/atomic_loong64.s:   JMP atomic·Load64(SB)
	src/runtime/internal/atomic/atomic_loong64.s:   JMP atomic·Load64(SB)
	src/runtime/internal/atomic/atomic_mips64x.s:   JMP atomic·Load(SB)
	src/runtime/internal/atomic/atomic_mips64x.s:   JMP atomic·Load64(SB)
	src/runtime/internal/atomic/atomic_mips64x.s:   JMP atomic·Load64(SB)

We could only handle unqualified or fully qualified references, like:

	JMP ·Load64(SB)
	JMP runtime∕internal∕atomic·Load64(SB)

Apparently, all three forms are equally valid.
Add a test case and fix it.

I checked whether referencing an imported package by its name worked;
it does not seem to be the case.
This feature appears to be restricted to the current package alone.

While here, we only need goPkgPath when we need to call listPackage.

Fixes #619.
1 year ago
Daniel Martí 7e95fcf1cb detect which objects are global in a simpler way
To detect if an object is global, our previous code did:

	obj.Pkg().Scope().Lookup(obj.Name()) == obj

However, that Lookup call is unnecessary.
It is a map lookup, which is cheap, but not free.
Instead, we can compare the scopes directly:

	obj.Pkg().Scope() == obj.Parent()

While here, reuse the calls to obj.Pkg(),
and avoid using fmt.Sprintf in this hot path.

Looked here since perf on Linux with a garble build
reports that we spend about 2.6% of CPU time in recordedObjectString.
Overall provides a very minor improvement:

	name      old time/op         new time/op         delta
	Build-16          21.8s ± 1%          21.7s ± 0%    ~     (p=0.200 n=4+4)

	name      old bin-B           new bin-B           delta
	Build-16          5.67M ± 0%          5.67M ± 0%    ~     (all equal)

	name      old cached-time/op  new cached-time/op  delta
	Build-16          734ms ± 3%          722ms ± 3%    ~     (p=0.486 n=4+4)

	name      old mallocs/op      new mallocs/op      delta
	Build-16          25.0M ± 0%          24.7M ± 0%  -1.28%  (p=0.029 n=4+4)

	name      old sys-time/op     new sys-time/op     delta
	Build-16          17.0s ± 1%          16.8s ± 1%    ~     (p=0.200 n=4+4)
1 year ago
Daniel Martí 1c4fe53fc1 skip over comments when obfuscating assembly
github.com/bytedance/sonic has many comments in its assembly files,
and one in particular caused us trouble:

	internal/rt/asm_amd64.s:2:// Code generated by asm2asm, DO NOT EDIT·

Since we looked for "middle dot" characters anywhere,
we would try to load the package "EDIT", which clearly does not exist.

To fix that, start skipping over comments. Thankfully,
Go's assembly syntax only supports "//" line comments,
so it's enough to read one line at a time and skip them quickly.
We also longer need a regular expression to find #include lines.

While here, two other minor bits of cleanup.
First, rename transformGo to transformGoFile, for clarity.
Second, use ".s" extensions for obfuscated assembly filenames,
just like we do with the comiler and ".go" files.

Updates #621.
1 year ago
pagran 22c177f088
remove all unexported func names with -tiny via the linker
The patch to the linker does this when generating the pclntab,
which is the binary section containing func names.
When `-tiny` is being used, we look for unexported funcs,
and set their names to the offset `0` - a shared empty string.

We also avoid including the original name in the binary,
which saves a significant amount of space.
The following stats were collected on GOOS=linux,
which show that `-tiny` is now about 4% smaller:

	go build                  1203067
	garble build               782336
	(old) garble -tiny build   688128
	(new) garble -tiny build   659456
2 years ago
Daniel Martí 417bcf27bb support referencing package paths with periods in assembly
The test case is lifted from github.com/bytedance/sonic/internal/native,
which had the following line:

	JMP github·com∕bytedance∕sonic∕internal∕native∕avx2·__quote(SB)

Updates #621.
2 years ago
pagran 6ace03322f
patch and rebuild cmd/link to modify the magic value in pclntab
This value is hard-coded in the linker and written in a header.
We could rewrite the final binary, like we used to do with import paths,
but that would require once again maintaining libraries to do so.

Instead, we're now modifying the linker to do what we want.
It's not particularly hard, as every Go install has its source code,
and rebuilding a slightly modified linker only takes a few seconds at most.

Thanks to `go build -overlay`, we only need to copy the files we modify,
and right now we're just modifying one file in the toolchain.
We use a git patch, as the change is fairly static and small,
and the patch is easier to understand and maintain.

The other side of this change is in the runtime,
as it also hard-codes the magic value when loading information.
We modify the code via syntax trees in that case, like `-tiny` does,
because the change is tiny (one literal) and the affected lines of code
are modified regularly between major Go releases.

Since rebuilding a slightly modified linker can take a few seconds,
and Go's build cache does not cache linked binaries,
we keep our own cached version of the rebuilt binary in `os.UserCacheDir`.

The feature isn't perfect, and will be improved in the future.
See the TODOs about the added dependency on `git`,
or how we are currently only able to cache one linker binary at once.

Fixes #622.
2 years ago
Daniel Martí 05d9b4ed26 avoid call to loadSharedCache for toolexec calls we skip
We were doing too much work for tools we don't need to wrap at all,
such as `go tool pack` or `go tool buildid`, which get run about as
often as the compiler does.

The overhead is small per call, but it adds up.

	name      old time/op         new time/op         delta
	Build-16          20.7s ± 0%          20.5s ± 1%    ~     (p=0.057 n=4+4)

	name      old bin-B           new bin-B           delta
	Build-16          5.67M ± 0%          5.66M ± 0%  -0.07%  (p=0.029 n=4+4)

	name      old cached-time/op  new cached-time/op  delta
	Build-16          707ms ± 0%          705ms ± 2%    ~     (p=0.886 n=4+4)

	name      old mallocs/op      new mallocs/op      delta
	Build-16          25.0M ± 0%          24.9M ± 0%  -0.26%  (p=0.029 n=4+4)

	name      old sys-time/op     new sys-time/op     delta
	Build-16          8.32s ± 2%          7.90s ± 3%  -5.05%  (p=0.029 n=4+4)
2 years ago
Daniel Martí 21a5eb1720 simplify types.TypeName panic logic
The logic

	if X || Y {
		if X {
			<X panic>
		}
		<Y panic>
	}

is equivalent to, and easier to read as:

	if X {
		<X panic>
	}
	if Y {
		<Y panic>
	}

It probably evolved unintentionally to be this way.
2 years ago
Daniel Martí 9a0d48c27e use errors.Is for listPackage errors 2 years ago
Daniel Martí d955196470 avoid using math/rand's global funcs like Seed and Intn
Go 1.20 is starting to deprecate the use of math/rand's global state,
per https://go.dev/issue/56319 and https://go.dev/issue/20661.
The reasoning is sound:

	Deprecated: Programs that call Seed and then expect a specific sequence
	of results from the global random source (using functions such as Int)
	can be broken when a dependency changes how much it consumes from the
	global random source. To avoid such breakages, programs that need a
	specific result sequence should use NewRand(NewSource(seed)) to obtain a
	random generator that other packages cannot access.

Aside from the tests, we used math/rand only for obfuscating literals,
which caused a deterministic series of calls like Intn. Our call to Seed
was also deterministic, per either GarbleActionID or the -seed flag.

However, our determinism was fragile. If any of our dependencies or
other packages made any calls to math/rand's global funcs, then our
determinism could be broken entirely, and it's hard to notice.

Start using separate math/rand.Rand objects for each use case.
Also make uses of crypto/rand use "cryptorand" for consistency.

Note that this requires a bit of a refactor in internal/literals
to start passing around Rand objects. We also do away with unnecessary
short funcs, especially since math/rand's Read never errors,
and we can obtain a byte via math/rand's Uint32.
2 years ago
Daniel Martí 4a0c2d0274 remove duplicate err!=nil check 2 years ago
Daniel Martí 9c0cdc61ef avoid reflect method call panics with GOGARBLE=*
We were obfuscating reflect's package path and its declared names,
but the toolchain wants to detect the presence of method reflection
to turn down the aggressiveness of dead code elimination.

Given that the obfuscation broke the detection,
we could easily end up in crashes when making reflect calls:

	fatal error: unreachable method called. linker bug?

	goroutine 1 [running]:
	runtime.throw({0x50c9b3?, 0x2?})
		runtime/panic.go:1047 +0x5d fp=0xc000063660 sp=0xc000063630 pc=0x43245d
	runtime.unreachableMethod()
		runtime/iface.go:532 +0x25 fp=0xc000063680 sp=0xc000063660 pc=0x40a845
	runtime.call16(0xc00010a360, 0xc00000e0a8, 0x0, 0x0, 0x0, 0x8, 0xc000063bb0)
		runtime/wcS9OpRFL:728 +0x49 fp=0xc0000636a0 sp=0xc000063680 pc=0x45eae9
	runtime.reflectcall(0xc00001c120?, 0x1?, 0x1?, 0x18110?, 0xc0?, 0x1?, 0x1?)
		<autogenerated>:1 +0x3c fp=0xc0000636e0 sp=0xc0000636a0 pc=0x462e9c

Avoid obfuscating the three names which cause problems: "reflect",
"Method", and "MethodByName".

While here, we also teach obfuscatedImportPath to skip "runtime",
as I also saw that the toolchain detects it for many reasons.
That wasn't a problem yet, as we do not obfuscate the runtime,
but it was likely going to become a problem in the future.
2 years ago
Daniel Martí 481e3a1f09 default to GOGARBLE=*, stop using GOPRIVATE
We can drop the code that kicked in when GOGARBLE was empty.
We can also add the value in addGarbleToHash unconditionally,
as we never allow it to be empty.

In the tests, remove all GOGARBLE lines where it just meant "obfuscate
everything" or "obfuscate the entire main module".

cgo.txtar had "obfuscate everything" as a separate step,
so remove it entirely.

linkname.txtar started failing because the imported package did not
import strings, so listPackage errored out. This wasn't a problem when
strings itself wasn't obfuscated, as transformLinkname silently left
strings.IndexByte untouched. It is a problem when IndexByte does get
obfuscated. Make that kind of listPackage error visible, and fix it.

reflect.txtar started failing with "unreachable method" runtime throws.
It's not clear to me why; it appears that GOGARBLE=* makes the linker
think that ExportedMethodName is suddenly unreachable.
Work around the problem by making the method explicitly reachable,
and leave a TODO as a reminder to investigate.

Finally, gogarble.txtar no longer needs to test for GOPRIVATE.
The rest of the test is left the same, as we still want the various
values for GOGARBLE to continue to work just like before.

Fixes #594.
2 years ago
Daniel Martí ffb4987756
clarify that "garble version" does more now (#612)
Since earlier this year, it has also printed build settings.
2 years ago
Daniel Martí 7d591830cd teach -debugdir to produce assembly files too
Up to this point, -debugdir only included obfuscated Go files.
Include assembly files and their headers as well.
While here, ensure that -debugdir supports the standard library too,
and that it behaves correctly with build tags as well.

Also use more consistent names for path strings, to clarify which are
only the basename, and which are the obfuscated version.
2 years ago
Daniel Martí e61317e7ae fix garble with newer Go tip versions
Some big changes landed in Go for the upcoming 1.20.

While here, remove the use of GOGC=off with make.bash,
as https://go.dev/cl/436235 makes that unnecessary now.
2 years ago
Daniel Martí ff521782f1 obfuscate all assembly filenames
We were still leaking the filenames for assembly files.
In our existing asm.txtar test's output binary,
the string `test/main/garble_main_amd64.s` was present.
This leaked full import paths on one hand,
and the filenames of each assembly file on the other.

We avoid this in Go files by using `/*line` directives,
but those are not supported in assembly files.
Instead, obfuscate the paths in the temporary directory.
Note that we still need a separate temporary directory per package,
because otherwise any included header files might collide.

We must remove the `main` package panic in obfuscatedImportPath,
as we now need to use that function for all packages.

While here, remove the outdated comment about `-trimpath`.

Fixes #605.
2 years ago