We now store how we obfuscated unexported names in the object file
itself, not a separate file. This means that the data can survive in the
build cache, whereas the separate file was being lost. Luckily, we can
just add an extra header to the archive, and other programs like the Go
linker will just ignore it.
Give the func a name that tells what the return value means.
Add missing newlines to printfs, use consistent quoting, and replace
"%s" with %q.
Document the Go 1.15 date.
Finally, fix the imports via goimports.
Many files were missing copyright, so also add a short script to add the
missing lines with the current year, and run it.
The AUTHORS file is also self-explanatory. Contributors can add
themselves there, or we can simply update it from time to time via
git-shortlog.
Since we have two scripts now, set up a directory for them.
Rework the features section in the README, leaving optional features at
the end of the list. Simplify the caveats list, too; the build cache and
exported field/method bits only need one point each. Overall, the
section was far too wordy for little reason.
Also redo the help text a bit. There's now a line to briefly introduce
the tool, as well as a link to the README with all the details. Finally,
the flags have shorter and more consistent help strings.
While at it, remove two unused global vars as spotted by staticcheck.
Finally, finally this is done. This allows import paths to be obfuscated by modifying
object/archive files and garbling import paths contained within. The bulk of the
code that makes parsing and writing Go object/archive files possible lives at
https://github.com/Binject/debug/tree/master/goobj2, which I wrote as well.
I have tested by garbling and checking for import paths via strings and grep
(in order of difficulty) https://github.com/lu4p/binclude, garble itself, and
https://github.com/dominikh/go-tools/tree/master/cmd/staticcheck.
This only supports object/archive files produced from the Go 1.15 compiler.
The object file format changed at 1.15, and 1.14 and earlier is not supported.
Fixes#13.
Most notably, x/mod now includes the GOPRIVATE pattern-matching API we
were copying before, so we can use it directly.
Also bump the Go version requirement to 1.15, in preparation for the
import path obfuscation PR, and don't let the gotip job fail the entire
workflow.
Before this change, obfuscating any package using unsafe.Pointer and
with GOPRIVATE="*" would result in errors like:
undefined: unsafe.ZrMmYd1lg
This is because the type isn't plain Go; it's rather a special type that
gets special treatment from the typechecker and compiler:
type Pointer *ArbitraryType
So, trying to obfuscate the name "unsafe.Pointer" will never work,
because there isn't a real Go type definition we can obfuscate along
with that.
Updates, but does not yet fully fix, #108.
This is important, because it would mean that we would obfuscate
nothing. At best, it would be confusing; at worst, it could mislead
the user into thinking the binary is obfuscated.
Fixes#20.
Updates #108.
We need to call 'go env GOPRIVATE' instead of just using os.Getenv so
that we pick up the value from the new ${CONFIG}/go/env file, written by
'go env -w'.
However, we were calling 'go env' at every process start, including the
often tens or hundreds of compiler calls to build all the dependencies.
Instead, do that only once on the first 'garble build' process, and use
os.Setenv to pass that along to future garble sub-processes.
name old time/op new time/op delta
Build-8 1.81s ± 0% 1.74s ± 4% -3.78% (p=0.030 n=5+6)
name old sys-time/op new sys-time/op delta
Build-8 1.45s ± 2% 1.22s ± 1% -16.07% (p=0.002 n=6+6)
name old user-time/op new user-time/op delta
Build-8 10.9s ± 1% 10.6s ± 1% -2.82% (p=0.004 n=6+5)
Error strings should never be capitalized.
A binsubstr line in one of the tests was duplicate and thus useless.
Remove duplicate or trailing spaces in test scripts.
Finally, add a TODO for an optimization I just spotted.
Fixes #2.
Line numbers are now obfuscated, via `//line` comments.
Filenames are now obfuscated via `//line` comments, instead of changing the actual filename.
New flag `-tiny` to reduce the binary size, at the cost of reversibility.
Fixes#93.
The second typecheck lead to the creation of different type objects,
which didn't match the types in the blacklist anymore.
It turns out we don't need the second typecheck,
therfore it is now removed.
First, our original append line was completely ineffective; we never
used that "flags" slice again. Second, we only attempted to use the flag
when we obfuscated a package.
In fact, we never care about debugging information here, so for any
package we compile, we can add "-dwarf=false". At the moment, we compile
all packages, even if they aren't to be obfuscated, due to the lack of
access to the build cache.
As such, we save a significant amount of work. The numbers below were
obtained on a quiet machine with "go test -bench=. -benchtime=10x", six
times before and after the change.
name old time/op new time/op delta
Build-8 2.06s ± 4% 1.87s ± 2% -9.21% (p=0.002 n=6+6)
name old sys-time/op new sys-time/op delta
Build-8 1.51s ± 2% 1.46s ± 1% -3.12% (p=0.004 n=6+5)
name old user-time/op new user-time/op delta
Build-8 11.9s ± 2% 10.8s ± 1% -8.71% (p=0.002 n=6+6)
While at it, only do CI builds on pushes and PRs to the master branch,
so that my PRs created from the same repo don't trigger duplicate
builds.
If the flags list included ["-o" "binary"], we would properly skip "-o",
but we wouldn't skip "binary".
Thus, 'go list' would receive "binary" as the first argument, and assume
that's the first parameter and the end of the flags.
And add a unit test case.
Fixes#82, again.
Otherwise any build flags like -tags won't be used, and we might easily
end up with errors or incorrect packages.
The common case with -tags is covered by one of the integration test
scripts. On top of that, we add a table-driven unit test to cover all
edge cases, since there are many we can do quickly in a unit test.
Fixes#82.
We don't really care about tools other than "compile" and "link". Stop
trying to keep a complete list.
Use "if err := f(); err != nil {" where it makes sense.
Simplify some declarations, and use a better variable name than "fW".
Instead of doing a 'go list' call every time we need to fetch a
dependency's export file, we now do a single 'go list' call before the
build begins. With the '-deps' flag, it gives us all the dependency
packages recursively.
We store that data in the gob format in a temporary file, and share it
with the future garble sub-processes via an env var.
This required lazy parsing of flags for the 'build' and 'test' commands,
since now we need to run 'go list' with the same package pattern
arguments.
Fixes#63.
The following identifiers are now skipped,
because they never show up in the binary:
- constant identifiers
- identifiers of local variables
(includes function params and named returns)
- identifiers of local types
Implement a literal obfuscator interface,
to allow the easy addition of new encodings.
Add literal obfuscation for byte literals.
Choose a random obfuscator on literal obfuscation,
useful when multiple obfuscators are implemented.
Fixes#62
Injected functions were mistaken for functions implemented outside go.
Asm functions:
obj.Scope().Pos() == 0
obj.Scope().End() == 0
Injected functions:
obj.Scope().Pos() == 0
obj.Scope().End() == 1
We now check for the End instead of the Pos.
This requires a bit of extra magic to replace one constant in
runtime/internal/sys, but that was simple enough given that we can reuse
a lot of the code to parse the files and write them to a temporary dir.
We can also drop the -X flags, as runtime.buildVersion is based on the
constant that we replace here.
Fixes#44, again.
Generating DWARF in the compiler object files could take as much as 10%
extra CPU time, while we ignore it entirely at the link stage.
Speeds up 'go test -short' from ~19.5s to ~18.5s on my laptop.
Whilst it may not be particularly common, it is legal to embed fields
where the type has universe scope (e.g. int, error, etc). This can
cause a panic in 2 difference places:
- When embedding `error`, a named type is resolved but the package is
nil. The call to `pkg.Name()` results in a panic
- When embedding a basic type such as `int`, no named type is resolved
at all. The call to `namedType(obj.Type()).Obj()` results in a panic
I'm assuming it is OK to return early when a named type cannot be
resolved.. we could let it continue but I think `pkg` should be set to
nil to be correct, so it'd end up returning straight away anyway.
In the added test, the unexported field used to be garbled.
Reflection can only reach exported methods, exported fields, and
unexported fields. Exported methods and fields are currently never
garbled, so unexported fields was the only missing piece.
Avoiding a type switch for the entire node prevents an indentation
level.
We can obtain obj and pkg early, and return early as well if either is
uninteresting. That means less nil checks later on, which means even
less indentation and complexity.
Also remove the -toolexec equivalent, as it's becoming longer now that
we have GARBLE_DIR, and it might become out of date in the future again.
We don't want users to assume it will work forever.
Carefully select a default that will do the right thing when inside a
module, as well as when building ad-hoc packages.
This means we no longer need to look at the compiler's -std flag, which
is nice.
Also replace foo.com/ with test/, as per golang/go#37641.
Fixes#7.
Spotted while trying to link a program using unix.Syscall, since its
implementation is assembly.
Telling if a function couldn't be garbled isn't trivial. If that
function belongs to an imported package, we only load its export data
instead of type-checking from source, so we don't have all the
information needed.
Instead, use the gc export data importer to import two versions of each
dependency: its original version, for the initial type-checking, and its
garbled version, to check if any of its exported names weren't garbled.
Updates #9.
We use 'go list -json -export' to locate required modules. This works
fine to locate direct module dependencies; since we're building in the
current module, we run 'go list' in the correct directory.
However, if we're building one of those module dependencies, and it has
other module dependencies of its own, we would fail with cryptic errors
like:
typecheck error: [...] go list error: updates to go.sum needed, disabled by -mod=readonly
This is because we would try to run 'go list' outside of the main
module, probably inside the module cache. Instead, use a $GARBLE_DIR env
var from the top-level 'garble build' call to always run 'go list' in
the original directory.
We add a few small modules to properly test this.
Updates #9.
In the added test case, we'd see a failure, since we garbled the name of
the "Embedded" type but not its use as an anonymous field. Garble both.
This might possibly break some reflect code, but it doesn't seem like we
have an option. When we garble a type, it's impossible to tell if it's
going to be used as an anonymous field later.
Updates #9.
This is important, because "-std -foo" and "-buildid -foo" are entirely
different cases. The first is equivalent to "-std=true -foo" since the
flag is boolean, but the second is equivalent to "-buildid=-foo" since
the flag isn't boolean.
We can keep track of which of the flags we're interested in are boolean,
which isn't much extra work. Also add unit tests; the build ID is a
hash, so it's very hard to write an end-to-end test that reliably has an
ID starting with a dash.
'go tool compile' receives the package path via the -p flag. This is
better than making up one.
We have to be careful with "-p main" though, as that's not part of the
standard library.
pagran
Daniel Martí
Andrew LeFevre
lu4p
pagran
Zachary Wasserman
Nicholas Jones