Forked from: github.com/burrowers/garble
You cannot select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
 
 
 
Go to file
Daniel Martí caa9831a63
fail if we are unexpectedly overwriting files (#418)
While investigating a bug report,
I noticed that garble was writing to the same temp file twice.
At best, writing to the same path on disk twice is wasteful,
as the design is careful to be deterministic and use unique paths.
At worst, the two writes could cause races at the filesystem level.

To prevent either of those situations,
we now create files with os.OpenFile and os.O_EXCL,
meaning that we will error if the file already exists.
That change uncovered a number of such unintended cases.

First, transformAsm would write obfuscated Go files twice.
This is because the Go toolchain actually runs:

	[...]/asm -gensymabis [...] foo.s bar.s
	[...]/asm [...] foo.s bar.s

That is, the first run is only meant to generate symbol ABIs,
which are then used by the compiler.
We need to obfuscate at that first stage,
because the symbol ABI descriptions need to use obfuscated names.

However, having already obfuscated the assembly on the first stage,
there is no need to do so again on the second stage.
If we detect gensymabis is missing, we simply reuse the previous files.

This first situation doesn't seem racy,
but obfuscating the Go assembly files twice is certainly unnecessary.

Second, saveKnownReflectAPIs wrote a gob file to the build cache.
Since the build cache can be kept between builds,
and since the build cache uses reproducible paths for each build,
running the same "garble build" twice could overwrite those files.

This could actually cause races at the filesystem level;
if two concurrent builds write to the same gob file on disk,
one of them could end up using a partially-written file.

Note that this is the only of the three cases not using temporary files.
As such, it is expected that the file may already exist.
In such a case, we simply avoid overwriting it rather than failing.

Third, when "garble build -a" was used,
and when we needed an export file not listed in importcfg,
we would end up calling roughly:

	go list -export -toolexec=garble -a <dependency>

This meant we would re-build and re-obfuscate those packages.
Which is unfortunate, because the parent process already did via:

	go build -toolexec=garble -a <main>

The repeated dependency builds tripped the new os.O_EXCL check,
as we would try to overwrite the same obfuscated Go files.
Beyond being wasteful, this could again cause subtle filesystem races.
To fix the problem, avoid passing flags like "-a" to nested go commands.

Overall, we should likely be using safer ways to write to disk,
be it via either atomic writes or locked files.
However, for now, catching duplicate writes is a big step.
I have left a self-assigned TODO for further improvements.

CI on the pull request found a failure on test-gotip.
The failure reproduces on master, so it seems to be related to gotip,
and not a regression introduced by this change.
For now, disable test-gotip until we can investigate.
3 years ago
.github fail if we are unexpectedly overwriting files (#418) 3 years ago
internal Fix minor typo in RecordUsedAsConstants docstring 3 years ago
scripts Obfuscate more packages of the standard library (#312) 3 years ago
testdata testdata: tweak typechecking error for Go 1.18 again 3 years ago
.gitattributes
.gitignore skip literals used in constant expressions 4 years ago
AUTHORS set up an AUTHORS file to attribute copyright 4 years ago
CHANGELOG.md prepare changelog for 0.4.0 3 years ago
CONTRIBUTING.md README: document commands 3 years ago
LICENSE set up an AUTHORS file to attribute copyright 4 years ago
README.md README: keep making a mention of reflect hints (#409) 3 years ago
bench_test.go rework the build benchmarks 3 years ago
go.mod fix runtime stripping for Go 1.18 3 years ago
go.sum fix runtime stripping for Go 1.18 3 years ago
hash.go update runtimeAndDeps for Go 1.18 3 years ago
main.go fail if we are unexpectedly overwriting files (#418) 3 years ago
main_test.go fail if we are unexpectedly overwriting files (#418) 3 years ago
position.go remove unused code spotted by -coverprofile 3 years ago
reverse.go fail if we are unexpectedly overwriting files (#418) 3 years ago
runtime_strip.go fix runtime stripping for Go 1.18 3 years ago
shared.go fail if we are unexpectedly overwriting files (#418) 3 years ago

README.md

garble

go install mvdan.cc/garble@latest

Obfuscate Go code by wrapping the Go toolchain. Requires Go 1.17 or later.

garble build [build flags] [packages]

The tool also supports garble test to run tests with obfuscated code, and garble reverse to de-obfuscate text such as stack traces. See garble -h for up to date usage information.

Purpose

Produce a binary that works as well as a regular build, but that has as little information about the original source code as possible.

The tool is designed to be:

  • Coupled with cmd/go, to support modules and build caching
  • Deterministic and reproducible, given the same initial source code
  • Reversible given the original source, to de-obfuscate panic stack traces

Mechanism

The tool wraps calls to the Go compiler and linker to transform the Go build, in order to:

  • Replace as many useful identifiers as possible with short base64 hashes
  • Replace package paths with short base64 hashes
  • Remove all build and module information
  • Strip filenames and shuffle position information
  • Strip debugging information and symbol tables via -ldflags="-w -s"
  • Obfuscate literals, if the -literals flag is given
  • Remove extra information, if the -tiny flag is given

By default, the tool obfuscates the packages under the current module. If not running in module mode, then only the main package is obfuscated. To specify what packages to obfuscate, set GOPRIVATE, documented at go help private.

Note that commands like garble build will use the go version found in your $PATH. To use different versions of Go, you can install them and set up $PATH with them. For example, for Go 1.17.1:

$ go install golang.org/dl/go1.17.1@latest
$ go1.17.1 download
$ PATH=$(go1.17.1 env GOROOT)/bin:${PATH} garble build

Literal obfuscation

Using the -literals flag causes literal expressions such as strings to be replaced with more complex variants, resolving to the same value at run-time. This feature is opt-in, as it can cause slow-downs depending on the input code.

Literal expressions used as constants cannot be obfuscated, since they are resolved at compile time. This includes any expressions part of a const declaration.

Tiny mode

When the -tiny flag is passed, extra information is stripped from the resulting Go binary. This includes line numbers, filenames, and code in the runtime that prints panics, fatal errors, and trace/debug info. All in all this can make binaries 2-5% smaller in our testing, as well as prevent extracting some more information.

With this flag, no panics or fatal runtime errors will ever be printed, but they can still be handled internally with recover as normal. In addition, the GODEBUG environmental variable will be ignored.

Note that this flag can make debugging crashes harder, as a panic will simply exit the entire program without printing a stack trace, and all source code positions are set to line 1. Similarly, garble reverse is generally not useful in this mode.

Speed

garble build should take about twice as long as go build, as it needs to complete two builds. The original build, to be able to load and type-check the input code, and then the obfuscated build.

Garble obfuscates one package at a time, mirroring how Go compiles one package at a time. This allows Garble to fully support Go's build cache; incremental garble build calls should only re-build and re-obfuscate modified code.

Determinism and seeds

Just like Go, garble builds are deterministic and reproducible if the inputs remain the same: the version of Go, the version of Garble, and the input code. This has significant benefits, such as caching builds or being able to use garble reverse to de-obfuscate stack traces.

However, it also means that an input package will be obfuscated in exactly the same way if none of those inputs change. If you want two builds of your program to be entirely different, you can use -seed to provide a new seed for the entire build, which will cause a full rebuild.

If any open source packages are being obfuscated, providing a custom seed can also provide extra protection. It could be possible to guess the versions of Go and garble given how a public package was obfuscated without a seed.

Caveats

Most of these can improve with time and effort. The purpose of this section is to document the current shortcomings of this tool.

  • Exported methods are never obfuscated at the moment, since they could be required by interfaces. This area is a work in progress; see #3.

  • Garble aims to automatically detect which Go types are used with reflection, as obfuscating those types might break your program. Note that Garble obfuscates one package at a time, so if your reflection code inspects a type from an imported package, and your program broke, you may need to add a "hint" in the imported package:

    type Message struct {
        Command string
        Args    string
    }
    
    // Never obfuscate the Message type.
    var _ = reflect.TypeOf(Message{})
    
  • Go declarations exported for cgo via //export are not obfuscated.

  • Go plugins are not currently supported; see #87.

Contributing

We welcome new contributors. If you would like to contribute, see CONTRIBUTING.md as a starting point.