|
|
|
env GOPRIVATE=test/main
|
|
|
|
|
|
|
|
garble -literals build
|
|
|
|
exec ./main$exe
|
|
|
|
cmp stderr main.stderr
|
|
|
|
|
|
|
|
binsubstr main$exe 'Skip this block' 'also skip this' 'skip typed const' 'skip typed var' 'skip typed var assign' 'stringTypeField strType' 'stringType lambda func return' 'testMap1 key' 'testMap2 key' 'testMap3 key' 'testMap1 value' 'testMap3 value' 'testMap1 new value' 'testMap3 new value' 'stringType func param' 'stringType return' 'skip untyped const'
|
|
|
|
! binsubstr main$exe 'garbleDecrypt' 'Lorem' 'dolor' 'first assign' 'second assign' 'First Line' 'Second Line' 'map value' 'to obfuscate' 'also obfuscate' 'stringTypeField String'
|
|
|
|
|
|
|
|
[short] stop # checking that the build is reproducible is slow
|
|
|
|
|
|
|
|
# Also check that the binary is reproducible.
|
|
|
|
cp main$exe main_old$exe
|
|
|
|
rm main$exe
|
|
|
|
garble -literals build
|
|
|
|
bincmp main$exe main_old$exe
|
|
|
|
|
|
|
|
# Check that the program works as expected without garble.
|
|
|
|
go build
|
|
|
|
exec ./main$exe
|
|
|
|
cmp stderr main.stderr
|
|
|
|
binsubstr main$exe 'Lorem' 'dolor' 'second assign' 'First Line' 'Second Line' 'map value' 'to obfuscate' 'also obfuscate' 'stringTypeField String'
|
|
|
|
|
do not try to obfuscate huge literals (#204)
It's common for asset bundling code generators to produce huge literals,
for example in strings. Our literal obfuscators are meant for relatively
small string-like literals that a human would write, such as URLs, file
paths, and English text.
I ran some quick experiments, and it seems like "garble build -literals"
appears to hang trying to obfuscate literals starting at 5-20KiB. It's
not really hung; it's just doing a lot of busy work obfuscating those
literals. The code it produces is also far from ideal, so it also takes
some time to finally compile.
The generated code also led to crashes. For example, using "garble build
-literals -tiny" on a package containing literals of over a megabyte,
our use of asthelper to remove comments and shuffle line numbers could
run out of stack memory.
This all points in one direction: we never designed "-literals" to deal
with large sizes. Set a source-code-size limit of 2KiB.
We alter the literals.txt test as well, to include a few 128KiB string
literals. Before this fix, "go test" would seemingly hang on that test
for over a minute (I did not wait any longer). With the fix, those large
literals are not obfuscated, so the test ends in its usual 1-3s.
As said in the const comment, I don't believe any of this is a big
problem. Come Go 1.16, most developers should stop using asset-bundling
code generators and use go:embed instead. If we wanted to somehow
obfuscate those, it would be an entirely separate feature.
And, if someone wants to work on obfuscating truly large literals for
any reason, we need good tests and benchmarks to ensure garble does not
consume CPU for minutes or run out of memory.
I also simplified the generate-literals test command. The only argument
that matters to the script is the filename, since it's used later on.
Fixes #178.
4 years ago
|
|
|
# Generate and write random literals into a separate file.
|
|
|
|
# Some of them will be huge; assuming that we don't try to obfuscate them, the
|
|
|
|
# test should generally run in under a second. If this test hangs for over ten
|
|
|
|
# seconds, it means we're trying to obfuscate them.
|
|
|
|
generate-literals extra_literals.go
|
|
|
|
|
|
|
|
# Also check that the binary is different from previous builds.
|
|
|
|
rm main$exe
|
refactor "current package" with TOOLEXEC_IMPORTPATH (#266)
Now that we've dropped support for Go 1.15.x, we can finally rely on
this environment variable for toolexec calls, present in Go 1.16.
Before, we had hacky ways of trying to figure out the current package's
import path, mostly from the -p flag. The biggest rough edge there was
that, for main packages, that was simply the package name, and not its
full import path.
To work around that, we had a restriction on a single main package, so
we could work around that issue. That restriction is now gone.
The new code is simpler, especially because we can set curPkg in a
single place for all toolexec transform funcs.
Since we can always rely on curPkg not being nil now, we can also start
reusing listedPackage.Private and avoid the majority of repeated calls
to isPrivate. The function is cheap, but still not free.
isPrivate itself can also get simpler. We no longer have to worry about
the "main" edge case. Plus, the sanity check for invalid package paths
is now unnecessary; we only got malformed paths from goobj2, and we now
require exact matches with the ImportPath field from "go list -json".
Another effect of clearing up the "main" edge case is that -debugdir now
uses the right directory for main packages. We also start using
consistent debugdir paths in the tests, for the sake of being easier to
read and maintain.
Finally, note that commandReverse did not need the extra call to "go
list -toolexec", as the "shared" call stored in the cache is enough. We
still call toolexecCmd to get said cache, which should probably be
simplified in a future PR.
While at it, replace the use of the "-std" compiler flag with the
Standard field from "go list -json".
3 years ago
|
|
|
garble -literals -debugdir=debug1 -seed=8J+Ri/Cfh6fwn4e+ build
|
|
|
|
! bincmp main$exe main_old$exe
|
|
|
|
|
|
|
|
exec ./main$exe
|
|
|
|
cmp stderr main.stderr
|
|
|
|
|
|
|
|
# Check obfuscators
|
|
|
|
|
|
|
|
# Xor obfuscator. Detect a[i] = a[i] (^|-|+) b[i]
|
refactor "current package" with TOOLEXEC_IMPORTPATH (#266)
Now that we've dropped support for Go 1.15.x, we can finally rely on
this environment variable for toolexec calls, present in Go 1.16.
Before, we had hacky ways of trying to figure out the current package's
import path, mostly from the -p flag. The biggest rough edge there was
that, for main packages, that was simply the package name, and not its
full import path.
To work around that, we had a restriction on a single main package, so
we could work around that issue. That restriction is now gone.
The new code is simpler, especially because we can set curPkg in a
single place for all toolexec transform funcs.
Since we can always rely on curPkg not being nil now, we can also start
reusing listedPackage.Private and avoid the majority of repeated calls
to isPrivate. The function is cheap, but still not free.
isPrivate itself can also get simpler. We no longer have to worry about
the "main" edge case. Plus, the sanity check for invalid package paths
is now unnecessary; we only got malformed paths from goobj2, and we now
require exact matches with the ImportPath field from "go list -json".
Another effect of clearing up the "main" edge case is that -debugdir now
uses the right directory for main packages. We also start using
consistent debugdir paths in the tests, for the sake of being easier to
read and maintain.
Finally, note that commandReverse did not need the extra call to "go
list -toolexec", as the "shared" call stored in the cache is enough. We
still call toolexecCmd to get said cache, which should probably be
simplified in a future PR.
While at it, replace the use of the "-std" compiler flag with the
Standard field from "go list -json".
3 years ago
|
|
|
grep '^\s+\w+\[\w+\] = \w+\[\w+\] [\^\-+] \w+$' debug1/test/main/extra_literals.go
|
|
|
|
|
|
|
|
# Swap obfuscator. Detect [...]byte|uint16|uint32|uint64{...}
|
refactor "current package" with TOOLEXEC_IMPORTPATH (#266)
Now that we've dropped support for Go 1.15.x, we can finally rely on
this environment variable for toolexec calls, present in Go 1.16.
Before, we had hacky ways of trying to figure out the current package's
import path, mostly from the -p flag. The biggest rough edge there was
that, for main packages, that was simply the package name, and not its
full import path.
To work around that, we had a restriction on a single main package, so
we could work around that issue. That restriction is now gone.
The new code is simpler, especially because we can set curPkg in a
single place for all toolexec transform funcs.
Since we can always rely on curPkg not being nil now, we can also start
reusing listedPackage.Private and avoid the majority of repeated calls
to isPrivate. The function is cheap, but still not free.
isPrivate itself can also get simpler. We no longer have to worry about
the "main" edge case. Plus, the sanity check for invalid package paths
is now unnecessary; we only got malformed paths from goobj2, and we now
require exact matches with the ImportPath field from "go list -json".
Another effect of clearing up the "main" edge case is that -debugdir now
uses the right directory for main packages. We also start using
consistent debugdir paths in the tests, for the sake of being easier to
read and maintain.
Finally, note that commandReverse did not need the extra call to "go
list -toolexec", as the "shared" call stored in the cache is enough. We
still call toolexecCmd to get said cache, which should probably be
simplified in a future PR.
While at it, replace the use of the "-std" compiler flag with the
Standard field from "go list -json".
3 years ago
|
|
|
grep '^\s+\w+ := \[\.{3}\](byte|uint16|uint32|uint64)\{[0-9\s,]+\}$' debug1/test/main/extra_literals.go
|
|
|
|
|
|
|
|
# Split obfuscator. Detect decryptKey ^= i * counter
|
refactor "current package" with TOOLEXEC_IMPORTPATH (#266)
Now that we've dropped support for Go 1.15.x, we can finally rely on
this environment variable for toolexec calls, present in Go 1.16.
Before, we had hacky ways of trying to figure out the current package's
import path, mostly from the -p flag. The biggest rough edge there was
that, for main packages, that was simply the package name, and not its
full import path.
To work around that, we had a restriction on a single main package, so
we could work around that issue. That restriction is now gone.
The new code is simpler, especially because we can set curPkg in a
single place for all toolexec transform funcs.
Since we can always rely on curPkg not being nil now, we can also start
reusing listedPackage.Private and avoid the majority of repeated calls
to isPrivate. The function is cheap, but still not free.
isPrivate itself can also get simpler. We no longer have to worry about
the "main" edge case. Plus, the sanity check for invalid package paths
is now unnecessary; we only got malformed paths from goobj2, and we now
require exact matches with the ImportPath field from "go list -json".
Another effect of clearing up the "main" edge case is that -debugdir now
uses the right directory for main packages. We also start using
consistent debugdir paths in the tests, for the sake of being easier to
read and maintain.
Finally, note that commandReverse did not need the extra call to "go
list -toolexec", as the "shared" call stored in the cache is enough. We
still call toolexecCmd to get said cache, which should probably be
simplified in a future PR.
While at it, replace the use of the "-std" compiler flag with the
Standard field from "go list -json".
3 years ago
|
|
|
grep '^\s+\w+ \^= \w+ \* \w+$' debug1/test/main/extra_literals.go
|
|
|
|
|
|
|
|
# XorShuffle obfuscator. Detect data = append(data, x (^|-|+) y...).
|
|
|
|
# Note that the line obfuscator adds an inline comment before the call.
|
|
|
|
grep '^\s+\w+ = .*\bappend\(\w+,(\s+\w+\[\d+\][\^\-+]\w+\[\d+\],?)+\)$' debug1/test/main/extra_literals.go
|
|
|
|
|
|
|
|
# XorSeed obfuscator. Detect type decFunc func(byte) decFunc
|
refactor "current package" with TOOLEXEC_IMPORTPATH (#266)
Now that we've dropped support for Go 1.15.x, we can finally rely on
this environment variable for toolexec calls, present in Go 1.16.
Before, we had hacky ways of trying to figure out the current package's
import path, mostly from the -p flag. The biggest rough edge there was
that, for main packages, that was simply the package name, and not its
full import path.
To work around that, we had a restriction on a single main package, so
we could work around that issue. That restriction is now gone.
The new code is simpler, especially because we can set curPkg in a
single place for all toolexec transform funcs.
Since we can always rely on curPkg not being nil now, we can also start
reusing listedPackage.Private and avoid the majority of repeated calls
to isPrivate. The function is cheap, but still not free.
isPrivate itself can also get simpler. We no longer have to worry about
the "main" edge case. Plus, the sanity check for invalid package paths
is now unnecessary; we only got malformed paths from goobj2, and we now
require exact matches with the ImportPath field from "go list -json".
Another effect of clearing up the "main" edge case is that -debugdir now
uses the right directory for main packages. We also start using
consistent debugdir paths in the tests, for the sake of being easier to
read and maintain.
Finally, note that commandReverse did not need the extra call to "go
list -toolexec", as the "shared" call stored in the cache is enough. We
still call toolexecCmd to get said cache, which should probably be
simplified in a future PR.
While at it, replace the use of the "-std" compiler flag with the
Standard field from "go list -json".
3 years ago
|
|
|
grep '^\s+type \w+ func\(byte\) \w+$' debug1/test/main/extra_literals.go
|
|
|
|
|
|
|
|
-- go.mod --
|
|
|
|
module test/main
|
|
|
|
|
|
|
|
go 1.16
|
|
|
|
-- main.go --
|
|
|
|
package main
|
|
|
|
|
|
|
|
type strucTest struct {
|
|
|
|
field string
|
|
|
|
anotherfield string
|
|
|
|
}
|
|
|
|
|
|
|
|
const (
|
|
|
|
cnst string = "Lorem"
|
|
|
|
multiline string = `First Line
|
|
|
|
Second Line`
|
|
|
|
)
|
|
|
|
|
|
|
|
const (
|
|
|
|
i = 1
|
|
|
|
boolean = true
|
|
|
|
|
|
|
|
skip1 = "Skip this block"
|
|
|
|
)
|
|
|
|
|
|
|
|
const (
|
|
|
|
foo = iota
|
|
|
|
bar
|
|
|
|
|
|
|
|
skip2 = "also skip this"
|
|
|
|
)
|
|
|
|
|
|
|
|
const arrayLen = 4
|
|
|
|
|
|
|
|
var array [arrayLen]byte
|
|
|
|
|
|
|
|
type typeAlias [arrayLen]byte
|
|
|
|
|
|
|
|
func main() {
|
|
|
|
empty := ""
|
|
|
|
|
|
|
|
localVar := "dolor"
|
|
|
|
|
|
|
|
reassign := "first assign"
|
|
|
|
reassign = "second assign"
|
|
|
|
|
|
|
|
add := "total" + " string"
|
|
|
|
|
|
|
|
println(cnst, boolean)
|
|
|
|
println(multiline, add)
|
|
|
|
println(localVar)
|
|
|
|
println(reassign)
|
|
|
|
println(empty)
|
|
|
|
|
|
|
|
x := strucTest{
|
|
|
|
field: "to obfuscate",
|
|
|
|
anotherfield: "also obfuscate",
|
|
|
|
}
|
|
|
|
|
|
|
|
lambda := func() string {
|
|
|
|
return "😅 😅"
|
|
|
|
}()
|
|
|
|
println(lambda)
|
|
|
|
|
|
|
|
println(x.field, x.anotherfield)
|
|
|
|
|
|
|
|
testMap := map[string]string{"map key": "map value"}
|
|
|
|
testMap["map key"] = "new value"
|
|
|
|
println(testMap["map key"])
|
|
|
|
println("another literal")
|
|
|
|
println(skip1, skip2)
|
|
|
|
println(i, foo, bar)
|
|
|
|
typedTest()
|
|
|
|
constantTest()
|
|
|
|
byteTest()
|
|
|
|
}
|
|
|
|
|
|
|
|
type stringType string
|
|
|
|
|
|
|
|
type stringTypeStruct struct {
|
|
|
|
str string
|
|
|
|
strType stringType
|
|
|
|
}
|
|
|
|
|
|
|
|
// typedTest types defined from string broke previously
|
|
|
|
func typedTest() {
|
|
|
|
const skipUntypedConst = "skip untyped const"
|
|
|
|
stringTypeFunc(skipUntypedConst)
|
|
|
|
|
|
|
|
const skipTypedConst stringType = "skip typed const" // skip
|
|
|
|
var skipTypedVar stringType = "skip typed var" // skip
|
|
|
|
|
|
|
|
var skipTypedVarAssign stringType
|
|
|
|
skipTypedVarAssign = "skip typed var assign" // skip
|
|
|
|
|
|
|
|
println(skipTypedConst, skipTypedVar, skipTypedVarAssign)
|
|
|
|
|
|
|
|
y := stringTypeStruct{
|
|
|
|
str: "stringTypeField String", // obfuscate
|
|
|
|
strType: "stringTypeField strType", // skip
|
|
|
|
}
|
|
|
|
println(y.str, y.strType)
|
|
|
|
|
|
|
|
z := func(s stringType) stringType {
|
|
|
|
return "stringType lambda func return" // skip
|
|
|
|
}("lambda call") // skip
|
|
|
|
println(z)
|
|
|
|
|
|
|
|
testMap1 := map[string]stringType{"testMap1 key": "testMap1 value"} // skip
|
|
|
|
testMap1["testMap1 key"] = "testMap1 new value" // skip
|
|
|
|
|
|
|
|
testMap2 := map[stringType]string{"testMap2 key": "testMap2 value"} // skip key
|
|
|
|
testMap2["testMap2 key"] = "testMap2 new value" // skip key
|
|
|
|
|
|
|
|
testMap3 := map[stringType]stringType{"testMap3 key": "testMap3 value"} // skip
|
|
|
|
testMap3["testMap3 key"] = "testMap3 new value" // skip
|
|
|
|
|
|
|
|
println(stringTypeFunc("stringType func param")) // skip
|
|
|
|
}
|
|
|
|
|
|
|
|
// constantTest tests that string constants which need to be constant are skipped
|
|
|
|
func constantTest() {
|
|
|
|
const a = "foo" // skip
|
|
|
|
const length = len(a)
|
|
|
|
|
|
|
|
const b = "bar" // skip
|
|
|
|
type T [len(b)]byte
|
|
|
|
|
|
|
|
const c = "foo" // skip
|
|
|
|
var _ [len(c)]byte
|
|
|
|
|
|
|
|
const d = "foo" // skip
|
|
|
|
var arr = [5]string{len(d): "foo"}
|
|
|
|
for _, elm := range arr {
|
|
|
|
if elm != "" {
|
|
|
|
println(elm)
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
const e = "foo" // skip
|
|
|
|
var slice = []string{len(e): "foo"}
|
|
|
|
for _, elm := range slice {
|
|
|
|
if elm != "" {
|
|
|
|
println(elm)
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
const f = "foo" // skip
|
|
|
|
const i = length + len(f)
|
|
|
|
println(length, i)
|
|
|
|
}
|
|
|
|
|
|
|
|
func byteTest() {
|
|
|
|
a := []byte{12, 13}
|
|
|
|
for _, elm := range a {
|
|
|
|
print(elm, ",")
|
|
|
|
}
|
|
|
|
println()
|
|
|
|
var b = []byte{12, 13}
|
|
|
|
for _, elm := range b {
|
|
|
|
print(elm, ",")
|
|
|
|
}
|
|
|
|
println()
|
|
|
|
|
|
|
|
var c = [2]byte{12, 13}
|
|
|
|
for _, elm := range c {
|
|
|
|
print(elm, ",")
|
|
|
|
}
|
|
|
|
println()
|
|
|
|
|
|
|
|
d := func() [4]byte {
|
|
|
|
return [4]byte{12, 13}
|
|
|
|
}()
|
|
|
|
for _, elm := range d {
|
|
|
|
print(elm, ",")
|
|
|
|
}
|
|
|
|
println()
|
|
|
|
}
|
|
|
|
|
|
|
|
func stringTypeFunc(s stringType) stringType {
|
|
|
|
println(s)
|
|
|
|
return "stringType return" // skip
|
|
|
|
}
|
|
|
|
|
|
|
|
// obfuscating this broke before
|
|
|
|
const (
|
|
|
|
iota0 uint8 = iota
|
|
|
|
iota1
|
|
|
|
)
|
set positions when using cursor.Replace
The regular obfuscation process simply modifies some simple nodes, such
as identifiers and strings. In those cases, we modify the nodes
in-place, meaning that their positions remain the same. This hasn't
caused any problems.
Literal obfuscation is trickier. Since we replace one expression with an
entirely different one, we use cursor.Replace. The new expression is
entirely made up on the spot, so it lacks position information.
This was causing problems. For example, in the added test input:
> garble -literals build
[stderr]
# test/main
dgcm4t6w.go:3: misplaced compiler directive
dgcm4t6w.go:4: misplaced compiler directive
dgcm4t6w.go:3: misplaced compiler directive
dgcm4t6w.go:6: misplaced compiler directive
dgcm4t6w.go:7: misplaced compiler directive
dgcm4t6w.go:3: misplaced compiler directive
dgcm4t6w.go:9: misplaced compiler directive
dgcm4t6w.go:3: misplaced compiler directive
dgcm4t6w.go:3: too many errors
The build errors are because we'd move the compiler directives, which
makes the compiler unhappy as they must be directly followed by a
function declaration.
The root cause there seems to be that, since the replacement nodes lack
position information, go/printer would try to estimate its printing
position by adding to the last known position. Since -literals adds
code, this would result in the printer position increasing rapidly, and
potentially printing directive comments earlier than needed.
For now, making the replacement nodes have the same position as the
original node seems to stop go/printer from making this mistake.
It's possible that this workaround won't be bulletproof forever, but it
works well for now, and I don't see a simpler workaround right now.
It would be possible to use fancier mechanisms like go/ast.CommentMap or
dave/dst, but those are a significant amount of added complexity as well.
Fixes #285.
3 years ago
|
|
|
-- directives.go --
|
|
|
|
// If we misplace any of the directives below,
|
|
|
|
// cmd/compile will complain with "misplaced compiler directive".
|
|
|
|
//
|
|
|
|
// We use many literals and functions, mixing different types,
|
|
|
|
// so that it's more likely that bugs will be caught.
|
|
|
|
package main
|
|
|
|
|
|
|
|
//go:noinline
|
|
|
|
func str0() { println("foo") }
|
|
|
|
|
|
|
|
//go:noinline
|
|
|
|
func str1() { println("foo") }
|
|
|
|
|
|
|
|
//go:noinline
|
|
|
|
func str2() { println("foo") }
|
|
|
|
|
|
|
|
//go:noinline
|
|
|
|
func str3() { println("foo") }
|
|
|
|
|
|
|
|
//go:noinline
|
|
|
|
func str4() { println("foo") }
|
|
|
|
|
|
|
|
//go:noinline
|
|
|
|
func arr0() { println(len([...]byte{0, 1, 2})) }
|
|
|
|
|
|
|
|
//go:noinline
|
|
|
|
func arr1() { println(len([...]byte{0, 1, 2})) }
|
|
|
|
|
|
|
|
//go:noinline
|
|
|
|
func slc0() { println([]byte{0, 1, 2}) }
|
|
|
|
|
|
|
|
//go:noinline
|
|
|
|
func slc1() { println([]byte{0, 1, 2}) }
|
|
|
|
|
|
|
|
//go:noinline
|
|
|
|
func str5() { println("foo") }
|
|
|
|
|
|
|
|
//go:noinline
|
|
|
|
func str6() { println("foo") }
|
|
|
|
|
|
|
|
//go:noinline
|
|
|
|
func str7() { println("foo") }
|
|
|
|
|
|
|
|
//go:noinline
|
|
|
|
func str8() { println("foo") }
|
|
|
|
|
|
|
|
//go:noinline
|
|
|
|
func str9() { println("foo") }
|
|
|
|
|
|
|
|
-- main.stderr --
|
|
|
|
Lorem true
|
|
|
|
First Line
|
|
|
|
Second Line total string
|
|
|
|
dolor
|
|
|
|
second assign
|
|
|
|
|
|
|
|
😅 😅
|
|
|
|
to obfuscate also obfuscate
|
|
|
|
new value
|
|
|
|
another literal
|
|
|
|
Skip this block also skip this
|
|
|
|
1 0 1
|
|
|
|
skip untyped const
|
|
|
|
skip typed const skip typed var skip typed var assign
|
|
|
|
stringTypeField String stringTypeField strType
|
|
|
|
stringType lambda func return
|
|
|
|
stringType func param
|
|
|
|
stringType return
|
|
|
|
foo
|
|
|
|
foo
|
|
|
|
3 6
|
|
|
|
12,13,
|
|
|
|
12,13,
|
|
|
|
12,13,
|
|
|
|
12,13,0,0,
|