deprecate using GOPRIVATE in favor of GOGARBLE (#427)
Piggybacking off of GOPRIVATE is great for a number of reasons:
* People tend to obfuscate private code, whose package paths will
generally be in GOPRIVATE already
* Its meaning and syntax are well understood
* It allows all the flexibility we need without adding our own env var
or config option
However, using GOPRIVATE directly has one main drawback.
It's fairly common to also want to obfuscate public dependencies,
to make the code in private packages even harder to follow.
However, using "GOPRIVATE=*" will result in two main downsides:
* GONOPROXY defaults to GOPRIVATE, so the proxy would be entirely disabled.
Downloading modules, such as when adding or updating dependencies,
or when the local cache is cold, can be less reliable.
* GONOSUMDB defaults to GOPRIVATE, so the sumdb would be entirely disabled.
Adding entries to go.sum, such as when adding or updating dependencies,
can be less secure.
We will continue to consume GOPRIVATE as a fallback,
but we now expect users to set GOGARBLE instead.
The new logic is documented in the README.
While here, rewrite some uses of "private" with "to obfuscate",
to make the code easier to follow and harder to misunderstand.
Fixes #276.
3 years ago
|
|
|
env GOGARBLE=test/main
|
simplify, improve, and test line obfuscation (#239)
First, remove the shuffling of the declarations list within each file.
This is what we used at the very start to shuffle positions. Ever since
we started obfuscating positions via //line comments, that has been
entirely unnecessary.
Second, add a proper test that will fail if we don't obfuscate line
numbers well enough. Filenames were already decently covered by other
tests.
Third, simplify the line obfuscation code. It does not require
astutil.Apply, and ranging over file.Decls is easier.
Finally, also obfuscate the position of top-level vars, since we only
used to do it for top-level funcs. Without that fix, the test would fail
as varLines was unexpectedly sorted.
4 years ago
|
|
|
|
|
|
|
|
garble build
|
|
|
|
|
exec ./main
|
testdata: use longer Go filenames for binsubstr
Every now and then, a CI run would fail:
FAIL: testdata/scripts/reflect.txt:7: unexpected match for ["main.go"] in main
These were rare, and very hard to reproduce or debug.
My best guess is that, since "main.go" is a short string and we use
random eight-character obfuscated filenames ending with ".go", it was
possible that the random filename happened to end in "main" in some
cases.
Given the base64 encoding, the chances of a single suffix collision are
about 0.000006%. Note, however, that a single obfuscated build will most
likely obfuscate many filenames, especially for the tests obfuscating
multiple packages. For a single CI run with many tests across three OSs,
the chances of any collision are likely very low, but realistic.
All this has a simple fix: use longer filenames to match with. We choose
"garble_main.go" since it's long enough, but also because it's still
clear it's a "main" Go file, and it's very unlikely to cause conflicts
with filenames in upstream Go given the "garble_" prefix.
4 years ago
|
|
|
! stdout 'garble_main\.go|garble_other_filename|is sorted'
|
|
|
|
|
|
testdata: use longer Go filenames for binsubstr
Every now and then, a CI run would fail:
FAIL: testdata/scripts/reflect.txt:7: unexpected match for ["main.go"] in main
These were rare, and very hard to reproduce or debug.
My best guess is that, since "main.go" is a short string and we use
random eight-character obfuscated filenames ending with ".go", it was
possible that the random filename happened to end in "main" in some
cases.
Given the base64 encoding, the chances of a single suffix collision are
about 0.000006%. Note, however, that a single obfuscated build will most
likely obfuscate many filenames, especially for the tests obfuscating
multiple packages. For a single CI run with many tests across three OSs,
the chances of any collision are likely very low, but realistic.
All this has a simple fix: use longer filenames to match with. We choose
"garble_main.go" since it's long enough, but also because it's still
clear it's a "main" Go file, and it's very unlikely to cause conflicts
with filenames in upstream Go given the "garble_" prefix.
4 years ago
|
|
|
! binsubstr main$exe 'garble_main.go' 'garble_other_filename'
|
simplify, improve, and test line obfuscation (#239)
First, remove the shuffling of the declarations list within each file.
This is what we used at the very start to shuffle positions. Ever since
we started obfuscating positions via //line comments, that has been
entirely unnecessary.
Second, add a proper test that will fail if we don't obfuscate line
numbers well enough. Filenames were already decently covered by other
tests.
Third, simplify the line obfuscation code. It does not require
astutil.Apply, and ranging over file.Decls is easier.
Finally, also obfuscate the position of top-level vars, since we only
used to do it for top-level funcs. Without that fix, the test would fail
as varLines was unexpectedly sorted.
4 years ago
|
|
|
|
|
|
|
|
[short] stop # no need to verify this with -short
|
|
|
|
|
|
|
|
|
|
go build
|
|
|
|
|
exec ./main
|
testdata: use longer Go filenames for binsubstr
Every now and then, a CI run would fail:
FAIL: testdata/scripts/reflect.txt:7: unexpected match for ["main.go"] in main
These were rare, and very hard to reproduce or debug.
My best guess is that, since "main.go" is a short string and we use
random eight-character obfuscated filenames ending with ".go", it was
possible that the random filename happened to end in "main" in some
cases.
Given the base64 encoding, the chances of a single suffix collision are
about 0.000006%. Note, however, that a single obfuscated build will most
likely obfuscate many filenames, especially for the tests obfuscating
multiple packages. For a single CI run with many tests across three OSs,
the chances of any collision are likely very low, but realistic.
All this has a simple fix: use longer filenames to match with. We choose
"garble_main.go" since it's long enough, but also because it's still
clear it's a "main" Go file, and it's very unlikely to cause conflicts
with filenames in upstream Go given the "garble_" prefix.
4 years ago
|
|
|
stdout 'garble_main.go'
|
|
|
|
|
stdout 'garble_other_filename'
|
simplify, improve, and test line obfuscation (#239)
First, remove the shuffling of the declarations list within each file.
This is what we used at the very start to shuffle positions. Ever since
we started obfuscating positions via //line comments, that has been
entirely unnecessary.
Second, add a proper test that will fail if we don't obfuscate line
numbers well enough. Filenames were already decently covered by other
tests.
Third, simplify the line obfuscation code. It does not require
astutil.Apply, and ranging over file.Decls is easier.
Finally, also obfuscate the position of top-level vars, since we only
used to do it for top-level funcs. Without that fix, the test would fail
as varLines was unexpectedly sorted.
4 years ago
|
|
|
stdout ':19: main'
|
|
|
|
|
stdout 'initPositions is sorted'
|
|
|
|
|
stdout 'varPositions is sorted'
|
simplify, improve, and test line obfuscation (#239)
First, remove the shuffling of the declarations list within each file.
This is what we used at the very start to shuffle positions. Ever since
we started obfuscating positions via //line comments, that has been
entirely unnecessary.
Second, add a proper test that will fail if we don't obfuscate line
numbers well enough. Filenames were already decently covered by other
tests.
Third, simplify the line obfuscation code. It does not require
astutil.Apply, and ranging over file.Decls is easier.
Finally, also obfuscate the position of top-level vars, since we only
used to do it for top-level funcs. Without that fix, the test would fail
as varLines was unexpectedly sorted.
4 years ago
|
|
|
|
|
|
|
|
-- go.mod --
|
|
|
|
|
module test/main
|
|
|
|
|
|
|
|
|
|
go 1.18
|
testdata: use longer Go filenames for binsubstr
Every now and then, a CI run would fail:
FAIL: testdata/scripts/reflect.txt:7: unexpected match for ["main.go"] in main
These were rare, and very hard to reproduce or debug.
My best guess is that, since "main.go" is a short string and we use
random eight-character obfuscated filenames ending with ".go", it was
possible that the random filename happened to end in "main" in some
cases.
Given the base64 encoding, the chances of a single suffix collision are
about 0.000006%. Note, however, that a single obfuscated build will most
likely obfuscate many filenames, especially for the tests obfuscating
multiple packages. For a single CI run with many tests across three OSs,
the chances of any collision are likely very low, but realistic.
All this has a simple fix: use longer filenames to match with. We choose
"garble_main.go" since it's long enough, but also because it's still
clear it's a "main" Go file, and it's very unlikely to cause conflicts
with filenames in upstream Go given the "garble_" prefix.
4 years ago
|
|
|
-- garble_main.go --
|
simplify, improve, and test line obfuscation (#239)
First, remove the shuffling of the declarations list within each file.
This is what we used at the very start to shuffle positions. Ever since
we started obfuscating positions via //line comments, that has been
entirely unnecessary.
Second, add a proper test that will fail if we don't obfuscate line
numbers well enough. Filenames were already decently covered by other
tests.
Third, simplify the line obfuscation code. It does not require
astutil.Apply, and ranging over file.Decls is easier.
Finally, also obfuscate the position of top-level vars, since we only
used to do it for top-level funcs. Without that fix, the test would fail
as varLines was unexpectedly sorted.
4 years ago
|
|
|
package main
|
|
|
|
|
|
|
|
|
|
import (
|
|
|
|
|
"fmt"
|
|
|
|
|
"runtime"
|
|
|
|
|
"sort"
|
|
|
|
|
)
|
|
|
|
|
|
|
|
|
|
var _, globalFile, globalLine, _ = runtime.Caller(0)
|
|
|
|
|
|
|
|
|
|
func init() {
|
|
|
|
|
_, file, line, _ := runtime.Caller(0)
|
|
|
|
|
fmt.Printf("%s:%d: init\n", file, line)
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
func main() {
|
|
|
|
|
fmt.Printf("%s:%d: global\n", globalFile, globalLine)
|
|
|
|
|
|
|
|
|
|
_, file, line, _ := runtime.Caller(0)
|
|
|
|
|
fmt.Printf("%s:%d: main\n", file, line)
|
|
|
|
|
|
|
|
|
|
funcDecl()
|
|
|
|
|
funcVar()
|
|
|
|
|
|
|
|
|
|
// initPositions is filled by ten consecutive funcs.
|
simplify, improve, and test line obfuscation (#239)
First, remove the shuffling of the declarations list within each file.
This is what we used at the very start to shuffle positions. Ever since
we started obfuscating positions via //line comments, that has been
entirely unnecessary.
Second, add a proper test that will fail if we don't obfuscate line
numbers well enough. Filenames were already decently covered by other
tests.
Third, simplify the line obfuscation code. It does not require
astutil.Apply, and ranging over file.Decls is easier.
Finally, also obfuscate the position of top-level vars, since we only
used to do it for top-level funcs. Without that fix, the test would fail
as varLines was unexpectedly sorted.
4 years ago
|
|
|
// If we are not shuffling or obfuscating line numbers,
|
|
|
|
|
// this list will be sorted.
|
|
|
|
|
// If we are, it's extremely unlikely it would remain sorted.
|
|
|
|
|
if sort.IsSorted(sort.StringSlice(initPositions)) {
|
|
|
|
|
fmt.Println("initPositions is sorted")
|
simplify, improve, and test line obfuscation (#239)
First, remove the shuffling of the declarations list within each file.
This is what we used at the very start to shuffle positions. Ever since
we started obfuscating positions via //line comments, that has been
entirely unnecessary.
Second, add a proper test that will fail if we don't obfuscate line
numbers well enough. Filenames were already decently covered by other
tests.
Third, simplify the line obfuscation code. It does not require
astutil.Apply, and ranging over file.Decls is easier.
Finally, also obfuscate the position of top-level vars, since we only
used to do it for top-level funcs. Without that fix, the test would fail
as varLines was unexpectedly sorted.
4 years ago
|
|
|
}
|
|
|
|
|
|
|
|
|
|
// Same as the above, but with vars.
|
|
|
|
|
if sort.IsSorted(sort.StringSlice(varPositions)) {
|
|
|
|
|
fmt.Println("varPositions is sorted")
|
simplify, improve, and test line obfuscation (#239)
First, remove the shuffling of the declarations list within each file.
This is what we used at the very start to shuffle positions. Ever since
we started obfuscating positions via //line comments, that has been
entirely unnecessary.
Second, add a proper test that will fail if we don't obfuscate line
numbers well enough. Filenames were already decently covered by other
tests.
Third, simplify the line obfuscation code. It does not require
astutil.Apply, and ranging over file.Decls is easier.
Finally, also obfuscate the position of top-level vars, since we only
used to do it for top-level funcs. Without that fix, the test would fail
as varLines was unexpectedly sorted.
4 years ago
|
|
|
}
|
|
|
|
|
|
|
|
|
|
// Adding "/*text*/" comments here is tricky,
|
|
|
|
|
// as we don't want to turn "a/b" into "a//*text*/b".
|
|
|
|
|
// We need "a/ /*text*/b" to preserve the syntax.
|
|
|
|
|
// The nested expression is needed to prevent spaces.
|
|
|
|
|
fmt.Printf("%v\n", 10*float64(3.0)/float64(4.0))
|
simplify, improve, and test line obfuscation (#239)
First, remove the shuffling of the declarations list within each file.
This is what we used at the very start to shuffle positions. Ever since
we started obfuscating positions via //line comments, that has been
entirely unnecessary.
Second, add a proper test that will fail if we don't obfuscate line
numbers well enough. Filenames were already decently covered by other
tests.
Third, simplify the line obfuscation code. It does not require
astutil.Apply, and ranging over file.Decls is easier.
Finally, also obfuscate the position of top-level vars, since we only
used to do it for top-level funcs. Without that fix, the test would fail
as varLines was unexpectedly sorted.
4 years ago
|
|
|
}
|
work around another go/printer bug to fix andybalholm/brotli
When obfuscating the following piece of code:
func issue_573(s struct{ f int }) {
var _ *int = &s.f
/*x*/
}
the function body would roughly end up printed as:
we would roughly end up with:
var _ *int = &dZ4xYx3N
/*x*/.rbg1IM3V
Note that the /*x*/ comment got moved earlier in the source code.
This happens because the new identifiers are longer, so the printer
thinks that the selector now ends past the comment.
That would be fine - we don't really mind where comments end up,
because these non-directive comments end up being removed anyway.
However, the resulting syntax is wrong, as the period for the selector
must be on the first line rather than the second.
This is a go/printer bug that we should fix upstream,
but until then, we must work around it in Go 1.18.x and 1.19.x.
The fix is somewhat obvious in hindsight. To reduce the chances that
go/printer will trip over comments and produce invalid syntax,
get rid of most comments before we use the printer.
We still keep the removal of comments after printing,
since go/printer consumes some comments in ast.Node Doc fields.
Add the minimized unit test case above, and add the upstream project
that found this bug to check-third-party.
andybalholm/brotli helps cover a compression algorithm and ccgo code
generation from C to Go, and it's also a fairly popular module,
particular with HTTP implementations which want pure-Go brotli.
While here, fix the check-third-party script: it was setting GOFLAGS
a bit too late, so it may run `go get` on the wrong mod file.
Fixes #573.
3 years ago
|
|
|
|
|
|
|
|
// Replacing `s` and `f` with obfuscated names triggered a bug in go/printer,
|
|
|
|
|
// where it would move `.f` after the comment, breaking the syntax.
|
|
|
|
|
func issue_573(s struct{ f int }) {
|
|
|
|
|
var _ *int = &s.f
|
|
|
|
|
/*x*/
|
|
|
|
|
}
|
testdata: use longer Go filenames for binsubstr
Every now and then, a CI run would fail:
FAIL: testdata/scripts/reflect.txt:7: unexpected match for ["main.go"] in main
These were rare, and very hard to reproduce or debug.
My best guess is that, since "main.go" is a short string and we use
random eight-character obfuscated filenames ending with ".go", it was
possible that the random filename happened to end in "main" in some
cases.
Given the base64 encoding, the chances of a single suffix collision are
about 0.000006%. Note, however, that a single obfuscated build will most
likely obfuscate many filenames, especially for the tests obfuscating
multiple packages. For a single CI run with many tests across three OSs,
the chances of any collision are likely very low, but realistic.
All this has a simple fix: use longer filenames to match with. We choose
"garble_main.go" since it's long enough, but also because it's still
clear it's a "main" Go file, and it's very unlikely to cause conflicts
with filenames in upstream Go given the "garble_" prefix.
4 years ago
|
|
|
-- garble_other_filename.go --
|
simplify, improve, and test line obfuscation (#239)
First, remove the shuffling of the declarations list within each file.
This is what we used at the very start to shuffle positions. Ever since
we started obfuscating positions via //line comments, that has been
entirely unnecessary.
Second, add a proper test that will fail if we don't obfuscate line
numbers well enough. Filenames were already decently covered by other
tests.
Third, simplify the line obfuscation code. It does not require
astutil.Apply, and ranging over file.Decls is easier.
Finally, also obfuscate the position of top-level vars, since we only
used to do it for top-level funcs. Without that fix, the test would fail
as varLines was unexpectedly sorted.
4 years ago
|
|
|
package main
|
|
|
|
|
|
|
|
|
|
import (
|
|
|
|
|
"fmt"
|
|
|
|
|
"runtime"
|
|
|
|
|
)
|
|
|
|
|
|
|
|
|
|
func funcDecl() {
|
|
|
|
|
_, file, line, _ := runtime.Caller(0)
|
|
|
|
|
fmt.Printf("%s:%d: func\n", file, line)
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
var funcVar = func() {
|
|
|
|
|
_, file, line, _ := runtime.Caller(0)
|
|
|
|
|
fmt.Printf("%s:%d: func var\n", file, line)
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
var initPositions []string
|
simplify, improve, and test line obfuscation (#239)
First, remove the shuffling of the declarations list within each file.
This is what we used at the very start to shuffle positions. Ever since
we started obfuscating positions via //line comments, that has been
entirely unnecessary.
Second, add a proper test that will fail if we don't obfuscate line
numbers well enough. Filenames were already decently covered by other
tests.
Third, simplify the line obfuscation code. It does not require
astutil.Apply, and ranging over file.Decls is easier.
Finally, also obfuscate the position of top-level vars, since we only
used to do it for top-level funcs. Without that fix, the test would fail
as varLines was unexpectedly sorted.
4 years ago
|
|
|
|
|
|
|
|
func curPos() string {
|
|
|
|
|
_, file, line, _ := runtime.Caller(1)
|
|
|
|
|
return fmt.Sprintf("%s:%d", file, line)
|
simplify, improve, and test line obfuscation (#239)
First, remove the shuffling of the declarations list within each file.
This is what we used at the very start to shuffle positions. Ever since
we started obfuscating positions via //line comments, that has been
entirely unnecessary.
Second, add a proper test that will fail if we don't obfuscate line
numbers well enough. Filenames were already decently covered by other
tests.
Third, simplify the line obfuscation code. It does not require
astutil.Apply, and ranging over file.Decls is easier.
Finally, also obfuscate the position of top-level vars, since we only
used to do it for top-level funcs. Without that fix, the test would fail
as varLines was unexpectedly sorted.
4 years ago
|
|
|
}
|
|
|
|
|
|
|
|
|
|
func init() { initPositions = append(initPositions, curPos()) }
|
|
|
|
|
func init() { initPositions = append(initPositions, curPos()) }
|
|
|
|
|
func init() { initPositions = append(initPositions, curPos()) }
|
|
|
|
|
func init() { initPositions = append(initPositions, curPos()) }
|
|
|
|
|
func init() { initPositions = append(initPositions, curPos()) }
|
|
|
|
|
func init() { initPositions = append(initPositions, curPos()) }
|
|
|
|
|
func init() { initPositions = append(initPositions, curPos()) }
|
|
|
|
|
func init() { initPositions = append(initPositions, curPos()) }
|
|
|
|
|
func init() { initPositions = append(initPositions, curPos()) }
|
|
|
|
|
func init() { initPositions = append(initPositions, curPos()) }
|
|
|
|
|
|
|
|
|
|
var varLine0 = curPos()
|
|
|
|
|
var varLine1 = curPos()
|
|
|
|
|
var varLine2 = curPos()
|
|
|
|
|
var varLine3 = curPos()
|
|
|
|
|
var varLine4 = curPos()
|
|
|
|
|
var varLine5 = curPos()
|
|
|
|
|
var varLine6 = curPos()
|
|
|
|
|
var varLine7 = curPos()
|
|
|
|
|
var varLine8 = curPos()
|
|
|
|
|
var varLine9 = curPos()
|
|
|
|
|
|
|
|
|
|
var varPositions = []string{
|
simplify, improve, and test line obfuscation (#239)
First, remove the shuffling of the declarations list within each file.
This is what we used at the very start to shuffle positions. Ever since
we started obfuscating positions via //line comments, that has been
entirely unnecessary.
Second, add a proper test that will fail if we don't obfuscate line
numbers well enough. Filenames were already decently covered by other
tests.
Third, simplify the line obfuscation code. It does not require
astutil.Apply, and ranging over file.Decls is easier.
Finally, also obfuscate the position of top-level vars, since we only
used to do it for top-level funcs. Without that fix, the test would fail
as varLines was unexpectedly sorted.
4 years ago
|
|
|
varLine0, varLine1, varLine2, varLine3, varLine4,
|
|
|
|
|
varLine5, varLine6, varLine7, varLine8, varLine9,
|
|
|
|
|
}
|
