|
|
|
# Note that this is the only test with a module where we rely on the detection
|
|
|
|
# of GOPRIVATE.
|
|
|
|
# Also note that, since this is the only test using "real" external modules
|
|
|
|
# fetched via GOPROXY, go.mod and go.sum should declare the dependencies.
|
|
|
|
|
|
|
|
# For now, use a throwaway module download cache instead of the host machine's.
|
|
|
|
# Usually it would be fine to reuse the host's, since we expose exact copies of
|
|
|
|
# some external modules in a local proxy, allowing 'go test' to work offline.
|
|
|
|
# However, for some reason, we end up with different hashes of the code for
|
|
|
|
# modules like rsc.io/quote, and it's unclear why. It might be a txtar-addmod bug.
|
|
|
|
# In any case, not worth our time to investigate right now, and "downloading"
|
|
|
|
# modules is instant since we just copy a handful of files.
|
|
|
|
#
|
|
|
|
# To reproduce the issue, remove the env line and run:
|
|
|
|
#
|
|
|
|
# go clean -modcache && go get -d rsc.io/quote@v1.5.2 && go test -short
|
|
|
|
env GOMODCACHE=$WORK/modcache
|
|
|
|
|
|
|
|
garble build -tags buildtag
|
|
|
|
exec ./main
|
|
|
|
cmp stdout main.stdout
|
|
|
|
|
|
|
|
! binsubstr main$exe 'ImportedVar' 'ImportedConst' 'ImportedFunc' 'ImportedType' 'main.go' 'test/main' 'imported.' 'NormalStruct' 'NormalExportedField' 'normalUnexportedField'
|
|
|
|
binsubstr main$exe 'ReflectInDefined' 'ExportedField2' 'unexportedField2'
|
|
|
|
|
|
|
|
[short] stop # checking that the build is reproducible is slow
|
|
|
|
|
|
|
|
# Also check that the binary is reproducible when many imports are involved.
|
initial support for build caching (#142)
As per the discussion in https://github.com/golang/go/issues/41145, it
turns out that we don't need special support for build caching in
-toolexec. We can simply modify the behavior of "[...]/compile -V=full"
and "[...]/link -V=full" so that they include garble's own version and
options in the printed build ID.
The part of the build ID that matters is the last, since it's the
"content ID" which is used to work out whether there is a need to redo
the action (build) or not. Since cmd/go parses the last word in the
output as "buildID=...", we simply add "+garble buildID=_/_/_/${hash}".
The slashes let us imitate a full binary build ID, but we assume that
the other components such as the action ID are not necessary, since the
only reader here is cmd/go and it only consumes the content ID.
The reported content ID includes the tool's original content ID,
garble's own content ID from the built binary, and the garble options
which modify how we obfuscate code. If any of the three changes, we
should use a different build cache key. GOPRIVATE also affects caching,
since a different GOPRIVATE value means that we might have to garble a
different set of packages.
Include tests, which mainly check that 'garble build -v' prints package
lines when we expect to always need to rebuild packages, and that it
prints nothing when we should be reusing the build cache even when the
built binary is missing.
After this change, 'go test' on Go 1.15.2 stabilizes at about 8s on my
machine, whereas it used to be at around 25s before.
4 years ago
|
|
|
# No packages should be rebuilt either, thanks to the build cache.
|
|
|
|
cp main$exe main_old$exe
|
|
|
|
rm main$exe
|
initial support for build caching (#142)
As per the discussion in https://github.com/golang/go/issues/41145, it
turns out that we don't need special support for build caching in
-toolexec. We can simply modify the behavior of "[...]/compile -V=full"
and "[...]/link -V=full" so that they include garble's own version and
options in the printed build ID.
The part of the build ID that matters is the last, since it's the
"content ID" which is used to work out whether there is a need to redo
the action (build) or not. Since cmd/go parses the last word in the
output as "buildID=...", we simply add "+garble buildID=_/_/_/${hash}".
The slashes let us imitate a full binary build ID, but we assume that
the other components such as the action ID are not necessary, since the
only reader here is cmd/go and it only consumes the content ID.
The reported content ID includes the tool's original content ID,
garble's own content ID from the built binary, and the garble options
which modify how we obfuscate code. If any of the three changes, we
should use a different build cache key. GOPRIVATE also affects caching,
since a different GOPRIVATE value means that we might have to garble a
different set of packages.
Include tests, which mainly check that 'garble build -v' prints package
lines when we expect to always need to rebuild packages, and that it
prints nothing when we should be reusing the build cache even when the
built binary is missing.
After this change, 'go test' on Go 1.15.2 stabilizes at about 8s on my
machine, whereas it used to be at around 25s before.
4 years ago
|
|
|
garble build -tags buildtag -v
|
|
|
|
! stderr .
|
|
|
|
bincmp main$exe main_old$exe
|
|
|
|
|
|
|
|
go build -tags buildtag
|
|
|
|
exec ./main
|
|
|
|
cmp stdout main.stdout
|
|
|
|
|
|
|
|
# Also check that -literals doesn't break anything
|
|
|
|
garble -literals build -tags buildtag
|
|
|
|
exec ./main
|
|
|
|
cmp stdout main.stdout
|
|
|
|
|
|
|
|
-- go.mod --
|
|
|
|
module test/main
|
|
|
|
|
|
|
|
go 1.15
|
|
|
|
|
make the handling of import paths more robust
First, make isPrivate panic on malformed import paths, since that should
never happen. This catches the errors that some users had run into with
packages like gopkg.in/yaml.v2 and github.com/satori/go.uuid:
panic: malformed import path "gopkg.in/garbletest%2ev2": invalid char '%'
This seems to trigger when a module path contains a dot after the first
element, *and* that module is fetched via the proxy. This results in the
toolchain URL-encoding the second dot, and garble ends up seeing that
encoded path.
We reproduce this behavior with a fake gopkg.in module added to the test
module proxy. Using yaml.v2 directly would have been easier, but it's
pretty large. Note that we tried a replace directive, but that does not
trigger the URL-encoding bug.
Also note that we do not obfuscate the gopkg.in package; that's fine, as
the isPrivate path validity check catches the bug either way.
For now, make initImport use url.PathUnescape to work around this issue.
The underlying bug is likely in either the goobj2 fork, or in the
upstream Go toolchain itself.
hashImport also gives a better error if it cannot find a package now,
rather than just an "empty seed" panic.
Finally, the sanity check in isPrivate unearthed the fact that we do not
support garbling test packages at all, since they were invalid paths
which never matched GOPRIVATE. Add an explicit check and TODO about
that.
Fixes #224.
Fixes #228.
3 years ago
|
|
|
require (
|
|
|
|
rsc.io/quote v1.5.2
|
|
|
|
gopkg.in/garbletest.v2 v2.999.0
|
|
|
|
)
|
|
|
|
-- go.sum --
|
|
|
|
golang.org/x/text v0.0.0-20170915032832-14c0d48ead0c h1:pvCbr/wm8HzDD3fVywevekufpn6tCGPY3spdHeZJEsw=
|
|
|
|
golang.org/x/text v0.0.0-20170915032832-14c0d48ead0c/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
|
make the handling of import paths more robust
First, make isPrivate panic on malformed import paths, since that should
never happen. This catches the errors that some users had run into with
packages like gopkg.in/yaml.v2 and github.com/satori/go.uuid:
panic: malformed import path "gopkg.in/garbletest%2ev2": invalid char '%'
This seems to trigger when a module path contains a dot after the first
element, *and* that module is fetched via the proxy. This results in the
toolchain URL-encoding the second dot, and garble ends up seeing that
encoded path.
We reproduce this behavior with a fake gopkg.in module added to the test
module proxy. Using yaml.v2 directly would have been easier, but it's
pretty large. Note that we tried a replace directive, but that does not
trigger the URL-encoding bug.
Also note that we do not obfuscate the gopkg.in package; that's fine, as
the isPrivate path validity check catches the bug either way.
For now, make initImport use url.PathUnescape to work around this issue.
The underlying bug is likely in either the goobj2 fork, or in the
upstream Go toolchain itself.
hashImport also gives a better error if it cannot find a package now,
rather than just an "empty seed" panic.
Finally, the sanity check in isPrivate unearthed the fact that we do not
support garbling test packages at all, since they were invalid paths
which never matched GOPRIVATE. Add an explicit check and TODO about
that.
Fixes #224.
Fixes #228.
3 years ago
|
|
|
gopkg.in/garbletest.v2 v2.999.0 h1:wiZfOKGiXX7DoYVgbNvnTaCjqElrpZQSvKg0HYouw/o=
|
|
|
|
gopkg.in/garbletest.v2 v2.999.0/go.mod h1:MF1BPTBjmDdc9x86+9UMLL9pAH2eMFPHvltohOvlGEw=
|
|
|
|
rsc.io/quote v1.5.2 h1:3fEykkD9k7lYzXqCYrwGAf7iNhbk4yCjHmKBN9td4L0=
|
|
|
|
rsc.io/quote v1.5.2/go.mod h1:LzX7hefJvL54yjefDEDHNONDjII0t9xZLPXsUe+TKr0=
|
|
|
|
rsc.io/sampler v1.3.0 h1:HLGR/BgEtI3r0uymSP/nl2uPLsUnNJX8toRyhfpBTII=
|
|
|
|
rsc.io/sampler v1.3.0/go.mod h1:T1hPZKmBbMNahiBKFy5HrXp6adAjACjK9JXDnKaTXpA=
|
|
|
|
-- main.go --
|
|
|
|
package main
|
|
|
|
|
|
|
|
import (
|
|
|
|
"fmt"
|
|
|
|
"reflect"
|
fix garbling names belonging to indirect imports (#203)
main.go includes a lengthy comment that documents this edge case, why it
happened, and how we are fixing it. To summarize, we should no longer
error with a build error in those cases. Read the comment for details.
A few other minor changes were done to allow writing this patch.
First, the actionID and contentID funcs were renamed, since they started
to collide with variable names.
Second, the logging has been improved a bit, which allowed me to debug
the issue.
Third, the "cache" global shared by all garble sub-processes now
includes the necessary parameters to run "go list -toolexec", including
the path to garble and the build flags being used.
Thanks to lu4p for writing a test case, which also applied gofmt to that
testdata Go file.
Fixes #180.
Closes #181, since it includes its test case.
4 years ago
|
|
|
"strings"
|
|
|
|
|
|
|
|
"test/main/imported"
|
|
|
|
|
|
|
|
"rsc.io/quote"
|
make the handling of import paths more robust
First, make isPrivate panic on malformed import paths, since that should
never happen. This catches the errors that some users had run into with
packages like gopkg.in/yaml.v2 and github.com/satori/go.uuid:
panic: malformed import path "gopkg.in/garbletest%2ev2": invalid char '%'
This seems to trigger when a module path contains a dot after the first
element, *and* that module is fetched via the proxy. This results in the
toolchain URL-encoding the second dot, and garble ends up seeing that
encoded path.
We reproduce this behavior with a fake gopkg.in module added to the test
module proxy. Using yaml.v2 directly would have been easier, but it's
pretty large. Note that we tried a replace directive, but that does not
trigger the URL-encoding bug.
Also note that we do not obfuscate the gopkg.in package; that's fine, as
the isPrivate path validity check catches the bug either way.
For now, make initImport use url.PathUnescape to work around this issue.
The underlying bug is likely in either the goobj2 fork, or in the
upstream Go toolchain itself.
hashImport also gives a better error if it cannot find a package now,
rather than just an "empty seed" panic.
Finally, the sanity check in isPrivate unearthed the fact that we do not
support garbling test packages at all, since they were invalid paths
which never matched GOPRIVATE. Add an explicit check and TODO about
that.
Fixes #224.
Fixes #228.
3 years ago
|
|
|
garbletest "gopkg.in/garbletest.v2"
|
|
|
|
)
|
|
|
|
|
|
|
|
func main() {
|
|
|
|
fmt.Println(imported.ImportedVar)
|
|
|
|
fmt.Println(imported.ImportedConst)
|
|
|
|
fmt.Println(imported.ImportedFunc('x'))
|
|
|
|
fmt.Println(imported.ImportedType(3))
|
|
|
|
fmt.Println(imported.ReflectInDefinedVar.ExportedField2)
|
|
|
|
fmt.Println(imported.ReflectInDefined{ExportedField2: 5})
|
fix garbling names belonging to indirect imports (#203)
main.go includes a lengthy comment that documents this edge case, why it
happened, and how we are fixing it. To summarize, we should no longer
error with a build error in those cases. Read the comment for details.
A few other minor changes were done to allow writing this patch.
First, the actionID and contentID funcs were renamed, since they started
to collide with variable names.
Second, the logging has been improved a bit, which allowed me to debug
the issue.
Third, the "cache" global shared by all garble sub-processes now
includes the necessary parameters to run "go list -toolexec", including
the path to garble and the build flags being used.
Thanks to lu4p for writing a test case, which also applied gofmt to that
testdata Go file.
Fixes #180.
Closes #181, since it includes its test case.
4 years ago
|
|
|
normal := imported.NormalStruct{SharedName: 3}
|
|
|
|
normal.IndirectStruct.Field = 23
|
|
|
|
fmt.Println(normal)
|
|
|
|
printfWithoutPackage("%T\n", imported.ReflectTypeOf(2))
|
|
|
|
printfWithoutPackage("%T\n", imported.ReflectTypeOfIndirect(4))
|
|
|
|
|
|
|
|
v := imported.ReflectValueOfVar
|
|
|
|
printfWithoutPackage("%#v\n", v)
|
|
|
|
method := reflect.ValueOf(&v).MethodByName("ExportedMethodName")
|
|
|
|
if method.IsValid() {
|
|
|
|
fmt.Println(method.Call(nil))
|
|
|
|
} else {
|
|
|
|
fmt.Println("method not found")
|
|
|
|
}
|
|
|
|
|
|
|
|
fmt.Println(quote.Go())
|
make the handling of import paths more robust
First, make isPrivate panic on malformed import paths, since that should
never happen. This catches the errors that some users had run into with
packages like gopkg.in/yaml.v2 and github.com/satori/go.uuid:
panic: malformed import path "gopkg.in/garbletest%2ev2": invalid char '%'
This seems to trigger when a module path contains a dot after the first
element, *and* that module is fetched via the proxy. This results in the
toolchain URL-encoding the second dot, and garble ends up seeing that
encoded path.
We reproduce this behavior with a fake gopkg.in module added to the test
module proxy. Using yaml.v2 directly would have been easier, but it's
pretty large. Note that we tried a replace directive, but that does not
trigger the URL-encoding bug.
Also note that we do not obfuscate the gopkg.in package; that's fine, as
the isPrivate path validity check catches the bug either way.
For now, make initImport use url.PathUnescape to work around this issue.
The underlying bug is likely in either the goobj2 fork, or in the
upstream Go toolchain itself.
hashImport also gives a better error if it cannot find a package now,
rather than just an "empty seed" panic.
Finally, the sanity check in isPrivate unearthed the fact that we do not
support garbling test packages at all, since they were invalid paths
which never matched GOPRIVATE. Add an explicit check and TODO about
that.
Fixes #224.
Fixes #228.
3 years ago
|
|
|
garbletest.Test()
|
|
|
|
}
|
|
|
|
|
|
|
|
func printfWithoutPackage(format string, v interface{}) {
|
|
|
|
fmt.Print(strings.Split(fmt.Sprintf(format, v), ".")[1])
|
|
|
|
}
|
|
|
|
-- notag_fail.go --
|
|
|
|
// +build !buildtag
|
|
|
|
|
|
|
|
package main
|
|
|
|
|
|
|
|
var foo int = "should be omitted by -tags"
|
|
|
|
-- withtag_success.go --
|
|
|
|
// +build buildtag
|
|
|
|
|
|
|
|
package main
|
|
|
|
|
|
|
|
import "fmt"
|
|
|
|
|
|
|
|
func init() { fmt.Println("buildtag init func") }
|
|
|
|
-- imported/imported.go --
|
|
|
|
package imported
|
|
|
|
|
fix garbling names belonging to indirect imports (#203)
main.go includes a lengthy comment that documents this edge case, why it
happened, and how we are fixing it. To summarize, we should no longer
error with a build error in those cases. Read the comment for details.
A few other minor changes were done to allow writing this patch.
First, the actionID and contentID funcs were renamed, since they started
to collide with variable names.
Second, the logging has been improved a bit, which allowed me to debug
the issue.
Third, the "cache" global shared by all garble sub-processes now
includes the necessary parameters to run "go list -toolexec", including
the path to garble and the build flags being used.
Thanks to lu4p for writing a test case, which also applied gofmt to that
testdata Go file.
Fixes #180.
Closes #181, since it includes its test case.
4 years ago
|
|
|
import (
|
|
|
|
"reflect"
|
|
|
|
"test/main/imported/indirect"
|
|
|
|
)
|
|
|
|
|
|
|
|
var ImportedVar = "imported var value"
|
|
|
|
|
|
|
|
const ImportedConst = "imported const value"
|
|
|
|
|
|
|
|
func ImportedFunc(param rune) string {
|
|
|
|
return string(param)
|
|
|
|
}
|
|
|
|
|
|
|
|
type ReflectTypeOf int
|
|
|
|
|
|
|
|
var _ = reflect.TypeOf(ReflectTypeOf(0))
|
|
|
|
|
|
|
|
type ReflectTypeOfIndirect int
|
|
|
|
|
|
|
|
var _ = reflect.TypeOf(new([]*ReflectTypeOfIndirect))
|
|
|
|
|
|
|
|
type ReflectValueOf struct {
|
fix garbling names belonging to indirect imports (#203)
main.go includes a lengthy comment that documents this edge case, why it
happened, and how we are fixing it. To summarize, we should no longer
error with a build error in those cases. Read the comment for details.
A few other minor changes were done to allow writing this patch.
First, the actionID and contentID funcs were renamed, since they started
to collide with variable names.
Second, the logging has been improved a bit, which allowed me to debug
the issue.
Third, the "cache" global shared by all garble sub-processes now
includes the necessary parameters to run "go list -toolexec", including
the path to garble and the build flags being used.
Thanks to lu4p for writing a test case, which also applied gofmt to that
testdata Go file.
Fixes #180.
Closes #181, since it includes its test case.
4 years ago
|
|
|
ExportedField string
|
|
|
|
|
|
|
|
unexportedField string
|
|
|
|
}
|
|
|
|
|
fix garbling names belonging to indirect imports (#203)
main.go includes a lengthy comment that documents this edge case, why it
happened, and how we are fixing it. To summarize, we should no longer
error with a build error in those cases. Read the comment for details.
A few other minor changes were done to allow writing this patch.
First, the actionID and contentID funcs were renamed, since they started
to collide with variable names.
Second, the logging has been improved a bit, which allowed me to debug
the issue.
Third, the "cache" global shared by all garble sub-processes now
includes the necessary parameters to run "go list -toolexec", including
the path to garble and the build flags being used.
Thanks to lu4p for writing a test case, which also applied gofmt to that
testdata Go file.
Fixes #180.
Closes #181, since it includes its test case.
4 years ago
|
|
|
func (r *ReflectValueOf) ExportedMethodName() string { return "method: " + r.ExportedField }
|
|
|
|
|
|
|
|
var ReflectValueOfVar = ReflectValueOf{ExportedField: "abc"}
|
|
|
|
|
|
|
|
var _ = reflect.TypeOf(ReflectValueOfVar)
|
|
|
|
|
|
|
|
type ReflectInDefined struct {
|
fix garbling names belonging to indirect imports (#203)
main.go includes a lengthy comment that documents this edge case, why it
happened, and how we are fixing it. To summarize, we should no longer
error with a build error in those cases. Read the comment for details.
A few other minor changes were done to allow writing this patch.
First, the actionID and contentID funcs were renamed, since they started
to collide with variable names.
Second, the logging has been improved a bit, which allowed me to debug
the issue.
Third, the "cache" global shared by all garble sub-processes now
includes the necessary parameters to run "go list -toolexec", including
the path to garble and the build flags being used.
Thanks to lu4p for writing a test case, which also applied gofmt to that
testdata Go file.
Fixes #180.
Closes #181, since it includes its test case.
4 years ago
|
|
|
ExportedField2 int
|
|
|
|
|
|
|
|
unexportedField2 int
|
|
|
|
}
|
|
|
|
|
|
|
|
var ReflectInDefinedVar = ReflectInDefined{ExportedField2: 9000}
|
|
|
|
|
|
|
|
var _ = reflect.TypeOf(ReflectInDefinedVar)
|
|
|
|
|
|
|
|
const SharedName = 2
|
|
|
|
|
|
|
|
type NormalStruct struct {
|
fix garbling names belonging to indirect imports (#203)
main.go includes a lengthy comment that documents this edge case, why it
happened, and how we are fixing it. To summarize, we should no longer
error with a build error in those cases. Read the comment for details.
A few other minor changes were done to allow writing this patch.
First, the actionID and contentID funcs were renamed, since they started
to collide with variable names.
Second, the logging has been improved a bit, which allowed me to debug
the issue.
Third, the "cache" global shared by all garble sub-processes now
includes the necessary parameters to run "go list -toolexec", including
the path to garble and the build flags being used.
Thanks to lu4p for writing a test case, which also applied gofmt to that
testdata Go file.
Fixes #180.
Closes #181, since it includes its test case.
4 years ago
|
|
|
SharedName int
|
|
|
|
IndirectStruct indirect.Indirect
|
|
|
|
normalUnexportedField int
|
|
|
|
}
|
|
|
|
|
|
|
|
// ImportedType comes after the calls to reflect, to ensure no false positives.
|
|
|
|
type ImportedType int
|
|
|
|
|
fix garbling names belonging to indirect imports (#203)
main.go includes a lengthy comment that documents this edge case, why it
happened, and how we are fixing it. To summarize, we should no longer
error with a build error in those cases. Read the comment for details.
A few other minor changes were done to allow writing this patch.
First, the actionID and contentID funcs were renamed, since they started
to collide with variable names.
Second, the logging has been improved a bit, which allowed me to debug
the issue.
Third, the "cache" global shared by all garble sub-processes now
includes the necessary parameters to run "go list -toolexec", including
the path to garble and the build flags being used.
Thanks to lu4p for writing a test case, which also applied gofmt to that
testdata Go file.
Fixes #180.
Closes #181, since it includes its test case.
4 years ago
|
|
|
-- imported/indirect/indirect.go --
|
|
|
|
package indirect
|
|
|
|
|
|
|
|
type Indirect struct {
|
|
|
|
Field int
|
|
|
|
}
|
|
|
|
|
|
|
|
-- main.stdout --
|
|
|
|
buildtag init func
|
|
|
|
imported var value
|
|
|
|
imported const value
|
|
|
|
x
|
|
|
|
3
|
|
|
|
9000
|
|
|
|
{5 0}
|
fix garbling names belonging to indirect imports (#203)
main.go includes a lengthy comment that documents this edge case, why it
happened, and how we are fixing it. To summarize, we should no longer
error with a build error in those cases. Read the comment for details.
A few other minor changes were done to allow writing this patch.
First, the actionID and contentID funcs were renamed, since they started
to collide with variable names.
Second, the logging has been improved a bit, which allowed me to debug
the issue.
Third, the "cache" global shared by all garble sub-processes now
includes the necessary parameters to run "go list -toolexec", including
the path to garble and the build flags being used.
Thanks to lu4p for writing a test case, which also applied gofmt to that
testdata Go file.
Fixes #180.
Closes #181, since it includes its test case.
4 years ago
|
|
|
{3 {23} 0}
|
|
|
|
ReflectTypeOf
|
|
|
|
ReflectTypeOfIndirect
|
|
|
|
ReflectValueOf{ExportedField:"abc", unexportedField:""}
|
|
|
|
[method: abc]
|
|
|
|
Don't communicate by sharing memory, share memory by communicating.
|