uber-go / atomic Goto Github PK
View Code? Open in Web Editor NEWWrapper types for sync/atomic which enforce atomic access
Home Page: https://go.uber.org/atomic
License: MIT License
Wrapper types for sync/atomic which enforce atomic access
Home Page: https://go.uber.org/atomic
License: MIT License
Setting minimum permissions to workflows is important to keep your repository safe against supply-chain attacks. I would like us to enforce least privilege access to workflows in the repository, similar to uber-go/multierr#76. The minimum permissions for the workflows, fossa.yaml
and go.yml
, would be contents: read
. Let me know if you agree with the changes and if you'd like to implement them, as done in uber-go/multierr, or otherwise I'm available to implement too.
I'm Gabriela and I work on behalf of Google and the OpenSSF suggesting supply-chain security changes :)
Hey,
I've encountered the following issue with atomic.String
atomicStr := atomic.NewString("")
atomicStr.Swap("string")
error
panic: interface conversion: interface {} is nil, not string [recovered]
panic: interface conversion: interface {} is nil, not string
go.uber.org/atomic.(*String).Swap(...)
/Users/dkrom/Dev/gonzo/pkg/mod/go.uber.org/atomic@v1.10.0/string.go:64
I guess its because NewString condition
func NewString(val string) *String {
x := &String{}
if val != _zeroString {
x.Store(val)
}
return x
}
I'm not sure if its by design or not?
go version go1.19.4 darwin/arm64
atomic @v1.10.0
Thanks a lot!
Tag v1.9.0
has support for go 1.13
(from go mod file) and in v1.10.0
it was changed to go 1.18
.
Isn't this a breaking change?
I want to upgrade dependencies of this project: https://github.com/actforgood/xconf and I get the following error:
go version
go version go1.16.4 darwin/amd64
go get -u
# go.uber.org/atomic
../../../go/pkg/mod/go.uber.org/[email protected]/error.go:55:12: x.v.CompareAndSwap undefined (type Value has no field or method CompareAndSwap)
../../../go/pkg/mod/go.uber.org/[email protected]/error.go:61:24: x.v.Swap undefined (type Value has no field or method Swap)
../../../go/pkg/mod/go.uber.org/[email protected]/string.go:58:12: x.v.CompareAndSwap undefined (type Value has no field or method CompareAndSwap)
../../../go/pkg/mod/go.uber.org/[email protected]/string.go:64:12: x.v.Swap undefined (type Value has no field or method Swap)
note: module requires Go 1.18
Methods Swap
and CompareAndSwap
from atomic.Value
were introduced in go1.17.
Bellow is a Dockerfile
the issue can be reproduced with.
Playing with image tag from top you can see that on 1.16 error is encountered, on 1.18 everything works smoothly.
FROM golang:1.16.0-alpine3.13
# FROM golang:1.18.0-alpine3.15
RUN apk add --no-cache git
RUN git clone https://github.com/actforgood/xconf /xconf
WORKDIR /xconf
CMD ["go", "get", "go.uber.org/[email protected]"]
docker build -q -f Dockerfile -t test_upgrade .
docker run --name test_container test_upgrade
docker rm test_container
docker image rm test_upgrade
What do you think of versioning your library?
Set git tag, make release and don't break the internet
Very minor thing: but here an example Float64 (same thing for Float32):
CAS
deprecated https://github.com/uber-go/atomic/blob/master/float64_ext.go#L49
Use of CAS
(same file) https://github.com/uber-go/atomic/blob/master/float64_ext.go#L35
Extra note: CAS is implemented by using the suggested replacement CompareAndSwap
(same file) https://github.com/uber-go/atomic/blob/master/float64_ext.go#L50
Could we add an atomic operation for BigDecimal?
panic using the uatomic.Duration Store on a 32-bit system
panic: unaligned 64-bit atomic operation
goroutine 132 [running]:
runtime/internal/atomic.panicUnaligned()
/home/ubuntu/BMI-HI3521A/go/src/runtime/internal/atomic/unaligned.go:8 +0x24
runtime/internal/atomic.Store64(0x45720ac, 0x3b9aca00)
/home/ubuntu/BMI-HI3521A/go/src/runtime/internal/atomic/atomic_arm.s:301 +0x14
go.uber.org/atomic.(*Int64).Store(...)
/StreamPlugins/gopath/pkg/mod/go.uber.org/[email protected]/int64.go:75
go.uber.org/atomic.(*Duration).Store(...)
/StreamPlugins/gopath/pkg/mod/go.uber.org/[email protected]/duration.go:55
Currently, it's possible to do,
x := atomic.NewInt32(1)
y := atomic.NewInt32(2)
fmt.Println(*x == *y)
This accesses the value of the integer in a non-atomic way.
We should consider disabling this by adding a _ [0]func()
field to the
atomic structs. This would add no runtime cost and disable struct-level
comparison of the values. (Ref: https://github.com/go4org/mem/blob/3dbcd07079b881f9bedb802b4099856c1e267fa1/mem.go#L42.)
Worth considering that this could constitute a breaking change. An argument
for in favor of the change is that the comparison was incorrect usage in the
first place, so it's not breaking to disallow it.
It would be very useful to have a similar atomic version for float32 as well for use cases where the precision of float64 is not necessary.
faced the problem while building for amd64 and ppc64le
assert_test.go:28:2: cannot find package "github.com/stretchr/testify/assert" in any of:
/home/travis/gopath/src/go.uber.org/atomic/vendor/github.com/stretchr/testify/assert (vendor tree)
/home/travis/.gimme/versions/go1.11.13.linux.amd64/src/github.com/stretchr/testify/assert (from $GOROOT)
/home/travis/gopath/src/github.com/stretchr/testify/assert (from $GOPATH)
FAIL go.uber.org/atomic [setup failed]
? go.uber.org/atomic/internal/gen-atomicint [no test files]
? go.uber.org/atomic/internal/gen-atomicwrapper [no test files]
Makefile:61: recipe for target 'cover' failed
make: *** [cover] Error 1
The command "make cover" exited with 2.
cache.2
store build cache
and add import line as below ,however it fails again as below can you check
go_import_path: github.com/stretchr/testify/assert
2.22s$ make cover
go test -coverprofile=cover.out -coverpkg github.com/stretchr/testify/assert,github.com/stretchr/testify/assert/internal/gen-atomicint,github.com/stretchr/testify/assert/internal/gen-atomicwrapper -v ./...
import cycle not allowed in test
package github.com/stretchr/testify/assert (test)
imports github.com/stretchr/testify/assert
FAIL github.com/stretchr/testify/assert [setup failed]
? github.com/stretchr/testify/assert/internal/gen-atomicint [no test files]
? github.com/stretchr/testify/assert/internal/gen-atomicwrapper [no test files]
Makefile:61: recipe for target 'cover' failed
make: *** [cover] Error 1
The command "make cover" exited with 2.
I am getting this error when Load()
happens before a call to Store()
(with bool as an example here).
[jacob@fedora fyne_demo]$ ./fyne_demo
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x816bba]
goroutine 1 [running, locked to thread]:
go.uber.org/atomic.(*Bool).Load(...)
/home/jacob/go/pkg/mod/go.uber.org/[email protected]/bool.go:49
I would personally have expected this to behave similarly to a regular type that would have it's default value returned when trying to access it before it has been changed. This would make the package a lot more useful for my use cases as I can avoid a lot of constructors or sync.Once
by just letting the atomic types be part of the struct fields and then stored when it changes.
Adding a Security Policy is important as it provides guidance on how to report potential vulnerabilities and inform the vulnerabilities disclosure window for this repo.
I recently recommended #132 and, like that change, this one also security-related.
If you agree, I can open a PR to suggest a Security Policy, and we can work together to communicate how the repo can best handle vulnerability reports.
Hi again! I'm Gabriela and I work on behalf of Google and the OpenSSF suggesting supply-chain security changes :)
Hello,
A new release with error.go included would be highly appreciated!
Thanks,
Should we move this library to a vanity import path?
Hi,
During package review requiring to update Kubo (IPfs) in Guix, I've faced with
an issue when I try to build github.com/uber/[email protected] on i686-linux machine:
# go.uber.org/atomic [go.uber.org/atomic.test]
src/go.uber.org/atomic/uintptr_test.go:72:29: constant 18446744073709551615 overflows uintptr
FAIL go.uber.org/atomic [build failed]
FAIL
To reproduce the issue:
guix time-machine --commit=42b4580f6bfcaab009a56e679bd6f3c408ef03c9 -- \
build --system=i686-linux go-go-uber-org-atomic
Thanks,
Oleg
Hi,golang 1.19 has support atomic.Pointer, how do you compatible with this feature? Thanks!
Atomic Pointers in Go 1.19
I'll be glad to work on it if it is good to go.
func LoadUintptr(addr *uintptr) (val uintptr)
func StoreUintptr(addr *uintptr, val uintptr)
func AddUintptr(addr *uintptr, delta uintptr) (new uintptr)
func SwapUintptr(addr *uintptr, new uintptr) (old uintptr)
func CompareAndSwapUintptr(addr *uintptr, old, new uintptr) (swapped bool)
func LoadPointer(addr *unsafe.Pointer) (val unsafe.Pointer)
func StorePointer(addr *unsafe.Pointer, val unsafe.Pointer)
func SwapPointer(addr *unsafe.Pointer, new unsafe.Pointer) (old unsafe.Pointer)
func CompareAndSwapPointer(addr *unsafe.Pointer, old, new unsafe.Pointer) (swapped bool)
If I have a struct with some atomic fields and try to marshal to json, all the json values are simply {}
Would be nice to have MarshalJSON() implementations for each type which return the underlying value by calling Load()
func TestTimeMarshal(t *testing.T) {
type Test struct {
Field1 time.Time
Field2 *atomic.Time
}
obj := &Test{Field1: time.Now()}
obj.Field2 = atomic.NewTime(obj.Field1)
jsonBytes, err := json.Marshal(obj)
if err != nil {
t.Error(err)
}
tmpMap := make(map[string]interface{})
err = json.Unmarshal(jsonBytes, &tmpMap)
if err != nil {
t.Error(err)
}
fmt.Println(fmt.Sprintf("JSON:\n%s", string(jsonBytes)))
if tmpMap["Field1"] != tmpMap["Field2"] { // Field2 is not even a string as of right now
t.Error("Marshaling not equal")
}
}
Output:
=== RUN TestTimeMarshal
JSON:
{"Field1":"2022-11-25T21:28:20.1068541+01:00","Field2":{}}
utils_test.go:143: Marshaling not equal
--- FAIL: TestTimeMarshal (0.00s)
FAIL
As demonstrated, Go's std time.Time is able to marshal to JSON but atomic.Time doesn't do the same.
Was there a reason behind this?
I don't find the dependent command bin/gen-atomicint
.
uname -a
Linux onecloud 6.1.63-current-meson #1 SMP Mon Nov 20 10:52:19 UTC 2023 armv7l GNU/Linux
goroutine 14 [running]:
runtime/internal/atomic.panicUnaligned()
/Users/z/.g/go[表情]c/runtime/internal/atomic/unaligned.go:8 +0x24
runtime/internal/atomic.Load64(0x10c61cc)
/Users/z/.g/go[表情]c/runtime/internal/atomic/atomic_arm.s:280 +0x14
go.uber.org/atomic.(*Int64).Load(...)
/Users/z/go/pkg/mod/go.uber.org/[email protected]/int64.go:45
While writing a fix + tests for #126, I realized where was a larger issue with how we're handling default values for wrapper-types. We have 2 representations for a default value: the underlying atomic Value is nil
, or the atomic Value has the default value. However, having 2 representations causes issues for CompareAndSwap
, since we only ever compare against the default value, so we'll never match the nil
value.
This simple test shows the issue:
var s atomic.String
fmt.Println(s.CompareAndSwap("", "foo"))
// Prints: false. Expect true.
This can be worse if there's a loop using CompareAndSwap
(typical way to use CompareAndSwap
), since it will never succeed when the old value is read from Load()
,
// s is an atomic.String.
// If s has never been set, the following loop will never end
for {
old := s.Load()
new := transform(old)
if s.CompareAndSwap(old, new) {
return nil
}
}
The same above code will work for atomic.Int32
and other wrappers around int primitives. This is only an issue with Value
primitives. From my testing, this affects String and Error, added in #111.
many atomic value support MarshalJSON
and UnmarshalJSON
, any plan for Pointer[T]
?
Uint64 and Uint32 only provide an Add which takes another uint. This is a little tricky to use if you want to subtract a value, see:
https://godoc.org/sync/atomic#AddUint32
We can either document this, or provide a Sub
or Subtract
helper.
Since the go1.19
update, there are lots of duplicate functionality between this package and sync/atomic
. This duplication is currently needed because go1.18
is still supported.
sync/atomic
has the following types that are duplicated here:
Bool
Int32
Int64
Pointer[T]
Uint32
Uint64
Uintptr
All types in sync/atomic
have the following methods:
CompareAndSwap
Load
Store
Swap
Integer-like types have the following, in addition to the above:
Add
But there is still some functionality in here go.uber.org/atomic
that is not there in sync/atomic
.
All thing considered, after go1.18
becomes unsupported, we could do the following:
gen-atomicint
to embed sync/atomic
types. Only the following additional methodsCAS
(Deprecated)Dec
Inc
Sub
String
MarshallJSON
UnmarshallJSON
Bool
to embed sync/atomic
type. Only the following additional methodsCAS
(Deprecated)String
Toggle
MarshallJSON
UnmarshallJSON
Pointer[T]
to embed sync/atomic
type.I just wanted to document all this stuff before I forget.
node_exporter]# make
checking code style
checking license header
running golangci-lint
GO111MODULE=on go list -e -compiled -test=true -export=false -deps=true -find=false -tags= -- ./... > /dev/null
go: go.uber.org/[email protected]: unrecognized import path "go.uber.org/atomic" (https fetch: Get https://go.uber.org/atomic?go-get=1: proxyconnect tcp: tls: first record does not look like a TLS handshake)
make: *** [common-lint] Error 1
node_exporter]# curl -I https://go.uber.org
HTTP/1.0 200 Connection established
HTTP/1.1 405 Method Not Allowed
Date: Fri, 27 Mar 2020 08:22:22 GMT
Content-Type: text/plain; charset=utf-8
Content-Length: 19
Allow: GET, OPTIONS
X-Content-Type-Options: nosniff
Via: 1.1 google
There have been a few small enhancements made since the 1.0 release - can we cut a point release?
I was wondering whether the noCopy
struct should be embedded to all of the atomic structs, as it is undefined to copy these values around without corresponding Load / Store operations.
if we embed noCopy as empty struct there shouldn't be any performance hit on struct sizes.
go data types uint8 and uint16 is not supported on "go.uber.org/atomic"
we required these datatypes supported on atomic for optimal memory and performance
Line 37 in 544d6aa
Why NewError need judge _zeroError
because I want to use this code to store the first error
err := NewError(nil)
err.CompareAndSwap(nil, errors.New("error message"))
because of _zeroError
I must use
err := NewError(nil)
err.Store(nil)
err.CompareAndSwap(nil, errors.New("error message"))
Hi,
The latest release introduced a new test for uintptr
(05e06dc). This test is now failing on armhf with:
github.com/uber-go/atomic/internal/gen-atomicint
cd obj-arm-linux-gnueabihf && go test -vet=off -v -p 4 github.com/uber-go/atomic github.com/uber-go/atomic/internal/gen-atomicint github.com/uber-go/atomic/internal/gen-atomicwrapper
# github.com/uber-go/atomic [github.com/uber-go/atomic.test]
src/github.com/uber-go/atomic/uintptr_test.go:72:29: constant 18446744073709551615 overflows uintptr
If you want to see the full log, you can check this URL:
On Fedora Rawhide with Go 1.16 beta 1, TestNocmpIntegration fails with:
testing: go.uber.org/atomic
go.uber.org/atomic
--- FAIL: TestNocmpIntegration (0.00s)
nocmp_test.go:163:
Error Trace: nocmp_test.go:163
Error: "missing $GOPATH\n" does not contain "struct containing nocmp cannot be compared"
Test: TestNocmpIntegration
FAIL
exit status 1
FAIL go.uber.org/atomic 0.018s
I suspect default gopath was removed in 1.16 thus you need to add it to the environment variables. I've used the following patch successfully but I'm not sure this is the correct way:
From 742e71bb16a909acd89bc9c8178ed1a14d09b9c3 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Robert-Andr=C3=A9=20Mauchin?= <[email protected]>
Date: Mon, 25 Jan 2021 04:13:53 +0100
Subject: [PATCH 1/1] Add missing GOPATH environment variable
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Signed-off-by: Robert-André Mauchin <[email protected]>
---
nocmp_test.go | 2 ++
1 file changed, 2 insertions(+)
diff --git a/nocmp_test.go b/nocmp_test.go
index d4be166..2156350 100644
--- a/nocmp_test.go
+++ b/nocmp_test.go
@@ -155,6 +155,8 @@ func TestNocmpIntegration(t *testing.T) {
// Forget OS build enviroment and set up a minimal one for "go build"
// to run.
cmd.Env = []string{
+ "GO111MODULE=off",
+ "GOPATH=" + filepath.Join(tempdir, "src"),
"GOCACHE=" + filepath.Join(tempdir, "gocache"),
}
cmd.Stderr = &stderr
--
2.29.2
var b atomic.Bool
b.Store(true) // true
b.Toggle() // false
b.CAS(false, true) // true ?
fmt.Println(b.Load()) // "false"
CAS
assumes a direct equality check with 1
or 0
but the rest of the functions only look at the lowest bit, which Toggle
takes advantage of.
The time.Duration wrapper is quite convenient and I would like to see one for time.Time as well
If I have an atomic.String and I try to use fmt.Sprintf("value: %v", atomicStr) I would expect "value: foo" but instead I get "value: {{{foo}}}"
Referencing actions by commit SHA in GitHub workflows, guarantees you are using an immutable version. Actions referenced by tags and branches are vulnerable to attacks, such as the tag being moved to a malicious commit, a malicious commit being pushed to the branch or typosquatting.
Although there are pros and cons for each reference, GitHub understands SHAs are more reliable, as does Scorecard security tool. This is specially important when using third-party actions.
If you agree, this would change, for example:
actions/checkout@v2 => actions/checkout@ee0669bd1cc54295c223e0bb666b733df41de1c5 # v2.7.0
The commit SHA is followed by a comment to keep the version readable. Both the SHA and the comment can be updated by Dependabot or Renovatebot for dependency management.
Additionally, we can take this moment to bump actions/checkout
to v4
and bump other actions too.
About me, I'm Gabriela and I work on behalf of Google and the OpenSSF suggesting supply-chain security changes :)
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.