viant / afs Goto Github PK

Hi!

I want to set the content type of the file when i save it to cloud storage (ex: image/jpeg), but cant find the correct option for my call:

wc, err := afs.New().NewWriter(ctx, dest, 0644)

(by the way thanks for this package, its awesome!)

Hi.
Tried to run some tests - but caught this runtime error:

$ go test -v -x
WORK=/tmp/go-build367362019
mkdir -p $WORK/b001/
mkdir -p $WORK/b024/
mkdir -p $WORK/b016/
mkdir -p $WORK/b040/
mkdir -p $WORK/b043/
mkdir -p $WORK/b042/
mkdir -p $WORK/b025/
mkdir -p $WORK/b044/
mkdir -p $WORK/b049/
mkdir -p $WORK/b012/
mkdir -p $WORK/b050/
mkdir -p $WORK/b039/
mkdir -p $WORK/b037/
mkdir -p $WORK/b057/
mkdir -p $WORK/b056/
mkdir -p $WORK/b079/
mkdir -p $WORK/b077/
mkdir -p $WORK/b076/
mkdir -p $WORK/b082/
mkdir -p $WORK/b084/
mkdir -p $WORK/b083/
mkdir -p $WORK/b080/
mkdir -p $WORK/b089/
mkdir -p $WORK/b090/
mkdir -p $WORK/b059/
mkdir -p $WORK/b091/
mkdir -p $WORK/b092/
mkdir -p $WORK/b096/
mkdir -p $WORK/b093/
mkdir -p $WORK/b053/
mkdir -p $WORK/b097/
mkdir -p $WORK/b008/
mkdir -p $WORK/b099/
mkdir -p $WORK/b098/
cat >$WORK/b001/importcfg.link << 'EOF' # internal
packagefile /home/oceanfish81/other_golang/afs.test=/home/oceanfish81/.cache/go-build/ff/fff91e1020f228c2c6937deebda67239b8f01b222833b4d42e8193322d1fc2b7-d
packagefile /home/oceanfish81/other_golang/afs=/home/oceanfish81/.cache/go-build/49/499cce08ff4968b086994ee7b46063c89fdc0451cda3a81cc6d706602884b519-d
packagefile /home/oceanfish81/other_golang/afs_test=/home/oceanfish81/.cache/go-build/c4/c4a5e50e3037587830d2a953c4eaf9e98cbf5d4b454e1ac9111a61f7130cfcb5-d
packagefile github.com/stretchr/testify/assert=/home/oceanfish81/go/pkg/gccgo_linux_amd64/github.com/stretchr/testify/libassert.a
packagefile github.com/viant/afs/asset=/home/oceanfish81/go/pkg/gccgo_linux_amd64/github.com/viant/afs/libasset.a
packagefile github.com/viant/afs/file=/home/oceanfish81/go/pkg/gccgo_linux_amd64/github.com/viant/afs/libfile.a
packagefile github.com/viant/afs/option=/home/oceanfish81/go/pkg/gccgo_linux_amd64/github.com/viant/afs/liboption.a
packagefile github.com/viant/afs/scp=/home/oceanfish81/go/pkg/gccgo_linux_amd64/github.com/viant/afs/libscp.a
packagefile github.com/viant/afs/storage=/home/oceanfish81/go/pkg/gccgo_linux_amd64/github.com/viant/afs/libstorage.a
packagefile github.com/viant/afs/url=/home/oceanfish81/go/pkg/gccgo_linux_amd64/github.com/viant/afs/liburl.a
packagefile github.com/pkg/errors=/home/oceanfish81/go/pkg/gccgo_linux_amd64/github.com/pkg/liberrors.a
packagefile github.com/viant/afs/base=/home/oceanfish81/go/pkg/gccgo_linux_amd64/github.com/viant/afs/libbase.a
packagefile github.com/viant/afs/http=/home/oceanfish81/go/pkg/gccgo_linux_amd64/github.com/viant/afs/libhttp.a
packagefile github.com/viant/afs/matcher=/home/oceanfish81/go/pkg/gccgo_linux_amd64/github.com/viant/afs/libmatcher.a
packagefile github.com/viant/afs/mem=/home/oceanfish81/go/pkg/gccgo_linux_amd64/github.com/viant/afs/libmem.a
packagefile github.com/viant/afs/ssh=/home/oceanfish81/go/pkg/gccgo_linux_amd64/github.com/viant/afs/libssh.a
packagefile github.com/viant/afs/tar=/home/oceanfish81/go/pkg/gccgo_linux_amd64/github.com/viant/afs/libtar.a
packagefile github.com/viant/afs/walker=/home/oceanfish81/go/pkg/gccgo_linux_amd64/github.com/viant/afs/libwalker.a
packagefile github.com/viant/afs/zip=/home/oceanfish81/go/pkg/gccgo_linux_amd64/github.com/viant/afs/libzip.a
packagefile github.com/viant/afs=/home/oceanfish81/go/pkg/gccgo_linux_amd64/github.com/viant/libafs.a
packagefile github.com/davecgh/go-spew/spew=/home/oceanfish81/go/pkg/gccgo_linux_amd64/github.com/davecgh/go-spew/libspew.a
packagefile github.com/pmezard/go-difflib/difflib=/home/oceanfish81/go/pkg/gccgo_linux_amd64/github.com/pmezard/go-difflib/libdifflib.a
packagefile gopkg.in/yaml.v3=/home/oceanfish81/go/pkg/gccgo_linux_amd64/gopkg.in/libyaml.v3.a
packagefile github.com/viant/afs/object=/home/oceanfish81/go/pkg/gccgo_linux_amd64/github.com/viant/afs/libobject.a
packagefile golang.org/x/crypto/ssh=/home/oceanfish81/go/pkg/gccgo_linux_amd64/golang.org/x/crypto/libssh.a
packagefile github.com/go-errors/errors=/home/oceanfish81/go/pkg/gccgo_linux_amd64/github.com/go-errors/liberrors.a
packagefile github.com/viant/afs/archive=/home/oceanfish81/go/pkg/gccgo_linux_amd64/github.com/viant/afs/libarchive.a
packagefile golang.org/x/crypto/chacha20=/home/oceanfish81/go/pkg/gccgo_linux_amd64/golang.org/x/crypto/libchacha20.a
packagefile golang.org/x/crypto/curve25519=/home/oceanfish81/go/pkg/gccgo_linux_amd64/golang.org/x/crypto/libcurve25519.a
packagefile golang.org/x/crypto/ed25519=/home/oceanfish81/go/pkg/gccgo_linux_amd64/golang.org/x/crypto/libed25519.a
packagefile golang.org/x/crypto/poly1305=/home/oceanfish81/go/pkg/gccgo_linux_amd64/golang.org/x/crypto/libpoly1305.a
packagefile golang.org/x/crypto/ssh/internal/bcrypt_pbkdf=/home/oceanfish81/go/pkg/gccgo_linux_amd64/golang.org/x/crypto/ssh/internal/libbcrypt_pbkdf.a
packagefile golang.org/x/crypto/internal/subtle=/home/oceanfish81/go/pkg/gccgo_linux_amd64/golang.org/x/crypto/internal/libsubtle.a
packagefile golang.org/x/crypto/blowfish=/home/oceanfish81/go/pkg/gccgo_linux_amd64/golang.org/x/crypto/libblowfish.a
EOF
cd .
/usr/local/bin/llvm-goc -o $WORK/b001/afs.test "-Wl,-(" -m64 -Wl,--whole-archive /home/oceanfish81/.cache/go-build/ff/fff91e1020f228c2c6937deebda67239b8f01b222833b4d42e8193322d1fc2b7-d /home/oceanfish81/.cache/go-build/49/499cce08ff4968b086994ee7b46063c89fdc0451cda3a81cc6d706602884b519-d /home/oceanfish81/.cache/go-build/c4/c4a5e50e3037587830d2a953c4eaf9e98cbf5d4b454e1ac9111a61f7130cfcb5-d /home/oceanfish81/go/pkg/gccgo_linux_amd64/github.com/stretchr/testify/libassert.a /home/oceanfish81/go/pkg/gccgo_linux_amd64/github.com/viant/afs/libasset.a /home/oceanfish81/go/pkg/gccgo_linux_amd64/github.com/viant/afs/libfile.a /home/oceanfish81/go/pkg/gccgo_linux_amd64/github.com/viant/afs/liboption.a /home/oceanfish81/go/pkg/gccgo_linux_amd64/github.com/viant/afs/libscp.a /home/oceanfish81/go/pkg/gccgo_linux_amd64/github.com/viant/afs/libstorage.a /home/oceanfish81/go/pkg/gccgo_linux_amd64/github.com/viant/afs/liburl.a /home/oceanfish81/go/pkg/gccgo_linux_amd64/github.com/pkg/liberrors.a /home/oceanfish81/go/pkg/gccgo_linux_amd64/github.com/viant/afs/libbase.a /home/oceanfish81/go/pkg/gccgo_linux_amd64/github.com/viant/afs/libhttp.a /home/oceanfish81/go/pkg/gccgo_linux_amd64/github.com/viant/afs/libmatcher.a /home/oceanfish81/go/pkg/gccgo_linux_amd64/github.com/viant/afs/libmem.a /home/oceanfish81/go/pkg/gccgo_linux_amd64/github.com/viant/afs/libssh.a /home/oceanfish81/go/pkg/gccgo_linux_amd64/github.com/viant/afs/libtar.a /home/oceanfish81/go/pkg/gccgo_linux_amd64/github.com/viant/afs/libwalker.a /home/oceanfish81/go/pkg/gccgo_linux_amd64/github.com/viant/afs/libzip.a /home/oceanfish81/go/pkg/gccgo_linux_amd64/github.com/viant/libafs.a /home/oceanfish81/go/pkg/gccgo_linux_amd64/github.com/davecgh/go-spew/libspew.a /home/oceanfish81/go/pkg/gccgo_linux_amd64/github.com/pmezard/go-difflib/libdifflib.a /home/oceanfish81/go/pkg/gccgo_linux_amd64/gopkg.in/libyaml.v3.a /home/oceanfish81/go/pkg/gccgo_linux_amd64/github.com/viant/afs/libobject.a /home/oceanfish81/go/pkg/gccgo_linux_amd64/golang.org/x/crypto/libssh.a /home/oceanfish81/go/pkg/gccgo_linux_amd64/github.com/go-errors/liberrors.a /home/oceanfish81/go/pkg/gccgo_linux_amd64/github.com/viant/afs/libarchive.a /home/oceanfish81/go/pkg/gccgo_linux_amd64/golang.org/x/crypto/libchacha20.a /home/oceanfish81/go/pkg/gccgo_linux_amd64/golang.org/x/crypto/libcurve25519.a /home/oceanfish81/go/pkg/gccgo_linux_amd64/golang.org/x/crypto/libed25519.a /home/oceanfish81/go/pkg/gccgo_linux_amd64/golang.org/x/crypto/libpoly1305.a /home/oceanfish81/go/pkg/gccgo_linux_amd64/golang.org/x/crypto/ssh/internal/libbcrypt_pbkdf.a /home/oceanfish81/go/pkg/gccgo_linux_amd64/golang.org/x/crypto/internal/libsubtle.a /home/oceanfish81/go/pkg/gccgo_linux_amd64/golang.org/x/crypto/libblowfish.a -Wl,--no-whole-archive "-Wl,-)" -Wl,--build-id=0x6233466a383071436478784f506d7962474462722f6b485634634d7144524962414a6938514a37546d2f71454b4e354a614a4a6c7473426b5a73544473492f6233466a383071436478784f506d796247446272
cat >$WORK/b008/vet.cfg << 'EOF' # internal
{
"ID": "/home/oceanfish81/other_golang/afs",
"Compiler": "gccgo",
"Dir": "/home/oceanfish81/other_golang/afs",
"ImportPath": "/home/oceanfish81/other_golang/afs",
"GoFiles": [
"/home/oceanfish81/other_golang/afs/copy.go",
"/home/oceanfish81/other_golang/afs/doc.go",
"/home/oceanfish81/other_golang/afs/faker.go",
"/home/oceanfish81/other_golang/afs/init.go",
"/home/oceanfish81/other_golang/afs/list.go",
"/home/oceanfish81/other_golang/afs/move.go",
"/home/oceanfish81/other_golang/afs/registry.go",
"/home/oceanfish81/other_golang/afs/service.go",
"/home/oceanfish81/other_golang/afs/uploader.go",
"/home/oceanfish81/other_golang/afs/walker.go",
"/home/oceanfish81/other_golang/afs/writer.go",
"/home/oceanfish81/other_golang/afs/copy_test.go",
"/home/oceanfish81/other_golang/afs/faker_test.go",
"/home/oceanfish81/other_golang/afs/list_test.go",
"/home/oceanfish81/other_golang/afs/move_test.go",
"/home/oceanfish81/other_golang/afs/service_test.go",
"/home/oceanfish81/other_golang/afs/uploader_test.go",
"/home/oceanfish81/other_golang/afs/walker_test.go"
],
"NonGoFiles": [],
"ImportMap": {
"bytes": "bytes",
"context": "context",
"errors": "errors",
"fmt": "fmt",
"github.com/pkg/errors": "github.com/pkg/errors",
"github.com/stretchr/testify/assert": "github.com/stretchr/testify/assert",
"github.com/viant/afs/asset": "github.com/viant/afs/asset",
"github.com/viant/afs/base": "github.com/viant/afs/base",
"github.com/viant/afs/file": "github.com/viant/afs/file",
"github.com/viant/afs/http": "github.com/viant/afs/http",
"github.com/viant/afs/matcher": "github.com/viant/afs/matcher",
"github.com/viant/afs/mem": "github.com/viant/afs/mem",
"github.com/viant/afs/option": "github.com/viant/afs/option",
"github.com/viant/afs/scp": "github.com/viant/afs/scp",
"github.com/viant/afs/ssh": "github.com/viant/afs/ssh",
"github.com/viant/afs/storage": "github.com/viant/afs/storage",
"github.com/viant/afs/tar": "github.com/viant/afs/tar",
"github.com/viant/afs/url": "github.com/viant/afs/url",
"github.com/viant/afs/walker": "github.com/viant/afs/walker",
"github.com/viant/afs/zip": "github.com/viant/afs/zip",
"io": "io",
"io/ioutil": "io/ioutil",
"os": "os",
"path": "path",
"strings": "strings",
"sync": "sync",
"testing": "testing",
"time": "time"
},
"PackageFile": {
"github.com/pkg/errors": "/home/oceanfish81/go/pkg/gccgo_linux_amd64/github.com/pkg/liberrors.a",
"github.com/stretchr/testify/assert": "/home/oceanfish81/go/pkg/gccgo_linux_amd64/github.com/stretchr/testify/libassert.a",
"github.com/viant/afs/asset": "/home/oceanfish81/go/pkg/gccgo_linux_amd64/github.com/viant/afs/libasset.a",
"github.com/viant/afs/base": "/home/oceanfish81/go/pkg/gccgo_linux_amd64/github.com/viant/afs/libbase.a",
"github.com/viant/afs/file": "/home/oceanfish81/go/pkg/gccgo_linux_amd64/github.com/viant/afs/libfile.a",
"github.com/viant/afs/http": "/home/oceanfish81/go/pkg/gccgo_linux_amd64/github.com/viant/afs/libhttp.a",
"github.com/viant/afs/matcher": "/home/oceanfish81/go/pkg/gccgo_linux_amd64/github.com/viant/afs/libmatcher.a",
"github.com/viant/afs/mem": "/home/oceanfish81/go/pkg/gccgo_linux_amd64/github.com/viant/afs/libmem.a",
"github.com/viant/afs/option": "/home/oceanfish81/go/pkg/gccgo_linux_amd64/github.com/viant/afs/liboption.a",
"github.com/viant/afs/scp": "/home/oceanfish81/go/pkg/gccgo_linux_amd64/github.com/viant/afs/libscp.a",
"github.com/viant/afs/ssh": "/home/oceanfish81/go/pkg/gccgo_linux_amd64/github.com/viant/afs/libssh.a",
"github.com/viant/afs/storage": "/home/oceanfish81/go/pkg/gccgo_linux_amd64/github.com/viant/afs/libstorage.a",
"github.com/viant/afs/tar": "/home/oceanfish81/go/pkg/gccgo_linux_amd64/github.com/viant/afs/libtar.a",
"github.com/viant/afs/url": "/home/oceanfish81/go/pkg/gccgo_linux_amd64/github.com/viant/afs/liburl.a",
"github.com/viant/afs/walker": "/home/oceanfish81/go/pkg/gccgo_linux_amd64/github.com/viant/afs/libwalker.a",
"github.com/viant/afs/zip": "/home/oceanfish81/go/pkg/gccgo_linux_amd64/github.com/viant/afs/libzip.a"
},
"Standard": {
"bytes": true,
"context": true,
"errors": true,
"fmt": true,
"io": true,
"io/ioutil": true,
"os": true,
"path": true,
"strings": true,
"sync": true,
"testing": true,
"time": true
},
"PackageVetx": {
"github.com/pkg/errors": "/home/oceanfish81/.cache/go-build/2c/2cb0dea5d565bc000f409b063d0236c03f79d602c0bef46262451652b4331997-d",
"github.com/stretchr/testify/assert": "/home/oceanfish81/.cache/go-build/e3/e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855-d",
"github.com/viant/afs/asset": "/home/oceanfish81/.cache/go-build/e3/e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855-d",
"github.com/viant/afs/base": "/home/oceanfish81/.cache/go-build/e3/e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855-d",
"github.com/viant/afs/file": "/home/oceanfish81/.cache/go-build/e3/e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855-d",
"github.com/viant/afs/http": "/home/oceanfish81/.cache/go-build/e3/e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855-d",
"github.com/viant/afs/matcher": "/home/oceanfish81/.cache/go-build/e3/e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855-d",
"github.com/viant/afs/mem": "/home/oceanfish81/.cache/go-build/e3/e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855-d",
"github.com/viant/afs/option": "/home/oceanfish81/.cache/go-build/e3/e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855-d",
"github.com/viant/afs/scp": "/home/oceanfish81/.cache/go-build/e3/e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855-d",
"github.com/viant/afs/ssh": "/home/oceanfish81/.cache/go-build/e3/e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855-d",
"github.com/viant/afs/storage": "/home/oceanfish81/.cache/go-build/e3/e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855-d",
"github.com/viant/afs/tar": "/home/oceanfish81/.cache/go-build/e3/e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855-d",
"github.com/viant/afs/url": "/home/oceanfish81/.cache/go-build/e3/e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855-d",
"github.com/viant/afs/walker": "/home/oceanfish81/.cache/go-build/e3/e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855-d",
"github.com/viant/afs/zip": "/home/oceanfish81/.cache/go-build/e3/e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855-d"
},
"VetxOnly": false,
"VetxOutput": "$WORK/b008/vet.out",
"SucceedOnTypecheckFailure": false
}
EOF
cd /home/oceanfish81/other_golang/afs
TERM='dumb' GCCGO='/usr/local/bin/llvm-goc' /usr/local/tools/vet -atomic -bool -buildtags -errorsas -ifaceassert -nilfunc -printf -stringintconv $WORK/b008/vet.cfg
cat >$WORK/b098/vet.cfg << 'EOF' # internal
{
"ID": "/home/oceanfish81/other_golang/afs_test",
"Compiler": "gccgo",
"Dir": "/home/oceanfish81/other_golang/afs",
"ImportPath": "_/home/oceanfish81/other_golang/afs_test",
"GoFiles": [
"/home/oceanfish81/other_golang/afs/example_test.go"
],
"NonGoFiles": [],
"ImportMap": {
"bytes": "bytes",
"context": "context",
"fmt": "fmt",
"github.com/viant/afs": "github.com/viant/afs",
"github.com/viant/afs/asset": "github.com/viant/afs/asset",
"github.com/viant/afs/file": "github.com/viant/afs/file",
"github.com/viant/afs/matcher": "github.com/viant/afs/matcher",
"github.com/viant/afs/option": "github.com/viant/afs/option",
"github.com/viant/afs/scp": "github.com/viant/afs/scp",
"io": "io",
"io/ioutil": "io/ioutil",
"log": "log",
"path": "path",
"strings": "strings"
},
"PackageFile": {
"github.com/viant/afs": "/home/oceanfish81/go/pkg/gccgo_linux_amd64/github.com/viant/libafs.a",
"github.com/viant/afs/asset": "/home/oceanfish81/go/pkg/gccgo_linux_amd64/github.com/viant/afs/libasset.a",
"github.com/viant/afs/file": "/home/oceanfish81/go/pkg/gccgo_linux_amd64/github.com/viant/afs/libfile.a",
"github.com/viant/afs/matcher": "/home/oceanfish81/go/pkg/gccgo_linux_amd64/github.com/viant/afs/libmatcher.a",
"github.com/viant/afs/option": "/home/oceanfish81/go/pkg/gccgo_linux_amd64/github.com/viant/afs/liboption.a",
"github.com/viant/afs/scp": "/home/oceanfish81/go/pkg/gccgo_linux_amd64/github.com/viant/afs/libscp.a"
},
"Standard": {
"bytes": true,
"context": true,
"fmt": true,
"io": true,
"io/ioutil": true,
"log": true,
"path": true,
"strings": true
},
"PackageVetx": {
"github.com/viant/afs": "/home/oceanfish81/.cache/go-build/e3/e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855-d",
"github.com/viant/afs/asset": "/home/oceanfish81/.cache/go-build/e3/e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855-d",
"github.com/viant/afs/file": "/home/oceanfish81/.cache/go-build/e3/e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855-d",
"github.com/viant/afs/matcher": "/home/oceanfish81/.cache/go-build/e3/e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855-d",
"github.com/viant/afs/option": "/home/oceanfish81/.cache/go-build/e3/e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855-d",
"github.com/viant/afs/scp": "/home/oceanfish81/.cache/go-build/e3/e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855-d"
},
"VetxOnly": false,
"VetxOutput": "$WORK/b098/vet.out",
"SucceedOnTypecheckFailure": false
}
EOF
TERM='dumb' GCCGO='/usr/local/bin/llvm-goc' /usr/local/tools/vet -atomic -bool -buildtags -errorsas -ifaceassert -nilfunc -printf -stringintconv $WORK/b098/vet.cfg
$WORK/b001/afs.test -test.timeout=10m0s -test.v=true
=== RUN TestService_Copy
copy_test.go:121: failed to lookup key location: [/home/oceanfish81/.secret/id_rsa /home/oceanfish81/.ssh/id_rsa]
--- SKIP: TestService_Copy (0.12s)
=== RUN TestNewFaker
--- FAIL: TestNewFaker (0.00s)
panic: runtime error: invalid memory address or nil pointer dereference [recovered]
panic: runtime error: invalid memory address or nil pointer dereference

goroutine 18 [running]:
testing.tRunner..func2
/home/oceanfish81/workarea/llvm-project/llvm/tools/gollvm/gofrontend/libgo/go/testing/testing.go:1112
testing.tRunner..func1
/home/oceanfish81/workarea/llvm-project/llvm/tools/gollvm/gofrontend/libgo/go/testing/testing.go:1072
panic
/home/oceanfish81/workarea/llvm-project/llvm/tools/gollvm/gofrontend/libgo/go/runtime/panic.go:712
sync.RWMutex.Lock
/home/oceanfish81/workarea/llvm-project/llvm/tools/gollvm/gofrontend/libgo/go/sync/rwmutex.go:98
github.x2ecom..z2fviant..z2fafs..z2fmem.Folder.putFile
/home/oceanfish81/go/src/github.com/viant/afs/mem/folder.go:60
github.x2ecom..z2fviant..z2fafs..z2fmem.Folder.Put
/home/oceanfish81/go/src/github.com/viant/afs/mem/folder.go:178
github.x2ecom..z2fviant..z2fafs..z2fmem.storager.Upload
/home/oceanfish81/go/src/github.com/viant/afs/mem/upload.go:61
github.x2ecom..z2fviant..z2fafs..z2fbase.Manager.Upload
/home/oceanfish81/go/src/github.com/viant/afs/base/manager.go:102
github.x2ecom..z2fviant..z2fafs..z2fmem.manager.Upload
/home/oceanfish81/go/src/github.com/viant/afs/mem/manager.go:44
_..z2fhome..z2foceanfish81..z2fother_golang..z2fafs.service.Upload
/home/oceanfish81/other_golang/afs/service.go:66
_..z2fhome..z2foceanfish81..z2fother_golang..z2fafs.TestNewFaker
/home/oceanfish81/other_golang/afs/faker_test.go:69
testing.tRunner
/home/oceanfish81/workarea/llvm-project/llvm/tools/gollvm/gofrontend/libgo/go/testing/testing.go:1163
created by testing.T.Run
/home/oceanfish81/workarea/llvm-project/llvm/tools/gollvm/gofrontend/libgo/go/testing/testing.go:1214 +0x41c
exit status 2
FAIL _/home/oceanfish81/other_golang/afs 1.064s
rm -r $WORK/b001/

CC @thanm , @cherrymui

Looks like Go files in various sub-folders could be built - but running any tests raises a panic.

My environment, for gollvm MinSizeRel build

$ go env && go version
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/oceanfish81/.cache/go-build"
GOENV="/home/oceanfish81/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/oceanfish81/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/oceanfish81/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/tools"
GCCGO="/usr/local/bin/llvm-goc"
AR="ar"
CC="/usr/bin/clang"
CXX="/usr/bin/clang++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build286061473=/tmp/go-build -gno-record-gcc-switches -funwind-tables"
go version go1.15.2 gollvm LLVM 12.0.0git linux/amd64

@adranwit , you can read here

Currently I do not see a way to pass a ssh.ClientConfig to the scp.Service. The readme on https://github.com/viant/afs/tree/master/scp shows an example for the scp.Storage to do so.

Reason for this is the default MACs set by Go for SSH are not compatible with the one's we currently expose on servers.

If this is not something currently possible, I can send a PR that adds the functionality, but I am unsure if I am maybe missing something obvious.

1. Is this still being actively maintained?
2. Are there any plans to add support for Azure?

Currently, a lot of data source reads are cached; this may be useful for certain situations, especially if within the API there is support for copying the same data source multiple times.

However, this cache is too aggressive by default - it will not clear the cache until the process restart. There is not much precedent in infrastructural tools exposing a file-system level API that will inherently cache so aggressively. There may need to be a general system that requires checking if a cache is dirty, or expiring the cache within a reasonable scope.

Hello, thanks for great lib.

If possible, can you briefly specify, what is afs different from https://github.com/spf13/afero in it's motivation and implementation.

And can we pass opened file to other tools which expects native file interface?

Thanks.

I am getting this build error:

../../vendor/github.com/viant/afs/embed/manager.go:5:2: cannot find package "embed" in any of:
        /home/Murph/go/src/github.com/bluearchive/main/vendor/embed (vendor tree)
        /usr/local/go/src/embed (from $GOROOT)
        /home/Murph/go/src/embed (from $GOPATH)

I am not trying to use the embed package, but it appears that what I am trying to use requires it. Here are my imports:

        "github.com/viant/afs"
        "github.com/viant/afs/scp"
        "github.com/viant/afs/option"

And here is all the code I have written to sfar:

auth := scp.NewAuthProvider (nil, option.NewBasicAuth ("user", "password"))
service := afs.New()

I ran dep ensure, which put the packages into my vendor/github.com/viant/afs directory. It also put the embed package there.

The start of afs/embed/manager.go looks like this:

package embed

import (
        "context"
        "embed"

This package appears to be trying to import itself, which I have never seen.

My Go version is 1.12.17.

Any ideas why this isn't working?

Thanks.

GCS seems to support both files AND directories (prefixes) with the same string.
For example, gs://test_bucket/test_file.txt and gs://test_bucket/test_file.txt/sub_file.txt are both valid GCS objects.

In the case this occurs, it looks like the ZIP detection isn't handling this correctly.
Specifically, github.com/viant/afsc/gs.(s *storager).List will return multiple files in the case that both a file and a prefix exist at the same path, and github.com/viant/afs/zip.newStorager() checks to make sure the List only returns one file https://github.com/viant/afs/blob/master/zip/storager.go#L241.

The currently coded behavior seems to work on the assumption that a file and a directory cannot be the same name (which is valid).

This may also coincide with the way that afsc/gs is handling the List method, as it seems to assume that the URL provided can be a prefix.

i use this code to zip and unzip folder

service := afs.New()
ctx := context.Background()
uploader := zip.NewBatchUploader(file.New())
err := service.Copy(ctx, "deployments", "zipped.zip", uploader)
if err != nil {
log.Fatal(err)
}

walker := zip.NewWalker(file.New())
err = service.Copy(ctx, "zipped.zip", "zipped", walker)
if err != nil {
log.Fatal(err)
}

zip - works properly , but uzip doesn`t

i zipped folder with one file "deployments\docker-compose\docker-compose.yml"
As result after unzip , i get "zipped\docker-compose\zipped.zip" . An intresting thing is that "zipped\docker-compose\zipped.zip" is a txt file with rigth conent . But name of folder is wrong

When running an operation using the option.NoCache with Source == option.NoCacheBaseURL, the Manager for the extension URL component does not get loaded. This seems unexpected, and prevents updated data sources from updating correctly.

6c357a8 addresses this.

Hello,

I was wondering how to specify the username of the remote user when using scp with LocalhostKeyAuth. It appears to try and connect as the user that is running the program (USER from env). I tried modifying the url to scp://user@host:22/path/to/file but that does not work.

Thanks!

-Will