git-lfs / lfs-test-server Goto Github PK

View Code? Open in Web Editor NEW

675.0 675.0 159.0 483 KB

Standalone Git LFS server

License: MIT License

Go 92.20% Shell 7.56% Dockerfile 0.25%

git-lfs go lfs-server

lfs-test-server's Introduction

Git Large File Storage

Git LFS is a command line extension and specification for managing large files with Git.

The client is written in Go, with pre-compiled binaries available for Mac, Windows, Linux, and FreeBSD. Check out the website for an overview of features.

Getting Started

Installing

On Linux

Debian and RPM packages are available from packagecloud, see the Linux installation instructions.

On macOS

Homebrew bottles are distributed and can be installed via brew install git-lfs.

On Windows

Git LFS is included in the distribution of Git for Windows. Alternatively, you can install a recent version of Git LFS from the Chocolatey package manager.

From binary

Binary packages are available for Linux, macOS, Windows, and FreeBSD. The binary packages include a script which will:

Install Git LFS binaries onto the system $PATH. On Windows in particular, you may need to restart your command shell so any change to $PATH will take effect and Git can locate the Git LFS binary.
Run git lfs install to perform required global configuration changes.

$ ./install.sh

Note that Debian and RPM packages are built for multiple Linux distributions and versions for both amd64 and i386. For arm64, only Debian packages are built and only for recent versions due to the cost of building in emulation.

From source

Ensure you have the latest version of Go, GNU make, and a standard Unix-compatible build environment installed.
On Windows, install goversioninfo with go install github.com/josephspurrier/goversioninfo/cmd/goversioninfo@latest.
Run make.
Place the git-lfs binary, which can be found in bin, on your system’s executable $PATH or equivalent.
Git LFS requires global configuration changes once per-machine. This can be done by running: git lfs install

Verifying releases

Releases are signed with the OpenPGP key of one of the core team members. To get these keys, you can run the following command, which will print them to standard output:

$ curl -L https://api.github.com/repos/git-lfs/git-lfs/tarball/core-gpg-keys | tar -Ozxf -

Once you have the keys, you can download the sha256sums.asc file and verify the file you want like so:

$ gpg -d sha256sums.asc | grep git-lfs-linux-amd64-v2.10.0.tar.gz | shasum -a 256 -c

For the convenience of distributors, we also provide a wider variety of signed hashes in the hashes.asc file. Those hashes are in the tagged BSD format, but can be verified with Perl's shasum or the GNU hash utilities, just like the ones in sha256sums.asc.

Example Usage

To begin using Git LFS within a Git repository that is not already configured for Git LFS, you can indicate which files you would like Git LFS to manage. This can be done by running the following from within a Git repository:

$ git lfs track "*.psd"

(Where *.psd is the pattern of filenames that you wish to track. You can read more about this pattern syntax here).

Note: the quotation marks surrounding the pattern are important to prevent the glob pattern from being expanded by the shell.

After any invocation of git-lfs-track(1) or git-lfs-untrack(1), you must commit changes to your .gitattributes file. This can be done by running:

$ git add .gitattributes
$ git commit -m "track *.psd files using Git LFS"

You can now interact with your Git repository as usual, and Git LFS will take care of managing your large files. For example, changing a file named my.psd (tracked above via *.psd):

$ git add my.psd
$ git commit -m "add psd"

Tip: if you have large files already in your repository's history, git lfs track will not track them retroactively. To migrate existing large files in your history to use Git LFS, use git lfs migrate. For example:
$ git lfs migrate import --include="*.psd" --everything
Note that this will rewrite history and change all of the Git object IDs in your repository, just like the export version of this command.

For more information, read git-lfs-migrate(1).

You can confirm that Git LFS is managing your PSD file:

$ git lfs ls-files
3c2f7aedfb * my.psd

Once you've made your commits, push your files to the Git remote:

$ git push origin main
Uploading LFS objects: 100% (1/1), 810 B, 1.2 KB/s
# ...
To https://github.com/git-lfs/git-lfs-test
   67fcf6a..47b2002  main -> main

Note: Git LFS requires at least Git 1.8.2 on Linux or 1.8.5 on macOS.

Uninstalling

If you've decided that Git LFS isn't right for you, you can convert your repository back to a plain Git repository with git lfs migrate as well. For example:

$ git lfs migrate export --include="*.psd" --everything

Note that this will rewrite history and change all of the Git object IDs in your repository, just like the import version of this command.

If there's some reason that things aren't working out for you, please let us know in an issue, and we'll definitely try to help or get it fixed.

Limitations

Git LFS maintains a list of currently known limitations, which you can find and edit here.

Git LFS source code utilizes Go modules in its build system, and therefore this project contains a go.mod file with a defined Go module path. However, we do not maintain a stable Go language API or ABI, as Git LFS is intended to be used solely as a compiled binary utility. Please do not import the git-lfs module into other Go code and do not rely on it as a source code dependency.

Need Help?

You can get help on specific commands directly:

$ git lfs help <subcommand>

The official documentation has command references and specifications for the tool. There's also a FAQ shipped with Git LFS which answers some common questions.

If you have a question on how to use Git LFS, aren't sure about something, or are looking for input from others on tips about best practices or use cases, feel free to start a discussion.

You can always open an issue, and one of the Core Team members will respond to you. Please be sure to include:

The output of git lfs env, which displays helpful information about your Git repository useful in debugging.
Any failed commands re-run with GIT_TRACE=1 in the environment, which displays additional information pertaining to why a command crashed.

Contributing

See CONTRIBUTING.md for info on working on Git LFS and sending patches. Related projects are listed on the Implementations wiki page.

See also SECURITY.md for info on how to submit reports of security vulnerabilities.

Core Team

These are the humans that form the Git LFS core team, which runs the project.

In alphabetical order:

@bk2204	@chrisd8088	@larsxschneider

PGP 0223B187	PGP 088335A9	PGP A5795889

Alumni

These are the humans that have in the past formed the Git LFS core team, or have otherwise contributed a significant amount to the project. Git LFS would not be possible without them.

In alphabetical order:

@andyneff	@PastelMobileSuit	@rubyist	@sinbad	@technoweenie	@ttaylorr

lfs-test-server's People

Contributors

Stargazers

Watchers

Forkers

folpindo godeep rgl peterdavehello gregsohl mocyuto pombredanne jmj chrhlnd krusmir nanomobile jsh bozaro aorjoa dipucian itsmikeq dirk ziggr yopai jezell shevchukpavel yoshimsa-yokoyama isc-labs richy882x signaljosh 1234- leenamba systemmonkey42 rbarraud suzuki-t-ce a780201 guojin2020 ivotron blackberrypassport ttaylorr rbramwell sinbad johndirek mgit-at feathers88 sasam6a lishengbing eddienko 2323monica holysynnr husinbetawi ikmski p12tic nengherr thejare satana-jhoni timxor fiona0407 kaben 13075723795 lukefrasera mohseenrm bayashka2017 bayashka1 kleopatra999 coftysweet angel1144 angel444 efateev57 neeke usernamealreadyis wg-lesta will-jj dotmpe hongnod linecode lift766 lsjostro gabrielsantossilva gabrielsantossilv mgmalind mixmarketing peterkinalex gergamel marypaz udoheyn erwinndog devdevmac smartmanru pcpartpicker trantung3103 dalavancloud ikuo nasrul5641 shyang seif-abaza bk2204 wingincheung saeedqq111 edartes biz-ad jpinkul navs rockylmc vinikeda

lfs-test-server's Issues

Watch out for read values of Bolt outside a transaction

At https://github.com/github/lfs-test-server/blob/master/meta_store.go#L68 (MetaStore.Get) you are using a value outside a transaction. This should not be done, see the bolt docs at:

Please note that values returned from Get() are only valid while the transaction is open. If you need to use a value outside of the transaction then you must use copy() to copy it to another byte slice.

Have to enter password twice during git clone

Have to enter password twice during git clone.

$ git clone ../repo.git
Cloning into 'repo'...
done.
Downloading sound.mp3 (4.80 MB)
Username for 'http://127.0.0.1:9999': fred
Password for 'http://[email protected]:9999': 
Username for 'http://127.0.0.1:9999': fred
Password for 'http://[email protected]:9999':

File lock failed: Repository or object not found

Hi. I get a problem with file locking.
Here is my server config:

LFS_LISTEN="tcp://:9999"
LFS_HOST="192.168.2.36:9999"
LFS_CONTENTPATH="/usr/local/lib/git-lfs-server/content"
LFS_METADB="/usr/local/lib/git-lfs-server/lfs.db"
LFS_ADMINUSER="cc"
LFS_ADMINPASS="cc"

I can track/push/pull files normally with git lfs commands, but when I want to lock a file with:
$ git lfs lock image.jpg
It fails and here is the message:

Lock failed: Repository or object not found: http://192.168.2.36:9999/locks
Check that it exists and that you have proper access to it

Can somebody help me with this? Thanks.

Turn harbour into a reference lfs server

We can turn harbour into a reference lfs server of sorts. It doesn't have to be able to handle GitHub levels of traffic, just implement the API in a simple way with a simple auth/storage backend.

Provide some kind of authorization hooks
Use a simple file store similar to alambic in enterprise
Probably remove the S3 URL generation (unless we want to provide that option)
Prepare for OSSing
Name it

/cc @github/lfs
/cc @bkeepers for thoughts/processes on preparing for OSS

Can't build riceboxes the same way

I was messing with the management page templates and tried rebuilding them (rice embed-go)...

Everytime I do so, I end up with a single "rice-box.go" file, compared to the two files you have. What am I doing wrong? Is it simply an different version of rice?

Kept getting 401 error on Windows without environment variables

Not sure if this should be considered an issue or note for the developers/contributors to update the docs

I kept trying to connect to the LFS test server that was running when I had NO environment variables set and I kept getting a 401 error no matter what I did.

PS H> .\LFS-Server-Latest.exe 
2019-10-21T18:06:44Z localhost lfs[21736] [proc.go:203]: version=0.4.0 fn=main msg=listening 
pid=21736 addr=tcp://:8080
2019-10-21T18:07:41Z localhost lfs[21736] [server.go:659]: url=/mgmt status=404 
request_id=c396f405-09e1-a7a8-3080-2ca6b8811b12 method=GET
2019-10-21T18:08:17Z localhost lfs[21736] [server.go:659]: method=POST url=/objects/batch 
status=401 request_id=53c1c8a3-8cb8-4638-e8cb-71a2b5be96ca
2019-10-21T18:08:18Z localhost lfs[21736] [server.go:659]: method=POST url=/objects/batch 
status=401 request_id=b6ab17c1-f61b-2b7e-dd44-ad97b25ff00f
2019-10-21T18:08:20Z localhost lfs[21736] [server.go:659]: method=POST url=/objects/batch 
status=401 request_id=63402406-fc3d-655d-23d6-662d445e29a9
OST url=/objects/batch status=401                                                                              `

After setting the following in PowerShell and running it finally was successful and showed a management interface as expected. So without those variables even the default didn't seem to work properly

set-item env:\LFS_HOST -Value "localhost:8080"
set-item env:\LFS_ADMINUSER -Value "unset"
set-item env:\LFS_ADMINPASS -Value "unset"
set-item env:\LFS_SCHEME -Value "http"
.\LFS-Server-Latest.exe

git push hangs

Hi all! Have this issue, we work with Atlassian. Do I do something wrong?
Here is the trace

GIT_TRACE=1 GIT_CURL_VERBOSE=1 git push

18:45:22.972872 git.c:344               trace: built-in: git 'push'
18:45:22.973464 run-command.c:334       trace: run_command: 'ssh' '[email protected]' 'git-receive-pack '\''groupp/repoo.git'\'''
18:45:24.445509 run-command.c:334       trace: run_command: '.git/hooks/pre-push' 'origin' '[email protected]:groupp/repoo.git'
18:45:24.449638 git.c:561               trace: exec: 'git-lfs' 'pre-push' 'origin' '[email protected]:groupp/repoo.git'
18:45:24.449724 run-command.c:334       trace: run_command: 'git-lfs' 'pre-push' 'origin' '[email protected]:groupp/repoo.git'
18:45:24.459124 trace git-lfs: run_command: 'git' version
18:45:24.462023 trace git-lfs: run_command: 'git' config -l
18:45:24.465080 trace git-lfs: tq: running as batched queue, batch size of 100
18:45:26.073199 trace git-lfs: pre-push: refs/heads/master d3075da831a34d08da1f9883b95521d2fa28dd5f refs/heads/master 1aefc2592af566cbe0bd848e61ae70af68055b2c
18:45:26.073503 trace git-lfs: run_command: git rev-list --objects d3075da831a34d08da1f9883b95521d2fa28dd5f --not --remotes=origin --
18:45:26.076189 trace git-lfs: run_command: git cat-file --batch
18:45:26.081487 trace git-lfs: tq: sending batch of size 1
18:45:26.081777 trace git-lfs: api: batch 1 files
18:45:26.081831 trace git-lfs: creds: git credential fill ("http", "server-o:9999", "")
Username for 'http://server-o:9999': user
Password for 'http://user@server-o:9999': 
18:45:30.111265 trace git-lfs: Filled credentials for http://server-o:9999/
18:45:30.111530 trace git-lfs: HTTP: POST http://server-o:9999/objects/batch
> POST /objects/batch HTTP/1.1
> Host: server-o:9999
> Accept: application/vnd.git-lfs+json; charset=utf-8
> Authorization: Basic * * * * *
> Content-Length: 123
> Content-Type: application/vnd.git-lfs+json; charset=utf-8
> User-Agent: git-lfs/2.3.2 (GitHub; linux amd64; go 1.8.1)
> 
{"operation":"upload","objects":[{"oid":"11f6bf651ae98a35d051d6ba7d0af90a239ce33891c787ff46b34da1906c37ec","size":307374}]}18:45:30.115886 trace git-lfs: HTTP: 200


< HTTP/1.1 200 OK
< Content-Length: 513
< Content-Type: application/vnd.git-lfs+json
< Date: Wed, 04 Oct 2017 16:44:57 GMT
< 
18:45:30.119045 trace git-lfs: HTTP: {"objects":[{"oid":"11f6bf651ae98a35d051d6ba7d0af90a239ce33891c787ff46b34da1906c37ec","size":307374,"actions":{"download":{"href":"http://server-o:9999/objects/11f6bf651ae98a35d051d6ba7d0af90a239ce33891c787ff46b34da1906c37ec","header":{"Accept":"application/vnd.git-lfs"},"expires_at":"0001-01-01T00:00:00Z"},"upload":{"href":"http://server-o:9999/objects/11f6bf651ae98a35d051d6ba7d0af90a239ce33891c787ff46b34da1906c37ec","header":{"Accept":"application/vnd.git-lfs"},"expires_at":"0001-01-01T00:00:00Z"}}}]}
{"objects":[{"oid":"11f6bf651ae98a35d051d6ba7d0af90a239ce33891c787ff46b34da1906c37ec","size":307374,"actions":{"download":{"href":"http://server-o:9999/objects/11f6bf651ae98a35d051d6ba7d0af90a239ce33891c787ff46b34da1906c37ec","header":{"Accept":"application/vnd.git-lfs"},"expires_at":"0001-01-01T00:00:00Z"},"upload":{"href":"http://server-o:9999/objects/11f6bf651ae98a35d051d6ba7d0af90a239ce33891c787ff46b34da1906c37ec","header":{"Accept":"application/vnd.git-lfs"},"expires_at":"0001-01-01T00:00:00Z"}}}]}18:45:30.119211 trace git-lfs: tq: starting transfer adapter "basic"
Git LFS: (0 of 1 files) 0 B / 300.17 KB                                                                                                                                        18:45:30.119472 trace git-lfs: creds: git credential fill ("http", "server-o:9999", "")
Git LFS: (0 of 1 files) 0 B / 300.17 KB                                                                                                                                        ^C
Exiting because of "interrupt" signal.
18:45:46.913179 trace git-lfs: tq: retrying object 11f6bf651ae98a35d051d6ba7d0af90a239ce33891c787ff46b34da1906c37ec: LFS: Git credentials for http://server-o:9999/ not found:
'git credential fill' error: signal: interrupt

log del server

2017-10-04T16:44:57Z aea0cdd6c40e lfs[191] [server.go:341]: method=POST url=/objects/batch status=200 request_id=b2cc7035-47ec-268d-7c4b-b904b9aa50d9

But when I click on http://server-o:9999/mgmt/raw/11f6bf651ae98a35d051d6ba7d0af90a239ce33891c787ff46b34da1906c37ec it returns Not Found

Got error with default LFS_SCHEME(https)

Both browser and git lfs client got this error, and by the way, even I didn't set LFS_SCHEME to http,
I can visit the panel via http protocol, is this a bug?

Error for POST https://localhost:8080/temp/test/objects

Post https://localhost:8080/temp/test/objects: tls: oversized record received with length 20527
goroutine 16 [running]:
github.com/github/git-lfs/lfs.Stack(0x0, 0x0, 0x0)
        /Users/rick/github/git-lfs/.vendor/src/github.com/github/git-lfs/lfs/errors.go:85 +0x89
github.com/github/git-lfs/commands.logPanic(0x7f938c1eac30, 0xc2080aa840, 0x7f938c049a00, 0x0, 0x0)
        /Users/rick/github/git-lfs/.vendor/src/github.com/github/git-lfs/commands/commands.go:148 +0xf17
github.com/github/git-lfs/commands.handlePanic(0x7f938c1eac30, 0xc2080aa840, 0x0, 0x0)
        /Users/rick/github/git-lfs/.vendor/src/github.com/github/git-lfs/commands/commands.go:100 +0x6c
github.com/github/git-lfs/commands.LoggedError(0x7f938c1eac30, 0xc2080aa840, 0xc208034040, 0x37, 0x0, 0x0, 0x0)
        /Users/rick/github/git-lfs/.vendor/src/github.com/github/git-lfs/commands/commands.go:65 +0x89
github.com/github/git-lfs/commands.Panic(0x7f938c1eac30, 0xc2080aa840, 0xc208034040, 0x37, 0x0, 0x0, 0x0)
        /Users/rick/github/git-lfs/.vendor/src/github.com/github/git-lfs/commands/commands.go:75 +0x68
github.com/github/git-lfs/commands.pushCommand(0x983f80, 0xc208022450, 0x2, 0x3)
        /Users/rick/github/git-lfs/.vendor/src/github.com/github/git-lfs/commands/command_push.go:120 +0x572
github.com/spf13/cobra.(*Command).execute(0x983f80, 0xc208034080, 0x3, 0x4, 0x0, 0x0)
        /Users/rick/github/git-lfs/.vendor/src/github.com/spf13/cobra/command.go:355 +0x17c
github.com/spf13/cobra.(*Command).findAndExecute(0x984680, 0xc20800e010, 0x4, 0x4, 0x0, 0x0)
        /Users/rick/github/git-lfs/.vendor/src/github.com/spf13/cobra/command.go:334 +0xd3
github.com/spf13/cobra.(*Command).Execute(0x984680, 0x0, 0x0)
        /Users/rick/github/git-lfs/.vendor/src/github.com/spf13/cobra/command.go:408 +0x7b0
github.com/github/git-lfs/commands.Run()
        /Users/rick/github/git-lfs/.vendor/src/github.com/github/git-lfs/commands/commands.go:80 +0x2a
main.main()
        /Users/rick/github/git-lfs/cmd/git-lfs.go:8 +0x1a

Large file uploads causing 500 error?

So in testing the server for using it as a folder tracking for large video files (yes I know this is not production but I need a personal folder tracker), I am so far doing well but some VERY LARGE video files are getting 500 errors as the sample below shows. Now the other situation is that I accidentally did a LARGE commit of lots of files that total 47 GB of data (141 in one commit), so there is that scenario, but only these few files show as not being successful. I am not sure if this is a server related bug or if it is related to the Git LFS client issue with big files as mentioned previously.

I am trying to do some more checks but wonder if this is a bug that needed to be investigated or is it a noob problem not knowing what I am doing.

< HTTP/1.1 500 Internal Server Error
< Connection: close
< Content-Length: 124
< Content-Type: text/plain; charset=utf-8
< Date: Thu, 24 Oct 2019 19:52:52 GMT
<
12:52:52.287691 trace git-lfs: xfer: adapter "basic" worker 7 finished job for "ea34c8f9d1619e90955e5fa89f2e2c661ba7ca5d
ec1368a0cc93e7bdeb8970c8"
12:52:52.287691 trace git-lfs: tq: retrying object ea34c8f9d1619e90955e5fa89f2e2c661ba7ca5dec1368a0cc93e7bdeb8970c8: Fat
al error: Server error: http://localhost:8080/objects/ea34c8f9d1619e90955e5fa89f2e2c661ba7ca5dec1368a0cc93e7bdeb89
70c8
12:52:52.288692 trace git-lfs: tq: enqueue retry #2 for "ea34c8f9d1619e90955e5fa89f2e2c661ba7ca5dec1368a0cc93e7bdeb8970c
8" (size: 3294909799)
12:52:52.317693 trace git-lfs: HTTP: 500

I tried to read all I can but not much was published about this error or scenario.

can remove boltdb to mysql ?

Unable to download file with its only url

Hi, thanks for the LFS-mechanism, that's great !

If I try to download an object directly with its url - the one I read in the , I got a 404.
http://localhost:8080/_user_/repo/objects/_sha_

It should rather return a 403, and allow downloading the file after authentication - and in password useless case (Issue #29), it should directly deliver the file.

And the management console, in the "Objects" tab, should link each sha to download the file.

Let me know your remarks about that, I can (try to) implement it in a PR.

Passwordless use case?

I'd like to use this test server to setup a public repository with some lfs-server backed files that, when cloned, does not prompt the user for a user name or password to pull the lfs files (just like any old public git repository on github). Is that possible?

It would be nice if there were an environment variable that could disable authentication or something like that.

LFS:Put connexion refused

Hello,

I like help on using lfs-test-server.

When I try to push lfs files, I got connection refused. I see inside lfs-test-server request

I have define a new user (user:matthieu password:matthieu) But I dont know how to set this user to LFS client configuration.

Can anyone help me ?

Regards

Matthieu

result from lfs-test-server 👍

[server.go:240]: request_id=8324f5a1-5165-8e5c-ec1b-c18602b87e44 method=POST url=/objects/batch status=200
2018-03-07T10:50:41Z LFRCRA02426 lfs[15276] [server.go:240]: method=POST url=/objects/batch status=200 request_id=6b6f8cc8-0562-6764-16c3-54e220dd3e23
2018-03-07T10:50:44Z LFRCRA02426 lfs[15276] [server.go:240]: method=POST url=/objects/batch status=200 request_id=2ec2b5e3-1647-7ee4-5706-fdeca0199157
2018-03-07T10:50:47Z LFRCRA02426 lfs[15276] [server.go:240]: method=POST url=/objects/batch status=200 request_id=f000cd28-56eb-9038-2b80-c59a42ef1255
2018-03-07T10:50:50Z LFRCRA02426 lfs[15276] [server.go:240]: method=POST url=/objects/batch status=200 request_id=57f36a43-f031-d552-1939-7b4209f6d0f1
2018-03-07T10:50:53Z LFRCRA02426 lfs[15276] [server.go:240]: method=POST url=/objects/batch status=200 request_id=0154e339-3cbf-7c24-32f3-946e2291a231
2018-03-07T10:50:56Z LFRCRA02426 lfs[15276] [server.go:240]: url=/objects/batch status=200 request_id=6a67820e-a87a-8db8-f2d3-505ccea8bb04 method=POST

LFS: Put http://localhost/objects/124ed7cb81447c8689ad5c5956ef913155612c285886dcdcad75db081ac6b30e: dial tcp [::1]:80: connectex: Aucune connexion n’a pu être établie car l’ordinateur cible l’a expressément refusée.

config file :

[core]
repositoryformatversion = 0
filemode = false
bare = false
logallrefupdates = true
ignorecase = true
[lfs]
url = "http://localhost:8080/"
[remote "remoteT5"]
url = z:/Git/t5/
fetch = +refs/heads/:refs/remotes/remoteT5/
[lfs "http://localhost:8080/"]
locksverify = false
access = basic

Use kv style logging

Add documentation

Bring code documentation up to standards.

> Doisjs

Doisjs

Originally posted by @phetdavan1 in git-lfs/git-lfs#4267 (comment)

Add request id

Add a request id that can be traced through the system.

How to use test-server 0.3.0 with lfs 1.0.2 ?

I am trying to use lfs-test-server 0.3 with lfs 1.0.2.

My repository is hosted on a locally installed Gerrit server listening at http://localhost:8084, my lfs-test-server listens on http://localhost:9999/.

I cloned the repository and ran
$ git lfs init
$ git track "*.bin"

here my .git/config

[core]
repositoryformatversion = 0
filemode = true
bare = false
logallrefupdates = true
ignorecase = true
precomposeunicode = true
[remote "origin"]
url = http://localhost:8084/lfs
fetch = +refs/heads/:refs/remotes/origin/
[branch "master"]
remote = origin
merge = refs/heads/master
rebase = true
[lfs "http://localhost:9999/"]
[lfs]
batch = false

after adding and committing a file "1.bin" when trying to push this to Gerrit / local lfs-test-server I hit the following error:

$ git push origin HEAD:refs/heads/master
(0 of 1 files) 0 B / 6 B
Repository or object not found: http://localhost:8084/lfs.git/info/lfs/objects
Check that it exists and that you have proper access to it
error: failed to push some refs to 'http://localhost:8084/lfs'

what's wrong here ?

lfs-test-server tricks client into violation of LFS spec

https://github.com/git-lfs/lfs-test-server/blob/master/server.go#L554

Server tells client to use Accept: application/vnd.git-lfs header when accessing various links, including /verify. However, LFS specification clearly states that client sends Accept: application/vnd.git-lfs+json when accessing verification link.

Current behavior turns our to work with native git-lfs implementation, but not with other clients (git-lfs-java, for example) and overall is very fragile and depends on the order in which client builds its HTTP headers list.

invalid character "\\" in host name

I'm having some trouble getting blobs to push into lfs-test-server. I've tried to construct a standalone setup on two different machines: a home laptop using Windows 8 and a (airgapped) work laptop using Windows 7. Results were the same for both.
As instructed in the lfs-test-server installation instructions, I in lfs-test-server setup I changed the environmental variables LFS_AdminUser (test) and LFS_AdminPass (testpass) in order to open \mgmt and add a user (testuser) and password (testpass). The only way I could figure out how to do this was using advanced properties in windows control panel and adding through the advanced system properties in the windows control panel. I can open mgmt template when I type http://localhost:8080/mgmt
All other environmental variables were left at default.

Here is my setup

Local Repository is on following directory c:\users\dad\desktop\GITOUT\GITREPO

a) .git\ config file content is as follows:
[core]
repositoryformatversion = 0
filemode = false
bare = false
logallrefupdates = true
symlinks = false
ignorecase = true

b) .lfsconfig file (staged and committed) as follows:
[lfs]
url = "http://localhost:8080"

Note: I also tried url =http://localhost:8080/ and results were the same (explained at end of message).

c) .gitattributes file (staged and committed) as follows:
*.pdf filter=lfs diff=lfs merge=lfs -text
d) LFS Environment on GITREPO is:
Endpoint=http://localhost:8080 (auth=none)
LocalWorkingDir=C:\Users\Dad\Desktop\GITOUT\GITREPO
LocalGitDir=C:\Users\Dad\Desktop\GITOUT\GITREPO.git
LocalGitStorageDir=C:\Users\Dad\Desktop\GITOUT\GITREPO.git\lfs\objects
LocalReferenceDirs=
TempDir=C:\Users\Dad\Desktop\GITOUT\GITREPO.git\lfs\tmp
ConcurrentTransfers=3
TusTransfers=false
BasicTransfersOnly=false
SkipDown loadErrors=false
FetchRecentAlways=false
FetchRecentRefsDays=7
FetchRecentCommitsDays=0
FetchRecentRefsIncludeRemotes=true
PruneOffsetDays=3
PruneVerifyRemoteAlways=false
PruneRemoteName=origin
LfsStorageDir=C:\Users\Dad\Desktop\GITOUT\GITREPO.git\lfs
AccessDownload=none
AccessUpload=none
DownloadTransfers=basic
UploadTransfers=basic
GIT_EXEC_PATH=C:/Program Files/Git/mingw64/libexec/git-core
git config filter.lfs.process = "git-lfs filter-process"
git config filter.lfs.smudge = "git-lfs smudge -- %f"
git config filter.lfs.clean = "git-lfs clean -- %f"

I cloned GITREPO to following directory c:\users\dad\desktop\GITOUT\GITCLONE

a) .git\ config file content is as follows:
[core]
bare = false
repositoryformatversion = 0
filemode = false
symlinks = false
ignorecase = true
logallrefupdates = true
[remote "origin"]
url = C:\Users\Dad\Desktop\GITOUT\GITREPO
fetch = +refs/heads/:refs/remotes/origin/
[branch "master"]
remote = origin
merge = refs/heads/master
[lfs "http://localhost:8080"]
locksverify = false
access = basic
b) .lfsconfig (unchanged) as follows:
[lfs]
url = "http://localhost:8080"
c) .gitattributes file (unchanged) as follows:
*.pdf filter=lfs diff=lfs merge=lfs -text
e) LFS Environment on GITCLONE is:
Endpoint=http://localhost:8080 (auth=none)
LocalWorkingDir=C:\Users\Dad\Desktop\GITOUT\GITCLONE
LocalGitDir=C:\Users\Dad\Desktop\GITOUT\GITCLONE.git
LocalGitStorageDir=C:\Users\Dad\Desktop\GITOUT\GITCLONE.git\lfs\objects
LocalReferenceDirs=
TempDir=C:\Users\Dad\Desktop\GITOUT\GITCLONE.git\lfs\tmp
ConcurrentTransfers=3
TusTransfers=false
BasicTransfersOnly=false
SkipDown loadErrors=false
FetchRecentAlways=false
FetchRecentRefsDays=7
FetchRecentCommitsDays=0
FetchRecentRefsIncludeRemotes=true
PruneOffsetDays=3
PruneVerifyRemoteAlways=false
PruneRemoteName=origin
LfsStorageDir=C:\Users\Dad\Desktop\GITOUT\GITCLONE.git\lfs
AccessDownload=none
AccessUpload=none
DownloadTransfers=basic
UploadTransfers=basic
GIT_EXEC_PATH=C:/Program Files/Git/mingw64/libexec/git-core
git config filter.lfs.process = "git-lfs filter-process"
git config filter.lfs.smudge = "git-lfs smudge -- %f"
git config filter.lfs.clean = "git-lfs clean -- %f"

I am able to push files that I don’t track from GITCLONE to GITREPO no problem. However, when I attempt to push the .pdf file from the clone to the repository I’m expecting the repository to have the SHA-1 pointer, and localhost:8080 to show the .pdf object in the \objects screen.

However, I am not asked for my credentials and I instead get the following message:

Uploading LFS objects: 0% (0/1), 0 B | 0 B/s, done
batch response: creds: parse //c:\users\administrator\desktop\GITOUT\GITREPO: invalid character “\” in host name
error: failed to push some refs to ‘c:\users\administrator\desktop\GITOUT\GITREPO’.

GITCLONE doesn’t push to GITREPO. I don’t even get the SHA-1 pointer in the GITREPO. I'm pretty sure that my lfs-test-server setup is incorrect. Is my problem that I set environmental variables for LFS_AdminUser and LFS_AdminPass within the advanced settings of windows? If so how do I set these without using windows? Should I change to https:?

I’m totally stumped.

following steps in README results in certificate errors

I tried following the README and creating a certificate to use for https access. Following the steps in the README results in an error.

LocalWorkingDir=/Users/louisb/tmp/test-gitlfs
LocalGitDir=/Users/louisb/tmp/test-gitlfs/.git
LocalMediaDir=/Users/louisb/tmp/test-gitlfs/.git/lfs/objects
TempDir=/Users/louisb/tmp/test-gitlfs/.git/lfs/tmp
GIT_PREFIX=

Error for POST https://localhost:9999/user/repo/objects

Post https://localhost:9999/user/repo/objects: x509: certificate signed by unknown authority
goroutine 16 [running]:
github.com/github/git-lfs/lfs.Stack(0x0, 0x0, 0x0)
    /Users/rick/github/git-lfs/.vendor/src/github.com/github/git-lfs/lfs/errors.go:85 +0x89
github.com/github/git-lfs/commands.logPanic(0x6674d0, 0xc2082dc360, 0x69da00, 0x0, 0x0)
    /Users/rick/github/git-lfs/.vendor/src/github.com/github/git-lfs/commands/commands.go:156 +0xf17
github.com/github/git-lfs/commands.handlePanic(0x6674d0, 0xc2082dc360, 0x0, 0x0)
    /Users/rick/github/git-lfs/.vendor/src/github.com/github/git-lfs/commands/commands.go:108 +0x6c
github.com/github/git-lfs/commands.LoggedError(0x6674d0, 0xc2082dc360, 0xc2082d4000, 0x37, 0x0, 0x0, 0x0)
    /Users/rick/github/git-lfs/.vendor/src/github.com/github/git-lfs/commands/commands.go:65 +0x89
github.com/github/git-lfs/commands.prePushCommand(0x594800, 0xc208044640, 0x2, 0x2)
    /Users/rick/github/git-lfs/.vendor/src/github.com/github/git-lfs/commands/commands_pre_push.go:97 +0x63f
github.com/spf13/cobra.(*Command).execute(0x594800, 0xc208044620, 0x2, 0x2, 0x0, 0x0)
    /Users/rick/github/git-lfs/.vendor/src/github.com/spf13/cobra/command.go:355 +0x17c
github.com/spf13/cobra.(*Command).findAndExecute(0x594700, 0xc20800e010, 0x3, 0x3, 0x0, 0x0)
    /Users/rick/github/git-lfs/.vendor/src/github.com/spf13/cobra/command.go:334 +0xd3
github.com/spf13/cobra.(*Command).Execute(0x594700, 0x0, 0x0)
    /Users/rick/github/git-lfs/.vendor/src/github.com/spf13/cobra/command.go:408 +0x7b0
github.com/github/git-lfs/commands.Run()
    /Users/rick/github/git-lfs/.vendor/src/github.com/github/git-lfs/commands/commands.go:80 +0x2a
main.main()
    /Users/rick/github/git-lfs/cmd/git-lfs/git-lfs.go:8 +0x1a

I assume that having

 [http]
        selfverify = false

in the .gitconfig is supposed to avoid this, but it doesn't seem to work. Switching from https to http resolved this error.

What is missing to make this server production ready?

The documentation says that the server is not production ready but doesn't explain why.
If people want to contribute to make it production ready, what do they need to look at?

Running git lfs ls-files on empty server causes index out of range error

Running git lfs ls-files before adding any files causes following error:

panic: runtime error: index out of range

goroutine 1 [running]:
github.com/github/git-lfs/git.ResolveRef(0x882d90, 0x4, 0x1, 0x0, 0x0)
    /Users/rick/go/src/github.com/github/git-lfs/git/git.go:68 +0x2f6
github.com/github/git-lfs/git.CurrentRef(0xacf3a0, 0x0, 0x0)
    /Users/rick/go/src/github.com/github/git-lfs/git/git.go:73 +0x38
github.com/github/git-lfs/commands.lsFilesCommand(0xaa55c0, 0xacf3a0, 0x0, 0x0)
    /Users/rick/go/src/github.com/github/git-lfs/commands/command_ls_files.go:26 +0x205
github.com/github/git-lfs/vendor/_nuts/github.com/spf13/cobra.(*Command).execute(0xaa55c0, 0xacf3a0, 0x0, 0x0, 0x0, 0x0)
    /Users/rick/go/src/github.com/github/git-lfs/vendor/_nuts/github.com/spf13/cobra/command.go:477 +0x403
github.com/github/git-lfs/vendor/_nuts/github.com/spf13/cobra.(*Command).Execute(0xaa6ae0, 0x0, 0x0)
    /Users/rick/go/src/github.com/github/git-lfs/vendor/_nuts/github.com/spf13/cobra/command.go:551 +0x46a
github.com/github/git-lfs/commands.Run()
    /Users/rick/go/src/github.com/github/git-lfs/commands/commands.go:85 +0x23
main.main()
    /Users/rick/go/src/github.com/github/git-lfs/git-lfs.go:9 +0x14

I followed the README to start a new server. I configured an empty repo to point to my new lfs-test-server. I then ran git lfs ls-files and got the error. After adding and committing a file to lfs I don't get the error any more.

Add Owner ID to S3 URLs

In order to track who owns an asset in our S3 logs we'll need the owner id added via a query param, i.e. /alambic/media/<oid>?owner=<id>

test server reverse proxy issues

Hello! I'm not very experienced with this so I might've overlooked something, but:

(1) I set up the lfs-test-server, and was able to successfully commit to it with a local repo and see the hash on github (using the remote lfs address of http://localhost:9999/testrepo/lfs).

(2) I then tried to use let's encrpt and nginx to handle the SSL and reverse proxy on the same machine. I set it up, and I'm able to get to the MGMT page from the public URL over HTTPS. When I try to push from a separate computer, it asks me for my password, and hangs at 0%. My logs don't show any errors, I get clean push requests, and the 'objects' part of the MGMT interface appears to update with hashes of relevant files, but if you click on them it gives a 'not found'.

My NGINX sites-enabled/default looks like the below. Any suggestions for how to troubleshoot this and/or what I'm doing wrong would be greatly appreciated! Thank you guys for making really useful stuff.

server {

    server_name   feedback.media.mit.edu;

    location / {

      proxy_set_header        Host $host;
      proxy_set_header        Referer $http_referer;
      proxy_set_header        X-Real-IP $remote_addr;
      proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
      proxy_set_header        X-Forwarded-Proto $scheme;

      # Fix the “It appears that your reverse proxy set up is broken" error.
      proxy_pass          http://localhost:9999/;
      proxy_read_timeout  100;
      proxy_send_timeout  100;
      proxy_connect_timeout    150;
      
      dav_methods  PUT DELETE MKCOL COPY MOVE;
      create_full_put_path   on;
      client_max_body_size 10000m;
      client_body_buffer_size 128k;

      proxy_redirect      http://localhost:9999 https://feedback.media.mit.edu/;
    }

    listen 443 ssl; # managed by Certbot
    ssl_certificate /etc/letsencrypt/live/feedback.media.mit.edu/fullchain.pem; # managed by Certbot
    ssl_certificate_key /etc/letsencrypt/live/feedback.media.mit.edu/privkey.pem; # managed by Certbot
    include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot

}

server {
    if ($host = feedback.media.mit.edu) {
        return 301 https://$host$request_uri;
    } # managed by Certbot


    listen 80;
    server_name   feedback.media.mit.edu;
    return 404; # managed by Certbot


}

setup lfs test server when git repository is not in same machine

hi,

was trying to setup a lfs server locally so that it acts as a storage for a bitbucket server running in cloud. I was able to start the lfs server and see that the credentials are being passed correctly. However its not working likely because lfs server know nothing about the bitbucket server instance.

we have similar setup working with artifactory:

this is my lfsconfig
[lfs]
url = "http://l-fossology:9999"

the bitbucket server is running in aws. im not sure if this use case is supported, can you give me some help with this?

this is my log

$ git push
15:12:46.319953 git.c:349               trace: built-in: git 'push'
15:12:46.321455 run-command.c:336       trace: run_command: 'ssh' '[email protected]' 'git-receive-pack '\''/spde/testlfs.git'\'''
15:12:48.416491 run-command.c:336       trace: run_command: '.git/hooks/pre-push' 'origin' 'ssh://[email protected]/spde/testlfs.git'
15:12:48.465527 git.c:563               trace: exec: 'git-lfs' 'pre-push' 'origin' 'ssh://[email protected]/spde/testlfs.git'
15:12:48.465527 run-command.c:336       trace: run_command: 'git-lfs' 'pre-push' 'origin' 'ssh://[email protected]/spde/testlfs.git'
trace git-lfs: run_command: 'git' config -l -f D:\prod\SPDE\testlfs\.lfsconfig
trace git-lfs: run_command: 'git' config -l
trace git-lfs: run_command: 'git' version
trace git-lfs: tq: running as batched queue, batch size of 100
trace git-lfs: creds: git credential fill ("http", "l-fossology:9999", "")
trace git-lfs: Filled credentials for http://l-fossology:9999
trace git-lfs: HTTP: POST http://l-fossology:9999/locks/verify
trace git-lfs: HTTP: 404
trace git-lfs: commands: disabling lock verification for "http://l-fossology:9999"
trace git-lfs: run_command: 'git' config lfs.http://l-fossology:9999.locksverify false
trace git-lfs: pre-push: refs/heads/master 59224486daa94d7a34a90f9207812fa2e6999138 refs/heads/master 325f461522ad4c2ce739da2628de305882aaa5fc
trace git-lfs: run_command: git rev-list --objects 59224486daa94d7a34a90f9207812fa2e6999138 --not --remotes=origin --
trace git-lfs: run_command: git cat-file --batch-check
trace git-lfs: run_command: git cat-file --batch
trace git-lfs: tq: sending batch of size 1
trace git-lfs: api: batch 1 files
trace git-lfs: creds: git credential fill ("http", "l-fossology:9999", "")
trace git-lfs: Filled credentials for http://l-fossology:9999
trace git-lfs: HTTP: POST http://l-fossology:9999/objects/batch
trace git-lfs: HTTP: 200
trace git-lfs: HTTP: {"objects":[{"oid":"7ee3a32978b896ccdf3ea44e91530e9e3d5bff9977fbe415bb0fd1db66adc91c","size":10818712,"actions":{"download":{"href":"http://127.0.0.1:9999/objects/7ee3a32978b896ccdf3ea44e91530e9e3d5bff9977fbe415bb0fd1db66adc91c","header":{"Accept":"application/vnd.git-lfs","Authorization":"Basic YnVpbGRtYXN0ZXI6cGRKNzIyTlU="},"expires_at":"0001-01-01T00:00:00Z"},"upload":{"href":"http://127.0.0.1:9999/objects/7ee3a32978b896ccdf3ea44e91530e9e3d5bff9977fbe415bb0fd1db66adc91c","header":{"Accept":"application/v
trace git-lfs: HTTP: nd.git-lfs","Authorization":"Basic YnVpbGRtYXN0ZXI6cGRKNzIyTlU="},"expires_at":"0001-01-01T00:00:00Z"}}}]}

trace git-lfs: tq: starting transfer adapter "basic"
Git LFS: (0 of 1 files) 0 B / 10.32 MB                                                                                                                                                                                                                                     trace git-lfs: HTTP: PUT http://127.0.0.1:9999/objects/7ee3a32978b896ccdf3ea44e91530e9e3d5bff9977fbe415bb0fd1db66adc91c
Git LFS: (0 of 1 files) 0 B / 10.32 MB                                                                                                                                                                                                                                     trace git-lfs: tq: retrying object 7ee3a32978b896ccdf3ea44e91530e9e3d5bff9977fbe415bb0fd1db66adc91c: LFS: Put http://127.0.0.1:9999/objects/7ee3a32978b896ccdf3ea44e91530e9e3d5bff9977fbe415bb0fd1db66adc91c: dial tcp 127.0.0.1:9999: connectex: No connection could be made because the target machine actively refused it.
trace git-lfs: tq: enqueue retry #1 for "7ee3a32978b896ccdf3ea44e91530e9e3d5bff9977fbe415bb0fd1db66adc91c" (size: 10818712)
trace git-lfs: tq: sending batch of size 1
trace git-lfs: api: batch 1 files
trace git-lfs: creds: git credential fill ("http", "l-fossology:9999", "")
trace git-lfs: Filled credentials for http://l-fossology:9999
trace git-lfs: HTTP: POST http://l-fossology:9999/objects/batch
trace git-lfs: HTTP: 200
trace git-lfs: HTTP: {"objects":[{"oid":"7ee3a32978b896ccdf3ea44e91530e9e3d5bff9977fbe415bb0fd1db66adc91c","size":10818712,"actions":{"download":{"href":"http://127.0.0.1:9999/objects/7ee3a32978b896ccdf3ea44e91530e9e3d5bff9977fbe415bb0fd1db66adc91c","header":{"Accept":"application/vnd.git-lfs","Authorization":"Basic YnVpbGRtYXN0ZXI6cGRKNzIyTlU="},"expires_at":"0001-01-01T00:00:00Z"},"upload":{"href":"http://127.0.0.1:9999/objects/7ee3a32978b896ccdf3ea44e91530e9e3d5bff9977fbe415bb0fd1db66adc91c","header":{"Accept":"application/v
trace git-lfs: HTTP: nd.git-lfs","Authorization":"Basic YnVpbGRtYXN0ZXI6cGRKNzIyTlU="},"expires_at":"0001-01-01T00:00:00Z"}}}]}

management page in server reports funny url if running over http instead of https

If running the server over http instead of https the management UI page reports an invalid URL to use in your .gitconfig . It's missing the http

Cleaning Up

So how would one go cleaning up this test server?

Is it just a question of remove the db and related objects and firing up the daemon, pushing up again to the server?

crypto/tls: private key does not match public key

I'm trying to specify my own ssl keys, but the server won't run with them. It gives:

2015-05-28T21:03:57Z 4f3e3d0bdc3c lfs[104] [proc.go:63]: fn=main msg=Using https
2015-05-28T21:03:58Z 4f3e3d0bdc3c lfs[104] [main.go:81]: fn=main err=Could not create https listener: crypto/tls: private key does not match public key

This key & crt was generated by Comodo or some similar paid service (can't remember) and they work fine with apache. Self-generated keys work fine.

Remove _workspace

There is no reason to put them into git repo, 'cause they come from other projects, you don't edit it or modify it in this project, no reason to track and waste history and space, what do you think?

Ref:
http://stackoverflow.com/questions/26334220/should-i-commit-godeps-workspace-or-is-godeps-json-enough

First push fails but second succeeds with large files and lfs-test-server

We have a project that needs to track large (>4GB) files so we spun up a little lfs-test-server on ubuntu18.04 LTS to go along with our repo on github.com. We're on the latest lfs-test-server (built from source) and latest git for windows. We started testing some things out with a dummy repo.

We're using the workaround of skipping smudge to work with these files on windows. It seems to work pretty well most of the time. But a few minutes ago I ran git push after a commit with a new big file, and got this output:

Uploading LFS objects: 0% (0/1), 9.1 GB | 104 MB/s, done Fatal error: Server error: http://10.100.0.1:9999/objects/2400552958e23519156f20f3256b49bc572cd9d728ef50a8d93702d886813ab8 error: failed to push some refs to '[email protected]:aggieNick02/hello-world.git

I checked the lfs-test-server's file store and the entire new file, with correct contents, was present (sha256 matched). The log for lfs-test-server showed no errors:

2018-12-06T20:18:08Z bmaster-dev lfs[17919] [server.go:342]: method=POST url=/objects/batch status=200 request_id=7448c73b-a8c8-e706-004b-f0c387c7b411 2018-12-06T20:18:08Z bmaster-dev lfs[17919] [server.go:342]: method=POST url=/objects/batch status=200 request_id=8324022a-491a-ab35-91ef-696d3b650979 2018-12-06T20:18:08Z bmaster-dev lfs[17919] [server.go:342]: url=/objects/batch status=200 request_id=f1cb516f-58bc-da0c-6c9c-18cd50faed95 method=POST 2018-12-06T20:18:08Z bmaster-dev lfs[17919] [server.go:342]: method=POST url=/objects/batch status=200 request_id=61caa3ad-87f8-4322-d697-8dcf248e71bd 2018-12-06T20:18:08Z bmaster-dev lfs[17919] [server.go:342]: method=POST url=/objects/batch status=200 request_id=053b138a-4d0f-a740-5968-bfa3391d7687 2018-12-06T20:18:08Z bmaster-dev lfs[17919] [server.go:342]: request_id=68dac289-c26f-41db-76b0-6c18f6ad0d65 method=POST url=/objects/batch status=200 2018-12-06T20:18:08Z bmaster-dev lfs[17919] [server.go:342]: method=POST url=/objects/batch status=200 request_id=f37540e9-2710-b812-43c8-eda61ebfac91 2018-12-06T20:18:08Z bmaster-dev lfs[17919] [server.go:342]: method=POST url=/objects/batch status=200 request_id=fc16bd86-5627-0d33-1309-389fef6457af 2018-12-06T20:18:11Z bmaster-dev lfs[17919] [server.go:361]: url=/objects/2400552958e23519156f20f3256b49bc572cd9d728ef50a8d93702d886813ab8 status=200 request_id=67556744-5757-00ab-aaf9-22de77eff880 method=PUT

At a loss, I ran git push again. This time, success. (The only thing worse than failure, because now it's an intermittent issue. ;-) ) Also bizarre was that the output showed uploading twice the size of the new file:

$ git push Uploading LFS objects: 100% (1/1), 18 GB | 104 MB/s, done Enumerating objects: 6, done. Counting objects: 100% (6/6), done. Delta compression using up to 8 threads Compressing objects: 100% (4/4), done. Writing objects: 100% (4/4), 514 bytes | 257.00 KiB/s, done. Total 4 (delta 1), reused 0 (delta 0) remote: Resolving deltas: 100% (1/1), completed with 1 local object. To github.com:aggieNick02/hello-world.git de87404..b617ba8 master -> master

If I repeat this whole sequence (new big file, add and commit, push, push), I get the same results. First push fails, second one succeeds with double the upload size... any ideas what's going on?

No binaries for Release 0.3.0

Please upload!

Circuit breakers around external API calls

No binaries for v0.4.0

Are there going to be binaries for v0.4.0? Thanks!

Panic when running latest Mac.AMD64.gz on macOS 10.13.4 (17E202)

$ ./lfs-test-server
panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xb code=0x1 addr=0x20 pc=0x1933f5]

goroutine 1 [running]:
github.com/boltdb/bolt.(*Bucket).inlineable(0x0, 0x10)
	/Users/scott/src/gocode/src/github.com/github/lfs-test-server/Godeps/_workspace/src/github.com/boltdb/bolt/bucket.go:553 +0x5
github.com/boltdb/bolt.(*Bucket).spill(0xc8200f6838, 0x0, 0x0)
	/Users/scott/src/gocode/src/github.com/github/lfs-test-server/Godeps/_workspace/src/github.com/boltdb/bolt/bucket.go:499 +0x105
github.com/boltdb/bolt.(*Tx).Commit(0xc8200f6820, 0x0, 0x0)
	/Users/scott/src/gocode/src/github.com/github/lfs-test-server/Godeps/_workspace/src/github.com/boltdb/bolt/tx.go:150 +0x1d4
github.com/boltdb/bolt.(*DB).Update(0xc820071520, 0xc8203e3d58, 0x0, 0x0)
	/Users/scott/src/gocode/src/github.com/github/lfs-test-server/Godeps/_workspace/src/github.com/boltdb/bolt/db.go:512 +0x11d
main.NewMetaStore(0x505cf8, 0x6, 0x779c40, 0x0, 0x0)
	/Users/scott/src/gocode/src/github.com/github/lfs-test-server/meta_store.go:47 +0xc3
main.main()
	/Users/scott/src/gocode/src/github.com/github/lfs-test-server/main.go:85 +0x8fc

goroutine 17 [syscall, locked to thread]:
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:1696 +0x1

goroutine 19 [syscall]:
os/signal.loop()
	/usr/local/go/src/os/signal/signal_unix.go:22 +0x18
created by os/signal.init.1
	/usr/local/go/src/os/signal/signal_unix.go:28 +0x37

LFS_HOST insufficient when using HTTPS -> HTTP reverse proxy

I have a use case, where I'm trying to use Apache reverse proxy to act as a HTTPS reverse proxy that redirect https://HOST/git-lfs/ to a backend lfs-test-server that's serving HTTP on locahost-only port 9999 (with non-root user that doesn't have access to SSL certificates).

The problem is, if I set LFS_HOST to https://HOST/git-lfs/, lfs-test-server builds malformed URLs like http://https://HOST/git-lfs/.

I propose checking if LFS_HOST starts with a protocol definition, and if so, omitting the http:// prefix it normally adds.

Clarifying .gitconfig setup and LFS url

Hi, I'm have some confusions in regards to the setup instructions (similar to #23), but thought it's better to start a new issue which can be answered and closed off.

Which .gitconfig file am I meant to update? Is it:
- the root ~/.gitconfig file
- the local repository .git/config file
- or a new .gitconfig file which I need to create inside my repository (which doesn't seem to be part of git convention?)
What is the correct LFS url I'm supposed to use?
- lfs-test-server README says [lfs] url = "https://host:port/jimdoe/lfsrepo
- If I go to https://localhost:9999/mgmt it says [lfs] url = "https://host:port/user/repo
- Other users seem to be using [lfs] url = "https://host:port/user/
- If I look at alternative implementations such as git-lfs-s3, it says to use [lfs]
  url = "http://yourserver.com"

I was able to get the lfs-test-server up and running, but then ran into other issues with the server hanging when I tried to do git-lfs push. Please confirm the above points 1 and 2 and I will re-visit my setup before looking into the hanging issue. Many thanks!

Using SSH keys

Is it possible to use ssh keys instead of user and pass?
So ssh keys of users permitted to do with project's git repository are also on the LFS server and there's no need for double credential in the project.

File Locking API

It would be great if this projects would also support the git file locking API:
https://github.com/git-lfs/git-lfs/blob/master/docs/api/locking.md

Test Server does not allow for testing Batch API

As far as I can tell, the lfs-test-server doesn't implement the batch API, so there's no way to use the tool to test the full git-lfs apis available to the client.

random 401 status in lfs.log causing client-side smudge errors

We use the lfs-test server to host the binary files in one of our git repos. This is on-premises and it happily runs on a Centos server.

Except that it randomly will throw a 401 status when asked for file. The git repo has about 100 binary files in the tree, and there's no telling when you'll get the 401. If you repeatedly pull, the client-side cache will eventually populate and the "error goes away" for the user.

There seems to be some sort of caching issue, because once ONE developer has finally got their repo synced for a particular commit, the next developer that pulls that commit from the git repo has NO trouble getting the latest 97 files from the lfs repo. The server seems to have no problems finding the files and supplying them.

If a dev pushes a new set of files, they will cause this grief once again when everyone pulls their new work - right up to the point the server has magically cached all of them and can now start serving them all without a problem.

We use https and authentication through the management interface with a user set up. Of course, git lfs now complains at every point that this isn't good enough, even if it did work fine when we first started with lfs all those months ago.

I have attached the server log and a client error log.

Is there something I'm missing? We run 9 instances of the test-lfs server for different repos (all instances have their own ports assigned) - and most seem absolutely fine.. but this one it a right pain. It does have the biggest number of binary files per commit though.

20171123T114847.555849927.log

lfs.log

the lfs server is 0.3.0, my client side git is 2.15. The network topology is all on site - nothing ventures outside our firewall to the Big Bad Internet. It's fedora desktop to Centos server.

I'm at a loss. All straws gratefully grabbed.

Return 413 on excessively large batch

If a user sends an excessively large batch (i.e., larger than the default), we should return a 413 so that alternative client implementations can test this, since some implementations won't want to handle these batches.

lfs server usage

Hi,

I'm playing around with the lfs server a bit, and I can't seem to make it work as I would expect: my lfs files are commited to the git repository instead. This is most likely me not understanding how to use the lfs server correctly, so please tell me if I'm doing something obviously wrong.

My setup is the following:

Create a repository on Github, clone it locally
Set up lfs to track all files ending in *.lfs

git lfs track "*.lfs"

Start the lfs-test-server

LFS_ADMINUSER=test LFS_ADMINPASS=test lfs-test-server

Add my [user] + password using the /mgmt interface
Set up .gitconfig in the local repository for my lfs test server.

[lfs]
url = "http://localhost:8080/myuser"

Add an lfs-tracked file

echo > mytest.lfs
git add mytest.lfs
git commit -m "add lfs-tracked file"
git push origin master

The expected result here is that lfs tracks the file and adds it to the lfs server directory.
Instead, it is added to the git repository as a normal, git-tracked file.

Management URLs should be relative

While building a reverse proxy solution ( see #66 ), I noticed the management page refers to the other pages (like /users), as well as the CSS file, with an absolute path: @import "/mgmt/css/primer.css". This doesn't work if mgmt has been mapped to a non-root path, such as: https://PROXY-SERVER/git-lfs/mgmt.

Could it be "../users" and "../css/primer.css" instead, or alternatively, use LFS_HOST to build the absolute URLs?

Git lfs ignores lfs url and doesn't push or upload to lfs server

Steps:

Set up lfs-test-server on http://mywebsite.com:9999 and run the lfs-test-server binary
Verify lfs-test-server works by checking out the /mgmt, logging in, and adding in a user
On my local, mkdir foo && cd foo && git init
On my local, git lfs install && git lfs track "*.png" && cp /some/file.png . && git add file.png && git commit
Add the line below to my "foo" repo's .gitconfig, .git/config, .lfsconfig. The line is what is suggested by the /mgmt page.

[lfs]
    url = "http://mywebsite.com:9999/"

Verify that pngs are tracked by checking git lfs ls-files and the contents of .gitattributes which all look good.
Still on local, in some other directory, do git clone /path/to/foo myfooclone
Back in foo repo, do git remote add origin /path/to/myfooclone
In foo repo, do git push origin master.

At this step, I expect it to push the commits to myfooclone, but upload LFS objects to the lfs.url. Instead, I just get a regular push as if git lfs wasn't doing anything. No errors, no activity on the git lfs server logs, no requests for authentication.

Ideas?

lfs test server is not accepting any credentials

We are trying to setup an LFS Test Server on a server we have in office. It seems to be running fine but we can't get anything to upload properly. When we try git lfs push origin master on our branches we are prompted for credentials forever. The only output we get it is:

Git LFS: (0 of 4 files) 0 B / 220 B

We can see the shas on the mgmt route in the object view, but when clicked on they simply say Not Found. The server sh file looks like so

#!/bin/bash

set -eu
set -o pipefail


LFS_LISTEN="tcp://:9999"
LFS_HOST="ourserver:9999"
LFS_CONTENTPATH="content"
LFS_ADMINUSER="admin"
LFS_ADMINPASS="admin"
LFS_CERT="mine.crt"
LFS_KEY="mine.key"
LFS_SCHEME="https"

export LFS_LISTEN LFS_HOST LFS_CONTENTPATH LFS_ADMINUSER LFS_ADMINPASS LFS_CERT LFS_KEY LFS_SCHEME

./lfs-test-server

and my gitconfig looks like so

[lfs]
	url = https://ourserver:9999/
[http]
	sslverify = false

I have tried entering admin and user credentials to finish the push, nothing seems to work. Any ideas?

permissions

How can I create user with read+write permissions, and user with read-only permissions?

If not possible, is there any workaround?

Does it mean the lfs-test-server can be used only privately, isn't it possible to set up a public lfs server?

exec format error

Hi, I am not sure what I am doing wrong, but when I unpack the file and run it (tried Mac version as well as Linux) they both give "exec format error". What am I missing here?

Following the example instructions for starting a server gives you Unsupported listener protocol

I wrote a bat file to start the test server with this content:

SETLOCAL
set LFS_LISTEN="tcp://:9999"
set LFS_HOST="127.0.0.1:9999"
set LFS_CONTENTPATH="content"
set LFS_ADMINUSER="adminuser"
set LFS_ADMINPASS="adminpass"
rem set LFS_CERT="mine.crt"
rem set LFS_KEY="mine.key"
set LFS_SCHEME="http"

.\lfs-test-server

Result when running was an error message:

2015-07-09T15:44:45Z CICERO2 lfs[13456] [main.go:72]: fn=main err=Could not create listener: Unsupported listener protocol:

Not setting LFS_SCHEME, LFS_LISTEN and LFS_HOST worked better.

request for /mgmt results in 404

I'm running the test server on Windows 10 from the binary like this:

D:\Program Files\git-lfs-test-server>SET LFS_ADMIN=admin & SET LFS_ADMINPASS=password & lfs-test-server.exe

2017-08-01T11:49:54Z KWS007 lfs[8588] [proc.go:111]: pid=8588 addr=tcp://:8080 version=0.3.0 fn=main msg=listening
2017-08-01T11:50:08Z KWS007 lfs[8588] [server.go:362]: method=GET url=/mgmt status=404 request_id=a72ed098-0c0c-6e53-a40b-0e90a3ee2f4b

And then requesting:

http://localhost:8080/mgmt

But this returns "Not Found" and a 404 is registered (see log above). Should there be some html files along with the executable, or are the (missing) admin page all generated code?