pure Go MTP library does not work with Samsung non-nexus devices about go-mtpfs HOT 26 CLOSED

Hi, I've just tried the new version and unfortunately it didn't solved the issue; actually - after the first (failed) time - I'm not able to connect the Galaxy S3 anymore, i.e. it is not detected.

root@debian:~/mtpgo/src/github.com/hanwen/go-mtpfs# git rev-parse HEAD
52ecddec552c20cac2fef0a6987ed62763e63dd3
root@debian:~/mtpgo/src/github.com/hanwen/go-mtpfs# cd /root/mtpgo/bin
root@debian:~/mtpgo/bin# PATH=$PATH:/root/mtp/fuse-2.8.3/util ./go-mtpfs mnt
2013/02/17 13:09:21 OpenSession failed: LIBUSB_ERROR_TIMEOUT; attempting reset
2013/02/17 13:09:25 Configure failed: OpenSession after reset: LIBUSB_ERROR_TIMEOUT
root@debian:~/mtpgo/bin# PATH=$PATH:/root/mtp/fuse-2.8.3/util ./go-mtpfs mnt
2013/02/17 13:09:28 detect failed: no MTP devices found

The result is the same if I disconnect and reconnect the phone, it is present on lsusb but go-mtpfs does not see it (with No MTP devices found message).

from go-mtpfs.

Sorry, I've to correct my previous post, because I've retried rebooting the virtual machine (that behaviour seemed too weird to me).
Now it (quite) works, in the way the phone is detected and the folders mounted, but then I cannot do ls on them:

root@debian:~/mtpgo/bin# PATH=$PATH:/root/mtp/fuse-2.8.3/util ./go-mtpfs mnt
2013/02/17 13:20:47 starting FUSE unknown
2013/02/17 13:21:02 GetObjectInfo failed: parsing time "20120101T000016.": extra text: .
2013/02/17 13:21:10 GetObjectInfo failed: parsing time "20120101T010022.": extra text: .

The two errors appear when I do this (on another terminal):

root@debian:~/mtpgo/bin# cd mnt/
root@debian:~/mtpgo/bin/mnt# ls
Card  Phone
root@debian:~/mtpgo/bin/mnt# ls Card/
ls: cannot open directory Card: Input/output error
root@debian:~/mtpgo/bin/mnt# ls Phone/
ls: cannot open directory Phone: Input/output error

... and the errors appear after each "ls".

Results of go test -v

from go-mtpfs.

On Sun, Feb 17, 2013 at 2:37 PM, virtualdj [email protected] wrote:

Sorry, I've to correct my previous post, because I've retried rebooting
the virtual machine (that behaviour seemed too weird to me).
Now it (quite) works, in the way the phone is detected and the folders
mounted, but then I cannot do ls on them:

root@debian:~/mtpgo/bin# PATH=$PATH:/root/mtp/fuse-2.8.3/util ./go-mtpfs mnt
2013/02/17 13:20:47 starting FUSE unknown
2013/02/17 13:21:02 GetObjectInfo failed: parsing time "20120101T000016.": extra text: .
2013/02/17 13:21:10 GetObjectInfo failed: parsing time "20120101T010022.": extra text: .

The two errors appear when I do this (on another terminal):

root@debian:/mtpgo/bin# cd mnt/
root@debian:/mtpgo/bin/mnt# ls
Card Phone
root@debian:~/mtpgo/bin/mnt# ls Card/
ls: cannot open directory Card: Input/output error

... and the error appears.

thanks, this looks very good.

Find a fix for the time issue in commit
5783584

Han-Wen Nienhuys - [email protected] - http://www.xs4all.nl/~hanwen

from go-mtpfs.

Also, can you plug the device and run

cd mtp ; go test -v

and send me the output?

On Sun, Feb 17, 2013 at 3:15 PM, Han-Wen Nienhuys [email protected] wrote:

On Sun, Feb 17, 2013 at 2:37 PM, virtualdj [email protected] wrote:

Sorry, I've to correct my previous post, because I've retried rebooting
the virtual machine (that behaviour seemed too weird to me).
Now it (quite) works, in the way the phone is detected and the folders
mounted, but then I cannot do ls on them:

root@debian:~/mtpgo/bin# PATH=$PATH:/root/mtp/fuse-2.8.3/util ./go-mtpfs
mnt
2013/02/17 13:20:47 starting FUSE unknown
2013/02/17 13:21:02 GetObjectInfo failed: parsing time "20120101T000016.":
extra text: .
2013/02/17 13:21:10 GetObjectInfo failed: parsing time "20120101T010022.":
extra text: .

The two errors appear when I do this (on another terminal):

root@debian:/mtpgo/bin# cd mnt/
root@debian:/mtpgo/bin/mnt# ls
Card Phone
root@debian:~/mtpgo/bin/mnt# ls Card/
ls: cannot open directory Card: Input/output error

... and the error appears.

thanks, this looks very good.

Find a fix for the time issue in commit
5783584

Han-Wen Nienhuys - [email protected] - http://www.xs4all.nl/~hanwen

Han-Wen Nienhuys - [email protected] - http://www.xs4all.nl/~hanwen

from go-mtpfs.

Find a fix for the time issue in commit

Great, now ls works! Unfortunately, the other commands do not...

root@debian:~/mtpgo/bin/mnt# cp /var/log/dmesg Card/
cp: cannot create regular file `Card/dmesg': Input/output error
2013/02/18 17:50:22 AndroidBeginEditObject failed: OperationNotSupported

root@debian:~/mtpgo/bin/mnt# echo "123" > Card/test.txt
-bash: Card/test.txt: Input/output error
2013/02/18 17:51:40 AndroidBeginEditObject failed: OperationNotSupported

root@debian:~/mtpgo/bin/mnt# cat Card/Prova.txt
cat: Card/Prova.txt: Input/output error
2013/02/18 17:52:20 AndroidGetPartialObject64 failed: OperationNotSupported

Also, can you plug the device and run cd mtp ; go test -v and send me the output?

I've updated the Gist link on my previous post, so you can compare the two versions.

from go-mtpfs.

Great, should be fixed - see 75b2f14

from go-mtpfs.

OK, now the commands work, but I'm still having issues.
Please consider this example:

root@debian:~/mtpgo/bin# echo "Test string" > t1.txt; cp t1.txt mnt/Card/t1.txt; ls -l mnt/Card/t1.txt; cp mnt/Card/t1.txt t2.txt; rm mnt/Card/t1.txt; diff t1.txt t2.txt
-rw-r--r-- 0 root root 12 Feb 18 19:46 mnt/Card/t1.txt
1d0
< Test string
root@debian:~/mtpgo/bin# ls -l t?.txt
-rw-r--r-- 1 root root 12 Feb 18 19:43 t1.txt
-rw-r--r-- 1 root root  0 Feb 18 19:43 t2.txt

Doesn't sound good, right?

So I'm able to read and copy files from the device, but unable to write new files correcly (all of them are 0 byte sized).
But we are improving! 👍

from go-mtpfs.

Predictably, it does work over here.

Can you figure out which of the two steps is going wrong? Is it the send-to-device, or the read-from-device? Can you receive files that are already on the device correctly?

Also, can you run

cd fs ; go test -v

and send that paste to me? I expect that it will say FAIL for some tests, but I want to see the logs.

If it says PASS, can you run

go-mtpfs -debug=fs,mtp,usb,data >& logfile

and run the failure scenario again, and attach the logfile?

thanks for bearing with me,

from go-mtpfs.

Can you figure out which of the two steps is going wrong? Is it the send-to-device, or the read-from-device? Can you receive files that are already on the device correctly?

You're right, I didn't do these simple tests. Reading a text file on the SD Memory Card:

root@debian:~/mtpgo/src/github.com/hanwen/go-mtpfs# git rev-parse HEAD
9eb1955afe122098d9ba916bb05b6e2e415a015c
root@debian:~/mtpgo/src/github.com/hanwen/go-mtpfs# cd /root/mtpgo/bin/
root@debian:~/mtpgo/bin# PATH=$PATH:/root/mtp/fuse-2.8.3/util ./go-mtpfs --usb-timeout=10000 mnt &
[1] 3131
root@debian:~/mtpgo/bin# 2013/02/20 17:33:10 starting FUSE unknown

root@debian:~/mtpgo/bin# ls -l mnt/Card/Textfile.txt
-rw-r--r-- 1 root root 69 Feb 18 21:43 mnt/Card/Textfile.txt
root@debian:~/mtpgo/bin# cp mnt/Card/Textfile.txt .
2013/02/20 17:34:01 fetched "Textfile.txt", 69 bytes in 89 ms. 0.0 MB/s
2013/02/20 17:34:01 trySplice: Splice write: bad address
root@debian:~/mtpgo/bin# ls -l Prova.txt
-rw-r--r-- 1 root root 69 Feb 20 17:34 Textfile.txt

There is an error ("Splice write"), but the received file is OK (same MD5 too).

About writing/uploading:

root@debian:~/mtpgo/bin# ls -l UploadedFile.txt
-rw-r--r-- 1 root root 36 Feb 20 17:38 UploadedFile.txt
root@debian:~/mtpgo/bin# cat UploadedFile.txt
File to be
uploaded to
GS3 SD card.
root@debian:~/mtpgo/bin# cp UploadedFile.txt mnt/Card/
2013/02/20 17:38:57 sending file "UploadedFile.txt" to device: 36 bytes.
2013/02/20 17:38:57 sent 36 bytes in 506 ms. 0.0 MB/s
root@debian:~/mtpgo/bin# ls -l mnt/Card/UploadedFile.txt
-rw-r--r-- 1 root root 36 Feb 20 17:38 mnt/Card/UploadedFile.txt
root@debian:~/mtpgo/bin# cat mnt/Card/UploadedFile.txt
2013/02/20 17:39:19 fetched "UploadedFile.txt", 36 bytes in 34 ms. 0.0 MB/s
2013/02/20 17:39:19 trySplice: Splice write: bad address

As you see, I'm unable to cat*, however now the file is successfully transferred on the device (same MD5 again), apart from the errors (maybe you've updated something)?

But the test of yesterday still doesn't succeed:

root@debian:~/mtpgo/bin# echo "Test string" > t1.txt; cp t1.txt mnt/Card/t1.txt; ls -l mnt/Card/t1.txt; cp mnt/Card/t1.txt t2.txt; rm mnt/Card/t1.txt; diff t1.txt t2.txt
2013/02/20 17:42:01 sending file "t1.txt" to device: 12 bytes.
2013/02/20 17:42:01 sent 12 bytes in 148 ms. 0.0 MB/s
-rw-r--r-- 0 root root 12 Feb 20 17:42 mnt/Card/t1.txt
2013/02/20 17:42:01 fetched "t1.txt", 12 bytes in 5 ms. 0.0 MB/s
2013/02/20 17:42:01 trySplice: Splice write: bad address
1d0
< Test string

root@debian:~/mtpgo/bin# ls -l t?.txt
-rw-r--r-- 1 root root 12 Feb 20 17:42 t1.txt
-rw-r--r-- 1 root root  0 Feb 20 17:42 t2.txt

This is quite funny.

Copying an 8 MB file from the Card to the VM works, but with more errors:

2013/02/20 17:49:10 trySplice: splice.Grow failed.
2013/02/20 17:49:10 trySplice: splice.Grow failed.
2013/02/20 17:49:10 trySplice: splice.Grow failed.
2013/02/20 17:49:10 trySplice: splice.Grow failed.
2013/02/20 17:49:10 trySplice: splice.Grow failed.
2013/02/20 17:49:10 trySplice: splice.Grow failed.
2013/02/20 17:49:10 trySplice: splice.Grow failed.
2013/02/20 17:49:10 trySplice: splice.Grow failed.
2013/02/20 17:49:10 trySplice: splice.Grow failed.
2013/02/20 17:49:10 trySplice: splice.Grow failed.
2013/02/20 17:49:10 trySplice: splice.Grow failed.
2013/02/20 17:49:10 trySplice: splice.Grow failed.

Copying a 300 MB file doesn't work either (it returns immediately, and changing the TEMP env. var. doesn't help):

root@debian:~/mtpgo/bin# cp 'mnt/Card/Video.mkv' .
2013/02/20 17:56:38 fetch failed: not enough space. Have 185671680, want 241849755
2013/02/20 17:56:38 fetch failed: not enough space. Have 185671680, want 241849755
cp: reading `mnt/Card/Video.mkv': Input/output error
cp: failed to extend `./Video.mkv': Input/output error
root@debian:~/mtpgo/bin# df -h | grep uuid
/dev/disk/by-uuid/e587c542-f39e-451e-b842-e25335787a27  7.6G  4.4G  2.8G  61% /

_Attachments_
fs-log.txt
go-mtpfs-log.txt

from go-mtpfs.

• I think have fixed the bug with files not being sent correctly: cb86654
• You need more space on /tmp . It ran out of space when you copied the large file. I've made the message more clear (5367aed ), and made the dir configurable with $TMPDIR (f0d8d53 ).

Please try again.

from go-mtpfs.

in the interest of privacy (the logs contain a dump of filenames on your phone), you may want to delete the test logs you've uploaded, btw.

from go-mtpfs.

OK, you've improved the app a lot! Now I'm able to copy files to and from the device successfully, but I still have some questions.

1. What do these messages mean?

trySplice: splice.Grow failed.
...
trySplice: Splice write: bad address

Normally they appear when copying files of hundreds of megabytes and the first message appears a lot of times; the copy however ends correctly, so what's happening?

2. Sometimes the LIBUSB_ERROR_TIMEOUT appears again just after launching go-mtpfs (see the attached logfile 1), however unmounting and then remounting fixes the issue. Maybe it's because I left out the usb-timeout parameter?

3. To test the large-file copy feature, I generated a 2.3 GB file on my VM and then copied it on the device. It worked (at about 6.2 MB/s, not too much but acceptable).
   Then I removed the file from the VM, but I wasn't able to copy the file back from the device.
   See the attached logfile 2; after that error, I had to unmount the folder because all the subsequent commands failed.

from go-mtpfs.

On Sat, Feb 23, 2013 at 2:39 PM, virtualdj [email protected] wrote:

OK, you've improved the app very well! Now I'm able to copy files to and
from the device successfully, but I still have some questions.

1. What do these messages mean?

trySplice: splice.Grow failed.
...
trySplice: Splice write: bad address

Normally they appear when copying files of hundreds of megabytes and the
first message appears a lot of times; the copy however ends correctly, so
what's happening?

it's a little bit technical, but should not be cause for concern? Which
linux kernel version do you run?

2. Sometimes the LIBUSB_ERROR_TIMEOUT appears again just after launching
   go-mtpfs (see the attached logfile 1https://gist.github.com/virtualdj/f771822f27df18da18ee),
   however unmounting and then remounting fixes the issue. Maybe it's because
   I left out the usb-timeout parameter?

It may help to increase the number.

3. To test the large-file copy feature, I generated a 2.3 GB file on my VM
   and then copied it on the device. It worked (at about 6.2 MB/s, not too
   much but acceptable).
   Then I removed the file from the VM, but I wasn't able to copy the file
   back from the device.
   See the attached logfile 2https://gist.github.com/virtualdj/7c21db8a06868323469f;
   after that error, I had to unmount the folder because all the subsequent
   commands failed.

Is this reproducible? Does it always happen, or only sometimes?

I would guess that it happens just part of the time; try copying 100
different 1mb files; some will succeed, and then it fails?

Or does it fail always fail on read? If true, the first file you try to
copy should already fail.

Han-Wen Nienhuys - [email protected] - http://www.xs4all.nl/~hanwen

from go-mtpfs.

I made the splice errors a bit more verbose. Can you post a short debug log of a session that causes these errors normally?

from go-mtpfs.

it's a little bit technical, but should not be cause for concern? Which linux kernel version do you run?

Linux debian 2.6.32-5-686 i686.

It may help to increase the number.

OK, for the next tests I've put 10000.

I made the splice errors a bit more verbose. Can you post a short debug log of a session that causes these errors normally?

Here's the log with the warnings.

Is this reproducible? Does it always happen, or only sometimes?

It is always reproducible. When I want to copy the 2.3 GB file from the device, I get these errors:

2013/02/23 16:50:38 fatal error got type 0 (CONTAINER_UNDEFINED) in response, want CONTAINER_RESPONSE.; closing connection.
2013/02/23 16:50:38 error fetching: got type 0 (CONTAINER_UNDEFINED) in response, want CONTAINER_RESPONSE.
2013/02/23 16:50:38 fetch failed: input/output error
2013/02/23 16:50:38 error fetching: mtp: cannot run operation GetObject, device is not open
2013/02/23 16:50:38 fetch failed: input/output error
2013/02/23 16:54:33 CreateFolder failed: mtp: cannot run operation SendObjectInfo, device is not open

And then every command I send doesn't work anymore. I have to unmount the folder, disconnect and reconnect the device and restart go-mtpfs.

I would guess that it happens just part of the time; try copying 100 different 1mb files; some will succeed, and then it fails? Or does it fail always fail on read? If true, the first file you try to copy should already fail.

I've tried copying 100 different 1 MB files and it works. Here you can see the log with the warnings too.
So maybe the problem is with a single > 2 GB file.

from go-mtpfs.

I've fixed the trySplice problem, hanwen/go-fuse@3d0a438.

from go-mtpfs.

I've added more debugging. Can you try to repro reading the 2.3G file with -debug=mtp,usb and send the log?

thanks

from go-mtpfs.

I've fixed the trySplice problem, hanwen/go-fuse@3d0a438.

Unfortunately it didn't fix the problem, the messages appear anyway (at the end of the transfer).
See the detailed log.

I've added more debugging. Can you try to repro reading the 2.3G file with -debug=mtp,usb and send the log?

Here's the log, this time the error appears just immediately.

from go-mtpfs.

On Sun, Feb 24, 2013 at 11:46 AM, virtualdj [email protected]:

I've fixed the trySplice problem, hanwen/go-fuse@3d0a438hanwen/go-fuse@3d0a438
.

Unfortunately it didn't fix the problem, the messages appear anyway (at
the end of the transfer).
See the detailed loghttps://gist.github.com/virtualdj/c1eab5f13afb6ce7b412
.

Are you sure you also update the go-fuse library? The problem and the fix
are in the fuse library, not the go-mtpfs itself.

I've added more debugging. Can you try to repro reading the 2.3G file
with -debug=mtp,usb and send the log?

Here's the log https://gist.github.com/virtualdj/fbf063c8053720d340d4,
this time the error appears just immediately.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/29#issuecomment-14006603.

Han-Wen Nienhuys - [email protected] - http://www.xs4all.nl/~hanwen

from go-mtpfs.

On Sun, Feb 24, 2013 at 1:06 PM, Han-Wen Nienhuys [email protected] wrote:

I've added more debugging. Can you try to repro reading the 2.3G file with

-debug=mtp,usb and send the log?

Here's the log https://gist.github.com/virtualdj/fbf063c8053720d340d4,
this time the error appears just immediately.

Thanks, I think I've found this problem; see:

e9b64b1

Han-Wen Nienhuys - [email protected] - http://www.xs4all.nl/~hanwen

from go-mtpfs.

Are you sure you also update the go-fuse library? The problem and the fix are in the fuse library, not the go-mtpfs itself.

I thought the same thing at first, but I normally erase all the folders and issue a "go get github.com/hanwen/go-mtpfs" command all the times.
However, just to be sure, I went into the folder "src/github.com/hanwen/go-fuse" and this is the result of git rev-parse HEAD:
3d0a438151bbd95...

from go-mtpfs.

Never mind , you're right.

I have this fancy var I set to determine whether to use splicing, but then
I never inspect it.

Hold on...

hanwen/go-fuse@9334ba5

On Sun, Feb 24, 2013 at 1:19 PM, virtualdj [email protected] wrote:

Are you sure you also update the go-fuse library? The problem and the fix
are in the fuse library, not the go-mtpfs itself.

I tought the same thing at first, but I normally erase all the folders and
issue a "go get github.com/hanwen/go-mtpfs" command all the times.
However, just to be sure, I went into the folder "src/
github.com/hanwen/go-fuse" and this is the result of git rev-parse HEAD:
3d0a438151bbd95...

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/29#issuecomment-14007622.

Han-Wen Nienhuys - [email protected] - http://www.xs4all.nl/~hanwen

from go-mtpfs.

Yes, yes, yes! With the latest version:

1. No more splicing errors;
2. I was able to move the 2.3 GB file out of the phone (even without setting the TEMPDIR so it means that it picked up from the environment)! Speed was 8.4 MB/s, and the MD5 hash matched.

What a success, and thanks for your cooperation!

As a side note, I would like to suggest you a tip: you can implement a "--version" option that prints both the go-mtpfs and go-fuse version (or git hash) so debugging is easier.
I don't know if it's possible to automate the "version injection" using go, but it would be useful because you don't have to remember with what version of the sources you built the application.

from go-mtpfs.

On Sun, Feb 24, 2013 at 1:40 PM, virtualdj [email protected] wrote:

Yes, yes, yes! With the latest version:

1. No more splicing errors;
2. I was able to move the 2.3 GB file out of the phone (even without
   setting the TEMPDIR so it means that it picked up from the environment)!
   Speed was 8.4 MB/s, and the MD5 hash matched.

What a success, and thanks for your cooperation!

As a side note, I would like to suggest you a tip: you can implement a
"--version" option that prints both the go-mtpfs and go-fuse version (or
git hash) so debugging is easier.
I don't know if it's possible to automate the "version injection" using
go, but it would be useful because you don't have to remember with what
version of the

sources you built the application.

I've been clamoring for this with go team, but no success so far.

Han-Wen Nienhuys - [email protected] - http://www.xs4all.nl/~hanwen

from go-mtpfs.

thanks, I'm closing this isssue. Did you use the default timeout, and if no, which number makes it behave robustly?

from go-mtpfs.

I always used -usb-timeout=10000.

from go-mtpfs.