invalid_scope: Empty or missing scope not allowed about gcsfs HOT 10 CLOSED

Seems related: googleapis/google-auth-library-python#217

from gcsfs.

The "service" route does already have the scopes set ( https://github.com/dask/gcsfs/blob/master/gcsfs/core.py#L370 ), but - shooting in the dark here - I can add it also to the file-based auth. I wouldn't have thought that made a difference, but you could try git+https://github.com/martindurant/gcsfs@scoped. If not, it would be useful to know via pdb what fs.method ended up as.

from gcsfs.

Ok, I will try that branch later today or tomorrow.

Do you recommend a different method of authentication?

from gcsfs.

If you have a .json file, you can always pass that directly to the constructor GCSFileSystem(token='/path/to/file.json', project='my-project'), and should also explicitly give the appropriate project=.

In the past, google-default and cloud auth mechanisms have worked without problem. If you give no indication to GCSFileSystem at all, it will try the various methods in turn, and it seems that something apparently gave no error (some auth was found), but the auth was actually incomplete. Or something. I would expect token='cloud' to be the smoothest method on GCE.

from gcsfs.

I am not on GCE. I am on pleiades.

I will try passing the token explicity.

from gcsfs.

It looks like specifying the token='/path/to/file.json' works. Sorry for the clutter.

However, it would be great if gcsfs would read the GOOGLE_APPLICATION_CREDENTIALS environment variable and use this to fill in token. I could try to put together a PR for this if you wish.

from gcsfs.

That's an impressive cluster!

I agree that GOOGLE_APPLICATION_CREDENTIALS should be respected, but I am surprised that gauth.default() doesn't pick this up already. Your original point about providing a more sensible error message, preferably when connecting as opposed to the first usage, still stands - but I am not sure how to achieve this.

from gcsfs.

OK, so what I understand is that google.auth.default() is supposed to respect GOOGLE_APPLICATION_CREDENTIALS, but "old" credentials can be missing scopes and/or project identifiers. Perhaps google.cloud.storage.Client knows how to deal with that. I can add the scopes to the default credentials, which apparently will use them "if necessary" - doesn't actually seem to make any difference to me.

from gcsfs.

I just created / downloaded the credentials yesterday. How can they be "old" already?

from gcsfs.

I just created / downloaded the credentials yesterday. How can they be "old" already?

I don't know :( It's hard to try to say what the google auth library actually did for your case.

from gcsfs.