Key File not found about spark-bigquery HOT 5 CLOSED

Hi @kurtmaile thanks for raising this, the problem is S3 isnt really a file system

Just mounting the s3 file wont work, I think you physically need to save the file to the cluster file system

You can test if this file exists by using the dbfs.utils.ls and the file should appear, or running a shell script using %sh ls

If the file isnt appearing then the application cannot see it

Hope that makes sense

from spark-bigquery.

Hey thanks alot for your quick response! :)

The file is there now - I headed it and saved the json contents (put it) to a new local dir, you can see in the dbox link the ls finds it (the head shows the right contents even though not displayed) but still get the same error. Odd!?

https://www.dropbox.com/s/4i6gcok6uyvwb2s/Screenshot%202017-04-12%2018.52.19.png?dl=0

Bit stumped on this rather simple part lol.

from spark-bigquery.

hmmm just seen there is a prefix of 'file:/' for local disk Ill try that!

from spark-bigquery.

ok so that worked thanks heaps! The little subtle things :) - legend thanks for helping out.

from spark-bigquery.

from spark-bigquery.