Comments (14)
For anyone stumbling here:
- update to the latest version (0.3 at the moment)
- there is now a 5th undocumented argument (opts) which supports the options
decode: false
Example:
{:ok, %Tesla.Env{body: body}} = Objects.storage_objects_get(conn, "my-bucket", "my-file.txt", [alt: "media"], decode: false)
IO.inspect(body, label: "file content")
from elixir-google-api.
Yeah, this is a pretty critical problem for me as well. GCS's usefulness is pretty limited if you can't pull the file. Here's a workaround for those who need it. It's just the same code as what's generated, it just skips the last decode command, and provides a friendlier wrapper around the whole thing.
defmodule Utils.GCS do
alias GoogleApi.Storage.V1.Connection
import GoogleApi.Storage.V1.RequestBuilder
def upload_file(bucket_id, bucket_path, file_path) do
# Authenticate.
conn = get_conn()
# Make the API request.
GoogleApi.Storage.V1.Api.Objects.storage_objects_insert_simple(
conn,
bucket_id,
"multipart",
%{name: bucket_path},
file_path
)
end
def download_file(bucket_id, bucket_path) do
conn = get_conn()
case storage_objects_get(conn, bucket_id, bucket_path, alt: :media) do
%Tesla.Env{body: <<"No such object:", _rest::binary>>} -> {:error, :file_not_found}
%Tesla.Env{body: body} -> {:ok, body}
end
end
defp get_conn() do
{:ok, token} = Goth.Token.for_scope("https://www.googleapis.com/auth/cloud-platform")
GoogleApi.Storage.V1.Connection.new(token.token)
end
defp storage_objects_get(connection, bucket, object, opts \\ []) do
optional_params = %{
:alt => :query,
:fields => :query,
:key => :query,
:oauth_token => :query,
:prettyPrint => :query,
:quotaUser => :query,
:userIp => :query,
:generation => :query,
:ifGenerationMatch => :query,
:ifGenerationNotMatch => :query,
:ifMetagenerationMatch => :query,
:ifMetagenerationNotMatch => :query,
:projection => :query,
:userProject => :query
}
%{}
|> method(:get)
|> url("/storage/v1/b/{bucket}/o/{object}", %{
"bucket" => URI.encode_www_form(bucket),
"object" => URI.encode_www_form(object)
})
|> add_optional_params(optional_params, opts)
|> Enum.into([])
|> (&Connection.request(connection, &1)).()
end
end
from elixir-google-api.
My stab at this issue:
defmodule Googly.Storage do
def download_file(bucket, name) do
url = "https://www.googleapis.com/download/storage/v1/b/#{bucket}/o/#{name}?alt=media"
GoogleApi.Gax.Request.new()
|> GoogleApi.Gax.Request.url(url)
|> Googly.Storage.Client.execute()
end
end
defmodule Googly.Storage.Client do
alias GoogleApi.Storage.V1.{
Connection,
}
def config(), do: Application.fetch_env!(:googly, Googly.Storage)
def connection() do
case Goth.Token.for_scope(config().scope) do
{:ok, %{token: token}} -> Connection.new(token)
err -> raise "Failed to acquire google storage token - #{inspect(err)}"
end
end
def execute(req) do
Connection.execute(connection(), req)
end
end
from elixir-google-api.
Just incase anyone else has some issues using @jeffdeville 's solution, you could try the almost exact code below
defmodule Utils.GCS do
alias GoogleApi.Gax.Request
alias GoogleApi.Storage.V1.{Connection, Api}
def upload_file(bucket_id, bucket_path, file_path) do
conn = get_conn()
Api.Objects.storage_objects_insert_simple(
conn,
bucket_id,
"multipart",
%{name: bucket_path},
file_path
)
end
def download_file(bucket_id, object_path) do
conn = get_conn()
case storage_objects_get(conn, bucket_id, object_path, alt: :media) do
{:error, %Tesla.Env{body: <<"No such object:", _rest::binary>>}} -> {:error, :file_not_found}
{:ok, %Tesla.Env{body: body}} -> {:ok, body}
end
end
# For Authentication
defp get_conn() do
{:ok, token} = Goth.Token.for_scope("https://www.googleapis.com/auth/cloud-platform")
Connection.new(token.token)
end
# Fetch the Object
defp storage_objects_get(connection, bucket, object, opts \\ []) do
optional_params = %{
:alt => :query,
:fields => :query,
:key => :query,
:oauth_token => :query,
:prettyPrint => :query,
:quotaUser => :query,
:userIp => :query,
:generation => :query,
:ifGenerationMatch => :query,
:ifGenerationNotMatch => :query,
:ifMetagenerationMatch => :query,
:ifMetagenerationNotMatch => :query,
:projection => :query,
:userProject => :query
}
request = Request.new()
|> Request.method(:get)
|> Request.url("/storage/v1/b/{bucket}/o/{object}", %{
"bucket" => URI.encode_www_form(bucket),
"object" => URI.encode_www_form(object)
})
|> Request.add_optional_params(optional_params, opts)
connection
|> Connection.execute(request)
end
end
from elixir-google-api.
Thanks for filing this issue.
This issue is hard because all of these client libraries are generated code from API definitions. This alt media parameter is really a one-off feature that is not accurately described by any API definition format. It might be possible to add a global option to any API call like deserialize: false
that would skip deserializing the response and return the Tesla.Env
response struct.
from elixir-google-api.
Hi @chingor13,
I agree. I think that's a pretty neat solution.
Happy to put together a PR.
from elixir-google-api.
That would be awesome!
We've started adding tests for the templates in clients/test_client/tests
which run on a client generated from specifications/gdd/TestClient-v1.json
.
from elixir-google-api.
@chingor13 @Jesse-Cameron
hi,
what's the status of this issue?
FYI this behaviour @Jesse-Cameron described above is is only for .json files, in other cases - returns an error like this: {:error, {:invalid, "\\", 1}}
the problem in the decode function, i think that in case of [alt: "media"]
you didn't need to decode the body, (but also - never return the Tesla.Env
in the response), just return the body as is.
def decode(%Tesla.Env{} = env, struct, opts) do
decode(response_format(env), env, struct, opts)
end
def decode("media", %Tesla.Env{body: body}, _struct, _opts) do
{:ok, body}
end
def decode(_alt, %Tesla.Env{body: body}, struct, dataWrapped: true) do
Poison.decode(body, as: %GoogleApi.Storage.V1.RequestBuilder.DataWrapper{}, struct: struct)
|> case do
{:ok, %{data: data}} -> {:ok, data}
error -> error
end
end
def decode(_alt, %Tesla.Env{body: body}, struct, _opts) do
Poison.decode(body, as: struct)
end
defp response_format(%Tesla.Env{query: query}) do
query
|> Enum.into(%{})
|> Map.get(:alt)
end
from elixir-google-api.
Hi,
i'm trying to download a file from a bucket. when i call storage_objects_get
without
[{:alt, "media"}]
it gives me Model.Object
which contains mediaLink
, on opening that link it says Anonymous caller does not have storage.objects.get access to <uri>
calling GoogleApi.Storage.V1.Api.Objects.storage_objects_get(conn, string, string, [{:alt, "media"}])
gives {:error, {:invalid, "%", 0}}
.
I want to know how to download the file.
from elixir-google-api.
The ability to download a file from GCP storage seems like a core feature that should be supported by this library. Is there a timeline for adding this?
from elixir-google-api.
Thank you @okoriko
from elixir-google-api.
@chingor13 Is this work complete? Can this be closed?
from elixir-google-api.
Verified that this is working on {:google_api_storage, "~> 0.9.0"}
from elixir-google-api.
It seems all is missing here is a proper typespec of {:ok, Tesla.Env.t()}
. This will go, for now, to the list of ignored dialyzer error as there is nothing I can do to typecheck this now.
I'll open a new issue.
from elixir-google-api.
Related Issues (20)
- WigWamWoodys HOT 1
- The task "google_apis.auth" could not be found HOT 3
- elixir-google-api broke with latest gcloud pub/sub emulator 0.7.1
- Speech: support v1p1beta1 HOT 2
- The Hex Docs are not being built HOT 2
- STOP GENERATING THOUSANDS OF MODULES!!! HOT 7
- Recommend configuration of Tesla adapter in README
- Copying Object in GCS
- Retail version does not match the doc HOT 5
- google_api_play_integrity package is not available on hex.pm HOT 1
- Version 0.25.0 of `dfa_reporting` is missing on hex.pm
- Synthesis failed for CloudIot HOT 1
- Is this still being maintained? HOT 1
- Synthesis failed for IDS HOT 33
- Is the new Google Identity's API available in this library? HOT 1
- GoogleCloudMlV1_PredictRequest Mount Wrong URL HOT 1
- Empty API References Page for v0.16.1 release for Google API {Spanner, Notebooks, Manufacturers} HOT 1
- Synthesis failed for Transcoder HOT 10
- is batch uploads supported? HOT 1
- dfareporting v4 has been released HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from elixir-google-api.