Code Monkey home page Code Monkey logo

Comments (14)

okoriko avatar okoriko commented on July 28, 2024 12

For anyone stumbling here:

  • update to the latest version (0.3 at the moment)
  • there is now a 5th undocumented argument (opts) which supports the options decode: false

Example:

{:ok, %Tesla.Env{body: body}} = Objects.storage_objects_get(conn, "my-bucket", "my-file.txt", [alt: "media"], decode: false)
IO.inspect(body, label: "file content")

from elixir-google-api.

jeffdeville avatar jeffdeville commented on July 28, 2024 1

Yeah, this is a pretty critical problem for me as well. GCS's usefulness is pretty limited if you can't pull the file. Here's a workaround for those who need it. It's just the same code as what's generated, it just skips the last decode command, and provides a friendlier wrapper around the whole thing.

defmodule Utils.GCS do
  alias GoogleApi.Storage.V1.Connection
  import GoogleApi.Storage.V1.RequestBuilder

  def upload_file(bucket_id, bucket_path, file_path) do
    # Authenticate.
    conn = get_conn()

    # Make the API request.
    GoogleApi.Storage.V1.Api.Objects.storage_objects_insert_simple(
      conn,
      bucket_id,
      "multipart",
      %{name: bucket_path},
      file_path
    )
  end

  def download_file(bucket_id, bucket_path) do
    conn = get_conn()

    case storage_objects_get(conn, bucket_id, bucket_path, alt: :media) do
      %Tesla.Env{body: <<"No such object:", _rest::binary>>} -> {:error, :file_not_found}
      %Tesla.Env{body: body} -> {:ok, body}
    end
  end

  defp get_conn() do
    {:ok, token} = Goth.Token.for_scope("https://www.googleapis.com/auth/cloud-platform")
    GoogleApi.Storage.V1.Connection.new(token.token)
  end

  defp storage_objects_get(connection, bucket, object, opts \\ []) do
    optional_params = %{
      :alt => :query,
      :fields => :query,
      :key => :query,
      :oauth_token => :query,
      :prettyPrint => :query,
      :quotaUser => :query,
      :userIp => :query,
      :generation => :query,
      :ifGenerationMatch => :query,
      :ifGenerationNotMatch => :query,
      :ifMetagenerationMatch => :query,
      :ifMetagenerationNotMatch => :query,
      :projection => :query,
      :userProject => :query
    }

    %{}
    |> method(:get)
    |> url("/storage/v1/b/{bucket}/o/{object}", %{
      "bucket" => URI.encode_www_form(bucket),
      "object" => URI.encode_www_form(object)
    })
    |> add_optional_params(optional_params, opts)
    |> Enum.into([])
    |> (&Connection.request(connection, &1)).()
  end
end

from elixir-google-api.

elbow-jason avatar elbow-jason commented on July 28, 2024 1

My stab at this issue:

defmodule Googly.Storage do
  def download_file(bucket, name) do
    url = "https://www.googleapis.com/download/storage/v1/b/#{bucket}/o/#{name}?alt=media"
    GoogleApi.Gax.Request.new()
    |> GoogleApi.Gax.Request.url(url)
    |> Googly.Storage.Client.execute()
  end
end

defmodule Googly.Storage.Client do
  alias GoogleApi.Storage.V1.{
    Connection,
  }

  def config(), do: Application.fetch_env!(:googly, Googly.Storage)

  def connection() do
    case Goth.Token.for_scope(config().scope) do
      {:ok, %{token: token}} -> Connection.new(token)
      err -> raise "Failed to acquire google storage token - #{inspect(err)}"
    end
  end

  def execute(req) do
    Connection.execute(connection(), req)
  end
end

from elixir-google-api.

IkechukwuAKalu avatar IkechukwuAKalu commented on July 28, 2024 1

Just incase anyone else has some issues using @jeffdeville 's solution, you could try the almost exact code below

defmodule Utils.GCS do
  alias GoogleApi.Gax.Request
  alias GoogleApi.Storage.V1.{Connection, Api}

  def upload_file(bucket_id, bucket_path, file_path) do
    conn = get_conn()
    Api.Objects.storage_objects_insert_simple(
      conn,
      bucket_id,
      "multipart",
      %{name: bucket_path},
      file_path
    )
  end

  def download_file(bucket_id, object_path) do
    conn = get_conn()
    case storage_objects_get(conn, bucket_id, object_path, alt: :media) do
      {:error, %Tesla.Env{body: <<"No such object:", _rest::binary>>}} -> {:error, :file_not_found}
      {:ok, %Tesla.Env{body: body}} -> {:ok, body}
    end
  end

  # For Authentication
  defp get_conn() do
    {:ok, token} = Goth.Token.for_scope("https://www.googleapis.com/auth/cloud-platform")
    Connection.new(token.token)
  end

  # Fetch the Object
  defp storage_objects_get(connection, bucket, object, opts \\ []) do
    optional_params = %{
      :alt => :query,
      :fields => :query,
      :key => :query,
      :oauth_token => :query,
      :prettyPrint => :query,
      :quotaUser => :query,
      :userIp => :query,
      :generation => :query,
      :ifGenerationMatch => :query,
      :ifGenerationNotMatch => :query,
      :ifMetagenerationMatch => :query,
      :ifMetagenerationNotMatch => :query,
      :projection => :query,
      :userProject => :query
    }

    request = Request.new()
    |> Request.method(:get)
    |> Request.url("/storage/v1/b/{bucket}/o/{object}", %{
      "bucket" => URI.encode_www_form(bucket),
      "object" => URI.encode_www_form(object)
    })
    |> Request.add_optional_params(optional_params, opts)

    connection
    |> Connection.execute(request)
  end
end

from elixir-google-api.

chingor13 avatar chingor13 commented on July 28, 2024

Thanks for filing this issue.

This issue is hard because all of these client libraries are generated code from API definitions. This alt media parameter is really a one-off feature that is not accurately described by any API definition format. It might be possible to add a global option to any API call like deserialize: false that would skip deserializing the response and return the Tesla.Env response struct.

from elixir-google-api.

Jesse-Cameron avatar Jesse-Cameron commented on July 28, 2024

Hi @chingor13,

I agree. I think that's a pretty neat solution.
Happy to put together a PR.

from elixir-google-api.

chingor13 avatar chingor13 commented on July 28, 2024

That would be awesome!

We've started adding tests for the templates in clients/test_client/tests which run on a client generated from specifications/gdd/TestClient-v1.json.

from elixir-google-api.

gVolop avatar gVolop commented on July 28, 2024

@chingor13 @Jesse-Cameron
hi,
what's the status of this issue?
FYI this behaviour @Jesse-Cameron described above is is only for .json files, in other cases - returns an error like this: {:error, {:invalid, "\\", 1}}
the problem in the decode function, i think that in case of [alt: "media"] you didn't need to decode the body, (but also - never return the Tesla.Env in the response), just return the body as is.


  def decode(%Tesla.Env{} = env, struct, opts) do
    decode(response_format(env), env, struct, opts)
  end

  def decode("media", %Tesla.Env{body: body}, _struct, _opts) do
    {:ok, body}
  end

  def decode(_alt, %Tesla.Env{body: body}, struct, dataWrapped: true) do
    Poison.decode(body, as: %GoogleApi.Storage.V1.RequestBuilder.DataWrapper{}, struct: struct)
    |> case do
      {:ok, %{data: data}} -> {:ok, data}
      error -> error
    end
  end

  def decode(_alt, %Tesla.Env{body: body}, struct, _opts) do
    Poison.decode(body, as: struct)
  end

  defp response_format(%Tesla.Env{query: query}) do
    query
    |> Enum.into(%{})
    |> Map.get(:alt)
  end

from elixir-google-api.

akumar1503 avatar akumar1503 commented on July 28, 2024

Hi,
i'm trying to download a file from a bucket. when i call storage_objects_get without
[{:alt, "media"}] it gives me Model.Object which contains mediaLink , on opening that link it says Anonymous caller does not have storage.objects.get access to <uri>
calling GoogleApi.Storage.V1.Api.Objects.storage_objects_get(conn, string, string, [{:alt, "media"}])
gives {:error, {:invalid, "%", 0}}.
I want to know how to download the file.

from elixir-google-api.

lpil avatar lpil commented on July 28, 2024

The ability to download a file from GCP storage seems like a core feature that should be supported by this library. Is there a timeline for adding this?

from elixir-google-api.

lpil avatar lpil commented on July 28, 2024

Thank you @okoriko

from elixir-google-api.

dazuma avatar dazuma commented on July 28, 2024

@chingor13 Is this work complete? Can this be closed?

from elixir-google-api.

tzumby avatar tzumby commented on July 28, 2024

Verified that this is working on {:google_api_storage, "~> 0.9.0"}

from elixir-google-api.

victorolinasc avatar victorolinasc commented on July 28, 2024

It seems all is missing here is a proper typespec of {:ok, Tesla.Env.t()}. This will go, for now, to the list of ignored dialyzer error as there is nothing I can do to typecheck this now.

I'll open a new issue.

from elixir-google-api.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.