tpitale / legato Goto Github PK
View Code? Open in Web Editor NEWGoogle Analytics Reporting API Client for Ruby
License: MIT License
Google Analytics Reporting API Client for Ruby
License: MIT License
I have a site that works fine for plenty of people - the user can authenticate with OAuth2, and Legato successfully displays data from their Google Analytics accounts.
But it's possible for someone to have a Google account, but no Analytics account. These users can authenticate successfully with Google - it asks if they should let my app "View your Google Analytics data". But when they return to my site I do:
user = Legato::User.new(access_token_obj)
which works fine, and then try to access user.accounts
or user.profiles
and I get OAuth2 errors:
OAuth2::Error - {"errors"=>[{"domain"=>"global", "reason"=>"insufficientPermissions", "message"=>"User does not have any Google Analytics account."}], "code"=>403, "message"=>"User does not have any Google Analytics account."}:
{"error":{"errors":[{"domain":"global","reason":"insufficientPermissions","message":"User does not have any Google Analytics account."}],"code":403,"message":"User does not have any Google Analytics account."}}:
I'm not clear how I should check whether the successfully authed user actually has an Analytics account before I try and do something with it. Have I missed something?
The big question: do we want Legato to support reading and writing from the Management API?
For example, WebProperties now support most CRUD operations, such as "insert": https://developers.google.com/analytics/devguides/config/mgmt/v3/mgmtReference/management/webproperties/insert
The list includes:
The alternative is to leave the management of these objects to the official Google gem.
I'm not sure if this is an issue/feature request or if I'm incorrectly using Legato. For a set of filters I want to filter keywords that match a set of regexes while excluding a set of keywords. I'm not sure how to accomplish that. Here's an example that will hopefully make it clear what I'm thinking:
Let's say I have an array of regular expressions and an array of excluded keywords:
regexes = ["REGEX1","REGEX2","REGEX3"]
keywords = ["KW1","KW2","KW3"]
I have my model class:
class Visits
extend Legato::Model
metrics :visits
filter :organic_searches, &lambda {matches(:medium, 'organic')}
filter :matching_regexes, &lambda { |regexes|
regexes.map { |regex| contains(:keyword, regex)}
}
filter :excluding, &lambda { |keywords|
keywords.map { |keyword| does_not_match(:keyword, keyword)}
}
end
Currently when I execute:
query = Visits.matching_regexes(regexes).excluding(keywords).organic_searches(profile).filters.to_params
I get:
`````` "ga:keyword=~REGEX1,ga:keyword=~REGEX2,ga:keyword=~REGEX3;ga:keyword!=KW1,ga:keyword!=KW2,ga:keyword!=KW3;ga:medium==organic"```
(the ga:keyword!= parameters are joined by OR (a comma))
What I'd like to be able to get:
"ga:keyword=~REGEX1,ga:keyword=~REGEX2,ga:keyword=~REGEX3;ga:keyword~=KW1;ga:keyword~=KW2;ga:keyword~=KW3;ga:medium==organic"
(the ga:keyword!= parameters are joined by AND (a semicolon))
Does this make sense?
I have a user asking that the requests to GA use a "HIGHER_PRECISION" sampling level query parameter. Without looking at the Legato docs, I added it to my query params and it returned data successfully. But then I looked further and it looked like it is not a supported param. Likelihood of supporting "samplingLevel"? Much appreciated.
See: https://developers.google.com/analytics/devguides/reporting/core/v3/reference#samplingLevel
I've been getting this error:
NoMethodError: undefined method `map' for nil:NilClass
from /app/vendor/bundle/ruby/1.9.1/gems/legato-0.0.7/lib/legato/response.rb:21:in `totals_for_all_results'
from /app/vendor/bundle/ruby/1.9.1/gems/legato-0.0.7/lib/legato/query.rb:119:in `load'
Line: https://github.com/tpitale/legato/blob/master/lib/legato/response.rb#L21
I think I've got an auth issue because it works locally but doesn't work when deployed. I think this error is masking the real problem though. Is there a way to do some error handling to see what the actual message is that google is returning?
Update: I am able to fetch profiles on a Legato::User so I'm not so sure it's an auth problem. Using the released 0.0.7 version.
Useful context:
Service accounts only work for Google Apps accounts. With a regular
@gmail.com
google account, you'll need an "Installed Application". See the google-oauth2-installed.
We just ran across this with one of our customers. They have more than 1000 GA profiles associated with their account. For this user the all
method in finder.rb
would only get the first 1000 profiles.
It's quite an edge case, potentially not worth fixing, but I thought you might want to know it exists in the wild.
When my query data range is a one year, it seems that google analytics returns my data in batches from different times throughout the year. For example,
{keyword="KEYWORD_A", visits="2", total_value="20192.75", revenue_per_transaction="20192.75", transactions="1", day="31", month="05"},
{keyword="KEYWORD_A", visits="1", total_value="5789.8", revenue_per_transaction="2894.9", transactions="2", day="12", month="07"}
Is there a way to prevent this? This a complete pain when consolidating because some values need to be summed and others averaged.
In the doc about filter, it says we can send a lambda as the second parameter. But if you do, you get a wrong number of arguments (2 for 1) error on the filter call.
So this is wrong:
filter :high_exits, lambda {gte(:exits, 2000)}
But those are right:
filter :high_exits, &lambda {gte(:exits, 2000)}
filter :high_exits, &-> {gte(:exits, 2000)}
filter :high_exits, &Proc.new {gte(:exits, 2000)}
filter :high_exits do
gte(:exits, 2000)
end
Is there any way to get the Account User Links information like
Doc Referred: https://developers.google.com/analytics/devguides/config/mgmt/v3/mgmtUserPermissions
https://gist.github.com/mjgiarlo/9961071
This gist shows that the google-api-client returns query results even for rows with 0 pageviews (for instance), and legato does not. It would be great if #results
could support an option for filling in these rows.
Hi,
I want to show the visits value on my web page. I have created a user object and also created a class which extents Legato::Model.
Here is the class
class DataExtender
extend Legato::Model
metrics :visitors, :pageviews, :visits
end
Fetching the results in following way
DataExtender.results(@Profile,:start_date => start_date, :end_date => yesterday_date)
giving the error like
NoMethodError: undefined method `user' for #Array:0xcd5a2e4
Now getting how can i fetch the metrics values
Thanks and Regards,
Anita Bharambe
Hi everyone,
I'm just writing because I couldn't find any docs on how to use Google Service accounts with Legato (I believe I need a service account since this is used for an automated reporting process), but I kind of hacked a working solution together so I thought I would share - maybe it would make a good wiki article.
The solution uses the google-api-client
gem to authorize a service account and get an access token which it then injects into the OAuth2 gem (this is a lot of unnecessary boilerplate code IMO and very hacky so please by all means if you see a way to improve it let me know!)
require 'google/api_client'
def build_legato(scope="https://www.googleapis.com/auth/analytics.readonly")
config = config("google")
client = Google::APIClient.new
key = Google::APIClient::PKCS12.load_key(config["private_key"], config["private_key_passphrase"])
service_account = Google::APIClient::JWTAsserter.new(config["email"], scope, key)
client.authorization = service_account.authorize
oauth_client = OAuth2::Client.new("", "", {
:authorize_url => 'https://accounts.google.com/o/oauth2/auth',
:token_url => 'https://accounts.google.com/o/oauth2/token'
})
token = OAuth2::AccessToken.new(oauth_client, client.authorization.access_token)
Legato::User.new(token)
end
the config("google")
call loads up the secure authorization data as a hash, I'll give you some dummy data to make it more clear whats going on
email: [email protected]
private_key: config/google-private-key.p12
private_key_passphrase: notasecret
So again this is very hacky and if you'd like to improve on this to make it simpler and not requiring the extra google-api-client
gem I'd love to hear about it!
:order, :limit, :offset, :start_date, :end_date
Possibly in a branch, separate gem. OAuth1 ruby gem has an annoyingly different API.
Add other methods to go back up the management chain for a given profile.
Given:
class Legato::SiteVisits
extend Legato::Model
metrics :visits
dimensions :date
end
I'm manually adding a dimension to a query, like so:
visits = SiteVisits.results(@client.profile, start_date: start_date, end_date: end_date)
visits.dimensions << :dimension1 if <some condition>
<add filters>
rs = visits.collection
I do this because I don't know until runtime which custom dimensions I'll need.
However, when I later create another query:
query = SiteVisits.results(@client.profile, start_date: start_date, end_date: end_date)
query.dimensions
shows @elements=[:date, :dimension1]
I would not expect :dimension1
to be in the dimensions collection.
Not sure if this is a bug, but it's unexpected behaviour.
Any suggestions for fixing or avoiding this would be appreciated. Thanks
rake oauth2:token generates an invalid grant token
invalid_grant:
{
"error" : "invalid_grant"
}
/Users/jonathan.phillips/.rvm/gems/ruby-1.9.3-p194@legato/gems/oauth2-0.8.0/lib/oauth2/client.rb:108:in request' /Users/jonathan.phillips/.rvm/gems/ruby-1.9.3-p194@legato/gems/oauth2-0.8.0/lib/oauth2/client.rb:131:in
get_token'
/Users/jonathan.phillips/.rvm/gems/ruby-1.9.3-p194@legato/gems/oauth2-0.8.0/lib/oauth2/strategy/auth_code.rb:29:in get_token' /Volumes/work/legato/Rakefile:35:in
block (2 levels) in <top (required)>'
/Users/jonathan.phillips/.rvm/gems/ruby-1.9.3-p194@legato/bin/ruby_noexec_wrapper:14:in eval' /Users/jonathan.phillips/.rvm/gems/ruby-1.9.3-p194@legato/bin/ruby_noexec_wrapper:14:in
If anyone wants it.
For example, \W will be quadruple escaped.
/^(.\W+)?(open)\W+(site)(\W+.)?$/
I'm using this gem to fetch my sites analytic's, every thing is working fine. Now I need to fetch goal settings for my profile, can't figure out any inbuilt classes to do the same using Legato.
Please help me out in this
Are there other ways to select user profiles except first, second, etc?
Bit of a noob and in the quick start guide it just says .first
Say I had 10 profiles, could I select which one I wanted to use by using the account_id returned by the Legato::Management::Account.all(user)?
Thanks
As highlighted by #55, errors that come back from GA are often lost and other random errors (often oauth-related). We should capture the GA error, and present that in a helpful way.
Hi,
The wiki doesn't say anything about storing the access_token that you get in the end, nor does it suggest that a more systematic process of getting access tokens in the future is needed. Is this a one time thing? Do I just get the access_token supplied by the code= variable and save that and use it forever?
Thanks,
Adam
In v3 you can now request which fields you want in the response. This is very helpful when querying for aggregate or total values. Using fields can really cut down on the size of the response object when you only need a small piece of data.
source: http://code.google.com/apis/analytics/docs/gdata/v3/reference.html#fields
In the README, a test query is defined as profile.exits
- when you define the Exit
class, it adds a profile#exit
method. I think the class should be defined as Exits
, right? I successfully called profile.exits
when I created another class called Exits
.
(I can whip up a pull-request if I'm right here, I'm just mostly making sure I'm doing this right... thanks!)
In the process of implementing Legato in a rather large analytics-based project, I have often found that I have to refer to the Legato code to figure out how to do certain things. For example, until I read the code, I didn't realize that it was possible to dynamically adding metrics to a Legato model. I have come across a few such "Ah-ha" moments, and I feel like new users could really benefit from not having to discover them, themselves.
I would be willing to write up some additional documentation and submit it to whomever currently manages the project. Is this something the Legato community would be interested in? If everyone ends up disliking it, feel free to throw it away!
Additionally, I have a nice compilation of example code for common Google Analytics queries. It would be great to add some examples to this library. Is this something people are interested in?
Thanks!
Hey guys,
the following took me some time to figure out.
I tried doing:
Query.results(ss.profile, start_date: date, end_date: date).results(limit: 10000, offset: 0).each do |result| ...
which resulted in:
Encoding::CompatibilityError: incompatible character encodings: UTF-8 and ASCII-8BIT
from /home/vagrant/.rvm/gems/ruby-2.0.0-p247/gems/oauth2-0.8.1/lib/oauth2/error.rb:21:in `join'
Then after lots of playing around I tried:
Query.results(ss.profile, start_date: date, end_date: date).results(limit: 10000, offset: 1).each do |result| ...
Which finally gave back the expected results.
Perhaps this should be caught or at least made clear in the documentation, that the offset starts with 1 not 0 ?!
I'm trying to figure out how to use Legato and one of the only sources on the web to learn how to use it is the wiki or by hard trial an error.
The wiki is confusing.
For example https://github.com/tpitale/legato/wiki/Model-Data is using a Page model but in other examples is using Exit... everything is unconnected and confusing. The https://github.com/tpitale/legato/wiki/Quick-Start is also mixing command line irb with a rails model... there should be a quick start guide with every step needed to have a Rails app retrieving data from GA.
I made it work, but it took me two hours... The barrier of entry should be much lower
I came across a problem while chaining filters:
Using either of the following 2 filters yields the expected result.
Using both filters at the same time results in
NoMethodError - undefined method `map' for nil:NilClass
/gems/legato-0.0.10/lib/legato/response.rb:21:in `totals_for_all_results'
Extending legato/lib/user.rb as suggested in Issue #18 with
puts raw_response
gives me the following hint at whats going on:
Net::HTTPUnauthorized 401 Unauthorized readbody=true
Here the relevant code:
class Visitors
extend Legato::Model
metrics :visitors, :goal_1_completions
dimensions :pagePath
filter :for_vertical, &lambda { |vertical| contains(:pagePath, "/#{vertical}") }
filter :for_medium, &lambda { |medium| matches(:medium, medium) }
end
Visitors.for_vertical("foobar").results(profile, ...) # OK
Visitors.for_medium("organic").results(profile, ...) # OK
Visitors.for_vertical("foobar").for_medium("organic").results(profile, ...) # Error
the resulting query looks like this:
"ids"=>"ga:XXXX", "start-date"=>"2013-06-24", "end-date"=>"2013-07-01", "filters"=>"ga:pagePath=~/foobar;ga:medium==organic", "fields"=>"columnHeaders/name,rows,totalResults,totalsForAllResults", "metrics"=>"ga:visitors,ga:goal1Completions", "dimensions"=>"ga:pagePath"}
At first I thought I misunderstood how to chain filters, but the HTTPUnauthorized 401 got me thinking this might be a bug.
Any solutions are much appreciated as I'm working on a project for a client.
When we get an OAuth2::Error we often get a useful message and we also know what request we were making. We should be able to improve the user experience by handling those errors and printing something helpful.
Hey tony,Is there any method to fetch filter information for a profile using legato gem in the same way we get the profile settings and goal information.By filter information I mean the following:
Given the correct configuration (key info), and the presence of the google-api-client (optional), we can build an oauth_token to use in for auth. This is currently documented as a process in the wiki. I think this is useful enough that we should make it easy to use this type of account.
I just discovered this repo as a recommendation from someone as the best maintained Google Analytics API gem at the moment, but it doesn't rank that well in searches for "Google analytics ruby" on Google or GitHub.
Changing "GA" to "Google Analytics" will make a big difference for the GitHub search and eventually on Google as well so more people and find and enjoy this lovely project!
After issue #40 was raised, I remembered that Garb allowed passing of metrics/dimensions options to the results method. We could reintroduce this feature.
Or, we could have a more explicit API on the query class to add_dimension(s) or add_metric(s) โฆ
NoMethodError: undefined method `code' for #<URI::InvalidURIError:0x1106a9080>
from /Users/tpitale/.rvm/gems/ree-1.8.7-2011.12@merchant_center/gems/legato-0.0.2/lib/legato/user.rb:24:in `request'
from /Users/tpitale/.rvm/gems/ree-1.8.7-2011.12@merchant_center/gems/legato-0.0.2/lib/legato/query.rb:185:in `request_for_query'
from /Users/tpitale/.rvm/gems/ree-1.8.7-2011.12@merchant_center/gems/legato-0.0.2/lib/legato/query.rb:111:in `load'
from /Users/tpitale/.rvm/gems/ree-1.8.7-2011.12@merchant_center/gems/legato-0.0.2/lib/legato/query.rb:116:in `to_a'
from (irb):14
I haven't been able to find a way to pass in a reverse sort
results(start_date: 7.days.ago, end_date: Time.now, sort: :pageviews, limit: 6)
to_params yields "sort"=>"ga:pageviews" which to reverse sort, you'd need "-ga:pageviews".
Is there a way to pass in the negative sign that I've overlooked?
Can we make use of the super proxy as part of Legato?
https://developers.google.com/analytics/solutions/google-analytics-super-proxy
Hey there,
is it possible to request infos from GA without using legato models?
The reason I'm asking is this: I have two classes that just define different dimensions while having the same filters. That is somehow crappy and not very modular. Next thing I have is having to select :pathpath.
class StatisticsService::Country
extend Legato::Model
dimensions :country
metrics :pageviews
filter :all, &lambda{|button_unique_code| contains(:page_path, "#{button_unique_code}")}
filter :button_click, &lambda {|button_unique_code| contains(:page_path, "^/pay/(index.php)?.id=#{button_unique_code}$")}
filter :button_view, &lambda {|button_unique_code| contains(:page_path, "^/dlbutton...php.id=#{button_unique_code}$")}
filter :button_download, &lambda {|button_unique_code| contains(:page_path, "^/pay/([a-z]*)/download.php.id=#{button_unique_code}$")}
end
class StatisticsService::Day
extend Legato::Model
dimensions :date
metrics :pageviews
filter :all, &lambda{|button_unique_code| contains(:page_path, "#{button_unique_code}")}
filter :button_click, &lambda {|button_unique_code| contains(:page_path, "^/pay/(index.php)?.id=#{button_unique_code}$")}
filter :button_view, &lambda {|button_unique_code| contains(:page_path, "^/dlbutton...php.id=#{button_unique_code}$")}
filter :button_download, &lambda {|button_unique_code| contains(:page_path, "^/pay/([a-z]*)/download.php.id=#{button_unique_code}$")}
end
Perhaps I'm also not doing this right.
Any advise is appreciated!
Thx for the work on this gem!
I've read the docs on Getting an OAuth2 Token but that only seems appropriate when your app needs to authenticate for the current user. In my case I just need to access my own analytics.
Twitter describes this: https://dev.twitter.com/docs/auth/oauth/single-user-with-examples
Is there a way to do this with legato, or else use Google's Simple API Access with an API key?
Calling each on Legato::Query causes an OAuth2::Error with "message"=>"Quota Error: profileId ga:123456 has too many concurrent connections."
Partial stack trace:
oauth2 (0.6.0) lib/oauth2/client.rb:108:in request' oauth2 (0.6.0) lib/oauth2/access_token.rb:98:in
request'
oauth2 (0.6.0) lib/oauth2/access_token.rb:105:in get' legato (0.0.7) lib/legato/user.rb:20:in
request'
legato (0.0.7) lib/legato/query.rb:206:in request_for_query' legato (0.0.7) lib/legato/query.rb:116:in
load'
legato (0.0.7) lib/legato/query.rb:124:in collection' legato (0.0.7) lib/legato/query.rb:140:in
each'
lib/analytics/ga_helper.rb:28:in `listings_pageviews_by_date'
Example Code:
class Exit
extend Legato::Model
metrics :exits
dimensions :page_path, :browser
filter :for_browser, &lambda {|browser| matches(:browser, browser)}
end
exits_from_safari = Exit.results(@Profile,
{
:start_date => Date.parse("20120601"),
:end_date => Date.parse("20120619"),
}).for_browser("Safari")
exits_from_safari.each do |exit|
puts exit
end
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.