Strange Yahoo search data about referer-parser HOT 14 OPEN

Perhaps these two images explain it best:

Normal search terms:

Not provided / advertising?:

Here's the query:

SELECT
  Distinct refr_term,
  "count"(*)
FROM
  "atomic".events
WHERE
  refr_urlhost = 'au.search.yahoo.com'
GROUP BY
  1
ORDER BY
  2 DESC

from referer-parser.

Right - sounds like we should add 'au.yahoo.com' to the unknown section at the top of the referers.yml, to reduce these search false positives (i.e. au.yahoo.com will be identified as unknown rather than search). Makes sense?

from referer-parser.

Sure. I am just a little hesitant because it looks like a pattern among many of the Yahoo domains in the yaml. Plus I don't have a huge amount of data to confirm this against what I'm seeing in my Snowplow install.

To name a few which also may be generating false positives:

  - yahoo.com
  - ar.yahoo.com
  - au.yahoo.com
  - br.yahoo.com
  - chinese.yahoo.com
  - de.yahoo.com
  - dk.yahoo.com
  - es.yahoo.com

from referer-parser.

Hey @kingo55 - @fblundun is back working on this library at the moment. Shall we add those 8 domains you list into our unknown section at the top to prevent false positives?

from referer-parser.

Ah awesome. Mind if I run a quick test first?

I want to make sure we're not going to mislabel genuine search traffic.
E.g. Secure searches where the keyword isn't provided.

PS. Massively excited to see all the other improvements coming!
On 26 Jun 2014 22:42, "Alexander Dean" [email protected] wrote:

Hey @kingo55 https://github.com/kingo55 - @fblundun
https://github.com/fblundun is back working on this library at the
moment. Shall we add those 8 domains you list into our unknown section at
the top to prevent false positives?

—
Reply to this email directly or view it on GitHub
#57 (comment)
.

from referer-parser.

No probs - go ahead Rob!

from referer-parser.

Ok, I can't find a way to produce referrers with au.yahoo.com, but Yahoo's
secure search appears to pass a referrer on with the visitor. You may want
to decide if you want to keep that particular referrer in.

Just second guessing since I notice Google Analytics includes that traffic
under search traffic, too.

On 26 Jun 2014 23:28, "Alexander Dean" [email protected] wrote:

No probs - go ahead Rob!

—
Reply to this email directly or view it on GitHub.

from referer-parser.

Yahoo search referrals always seem to come under r.search.yahoo.com/__ylt=...

On 27 Jun 2014 00:02, "Rob Kingston" [email protected] wrote:

Ok, I can't find a way to produce referrers with au.yahoo.com, but
Yahoo's secure search appears to pass a referrer on with the visitor. You
may want to decide if you want to keep that particular referrer in.

Just second guessing since I notice Google Analytics includes that traffic
under search traffic, too.

On 26 Jun 2014 23:28, "Alexander Dean" [email protected] wrote:

No probs - go ahead Rob!

—
Reply to this email directly or view it on GitHub.

from referer-parser.

Hi @kingo55 ,
Just asking to clarify: Do you think that those eight domains should all go into the unknown section? Or do you think that some of them might be genuine "search" referers?

from referer-parser.

Hi @fblunden

Which domains are you referring to?
On 28 Jun 2014 00:14, "Fred Blundun" [email protected] wrote:

Reopened #57 #57.

—
Reply to this email directly or view it on GitHub
#57 (comment).

from referer-parser.

Your list from April:

  - yahoo.com
  - ar.yahoo.com
  - au.yahoo.com
  - br.yahoo.com
  - chinese.yahoo.com
  - de.yahoo.com
  - dk.yahoo.com
  - es.yahoo.com

from referer-parser.

Ah yes I was looking at the wrong PR.

Up to you guys how you want to treat those referrers. My findings indicated
they should be OK to add as unknown. I was skeptical that they might mask
Yahoo secure searches, but I don't think so anymore.
On 28 Jun 2014 00:23, "Fred Blundun" [email protected] wrote:

Your list from April:

• yahoo.com
• ar.yahoo.com
• au.yahoo.com
• br.yahoo.com
• chinese.yahoo.com
• de.yahoo.com
• dk.yahoo.com
• es.yahoo.com

—
Reply to this email directly or view it on GitHub
#57 (comment)
.

from referer-parser.

Okay great - let's add them to the list of unknowns then Fred! Thanks Rob...

from referer-parser.

Hmm - there's a lot of other country domains in there. I'm nervous someone is going to not like us changing this. Suggest we pause and think about #19 instead

from referer-parser.