site search broken? about googler HOT 11 CLOSED

On second thoughts, I am almost convinced we shouldn't show these People who ask results from other sites in our results.

from googler.

This is not a regression. Here's what you get from 2.3:

> /usr/local/bin/googler -c in -n 4 -w imdb.com mr robot
 1  Mr. Robot (TV Series 2015– ) - IMDb
http://www.imdb.com/title/tt4158110/
... who suffers from social anxiety disorder and forms connections through hacking. He's recruited by a mysterious
anarchist, who calls himself Mr. Robot.

 2  Elliot played by Rami Malek | Characters & Crew | Mr
http://www.usanetwork.com/mrrobot/cast/elliot
GET DISCOVERED. Enhance your IMDb Page. Go to IMDbPro ». |; Help · Login · Register · Login · Mr. Robot (TV Series
2015– ) Poster · Mr. Robot (2015– ) ...

 3  Mr. Robot (TV Series 2015– ) - Full Cast & Crew - IMDb
http://www.imdb.com/title/tt4158110/fullcredits/
Mr. Robot (TV Series 2015– ) cast and crew credits, including actors, actresses, directors, writers and more.

 4  "Mr. Robot" Episode #2.1 (TV Episode 2016) - IMDb
http://www.imdb.com/title/tt4901088/
Directed by Sam Esmail. With Rami Malek, Christian Slater, Carly Chaikin, Grace Gummer.

Enter n, p, result number or new keywords (? for help)

Note that it is showing the "People also ask" results as 2, 3, 4, then randomly got cut off, seemingly due to half-brokenness of the legacy parser logic.

from googler.

In this case these results are harder to cut since they satisfy all the constraints we are currently imposing on valid results. What I'll do is to implement a generic ignore feature to blacklist certain tag/class combinations, e.g. div.related-question-pair in this case. Then it'll be easy to ignore other things we don't want to show on a case-by-case basis with a simple addition to a ignore list.

CSS selectors would be ideal for this purpose, but we don't have access to external libraries, so there's only so much I can do without bloat.

from googler.

implement a generic ignore feature to blacklist certain tag/class combinations

Works for me. No external libs please. The current imports take 0.16 seconds... approximately 16% of a resultset fetch time on my home network. Plus there are other regular complexities of additional dependencies.

from googler.

Yeah, no plan to use external libs.

The current imports take 0.16 seconds...

It takes 96ms here (averaged over 100 runs). Anyway, very typical for Python.

from googler.

very typical for Python

True. I've removed 2 imports as conditional. That's the best we can do for now.

from googler.

I've removed 2 imports as conditional.

It does help a tiny bit, but it's considered bad practice per PEP 8. I personally avoid that style as much as possible.

from googler.

it's considered bad practice per PEP 8

We'll live with it for debug and json, which don't come in our regular workflow.

from googler.

I'm okay with it. Just pointing out that I avoid it personally.

from googler.

okies. i'm gonna sleep now. have a great day! 👯

from googler.

Good night. You'll have a PR by the time you wake up.

from googler.