Code Monkey home page Code Monkey logo

startpage-parser's Introduction

startpage-parser

parsing search results from startpage search engine (based on google.com results) you can use it if you need get some results without get banned from google

Untitled-Diagram

INSTALL

  • from pip
$ pip install startpage-parser
  • from source
$ git clone https://github.com/knassar702/startpage-parser
$ cd startpage-parser
$ pip install -e .

>>> from startpage import StartPage
>>> task = StartPage()
>>> task.search("Hello World",page=1) # page = number of pages (page=10 > from page number one to page number ten)
>>> # All results stored in .results 
>>> # print(task.results)
>>> # {'page number':"Results"}
>>> print(task.results)
{'1': [{'title': '"Hello, World!" program - Wikipedia', 'link': 'https://en.wikipedia.org/wiki/%22Hello,_World!%22_program', 'description': 'A "'}, {'title': 'Hello World (film) - Wikipedia', 'link': 'https://en.wikipedia.org/wiki/Hello_World_(film)', 'description': <b>Hello World</b>}, {'title': 'hello world - YouTube', 'link': 'https://www.youtube.com/watch?v=Yw6u6YkTgQ4', 'description': '30 Mar 2018 '}, {'title': 'Total immersion, Serious fun! with Hello-World!', 'link': 'https://www.hello-world.com/', 'description': 'Main index for '}, {'title': 'Hello, World! - Learn Python - Free Interactive Python Tutorial', 'link': 'https://www.learnpython.org/en/Hello,_World!', 'description': <b>Hello</b>}, {'title': 'C "Hello, World!" Program', 'link': 'https://www.programiz.com/c-programming/examples/print-sentence', 'description': 'In this example, you will learn to print "'}, {'title': 'C++ "Hello, World!" Program', 'link': 'https://www.programiz.com/cpp-programming/examples/print-sentence', 'description': 'In this example, we will learn to create a simple program named "'}, {'title': "The History of 'Hello, World' - HackerRank Blog", 'link': 'https://blog.hackerrank.com/the-history-of-hello-world/', 'description': '21 Apr 2015 '}, {'title': 'Hello World Studio', 'link': 'https://www.helloworldstudio.org/', 'description': <b>Hello World</b>}, {'title': 'Hello World - Go by Example', 'link': 'https://gobyexample.com/hello-world', 'description': 'To run the program, put the code in '}]}

examples

from startpage import StartPage

task = StartPage()
task.search("Hello World",page=1)
for page_num,results in task.results.items():
    print(f'PAGE: {page_num}\n---------------')
    for res in results:
        print(f'Title: {res["title"]}\n Description: {res["description"]}\n Link: {res["link"]}\n======')
"""
PAGE: 1
---------------
Title: "Hello, World!" program - Wikipedia
 Description: A "
 Link: https://en.wikipedia.org/wiki/%22Hello,_World!%22_program
======
Title: Hello World (film) - Wikipedia
 Description: <b>Hello World</b>
 Link: https://en.wikipedia.org/wiki/Hello_World_(film)
======
Title: hello world - YouTube
 Description: 30 Mar 2018 
 Link: https://www.youtube.com/watch?v=Yw6u6YkTgQ4
======
Title: Total immersion, Serious fun! with Hello-World!
 Description: Main index for 
 Link: https://www.hello-world.com/
======
Title: Hello, World! - Learn Python - Free Interactive Python Tutorial
 Description: <b>Hello</b>
 Link: https://www.learnpython.org/en/Hello,_World!
======
"""

2

3

startpage-parser's People

Contributors

knassar702 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

Forkers

newtux a1k-ghaz1

startpage-parser's Issues

No results after 4 or 5 search requests

Hey. Your script here is awesome and I like that is is really minimal. But sadly I can not use it for my current project.

I got no results after a 4 or 5th search request. The result is then:
[]

I could imagine that startpage.com detect the bot/script and give you a captcha page like in this bug here:
Garee/sp#12

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.