Code Monkey home page Code Monkey logo

pys2's People

Contributors

mirandrom avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

pys2's Issues

Don't scrape all of a paper's neighbours by default when adding it to the S2Graph

if self.hopper.hop(gpath, self.graph):

Currently, adding a paper to the S2Graph implies scraping and adding all of its neighbours as well. This choice was made to allow dynamic graph exploration (with e.g. a reinforcement learning based GraphHopper) that requires information about paper (and thus scraping it) before hopping to it.

However, for simple rule-based GraphHoppers, this can add a lot of undesirable overhead when papers have large amounts of citations/references. I think the best solution would be to let S2DataStore objects lazily query the API when a paper/author is not locally cached. That way if the GraphHopper doesn't need scraped paper information (e.g. if the decision is based only on the edge type), then API calls are avoided.

add function for construction citation graph of a paper

Is your feature request related to a problem? Please describe.
Constructing a paper's citation graph is a common use case for the S2 API.

Describe the solution you'd like
A function for creating the citation graph of a given paperId, with a specified traversal depth in the direction of references and of citations. Include intermediate data structures to allow recovery in case of interruption (i.e. deque of unscraped paperId's; set of scraped paperId's; set of papers with errors; set of papers not found). Decide on final data structure of citation graph, how its stored/saved, and possibly an html visualization.

Describe alternatives you've considered
NA

Additional context
NA

add logging

Is your feature request related to a problem? Please describe.
Currently, if a response contains an error status code, information about the paperId/authorId is printed. This doesn't work well when trying to perform large amounts of requests over an extended period of time.

Describe the solution you'd like
Use logging

Describe alternatives you've considered
NA

Additional context
NA

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.