useful python scarping script
- This script scrapes infomation of all papers in one topic
- The information contains: [title, abstract, authors, journal, date, universities]
problem ==>
- after a time, the web refuse the request
- Scraping all relevant information on ssrn papers in the Financial Economic Network
- there are 200,000 urls, we use the Scraper API, as it is more stable
- Scarper API: https://www.scraperapi.com/documentation/