A simple scraper and NPM module to get all article links from Aftonbladet, currently 900k+ articles and counting!
No support is given for Node versions under v8.0.0!
npm i --save aftonbladet-links
This should be pretty straight forward, but I included an example that dumps all article links to a json file. :) Click here!
DEBUG=* && npm i --dev && npm test
Will get all URLs to children sitemaps (one for each month) from parent (main)
Parameters:
None :)
The array contains all child sitemap URLs!
Will get all URLs for articles (in code refered as a "link") from children sitemap
Parameters:
url
string The child sitemap URL you want to get article links from :)
The array contains all article URLs!
Will get all URLs for articles (in code refered as a "link") for all articles on the site!
Parameters:
limit
string How many concurrent requests to use! (optional, default5
)
The array contains all article URLs!