Comments (3)
aloha. @jpt-c
OK, in the scrapy shell when fetching the url, im getting the below error.
(i know the url looks weird. the website uses an iframe of agenda pdfs stored in a google drive folder, and the url is the iframe url)
>>> fetch('https://drive.google.com/embeddedfolderview?id=1BlNtMxhYJTbrjeqGQUkZhj_8RtqrX1eI#list')
2022-09-28 23:24:57 [scrapy.downloadermiddlewares.robotstxt] DEBUG: Forbidden by robots.txt: <GET https://drive.google.com/embeddedfolderview?id=1BlNtMxhYJTbrjeqGQUkZhj_8RtqrX1eI#list>
>>>
from city-scrapers-fresno.
Hey, just replied on Slack too :)
Don't try to scrape Google Drive, easy way to get blocked by Google altogether which is no fun.
If that's the source for their agenda we'll need people to handle this in a different way. Let's leave this one be.
from city-scrapers-fresno.
using google drive to store their agendas instead of displaying the agendas directly on their website. canβt scrape google drive (they block very fast, and it could affect your home use of google).
from city-scrapers-fresno.
Related Issues (20)
- New Scraper: Avenal City Council
- New Scraper: San Joaquin Valley Air Pollution Control District
- New Scraper: Friant Water Authority HOT 4
- New Scraper: Fresno Irrigation District HOT 1
- New Scraper: San Joaquin River Conservancy HOT 2
- New Scraper: Madera Irrigation District
- New Scraper: Westlands Water District HOT 1
- New Scraper: Fresno Parks and Arts Commission HOT 1
- New Scraper: Fresno Bicycle and Pedestrian Advisory Committee
- New Scraper: Fresno City/County Housing Authority
- New Scraper: City of Fresno
- New Scraper: City of Fresno
- New Scraper: City of Fresno
- New Scraper: City of Fresno
- New Scraper: City of Fresno
- New Scraper: City of Fresno
- New Scraper: City of Fresno
- New Scraper: Fresno County Planning Commission
- New Scraper: Measure C Committee
- Spider Revision: San Joaquin River Conservancy
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
π Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. πππ
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google β€οΈ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from city-scrapers-fresno.