stummjr / flake8-scrapy Goto Github PK
View Code? Open in Web Editor NEWA Flake8 plugin to catch common issues on Scrapy spiders
License: MIT License
A Flake8 plugin to catch common issues on Scrapy spiders
License: MIT License
When starting the fakehaven stage in CI, I received the following error:
$ flakeheaven lint --format=grouped --exit-zero --import-order-style pep8 --application-import-names directories
multiprocessing.pool.RemoteTraceback:
"""
Traceback (most recent call last):
File "/usr/local/lib/python3.7/multiprocessing/pool.py", line 121, in worker
result = (True, func(*args, **kwds))
File "/usr/local/lib/python3.7/multiprocessing/pool.py", line 44, in mapstar
return list(map(*args))
File "/usr/local/lib/python3.7/site-packages/flake8/checker.py", line 687, in _run_checks
return checker.run_checks()
File "/usr/local/lib/python3.7/site-packages/flakeheaven/_patched/_checkers.py", line 282, in run_checks
return super().run_checks()
File "/usr/local/lib/python3.7/site-packages/flake8/checker.py", line 597, in run_checks
self.run_ast_checks()
File "/usr/local/lib/python3.7/site-packages/flake8/checker.py", line 500, in run_ast_checks
for (line_number, offset, text, _) in runner:
File "/usr/local/lib/python3.7/site-packages/flake8_scrapy.py", line 55, in run
finder.visit(self.tree)
File "/usr/local/lib/python3.7/ast.py", line 271, in visit
return visitor(node)
File "/usr/local/lib/python3.7/ast.py", line 279, in generic_visit
self.visit(item)
File "/usr/local/lib/python3.7/ast.py", line 271, in visit
return visitor(node)
File "/usr/local/lib/python3.7/ast.py", line 279, in generic_visit
self.visit(item)
File "/usr/local/lib/python3.7/ast.py", line 271, in visit
return visitor(node)
File "/usr/local/lib/python3.7/site-packages/flake8_scrapy.py", line 38, in visit_Assign
self.find_issues_visitor('Assign', node)
File "/usr/local/lib/python3.7/site-packages/flake8_scrapy.py", line 35, in find_issues_visitor
self.generic_visit(node)
File "/usr/local/lib/python3.7/ast.py", line 281, in generic_visit
self.visit(value)
File "/usr/local/lib/python3.7/ast.py", line 271, in visit
return visitor(node)
File "/usr/local/lib/python3.7/site-packages/flake8_scrapy.py", line 41, in visit_Call
self.find_issues_visitor('Call', node)
File "/usr/local/lib/python3.7/site-packages/flake8_scrapy.py", line 35, in find_issues_visitor
self.generic_visit(node)
File "/usr/local/lib/python3.7/ast.py", line 279, in generic_visit
self.visit(item)
File "/usr/local/lib/python3.7/ast.py", line 271, in visit
return visitor(node)
File "/usr/local/lib/python3.7/ast.py", line 281, in generic_visit
self.visit(value)
File "/usr/local/lib/python3.7/ast.py", line 271, in visit
return visitor(node)
File "/usr/local/lib/python3.7/site-packages/flake8_scrapy.py", line 41, in visit_Call
self.find_issues_visitor('Call', node)
File "/usr/local/lib/python3.7/site-packages/flake8_scrapy.py", line 34, in find_issues_visitor
self.issues.extend(list(issues))
File "/usr/local/lib/python3.7/site-packages/finders/oldstyle.py", line 18, in find_issues
if first_param.value.id == 'response' and first_param.attr == 'url':
AttributeError: 'Attribute' object has no attribute 'id'
"""
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/local/bin/flakeheaven", line 8, in <module>
sys.exit(entrypoint())
File "/usr/local/lib/python3.7/site-packages/flakeheaven/_cli.py", line 40, in entrypoint
exit_code, msg = main(argv)
File "/usr/local/lib/python3.7/site-packages/flakeheaven/_cli.py", line 32, in main
return COMMANDS[command_name](argv=argv[1:])
File "/usr/local/lib/python3.7/site-packages/flakeheaven/commands/_lint.py", line 12, in lint_command
app.run(argv)
File "/usr/local/lib/python3.7/site-packages/flake8/main/application.py", line 375, in run
self._run(argv)
File "/usr/local/lib/python3.7/site-packages/flake8/main/application.py", line 364, in _run
self.run_checks()
File "/usr/local/lib/python3.7/site-packages/flake8/main/application.py", line 271, in run_checks
self.file_checker_manager.run()
File "/usr/local/lib/python3.7/site-packages/flake8/checker.py", line 309, in run
self.run_parallel()
File "/usr/local/lib/python3.7/site-packages/flake8/checker.py", line 275, in run_parallel
for ret in pool_map:
File "/usr/local/lib/python3.7/multiprocessing/pool.py", line 354, in <genexpr>
return (item for chunk in result for item in chunk)
File "/usr/local/lib/python3.7/multiprocessing/pool.py", line 748, in next
raise value
AttributeError: 'Attribute' object has no attribute 'id'
The problem occurs in this line:
urljoin(settings.SERVICE_URLS.PD, '/path')
Where are the settings
:
from pydantic import BaseSettings, BaseModel
class ServiceUrlsSchema(BaseModel):
PD: str
class Settings(BaseSettings):
SERVICE_URLS: ServiceUrlsSchema
Capture the usage of:
response.css()[0].extract()
response.css().extract()[0]
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.