hadialqattan / pycln Goto Github PK

Description
When checking the "project".pth file in virtual environment pycln is appending /init.py to the filename instead of the contents.

To Reproduce

1. Run:

$ pycln project_name/

within poetry created virtual environment that exists in project directory

2. Error traceback:

/Users/user/Documents/repos/repo/.venv/lib/python3.7/site-packages/repo.pth/__init__.py Permission denied [READ] [Errno 13] ⛔

3. I believe this is happening because contents of repo.pth should be used:
   /Users/user/Documents/repos/repo

Expected behavior:

/Users/user/Documents/repos/repo/__init__.py
should be opened for reading not
/Users/user/Documents/repos/repo/.venv/lib/python3.7/site-packages/repo.pth/__init__.py

Environment:

• Python Version: 3.7
• Pycln Version: 1.2.5
• OS Type: MacOS

Additional context
I changed the username in path and project name for confidentiality reasons. If this is happening because of my misunderstanding of pycln usage please let me know. Thank you for your understanding.

I don't know how python's modules work too well, but it seems that the 'package' can be None, which breaks things.

To Reproduce:

1. Create a directory with the following files:

src/
    utils/
        __init__.py
        test.txt
    __init__.py
    code.py

Where the __init__.py file in the utils directory contains the following code:

from pathlib import Path

text_file = Path(__file__).parent / "blahblah.txt"

the text.txt file can contain any text,

the __init__.py file in the src/ directory contains the following code:

from . import utils

and finally, code contains:

from utils import text_file

print(text_file)

2. Run this Pycln command:

   $ pycln .

3. Error traceback:

Traceback (most recent call last):
  File "C:\Users\john\AppData\Local\Programs\Python\Python36\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "C:\Users\john\AppData\Local\Programs\Python\Python36\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\Users\john\code\test_pycln\.venv\Scripts\pycln.exe\__main__.py", line 9, in <module>
  File "c:\users\john\code\test_pycln\.venv\lib\site-packages\typer\main.py", line 214, in __call__
    return get_command(self)(*args, **kwargs)
  File "c:\users\john\code\test_pycln\.venv\lib\site-packages\click\core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "c:\users\john\code\test_pycln\.venv\lib\site-packages\click\core.py", line 782, in main
    rv = self.invoke(ctx)
  File "c:\users\john\code\test_pycln\.venv\lib\site-packages\click\core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "c:\users\john\code\test_pycln\.venv\lib\site-packages\click\core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "c:\users\john\code\test_pycln\.venv\lib\site-packages\typer\main.py", line 497, in wrapper
    return callback(**use_params)  # type: ignore
  File "c:\users\john\code\test_pycln\.venv\lib\site-packages\pycln\cli.py", line 155, in main
    session_maker.session(source)
  File "c:\users\john\code\test_pycln\.venv\lib\site-packages\pycln\utils\refactor.py", line 113, in session
    fixed_lines = self._code_session(content).splitlines(True)
  File "c:\users\john\code\test_pycln\.venv\lib\site-packages\pycln\utils\refactor.py", line 144, in _code_session
    return self._refactor(original_lines)
  File "c:\users\john\code\test_pycln\.venv\lib\site-packages\pycln\utils\refactor.py", line 214, in _refactor
    used_names = self._get_used_names(node, is_star)
  File "c:\users\john\code\test_pycln\.venv\lib\site-packages\pycln\utils\refactor.py", line 255, in _get_used_names
    node, alias, is_star
  File "c:\users\john\code\test_pycln\.venv\lib\site-packages\pycln\utils\refactor.py", line 351, in _should_remove
    if self._has_side_effects(alias.name, node) in (
  File "c:\users\john\code\test_pycln\.venv\lib\site-packages\pycln\utils\refactor.py", line 393, in _has_side_effects
    self._path, module, node.module, node.level
  File "c:\users\john\code\test_pycln\.venv\lib\site-packages\pycln\utils\pathu.py", line 335, in get_import_from_path
    mpath = get_local_import_from_path(path, module, package, level)
  File "c:\users\john\code\test_pycln\.venv\lib\site-packages\pycln\utils\pathu.py", line 274, in get_local_import_from_path
    if os.path.isfile(mpath) and package.split(".")[0] in mpath:
AttributeError: 'NoneType' object has no attribute 'split'

Environment:

• Python Version: 3.6.8
• Pycln Version: 1.0.3
• OS Type: Windows

Let me know if there is anything more I can do to help.

Describe the issue

There's an info I'd like to find in the docs, and I wasn't able to.
The documentation (and README) specify that the tool can run on Python 3.6 to 3.10. What it doesn't say is if it is also able to operate on code that is running with 3.6 to 3.10, and more importantly, if the version the tool runs on is required to be the same version as the version the codebase we use the tool on.

In other words, are there places in the code where the tool needs to figure out if it's working on a py3.6 or a py3.10 python file, and it uses sys.version to know ?

Other linting tools such as black and isort have a --python-version (or equivalent) flag to disambiguate. On the other hand, flake8 has a note in the docs saying it must run on the same version it operates on.

When using pre-commit, I want to know if I need to put language_version: python3.x or if I can just let it select the most recent python version.

Describe the bug A clear and concise description of what the bug is.
Pycln is failing due to a missing dependency from Typer

To Reproduce Steps to reproduce the behavior:

1. Take this code snippet:

  - repo: https://github.com/hadialqattan/pycln
    rev: v1.0.3 # Possible releases: https://github.com/hadialqattan/pycln/releases
    hooks:
    - id: pycln
      name: "Clean unused python imports"
      args: [--config=.config/yadm/pyproject.toml]
      files: '.*\.py'

2. Run this Pycln command:

   $ pre-commit run --all-files

3. Error traceback or unexpected output (if present):

pycln....................................................................Failed
- hook id: pycln
- exit code: 1

Traceback (most recent call last):
  File "/Users/adam/.cache/pre-commit/repoi53mrfw6/py_env-python3/bin/pycln", line 5, in <module>
    from pycln.cli import app
  File "/Users/adam/.cache/pre-commit/repoi53mrfw6/py_env-python3/lib/python3.9/site-packages/pycln/__init__.py", line 7, in <module>
    import typer
  File "/Users/adam/.cache/pre-commit/repoi53mrfw6/py_env-python3/lib/python3.9/site-packages/typer/__init__.py", line 29, in <module>
    from .main import Typer as Typer
  File "/Users/adam/.cache/pre-commit/repoi53mrfw6/py_env-python3/lib/python3.9/site-packages/typer/main.py", line 11, in <module>
    from .completion import get_completion_inspect_parameters
  File "/Users/adam/.cache/pre-commit/repoi53mrfw6/py_env-python3/lib/python3.9/site-packages/typer/completion.py", line 10, in <module>
    import click._bashcomplete
ModuleNotFoundError: No module named 'click._bashcomplete'

Environment (please complete the following informations):

• Python Version: 3.9
• Pycln Version: 1.0.3
• OS Type: macOS

Additional context Add any other context about the problem here.

Where do I report a security bug?

C modules import star expanding related vulnerability.

Environment:

• Python Version: Python 3.8 (affects all versions).
• Pycln Version: v0.0.1-alpha.2
• OS Type: Linux (affects all platforms)

Version 1.3.0 is broken please upgrade to 1.3.1 ...

Wait what?!
I've added a new directory outside the pycln/ directory called vendor/, and I forgot to include it in the final distribution.

Then what?
Pycln v1.3.0 worked properly (locally) because the vendor/ directory exists on my machine.

Aha, tell more!
As soon as I had published v1.3.0 I realized that the vendor/ directory wasn't included in the final distribution.

What does that mean?
Whenever I run Pycln v1.3.0, which was installed via pip, it crashes with a No module named 'vendor' error making v1.3.0 unusable!

At the end of this kinda sad story, I apologize for releasing a broken release!

Fix: #26, #27, #32.

Describe the bug
The flags --check and --diff should work together, returning the correct exit code like with --check alone, and the diff output like with --diff.
This would replicate the behavior of tools like black and isort where both flags can be combined as expected. An application for this would be a pre-commit hook so that commits with unnecessary imports can be declined and the pycln diff explains why.

To Reproduce

1. Take this code snippet:

import math

2. Run this Pycln command:

$ pycln --check --diff

3. unexpected output (if present):

All done! 💪 😎
1 import would be removed, 1 file would be changed.
$ echo $?
1

Expected behavior:

1. Description: Shows diff and returns 1 as exit code
2. Expected output (if present):

--- original/ test.py
+++ fixed/ test.py
@@ -1 +0,0 @@
-import math



All done! 💪 😎
1 import would be removed, 1 file would be changed.
$ echo $?
1

Environment (please complete the following informations):

• Python Version: 3.10.5
• Pycln Version: 1.3.5
• OS Type: Arch Linux

Is your feature request related to a problem? Please describe.

I'm always frustrated when I find that after running this tool, the line break are all replaced by CRLF on my windows PC.

Describe the solution you'd like

Keep the original format of line break.

Describe the bug A clear and concise description of what the bug is.

Pycln doesn't remove two unused imports

To Reproduce Steps to reproduce the behavior:

1. Take this code snippet:

    import nanoid
    from django.core.validators import MaxValueValidator, MinValueValidator
    from django.db.models import (
        SET_NULL,
        CharField,
        DurationField,
        FileField,
        ForeignKey,
        TimeField,
        UUIDField,
    )

   In the above, MaxValueValidator, MinValueValidator, and TimeField are all unused.

2. Run this Pycln command:

   $ pycln that_file.py

3. Error traceback or unexpected output (if present):

    Looks good! ✨ 🍰 ✨
    1 file left unchanged.

Expected behavior:

I expect the unused imports to be removed.

Environment (please complete the following informations):

• Python Version: 3.9.10
• Pycln Version: 1.1.0
• OS Type: macOS

Assume we have a directory called potato containing a critical code that we don't want any QA tool like pycln to touch it, then the only option is to use --exclude, which overrides the default EXCLUDE_REGEX.

What if we want to add this exclusion without overriding the default exclusions?

For now, the only way to do it is by rewriting the entire exclude regex with the additional exclusion(s):

pycln . --exclude "(\.eggs|\.git|\.hg|\.mypy_cache|__pycache__|\.nox|\.tox|\.venv|\.svn|buck-out|build|dist|potato)/"

While after adding this feature, we could do something more elegant like this:

pycln . --extend-exclude "(potato)/"

Describe the bug pycln removes imported names that should be exported, when __all__ is not used.

To Reproduce Steps to reproduce the behavior:

1. Create a folder with the following structure:

project/
  main.py
  foo/
    __init__.py
    bar.py

and the following contents:

# foo/bar.py
Bar = 1

# foo/__init__.py
from .bar import Bar

# main.py
from foo import Bar

print(Bar)

2. Run this Pycln command:

   $ pycln --all --diff .

3. Unexpected fixed code (if present):

--- original/ foo/__init__.py
+++ fixed/ foo/__init__.py
@@ -1 +0,0 @@
-from .bar import Bar

Expected behavior:

1. Description: pycln should not remove the import from __init__.py because it is used by main.py.

Environment (please complete the following informations):

• Python Version: 3.9.7
• Pycln Version: v1.0.3
• OS Type: Linux

Additional context
I know that I can fix it by adding __all__ = ["Bar"] to __init__.py, so it is probably not supported to check other files if they import a name recursively.

If this bug shouldn't be fixed, could you atleast mention it somewhere that pycln scans each file individually and thus you need to use __all__ if you want to re-export an import.
Furthermore it might make sense to output a warning if an import is removed from a file named __init__.py and it doesn't contain an __all__ variable.

In docs/CONTRIBUTING.md line 119, the link of CHANGELOG.md isn't same as it's should be, it's just CHANGELOG rather than CHANGELOG.md, which redirect the user to undefined GitHub page.

Is your feature request related to a problem? Please describe.
No

Describe the solution you'd like
Would it be possible to add support for stub (*.pyi) files?

Describe alternatives you've considered

PS> pycln mystub.pyi --all"
No Python files are present to be cleaned. Nothing to do 😴

Additional context
N/A

Error on comment line.

To Reproduce Steps to reproduce the behavior:

1. Take this code snippet:

   def addPortalMessage(message, type='info', request=None):
       # Note: no docstring please, to avoid reflected XSS.
       # This might not be possible, but type="structure" below sounds dangerous,
       # although I find no support for it in code.
   
       # The arguments are:
       #     message:   a string, with the text message you want to show,
       #                or a HTML fragment (see type='structure' below)
       #     type:      optional, defaults to 'info'. The type determines how
       #                the message will be rendered, as it is used to select
       #                the CSS class for the message. Predefined types are:
       #                'info' - for informational messages
       #                'warning' - for warning messages
       #                'error' - for messages about restricted access or
       #                          errors.
   
       pass
   

   Code reduced from https://github.com/plone/Products.CMFPlone/blob/58352686898521dd9da24d0239fd9848d04a02c5/Products/CMFPlone/PloneTool.py#L538-L567

2. Run this Pycln command:

   $ pycln testfile.py

3. Error traceback or unexpected output (if present):

   pyctnfailure.py .:1:1 SyntaxError: invalid syntax "optional, defaults to 'info'. The type determines how" ⛔
   

4. Unexpected fixed code (if present):

   Does not apply

Expected behavior:

1. Description: A clear and concise description of what you expected to happen.

   No error, just pass and let file as is.

2. Expected output (if present):

   No error.

3. Expected fixed code (if present):

   Does not apply

Environment (please complete the following informations):

• Python Version: 3.9.5
• Pycln Version: 0.0.3
• OS Type: Linux

xxx should not be considered as unused:

from typing import cast
import xxx
import y_obj

z_obj = cast("xxx", y_obj)

Hints:

• related to visit_Call on pycln/utils/scan.py.

Describe the bug A clear and concise description of what the bug is.

Under certain circumstances, pycln removes pass statements, which can lead to syntax errors in code.
Seems similar to #99, however for try-except-finally.

To Reproduce Steps to reproduce the behavior:

1. Take this code snippet:

   import os
   
   def f():
       try:
           x = 1
           return
       finally:
           pass

2. Run this Pycln command:

   $ pycln test_file.py

3. Error traceback or unexpected output (if present):

4. Unexpected fixed code (if present):

   def f():
       try:
           x = 1
           return
       finally:

Expected behavior:

1. Description: A clear and concise description of what you expected to happen.

   no syntax errors

2. Expected output (if present):

3. Expected fixed code (if present):

   def f():
       try:
           x = 1
           return
       finally:
           pass

Environment (please complete the following informations):

• Python Version: Python 3.9
• Pycln Version: v1.3.3
• OS Type: Windows

Additional context Add any other context about the problem here.

Description: Running pycln with the -a flag crashes with SyntaxError when it has cleaned up a multiline import

To Reproduce:

1. Take this code snippet:
   pycln_test.py

# -*- coding: utf-8 -*-
from urllib.parse import quote, unquote, urlparse

from flask import Blueprint, Response, abort, flash, g, jsonify, redirect, render_template, request, session, url_for
from flask_wtf import FlaskForm
from wtforms import (FieldList, Form, FormField, HiddenField, PasswordField, RadioField, SelectField, StringField,
                     SubmitField, TextAreaField, validators)
from wtforms.fields.html5 import EmailField, TelField
from wtforms.validators import DataRequired, Email, InputRequired, Length, Optional, Required


used = [Blueprint, Response, abort, redirect, render_template, request, session, url_for,
        FlaskForm, FieldList, FormField, SelectField, StringField, SubmitField, TextAreaField,
        Required]

2. Run this Pycln command:

$ pycln -a pycln_test.py

3. Error traceback or unexpected output (if present):

  pycln_test.py:2:0 'from urllib.parse import quote' has removed! 🔮
  pycln_test.py:2:0 'from urllib.parse import unquote' has removed! 🔮
  pycln_test.py:2:0 'from urllib.parse import urlparse' has removed! 🔮
  pycln_test.py:4:0 'from flask import flash' has removed! 🔮
  pycln_test.py:4:0 'from flask import g' has removed! 🔮
  pycln_test.py:4:0 'from flask import jsonify' has removed! 🔮
  pycln_test.py:6:0 'from wtforms import Form' has removed! 🔮
  pycln_test.py:6:0 'from wtforms import HiddenField' has removed! 🔮
  pycln_test.py:6:0 'from wtforms import PasswordField' has removed! 🔮
  pycln_test.py:6:0 'from wtforms import RadioField' has removed! 🔮
  pycln_test.py:6:0 'from wtforms import validators' has removed! 🔮
  pycln_test.py:8:0 'from wtforms.fields.html5 import EmailField' has removed! 🔮
  pycln_test.py:8:0 'from wtforms.fields.html5 import TelField' has removed! 🔮
  pycln_test.py:9:0 'from wtforms.validators import DataRequired' has removed! 🔮
  pycln_test.py:9:0 'from wtforms.validators import Email' has removed! 🔮
  pycln_test.py:9:0 'from wtforms.validators import InputRequired' has removed! 🔮
  pycln_test.py:9:0 'from wtforms.validators import Length' has removed! 🔮
  pycln_test.py:9:0 'from wtforms.validators import Optional' has removed! 🔮
  pycln_test.py 18 imports has removed! 🚀
Traceback (most recent call last):
  File "/Users/mango/  File "/Users/mango/  File "/Users/mango/  File "/Users/mango/   sys.exit(app())
  File "/Users/mango/my_project/my_projectPassion/venv/lib/python3.8/site-packages/typer/main.py", line 214, in __call__
    return get_command(self)(*args, **kwargs)
  File "/Users/mango/my_project/my_projectPassion/venv/lib/python3.8/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/Users/mango/my_project/my_proj  File "/Users/mango/my_project/my_proj  File "/Users/mango/my_project/my_proj  Filerv  File "/Users/mango/my_project/my_proj  File "/Users/mango/my_project/my_proj  File ".8  File "/Users/mango/my_project/my_proj  File "/Users/mango/my_project/my_proj el  File "/Users/mango/my_project/my_proj  File "/Users/mango/my_project/my_proj  File "/th  File "/Users/mango/my_project/my_proj  File "/Users/mango/my_project/my_proj  *args, **kwargs)
  File "/Users/mango/my_project/my_projectPassion/venv/lib/python3.8/site-packages/typer/main.py", line 497, in wrapper
    return callback(**use_params)  # type: ignore
  File "/Users/mango/my_project/my_projectPassion/venv/lib/python3.8/site-packages/pyc  File "/Users/mango/my_project/my_projectPassion/venv/lib/pythrce)
  File "/Users/mango/my_project/my_projectPassion/venv/li  File "/Users/mango/my_project/my_projectPassioor  File "/Users/mango/my_project/my_projectPassion/venv/li  File "/Users/mango/my_project/my_projectPassioor  File "/Users/mrs  File "/Users/mango/my_project/netiga/lib/python3.8/site-packages/pycln/utils/refactor.py", line 165, in _output
    fixed_lines = Refactor.remove_useless_passes(fixed_lines)
  File "/Users/mango/my_project/my_projectPassion/venv/lib/python3.8/site-packages/pycln/utils/refactor.py", line 85, in remove_useless_passes
    tree = ast.parse("".join(source_lines))
  File "/usr/local/Cellar/[email protected]/3.8.6_1/Frameworks/Python.framework/Versions/3.8/lib/python3.8/ast.py", line 47, in parse
    return compile(source, filename, mode, flags,
  File "<unknown>", line 7
    from wtforms.validators import Required
    ^
SyntaxError: invalid syntax

4. Unexpected fixed code:

# -*- coding: utf-8 -*-
from urllib.parse import quote, unquote, urlparse

from flask import Blueprint, Response, abort, flash, g, jsonify, redirect, render_template, request, session, url_for
from flask_wtf import FlaskForm
from wtforms import FieldList, FormField, SelectField, StringField, SubmitField, TextAreaField
from wtforms.fields.html5 import EmailField, TelField
from wtforms.validators import DataRequired, Email, InputRequired, Length, Optional, Required


used = [Blueprint, Response, abort, redirect, render_template, request, session, url_for,
        FlaskForm, FieldList, FormField, SelectField, StringField, SubmitField, TextAreaField,
        Required]

Expected behavior:

1. Description: The could to run to completion without Exception
2. Expected output (if present):

  pycln_test.py:2:0 'from urllib.parse import quote' has removed! 🔮
  pycln_test.py:2:0 'from urllib.parse import unquote' has removed! 🔮
  pycln_test.py:2:0 'from urllib.parse import urlparse' has removed! 🔮
  pycln_test.py:4:0 'from flask import flash' has removed! 🔮
  pycln_test.py:4:0 'from flask import g' has removed! 🔮
  pycln_test.py:4:0 'from flask import jsonify' has removed! 🔮
  pycln_test.py:7:0 'from wtforms.fields.html5 import EmailField' has removed! 🔮
  pycln_test.py:7:0 'from wtforms.fields.html5 import TelField' has removed! 🔮
  pycln_test.py:8:0 'from wtforms.validators import DataRequired' has removed! 🔮
  pycln_test.py:8:0 'from wtforms.validators import Email' has removed! 🔮
  pycln_test.py:8:0 'from wtforms.validators import InputRequired' has removed! 🔮
  pycln_test.py:8:0 'from wtforms.validators import Length' has removed! 🔮
  pycln_test.py:8:0 'from wtforms.validators import Optional' has removed! 🔮
  pycln_test.py 13 imports has removed! 🚀

All done! 💪 😎
13 imports has removed, 1 file has changed.

3. Expected fixed code (if present):

# -*- coding: utf-8 -*-

from flask import Blueprint, Response, abort, redirect, render_template, request, session, url_for
from flask_wtf import FlaskForm
from wtforms import FieldList, FormField, SelectField, StringField, SubmitField, TextAreaField
from wtforms.validators import Required


used = [Blueprint, Response, abort, redirect, render_template, request, session, url_for,
        FlaskForm, FieldList, FormField, SelectField, StringField, SubmitField, TextAreaField,
        Required]

Environment:

• Python Version: Python 3.8.6
• Pycln Version: pycln, version 1.0.3
• OS Type: MacOs 11.6 (Big Sur)

When we import something with this type of syntax

from x import (
    c,
    a as b,
)

pre-commit doesnt catch it (and hence doesnt fix it). But it gets caught by the linter and fails

I would like pre-commit to do it before linter can fail

These imports should be considered as used:

import x, y
import i, j

__all__ = ["x", "y"] + ["i", "j"]

Is your feature request related to a problem? Please describe.

PEP 484 states

Modules and variables imported into the stub are not considered exported from the stub unless the import uses the import ... as ... form or the equivalent from ... import ... as ... form. (UPDATE: To clarify, the intention here is that only names imported using the form X as X will be exported, i.e. the name before and after as must be the same.)

Pyright calls

• import A as A a redundant module alias
• from X import A as A a redundant symbol alias

These imports are needed, but not actually used:

• per PEP 484, to allow static type checkers to see the module's contents, and
• for package maintainers, as a courtesy to users to avoid them having to use long dotted access lookups (Law of Demeter Violation)

These typically occur in __init__.py and __init__.pyi files, but I don't want to ignore these files entirely. I would also like to run pycln on them to remove old-form uppercase typing module types (e.g. List, Optional, Dict, etc.) after running pyupgrade and to find genuinely unused imports.

Currently, the pycln CLI arguments --exclude and --extend-exclude only accept files and path regex's, not regex's matching specific lines in modules.

To have pycln ignore these aliases, one currently must add a # nopycln: import to each import in the project that is being exported, which is quite tedious.

Describe the solution you'd like. A clear and concise description of what you want to happen.

Since PEP 484 redundant aliases have a specific form, it would be nice to have a regex feature to exclude imports of a specific form; i.e. those matching the patterns

• import A as A and
• from X import A as A

A backreference could be used in a regular expression, like so:

/import (\w*) as \1/gm

and

/from (\w*) import (\w*) as (\2)/gm

respectively.

Since the module/symbol name is being repeated (A) instead of renamed, it is clear these are intended for the purpose of exporting (as opposed to, say, import numpy as np or from matplotlib import pyplot as plt), so there is minimal risk of accidentally skipping an actually unused import.

Describe alternatives you've considered A clear and concise description of any alternative solutions or features you've considered.
N/A

Additional context Add any other context or screenshots about the feature request here.
N/A

Describe the bug A clear and concise description of what the bug is.
Under certain circumstances, pycln will remove pass statements, which can lead to syntax errors in code. As shown in the code snippet below, it has something to do with nested if statements and that conclude with else: pass.

I have trimmed the code down as much as possible to narrow in on the situations where this occurs. This does not appear to happen if there are not nested conditionals. Also, if there is not another assignment before the inner conditional, this also does not happen.

While this is example code, I am seeing this happening in production code.

To Reproduce Steps to reproduce the behavior:

1. Take this code snippet:

   import os
   
   
   def cleaner():
       if bool(True):
           no_thing = None
           if no_thing:
              pass
           else:
              pass
       else:
           pass

2. Run this Pycln command:

   $ pycln test_file.py

3. Unexpected fixed code (if present):
   Note: there are two empty lines preceding the cleaner() method that are hidden by Github

   def cleaner():
       if bool(True):
           no_thing = None
           if no_thing:
               pass
           else:
               pass
       else:

Expected behavior:

1. I expected the pass statement to not be removed by pycln unexpectedly.
2. Expected fixed code (if present):

def cleaner():
    if bool(True):
        no_thing = None
        if no_thing:
            pass
        else:
            pass
    else:
        pass

Environment (please complete the following information):

• Python Version: Python 3.9.6 and Python 3.10.1
• Pycln Version: v1.2.1
• OS Type: Linux, MacOS

Describe the bug

pycln behaves weirdly when you do a star import of an empty file. I agree it's an edge case, and what am I doing importing from an empty file anyway?
Still, the behaviour is unexpected, and maybe there's an actual issue behind it all?

To Reproduce

1. Create a python package with three files:

__init__.py:

from .api import *
from .admin import *

api.py:

def my_view(request):
 ...

admin.py:

# Purposefully left empty
# For example django's `startapp` command will create empty files to put eg. your views

2. Run this Pycln command:

$ pycln .
  views/__init__.py:2:0 'from .admin import *' has removed! 🔮

All done! 💪 😎
1 import has removed, 3 files left unchanged.

The issue is that nothing is actually removed 🙆‍♀️ Running the same command again yields the same result: report says an import has been removed, but it's still present.

Note that running the same command with --all:

$ pycln . --all
  views/__init__.py:1:0 'from .api import *' has removed! 🔮
  views/__init__.py:2:0 'from .admin import *' has removed! 🔮
  views/__init__.py 2 imports has removed! 🚀

All done! 💪 😎
2 imports has removed, 1 file has changed, 2 files left unchanged.

does remove the from .api import * import, but not the from .admin import * one 🕵️

3. Error traceback or unexpected output (if present):

In the default behaviour, I expect the reporting to say that nothing needs to be changed.
In the --all behaviour, I expect both star imports to be removed.

Expected behavior:

☝️ Described above

Environment (please complete the following informations):

• Python Version: Python 3.8
• Pycln Version: 1.1.0
• OS Type: Ubuntu

Additional context Add any other context about the problem here.

I'll try and look into this myself 👀

To Reproduce Steps to reproduce the behavior:

1. Take this code snippet:

import random
import time


def foo():
    pass


if random.randint(0, 1):
    foo()
    if random.randint(0, 1):
        foo()
else:
    pass

2. Run this Pycln command:

 $ python3.8 -m pycln test.py

3. Unexpected fixed code (if present):

import random


def foo():
    pass


if random.randint(0, 1):
    foo()
    if random.randint(0, 1):
        foo()
else:

Expected behavior:

The generated python code should be legal.

Environment (please complete the following informations):

• Python Version: python3.8
• Pycln Version: pycln, version 1.1.0
• OS Type: Linux

Pycln always exits with 0 status code!

Environment (please complete the following informations):

• Python Version: Python 3.8
• Pycln Version: v0.0.1
• OS Type: Linux

Describe the bug

At wntrblm/nox#590, we tried to adress a Black issue with a breaking Click update. However, we noticed that also pycln was affected.

At least, I can reproduce this with GitHub Actions.

To Reproduce

Steps to reproduce the behavior:

1. Run pycln anywhere, with the latest click version.

2. See the error:

   pycln....................................................................Failed
   - hook id: pycln
   - exit code: 1
   
   Traceback (most recent call last):
     File "/home/runner/.cache/pre-commit/repo4hw2hs9j/py_env-python3/bin/pycln", line 5, in <module>
       from pycln.cli import app
     File "/home/runner/.cache/pre-commit/repo4hw2hs9j/py_env-python3/lib/python3.9/site-packages/pycln/__init__.py", line 7, in <module>
       import typer
     File "/home/runner/.cache/pre-commit/repo4hw2hs9j/py_env-python3/lib/python3.9/site-packages/typer/__init__.py", line 12, in <module>
       from click.termui import get_terminal_size as get_terminal_size
   ImportError: cannot import name 'get_terminal_size' from 'click.termui' (/home/runner/.cache/pre-commit/repo4hw2hs9j/py_env-python3/lib/python3.9/site-packages/click/termui.py)
   Traceback (most recent call last):
     File "/home/runner/.cache/pre-commit/repo4hw2hs9j/py_env-python3/bin/pycln", line 5, in <module>
       from pycln.cli import app
     File "/home/runner/.cache/pre-commit/repo4hw2hs9j/py_env-python3/lib/python3.9/site-packages/pycln/__init__.py", line 7, in <module>
       import typer
     File "/home/runner/.cache/pre-commit/repo4hw2hs9j/py_env-python3/lib/python3.9/site-packages/typer/__init__.py", line 12, in <module>
       from click.termui import get_terminal_size as get_terminal_size
   ImportError: cannot import name 'get_terminal_size' from 'click.termui' (/home/runner/.cache/pre-commit/repo4hw2hs9j/py_env-python3/lib/python3.9/site-packages/click/termui.py)

Expected behavior:

1. Description: I expected pycln to finish cleanly.

2. Expected output (if present):

3. Expected fixed code (if present):

Environment (please complete the following informations):

• Python Version: python-version: 3.9
• Pycln Version: 1.2.5 (but also checked using 1.1.0)
• OS Type: runs-on: ubuntu-20.04

Additional context

I think Click 8.1.0 is the responsible, like the issue with Black (psf/black#2964). However, maybe something else is involved.

Is your feature request related to a problem?
Use of this project isn't always convenient/ visual for the user.

Describe the solution you'd like
It would be nice to have an extension for vs code / pycharm. I also believe this will get more eyes on the project.

Describe alternatives you've considered
using version control integration would work great, but in my opinion, an extension would be the preferred use case.

Additional context
Not sure how hard this is, just a suggestion 😃

example.py

from ast import Import
from typing import List

def foo(bar: List["Import"]):
    pass

Current behavior, ast.Import is considered as unused:

from typing import List

def foo(bar: List["Import"]):
    pass

Expected behavior, nothing should change:

from ast import Import
from typing import List

def foo(bar: List["Import"]):
    pass

In order to add pycln as an ALE fixer using nvim, IIRC, I'd need pycln to be able to format files from stdin.

Very much like black does:

cat some_file.py | black -

Other than my niche use case, I think this would be a great addition for some other people.

pycln is currently not installable in Python 3.10 via pip:

> pip install pycln
ERROR: Could not find a version that satisfies the requirement pycln (from versions: none)
ERROR: No matching distribution found for pycln

Describe the bug Semi string type hints being removed when the type is not a built in type variable as defined in https://github.com/hadialqattan/pycln/pull/36/files

To Reproduce Steps to reproduce the behavior:

1. Take this code snippet:

    # cats.py
    from django.models import QuerySet
    from typing import TYPE_CHECKING
    
    if TYPE_CHECKING:
        from .models import Cat
    
    def make_cats_meow(cat_queryset: QuerySet["Cat"]) -> None:
        pass

2. Run this Pycln command:

   pycln -a cats.py

3. Error traceback or unexpected output (if present):

    cats3.py:5:4 'from .models import Cat' was removed! 🔮
    cats.py 1 import was removed! 🚀
   

4. Unexpected fixed code (if present):

    from django.models import QuerySet
    from typing import TYPE_CHECKING
    
    if TYPE_CHECKING:
        pass
    
    def make_cats_meow(cat_queryset: QuerySet["Cat"]) -> None:
        pass

Expected behavior:

1. Description: A clear and concise description of what you expected to happen.

I expect Cat to not be removed

2. Expected output (if present):
   Environment (please complete the following informations):

• Python Version: Python 3.10
• Pycln Version: v1.3.1
• OS Type: MacOS

Additional context
Only the built in types from https://github.com/hadialqattan/pycln/pull/36/files are supported.
Other subscriptable types aka QuerySet from django are not.

Every time I run it it gets stuck in this loop until it gets a precision error:

\pycln\utils\scan.py", line 718, in generic_visit
self.visit(item)
File "python39\lib\ast.py", line 407, in visit
return visitor(node)
\pycln\utils\scan.py", line 104, in wrapper
func(self, *args, **kwargs)
\pycln\utils\scan.py", line 639, in visit_ImportFrom
node = cast(ast.ImportFrom, expand_import_star(node, self._path))
\pycln\utils\scan.py", line 855, in expand_import_star
analyzer.visit(tree)
File "python39\lib\ast.py", line 407, in visit
return visitor(node)
\pycln\utils\scan.py", line 718, in generic_visit
self.visit(item)
File "python39\lib\ast.py", line 407, in visit
return visitor(node)
\pycln\utils\scan.py", line 104, in wrapper
func(self, *args, **kwargs)
\pycln\utils\scan.py", line 639, in visit_ImportFrom
node = cast(ast.ImportFrom, expand_import_star(node, self._path))
\pycln\utils\scan.py", line 855, in expand_import_star
RecursionError: maximum recursion depth exceeded while calling a Python object

Is there a way to specify seemingly unneeded imports globally across all files? If not, could this be added, e.g. as a global config option?

Hello,

Running pycln with only a config file as CLI options seems to ignore the path specifiied in the provided config file

To reproduce
According to the following pyproject.toml file

[tool.pycln]
path = "/user_management_rest_api/"

Running following command

pycln --config pyproject.toml

Produce the following output

No Path provided. Nothing to do 😴

While I expected pycln to run over /user_management_rest_api/ folder specified in provided config file

I tried to set verbose = true in my pyproject.toml file to ensure it was loaded and it worked as expected. Just the path looks like to be somehow ignored

Environment

• Python Version: 3.7
• Pycln Version: 1.1.0
• OS Type: [e.g.macOS Big Sur]

All the imports bellow should be considered as used.

append:

import x, y
import i

__all__ = ["x", "y"]
__all__.append("i")

extend:

import x, y
import i, j

__all__ = ["x", "y"]
__all__.extend(["i", "j"])

Describe the bug When I run pycln over my project, all I get is 16 lines of errors.

To Reproduce Steps to reproduce the behavior:

$ git clone https://github.com/qutebrowser/qutebrowser
$ cd qutebrowser
$ python3 -m venv .venv
$ .venv/bin/pip install -e .   # not doing this seems to prevent the issue
$ .venv/bin/pip install pycln
$ .venv/bin/pycln qutebrowser/

3. Error traceback or unexpected output (if present):

     /home/florian/tmp/qutebrowser/.venv/lib/python3.8/site-packages/qutebrowser.egg-link/__init__.py Permission denied [READ] [Errno 13] ⛔
     /home/florian/tmp/qutebrowser/.venv/lib/python3.8/site-packages/qutebrowser.egg-link/__init__.py Permission denied [READ] [Errno 13] ⛔
     /home/florian/tmp/qutebrowser/.venv/lib/python3.8/site-packages/qutebrowser.egg-link/__init__.py Permission denied [READ] [Errno 13] ⛔
     /home/florian/tmp/qutebrowser/.venv/lib/python3.8/site-packages/qutebrowser.egg-link/__init__.py Permission denied [READ] [Errno 13] ⛔
     /home/florian/tmp/qutebrowser/.venv/lib/python3.8/site-packages/qutebrowser.egg-link/__init__.py Permission denied [READ] [Errno 13] ⛔
     /home/florian/tmp/qutebrowser/.venv/lib/python3.8/site-packages/qutebrowser.egg-link/__init__.py Permission denied [READ] [Errno 13] ⛔
     /home/florian/tmp/qutebrowser/.venv/lib/python3.8/site-packages/qutebrowser.egg-link/__init__.py Permission denied [READ] [Errno 13] ⛔
     /home/florian/tmp/qutebrowser/.venv/lib/python3.8/site-packages/qutebrowser.egg-link/__init__.py Permission denied [READ] [Errno 13] ⛔
     /home/florian/tmp/qutebrowser/.venv/lib/python3.8/site-packages/qutebrowser.egg-link/__init__.py Permission denied [READ] [Errno 13] ⛔
     /home/florian/tmp/qutebrowser/.venv/lib/python3.8/site-packages/qutebrowser.egg-link/__init__.py Permission denied [READ] [Errno 13] ⛔
     /home/florian/tmp/qutebrowser/.venv/lib/python3.8/site-packages/qutebrowser.egg-link/__init__.py Permission denied [READ] [Errno 13] ⛔
     /home/florian/tmp/qutebrowser/.venv/lib/python3.8/site-packages/qutebrowser.egg-link/__init__.py Permission denied [READ] [Errno 13] ⛔
     /home/florian/tmp/qutebrowser/.venv/lib/python3.8/site-packages/qutebrowser.egg-link/__init__.py Permission denied [READ] [Errno 13] ⛔
     /home/florian/tmp/qutebrowser/.venv/lib/python3.8/site-packages/qutebrowser.egg-link/__init__.py Permission denied [READ] [Errno 13] ⛔
     /home/florian/tmp/qutebrowser/.venv/lib/python3.8/site-packages/qutebrowser.egg-link/__init__.py Permission denied [READ] [Errno 13] ⛔
     /home/florian/tmp/qutebrowser/.venv/lib/python3.8/site-packages/qutebrowser.egg-link/__init__.py Permission denied [READ] [Errno 13] ⛔
   
   Oh no, there are errors! 💔 ☹️
   179 files left unchanged, 16 files has failed to be cleaned.

Environment (please complete the following informations):

• Python Version: Python 3.8
• Pycln Version: 0.0.1a3
• OS Type: Archlinux

Additional context

Might be reproducible with any Python package, I didn't check yet.

Describe the bug

When using a library with a generic type (in my case sqlalchemy's Mapped type) pycln is removing in-use type checking imports.

I've tested with a few variations, here's what I found:

• 'OtherClass' works fine
• dict['OtherClass'] works fine
• Mapped['OtherClass'] is not recognized and the import is removed

To Reproduce Steps to reproduce the behavior:

I've taken my sqlalchemy model definition and reduced it down to the smallest reproducible case I could and then changed names around to avoid leaking codebase details.

import typing

from sqlalchemy.orm import Mapped

if typing.TYPE_CHECKING:
    from project import OtherClass


class Object:
    other: Mapped['OtherClass']

Used this command

$ pycln --all <file>.py

Resulted in this unexpected code:

import typing

from sqlalchemy.orm import Mapped

if typing.TYPE_CHECKING:
    pass


class Object:
    other: Mapped['OtherClass']

Expected behavior:

1. Description: A clear and concise description of what you expected to happen.
   I expect that this would not remove the output

Environment (please complete the following informations):

• Python Version: 3.10
• Pycln Version: 2.0.2
• OS Type: macos

Hi, colleague of my mine was unable to run pycln as a pre-commit hook on his Windows machine.
I have tried it myself and it fails with the same error message:

UnicodeEncodeError: 'charmap' codec can't encode character '\u2728' in position 12: character maps to <undefined>

Problematic line:

typer.echo(str(reporter), nl=False)

I have tried to fix it and encoding it to UTF-8 seems to help:

typer.echo(str(reporter).encode("utf-8"), nl=False)

I will prepare PR for this.

Environment:

• Python Version: Python 3.9
• Pycln Version: v0.0.2
• OS Type: Windows 10

Describe the issue
When installing pycln via pipenv the pathspec version clashes with blacks requirements:

[pipenv.exceptions.ResolutionFailure]: Warning: Your dependencies could not be resolved. You likely have a mismatch in your sub-dependencies.
  First try clearing your dependency cache with $ pipenv lock --clear, then try the original command again.
 Alternatively, you can use $ pipenv install --skip-lock to bypass this mechanism, then run $ pipenv graph to inspect the situation.
  Hint: try $ pipenv lock --pre if it is a pre-release dependency.
ERROR: Could not find a version that matches pathspec<0.9.0,<1,>=0.8.0,>=0.9.0 (from pycln==1.0.3->-r /var/folders/j2/20r03k5j6y9f7tg50byp06cm0000gn/T/pipenvandssn03requirements/pipenv-yhtk_fzz-constraints.txt (line 6))
Tried: 0.2.2, 0.3.0, 0.3.1, 0.3.2, 0.3.3, 0.3.4, 0.4.0, 0.5.0, 0.5.1, 0.5.2, 0.5.3, 0.5.4, 0.5.5, 0.5.6, 0.5.7, 0.5.8, 0.5.9, 0.6.0, 0.7.0, 0.7.0, 0.8.0, 0.8.0, 0.8.1, 0.8.1, 0.9.0, 0.9.0
There are incompatible versions in the resolved dependencies:
  pathspec<0.9.0,>=0.8.0 (from pycln==1.0.3->-r /var/folders/j2/20r03k5j6y9f7tg50byp06cm0000gn/T/pipenvandssn03requirements/pipenv-yhtk_fzz-constraints.txt (line 6))
  pathspec<1,>=0.9.0 (from black==21.8b0->-r /var/folders/j2/20r03k5j6y9f7tg50byp06cm0000gn/T/pipenvandssn03requirements/pipenv-yhtk_fzz-constraints.txt (line 8))

I know pipenv sucks and we should use poetry but I haven't managed to convince my team of that yet 😅
Also I think this is a dependency problem in general, isn't it?

Environment (please complete the following informations):

• Python Version: 3.9.5
• Pycln Version: 1.0.3
• OS Type: M1 Mac

Additional Notes:

• Really love this package so far ❤️

os should not be considered as unused:

import os.path
print(os.R_OK)

Hints:

• related to has_used on utils/refactor.py.
• import os.path should be refactored to import os (optional).

from importlib import import_module

time = import_module("time")  # used
os = import_module("os")  # unused

print(time.time())

Current behavior, nothing would change:

from importlib import import_module

time = import_module("time")  # used
os = import_module("os")  # unused

print(time.time())

Expected behavior, remove os:

from importlib import import_module

time = import_module("time")  # used

print(time.time())

Hi,

Thanks for creating this project. It's been super helpful 👏

Ran into an issue with match statement syntax on the last version (1.2.5), running the project as a pre-commit hook. Looks like match statement support was added a while ago, so not sure if I'm just tired or there's an actual bug in the hook 🙃

Reproducible example

Python code

# match.py
match 1:
    case 1:
        print('nice')
    case _:
        print('what')

Pre-commit config

# pre-commit-config.yaml
repos:
  - repo: https://github.com/hadialqattan/pycln
    rev: v1.2.5
    hooks:
      - id: pycln

Output

  match.py:1:7 SyntaxError: invalid syntax 'match 1:' ⛔

Describe the bug

Here's a sample output I get:

$> pycln .
  path/to/file.py:34:0 'import time' has removed! 🔮
  path/to/file.py:35:0 'import json' has removed! 🔮
  path/to/file.py 2 imports has removed! 🚀

All done! 💪 😎
2 imports has removed, 1 file has changed, 201 files left unchanged.

For each removed import I expect to read "1 import was removed" or "1 import has been removed", not "1 import has removed".
In the totals (for each file and overall), the plural form of the verb should be handled. I expect to read "2 imports were removed" or "2 imports have been removed", not "2 imports has removed".

To Reproduce

Every output behaves this way so it should be easy enough to reproduce :> It's even asserted on in the tests :o

Expected behavior

Already talked about above

Environment (please complete the following informations):

• Python Version: Python 3.7
• Pycln Version: 1.1.0 (but the issue is present on master)
• OS Type: Ubuntu

Additional context

I'm not a native English speaker so maybe I'm flat out wrong. Do tell if I am :>
I'll try and open a PR to fix this :)

Describe the bug Spaces inside Literal type annotations break pycln.

To Reproduce Steps to reproduce the behaviour:

1. Take this code snippet:

    # test-pycln.py
    from typing_extensions import Literal
    import math
    
    # spaces inside Literal type annotations break pycln
    LITERAL_WITH_SPACE: Literal["literal with space"] = "literal with space"
    NO_SPACE: Literal["no_space"] = "no_space"

2. Run this Pycln command:

    $ pipx list | grep pycln
       package pycln 0.0.1b0, Python 3.6.8
    $ mypy test-pycln.py
        Success: no issues found in 1 source file
    $ pycln test-pycln.py
      test-pycln.py .:1:12 SyntaxError: invalid syntax 'literal with space' ⛔
    
    Oh no, there is an error! 💔 ☹️
    1 file left unchanged, 1 file has failed to be cleaned.

Expected behavior:

1. pycln successfully processes the file.
2. Expected output (if present):

   $ pycln test-pycln.py 
   Looks good! ✨ 🍰 ✨
   1 file changed.

Environment (please complete the following information):

• Python Version: Python 3.6.8 Python 3.8.2
• Pycln Version: 0.0.1b0
• OS Type: Linux, Windows

pycln installed into an isolated virtual environment by pipx

My projects style use trailing commas.

Due to this pycln does this a lot:

--- original/ manim\cli\new\group.py
+++ fixed/ manim\cli\new\group.py
@@ -9,7 +9,7 @@
     add_import_statement,
     copy_template_files,
     get_template_names,
-    get_template_path,
+    get_template_path
 )

Could I have an option that stops this.

Description: Running pycln with the -a, lines with # NOQA are removed

To Reproduce Steps to reproduce the behavior:

1. Take this code snippet:

from models.blueprint import \
    Blueprint  # NOQA - required for stand-alone runs or sqlalchemy gets lost wrt cross-table reference

print("Test")

2. Run this Pycln command:

   $ pycln -a pycln_test.py

3. Error traceback or unexpected output (if present):

  pycln_test.py:1:0 'from models.blueprint import Blueprint' has removed! 🔮
  pycln_test.py 1 import has removed! 🚀

All done! 💪 😎
1 import has removed, 1 file has changed.

4. Unexpected fixed code (if present):

print("Test")

Expected behavior:

Lines marked NOQA should remain (at least on the lowest aggressivity level)

Environment (please complete the following informations):

• Python Version: Python 3.8
• Pycln Version: version 1.0.3
• OS Type: MacOs 11.6 (Big Sur)

example.py

from ast import Import
from typing import List

def foo(bar: "List['Import']"):
    pass

Current behavior, ast.Import is considered as unused:

from typing import List

def foo(bar: "List['Import']"):
    pass

Expected behavior, nothing should be changed:

from ast import Import
from typing import List

def foo(bar: "List['Import']"):
    pass

/usr/lib/python3.10/lib-dynload/math.cpython-310-x86_64-linux-gnu.so/__init__.py Permission denied [READ] [Errno 13]
hi, i'm getting this error, and the only thing i can tell is that /usr/lib/python3.10/lib-dynload/math.cpython-310-x86_64-linux-gnu.so is a file.
my user has permissions over this directory

Hi!

I see pycln recently added support for .pyi files. That's awesome!

I'm interested in possibly using pycln in typeshed's CI. However, it seems like there's one part of PEP 484 that pycln isn't quite compliant with when it comes to stub files, namely this:

[...] all objects imported into a stub using from ... import * are considered exported. (This makes it easier to re-export all objects from a given module that may vary by Python version.)

If I run pycln on typeshed at the moment, it erroneously deletes a large number of "unused" from foo import * imports. These imports are all deliberate -- if a type checker sees a from foo import * import in a stub file bar.pyi, it understands all symbols from module foo to be re-exported in module bar.

Playing around with pycln, it looks like this one-line patch solves the problem:

diff --git a/pycln/utils/refactor.py b/pycln/utils/refactor.py
index 4739e72..a241fc8 100644
--- a/pycln/utils/refactor.py
+++ b/pycln/utils/refactor.py
@@ -279,7 +279,7 @@ class Refactor:

                 # Expand any import '*' before checking.
                 node, is_star = self._expand_import_star(node)
-                if is_star is None:
+                if is_star is None or (is_star and self._path.is_stub):
                     continue

                 # Get set of used names.

I'd file a PR, but I'm struggling a little bit with the test suite 😅 (I'm not too familiar with the advanced features of pytest, I'm afraid.)