ruedigervoigt / salted Goto Github PK

As salted uses sqlite as a cache and does not do anything low-level, using SQLAlchemy seems a good choice.

Create a simple Graphical User Interface that can be opened via the command line parameter --gui.

Due to the licenses and their deep integration into Python Tkinter or wxPython seem to be good choices.

Hyperlinks might contain mailto:mailadress in the href field.

• Ignore those or do basic check for validity.
• Beware: sometimes those mailto links are scrambeld in raw HTML to avoid that spammers extract mail addresses.

Some servers behave differently if the user agent discloses this is a Python script. For example they will not answer HTTP requests because they suspect malicious intent.

1. salted should by default sent an user agent like "salted - validating hyperlinks"

2. The user should be able to set the UA string to impersonate a browser if necessary (for example if the rate of false results is too high).

There should be a CLI parameter to specify a logfile

This code in parser.py:

        soup = BeautifulSoup(file_content, 'html.parser')
        for link in soup.find_all('a'):

does not match if code falsely uses upper case <A HREF='https://www.example.com/'>.. - like for example the export function from the Chrome bookmark manager.

The simple solution would be to convert everything to lowercase.
This would also mean faster matching.
Yet it looks wrong in the report.

The number of features has grown and likely will grow even more. Therefore salted should use a config file. Simply running from the CLI should produces a result. The config file could be version controlled by the user.

• Look for a config file in the current working directory.
• Make it possible to provide the path to an alternative config file
• If there is a config file, read it.
• Catch reading error.
• Catch if the config file is corrupted.
• Check if CLI parameters overwrite parts of the config file.
• Check if the config (file + CLI) misses flags. Set reasonable defaults.
• Check if all strictly necessary parameters are set.
• salted can be run from the CLI without creating an object.

Salted should be able to check links in markdown files.

• The GitHub markdown syntax [link text](url) is easy to parse.
• The pandoc dialect of markdown additionally knows the automatic link <url> and it can include raw HTML into a markdown document.

Salted has basic support for .tex files, but in scientific projects most links are stored in a BibTeX literature database.

I suspect it's because you need to do

from salted.__main__ import Salted as Salted

instead of

from salted.__main__ import Salted

Or typecheckers won't consider it a re-export.
https://www.python.org/dev/peps/pep-0484/#stub-files

I'm not 100% sure - since the pep only makes the claim about stub files. It's pretty easily reproable by setting up

from salted import Salted

And running mypy --strict testfile.py