sky-uk / bslint Goto Github PK

View Code? Open in Web Editor NEW

24.0 24.0 9.0 717 KB

A linter for the BrightScript language.

License: BSD 3-Clause "New" or "Revised" License

Python 97.44% Brightscript 2.56%

bslint's People

Contributors

Stargazers

Watchers

Forkers

vasiuadriana prototype127001 gabilan seiji-fli alexmakii wardy pawelhertman unwiredben

bslint's Issues

Crashes with Python 3.8

When using Python 3.8 I get the following error:

Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.8/bin/bslint", line 8, in <module>
    sys.exit(main())
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/bslint/bslint.py", line 12, in main
    interface_handler.start()
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/process.py", line 121, in start
    self._popen = self._Popen(self)
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/context.py", line 224, in _Popen
    return _default_context.get_context().Process._Popen(process_obj)
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/context.py", line 283, in _Popen
    return Popen(process_obj)
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/popen_spawn_posix.py", line 32, in __init__
    super().__init__(process_obj)
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/popen_fork.py", line 19, in __init__
    self._launch(process_obj)
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/popen_spawn_posix.py", line 47, in _launch
    reduction.dump(process_obj, fp)
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/reduction.py", line 60, in dump
    ForkingPickler(file, protocol).dump(obj)
TypeError: cannot pickle '_io.TextIOWrapper' object

This is probably due to the changes in the multiprocessing library.

Bslint uses libraries which does not exist for windows 64bit. I tried this with python 32bit and get an error related to the OS (source_process = _winapi.OpenProcess(OSError: [WinError 87] The parameter is incorrect). I tried this with python 64bit and get an error related to 'enchant' C library. Such a library does not exist for python 64bit.

a possible solution for python 32bit could be the following: multiprocessing in windows is different than linux/mac - specifically, windows uses spawn while linux/mac uses fork. You can tell multiprocessing to use spawn instead of fork by something like this

import multiprocessing
multiprocessing.set_start_method('spawn')

Please fix it for Windows

False parsing error for sample file from @siekermantechnology

At least one false parsing error was reported in an example file from @siekermantechnology.

ERROR: Parsing program failed. Line number: 796

File: utils_brightscript.brs.txt

bslintrc used: bslintrc.txt

Further investigation needs to be done to find the exact reason that parsing failed, and resolve it

Operator spacing should not be flagged for ++ or -1

Switching on the rule spaces_around_operators will be flagged up incorrectly with new brightscript increment(++) or decrement (--) operations. Also negative numbers are flagged up.

Repository Archival Notification

As part of a GitHub wide review by Cyber security, this repo was identified in May 2024 as not being 'active' for > 18 months. Following on from comms and consultation with users, admins and business reps this repository has been archived. More details on this campaign can be found on SharePoint https://skyglobal.sharepoint.com/sites/GroupCyberSecurity10/SitePages/Repository-Archiving.aspx. If you wish this to be unarchived please complete the form on https://tech.at.sky/techss?id=tss_cat_item&sys_id=576125771b4e4a18923443f6b04bcb94

[Spike] Package the application using pip wheel

False positive for spaces around <> operators warning

@siekermantechnology Reported that the "spaces around operators" check is giving false positive warnings on if statements with a <> operator, such as if (parsedJSONObject <> invalid).

Use requirements.txt file for dependency management

All libraries should be added to requirements.txt

(then they can be installed using pip install -r requirements.txt)

Update the readme file with this step

File encoding not checked.

Although the 'check file encoding' command has been tested directly, it is not actually implemented when running bslint from the command line.
The command is executed in the file reader method in the interface handler:

@staticmethod
    def file_reader(file_to_lex):
        with open(file_to_lex, "r+") as file:
            str_to_lex = file.read()
        return {"invalid_encoding": commands.check_file_encoding(file_to_lex), "file_content": str_to_lex}

However, the 'invalid_encoding' field is never actually used:

    def lint_file(self, filepath):
     ...
            file_content = self.file_reader(filename)['file_content']
     ...

linter breaking when encountering ;

in prints for example:
print "[";_logLevel;"] ";isoDateTime;" ";message
you will get:
ERROR: You have unmatched quotation marks at ;messag on line number: 67

ERROR: Parsing program failed

Version: 1.0.1

Failure code block

function TestClass() as object
    if m.TestClass = invalid then
        obj = CreateObject("roAssociativeArray")
        obj.var = true

        m.TestClass = obj
    end if

    return m.TestClass
end function

ERROR: Parsing program failed. Line number: 10

Parsing works

function TestClass() as object
    if true then
        obj = CreateObject("roAssociativeArray")
        obj.var = true

        m.TestClass = obj
    end if

    return m.TestClass
end function

Main documentation is missing parameters

I just pulled in bslint version 1.0.1 through pip3. When I run bslint -h, it gives me:

usage: bslint [-h] [--path PATH] [--lex] [--version]

BSLint: A linter for BrightScript.

optional arguments:
  -h, --help            show this help message and exit
  --path PATH, -p PATH  Specify directory or file
  --lex, -l             Runs only the lexer, without parsing the code
  --version, -v         Get current version

The documentation at https://github.com/sky-uk/bslint/blob/master/README.md doesn't mention the arguments, even though the --path / -p parameter is indeed needed.

Wrong indentation for Arrays

"check_indentation" rule enforces wrong indentations for arrays.

When it's turned on this code is correct:

variable = [
'sdfsdfg',
'sdfgsdfg',
]

while this is not:

variable = [
  'sdfsdfg',
  'sdfgsdfg',
]

In (not only) my opinion the second notation should be accepted.
For associative arrays the rule works correct.

I'm not familiar with Python, so I'm not able to contribute, but I guess the problem occurs because in lexer/regexs.py signs [ and ] are matched with "NO_INDENTATION"

[r"\[", const.OPEN_SQUARE_BRACKET, const.OPEN_SQUARE_BRACKET, const.NO_INDENTATION],
    [r"\]", const.CLOSE_SQUARE_BRACKET, const.CLOSE_SQUARE_BRACKET, const.NO_INDENTATION],

what in fact is correct because these signs can be used as Array Operator to access array's value.
So the solution probably is to create another regexes to recognize brackets as array initialization when they appear after = sign.

Use virtualenv and virtualenvwrapper for development purposes

http://docs.python-guide.org/en/latest/dev/virtualenvs/

Update read file with dev instructions on how to get setup

Configurable message colours

Investigate ways to configure the colour of the messages printed out. For example, the yellow colour currently used for warnings is difficult to read on a white terminal background.

It may be possible to detect the user's terminal preferences and use default colours.
Alternatively (or possibly additionally), the user should be able to configure the colours themselves.

Veracode Scan Trigger - December 2023

trigger-veracode-scan-sast
trigger-veracode-scan-sca

This is a trigger for a Veracode scan conducted by the Sky Cyber Software Security Team.
This issue will trigger a scan on the repository.You do not need to take any action.
See the below link for more information.

https://skyglobal.sharepoint.com/sites/GroupCyberSecurity10/SitePages/Software-Security.aspx

Sporadic pyenchant exception

Following exception sometimes thrown by pyenchant. Re-running the command removes the exception.

Exception ignored in: <bound method Dict.__del__ of <enchant.DictWithPWL object at 0x101993f60>>
Traceback (most recent call last):
  File "/Users/jin05/Documents/bslint/venv/lib/python3.5/site-packages/enchant/__init__.py", line 572, in __del__
  File "/Users/jin05/Documents/bslint/venv/lib/python3.5/site-packages/enchant/__init__.py", line 847, in _free
  File "/Users/jin05/Documents/bslint/venv/lib/python3.5/site-packages/enchant/__init__.py", line 634, in _free
  File "/Users/jin05/Documents/bslint/venv/lib/python3.5/site-packages/enchant/__init__.py", line 346, in _free_dict
  File "/Users/jin05/Documents/bslint/venv/lib/python3.5/site-packages/enchant/__init__.py", line 353, in _free_dict_data
TypeError: 'NoneType' object is not callable
Exception ignored in: <bound method Dict.__del__ of <enchant.Dict object at 0x101d692b0>>
Traceback (most recent call last):
  File "/Users/jin05/Documents/bslint/venv/lib/python3.5/site-packages/enchant/__init__.py", line 572, in __del__
  File "/Users/jin05/Documents/bslint/venv/lib/python3.5/site-packages/enchant/__init__.py", line 634, in _free
  File "/Users/jin05/Documents/bslint/venv/lib/python3.5/site-packages/enchant/__init__.py", line 346, in _free_dict
  File "/Users/jin05/Documents/bslint/venv/lib/python3.5/site-packages/enchant/__init__.py", line 353, in _free_dict_data
TypeError: 'NoneType' object is not callable

Use absolute paths for test resources

This is so that we can run tests from any location.

Also, after this enhancement all the if statements about nose tests can be removed.

To test that all tests are running fine from top level, sub-directory level etc. you can use

nosetests

For this you need the nose library

On macOS, users other than the owner of /usr/local/lib cannot run bslint

It looks like bslint, by nature of using enchant with a PWL, needs the PWL file to be writable by the user running it. (I did some digging and found out that enchant opens the PWL in a+b mode when attempting to load it) This can be problematic in CI setups where the CI server is run as its own user.

Here is some sample output that corroborates this:

$ sudo su jenkins                                              
Password:
bash-3.2$ bslint
Traceback (most recent call last):
  File "/usr/local/bin/bslint", line 7, in <module>
    from bslint.bslint import main
  File "/usr/local/lib/python3.6/site-packages/bslint/__init__.py", line 11, in <module>
    from bslint.lexer.commands import *
  File "/usr/local/lib/python3.6/site-packages/bslint/lexer/commands.py", line 11, in <module>
    DICTIONARY = words_dict.get_new_dictionary()
  File "/usr/local/lib/python3.6/site-packages/bslint/lexer/words_dictionary.py", line 9, in get_new_dictionary
    return enchant.DictWithPWL(dictionary_lang, personal_words_list_path)
  File "/usr/local/lib/python3.6/site-packages/enchant/__init__.py", line 822, in __init__
    self.pwl = self._broker.request_pwl_dict(pwl)
  File "/usr/local/lib/python3.6/site-packages/enchant/__init__.py", line 330, in request_pwl_dict
    self._raise_error(eStr % (pwl,))
  File "/usr/local/lib/python3.6/site-packages/enchant/__init__.py", line 259, in _raise_error
    raise eclass(EnchantStr("").decode(err))
enchant.errors.Error: Couldn't open personal wordlist '/usr/local/lib/python3.6/site-packages/bslint/config/personal-words-list.txt'

I propose that the PWL be copied somewhere in the user's home directory so that bslint can run correctly as any user.

Give more useful line numbers for parsing errors

Give the correct line number when a specific code block causes a parsing error, not the final line number of the enclosing function e.g. END FUNCTION.

Use pylint linter

pylint is good for making sure the code is styled according to PEP-8

Various remarks after testing

We've been experimenting with the bslint tool on our utility functions .brs file. Below a list of remarks we encountered while working on it.

Documentation on the various parameters in the configuration file would be welcome. Part of it can be figured out based on the naming of the parameters in the sample file, but part of it is not clear and not documented yet.
Change the default colour of the lex output from bright yellow to something which is easier to read on a white background.
Running bslint without the -l / --lex flag resulted in: ERROR: Parsing program failed. Line number: 796
WARNING: Operators must be surrounded by 1 space(s). => Gives falls positive on if statements with a <> operator, such as "if (parsedJSONObject <> invalid)".
WARNING: Print statements not allowed. => We haven't found a way to disable this check in the configuration file. Would be a good setting to add.
WARNING: You have spelling mistakes in your code. => We disabled this check (spell_check) immediately, because it gave too many false positives to be useful.
WARNING: Invalid indentation, you must indent with tabs. => We indent with 4 spaces, so we modified the setting "only_tab_indents" in "check_indentation" to "false". It's not clear if "false" allows only spaces or tabs + spaces.
Also, after setting to false, we get a lot of the following warnings: WARNING: Tab indentation should be set to 4 spaces. => The weird thing is that this happens all over the place, also in lines which only contain a comment, so looks like something is going wrong there.

Repo Archiving Notification

On the 20th of May 2024 this repository will be archived.
When a repository is archived, existing users will be removed and not able to access the repository.
A limited number of GitHub super users will be added to the repository to manage unarchiving and access permissions in the event of an incident.
All other unarchiving requests will go through the ServiceNow tech store (please see the FAQ’s for more information).
You can review the repositories to be archived, listed on the Cyber Security SharePoint page. You can filter by the owner’s email address, division, and repository name.
This repository is in scope for archiving and had not identified owner. You can claim these repositories by completing the Opt-out form.

Use python conventions for file names and function or method names

Classes should use lowercase_and_underscores

Test classes should use lowercase_underscores_test.py

Functions/methods should use lowercase_and_underscores

Conditional Compilation Parser Support

Parser gives errors for Conditional Compilation grammar

https://forums.roku.com/viewtopic.php?t=106057

https://sdkdocs.roku.com/display/sdkdoc/Conditional+Compilation

Handle keyboard interrupts cleanly

Since making BSLint use the multiprocessing library (to implement the spinner), keyboard interrupts are no longer handled cleanly. Instead, a long stacktrace is now printed out. See example below:

Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/multiprocessing/popen_fork.py", line 29, in poll
    pid, sts = os.waitpid(self.pid, flag)
KeyboardInterrupt
Process InterfaceHandler-1:
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/multiprocessing/process.py", line 249, in _bootstrap
    self.run()
  File "/Users/jin05/Documents/bslint/venv/lib/python3.5/site-packages/bslint/interface_handler.py", line 48, in run
    self.lint()
  File "/Users/jin05/Documents/bslint/venv/lib/python3.5/site-packages/bslint/interface_handler.py", line 61, in lint
    self.lint_all(pathname)
  File "/Users/jin05/Documents/bslint/venv/lib/python3.5/site-packages/bslint/interface_handler.py", line 107, in lint_all
    self.lint_directory(dir_name, files)
  File "/Users/jin05/Documents/bslint/venv/lib/python3.5/site-packages/bslint/interface_handler.py", line 112, in lint_directory
    self.lint_file(filepath)
  File "/Users/jin05/Documents/bslint/venv/lib/python3.5/site-packages/bslint/interface_handler.py", line 122, in lint_file
    lex_result = Parser().parse(file_content)
  File "/Users/jin05/Documents/bslint/venv/lib/python3.5/site-packages/bslint/parser/parser.py", line 44, in parse
    self.check_statement_validity(self.tokens[self.current_token_index:])
  File "/Users/jin05/Documents/bslint/venv/lib/python3.5/site-packages/bslint/parser/parser.py", line 59, in check_statement_validity
    self.handle_statement()
  File "/Users/jin05/Documents/bslint/venv/lib/python3.5/site-packages/bslint/parser/parser.py", line 71, in handle_statement
    self.iterate_over_inner_statement()
  File "/Users/jin05/Documents/bslint/venv/lib/python3.5/site-packages/bslint/parser/parser.py", line 99, in iterate_over_inner_statement
    self.iterate_over_stmt()
  File "/Users/jin05/Documents/bslint/venv/lib/python3.5/site-packages/bslint/parser/parser.py", line 112, in iterate_over_stmt
    self.reduce_and_handle_error()
  File "/Users/jin05/Documents/bslint/venv/lib/python3.5/site-packages/bslint/parser/parser.py", line 148, in reduce_and_handle_error
    self.reduce_and_handle_error()
  File "/Users/jin05/Documents/bslint/venv/lib/python3.5/site-packages/bslint/parser/parser.py", line 139, in reduce_and_handle_error
    possible_production_rules = self._get_possible_production_rules(self.current_statement[-(index + 1)])
  File "/Users/jin05/Documents/bslint/venv/lib/python3.5/site-packages/bslint/parser/parser.py", line 195, in _get_possible_production_rules
    if current_token not in self.current_grammar.RULES[self.current_priority_level]:
KeyboardInterrupt