thoth-station / kebechet Goto Github PK

I'm Kebechet bot, goddess of freshness - I will keep your source code fresh and up-to-date

Python 98.18% Shell 0.63% Dockerfile 0.61% Gherkin 0.58%

thoth dependency-manager dependency-analysis sourceops version-manager bot python python3 artificial-intelligence hacktoberfest

kebechet's Introduction

Kebechet

Kebechet is an extensible system of repository managers for GitHub, GitLab, and Pagure.

To use it on your repository, all you have to do is set up the authentication information and then run the following command:

> pipenv run PYTHON_PATH=. kebecht-cli run-url -u <url-to-github-repo> -s <GITHUB|GITLAB|PAGURE>

Features

Python package dependency management (see update and thoth-advise manager)
Version releases (version manager)
and MORE

Contribute

Issue Tracker:github.com/thoth-station/kebechet/issues
Source Code: github.com/thoth-station/kebechet

Support

If you are having issues, please let us know by opening an issue

License

The project is licensed under the GPL-3.0 license.

Full documentation

For full documentation go here

kebechet's People

Contributors

Stargazers

Watchers

kebechet's Issues

GitLab support

As a user I would like to use Kebechet with GitLab.

The configuration of kebechet should provide a github/gitlab switch.
All API calls to GitHub have have their GitLab alternatives (see libraries that abstract this).
We should support also hosted github - once the config file will be extended, state also API URL of the hosted GitHub instance.

uncomment once AICoE index will be fixed

kebechet/kebechet/managers/origin/origin.py

Lines 78 to 83 in 454a096

    
           # TODO: uncomment once AICoE index will be fixed 
        
           #url = f"{source_url}/{package_name}/" 
        
           _LOGGER.debug(f"Discovering package %r artifacts from %r", package_name, url) 
        
           response = requests.get(url, verify=verify_tls) 
        
           response.raise_for_status() 
        
           soup = BeautifulSoup(response.text, 'lxml')

This issue was generated by todo based on a `TODO` comment in `454a096` when #135 was merged. cc @fridex.

refactor update logic and sourceops logic

see https://trello.com/c/h4h32uH0

handle this exception by opening a feature request

The exception at

kebechet/kebechet/update.py

Line 457 in dfe1d81

raise DependencyManagementError("There was found an issue in your dependency "

could be handled by opening a feature request to add a Pipfile or requirements.in

Add support for requirements-dev

As a developer who is not using pipenv, I would like to use Kebechet for managing requirements-dev.in`. Besides requirements.txtI would get updated alsorequirements.in. The input file for pinning these files would be requirements-dev.in``.

assignees is a list...

Describe the bug


2018-07-28 08:15:36,007 [1] ERROR    kebechet.managers.version.version: Failed to assign ['sesheta'] to issue #15
--
  | Traceback (most recent call last):
  | File "/usr/local/lib/python3.6/site-packages/kebechet/managers/version/version.py", line 192, in run
  | self.sm.assign(issue, assignees)
  | File "/usr/local/lib/python3.6/site-packages/kebechet/source_management.py", line 169, in assign
  | issue.assign(users)
  | File "/usr/local/lib/python3.6/site-packages/IGitt/GitHub/GitHubIssue.py", line 152, in assign
  | {'assignees': [user.username for user in users]})
  | File "/usr/local/lib/python3.6/site-packages/IGitt/GitHub/GitHubIssue.py", line 152, in <listcomp>
  | {'assignees': [user.username for user in users]})
  | AttributeError: 'list' object has no attribute 'username'

Propagate Zuul configuration options from new version release issues

Is your feature request related to a problem? Please describe.

We would like to make sure that releases are performed with specified "Depends-On" or other Zuul conifuguration options. This way we can declaratively state which PRs should go in before a release is done and we do not need to wait for PRs to be merged before opening new release issues.

Describe the solution you'd like

Kebechet's version manager copies/propagates "Depends-On" (and possibly others) lines from issues triggering new releases.

package.version not found for kebechet itself?!

Describe the bug


2018-07-28 08:03:18,279 [1] INFO     kebechet.utils: Cloning repository [email protected]:thoth-station/kebechet.git to /tmp/tmpjpnb5xrb
--
  | 2018-07-28 08:03:18,811 [1] INFO     kebechet.managers.update.update: Using Pipfile for dependency management
  | 2018-07-28 08:03:19,120 [1] DEBUG    kebechet.source_management: Issue 'No dependency management found' not found, not closing it
  | 2018-07-28 08:03:19,403 [1] DEBUG    kebechet.source_management: Issue 'Failed to perform initial lock of software stack' not found, not closing it
  | 2018-07-28 08:03:19,408 [1] ERROR    kebechet.config: An error occurred during run of manager {} <class 'kebechet.managers.update.update.UpdateManager'> for thoth-station/kebechet, skipping
  | Traceback (most recent call last):
  | File "/usr/local/lib/python3.6/site-packages/kebechet/config.py", line 168, in run
  | instance.run(**manager_configuration)
  | File "/usr/local/lib/python3.6/site-packages/kebechet/managers/update/update.py", line 595, in run
  | result = self._do_update(labels, pipenv_used=True)
  | File "/usr/local/lib/python3.6/site-packages/kebechet/managers/update/update.py", line 489, in _do_update
  | old_environment = self._get_all_packages_versions()
  | File "/usr/local/lib/python3.6/site-packages/kebechet/managers/update/update.py", line 148, in _get_all_packages_versions
  | 'version': package_info['version'][len('=='):]
  | KeyError: 'version'
  | 2018-07-28 08:03:19,409 [1] INFO     kebechet.config: Running manager 'info' for 'thoth-station/kebechet'

Kebechet info

pre approved PR?

does

kebechet/config/thoth.yaml

Line 4 in 7f374d0

label: ['approved', 'bot']

mean that PR opened by kebechet are pre-approved? if so? they ignore any CI job result, I would consider that a bug :)

Extent version manager with version.py file support

https://packaging.python.org/guides/single-sourcing-package-version/

Automatically rebase on top of the current master

As Kebechet I want to make sure that all PRs raised are made on top of the current master so that CI checks are triggered on the current codebase. This should also handle merge conflicts when operating on same requirements.txt/Pipfile.lock file.

Failed to update dependencies to their latest version

Automatic dependency update failed for the current master with SHA ab29d90.

The automatic dependency management cannot continue. Please fix errors reported bellow.

Command

  $ pipenv update --dev

Standard output

Running $ pipenv lock then $ pipenv sync.

Standard error

Creating a virtualenv for this project...
Pipfile: /tmp/tmpm0w9d5i_/Pipfile
Using /usr/bin/python3.6m (3.6.5) to create virtualenv...
Running virtualenv with interpreter /usr/bin/python3.6m
Using base prefix '/usr'
New python executable in /tmp/tmpm0w9d5i_/.venv/bin/python3.6m
Also creating executable in /tmp/tmpm0w9d5i_/.venv/bin/python
Installing setuptools, pip, wheel...done.

Virtualenv location: /tmp/tmpm0w9d5i_/.venv
Locking [dev-packages] dependencies...

Warning: Your dependencies could not be resolved. You likely have a mismatch in your sub-dependencies.
  You can use $ pipenv install --skip-lock to bypass this mechanism, then run $ pipenv graph to inspect the situation.
  Hint: try $ pipenv lock --pre if it is a pre-release dependency.
Could not find a version that matches pydocstyle==1.1,==2.1.1 (from -r /tmp/pipenv-ani508v4-requirements/pipenv-sl2r9pai-constraints.txt (line 5))
Tried: 1.0.0, 1.0.0, 1.1.0, 1.1.0, 1.1.1, 1.1.1, 2.0.0, 2.0.0, 2.1.0, 2.1.0, 2.1.1, 2.1.1, 2.1.1
There are incompatible versions in the resolved dependencies.

Environment details


Kebechet version: 1.0.0
Python version: 3.6.5
Platform: Linux-4.17.3-200.fc28.x86_64-x86_64-with-fedora-28-Twenty_Eight
pipenv version: pipenv, version 2018.7.1

Dependency graph

click==6.7
coala==0.11.0
  - appdirs [required: ~=1.4, installed: 1.4.3]
  - coala-utils [required: ~=0.6.2, installed: 0.6.7]
    - appdirs [required: ==1.4.*, installed: 1.4.3]
    - pyprint [required: ==0.2.*, installed: 0.2.6]
      - colorama [required: ~=0.3.7, installed: 0.3.9]
      - termcolor [required: ~=1.1.0, installed: 1.1.0]
  - colorlog [required: ~=2.7, installed: 2.10.0]
  - dependency-management [required: ~=0.4.0, installed: 0.4.0]
    - coala-utils [required: ~=0.6, installed: 0.6.7]
      - appdirs [required: ==1.4.*, installed: 1.4.3]
      - pyprint [required: ==0.2.*, installed: 0.2.6]
        - colorama [required: ~=0.3.7, installed: 0.3.9]
        - termcolor [required: ~=1.1.0, installed: 1.1.0]
    - sarge [required: ~=0.1.4, installed: 0.1.5.post0]
  - libclang-py3 [required: ~=3.4.0, installed: 3.4.0]
  - Pygments [required: ~=2.1, installed: 2.2.0]
  - PyPrint [required: ~=0.2.6, installed: 0.2.6]
    - colorama [required: ~=0.3.7, installed: 0.3.9]
    - termcolor [required: ~=1.1.0, installed: 1.1.0]
  - requests [required: ~=2.12, installed: 2.19.1]
    - certifi [required: >=2017.4.17, installed: 2018.4.16]
    - chardet [required: >=3.0.2,<3.1.0, installed: 3.0.4]
    - idna [required: >=2.5,<2.8, installed: 2.7]
    - urllib3 [required: >=1.21.1,<1.24, installed: 1.23]
  - setuptools [required: >=17.0, installed: 39.2.0]
  - unidiff [required: ~=0.5.2, installed: 0.5.5]
daiquiri==1.3.0
delegator.py==0.1.0
  - pexpect [required: >=4.1.0, installed: 4.6.0]
    - ptyprocess [required: >=0.5, installed: 0.6.0]
GitPython==2.1.10
  - gitdb2 [required: >=2.0.0, installed: 2.0.3]
    - smmap2 [required: >=2.0.0, installed: 2.0.3]
pipenv==2018.7.1
  - certifi [required: Any, installed: 2018.4.16]
  - pip [required: >=9.0.1, installed: 10.0.1]
  - setuptools [required: >=36.2.1, installed: 39.2.0]
  - virtualenv [required: Any, installed: 16.0.0]
  - virtualenv-clone [required: >=0.2.5, installed: 0.3.0]
PycodestyleBear==0.10.0
  - pycodestyle [required: ==2.2, installed: 2.2.0]
PyDocStyleBear==0.10.0
  - pydocstyle [required: ==1.1, installed: 1.1.0]
pylint==1.9.2
  - astroid [required: >=1.6,<2.0, installed: 1.6.5]
    - lazy-object-proxy [required: Any, installed: 1.3.1]
    - six [required: Any, installed: 1.11.0]
    - wrapt [required: Any, installed: 1.10.11]
  - isort [required: >=4.2.5, installed: 4.3.4]
  - mccabe [required: Any, installed: 0.6.1]
  - six [required: Any, installed: 1.11.0]
pytest-cov==2.5.1
  - coverage [required: >=3.7.1, installed: 4.5.1]
  - pytest [required: >=2.6.0, installed: 3.6.3]
    - atomicwrites [required: >=1.0, installed: 1.1.5]
    - attrs [required: >=17.4.0, installed: 18.1.0]
    - more-itertools [required: >=4.0.0, installed: 4.2.0]
      - six [required: >=1.0.0,<2.0.0, installed: 1.11.0]
    - pluggy [required: >=0.5,<0.7, installed: 0.6.0]
    - py [required: >=1.5.0, installed: 1.5.4]
    - setuptools [required: Any, installed: 39.2.0]
    - six [required: >=1.10.0, installed: 1.11.0]
pytest-timeout==1.3.0
  - pytest [required: >=3.6.0, installed: 3.6.3]
    - atomicwrites [required: >=1.0, installed: 1.1.5]
    - attrs [required: >=17.4.0, installed: 18.1.0]
    - more-itertools [required: >=4.0.0, installed: 4.2.0]
      - six [required: >=1.0.0,<2.0.0, installed: 1.11.0]
    - pluggy [required: >=0.5,<0.7, installed: 0.6.0]
    - py [required: >=1.5.0, installed: 1.5.4]
    - setuptools [required: Any, installed: 39.2.0]
    - six [required: >=1.10.0, installed: 1.11.0]
snowballstemmer==1.2.1
toml==0.9.4
YAMLLintBear==0.10.0
  - yamllint [required: ==1.6.1, installed: 1.6.1]
    - pyyaml [required: Any, installed: 3.13]

Notes

For more information, see Pipfile and Pipfile.lock.

Once this issue is resolved, the issue will be automatically closed by bot.

Generate changelog in version manager

I, as a package maintainer, would like to have automatically generated changelog based on git log entries (first commit message line) on version releases. This can be automatically handled by our version manager - each time there should be issued a new release, add changelog entries into a change log file (e.g. CHANGELOG.md) that occurred between the upcoming and the current release.

Kebechet raises and exception if a dependency cannot be found in old lock

Kebechet currently raises an exception if a dependency was not present in the previous lock (but is present in the new one). Kebechet should recover from this failure - this can be caused by the fact that a user added a dependency to Pipfile/requirements.in and did not perform lock.

is this import required?

kebechet/kebechet/cli.py

Line 27 in c1c642a

from kebechet.managers import UpdateManager

Failed to update dependencies to their latest version

Automatic dependency update failed for the current master with sha ab29d90.

The automatic dependency management cannot continue. Please fix errors reported bellow.

Command

  $ pipenv update --dev

Standard output

Running $ pipenv lock then $ pipenv sync.

Standard error

Creating a virtualenv for this project...
Pipfile: /tmp/tmpgnjyg1fp/Pipfile
Using /usr/bin/python3.6m (3.6.5) to create virtualenv...
Running virtualenv with interpreter /usr/bin/python3.6m
Using base prefix '/usr'
New python executable in /tmp/tmpgnjyg1fp/.venv/bin/python3.6m
Also creating executable in /tmp/tmpgnjyg1fp/.venv/bin/python
Installing setuptools, pip, wheel...done.

Virtualenv location: /tmp/tmpgnjyg1fp/.venv
Locking [dev-packages] dependencies...

Warning: Your dependencies could not be resolved. You likely have a mismatch in your sub-dependencies.
  You can use $ pipenv install --skip-lock to bypass this mechanism, then run $ pipenv graph to inspect the situation.
  Hint: try $ pipenv lock --pre if it is a pre-release dependency.
Could not find a version that matches pydocstyle==1.1,==2.1.1 (from -r /tmp/pipenv-3js608n8-requirements/pipenv-75_iduxc-constraints.txt (line 5))
Tried: 1.0.0, 1.0.0, 1.1.0, 1.1.0, 1.1.1, 1.1.1, 2.0.0, 2.0.0, 2.1.0, 2.1.0, 2.1.1, 2.1.1, 2.1.1
There are incompatible versions in the resolved dependencies.

Environment details


Kebechet version: 1.0.0
Python version: 3.6.5
Platform: Linux-4.17.3-200.fc28.x86_64-x86_64-with-fedora-28-Twenty_Eight
pipenv version: pipenv, version 2018.7.1

Dependency graph

click==6.7
coala==0.11.0
  - appdirs [required: ~=1.4, installed: 1.4.3]
  - coala-utils [required: ~=0.6.2, installed: 0.6.7]
    - appdirs [required: ==1.4.*, installed: 1.4.3]
    - pyprint [required: ==0.2.*, installed: 0.2.6]
      - colorama [required: ~=0.3.7, installed: 0.3.9]
      - termcolor [required: ~=1.1.0, installed: 1.1.0]
  - colorlog [required: ~=2.7, installed: 2.10.0]
  - dependency-management [required: ~=0.4.0, installed: 0.4.0]
    - coala-utils [required: ~=0.6, installed: 0.6.7]
      - appdirs [required: ==1.4.*, installed: 1.4.3]
      - pyprint [required: ==0.2.*, installed: 0.2.6]
        - colorama [required: ~=0.3.7, installed: 0.3.9]
        - termcolor [required: ~=1.1.0, installed: 1.1.0]
    - sarge [required: ~=0.1.4, installed: 0.1.5.post0]
  - libclang-py3 [required: ~=3.4.0, installed: 3.4.0]
  - Pygments [required: ~=2.1, installed: 2.2.0]
  - PyPrint [required: ~=0.2.6, installed: 0.2.6]
    - colorama [required: ~=0.3.7, installed: 0.3.9]
    - termcolor [required: ~=1.1.0, installed: 1.1.0]
  - requests [required: ~=2.12, installed: 2.19.1]
    - certifi [required: >=2017.4.17, installed: 2018.4.16]
    - chardet [required: >=3.0.2,<3.1.0, installed: 3.0.4]
    - idna [required: >=2.5,<2.8, installed: 2.7]
    - urllib3 [required: >=1.21.1,<1.24, installed: 1.23]
  - setuptools [required: >=17.0, installed: 39.2.0]
  - unidiff [required: ~=0.5.2, installed: 0.5.5]
daiquiri==1.3.0
delegator.py==0.1.0
  - pexpect [required: >=4.1.0, installed: 4.6.0]
    - ptyprocess [required: >=0.5, installed: 0.6.0]
GitPython==2.1.10
  - gitdb2 [required: >=2.0.0, installed: 2.0.3]
    - smmap2 [required: >=2.0.0, installed: 2.0.3]
pipenv==2018.7.1
  - certifi [required: Any, installed: 2018.4.16]
  - pip [required: >=9.0.1, installed: 10.0.1]
  - setuptools [required: >=36.2.1, installed: 39.2.0]
  - virtualenv [required: Any, installed: 16.0.0]
  - virtualenv-clone [required: >=0.2.5, installed: 0.3.0]
PycodestyleBear==0.10.0
  - pycodestyle [required: ==2.2, installed: 2.2.0]
PyDocStyleBear==0.10.0
  - pydocstyle [required: ==1.1, installed: 1.1.0]
pylint==1.9.2
  - astroid [required: >=1.6,<2.0, installed: 1.6.5]
    - lazy-object-proxy [required: Any, installed: 1.3.1]
    - six [required: Any, installed: 1.11.0]
    - wrapt [required: Any, installed: 1.10.11]
  - isort [required: >=4.2.5, installed: 4.3.4]
  - mccabe [required: Any, installed: 0.6.1]
  - six [required: Any, installed: 1.11.0]
pytest-cov==2.5.1
  - coverage [required: >=3.7.1, installed: 4.5.1]
  - pytest [required: >=2.6.0, installed: 3.6.3]
    - atomicwrites [required: >=1.0, installed: 1.1.5]
    - attrs [required: >=17.4.0, installed: 18.1.0]
    - more-itertools [required: >=4.0.0, installed: 4.2.0]
      - six [required: >=1.0.0,<2.0.0, installed: 1.11.0]
    - pluggy [required: >=0.5,<0.7, installed: 0.6.0]
    - py [required: >=1.5.0, installed: 1.5.4]
    - setuptools [required: Any, installed: 39.2.0]
    - six [required: >=1.10.0, installed: 1.11.0]
pytest-timeout==1.3.0
  - pytest [required: >=3.6.0, installed: 3.6.3]
    - atomicwrites [required: >=1.0, installed: 1.1.5]
    - attrs [required: >=17.4.0, installed: 18.1.0]
    - more-itertools [required: >=4.0.0, installed: 4.2.0]
      - six [required: >=1.0.0,<2.0.0, installed: 1.11.0]
    - pluggy [required: >=0.5,<0.7, installed: 0.6.0]
    - py [required: >=1.5.0, installed: 1.5.4]
    - setuptools [required: Any, installed: 39.2.0]
    - six [required: >=1.10.0, installed: 1.11.0]
snowballstemmer==1.2.1
toml==0.9.4
YAMLLintBear==0.10.0
  - yamllint [required: ==1.6.1, installed: 1.6.1]
    - pyyaml [required: Any, installed: 3.13]

Notes

For more information, see Pipfile and Pipfile.lock.

Once you resolve this issue, close this report to keep the bot updating your dependencies properly.

print version on startup

Is your feature request related to a problem? Please describe.
Its hard to figure out what version of kebechet is run by a cronjob as the log does not contain a version identifier.

Describe the solution you'd like
just print version as logger.INFO

Describe alternatives you've considered
none

Additional context
looking at cronjob's pod log...

Fix URLs to raw files in reports for GitLab

Current links in reports (e.g. to Pipfile and Pipfile.lock) work only with GitHub. Reported URLs should also work with GItLab when GitLab is used.

Let users of Kebechet own configuration logic

I, as a user, would like to have full control of bot configuration. I don't want to open pull requests in this repository, but I would rather maintain my .kebechet.yaml configuration so I don't bother Kebechet upstream with configuration changes.

Ideally, we can maintain a list of repositories we manage and in each repository, there is present .kebechet.yaml with enabled managers and their configuration.

Update configuration docs

It looks like our configuration is currently out of date. Let's update it:

https://github.com/thoth-station/kebechet#configuration-of-kebechet

kebechet.exception.InternalError: Failed to retrieve version information for dependency igitt, (dev: False)

From logs:

2018-07-24 11:17:09,973 [1] DEBUG    kebechet.config: Using token '***************************************'
--
  | 2018-07-24 11:17:09,973 [1] INFO     kebechet.config: Running manager 'update' for 'thoth-station/kebechet'
  | 2018-07-24 11:17:09,973 [1] INFO     kebechet.utils: Cloning repository git@github.com:thoth-station/kebechet.git to /tmp/tmpxzawajqy
  | 2018-07-24 11:17:10,459 [1] INFO     kebechet.managers.update: Using Pipfile for dependency management
  | 2018-07-24 11:17:12,492 [1] DEBUG    kebechet.source_management: Issue 'No dependency management found' not found, not closing it
  | 2018-07-24 11:17:15,011 [1] DEBUG    kebechet.source_management: Issue 'Failed to perform initial lock of software stack' not found, not closing it
  | 2018-07-24 11:17:15,017 [1] ERROR    kebechet.config: An error occurred during run of manager 'update' <class 'kebechet.managers.update.UpdateManager'> for thoth-station/kebechet, skipping
  | Traceback (most recent call last):
  | File "/usr/local/lib/python3.6/site-packages/kebechet/config.py", line 129, in run
  | kebechet_manager(slug, ServiceType.by_name(service_type), service_url, token).run(labels)
  | File "/usr/local/lib/python3.6/site-packages/kebechet/managers/update.py", line 595, in run
  | result = self._do_update(labels, pipenv_used=True)
  | File "/usr/local/lib/python3.6/site-packages/kebechet/managers/update.py", line 490, in _do_update
  | old_direct_dependencies_version = self._get_direct_dependencies_version()
  | File "/usr/local/lib/python3.6/site-packages/kebechet/managers/update.py", line 168, in _get_direct_dependencies_version
  | version = cls._get_dependency_version(dependency, is_dev=is_dev)
  | File "/usr/local/lib/python3.6/site-packages/kebechet/managers/update.py", line 99, in _get_dependency_version
  | f"Failed to retrieve version information for dependency {dependency}, (dev: {is_dev})")
  | kebechet.exception.InternalError: Failed to retrieve version information for dependency igitt, (dev: False)

Changelog is not computed correctly

Describe the bug

The changelog that is computed is not computed correctly.

To Reproduce
Steps to reproduce the behavior:

Run version manager on any repo
Ask to release a version
Changelog is not computed correctly in the opened PR
See error

Expected behavior

Changelog should be computed.

refactor to utility method?

refactor

kebechet/kebechet/managers/info.py

Line 48 in c1c642a

INFO_REPORT.format(

into a utility method so that we can create the same report via a newly created comment we observed via a webhook

Provide git commit hashes in changelog entries

Is your feature request related to a problem? Please describe.

I would like to be pointed directly to commit/pull request that introduced the given feature stated in the generated changelog - see for example thoth-station/common#90 (hashes are missing there).

Describe the solution you'd like

Add hashes to the listing so that a user can directly go to change/pull request that introduced the given feature/fix.

Adjust build config so it does not cache images

Is your feature request related to a problem? Please describe.

I'm always frustrated when I want to propagate changes to kebechet image - I click on build image in OpenShift UI and the build is not updated as it keeps cached images. I need to trigger build manually:

oc start-build kebechet-job --no-cache=true

Describe the solution you'd like

When I click on build in the openshift console, the build is automatically started without reusing the cached image.

Describe alternatives you've considered

None.

implement Metrics manager

Is your feature request related to a problem? Please describe.
no

Describe the solution you'd like
I'd like to have kebechet push metrics about each manager and repo to prometheus

Describe alternatives you've considered
n/a

Additional context
n/a

Implement Sentry monitoring

Is your feature request related to a problem? Please describe.

We would like to capture all the exceptions happening in the implementation so that we are sure nothing bad is happening.

Describe the solution you'd like

Sentry integration implemented as in thoth-station/common#108

Describe alternatives you've considered

None.

Additional context

None.

change owners file...

@harshad16 do you want to add yourself to the reviewers section of the OWNERS file?

New patch release

Handle Exception on config error

Describe the bug


2018-07-28 05:58:57,425 [1] ERROR    kebechet.config: An error occurred during run of manager {} <class 'kebechet.managers.version.version.VersionManager'> for thoth-station/storages, skipping
--
  | Traceback (most recent call last):
  | File "/usr/local/lib/python3.6/site-packages/kebechet/config.py", line 168, in run
  | instance.run(**manager_configuration)
  | TypeError: run() got an unexpected keyword argument 'assignees'

Expected behavior
just log the exception...

Additional context
This was code before #118 seeing a configfile after #118

Kebechet should delete branches that have been merged

As Kebechet creates branches to update dependencies, it should delete branches which have been merged...

refactor out an ansible role

have a look at https://github.com/goern/thoth-kebechet-bot and fork it to create an ansible role.

AttributeError: 'GitHubIssue' object has no attribute 'commnets'

From logs:

2018-07-24 11:15:58,400 [1] DEBUG    kebechet.source_management: Reporting issue 'Failed to update dependencies to their latest version'
--
  | 2018-07-24 11:15:58,929 [1] INFO     kebechet.source_management: Issue already noted on upstream with id #58
  | 2018-07-24 11:15:58,981 [1] ERROR    kebechet.config: An error occurred during run of manager 'update' <class 'kebechet.managers.update.UpdateManager'> for thoth-station/storages, skipping
  | Traceback (most recent call last):
  | File "/usr/local/lib/python3.6/site-packages/kebechet/managers/update.py", line 492, in _do_update
  | self._pipenv_update_all()
  | File "/usr/local/lib/python3.6/site-packages/kebechet/managers/update.py", line 390, in _pipenv_update_all
  | cls.run_pipenv('pipenv update --dev')
  | File "/usr/local/lib/python3.6/site-packages/kebechet/managers/manager.py", line 99, in run_pipenv
  | raise PipenvError(result)
  | kebechet.exception.PipenvError
  | During handling of the above exception, another exception occurred:
  | Traceback (most recent call last):
  | File "/usr/local/lib/python3.6/site-packages/kebechet/config.py", line 129, in run
  | kebechet_manager(slug, ServiceType.by_name(service_type), service_url, token).run(labels)
  | File "/usr/local/lib/python3.6/site-packages/kebechet/managers/update.py", line 595, in run
  | result = self._do_update(labels, pipenv_used=True)
  | File "/usr/local/lib/python3.6/site-packages/kebechet/managers/update.py", line 505, in _do_update
  | labels=labels
  | File "/usr/local/lib/python3.6/site-packages/kebechet/source_management.py", line 81, in open_issue_if_not_exist
  | comment_body = refresh_comment(issue)
  | File "/usr/local/lib/python3.6/site-packages/kebechet/managers/update.py", line 399, in _add_refresh_comment
  | for issue_comment in issue.commnets:
  | AttributeError: 'GitHubIssue' object has no attribute 'commnets'

Group fixes in the changelog

As a followup from #146, we could generate a new part of changelog that would group fixes done in the release. This can be based based on "fixes ..." stated in the PR message/git commit as linked on GitHub/GitLab.

Example:

* Fixes in this release:
   * Handle Exception on config error #120
   * Explicitly perform lock if the given package was not found in the lock #82

Extend version manager with automatic semantic versioning

I, as a package maintainer, don't want to know what is the current version of package to bump it - e.g. by opening issue "1.0.3 release". Instead, I want to just open issue "New major release", "New minor release", "New patch release" respecting semver (optionally with "release candidate" for all of them - for example "New major release candidate"). This way I don't need to explicitly say what version I want to have and the version manager will do all the version magic for me.

Kebechet should open an issue if the given environment cannot be replicated

From Kebechet logs:

2018-07-06 19:23:39,999 [1] INFO     kebechet.update: Cloning repository [email protected]:thoth-station/package-extract.git to /tmp/tmpevoy6kq6
2018-07-06 19:23:43,665 [1] INFO     kebechet.update: Using Pipfile for dependency management
2018-07-06 19:23:43,671 [1] INFO     kebechet.update: Updating all dependencies to their latest version
2018-07-06 19:23:43,671 [1] DEBUG    kebechet.update: Running pipenv command 'pipenv update --dev'
2018-07-06 19:24:05,918 [1] DEBUG    kebechet.update: Running pipenv command 'pipenv lock'
2018-07-06 19:24:14,780 [1] DEBUG    kebechet.update: Issue 'Failed to update dependencies to their latest version' not found, not closing it
2018-07-06 19:24:14,783 [1] DEBUG    kebechet.update: Found new update for thoth-common: 0.0.8 -> 0.1.0 (dev: False)
2018-07-06 19:24:14,783 [1] DEBUG    kebechet.update: Found new update for thoth-analyzer: 0.0.6 -> 0.0.7 (dev: False)
2018-07-06 19:24:14,783 [1] DEBUG    kebechet.update: Found new update for pytest: 3.6.2 -> 3.6.3 (dev: True)
2018-07-06 19:24:14,783 [1] INFO     kebechet.update: Outdated: {'thoth-common': {'dev': False, 'old_version': '0.0.8', 'new_version': '0.1.0'}, 'thoth-analyzer': {'dev': False, 'old_version': '0.0.6', 'new_version': '0.0.7'}, 'pytest': {'dev': True, 'old_version': '3.6.2', 'new_version': '3.6.3'}}
2018-07-06 19:24:15,794 [1] DEBUG    kebechet.update: No pull request was found for update of thoth-common to version 0.1.0
2018-07-06 19:24:15,795 [1] INFO     kebechet.update: Replicating old environment for incremental update
2018-07-06 19:24:15,795 [1] DEBUG    kebechet.update: Running pipenv command 'pipenv sync --dev'
2018-07-06 19:24:25,354 [1] ERROR    kebechet.update:   Could not find a version that satisfies the requirement pyyaml==4.1 (from -r /tmp/pipenv-mar91aye-requirements/pipenv-h4g097xy-requirement.txt (line 1)) (from versions: 3.10, 3.11, 3.12, 3.13b1, 3.13rc1, 3.13, 4.2b1, 4.2b2, 4.2b4)
No matching distribution found for pyyaml==4.1 (from -r /tmp/pipenv-mar91aye-requirements/pipenv-h4g097xy-requirement.txt (line 1))

We should open an issue, try to re-lock dependencies and open a PR for this automatically. PyYAML==4.1 was removed from PyPI.

Add support for managing lock files in nested directories

As a user of Kebechet I would like to make sure Kebechet also manages nested Pipfile.lock and requirements.txt files. This can be accomplished either by searching for files matching the given filename or explicitly by stating files to be monitored in the configuration file (the latter is preferred for us).

Failed to perform initial lock of software stack

Failed to perform initial lock of your dependencies based on your Pipfile.

See attached report below to inspect this issue.

Command

  $ pipenv lock

Standard output

Standard error

Creating a virtualenv for this project...
Pipfile: /tmp/tmpei23a5hg/Pipfile
Using /usr/bin/python3.6m (3.6.6) to create virtualenv...
Running virtualenv with interpreter /usr/bin/python3.6m
Using base prefix '/usr'
New python executable in /tmp/tmpei23a5hg/.venv/bin/python3.6m
Also creating executable in /tmp/tmpei23a5hg/.venv/bin/python
Installing setuptools, pip, wheel...done.

Virtualenv location: /tmp/tmpei23a5hg/.venv
Locking [dev-packages] dependencies...
Locking [packages] dependencies...

Warning: Your dependencies could not be resolved. You likely have a mismatch in your sub-dependencies.
  You can use $ pipenv install --skip-lock to bypass this mechanism, then run $ pipenv graph to inspect the situation.
  Hint: try $ pipenv lock --pre if it is a pre-release dependency.
Could not find a version that matches gitpython==1.0.1,==2.1.11 (from -r /tmp/pipenv-byl_c4ht-requirements/pipenv-dclvixhy-constraints.txt (line 8))
Tried: 0.1.7, 0.3.2, 0.3.2.1, 0.3.3, 0.3.4, 0.3.5, 0.3.6, 0.3.7, 1.0.0, 1.0.1, 1.0.2, 2.0.0, 2.0.1, 2.0.1, 2.0.2, 2.0.2, 2.0.3, 2.0.3, 2.0.4, 2.0.4, 2.0.5, 2.0.5, 2.0.6, 2.0.6, 2.0.7, 2.0.8, 2.0.9, 2.0.9, 2.1.0, 2.1.0, 2.1.1, 2.1.1, 2.1.3, 2.1.3, 2.1.4, 2.1.4, 2.1.5, 2.1.5, 2.1.6, 2.1.6, 2.1.7, 2.1.7, 2.1.8, 2.1.8, 2.1.9, 2.1.9, 2.1.10, 2.1.10, 2.1.11, 2.1.11
Skipped pre-versions: 0.2.0b1, 0.3.0b1, 0.3.0b2, 0.3.1b2, 0.3.2rc1, 2.0.9.dev0, 2.0.9.dev1
There are incompatible versions in the resolved dependencies.

Environment details


Kebechet version: 1.0.0
Python version: 3.6.6
Platform: Linux-3.10.0-862.9.1.el7.x86_64-x86_64-with-fedora-28-Twenty_Eight
pipenv version: pipenv, version 2018.7.1

rebase feature on master

Is your feature request related to a problem? Please describe.
Sometimes the auto merge pipeline is blocked by outdated feature branches. It would be nice if Kebechet could rebase the branch on master (and update the PR).

Describe the solution you'd like
...

Describe alternatives you've considered
rebase manually

add Sesheta

please let Kebechet take care of Sesheta

check if PR already exists

As Kebechet
I want to check if a PR that I want to open already exists,
so that I do not open a PR twice: one closed and one new opened.

Group updates in the changelog entries

As of now we produce changelog that is solely made of git commit messages. It would be nice to provide a well structured changelog - the very first step could be grouping of updates done by bots into a single entry - something like:

* Dependency updates:
  * dep1 (old version -> new version)
  * dep2 (old version -> new version)

1.0.1 release

Key error on configuration entries

  | 2018-07-26 09:53:17,124 [1] DEBUG    kebechet: Debug mode turned on
-- | --
  | 2018-07-26 09:53:17,273 [1] DEBUG    kebechet.config: Using token '569*************************************'
  | 2018-07-26 09:53:17,273 [1] INFO     kebechet.config: Running manager 'update' for 'thoth-station/adviser'
  | 2018-07-26 09:53:17,274 [1] INFO     kebechet.utils: Cloning repository git@github.com:thoth-station/adviser.git to /tmp/tmpv9jx0hzn
  | 2018-07-26 09:53:17,720 [1] INFO     kebechet.managers.update.update: Using Pipfile for dependency management
  | 2018-07-26 09:53:18,108 [1] DEBUG    kebechet.source_management: Issue 'No dependency management found' not found, not closing it
  | 2018-07-26 09:53:18,544 [1] DEBUG    kebechet.source_management: Issue 'Failed to perform initial lock of software stack' not found, not closing it
  | 2018-07-26 09:53:18,547 [1] INFO     kebechet.managers.update.update: Updating all dependencies to their latest version
  | 2018-07-26 09:53:18,547 [1] DEBUG    kebechet.managers.manager: Running pipenv command 'pipenv update --dev'
  | 2018-07-26 09:55:58,214 [1] DEBUG    kebechet.managers.manager: Running pipenv command 'pipenv lock'
  | 2018-07-26 09:56:09,472 [1] DEBUG    kebechet.source_management: Issue 'Failed to update dependencies to their latest version' not found, not closing it
  | 2018-07-26 09:56:09,474 [1] INFO     kebechet.managers.update.update: Outdated: {}
  | 2018-07-26 09:56:09,884 [1] DEBUG    kebechet.source_management: Issue 'Failed to replicate environment for updates' not found, not closing it
  | 2018-07-26 09:56:10,655 [1] INFO     kebechet.config: Running manager 'info' for 'thoth-station/adviser'
  | 2018-07-26 09:56:11,226 [1] INFO     kebechet.managers.info.info: No issue to report to, exiting
  | 2018-07-26 09:56:11,226 [1] INFO     kebechet.config: Running manager 'version' for 'thoth-station/adviser'
  | 2018-07-26 09:56:11,226 [1] ERROR    kebechet.config: An error occurred during run of manager {} <class 'kebechet.managers.version.version.VersionManager'> for thoth-station/adviser, skipping
  | Traceback (most recent call last):
  | File "/usr/local/lib/python3.6/site-packages/kebechet/config.py", line 139, in run
  | instance.run(**manager_configuration)
  | TypeError: run() got an unexpected keyword argument 'maintainers'

Hint: Most probably this is caused by poping things from config:

2018-07-26 09:56:11,227 [1] ERROR    kebechet.config: No manager name provided in configuration entry for thoth-station/cleanup-job, ignoring entry
--
  | Traceback (most recent call last):
  | File "/usr/local/lib/python3.6/site-packages/kebechet/config.py", line 122, in run
  | manager_name = manager.pop('name')

Start using PyPI for releases

Currently, we install Kebechet directly from source repo that causes sometimes failures - as containers are cached, we sometimes end up with old code in container. We should start do proper releases (now with version manager, that should be transparent) and trigger OpenShift generic trigger to build based on PyPI release.

	# TODO: uncomment once AICoE index will be fixed
	#url = f"{source_url}/{package_name}/"
	_LOGGER.debug(f"Discovering package %r artifacts from %r", package_name, url)
	response = requests.get(url, verify=verify_tls)
	response.raise_for_status()
	soup = BeautifulSoup(response.text, 'lxml')

thoth-station / kebechet Goto Github PK

kebechet's Introduction

Kebechet

Features

Contribute

Support

License

Full documentation

kebechet's People

Contributors

Stargazers

Watchers

Forkers

kebechet's Issues

This issue was generated by todo based on a TODO comment in 454a096 when #135 was merged. cc @fridex.

Command

Notes

Command

Notes

Command

Recommend Projects

Recommend Topics

Recommend Org

This issue was generated by todo based on a `TODO` comment in `454a096` when #135 was merged. cc @fridex.