pyupio / pyup Goto Github PK

My invoices are going straight to junk. Please fix.

(this might be the wrong place to report this, sorry if that's the case)

Environment markers are currently not supported:

I use Environment Markers (PEP 496) in my requirements.txt. But it seems pyup-bot doesn't know what it is and deletes them, for example in this PR: https://github.com/bayandin/devtools-proxy/pull/16/files. Hope it will be possible to keep the environment markets in requirements.txt.

pyup-bot's initial PR updated pytest-html from 1.13.0 to 1.14.2.

The new version has apparently added a dependency on pytest-metadata, which means the Travis run failed with:

In --require-hashes mode, all requirements must have their versions pinned with ==. These do not:
    ...
    pytest-metadata from https://pypi.python.org/packages/71/33/9037033094e0f8da4589ab147bf3d931746f9a322300e705e49c84db5ed1/pytest_metadata-1.3.0-py2.py3-none-any.whl#md5=f9ad72f4d02315c99390f673c5bbd582 (from pytest-html==1.14.2->-r requirements/dev.txt (line 84))

(https://travis-ci.org/mozilla/treeherder/jobs/219388305)

To fix, the new package will need adding to the requirements file, along with an appropriate version number and hash.

This may be beyond the scope of pyup, but filing just in case it's something you'd want to support.

@pyup-bot is too noizy creating and closing new issues. It will help if it could update it's own issues, and instead of "Update sphinx to 1.4.2 by pyup-bot" create "Autoupdate of sphinx [1.4.0 -> 1.4.2]" and update the header as new versions are coming out.

If I'm not wrong, the issue here is that the bot pushes to the created branch after every commit. If all commits were done first, and then the branch pushed, Travis would only trigger one build for the push. Does this sound reasonable? This behavior should be an easy thing to configure with a setting.

Have you thought about adding other non-django requirement files into the pyup check? Most specifically I'm talking about the requirements file provided by hitchtest, which is found in "/hitch/hitchreqs.txt".

Maybe an option to define other folders/files within the project would do the job!

Running pyup --repo=****/***** --user-token=******* --initial

Traceback (most recent call last):
  File "/usr/bin/pyup", line 7, in <module>
    from pyup.cli import main
  File "/usr/lib/python2.7/site-packages/pyup/cli.py", line 3, in <module>
    from pyup.bot import DryBot, Bot
  File "/usr/lib/python2.7/site-packages/pyup/bot.py", line 5, in <module>
    from .requirements import RequirementsBundle
  File "/usr/lib/python2.7/site-packages/pyup/requirements.py", line 4, in <module>
    from pkg_resources._vendor.packaging.specifiers import SpecifierSet

Was able to fix the error by installing an older version of setuptools (it isn't defined in your setup.py)

@pyup-bot's PR descriptions and https://pyup.io/ use the name "PyPi", but all of the branding on https://pypi.python.org/pypi and the new https://pypi.org/ refer to the package repository "PyPI (Python Package Index)".

Trying to sync my repos I'm getting an alert: "An unknown error occurred. Please contact support at [email protected]."

Looking in the console, I'm getting a 500 response from https://pyup.io/account/repos/fetch-repos/ sure enough if I go to that url I get 500 too.

I got this the other day, I think it was intermittent then but seems to be constant now.

Private repository (private access enabled obviously). I pushed a new branch 'develop' to my repository and changed it to be the default. I closed a pull request and deleted the branch, however the new PR was still against the old 'master' branch (which is no longer default and has been set to be a protected branch). I have tried removing the repository from pyup.io and re-adding it and I still got the same behavior.

I had to add a .pyup.yml file with branch = develop to get the correct behavior but the docs state that pyup makes PR's against the default branch on the repository (usually master)

I need requirements in setup.py in the install_requires config setting, but these aren't picked up in setup.py.

I know its probably difficult, but do you have any plans on support for this?

Our requirements file contains the line (link):
taskcluster==0.3.4 --hash sha256:d4fe5e2a44fe27e195b92830ece0a6eb9eb7ad9dc556a0cb16f6f2a6429f1b65

However the pyup overview (https://pyup.io/account/repos/github/mozilla/treeherder/) does not list that package, and the bot hasn't opened a PR to update to the newer 1.2.0 release.

The package can be found here:
https://pypi.python.org/pypi/taskcluster
https://github.com/taskcluster/taskcluster-client.py

I'm interested to adopt pyup for my gitlab hosted projects.
Future plans include support to different git hosting providers?

I'm not sure I'm able to engage this task, but where I can find some info on how to start?
I would need only to properly create a pyup/providers/gitlab.py file?

Hi,

We've had a feature request from one of our devs regarding PRs generated by pyup. Several of our teams are using Waffle.io for tracking PRs and Issues and would like to be able define a repo-specific label to be applied to all PRs generated by pyup for that repo. I think the workflow would look like this:

• Add repo in pyup
• On the repo specific screen, an optional (unused if empty) label field would be available
• If updated with a value, any existing PRs generated by pyup would have that label edited on to the PRs and new PRs would be generated with the label value

I believe that pygithub has support for all of this already from a quick perusal. I've sent this to our dev to confirm that would be his expected workflow. Does this sound in-spec for pyup?

Caveat: this would ideally be open to accounts with org perms, rather than only limited to the original creation account.

After adding a repo, the initial PR was opened updating 48 packages. Unfortunately some of those packages shouldn't be updated, so will need filtering (I would have done that first, but for #207).

In the initial PR description it says:

This is my first visit to this fine repo so I have bundled all updates in a single pull request to make things easier for you to merge.
Close this pull request and delete the branch if you want me to start with single pull requests right away

So if I just deleted the branch, I'd end up with 40+ PRs being opened, which is pretty spammy.

Instead, I'd just like the initial PR to be updated to take into account the filters, but it's not clear how to achieve that. (Perhaps by deleting the repo from my pyup account page and re-adding?)

Just saw this in an Initial Update message from @pyup-bot:

Once you have closed this issue, I'll create seperate pull requests for every update as soon as I find one.

It should be spelt “separate”, not “seperate”. I’d raise a PR for it, but I can’t actually find where this string comes from in the code.

The github provider raises NoPermissionErrors on all github exceptions. This should be fixed.

Pipfiles are on its way to be a thing.

It'd great if pyup could support this as soon as the internal API is stable: pypa/pipfile#57

Taking the example from #50 further, if your package does depend on requests, and you don't want a downstream lib preventing you from using a new version.

your-package-requirements.txt

some-package==1.0.3
requests>=2.7  # some-package uses requests==2.5, but I want a newer version

This may cause a version conflict in pip unless some-package has requests>=2.5,<3.0 in requirements.txt. But now that requests isn't "pinned" in some-package, the security checks don't work.

I read that you support filters, so some-package could do

requests  # pyup: >=2.5,<3.0

But that would require rewriting all ranged requirements. Would there be a downside to natively recognizing?

requests>=2.5,<3.0

I had read the pyup docs and so knew we'd need to add filters to our requirements files, since not all packages are safe to update (eg Celery 4 causes a few regressions for us).

However it wasn't clear that immediately after adding the repository the initial PR would be opened before I'd had a chance to add the filter statements to our requirements files.

Ideally the "add a repo" flow would either:

1. Make it clearer that the initial PR will be opened straight away, so if filters are required, the user should add them first
2. Make adding a repo not create the initial PR, until a 2nd step is completed

As-is I now have to delete the initial PR, but apparently that will create multiple separate PRs, which will be really spammy. (Perhaps I can delete the repo from my pyup account and start over, to get the combined initial PR again? I'll file a separate issue for this.)

When I go to https://pyup.io/account/repos/add/ and click "Add" next to any repo nothing happens.

Opening the inspector shows this error:

Tested on Chrome & Firefox on Windows 10.

There has been a lot of discussion during the beta if packages that have a >= range should be pinned. This came up recently again and I think it should be discussed.

What is a >= range?
A line with e.g. requests>=2.7 tells pip to install requests with at least 2.7 and everything above that.

What's happening
The updater won't pin the package.

Why?
There is a use case that makes the current behaviour pretty neat: Indirect dependencies.
Let's consider the following fictional example: A codebases has a direct dependency some-package.

some-package=1.0.3

some-package is pulling in a second, indirect dependency requests. This happens all the time and is nothing to worry about per se, but can lead to problems if some-package's setup.py is not very well maintained and if a specific version of requests is known broken with it.

This can be fixed on the project level with a requirement file like this:

some-package=1.0.3
requests>=2.7 # some package has problems with requests <=2.7, just make sure we don't use that

It indicates that the project itself doesn't really care about requests and just want to make sure that everything above 2.7. is installed.

On https://pyup.io/account/repos/add/, cookiecutter-pypackage doesn't show up even when I click "Reload" or search the page.

Perhaps I have too many repos?

In this PR:
mozilla/treeherder#2370

The diff is:

diff --git a/requirements/common.txt b/requirements/common.txt
index 273ce35..841006d 100644
--- a/requirements/common.txt
+++ b/requirements/common.txt
@@ -9,7 +9,9 @@ Brotli==0.5.2 --hash=sha256:3411b9acd2a2056e55084acf7a6ab3e4a8540c2ef37a4435bca6
 
 Django==1.10.7 --hash=sha256:e68fd450154ad7ee2c88472bb812350490232462adc6e3c6bcb544abe5212134  # pyup: <1.11 # Bug 1353561
 
-celery==3.1.24 --hash=sha256:25396191954521184cc15018f776a2a2278b04dd4213d94f795daef4b7961b4b  # pyup: <4 # Bug 1337717
+celery==3.1.25  # pyup: <4 # Bug 1337717 \
+    --hash=sha256:1954a224805f3835e5b6f5998ec9fe51db3413cc49e59fc720d314c7913427cf \
+    --hash=sha256:6ced63033bc663e60c992564954dbb5c84c43899f7f1a04b739957350f6b55f3
 
 kombu==3.0.37 --hash=sha256:7ceab743e3e974f3e5736082e8cc514c009e254e646d6167342e0e192aee81a6  # pyup: <4 # Bug 1337717

Unfortunately adding the filter comments mid package definition confuses pip, causing the following failure:

Hashes are required in --require-hashes mode, but they are missing from some requirements. Here is a list of those requirements along with the hashes their downloaded archives actually had. Add lines like these to your requirements files to prevent tampering. (If you did not enable --require-hashes manually, note that it turns on automatically when any package has a hash.)

    celery==3.1.25 --hash=sha256:1954a224805f3835e5b6f5998ec9fe51db3413cc49e59fc720d314c7913427cf

(https://travis-ci.org/mozilla/treeherder/jobs/223178218)

If the project uses github (or similar) and it has tags for the releases, it would be possible to include a link to a page that shows a diff for all the code changes between the two releases. eg this link (https://github.com/twilio/twilio-python/compare/5.6.0...5.7.0)

Even better (but would require a new service) would be something that automatically shows a visual diff between two given versions of the downloaded source from PyPI. That is, it would be a real release comparison, not one relying on accurate tags in the repo.

With github integration, we're getting constant pull requests "Update openpyxl to 2.4.3".

That version doesn't exist on pypi. We've tried closing the PR and deleting the branch and another PR is always created.

https://pyup.io/changelogs/openpyxl/ doesn't match https://pypi.python.org/pypi/openpyxl/json at all. Lots of version (2.5, 3.2, 3.3) don't exist on pypi.

The Requirement class has a property called is_insecure which is currently not implemented.

Currently, there's no central place to query for known security vulnerabilites.

pypi/warehouse#798

I was hoping to use https://pyup.io/tools/requirements-checker/ to preview the changes that would be made to a requirements file that is currently ignored (using # pyup: ignore file), so I could check my filter lines (eg # pyup: <4) were correct before unleashing the PRs.

However after pasting in this requirements file (with the ignore line omitted):
https://github.com/mozilla/treeherder/blob/ab82e6152635b8150f4a612bdba5be96131406c0/requirements/common.txt

...I get:
"Ensure this value has at most 6000 characters (it has 6111)."

I had to manually remove a couple of comment lines to get it under the limit.

With pip hashes, requirements files can now be reasonably large, so the limit is probably a bit low.

Tested against 0.5.0, supposedly master is affected too.

  File "/Users/prophet/.envs/mybook/bin/pyup", line 11, in <module>
    sys.exit(main())
  File "/Users/prophet/.envs/mybook/lib/python2.7/site-packages/click/core.py", line 716, in __call__
    return self.main(*args, **kwargs)
  File "/Users/prophet/.envs/mybook/lib/python2.7/site-packages/click/core.py", line 696, in main
    rv = self.invoke(ctx)
  File "/Users/prophet/.envs/mybook/lib/python2.7/site-packages/click/core.py", line 889, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/prophet/.envs/mybook/lib/python2.7/site-packages/click/core.py", line 534, in invoke
    return callback(*args, **kwargs)
  File "/Users/prophet/.envs/mybook/lib/python2.7/site-packages/pyup/cli.py", line 44, in main
    bot.update(branch=branch, initial=initial, pin=pin, close_prs=close_prs)
  File "/Users/prophet/.envs/mybook/lib/python2.7/site-packages/pyup/bot.py", line 94, in update
    scheduled=kwargs.get("scheduled", False)
  File "/Users/prophet/.envs/mybook/lib/python2.7/site-packages/pyup/bot.py", line 155, in apply_updates
    assignees=self.config.assignees
TypeError: commit_and_pull() got an unexpected keyword argument 'pr_label'

Currently, the #rq.filter filter from requires.io is used but not documented.

Todo:

• document the # rq.filter: filter
• add a # pyup: filter using the # requires: rules
• add a filter that completely excludes a requirement
• add a filter that completely excludes a whole file

Currently you only accept a token, but for the version of github used internally this is not a valid login method (a username is also required).

Could you add an option to also specify a username with the token, I checked into pygithub and it already supports both username, password along with token.

I updated python-pdf three days ago (16th) however no pull request has been created for the change and nothing shows in the pyup interface.

The update was from v0.23 to v0.3.

Unfortunately the repo using python-pdf is private so I can't provide a link to the lack of PR.

Possibly related: the badge from shields.io hasn't updated either. I guess it's remotely possibly you use the same mechanism of prompting updates and pypi had a problem where the prompt wasn't fired for this package?

The documentation says I can create a .pyup.yml to control which requirements files are updated as I did here:

search: False

requirements:
  - requirements.txt

branch: python2-pytest

but @pyup-bot still created PRs for all my nested requirements files:

Are configuration files actually supported right now?

One feature I miss from requires.io is the ability to jump straight to PyPI and/or the package repo from eg https://pyup.io/account/repos/github/mozilla/treeherder/

Currently if the changelog is missing, I have to Google "pypi {package name}" to get the PyPI page.

Branches where the commit failed for whatever reason should be deleted.

For example, requests[security] becomes requests==2.11.1 rather than requests[security]==2.11.1. See, e.g. https://github.com/jimr/noterator/pull/2/files

After signing up and adding a repo, I used the "Requirements" button, which takes me to:
https://pyup.io/account/repos/github/mozilla/treeherder/

However that page HTTP 500s.

A .pyup.yml configuration file is needed that holds some basic information about the project.

As a start, it'd be good to have a pointer to the requirement files the project is using that overrides the basic search function.

I'm not sure where the Python 3 compatibility logic is, but for, e.g bumpversion it seems incorrect; pyup reports it as incompatible but the package claims Python 3 support: https://github.com/peritus/bumpversion/blob/master/setup.py#L35 vs https://pyup.io/account/repos/github/jimr/Showcase/

One thing that gets me though is that your generated files just have a space before the comment in the requirements file, where as mine have tabs, leading to diffs similar to this

-Jinja2==2.9.4 # via flask
+jinja2==2.9.4 # via flask

cc @jimjkelly

Hey, @jayfk! I like pyup idea and want to use pyup. Here are problems that blocked my progress to making any pull requests, because not being able to have a green tests:

1. Tox build is broken with error Could not detect requirement name, please specify one with #egg= (hashin is not installed)
   https://travis-ci.org/pyupio/pyup/jobs/195612487#L219

2. python setup.py test fails under python <3.5: AttributeError: 'module' object has no attribute 'test_bot'

Traceback (most recent call last):
  File "setup.py", line 68, in <module>
    tests_require=test_requirements,
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/distutils/core.py", line 148, in setup
    dist.run_commands()
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/distutils/dist.py", line 955, in run_commands
    self.run_command(cmd)
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "/Users/prophet/work/forks/pyup/.tox/py34/lib/python3.4/site-packages/setuptools/command/test.py", line 211, in run
    self.run_tests()
  File "/Users/prophet/work/forks/pyup/.tox/py34/lib/python3.4/site-packages/setuptools/command/test.py", line 234, in run_tests
    **exit_kwarg
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/unittest/main.py", line 92, in __init__
    self.parseArgs(argv)
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/unittest/main.py", line 139, in parseArgs
    self.createTests()
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/unittest/main.py", line 146, in createTests
    self.module)
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/unittest/loader.py", line 146, in loadTestsFromNames
    suites = [self.loadTestsFromName(name, module) for name in names]
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/unittest/loader.py", line 146, in <listcomp>
    suites = [self.loadTestsFromName(name, module) for name in names]
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/unittest/loader.py", line 117, in loadTestsFromName
    return self.loadTestsFromModule(obj)
  File "/Users/prophet/work/forks/pyup/.tox/py34/lib/python3.4/site-packages/setuptools/command/test.py", line 43, in loadTestsFromModule
    tests.append(self.loadTestsFromName(submodule))
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/unittest/loader.py", line 114, in loadTestsFromName
    parent, obj = obj, getattr(obj, part)
AttributeError: 'module' object has no attribute 'test_bot'

3. Setuptools internal import does not work with latest setuptools #192

pyup --repo=Eksmo/eksmo --user-token=mytoken --initial crashed cli (version 0.5.0)

Traceback (most recent call last):
  File "/Users/prophet/.envs/mybook/bin/pyup", line 11, in <module>
    sys.exit(main())
  File "/Users/prophet/.envs/mybook/lib/python2.7/site-packages/click/core.py", line 716, in __call__
    return self.main(*args, **kwargs)
  File "/Users/prophet/.envs/mybook/lib/python2.7/site-packages/click/core.py", line 696, in main
    rv = self.invoke(ctx)
  File "/Users/prophet/.envs/mybook/lib/python2.7/site-packages/click/core.py", line 889, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/prophet/.envs/mybook/lib/python2.7/site-packages/click/core.py", line 534, in invoke
    return callback(*args, **kwargs)
  File "/Users/prophet/.envs/mybook/lib/python2.7/site-packages/pyup/cli.py", line 44, in main
    bot.update(branch=branch, initial=initial, pin=pin, close_prs=close_prs)
  File "/Users/prophet/.envs/mybook/lib/python2.7/site-packages/pyup/bot.py", line 94, in update
    scheduled=kwargs.get("scheduled", False)
  File "/Users/prophet/.envs/mybook/lib/python2.7/site-packages/pyup/bot.py", line 117, in apply_updates
    self.iter_updates(initial, scheduled)
TypeError: iter_updates() takes exactly 2 arguments (3 given)

When accessing pyup from a non-logged in browser, you end up on a page without any "bail out" behavior, e.g. no links to homepage or the account login. My particular URL structure was of this type:
https://pyup.io/repos/github/org/repo/commits/commit/

Out of the two available options already on page, the large logo is a good candidate or the small pyup in the bottom left

<div class="padding-top-40">
    <img src="https://pyup.io/static/images/logo.png" class="img img-responsive"/>
</div>
...
<div class="col-md-6">
    pyup
</div>

STR:

1. Add https://github.com/mozilla/treeherder to pyup.io
2. Wait for the initial PR test to complete
3. Notice that one of the updates causes a test failure (in this case, isort was updated meaning it now catches a lint case it missed before)
4. Add a commit to the pyup-initial-update branch locally to fix the issue
5. Use interactive rebase to (a) move the test-fixup commit to before the others (since that way if bisected, the repository always passes tests at any revision), (b) tweak the commit message for the pyup-bot commits
6. Force push to the branch to update the PR

Expected:
pyup.io bot does nothing until the PR is merged/closed.

Actual:
pyup.io bot opens 18 PRs (one for each of the dependencies in the initial PR), which is pretty spammy.

There's an issue with the github API when uploading new file contents too fast.

GithubException: 409 {'documentation_url': 'https://developer.github.com/v3/repos/contents/', 'message': 'refs/heads/pyup-initial-update expected to be at ab6461b0bffc5323bbeba454795bcb8db5e8d6d2'}
(5 additional frame(s) were not displayed)
...
  File "pyup/bot.py", line 117, in commit_and_pull
committer=self.bot if self.bot_token else self.user,
  File "pyup/providers/github.py", line 65, in create_commit
committer=self.get_committer_data(committer),
  File "github/Repository.py", line 1086, in update_content
input=post_parameters
  File "github/Requester.py", line 171, in requestJsonAndCheck
return self.__check(*self.requestJson(verb, url, parameters, headers, input, cnx))
  File "github/Requester.py", line 179, in __check
raise self.__createException(status, responseHeaders, output)

More background on this: http://stackoverflow.com/questions/19576601/github-api-issue-with-file-upload

PyGithub is currently installed from git because the version in pypi does not support the update_content function.

There's a PR open here: PyGithub/PyGithub#316

It would be awesome if it could show changelog information.

There are a bunch of services on the homepage of http://shields.io/ that allow you to customize the style of the shield. We like to use the flat-squared style over at dj-stripe; it'd be great to be able to use the pyup badge