Bitbucket private repos about git-updater HOT 49 CLOSED

I'm still having issues with this. I can get notifications but actually updating isn't quite working. I've got a couple more ideas to try.

from git-updater.

@pdclark, I'm at a loss. There was one time, unless I'm being delusional, when I thought I actually got a private repo to update. I can't get it done now. I do have http_request_args passing Basic Authentication headers correctly.

from git-updater.

You are doing a wonderful job there... I will be finding a gap and try to solve this problem with you :)

from git-updater.

Thanks @nadimtuhin I can use the help on this one. I'm at an impasse.

from git-updater.

https://bitbucket.org/site/master/issue/1087/private-repository-public-downloads-bb-733

Justen Stepka
changed status to wontfix
We have decided that we will not be implementing public downloads for repos with private source. Our goal is to build a product where folks collaborate around writing source code and not build a distribution platform for binary files. There are several low cost services that are optimized for distributing binary files, such as Amazon S3 or Rackspace which we recommend.
Respectfully, Justen -- Bitbucket product manager
6/14/2014

It looks as if support for Bitbucket private repos is not going to be possible. Sorry.

@jazzsequence @pdclark

from git-updater.

Looks like that just says that for a repo that requires authentication, the zip will require authentication as well. Doesn't mean updates for private repos aren't possible; just that they have to be downloaded with an authorized user account.

from git-updater.

But Paul I think they mean a logged in user account, not just authorize. We are both using Basic Auth and it's not working.

from git-updater.

Like it's giving you an access denied error?

If that's the case, then it may just mean authorizing with oAuth, which then puts it in the same class as private Github repos — difficult, but possible.

from git-updater.

I think the only way to do Bitbucket private repos is with oAuth. They don't make an access token like GitHub and it doesn't look like they're going too.

Creating a Bitbucket oAuth token might be a whole project to itself.

On another note, Paul it's great to hear from you.

from git-updater.

Nice to be here! Sorry my new job has had me under water for months. Just released three WordPress sites for ted.com.

Agreed that oAuth certainly isn't trivial. There do tend to be a lot of libraries that make it much easier. Not sure about the access token. I know I write a Github oAuth downloader at one time, but can't remember if the access token for downloads was Github-specific or part of the oAuth spec (which does provide something called an access token).

from git-updater.

I've done a lot more stuff here. You should take another look.

I seem to remember your oAuth stuff being a PR to @jkudish's plugin. I'll try to find it again.

from git-updater.

Found the oAuth code and it looks very specific to GitHub. 😦

from git-updater.

Yes, but the oAuth process is pretty much the same for any site. Mostly just the URLs and data format change.

from git-updater.

Might be a good project to build an oAuth plugin and pass the secret key and token back to the caller.

from git-updater.

Maybe using https://oAuth.io would somehow work?

from git-updater.

Looks like a great service!!

On Sunday, June 22, 2014 at 10:27 PM, Andy Fragen wrote:

Maybe using http://oAuth.io would somehow work?

—
Reply to this email directly or view it on GitHub (#59 (comment)).

from git-updater.

Hey Paul! I discovered something very interesting while searching Google. The following are headers of downloaded private and public Bitbucket repos. https://core.trac.wordpress.org/ticket/22913

Private repository
Array
(
    [headers] => Array
        (
            [server] => nginx/1.5.10
            [date] => Mon, 23 Jun 2014 18:17:33 GMT
            [content-type] => text/html; charset=utf-8
            [connection] => close
            [x-served-by] => app16
            [x-render-time] => 0.0786230564117
            [content-language] => en
            [x-static-version] => 2668896b5f12
            [expires] => Mon, 23 Jun 2014 18:17:33 GMT
            [vary] => Cookie, Accept-Language
            [last-modified] => Mon, 23 Jun 2014 18:17:33 GMT
            [x-version] => 2668896b5f12
            [x-request-count] => 392
            [cache-control] => max-age=0
            [x-frame-options] => SAMEORIGIN
            [set-cookie] => csrftoken=RySijN7Dcz21hKlSFtDAQXjc0TdpI1lX; expires=Mon, 22-Jun-2015 18:17:33 GMT; Max-Age=31449600; Path=/; secure
            [strict-transport-security] => max-age=31536000
            [x-content-type-options] => nosniff
            [content-encoding] => gzip
        )
    [body] => 37745: 3c21444f435459504520
    [response] => Array
        (
            [code] => 200
            [message] => OK
        )
)
Public repository
Array
(
    [headers] => Array
        (
            [server] => nginx/1.5.10
            [date] => Mon, 23 Jun 2014 18:17:33 GMT
            [content-type] => application/zip; charset=utf-8
            [content-length] => 1448
            [connection] => close
            [last-modified] => Mon, 23 Jun 2014 18:02:38 GMT
            [cache-control] => max-age=0
            [expires] => Mon, 23 Jun 2014 18:17:33 GMT
            [content-disposition] => attachment; filename=afragen-test-bitbucket-plugin-8a9805ead42c.zip
            [etag] => "53a86bbe-5a8"
            [accept-ranges] => bytes
        )
    [body] => 1448: 504b03040a0000000000
    [response] => Array
        (
            [code] => 200
            [message] => OK
        )
)

I'm betting the content-type disparity and/or other headers are what is causing the issue. Is there some filter to try and change it?

from git-updater.

These look like response headers, rather than request headers. Response
headers are up to the responding server... And it does look like private is
sending you HTML, not a ZIP. You can filter outgoing headers, but response
is just information about what your'e getting.

Paul Clark
pdclark.com

On Mon, Jun 23, 2014 at 11:50 AM, Andy Fragen [email protected]
wrote:

Hey Paul! I discovered something very interesting while searching Google.
The following are headers of downloaded private and public Bitbucket repos.
https://core.trac.wordpress.org/ticket/22913

Private repository
Array
(
[headers] => Array
(
[server] => nginx/1.5.10
[date] => Mon, 23 Jun 2014 18:17:33 GMT
[content-type] => text/html; charset=utf-8
[connection] => close
[x-served-by] => app16
[x-render-time] => 0.0786230564117
[content-language] => en
[x-static-version] => 2668896b5f12
[expires] => Mon, 23 Jun 2014 18:17:33 GMT
[vary] => Cookie, Accept-Language
[last-modified] => Mon, 23 Jun 2014 18:17:33 GMT
[x-version] => 2668896b5f12
[x-request-count] => 392
[cache-control] => max-age=0
[x-frame-options] => SAMEORIGIN
[set-cookie] => csrftoken=RySijN7Dcz21hKlSFtDAQXjc0TdpI1lX; expires=Mon, 22-Jun-2015 18:17:33 GMT; Max-Age=31449600; Path=/; secure
[strict-transport-security] => max-age=31536000
[x-content-type-options] => nosniff
[content-encoding] => gzip
)

[body] => 37745: 3c21444f435459504520
[response] => Array
    (
        [code] => 200
        [message] => OK
    )

)

Public repository

Array
(
[headers] => Array
(
[server] => nginx/1.5.10
[date] => Mon, 23 Jun 2014 18:17:33 GMT
[content-type] => application/zip; charset=utf-8
[content-length] => 1448
[connection] => close
[last-modified] => Mon, 23 Jun 2014 18:02:38 GMT
[cache-control] => max-age=0
[expires] => Mon, 23 Jun 2014 18:17:33 GMT
[content-disposition] => attachment; filename=afragen-test-bitbucket-plugin-8a9805ead42c.zip
[etag] => "53a86bbe-5a8"
[accept-ranges] => bytes
)

[body] => 1448: 504b03040a0000000000
[response] => Array
    (
        [code] => 200
        [message] => OK
    )

)

I'm betting the content-type disparity and/or other headers are what is
causing the issue. Is there some filter to try and change it?

Reply to this email directly or view it on GitHub
#59 (comment)
.

from git-updater.

So it looks like the issue is that Bitbucket is sending the file incorrectly. It does, however, download a zip file on the desktop.

I wonder if these response headers are causing the error. If so, how do we fix?

from git-updater.

I wonder if some of the authentication code from this project could be used here?

https://bitbucket.org/alixandru/bitbucket-sync/overview

from git-updater.

Not sure the authentication is the issue. When I get back from islands I'll see if changing wp_get_remote to file_get_contents or curl for the update DL makes a difference.

from git-updater.

It may help you investigating this WP plugin: http://wordpress.org/plugins/wp-bucket/ - it is also available on BB and has a great wiki page: https://bitbucket.org/khosroblog/wp_bucket/wiki/Home

So this statement, while using the plugin's classes and after setup and confirmation of an OAuth consumer key, will return a zip file of a private master branch:
$WP_Bucket->download_stream("my_repo_slug", "my_username", "master");

Good luck!

from git-updater.

Sorry about my previous posting - the plugin won't help probably. OAuth always needs interaction with the requesting user... So I had to switch to GH meanwhile.

from git-updater.

@hostz-frank it was an interesting plugin though I was reaching he same conclusion. Too bad.

from git-updater.

I was also looking for a plugin that could even update WP plugins and themes from Gitlab - but no luck.

Since all these platforms are accessible via Git itself - isn't it a reasonable way to think in another direction and let Git itself do the work (without any OAuth and API's) ... :-)

from git-updater.

@hostz-frank I believe there's another issue re:Gitlab integration. Can we move this discussion there? I could also use some help on that as I don't use Gitlab.

from git-updater.

Exciting news! I was just testing my new Settings page, the password is now stored in the Settings API. I also made significant changes to the authentication code.

Updating now seems to work in testing! Please check.

<?php
/*
Plugin Name:          Test Bitbucket Private Plugin
Plugin URI:           https://bitbucket.org/afragen/test-bitbucket-private-plugin
Description:          This plugin is used for testing functionality of Bitbucket updating of plugins.
Version:              0.4.0
Author:               Andy Fragen
License:              GNU General Public License v2
License URI:          http://www.gnu.org/licenses/old-licenses/gpl-2.0.html
Bitbucket Plugin URI: https://bitbucket.org/afragen/test-bitbucket-private-plugin
Bitbucket Branch:     master
*/

The above is what my test private repo now looks like.

Could everyone please test and let me know success or problems. Click the Check Again button. Your password should automatically be placed in the Settings page if it is in the header. It should stay in the Settings API.

from git-updater.

Thanks for all your work on this, I was elated when I saw you'd figured this out.

I'm running into a problem where repo_meta isn't getting populated so the password option isn't being added to the settings page. I think it's due to the fact that BitBucket is returning a 401 status for $response = wp_remote_get( $this->get_api_url( $url ) ); on line 92 in class-bitbucket-api.php and thus we fail if ( $repo_api->get_remote_info( 'style.css' ) ) { on line 62 in class-theme-updater.php and $repo_api->get_repo_meta(); doesn't get called.

I can dump all my debug info or add you to the private repo, if either of those would help.

Thanks again!

from git-updater.

@coderaaron let me think about how to ensure do that. I've spent a lot of time trying to make sure the extra stuff doesn't show up in settings page. I missed the part about what happens the first time a private repo is added.

from git-updater.

@coderaaron update to latest develop branch and let me know if it works. I've set it up so it should display in the Settings until a password is entered if the Bitbucket repo is private. Thanks as I had to rework it for GitHub too.

Let me know if it's working.

from git-updater.

@afragen I've updated to version 3.0.7.9, and it got rid of the no private repo message on the settings page. I've added my bitbucket password and saved on the settings page, but I've gone into the database and see the github-update value is "a:1:{s:14:"github-updater";N;}".

from git-updater.

I just pushed 3.0.7.10 which, I think, fixed an odd problem where update_site_option wasn't working.

You should be looking for database value github_updater.

I've been testing locally with v3.0.7.10 and it's working great. If anyone is having issues with this version please let me know. Also, tell me if it's working for you.

from git-updater.

Updated but still seeing no change in the database after saving settings :(

from git-updater.

Can you email me a zip of the plugin?

from git-updater.

@haleeben if you're testing on a multisite install the github_updater data is in wp_sitemeta on a single install the site options are located in wp_options

from git-updater.

Sorry everyone. I just pushed to develop so that settings on single site installs should properly save settings.

from git-updater.

Thanks @afragen , working now...........awesome.

from git-updater.

@haleeben thanks for confirming. This version was tagged and released.

from git-updater.

I tested this and it is working great!

Maybe this isn't the right issue to post this on, but is there any way to encode the password so that if someone gains access to the admin they don't have the keys to the kingdom for our Bitbucket account?

from git-updater.

@zachthezman a new issue might be a better place, but the short answer is if we encode the password we have to decode password and all that coding/decoding will be present in the plugin. This would mean that any user would be able to discern the password. The plugin needs to have the access to the password.

I would recommend setting up individual accounts for each private repo you wish to mass distribute. Not many other ways around it.

from git-updater.

Hey everyone, I've got a better idea that's now on develop. Now in the Settings page you enter your personal Bitbucket user/pass and check the private repositories. If you don't own the private repo you must have at least read privileges to authenticate. Now you can add clients to your private repos with read-only access and not expose your credentials to them.

from git-updater.

Awsome, just tried out your plugin and was just about to ask about support for organization repos in bitbucket. The new settings with username / password works, thanks!

from git-updater.

@carlgrundberg thanks for testing this. Out of curiosity, what privileges do your organization members hold?

from git-updater.

I was able to add a private Bitbucket repo owned by our organization. However when I push an update to Bitbucket and go to check the plugin - it never sees the updated version is available.

Another issue (and probably should be opened as another issue) is that it is not remembering I selected "private repository" on installation. I am having to go back in and re-check that after the authentication fails when re-loading the plugin page.

I am running version 4.2.0

from git-updater.

Remote installation and updating occur through different mechanisms. The Settings page must be updated after a remote install of a private repo.

Did that solve your issue?

from git-updater.

no not solved. The main issue im having is that when i update a plugin, i
dont see the update available in the plugins page on WP.

i just thought it was weird i had to check "private repository" twice when
the second one should know i've already checked the first one.

Andy Brudtkuhl
@abrudtkuhl http://twitter.com/abrudtkuhl

On Fri, Mar 6, 2015 at 3:23 PM, Andy Fragen [email protected]
wrote:

Remote installation and updating occur through different mechanisms. The
Settings page must be updated after a remote install of a private repo.

Did that solve your issue?

—
Reply to this email directly or view it on GitHub
#59 (comment)
.

from git-updater.

no not solved. The main issue im having is that when i update a plugin, i
dont see the update available in the plugins page on WP.

Is your Settings page correct?

i just thought it was weird i had to check "private repository" twice when
the second one should know i've already checked the first one.

The issue is that the private repo won't be listed in the Settings page until it is installed. There's no place to put the setting for a private repo because it didn't exist prior to installing the app.

I'll consider it a feature request. Maybe I can figure something out.

from git-updater.

i just thought it was weird i had to check "private repository" twice when
the second one should know i've already checked the first one.

The issue is that the private repo won't be listed in the Settings page until it is installed. There's no place to put the setting for a private repo because it didn't exist prior to installing the app.

I'll consider it a feature request. Maybe I can figure something out.

86ac601

from git-updater.

no not solved. The main issue im having is that when i update a plugin, i
dont see the update available in the plugins page on WP.

You might have to clear the transients by clicking Check Again on the update-core.php page.

from git-updater.