See: https://docs.google.com/document/d/1NLobogbVfSYofcm7b61tesTirlXpmxwIRYd6Gsd3tt8/edit#heading=h.xrdvvhrlt660

We're opening this issue because your project has used Travis CI within the last 6 months. If you have already migrated off it, you can close and ignore this issue.

Travis CI is ending free builds on public repositories. travis-ci.com stopped providingthem in early November, and travis-ci.org will stop after December 31, 2020. To avoid disruptions to your workflows, you must migrate to another CI service.

For production use cases, we recommend switching to CircleCI. This service is already widely used within Mozilla. There is a guide to migrating from Travis CI to CircleCI available here.

For non production use cases, we recommend either CircleCI or Github Actions. There is a guide to migrating from Travis CI to Github Actions available here. Github Actions usage within Mozilla is new, and you will have to work with our github administrators to enable specific actions following this process.

If you have any questions, reach out in #github-admin:mozilla.org on matrix.

From the readme, I don't have any idea of what this repository is doing (I don't know what bouncer is nor its redirector portion)

From time to time product-details and ship-it get have been known to accidentally get out of sync, at a test for this.

• Add a test for ship-it and product details that asserts firefox_versions.json are in sync
• Remove _extract_windows_version_num(), this method isn't used by any of the tests.
• Update docstings, s/:arg/:param/

We want to enable pyup.io for our projects to take care of automating the updates to our dependencies. Here's what's needed:

• Move dependencies out of tox.ini and into requirements files. See mozilla/PyPOM@ad21988 for an example of this.
• Add repository to pyup.io (@davehunt can take care of this once the above is merged).
• Add pyup.io badges for updates and python 3 readyness. See mozilla/PyPOM@931c41b for an example of this.

Conftest.py adds a '--productoption that is being used bytest_stub_installer_redirect_for_en_us_and_win`.

I believe we designed it this way with the thought that we might want hand test, in a bestoke artisanal way, individual products. Not just firefox-stub.

I don't see a reason to keep this complexity in the code. Remove this and hard-code firefox-stub directly to the test.

• Remove conftest.py
• Update stub test to use firefox-stub alias
• Update stub test to use the cdn list

A new ESR build for IE6 has dropped, `firefox-52.0.2esr.exe'. Update the test to reflect this and xfail the effected tests that redirecting to the incorrect product.

The following need to be xfailed:

• firefox-38.5.1esr
• firefox-38.5.2esr
• firefox-38.5.3esr
• firefox-38.6.3esr
• firefox-40.0.0esr
• firefox-beta-latest
• firefox-49.0b8
• firefox-49.0b10
• firefox-49.0b37

https://bugzilla.mozilla.org/show_bug.cgi?id=1357378

(Originally filed as https://bugzilla.mozilla.org/show_bug.cgi?id=1878168)

Right now, incomplete URLs such as https://download.mozilla.org/?product=firefox-latest redirect to the Windows stub installer. Unless there's a very a good reason we're doing this, we should stop. Bouncer is essentially an API, so it's best to require complete requests rather than making assumptions.

If there are places where we're providing incomplete links for some reason, we should fix those.

We need to create SHA1 only mirrors and redirect SHA1 clients there.

https://github.com/Masterminds/glide

This could ease adding dependencies as in #58 and optionally remove the need to check vendor into source control.

As of January 1 2019, Mozilla requires that all GitHub projects include this CODE_OF_CONDUCT.md file in the project root. The file has two parts:

1. Required Text - All text under the headings Community Participation Guidelines and How to Report, are required, and should not be altered.
2. Optional Text - The Project Specific Etiquette heading provides a space to speak more specifically about ways people can work effectively and inclusively together. Some examples of those can be found on the Firefox Debugger project, and Common Voice. (The optional part is commented out in the raw template file, and will not be visible until you modify and uncomment that part.)

If you have any questions about this file, or Code of Conduct policies and procedures, please reach out to [email protected].

(Message COC001)

Releng has recently removed the firefox-esr-sha1 rule set on their end. Update our winxp tests to reflect this change.

• Specifically http://download.mozilla.org/?product=firefox-esr-sha1 now returns a 404 when visited by ie6

This happened when we attempted to deploy to stage... See also: docker-library/golang#467

We need Dependabot to keep our dependencies up to date.

Configure pyup to initiate pull requests only on security releases

https://bugzilla.mozilla.org/show_bug.cgi?id=1240511

This issue is similar to mozilla/bedrock#10524 (which has more info). We introduced a check in #347, which we would likely need to enhance in order to better decode the value of the content parameter.

WinXP and Vista users will now only receive 52.9 esr builds per bug 1319164.

Add option for datadog (statsd) integration.

bouncer.download counter with tags: product, os, language.

Stage is set to sync against prod details every 2 hours. Update the test job (https://qa-master.fxtest.jenkins.stage.mozaws.net/job/go-bouncer.stage/) to run on a sane default10 H/2 * * *.

Description

Ignore attribution for RTAMO if the referral wasn't from www.mozilla.org , using a referer header check.

Success Criteria

• RTAMO attribution should be present if the user was referred by www.mozilla.org and all other validation is successful.
• RTAMO attribution should be ignored if the user did not start on www.mozilla.org.

cc @mostlygeek

Changes made to prod appear to have propagated to stage.

Reference bug is bug 1351900.

Requests for certain products or aliases like 'firefox-latest' can be done via either http or https, depending on context. Our stub installer requests the installer via http and does independent verification of the installer.

Most other requests of 'firefox-latest' are done initially via https to download.mozilla.org. However, bouncer at this point redirects them to an http location.

I'd like to change how bouncer works in this case. When a request is originally done via https, the redirect should always be to an https location. We should never downgrade from https to http.

We are using Go 1.14 at the moment (apparently), that is a bit old now.

@davehunt my brain is mush at this point - helpful pointers, if you would?

https://qa-preprod-master.fxtest.jenkins.stage.mozaws.net/job/go-bouncer.adhoc/137/ (and even running locally, for me, against master) yields:

02:40:48 Step 4 : COPY requirements.txt /src
02:40:48  ---> Using cache
02:40:48  ---> 5bf630fc453d
02:40:48 Step 5 : RUN pip install pipenv
02:40:48  ---> Using cache
02:40:48  ---> 85735dd5b5ef
02:40:48 Step 6 : RUN pipenv install --system -r requirements.txt --skip-lock --deploy
02:40:49  ---> Running in b738bc7afcb4
02:40:50 Creating a Pipfile for this project…
02:40:50 Traceback (most recent call last):
<snip>
TypeError: 'NoneType' object has no attribute '__getitem__'

Fuller output below:

02:40:48 Checking out Revision 58acc2ca54237e19299a7e616fb87212746e269f (origin/pyup-update-pytest-3.4.1-to-3.4.2)
02:40:48  > git config core.sparsecheckout # timeout=10
02:40:48  > git checkout -f 58acc2ca54237e19299a7e616fb87212746e269f
02:40:48 Commit message: "Update pytest from 3.4.1 to 3.4.2"
[Pipeline] withEnv
[Pipeline] {
[Pipeline] readFile
[Pipeline] sh
02:40:48 [go-bouncer.adhoc@2] Running shell script
02:40:48 + docker build -t a8cf67e9f83d680ef17c18ffab414241b9ded0e2 -f tests/e2e/Dockerfile tests/e2e
02:40:48 Sending build context to Docker daemon  72.7 kB

02:40:48 Step 1 : FROM python:2.7-alpine
02:40:48  ---> 3e4f5b6dfb77
02:40:48 Step 2 : RUN apk add --update gcc libffi-dev musl-dev openssl-dev python-dev
02:40:48  ---> Using cache
02:40:48  ---> 944e88cd23b8
02:40:48 Step 3 : WORKDIR /src
02:40:48  ---> Using cache
02:40:48  ---> 40ba2f4b58b5
02:40:48 Step 4 : COPY requirements.txt /src
02:40:48  ---> Using cache
02:40:48  ---> 5bf630fc453d
02:40:48 Step 5 : RUN pip install pipenv
02:40:48  ---> Using cache
02:40:48  ---> 85735dd5b5ef
02:40:48 Step 6 : RUN pipenv install --system -r requirements.txt --skip-lock --deploy
02:40:49  ---> Running in b738bc7afcb4
02:40:50 Creating a Pipfile for this project…
02:40:50 Traceback (most recent call last):
02:40:50   File "/usr/local/bin/pipenv", line 11, in <module>
02:40:50     sys.exit(cli())
02:40:50   File "/usr/local/lib/python2.7/site-packages/pipenv/vendor/click/core.py", line 722, in __call__
02:40:50     return self.main(*args, **kwargs)
02:40:50   File "/usr/local/lib/python2.7/site-packages/pipenv/vendor/click/core.py", line 697, in main
02:40:50     rv = self.invoke(ctx)
02:40:50   File "/usr/local/lib/python2.7/site-packages/pipenv/vendor/click/core.py", line 1066, in invoke
02:40:50     return _process_result(sub_ctx.command.invoke(sub_ctx))
02:40:50   File "/usr/local/lib/python2.7/site-packages/pipenv/vendor/click/core.py", line 895, in invoke
02:40:50     return ctx.invoke(self.callback, **ctx.params)
02:40:50   File "/usr/local/lib/python2.7/site-packages/pipenv/vendor/click/core.py", line 535, in invoke
02:40:50     return callback(*args, **kwargs)
02:40:50   File "/usr/local/lib/python2.7/site-packages/pipenv/cli.py", line 197, in install
02:40:50     selective_upgrade=selective_upgrade
02:40:50   File "/usr/local/lib/python2.7/site-packages/pipenv/core.py", line 1722, in do_install
02:40:50     ensure_project(three=three, python=python, system=system, warn=True, deploy=deploy, skip_requirements=skip_requirements)
02:40:50   File "/usr/local/lib/python2.7/site-packages/pipenv/core.py", line 653, in ensure_project
02:40:50     ensure_pipfile(validate=validate, skip_requirements=skip_requirements)
02:40:50   File "/usr/local/lib/python2.7/site-packages/pipenv/core.py", line 343, in ensure_pipfile
02:40:50     project.create_pipfile(python=python)
02:40:50   File "/usr/local/lib/python2.7/site-packages/pipenv/project.py", line 435, in create_pipfile
02:40:50     data[u'requires'] = {'python_version': python_version(self.which('python'))[:len('2.7')]}
02:40:50 TypeError: 'NoneType' object has no attribute '__getitem__'
02:40:50 The command '/bin/sh -c pipenv install --system -r requirements.txt --skip-lock --deploy' returned a non-zero code: 1

The intention was to send users on old OSX to ESR78 installers; that worked while the esr-next aliases pointed at esr78, but that has not been the case for several years now, so this code stopped working with the initial esr91 release.

Please install https://github.com/integration/taskcluster for this repository so that TaskCluster will publish revisions and Treeherder will ingest these into https://treeherder.allizom.org/#/jobs?repo=go-bouncer (and https://treeherder.mozilla.org/#/jobs?repo=go-bouncer after the next deployment). This will allow us to then submit test results to Treeherder.

In issue #60 @stephendonner correctly noted that we need to update the stub installer tests to verify several additional components.

Namely adding in SHA-1 support, given https://download-sha1.allizom.org/ and its corresponding CDN https://download-sha1.cdn.mozilla.net/ are live.

Let's configure Codecov to get code coverage reports.

If the attribution_code parameter is set, redirect to the stub downloader service with the product, os, lang and attribution_code parameters set.

@pmac

Line 89, response.url is referenced but response is never assigned when an exception is thrown.

try:
    response = requests.head(url, headers=headers, verify=False, timeout=15, params=params, allow_redirects=True)
    except requests.RequestException as e:
        request_url = '%s/?%s' % (url, urlencode(params))
        raise AssertionError('Failing URL: %s redirected to %s Error message: %s' % (request_url, response.url, e))

Tests are failing due to https://bugzilla.mozilla.org/show_bug.cgi?id=1351879

Remove the xfail fromtest_verify_product_details_is_in_sync_with_ship_it().

Currently the test is xfailed due to our CI not having access to ship-it. Once this is remedied we can remove this xfail. I'll follow up shortly with a bug number.

https://github.com/mozilla-services/go-bouncer/blob/master/tests/e2e/tests/test_smoketests.py#L27

• Update the winxp tests to track 'firefox-esr-latest' in https://product-details.mozilla.org/1.0/firefox_versions.json
• Update winxp esr tests to request the following aliases

        'stub',
        'latest',
        'sha1',
        'esr-latest',
        'esr-sha1',
        'esr-stub',
        'beta',
        'beta-latest',
        'beta-sha',
        'beta-stub',
        '38.5.1esr',
        '40.0.0esr',
        '58.0.0esr',
        '42.0',
        '43.0.1',
        '49.0b8',
        '49.0b8-ssl',
        '100.0',
        'cats'
    ]