claustromaniac / httpz Goto Github PK

Firefox 71.0 with HTTPZ 0.12.0, Wayback Machine 1.8.6 and other extensions.

http://yourjavascript.com/

IIRC a subsequent manual visit to http://yourjavascript.com/ resulted in an attempt to use HTTPS but there was no HTTPZ icon.

Also IIRC I had this (can't remember when or why I disabled the option):

☐ Handle non-standard redirections to HTTP

That's now enabled.

"Remember Secure Sites" has a warning

Do not disable this unless you know what you're doing!

So what is so bad about it? To me it just seems like speed/storage (privacy) balance: either skip if URL matches a blacklist or retry every time.

Surely it helps when the user is actively being MITMed, but how likely is that?

^ What the title says. ^ Instead of redirecting back to http://archives.nypl.org, you get stuck on https://archives.nypl.org, which shows you this error message:

archives.nypl.org -
SSL Connection failed
Error code 29
SSL is not supported

Attempting to access http://silta.piraatit.fi:61825/robots.txt turns it into https://silta.piraatit.fi:61825/robots.txt which errors. That port is TeleIRC and port 443 is used by OpenSSH.

Sorry about this!

Three frames from a screen recording:

Add, quit, start Firefox, view the list: no listing.

I'm almost certain that it occurred on more than one occasion.

http://thegearcalculator.appspot.com/

The recording: 2019-12-04 13:21.zip

IIRC my first realisation of the issue was when the secure page, which should have been excluded, was automatically loaded at Firefox start time (session restoration).

Extension version 0.11.0b7

grahamperrin@momh167-gjp4-8570p:~ % pkg query '%o %v %R' firefox
www/firefox 71.0_1,1 FreeBSD
grahamperrin@momh167-gjp4-8570p:~ % 

PS

at Firefox start time (session restoration).

– that's why I began this issue as separate from #42. For misbehaviour of any extension at session restoration, time I lazily think first of https://bugzilla.mozilla.org/show_bug.cgi?id=1378459 (i.e. not an issue with the extension) but on second thoughts, this issue is probably not 1378459. Sorry for rushing into this one.

❗ This issue is locked. Open a new issue instead.

Since version 0.7.0, HTTPZ no longer checks the type of error thrown to decide what to do. Site-specific issues are now rare and they no longer have a common cause, so I prefer to address them separately on a case-by-case basis.

Please use this issue to report any site-specific errors that you come across. Firefox APIs throw a limited number of errors, but this extension won't always handle them all, mostly because

1. I'm only human and I can (and do) miss some.
2. Mozilla can add new errors over time, and according to documentation the error messages are not guaranteed to stay the same as the browser is updated.

Make sure to include the URL of the site you're trying to visit so I can try to reproduce your issue on my end and fix it.

Hi claustromaniac,
Should I remove Decentraleyes if I want to use HTTPZ (or any other alternative to HTTPS Everywhere)? (I think the answer's yes but I'm not sure.) TIA.

I stumbled across your add-on over on Reddit and I wanted to know what the benefits / drawbacks are of HTTPZ vs HTTPS Everywhere.
This thread, while old, seems to suggest that HTTPS Everywhere isn't exactly perfect. No idea if the contents of what it mentions still stand or not, but regardless, I'm curious as to whether I should swap to HTTPZ.

Thanks in advance!

This is an example of Tor-based site which should not be routed to HTTPS, because Tor internal transfers are already encrypted. Currently Httpz makes it impossible to connect to such resources.

I'm sure you are aware that there is a page filled with messages in a large font that sometimes momentarily appears when HTTPZ unsuccessfully tries to load a page through HTTPS.

Where do those strings come from? Can we edit them somehow?

I tried creating a bunch of bookmarks for sites that I know don't work with HTTPS. I then loaded all the bookmarks at once (about 15 of them) with HTTPZ 0.7.1 enabled.

The result was that the following sites get redirected to HTTPS by HTTPZ, but never get redirected back to HTTP when they fail HTTPS:

http://blog.bahraniapps.com/gifcam/
http://liwi.io/
http://recordit.co/
http://www.onyxbits.de/

Spun off from #37 (comment) test 5:

http://thegearcalculator.appspot.com

Pre-release 0.10.0b4 with Firefox,

grahamperrin@momh167-gjp4-8570p:~ % date ; uname -v
Tue 19 Nov 2019 07:29:26 GMT
FreeBSD 13.0-CURRENT #36 r354616: Tue Nov 12 01:28:03 GMT 2019     root@momh167-gjp4-8570p:/usr/obj/usr/src/amd64.amd64/sys/GENERIC-NODEBUG 
grahamperrin@momh167-gjp4-8570p:~ % pkg query '%o %v %R' firefox waterfox
www/firefox 70.0.1_3,1 FreeBSD
www/waterfox 2019.10.c.1 unknown-repository
grahamperrin@momh167-gjp4-8570p:~ % 

Sometimes the page action disappears (appears greyed out beneath the ⋯ menu). When this occurs after whitelisting, it's necessary to use the preferences for the extension if the listing is to be removed.

Parts of these two screen recordings might suggest transient improvement after disabling enhanced tracking protection, but there's a sense of randomness; I have not figured out how to make things consistently reproducible.

• https://put.re/player/77kHP2wV.mp4 2019-11-19 07:18.mp4
• https://put.re/player/Hgr6rcg3.mp4 2019-11-19 07:24.mp4
• two screen recordings.zip

Refreshing the profile is not a workaround.

Please note, I have not tested release 0.9.4 with the site with Firefox.

(I hope that the Waterfox Classic (Firefox 56.⋯)-oriented enhancements in 0.10.0b4 are not detrimental to Firefox Quantum use cases.)

can this be used as a SkipCertError replacement?

Basically, if an invalid cert is found when visiting sites like this, it will skip the "Your connection is not secure" warning and auto-add a temporary exception.

Related to #8

Here is what I'm thinking:

• flip the default state of the Automatic Mode to false
• add a button or checkbox to error.htm like don't show me these warnings again that, when pressed, shows a warning recommending users to leave the Automatic Mode off if they installed the extension recently or if they're connecting over an untrusted network (such as a wifi hotspot)
• add the same info to the options page for users that installed the extension prior to that update... maybe. I could even use an upboarding page for this but I feel like that would be overdoing it. I didn't make this extension to teach users how to use their heads after all...
  • alternatively, I could flip the Automatic Mode itself to false, not just the default state, but that would definitely annoy (and maybe even offend) a certain number of users. My peace of mind should not matter more than their personal preferences, at least not in this case. Then again, if it's a one-time thing, maybe they'll tolerate it.
• should also reword the option for remembering secure sites. Specifically, the last part, where it says Note that this is only meaningful in automatic mode. I should remove that statement and instead recommend users not to disable that option unless they understand the consequences.

FF 67.0.4
httpz 0.8.1

auto mode = off
remember = off
ignore = until FF is restarted
whitelist is empty

load truthinmedia.com, or http://truthinmedia.com and there is no warning

the domain was accessed by my RSS reader prior to this, though i never opened the tab, but i clicked 'forget ignored sites' to test again but the behavior was the same

1st time i've seen this

http://thegearcalculator.appspot.com/

Screen recording:

• https://put.re/player/dfmt3vXQ.mp4
• 2019-11-21 14:12.zip

1. 01:35 addition to whitelist (because the content is wrong with https)
2. 01:45 (page action) apparent removal from the whitelist, automated reload of the http URL
3. 01:49 page action unavailable, greyed out
4. 02:00 presence in the whitelist
5. 02:13 restart of Firefox
6. 02:24 page action available, with the option to remove from the whitelist.

In this case: step (2) is probably not a sane action (no sane user would require the wrong content), however the presence of the listing at point (4) on the timeline seems inconsistent with the click to remove the listing at step (2).

Edge case? Or is this somehow related to the criteria at #42 (comment)?

TIA

Cloudflare's Rocket Loader lazy loading script won't work when HTTPZ is enabled and sites that use it like https://thehackernews.com/ won't work properly.

The error thrown in the console is:

 rocket-loader.min.js:1:9661: Error: Permission denied to access property "__rocketLoaderInlineHandlerProxy"

It might be related to this and might need a similar fix:
kanap0nta/QuickDrag-WE#52

Creating a new issue because #2 is locked. Apologies if it's the wrong course of action.
Thank you for the extension.

Have you considered porting this to Chromium in order to make it work in browsers like Chrome, Opera and Edge (and upload to their web stores)?

Edit: per upcoming manifest v3, #43 is required too.

this was brought up on the privacytools.io repo and i'm wondering if you could provide some insight claustro - thanks! ...

This is the issue with HTTPZ, and why I think HTTPS Everywhere's whitelist solution is the more secure option, even if it isn't as comprehensive. I haven't seen any evidence that HTTPZ has any kind of downgrade attack prevention.

For example, if I was an attacker in between a client and a webserver, say privacytools.io, I could block HTTPS access to https://privacytools.io, by blocking port 443 or whatever...

• From what I can tell, HTTPZ would just say "this resource isn't available over HTTPS" and happily load http://privacytools.io, because it has no way of knowing if the site should be loaded over HTTPS.

• On the other hand (assuming privacytools.io is in the whitelist), as far as I understand it, HTTPS Everywhere will see that access to https://privacytools.io is blocked (by the MITM) and not allow any connections over HTTP, because it knows that privacytools.io needs to be loaded over HTTPS. That's the entire point of the whitelist which I think is what is being missed in this thread.

Yes, HTTPZ may upgrade more connections to HTTPS, which is cool and all, but it doesn't prevent downgrade attacks which is a security flaw. This is the same issue I brought up when recommending to close #810 (comment).

Hi @claustromaniac , I tried httpz-0.6.0b-an+fx.xpi (downloaded from GitHub, not yet available at AOM) and the extension is brilliant, combines all we've been talking about at #2

Your Readme.md explains perfectly well what HTTPZ performs as well the known issues.

There is unfortunately another issue, not related to your code but to to the very http-to-https, this time when https is indeed available (the easiest scenario).

I'm referring to a page which calls 3rd-party servers via http when these 3rd-party servers will include mixed-content.

If HTTPZ successfully starts the http site via https
and
if Mixed Content is called via http,
then,
if the Mixed Content is passive (content such as images) then the user's security.mixed_content.block_display_content will have to be set to false (default=false)
but
if the Mixed Content is active (content such as scripts) then the user's security.mixed_content.block_active_content will have to be set to true (default=false)

An example:http://www.internetlivestats.com/

If that site is called via https (and it does accept https) then, in order to view the page displayed correctly, I need to set security.mixed_content.block_active_content to false which is unsecure.

In other words, there are sites which still need to be accessed via http even though they are accessible via https. This means that HTTPZ would require a whitelist for sites always accessed via http even when https is available. Getting complex.

Sorry for having been lengthy, but what i'm afraid of is that the very concept of try https - keep if ok - revert to http otherwise appears to be far more problematic than one could believe initially

STR:

1. Open the native bookmarks library in Mozilla Firefox 66.0.3
2. Search for "http://"
3. Multi-select 10-20 http URLs
4. Right-click a url, and select "Open in tabs" to open all the selected bookmarks.
5. Notice that some tabs that fail on HTTPS don't get reverted to HTTP by HTTPZ. HTTPZ works fine on those sites when opened individually.

Add an ability to translate the addon into other languages.

Version: 0.7.0b2

Settings:

{
	"ignorePeriod": 0,
	"ignored": {},
	"incognitoWhitelist": {},
	"knownSecure": {},
	"rememberSecureSites": false,
	"whitelist": {}
}

Browser: Firefox 66.0.3 (64-bit)

Sample URL: example.com

This is in continuation of the discussion which started here. I have come across the following issues while testing the latest beta version:

1. Sites whitelisted in incognito mode do not appear in the exported settings file. Also, the whitelisting doesn't last once the tab/window is closed (if it worked in the first place).

2. The HTTPZ icon for an http site redirected to https is not always enabled. Sometimes it gets enabled after either reloading the tab, restarting browser, clearing cache or refreshing after a minute, etc. The result also varies depending on whether / is present at the end or www is present at the beginning of URL.

3. Sites whitelisted in normal mode behave erratically. Sometimes the whitelist is not respected and only kicks in after trying the random tricks mentioned above. Again, the presence of / at the end or www at the beginning of URL may affect results.

4. The Import option does not work and throws up Error. Invalid file (?) message.

Another one for the back burner.

I think some options are worth clustering into an "advanced" section.

Unless I'm mistaken, currently user's exceptions aren't listed anywhere in addon options so it's hard track them. I think it would be nice to have list of them with the ability to delete specific sites.

Take the https://addons.mozilla.org/en-GB/firefox/addon/fakesradar/ route to https://fakesradar.com/

A screen recording of Waterfox Classic 2019.12, I get much the same with Firefox 71.0:

2019-12-27 15:09.zip

When I visit the following sites with HTTPZ addon enabled:

allergologia.hsanmartino.it

www.ospedalesanmartino.it/servizi-alla-persona/informazioni-su-prelievi-ed-esami/laboratorio-di-analisi.html

**EDIT:**
It is a lot more than these 2 sites. It seems to have something to do with speed of loading. Slow sites are more likely to get stuck with this error.

I get an HTTPZ error page with NS_ERROR_CONNECTION_REFUSED message, and Proceed over HTTP & Retry buttons--clicking both results in the same error page. So the website remains inaccessible.

I can visit the above mentioned sites if I do the following:

1. Disable HTTPZ
2. Load the site
3. Enable HTTPZ
4. Refresh the site
5. It will work from that point on and there is no error page (HTTP)

Firefox 72.0.2
HTTPZ 0.12.0

Faster to show than to explain.

STR:

1. Create a bookmark to http://blog.bahraniapps.com/gifcam/
2. Make sure you have at least one more bookmark
3. Open FF's Bookmark Library
4. Open the bookmark created in step 1 by double-clicking on it
5. Within a few seconds, open another bookmark my double-clicking on it
6. Notice that bookmark opened in step 5 gets replaced by the site opened in step 4

This makes sense when you understand what HTTPZ is doing behind the scenes, but is definitely unexpected behavior if you don't.

http://www.searchdaimon.com/ (referred from https://github.com/searchdaimon/enterprise-search)

I can work around by setting a maximum wait before fallback (I see #7; sixty seconds seems to work in this case).

Without the workaround:

Technically, is it possible to show the HTTPZ icon in cases such as this? A visual hint that the extension has modified the URL.

I'm using Httpz in "automatic mode". When visiting several http sites I have bookmarked (for example: http://www.easysurf.cc/scintd.htm), I briefly see an error page with the following lines before seeing the correct page:

Oops.
Log in to network
Hmm. We’re having trouble finding that site.
File not found
Access to the file was denied
Hmm. That address doesn’t look right.
The address wasn’t understood
etc...

It only happens the first time I visit a website.

I searched and saw something similar reported at #2

With HTTPZ 0.11.3 disabled:

• http://helpx.adobe.com/flash-player.html redirects to HTTPS.

With the extension enabled:

• http://helpx.adobe.com/flash-player.html redirects to HTTPS
• the page action dialogue states that redirection was performed by HTTPZ
• opting to add an exclusion does result in a listing at about:addons but the listed exclusion is not effective.

A minor UX issue, I reckon.

HTTPZ does not currently treat gateway errors as errors, but this would be of use in some scenarios.
I want to do one of two things. Either:

1. allow users to specify that they are behind a forward proxy, via a checkbox in the options, or...
2. check the browser's proxy settings and act accordingly

1 requires user interaction, but 2 is useless to users behind transparent proxies routing traffic at the OS level or in the local network, etc. 1 sounds like a better idea.

d7d8856#diff-051a8a7bcae8db1e4cee8eb09b52e619L5

b3c55c9#diff-051a8a7bcae8db1e4cee8eb09b52e619L5

Loosely speaking: should users of Waterfox Classic e.g. 56.2.14 expect some features of 0.6.0 and greater to be simply non-functional?

https://addons.mozilla.org/addon/httpz/versions/

tl;dr Waterfox 56.0 was based on Firefox 56.0.2 and to be clear, I'm not suggesting that (#32) Waterfox Classic should be another supported browser.

I'm just curious about functionality. Background: https://www.reddit.com/comments/dahnh0/-/f1te63v/

TIA

HTTPZ 0.5.0 / Firefox 64.0 (x64) / Windows 7 (x64)

Hi,

As mentioned in the title, besides an unavailable httpz toolbar icon the extension here fails to redirect to http on http-only sites, but https-available site called via http was correctly redirected to https.

Examples:

http://www.cartesfrance.fr/ : no https, was not redirected to http
http://www.acidtests.org/ : https available, was redirected successfully to https

Once these issues fixed (provided they are not explained by a compatibility reason with other extensions, settings of mine) I'd wish to know if HTTPZ handles 3rd-party sites : for instance cartesfrance.fr mentioned above calls http servers (but https available) which are handled correctly by the 'HTTPS Everywhere' extension. Does HTTPZ handle those 3rd-party servers as well?

I already use your three other Firefox extensions (Etag Stoppa, Poop, True Sight). I'd love to add HTTPZ to the lot and abandon 'HTTPS Everywhere' whatever its qualities.

http://living.corriere.it goes into a death spiral (constantly reloading) when HTTPZ is enabled. Lots of mixed content errors in the console. I'm not sure if there's anything to be done, other than whitelist the site. (Also not sure if I should have opened a new issue for this, but this seemed to be the right place.) In any case, thanks for the extension!

Originally posted by @practik in #2 (comment)

On home page desktop mode, Click on Samsung Firmware's tab up top. When you enter a model number in the box and attempt to search, nothing happens. Disabling HTTPZ fixes issue. No issues with HTTPS Everywhere or Smart HTTPS.

Motivation, ideas: #13, #42

HTTPZ for example is at v0.7.2 released 13 days ago, yet it appears 2nd in the list (as of this writing) on AMO if you sort extensions by 'recently updated'...

https://addons.mozilla.org/en-US/firefox/search/?sort=updated&type=extension

you know what's up with that? i'm seeing this with other add-ons as well

Unlike HTTPS Everywhere, this extension doesn't take care of sub-requests triggered from HTTP-only sites. For now, it outright ignores those requests, because using the same approach with those (retrying on error) is very complicated and has significant drawbacks.

What about adding an option to use the Upgrade-Insecure-Requests header, similar to Smart HTTPS? That way the browser would handle everything properly, without needing the extension to redirect links back and forth.

There is also a flag security.mixed_content.upgrade_display_content but I think it is not exposed to extensions just yet. Bug 1435733

1. Disable Enable interception of server-initiated redirections to http
2. maybe also disable enable handling of non-standard redirections to http
3. visit http://www.fuelly.com/car/nissan/elgrand_/2000
4. Proceed with redirection

… at least twice I found version 0.11.0b7 not proceeding – IIRC just a blank white extensions page – but (sorry) I didn't take a screenshot and now I can't reproduce the behaviour.

Re: #39 (comment)

With my heavily-extended everyday Firefox profile, the new layout appears as it should.

With a test profile, HTTPZ 0.11.0b3 alone enabled, I got this:

In this profile, extensions.htmlaboutaddons.inline-options.enabled was false. Making it true (a Firefox 71.0 default) worked around the issue.

I don't know whether extensions.htmlaboutaddons.inline-options.enabled is (or will be) redundant …

Occasionally, I will encounter sites that partially work with HTTPS, but don't fully work (for example, all images on the page might not be displayed when accessed via HTTPS).

The next time I come across an example, I'll try to remember to post here.

It might be useful for this extension to support an exception list where the user can add sites that fall under that category. It can be real simple: a text field in the options screen where the user can enter a comma-separated list of hosts. For flexibility, you can allow * for the host to act as a wildcard (ex. "*.site.com"). URL's matching any item in this list will not be converted from HTTP to HTTPS.

Would it be possible to specify the maximum time duration to wait (the first time, or after a reset) before declaring that a site does not support HTTPS?

It would possibly not be 100% accurate to specify a shorter timeout, but it would greatly increase usability.

For example, sites like http://recordit.co/ take a long time to timeout when attempting HTTPS connections.

Hi

At a glance:

In full:

At a glance

One preference is open to misinterpretation:

☑ Automatic mode

Maybe better:

☑ Silent fallback

Order

FWIW I'd have the whitelist uppermost, followed by the ignored section.

Save button

I sort of wish for Save buttons at both top and bottom, to compensate for the freakiness of Mozilla's redesign –

– but doubling up is debatably freaky, so imagine the one Save button moving to the top of the page (to not require scrolling after e.g. editing the shortlist). It's a bit of an awkward idea, because it is nice to have Export, Import and Save buttons neatly aligned together but still, I wish for the Save button to be more accessible without scrolling/paging.

When I set "When a site does not support HTTPS, ignore requests to it..." to 10 days (or more) and click the save button, save button becomes ❌ instead of ✔ and does not save my preferences.

HTTPZ Version: 0.9.3
Firefox version: 69.0.3
OS: Windows 10

Opening https://http.badssl.com makes the server redirect to HTTP (status code 301), but HTTPZ does not intercept the redirection. Why?

Tried with HTTPZ 0.9.3 using settings

1. Automatic, intercept, handle, remember
2. Intercept, handle, remember
3. Intercept, handle (secure sites forgotten)
4. Intercept
5. Automatic, intercept
6. Automatic, intercept, handle

I think is good, add an feature to use HTTPS Everywhere database, so, if the site is on db, do not TRY HTTPS, if isn't on db, then TRY, this feature can be optional, and db can be auto updated like HTTPS Everywhere. What you think?