Hi,

This would be great as we can use big CDNs like Google Drive and Yandex as additional mirrors.

https://rclone.org
https://github.com/ncw/rclone

Thanks,

Changing the weight from int to string in 048833d breaks the piegraph in the mirrorlist.

See comment at 048833d#commitcomment-12490100

Support multiple instances of mirrorbits on different servers but serving the same repository.

From the looks of it it's not possible (or easy) to (re)load the new GeoIP database.

The current implementation of the web pages is not good enough and since I'm not a web guy, any help on this side would be greatly appreciated.

This should obviously be added to the 'edit' subcommand.

Hi, we are migrating the parrotsec.org redirector to mirrorbits and i'm pretty new to this software.
is there any way to scan a mirror without rsync or ftp?

our previous mirror redirector (mirrorbrain) allowed http repositories to be easily (but slowly) scanned via http if no rsync/ftp url was provided

There seems to be a missing nil check in http.go#243 that causes it to deref a nil variable when there's no fallbacks specified in the config file..

2015/07/29 09:48:38 http: panic serving 127.0.0.1:33857: runtime error: invalid memory address or nil pointer dereference
goroutine 2782 [running]:
net/http.func·011()
        /usr/local/go/src/pkg/net/http/server.go:1100 +0xb7
runtime.panic(0x7f2960, 0xc84db3)
        /usr/local/go/src/pkg/runtime/panic.c:248 +0x18d
main.(*HTTP).mirrorHandler(0xc20802a540, 0x7fcdd7478288, 0xc2086b1e00, 0xc20816e410, 0xc2087558c0)
        /home/mirrorbits/gocode/src/github.com/etix/mirrorbits/http.go:243 +0x1034
main.(*HTTP).requestDispatcher(0xc20802a540, 0x7fcdd7478288, 0xc2086b1e00, 0xc20816e410)
        /home/mirrorbits/gocode/src/github.com/etix/mirrorbits/http.go:188 +0x130
main.*HTTP.(main.requestDispatcher)·fm(0x7fcdd7478288, 0xc2086b1e00, 0xc20816e410)
        /home/mirrorbits/gocode/src/github.com/etix/mirrorbits/http.go:78 +0x44
main.func·003(0x7fcdd7478288, 0xc2086b1e00, 0xc20816e410)
        /home/mirrorbits/gocode/src/github.com/etix/mirrorbits/gzip.go:33 +0x2ec
net/http.HandlerFunc.ServeHTTP(0xc2080f8db0, 0x7fcdd7478288, 0xc2086b1e00, 0xc20816e410)
        /usr/local/go/src/pkg/net/http/server.go:1235 +0x40
net/http.(*ServeMux).ServeHTTP(0xc208026b70, 0x7fcdd7478288, 0xc2086b1e00, 0xc20816e410)
        /usr/local/go/src/pkg/net/http/server.go:1511 +0x1a3
net/http.serverHandler.ServeHTTP(0xc2080f2420, 0x7fcdd7478288, 0xc2086b1e00, 0xc20816e410)
        /usr/local/go/src/pkg/net/http/server.go:1673 +0x19f
net/http.(*conn).serve(0xc208218280)
        /usr/local/go/src/pkg/net/http/server.go:1174 +0xa7e
created by net/http.(*Server).Serve
        /usr/local/go/src/pkg/net/http/server.go:1721 +0x313

Hey!

My builds are automatically pushed to GH Releases.
Is it possible to use GH as a primary source for mirroring?
Will Mirrorbits pick downloads up and sync them to mirrors automatically?

Best, Jens

This ticket come after few days of conversations around the security of VLC download procedures, starting from an HTTP clear-text website www.videolan.org along with get.videolan.org that could, and hopefully will, be upgrade to HTTPS-only with HSTS.

One of the claim of the team of VLC for not upgrading to HTTPS-only their website www.videolan.org and get.videolan.org (running mirrorbits) is that their mirrors are anyhow also HTTP-cleartext.

Aside that there are 35 out 88 mirrors already with HTTPS, this ticket is to propose a different approach to keep the "root of trust" on secured HTTPS-only version of videolan.org/get.videolan.org while keeping the mirrors also in HTTP in clear.

The idea is to make a small Javascript widget, using HTML5 Download API, that would be safely loaded from https://get.videolan.org when clicking download and will:

• download using XMLHTTPRequest the binary of the installation as a JS blob
• will automatically verify the signature from an HTTPS-only preloaded list of file hashes
• if the signature will match, will trigger the browser download from a data:URI (with a similar approach to http://danml.com/download.html)

That way the root of trust, that today is broken because of missing HTTPS-only website, can stay there regardless of the security of the mirrors, the-facto preventing MITM attacks also without having to deploy the strict HTTPS+HSTS across all of the mirrors (but only on the main videolan.org domain sites).

Similar to what the 'edit' do but just for printing on the console.

-drop-

We need something like what has been done on #31 to specify the endpoint in the configuration file instead of relying only on query arguments. The only legitimate use of the query arguments would be ?mirrorlist and ?stats (and thus remove ?mirrorstats and ?downloadstats).

I've been wondering whether it's possible to run multiple instances of mirrorbits (meaning: for different projects, with a different set of mirrors) on the same host.
As you can use multiple config files and the config does allow you to set RedisDB which then seems to select a different DB, this should probably work?
However, the instances then still detect each other on startup: 2018/01/28 11:04:07.140 CET -> Node mirrorbits-21868 joined the cluster
As far as I understood the code, this is because it does some broadcast communication through the database, and that just uses the static string "HELLO" as a marker.
My attempt to fix this was to add the RedisDB-number to the string like that:

--- a/daemon/cluster.go
+++ b/daemon/cluster.go
@@ -11,6 +11,7 @@ import (
        "sync"
        "time"

+       . "github.com/etix/mirrorbits/config"
        "github.com/etix/mirrorbits/database"
        "github.com/etix/mirrorbits/mirrors"
        "github.com/etix/mirrorbits/utils"
@@ -94,6 +95,7 @@ func (c *cluster) Stop() {
 func (c *cluster) clusterLoop() {
        clusterChan := make(chan string, 10)
        announceTicker := time.NewTicker(1 * time.Second)
+       clusterAnnWithId := clusterAnnounce+fmt.Sprintf("%d", GetConfig().RedisDB)

        c.refreshNodeList(c.nodeID, c.nodeID)
        c.redis.Pubsub.SubscribeEvent(database.CLUSTER, clusterChan)
@@ -106,18 +108,18 @@ func (c *cluster) clusterLoop() {
                case <-announceTicker.C:
                        c.announce()
                case data := <-clusterChan:
-                       if !strings.HasPrefix(data, clusterAnnounce+" ") {
+                       if !strings.HasPrefix(data, clusterAnnWithId+" ") {
                                // Garbage
                                continue
                        }
-                       c.refreshNodeList(data[len(clusterAnnounce)+1:], c.nodeID)
+                       c.refreshNodeList(data[len(clusterAnnWithId)+1:], c.nodeID)
                }
        }
 }

 func (c *cluster) announce() {
        r := c.redis.Get()
-       database.Publish(r, database.CLUSTER, fmt.Sprintf("%s %s", clusterAnnounce, c.nodeID))
+       database.Publish(r, database.CLUSTER, fmt.Sprintf("%s%d %s", clusterAnnounce, GetConfig().RedisDB, c.nodeID))
        r.Close()
 }

That seems to work, the instances no longer see each other. Is that really all that's needed, or am I overlooking something here that will cause problems down the road?

Should be easy to do.

Hi :)

I'm wanting to use mirrorbits on a machine that already runs redis. So the database "0" is already in use. In mirrorbits.conf I can see:

RedisAddress: 127.0.0.1:6379
RedisPassword:

but I don't see a parameter to connect to another database than DB 0;

Is there a way to change the database index used by mirrorbits?

Raphaël

Metalink support would enable traffic distribution, integrity check (file checksum) and nice download resume (most meta4 aware download clients track download progress and can resume download in any point). Optional fragment checksum feature would enable re-downloading only broken parts. New ?meta4 endpoint would contain data from ?mirrorlist and ?sha1/md5, could also contain file size (that currently can be obtained only from Content-Length header) so it would help allocate disk space before downloading.

I think it would be wise to scan a mirror that was "unreachable" before mirrorbits auto enables it.

For instance

Mirror goes down,

mirrorbits sees this and disables it.

While mirror is down, some directory's are removed.

mirror back up, mirrorbits blindly enables mirror assuming it has all the files it did when it went down.

users get a 404 for missing files.

Maybe there's a timer where it will do this automatically.

Keep up the good work : ]

As title says. Currently data usage in mirrorstats is onnly shown per mirror when you hover over it. However it's only current day total so to actually know the full day usage you would have to check at specific time.
If possible a counter for current day / month / year would help doing estimations of traffic (same as done for download counts).
As extra a global counter for all mirrors combined would be a nice extra

@kib will probably already see what's available and adjust the layout

Hi,

We're using mirrorbits for lineageos (https://download.lineageos.org / https://mirrorbits.lineageos.org). I'm running into a problem where artifacts occasionally show "Service Unavailable", even when they're on disk. The problem goes away when mirrorbits is restarted. There's nothing applicable in the log file.

Last example (we've since restarted so its available again): https://mirrorbits.lineageos.org/full/kiwi/20170125/lineage-14.1-20170125-nightly-kiwi-signed.zip

And, redis information for that file:

127.0.0.1:6379> KEYS *lineage-14.1-20170125-nightly-kiwi-signed.zip
1) "FILEINFO_ftp.tugraz.at_/full/kiwi/20170125/lineage-14.1-20170125-nightly-kiwi-signed.zip"
2) "FILEINFO_main_/full/kiwi/20170125/lineage-14.1-20170125-nightly-kiwi-signed.zip"
3) "FILEINFO_acc.umu.se_/full/kiwi/20170125/lineage-14.1-20170125-nightly-kiwi-signed.zip"
4) "FILEINFO_mirror.karneval.cz_/full/kiwi/20170125/lineage-14.1-20170125-nightly-kiwi-signed.zip"
5) "FILEMIRRORS_/full/kiwi/20170125/lineage-14.1-20170125-nightly-kiwi-signed.zip"
6) "FILE_/full/kiwi/20170125/lineage-14.1-20170125-nightly-kiwi-signed.zip"
7) "FILEINFO_mirror.brauser.io_/full/kiwi/20170125/lineage-14.1-20170125-nightly-kiwi-signed.zip"

127.0.0.1:6379> HGETALL FILEINFO_ftp.tugraz.at_/full/kiwi/20170125/lineage-14.1-20170125-nightly-kiwi-signed.zip
1) "size"
2) "525981555"
127.0.0.1:6379> HGETALL FILEINFO_main_/full/kiwi/20170125/lineage-14.1-20170125-nightly-kiwi-signed.zip
1) "size"
2) "525981555"
127.0.0.1:6379> HGETALL FILEINFO_acc.umu.se_/full/kiwi/20170125/lineage-14.1-20170125-nightly-kiwi-signed.zip
1) "size"
2) "525981555"
127.0.0.1:6379> HGETALL FILEINFO_mirror.karneval.cz_/full/kiwi/20170125/lineage-14.1-20170125-nightly-kiwi-signed.zip
1) "size"
2) "525981555"
127.0.0.1:6379> HGETALL FILEMIRRORS_/full/kiwi/20170125/lineage-14.1-20170125-nightly-kiwi-signed.zip
(error) WRONGTYPE Operation against a key holding the wrong kind of value
127.0.0.1:6379> SMEMBERS FILEMIRRORS_/full/kiwi/20170125/lineage-14.1-20170125-nightly-kiwi-signed.zip
1) "mirror.karneval.cz"
2) "mirror.brauser.io"
3) "main"
4) "ftp.tugraz.at"
5) "acc.umu.se"
127.0.0.1:6379> HGETALL FILEINFO_mirror.brauser.io_/full/kiwi/20170125/lineage-14.1-20170125-nightly-kiwi-signed.zip
1) "size"
2) "525981555"
127.0.0.1:6379> HGETALL FILE_/full/kiwi/20170125/lineage-14.1-20170125-nightly-kiwi-signed.zip
 1) "size"
 2) "525981555"
 3) "modTime"
 4) "2017-01-25 07:24:10 +0000 UTC"
 5) "sha1"
 6) "71fe164924d74b02a7fec80e90f70dca23c3e7af"
 7) "sha256"
 8) ""
 9) "md5"
10) ""

Any ideas on how I can track down what's causing this? Thank you!

I plan on evaluating mirrorbits over the next week or so before we potentially swap out mirrorbrain. One of the key features that attracts us to mirrorbits is the customized download page feature:

"mirrorbits also has a JSON query API to be able to generate your own customized download page for showing alternate mirrors or a sponsor logo along with a countdown"

This is something we had planned on trying to put together on mirrorbrain for sometime but never got round to it. Is there any documentation / sample pages on how to do this? I had a quick look in the repo but couldn't see anything obvious....

thanks

We are hosting an APT repository with pdiffs, and I noticed Mirrorbit's scan feature uses file sizes to check file integrity.

The Index file on an APT repository may not change in size when it is updated.

If you look at http://realtime.mirror.osmc.tv//osmc/apt/dists/jessie-devel/main/binary-armhf/Packages.diff/Index, you will see that the current checksum is stored as are a few previous ones. But this file never grows in size as checksums are simply replaced. This can give a 'Hash Sum Mismatch' error when running apt-get update.

It would be great if we could use a fallback specified in the mirrorbits.conf for certain paths or filenames.

Hi,

It would be awesome if mirrorbits has support for automatic 'cron-like' syncing of another mirror to the local repository, so that practiclty the first step in the quick start can be replaced with a mirrorbits feature.

I would like to have this because if you currently have to maintain a (software/Linux Distro)mirror, you need to maintain a bunch of scripts and cron's to keep it up-to-date with the official ones.
It would make my life a lot easier when I'm able to just do something like

mirrorbits add -http http://[official_mirror_x] -rsync rsync://[official_mirror_x] mirror_x
mirrorbits pull mirrox_x
mirrorbits scan -enable mirror_x
mirrorbits update

And were 'pull' starts to sync the mirrors content over http/rsync to the local repository.
After which you can use the local repository as the preferred one for serving requests, and in case of an local error redirect/proxy users to another working one.

If I go over this quickly, this means that their will need to be some sort of another mode in mirrorbits in the first place which allows it to serve requests from it's local repository if preferred right?

Is this something you guys would be interested in adding?

When one disables all hashing algorithms in the config because they are just not interested in mirrorbits computing them for all files, it becomes impossible to use "mirrorbits edit" to edit mirror settings. The reason is that

When there are two mirrors with very similar names the show command will fuzzy logic list both mirrors, in my opinion this is a good thing.

The issue however is threefold:

• The edit command will do the same, and does not use strict checking
• There is no way (I found) to change the descriptive name of a mirror
• (SOLVED) It is impossible to add a mirror with the same name as one that was previously removed.

Example:

First I add a few mirrors called test.example.com and secondtest.example.com

$ mirrorbits add -http="http://test.example.com" test.example.com 
Warning: unable to guess the geographic location of test.example.com
Mirror added successfull
$ mirrorbits add -http="http://secondtest.example.com" secondtest.example.com
Warning: unable to guess the geographic location of test.example.com
Mirror added successfull

It is clear this worked from the output of the show command

$ mirrorbits show example
secondtest.example.com
test.example.com

But I can't edit test.example.com

$ mirrorbits edit test.example.com
secondtest.example.com
test.example.com

I proceed to remove secondtest.example.com so i can edit test.example.com

$ mirrorbits remove secondtest.example.com
Mirror removed successfully
$ mirrorbits edit test.example.com

I then find out I can't edit the displayname of this mirror.
I end up removing it completely and recreating it with the new displayname

When I tried this previously I also was not able to recreate the same mirrorname, it complains the mirror already exists:

$ mirrorbits add <lots of info> z.os6.org
Mirror z.os6.org already exists!
$ mirrorbits edit z.os6.org
No match for z.os6.org

In mirrorbrain there was an option that each listed file could be called with an added ".md5" which returned the hash stored in the database
Example:
"http://mirrors.kodi.tv/addons/gotham/metadata.themoviedb.org/metadata.themoviedb.org-3.8.4.zip.md5"

Storing the hash is already added in e566e93

Proposal:
add a handler like ?mirrorlist that will provide a template like mirrorlist.html to give us the md5 that is already in the db

Trying to run this (current git checkout) on Ubuntu 14.04, with sentinel disabled, every command crashes as follows:

panic: runtime error: index out of range

goroutine 1 [running]:
main.(*redisobj).askRole(0xc208038c40, 0x7f10d8e8f5b8, 0xc20805d220, 0x0, 0x0, 0x0, 0x0)
        /home/mirrorbits/src/github.com/etix/mirrorbits/redis.go:184 +0x180
main.(*redisobj).connect(0xc208038c40, 0x0, 0x0, 0x0, 0x0)
        /home/mirrorbits/src/github.com/etix/mirrorbits/redis.go:161 +0x360
main.(*cli).CmdAdd(0xcb4bd0, 0xc20800a020, 0x4, 0x4, 0x0, 0x0)
        /home/mirrorbits/src/github.com/etix/mirrorbits/commands.go:269 +0xc25

Commenting out the role, err := r.askRole(c) on line 161 (and the following ifs) makes it run without crashing.

For historical reasons we have some symlinked folders, like "2016 -> 33c3".

Files in the symlinked location are reported as "not present on any mirror" and served by our fallbacks.

• https://cdn.media.ccc.de/congress/2016/h264-hd/33c3-7811-deu-Irren_ist_staatlich.mp4?mirrorlist
• https://cdn.media.ccc.de/congress/33c3/h264-hd/33c3-7811-deu-Irren_ist_staatlich.mp4?mirrorlist

Some of our mirrors only offer password protected rsync access.

Credentials are not supported in rsync URLs:

mirrorbits add -rsync="rsync://user:[email protected]/ftp" -http="http://berlin.ftp.media.ccc.de/" berlin.media.ccc.de

A mirror, once added cannot be renamed. We need a way to change that without causing too much trouble within a cluster.

the script for updating the GeoIP databases still assumes apache is just (like it was with mirrorbrain). Guess that should be removed and adjusted to mirrorbits

See http://mirrors.kodi.tv/test-builds/test.txt?mirrorlist

I've added a test file because we were debugging another issue (excessive disk reads by mirorrbits child processes) and wanted to take a look at the mirrorlist for this file.
Obviously because the file is not yet refreshed from local disk, this doesn't work but should probably be handled better.

template: mirrorlist.html:78:89: executing "body" at <$v.FileInfo.Path>: nil pointer evaluating *main.FileInfo.Path

Has to do with last changed made in #9

When connecting from mobile devices mirrorbits 'sees' two external IP's for the device, and is unable to select an ASN. The result is that the client does not receive a download at optimal speed.

For example: http://up.kibje.com/20150803/Screenshot_2015-08-03-17-27-09.png
I have IP's 212.67.184.99 (ASN8608) and 66.249.81.223 (ASN15169)

For mobile the first address almost always corresponds to the location-defining IP of the client. In my case when I am on WLAN I get the proxy of my location, when I am not on WLAN I get the proxy address of my mobile phone provider.
The second IP address is caused by mobile browser optimization services (eg Google / Akamai / Apple) which most phone manufacturers are using on their OS's these days. They can also be caused by webpage filtering software but that is less common.

In my opinion the first address should always be selected if two or more addresses are available.

Currently several back up locations are added to the response next to the main redirect. Since not all software actually use or needs the extra locations it would be great if this could be configured to the "needed" amount.
This should greatly reduce traffic from our server.

If this option is ever merged it should be made optional.

The question is, what are we supposed to do if a .sha1 file already exists on the filesystem?

• Treat it like any other file and redirect on a mirror?
• Show the file on the filesystem (considering that it might be different)?
• Show the one computed by mirrorbits?
• Something else?

Easy feature:
- Make MirrorBits output a serverlist that is compatible with mirmon

Better feature:
Have something like "beaconFile" in config, which is a file that every server has with a timestamp in it. Save the content of beaconfile in the redis database and show the diff betweeen MirrorBits beaconFile and Mirrors beaconFile.

This way, ?mirrorlist can show a table as:

The ?mirrorlist works fine but the moment i try and grab a file i get this.

2015/05/08 20:25:17 http: panic serving 107.185.xx.xxx:22686: runtime error: index out of range
goroutine 40 [running]:
net/http.func·011()
/usr/lib/go/src/pkg/net/http/server.go:1100 +0xb7
runtime.panic(0x7e3260, 0xc6c81c)
/usr/lib/go/src/pkg/runtime/panic.c:248 +0x18d
main.(_RedirectRenderer).Write(0xc84fc0, 0xc208379980, 0xc20845a600, 0x8, 0x0, 0x0)
/home/etix/dev/go/src/github.com/etix/mirrorbits/pagerenderer.go:65 +0x588
main.(_HTTP).mirrorHandler(0xc20801a320, 0x7f41fe644978, 0xc20804cf00, 0xc2081225b0, 0xc208379980)
/home/etix/dev/go/src/github.com/etix/mirrorbits/http.go:251 +0x5e2
main.(_HTTP).requestDispatcher(0xc20801a320, 0x7f41fe644978, 0xc20804cf00, 0xc2081225b0)
/home/etix/dev/go/src/github.com/etix/mirrorbits/http.go:157 +0x130
main._HTTP.(main.requestDispatcher)·fm(0x7f41fe644978, 0xc20804cf00, 0xc2081225b0)
/home/etix/dev/go/src/github.com/etix/mirrorbits/http.go:71 +0x44
main.func·002(0x7f41fe644978, 0xc20804cf00, 0xc2081225b0)
/home/etix/dev/go/src/github.com/etix/mirrorbits/gzip.go:33 +0x2ec
net/http.HandlerFunc.ServeHTTP(0xc208000eb0, 0x7f41fe644978, 0xc20804cf00, 0xc2081225b0)
/usr/lib/go/src/pkg/net/http/server.go:1235 +0x40
net/http.(_ServeMux).ServeHTTP(0xc208026ba0, 0x7f41fe644978, 0xc20804cf00, 0xc2081225b0)
/usr/lib/go/src/pkg/net/http/server.go:1511 +0x1a3
net/http.serverHandler.ServeHTTP(0xc2080d44e0, 0x7f41fe644978, 0xc20804cf00, 0xc2081225b0)
/usr/lib/go/src/pkg/net/http/server.go:1673 +0x19f
net/http.(_conn).serve(0xc208111680)
/usr/lib/go/src/pkg/net/http/server.go:1174 +0xa7e
created by net/http.(*Server).Serve
/usr/lib/go/src/pkg/net/http/server.go:1721 +0x313

Maybe more verbose than the main output so we can track what happened exactly on each mirror (new/changed/removed files).

The monitor loop

-monitor=true 

seems to be scanning all files on disk by reading the full file, then waiting 5 minutes and starting over again. We are seeing continuous 50-100 MB/sec disk read.

It seems (or so i am told by people smarter than me) that this could be implemented by using the linux inotify system for any loops after the first one. inotify is supported in go it seems.
That way mirrorbits would be notified of filesystem changes in the folder, and could selectively add files / hash them.

For now we have had to disable the monitor loop and manually set up crontabs. However this feels cludgy. :)

Hi,

Is there a WordPress plugin for this software?

Thanks,

Client sites at present don't have a requirement to include a link to report a mirror issue. Similarly, one cannot provide "feedback" in the form of location correction, which might be fairly important.

I mention this because there are a number of mirrors for a particular client that are adversely narrow in bandwidth - I just started a download of VLC that would have taken half an hour to complete had I not located the files on another mirror.

Additionally, my development server is located in France. However, rather than a French provider, the provider selected is located in the British Midlands. Geographically, there are at least four better options.

It seems the GeoLite IP database mirrorbits has been moved to legacy.

Legacy downloads:
http://dev.maxmind.com/geoip/legacy/geolite/

New database downloads:
http://dev.maxmind.com/geoip/geoip2/geolite2/

Perhaps it's better to move to the new v2 version as it seems to be a complete set instead of separate v4 and v6 addresses.
http://dev.maxmind.com/geoip/geoip2/whats-new-in-geoip2/

Google Maps returns "The Google Maps API server rejected your request. This service requires an API key.".

According to https://stackoverflow.com/questions/38462525/google-maps-static-api-is-asking-for-api-key, all new servers online after June 22nd 2016 require an API key.

I don't know if adding an API key is feasible or not, due to restrictions from google on number of requested maps per day, or if it would be better to look at implementing a different provider.

(I'm willing to do either in a PR)

We have one case where a mirror in Russia doesn't allow downloads from Ukrainian users.

EDIT: Oops, it's Ukraine blocking connections to Russia.

Following Snowden NSA and Wikileaks CIA releases the internet community has been shaked in the understanding of how massive surveillance and targeted attacks works.

The basic attacks to infect computer with software malware is to intercept "on the wire" the package download, infect it "on-the-fly" and let the end-user finish the download and install it.

This major attack can be prevented by having a default HTTPS transport from the download mirror, considering that the additional costs of HTTPS vs HTTP is today negligible.

This ticket is to have mirrorbits to enforce all of the mirror download URLs to be only HTTPS, operating a plan to enforce HTTPS as the only download method, eventually working closely with the download mirrors.

For our about page (https://media.ccc.de/about.html) we use JSON to render a table of all mirrors.
Previously we would list all active mirrors, their URLs and the number of files on the mirror.

I thought about writing a separate go program later this month to get the information from the redis database, but maybe it would make sense to include this in mirrorbits?

When adding a new mirror i always try a manual scan to make sure the scan works. Currently even when the rsync url is wrong there's no cmd line output if it was successful or not.

Perhaps useful to report this on cmd?

Is it possible to add the md5 hash of a file and return it via json as well as the sha1?

Long term it would be nice if you could add the ability to append .sha1 or .md5 to a given filename and have it download the hash in a similar way to how mirrorbrain works.

Hi,
Can you add support for HTTP dynamic header manipulation?

This way we can do things like pushing a different filename upon download, using the Content-Disposition HTTP header for example,among other similar things,like optionally pushing file checksum in the header.

https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Disposition

Thanks,